Understanding Big O notation - Cracking the coding

Question

I got stuck with this two codes.

Code 1

int f(int n){
    if (n <= 1){
         return 1;
    }
    return f(n-1) + f(n-1);
}

Code 2 (Balanced binary search tree)

int sum(Node node){
    if(node == null){
        return 0;
    }
    return sum(node.left) + node.value + sum(node.right);
}

the author says the runtime of Code 1 is O(2^n) and space complexity is O(n)

And Code 2 is O(N)

I have no idea what's different between those two codes. it looks like both are the same binary trees

Answer 1

First of all, it's important to understand what N is in both cases. In the first example it's pretty obvious, because you see it directly in the code. For your first case, if you build the tree of f(i) calls, you'll see that it contains O(2^N) elements. Indeed,

            f(N)             // 1 node
        /          \ 
   f(N-1)         f(N-1)     // 2 nodes
  /      \      /      \
f(N-2) f(N-2)  f(N-2) f(N-2) // 2^2 nodes
...
f(1) ........ f(1)           // 2^(N-1) nodes

In the second case, N is (most likely) a number of elements in the tree. As you may see from the code, we walk through every element exactly once - you may realize it as you see that node.value is invoked once for each tree node. Hence O(N).

Note that in such tasks N normally means the size of the input, while what the input is depends on your problem. It can be just a number (like in your first problem), a one-dimensional array, a binary tree (like in your second problem), or even a matrix (although in the latter case you may expect to see explicit statement like "a matrix with a size M*N").

So your confusion probably comes from the fact that the "definition of N" differs between those two problems. In other words, I might say that n2 = 2^n1.

Answer 2

Well there's a mistake because the first snippet runs in O(2^n) not O(n^2).

The explanation is: In every step we decrement n but create twice the number of calls, so for n we'll call twice with f(n-1) and for each one of the calls of n-1 we'll call twice with f(n-2) - which is 4 calls, and if we'll go another level down we'll call 8 times with f(n-3): so the number of calls is: 2^1, then 2^2, then 2^3, 2^4, ..., 2^n.

The second snippet is doing one pass on a binary tree and reaches every node exactly once, so it's O(n).

Answer 3

Code 1:

The if() statement runs n times according to whatever is passed into the parameter, but the the function call itself n-1 times. To simplify:

n * (n-1) = n^2 - n = O(n^2 - n) = O(n^2)

Code 2:

The search traverses every element of the tree only once, and the function itself doesn't have any for(). Since there are n items and they are visited only once, it is O(n).

Answer 4

For Code 2, to determine the Big O of a function, didn't we have to consider the cost of the recurrence and also how many times the recurrence was run?

If we use two approach to estimate the Big O using recursive tree and master theorem:

Recursive tree: total cost in each level will be cn for each level as the number of recursive call and the fraction of input are equal, and the level of tree is lg(n) since it's a balanced binary search tree. So the run time should be nlg(n)?

Master Theorem: This should be a case 2 since f(n) = n^logbase a (b). So according to the master theorem, it should be nlg(n) running time?

Answer 5

The first code is indeed O(2^n).

But the second code cannot be O(n), because there is no n there. That's a thing which many forget and usually they assume what n is without clarifying it.

In fact you can estimate growth speed of anything based on anything. Sometimes it's a size of input (which in the first code is O(1) or O(log n) depending on usage of big numbers), sometimes just on argument if it's numeric.

So when we start thinking about what time and memory depend on in the second code we can get these things:

1. time=O(number_of_nodes_in_tree)
2. time=O(2^height_of_tree)
3. additional_space=O(height_of_tree)
4. additional_space=O(log(number_of_nodes)) (if the tree is balanced)

All of them are correct at the same time - they just relate something to different things.