How to find “not a procedure” error

(define (comp f g)
  (lambda (x)(f (g x))))

(define (complement f) (cond ((equal? (comp f (lambda (g) g)) #t) #f)
                             ((equal? (comp f (lambda (g) g)) #f) #t)))

((complement odd?)2)

It keeps saying that ((complement odd?)2) is not a procedure. I'm not sure how to fix it.

When you run this code you'll see that ((complement odd?) 2) is red in the definitions and you get the following error:

application: not a procedure;
 expected a procedure that can be applied to arguments
  given: #<void>

So that would mean (complement odd?) doesn't return a procedure but the value #<void>. Lets try that:

(complement odd?)
; ==> nothing (aka #<void>)

If you really want to see it use it somewhere:

(list (complement odd?))
; ==> (#<void>) now you see it

That means you are not handling all your possible checks in the cond in complement and I see why.. Have you tried comp?

(comp f (lambda (g) g)) ; ==> #<procedure>

Surely enough the use of comp becomes a procedure. Not strange since the body has one lambda form that indicates the return would be a procedure. It will never be #t or #f When you don't have an else (default) term for when neither of your predicates became true cond returns an implementation specific defauls value. In Racket this is #<void> which is supressed by the REPL, but in other implementations it can be banana or whatever the implementers want it to be so you should always have a else clause. If you don't think you need it then do (else (error "should never happen")) and you're good to go. (try it)

The consequences in your cond are #t and #f. That means that if your code sould have worked you would have got this error message instead:

application: not a procedure;
 expected a procedure that can be applied to arguments
  given: #t

For each place your return smething that is not a procedure that would be an error since you are using the result as a procedure. You need to change your implementation such that it returns a procedure always.

So, this is the answer to how to find “not a procedure” error. It's not the answer to how to fix your procedure but since this is the simplest procedure in the world I'll add it. You have a procedure comp that takes two procedures and return the composition of the two. eg. if you want add2 you can (define add2 (comp add1 add1)). The complement must be that the false value #f turns #t while all the true values turn #f. not does this so the composition of not and odd? would become the procedure that works the same as even?:

(define (complement f)
  (comp not f)) ; comp returns a procedure always

(define my-even? (complement odd?))
(my-even? 2) ; ==> #t

Since we don't mutate anything you can use substitution method to check what this does:

(my-even? 2)                      ; ==>
((comp not odd?) 2)               ; ==>
(((lambda (x) (not (odd? x))) 2)  ; ==>
(not (odd? 2))                    ; ==>
(not #f)                          ; ==>
#t