Is there a straightforward lisp equivalent of Pyth

In Python you can write this:

def firstn(n):
     num = 0
     while num < n:
         yield num
         num += 1

What is the lisp equivalent of this?

Existing package

Download, install and load the GENERATORS system with Quicklisp. Then, use package :generators (or preferably, define your own package first).

(ql:quickload :generators)
(use-package :generators)

Define an infinite generator for random values:

(defun dice (n)
  (make-generator ()
    ;; repeatedly return a random value between 1 and N
    (loop (yield (1+ (random n))))))

Use the generator:

(loop
   with dice = (dice 6)
   repeat 20
   collect (next dice))

=> (1 2 6 1 1 4 4 2 4 3 6 2 1 5 6 5 1 5 1 2)

Note however what the author of the library says:

This library is more of an interesting toy, though as far as I know it does work. I dont think I have ever used this in application code, though I think that with care, it could be.

Closures

In practice, CL does not rely that much on generators as popularized by Python. What happens instead is that when people need lazy sequences, they rely on closures:

(defun dice (n)
  (lambda ()
    (1+ (random n))))

Then, the equivalent of next is simply a call to the thunk generated by dice:

(loop
   with dice = (dice 6)
   repeat 20
   collect (funcall dice))

This is the approach that is preferred, in particular because there is no need to rely on delimited continuations like with generators. Your example involves a state, which the dice example does not require (well, there is a hidden state that influences random, but that's another story) . Here is how your counter is typically implemented:

(defun first-n (n)
  (let ((counter -1))
    (lambda ()
      (when (< counter n)
        (incf counter)))))

Higher-order functions

Alternatively, you design a generator that accepts a callback function which is called by your generator for each value. Any funcallable can be used, which allows the caller to retain control over code execution:

(defun repeatedly-throw-dice (n callback)
  (loop (funcall callback (1+ (random n)))))

Then, you can use it as follows:

(prog ((counter 0) stack)
  (repeatedly-throw-dice 6 
    (lambda (value)
      (if (<= (incf counter) 20)
        (push value stack)
        (return (nreverse stack))))))

See documentation for PROG.

`do-traversal` idiom

Instead of building a function, data sources that provides a custom way of generating values (like matches of a regular expressions in a string) also regularly provide a macro that abstracts their control-flow. You would use it as follows:

(block 'outer
  (let ((counter 0) stack)
    (do-repeatedly-throw-dice (value 6)

      ;; For each iteration of the infinite loop,
      ;; VALUE is bound to a new random value.

      (if (<= (incf counter) 20)
        (push value stack)
        (return-from 'outer (nreverse stack))))))

The difference with the above is that the block is explicitely named. This is because DO-X usually can be expected to define a NIL block around their body, which means that any enclosing NIL block is shadowed. The implicit NIL block allows you to exit from the iteration easily:

 (let ((counter 0)  stack)
   (do-repeatedly-throw-dice (value 6)
     (if (<= (incf counter) 20)
       (push value stack)
       (return (nreverse stack))))))

A possible implementation for the macro is to wrap the body in a lambda form and use the callback-based version defined above:

(defmacro do-repeatedly-throw-dice ((var n) &body body)
  `(block nil (repeatedly-throw-dice ,n (lambda (,var) ,@body))))

Directly expanding into a loop would be possible too:

(defmacro do-repeatedly-throw-dice ((var n) &body body)
  (let ((max (gensym)) (label (make-symbol "NEXT")))
    `(prog ((,max ,n) ,var)
        ,label
        (setf ,var (1+ (random ,max)))
        (progn ,@body)
        (go ,label))))

One step of macroexpansion for above form:

(BLOCK 'OUTER
  (LET ((COUNTER 0) STACK)
    (PROG ((#:G16053 6) VALUE)
     #:NEXT (SETF VALUE (1+ (RANDOM #:G16053)))
            (PROGN (IF (<= (INCF COUNTER) 20)
                      (PUSH VALUE STACK)
                      (RETURN-FROM 'OUTER (NREVERSE STACK))))
            (GO #:NEXT))))

Bindings

Broadly speaking, building a generator with higher-order functions or directly with a do- macro gives the same result. You can implement one with the other (personally, I prefer to define first the macro and then the function using the macro, but doing the opposite is also interesting, since you can redefine the function without recompiling all usages of the macro).

However, there is still a difference: the macro reuses the same variable across iterations, whereas the closure introduces a fresh binding each time. For example:

(let ((list))
  (dotimes (i 10) (push (lambda () i) list))
  (mapcar #'funcall list))

.... returns:

(10 10 10 10 10 10 10 10 10 10)

Most (if not all) iterators in Common Lisp tend to work like this, and it should not come as a surprise for experienced users (the opposite would be surprising, in fact). If dotimes was implemented by repeatedly calling a closure, the result would be different:

(defmacro my-dotimes ((var count-form &optional result-form) &body body)
  `(block nil
     (alexandria:map-iota (lambda (,var) ,@body) ,count-form)
     ,result-form))

With the above definition, we can see that:

(let ((list))
  (my-dotimes (i 10) (push (lambda () i) list))
  (mapcar #'funcall list))

... returns:

(9 8 7 6 5 4 3 2 1 0)

In order to have the same result with the standard dotimes, you only need to evaluate the iteration variable before building the closure:

(let ((list))
  (dotimes (i 10) 
    (let ((j i))
      (push (lambda () j) list))))

If you wanted to, you could always introduce that inner let from the macro, but this is rarely done.