Practical example of Lisp's flexibility? [clos

Question

Someone is trying to sell Lisp to me, as a super powerful language that can do everything ever, and then some.

Is there a practical code example of Lisp's power?
(Preferably alongside equivalent logic coded in a regular language.)

Answer 1

I was an AI student at MIT in the 1970s. Like every other student, I thought language was paramount. Nevertheless, Lisp was the primary language. These are some things I still think it is pretty good for:

• Symbolic math. It is easy and instructive to write symbolic differentiation of an expression, and algebraic simplification. I still do those, even though I do them in C-whatever.

• Theorem proving. Every now & then I go on a temporary AI binge, like trying to prove that insertion sort is correct. For that I need to do symbolic manipulation, and I usually fall back on Lisp.

• Little domain-specific-languages. I know Lisp isn't really practical, but if I want to try out a little DSL without having to get all wrapped up in parsing, etc., Lisp macros make it easy.

• Little play algorithms like minimax game tree search can be done in like three lines.

• Want to try lambda calculus? It's easy in Lisp.

Mainly what Lisp does for me is mental exercise. Then I can carry that over into more practical languages.

P.S. Speaking of lambda calculus, what also started in the 1970s, in that same AI millieu, was that OO started invading everybody's brain, and somehow, interest in what it is seems to have crowded out much interest in what it is good for. I.e. work on machine learning, natural language, vision, problem solving, all sort of went to the back of the room while classes, messages, types, polymorphism, etc. went to the front.

Answer 2

One thing I like is the fact that I can upgrade code "run-time" without losing application state. It's a thing only useful in some cases, but when it is useful, having it already there (or, for only a minimal cost during development) is MUCH cheaper than having to implement it from scratch. Especially since this comes at "no to almost no" cost.

Answer 3

Actually, a good practical example is the Lisp LOOP Macro.

http://www.ai.sri.com/pkarp/loop.html

The LOOP macro is simply that -- a Lisp macro. Yet it basically defines a mini looping DSL (Domain Specific Language).

When you browse through that little tutorial, you can see (even as a novice) that it's difficult to know what part of the code is part of the Loop macro, and which is "normal" Lisp.

And that's one of the key components of Lisps expressiveness, that the new code really can't be distinguished from the system.

While in, say, Java, you may not (at a glance) be able to know what part of a program comes from the standard Java library versus your own code, or even a 3rd party library, you DO know what part of the code is the Java language rather than simply method calls on classes. Granted, it's ALL the "Java language", but as programmer, you are limited to only expressing your application as a combination of classes and methods (and now, annotations). Whereas in Lisp, literally everything is up for grabs.

Consider the Common SQL interface to connect Common Lisp to SQL. Here, http://clsql.b9.com/manual/loop-tuples.html, they show how the CL Loop macro is extended to make the SQL binding a "first class citizen".

You can also observe constructs such as "[select [first-name] [last-name] :from [employee] :order-by [last-name]]". This is part of the CL-SQL package and implemented as a "reader macro".

See, in Lisp, not only can you make macros to create new constructs, like data structures, control structures, etc. But you can even change the syntax of the language through a reader macro. Here, they're using a reader macro (in the case, the '[' symbol) to drop in to a SQL mode to make SQL work like embedded SQL, rather than as just raw strings like in many other languages.

As application developers, our task is to convert our processes and constructs in to a form that the processor can understand. That means we, inevitably, have to "talk down" to the computer language, since it "doesn't understand" us.

Common Lisp is one of the few environments where we can not only build our application from the top down, but where we can lift the language and environment up to meet us half way. We can code at both ends.

Mind, as elegant as this can be, it's no panacea. Obviously there are other factors that influence language and environment choice. But it's certainly worth learning and playing with. I think learning Lisp is a great way to advance your programming, even in other languages.

Answer 4

You might find this post by Eric Normand helpful. He describes how as a codebase grows, Lisp helps by letting you build the language up to your application. While this often takes extra effort early on, it gives you a big advantage later.

Answer 5

Have you taken a look at this explanation of why macros are powerful and flexible? No examples in other languages though, sorry, but it might sell you on macros.

Answer 6

I like macros.

Here's code to stuff away attributes for people from LDAP. I just happened to have that code lying around and fiigured it'd be useful for others.

Some people are confused over a supposed runtime penalty of macros, so I've added an attempt at clarifying things at the end.

In The Beginning, There Was Duplication

(defun ldap-users ()
  (let ((people (make-hash-table :test 'equal)))
    (ldap:dosearch (ent (ldap:search *ldap* "(&(telephonenumber=*) (cn=*))"))
                   (let ((mail  (car (ldap:attr-value ent 'mail)))
                         (uid   (car (ldap:attr-value ent 'uid)))
                         (name  (car (ldap:attr-value ent 'cn)))
                         (phonenumber (car (ldap:attr-value ent 'telephonenumber))))
                      (setf (gethash uid people)
                            (list mail name phonenumber))))
    people))

You can think of a "let binding" as a local variable, that disappears outside the LET form. Notice the form of the bindings -- they are very similar, differing only in the attribute of the LDAP entity and the name ("local variable") to bind the value to. Useful, but a bit verbose and contains duplication.

On the Quest for Beauty

Now, wouldn't it be nice if we didn't have to have all that duplication? A common idiom is is WITH-... macros, that binds values based on an expression that you can grab the values from. Let's introduce our own macro that works like that, WITH-LDAP-ATTRS, and replace it in our original code.

(defun ldap-users ()
  (let ((people (make-hash-table :test 'equal))) ; equal so strings compare equal!
    (ldap:dosearch (ent (ldap:search *ldap* "(&(telephonenumber=*) (cn=*))"))
                   (with-ldap-attrs (mail uid name phonenumber) ent
                       (setf (gethash uid people)
                             (list mail name phonenumber))))
    people))

Did you see how a bunch of lines suddenly disappeared, and was replaced with just one single line? How to do this? Using macros, of course -- code that writes code! Macros in Lisp is a totally different animal than the ones you can find in C/C++ through the use of the pre-processor: here, you can run real Lisp code (not the #define fluff in cpp) that generates Lisp code, before the other code is compiled. Macros can use any real Lisp code, i.e., ordinary functions. Essentially no limits.

Getting Rid of Ugly

So, let's see how this was done. To replace one attribute, we define a function.

(defun ldap-attr (entity attr)
  `(,attr (car (ldap:attr-value ,entity ',attr))))

The backquote syntax looks a bit hairy, but what it does is easy. When you call LDAP-ATTRS, it'll spit out a list that contains the value of attr (that's the comma), followed by car ("first element in the list" (cons pair, actually), and there is in fact a function called first you can use, too), which receives the first value in the list returned by ldap:attr-value. Because this isn't code we want to run when we compile the code (getting the attribute values is what we want to do when we run the program), we don't add a comma before the call.

Anyway. Moving along, to the rest of the macro.

(defmacro with-ldap-attrs (attrs ent &rest body)
  `(let ,(loop for attr in attrs
         collecting `,(ldap-attr ent attr))
     ,@body)) 

The ,@-syntax is to put the contents of a list somewhere, instead of the actual list.

Result

You can easily verify that this will give you the right thing. Macros are often written this way: you start off with code you want to make simpler (the output), what you want to write instead (the input), and then you start molding the macro until your input gives the correct output. The function macroexpand-1 will tell you if your macro is correct:

(macroexpand-1 '(with-ldap-attrs (mail phonenumber) ent
                  (format t "~a with ~a" mail phonenumber)))

evaluates to

(let ((mail (car (trivial-ldap:attr-value ent 'mail)))
      (phonenumber (car (trivial-ldap:attr-value ent 'phonenumber))))
  (format t "~a with ~a" mail phonenumber))

If you compare the LET-bindings of the expanded macro with the code in the beginning, you'll find that it is in the same form!

Compile-time vs Runtime: Macros vs Functions

A macro is code that is run at compile-time, with the added twist that they can call any ordinary function or macro as they please! It's not much more than a fancy filter, taking some arguments, applying some transformations and then feeding the compiler the resulting s-exps.

Basically, it lets you write your code in verbs that can be found in the problem domain, instead of low-level primitives from the language! As a silly example, consider the following (if when wasn't already a built-in)::

(defmacro my-when (test &rest body)
  `(if ,test 
     (progn ,@body)))

if is a built-in primitive that will only let you execute one form in the branches, and if you want to have more than one, well, you need to use progn::

;; one form
(if (numberp 1)
  (print "yay, a number"))

;; two forms
(if (numberp 1)
  (progn
    (assert-world-is-sane t)
    (print "phew!"))))

With our new friend, my-when, we could both a) use the more appropriate verb if we don't have a false branch, and b) add an implicit sequencing operator, i.e. progn::

(my-when (numberp 1)
  (assert-world-is-sane t)
  (print "phew!"))

The compiled code will never contain my-when, though, because in the first pass, all macros are expanded so there is no runtime penalty involved!

Lisp> (macroexpand-1 '(my-when (numberp 1)
                        (print "yay!")))

(if (numberp 1)
  (progn (print "yay!")))

Note that macroexpand-1 only does one level of expansions; it's possible (most likely, in fact!) that the expansion continues further down. However, eventually you'll hit the compiler-specific implementation details which are often not very interesting. But continuing expanding the result will eventually either get you more details, or just your input s-exp back.

Hope that clarifies things. Macros is a powerful tool, and one of the features in Lisp I like.