How to live with Emacs Lisp dynamic scoping?

Question

I've learned Clojure previously and really like the language. I also love Emacs and have hacked some simple stuff with Emacs Lisp. There is one thing which prevents me mentally from doing anything more substantial with Elisp though. It's the concept of dynamic scoping. I'm just scared of it since it's so alien to me and smells like semi-global variables.

So with variable declarations I don't know which things are safe to do and which are dangerous. From what I've understood, variables set with setq fall under dynamic scoping (is that right?) What about let variables? Somewhere I've read that let allows you to do plain lexical scoping, but somewhere else I read that let vars also are dynamically scoped.

I quess my biggest worry is that my code (using setq or let) accidentally breaks some variables from platform or third-party code that I call or that after such call my local variables are messed up accidentally. How can I avoid this?

Are there a few simple rules of thumb that I can just follow and know exactly what happens with the scope without being bitten in some weird, hard-to-debug way?

Answer 1

In addition to the last paragraph of Gilles answer, here is how RMS argues in favor of dynamic scoping in an extensible system:

Some language designers believe that dynamic binding should be avoided, and explicit argument passing should be used instead. Imagine that function A binds the variable FOO, and calls the function B, which calls the function C, and C uses the value of FOO. Supposedly A should pass the value as an argument to B, which should pass it as an argument to C.

This cannot be done in an extensible system, however, because the author of the system cannot know what all the parameters will be. Imagine that the functions A and C are part of a user extension, while B is part of the standard system. The variable FOO does not exist in the standard system; it is part of the extension. To use explicit argument passing would require adding a new argument to B, which means rewriting B and everything that calls B. In the most common case, B is the editor command dispatcher loop, which is called from an awful number of places.

What's worse, C must also be passed an additional argument. B doesn't refer to C by name (C did not exist when B was written). It probably finds a pointer to C in the command dispatch table. This means that the same call which sometimes calls C might equally well call any editor command definition. So all the editing commands must be rewritten to accept and ignore the additional argument. By now, none of the original system is left!

Personally, I think that if there is a problem with Emacs-Lisp, it is not dynamic scoping per se, but that it is the default, and that it is not possible to achieve lexical scoping without resorting to extensions. In CL, both dynamic and lexical scoping can be used, and -- except for top-level (which is adressed by several deflex-implementations) and globally declared special variables -- the default is lexical scoping. In Clojure, too, you can use both lexical and dynamic scoping.

To quote RMS again:

It is not necessary for dynamic scope to be the only scope rule provided, just useful for it to be available.

Answer 2

Just don't.

Emacs-24 lets you use lexical-scope. Just run

(setq lexical-binding t)

or add

;; -*- lexical-binding: t -*-

at the beginning of your file.

Answer 3

Are there a few simple rules of thumb that I can just follow and know exactly what happens with the scope without being bitten in some weird, hard-to-debug way?

Read Emacs Lisp Reference, you'll have many details like this one :

• Special Form: setq [symbol form]... This special form is the most common method of changing a variable's value. Each SYMBOL is given a new value, which is the result of evaluating the corresponding FORM. The most-local existing binding of the symbol is changed.

Here is an example :

(defun foo () (setq tata "foo"))

(defun bar (tata) (setq tata "bar"))


(foo)
(message tata)
    ===> "foo"


(bar tata)
(message tata)
    ===> "foo"

Answer 4

First, elisp has separate variable and function bindings, so some pitfalls of dynamic scoping are not relevant.

Second, you can still use setq to set variables, but the value set does not survive the exit of the dynamic scope it is done in. This isn't, fundamentally, different from lexical scoping, with the difference that with dynamic scoping a setq in a function you call can affect the value you see after the function call.

There's lexical-let, a macro that (essentially) imitates lexical bindings (I believe it does this by walking the body and changing all occurrences of the lexically let variables to a gensymmed name, eventually uninterning the symbol), if you absolutely need to.

I'd say "write code as normal". There are times when the dynamic nature of elisp will bite you, but I've found that in practice that is surprisingly seldom.

Here's an example of what I was saying about setq and dynamically-bound variables (recently evaluated in a nearby scratch buffer):

(let ((a nil))
  (list (let ((a nil))
          (setq a 'value)
          a)
        a))

(value nil)

Answer 5

Dynamic and lexical scoping have different behaviors when a piece of code is used in a different scope than the one it was defined in. In practice, there are two patterns that cover most troublesome cases:

• A function shadows a global variable, then calls another function that uses that global variable.

  (defvar x 3)
  (defun foo ()
    x)
  (defun bar (x)
    (+ (foo) x))
  (bar 0) ⇒ 0
  

  This doesn't come up often in Emacs because local variables tend to have short names (often single-word) whereas global variables tend to have long names (often prefixed by packagename-). Many standard functions have names that are tempting to use as local variables like list and point, but functions and variables live in separate name spaces are local functions are not used very often.

• A function is defined in one lexical context and used outside this lexical context because it's passed to a higher-order function.

  (let ((cl-y 10))
    (mapcar* (lambda (elt) (* cl-y elt)) '(1 2 3)))
  ⇒ (10 20 30)
  (let ((cl-x 10))
    (mapcar* (lambda (elt) (* cl-x elt)) '(1 2 3)))
  ⇑ (wrong-type-argument number-or-marker-p (1 2 3))
  

  The error is due to the use of cl-x as a variable name in mapcar* (from the cl package). Note that the cl package uses cl- as a prefix even for its local variables in higher-order functions. This works reasonably well in practice, as long as you take care not to use the same variable as a global name and as a local name, and you don't need to write a recursive higher-order function.

P.S. Emacs Lisp's age isn't the only reason why it's dynamically scoped. True, in those days, lisps tended towards dynamic scoping — Scheme and Common Lisp hadn't really taken on yet. But dynamic scoping is also an asset in a language targeted towards extending a system dynamically: it lets you hook into more places without any special effort. With great power comes great rope to hang yourself: you risk accidentally hooking into a place you didn't know about.

Answer 6

I entirely feel your pain. I find the lack of lexical binding in emacs rather annoying - especially not being able to use lexical closures, which seems to be a solution I think of a lot, coming from more modern languages.

While I don't have any more advice on working around the lacking features that the previous answers didn't cover yet, I'd like to point out the existance of an emacs branch called `lexbind', implementing lexical binding in a backward-compatible way. In my experience lexical closures are still a little buggy in some circumstances, but that branch appears to a promising approach.