Common Lisp Double-Backquote, Unquote, Quote, Unqu

Question

I'm reading Let Over Lambda, which deals with some pretty deeply layered macro authoring. It's fascinating and I'm mostly managing to keep up with it.

In Chapter 4 Hoyte implements reader macros for CL-PPCRE match and replace functions, such that you can do things like:

(#~m/(foo|bar)\d+/ "Some foo99")    ; matches!
(#~s/foo(\d+)/bar\1/, "Some foo99") ; "Some bar99

In order to achieve this, we define a macro that uses the double-backquote, since it is actually expanded by a wrapper macro, which needs the quoted value (it returns a lambda form). Within the quasi-quoted list, there is some use of the following sequence ,',varname, which I can't get my head around. What does the initial ,' do here?

(defmacro! pcre/match-lambda-form (o!args)
  "Expands to a lambda that applies CL-PPCRE:SCAN"
  ``(lambda (,',g!str)
      (cl-ppcre:scan ,(car ,g!args)
                     ,',g!str)))

Actually, it's probably better that I distill that down to something that uses just defmacro, for clarity if you haven't read the book. str is a symbol and args is a list:

(defmacro pcre/match-lambda-form (args)
  "Expands to a lambda that applies CL-PPCRE:SCAN"
  ``(lambda (,',str)
      (cl-ppcre:scan ,(car ,args)
                     ,',str)))

Are the quotes basically double-quoting the inner parts, so that the result can be unquoted twice? Effectively putting 'str into the expanded form, instead of just str?

EDIT | Thanks to Terje D. and some playing around in the REPL, this is pretty much the situation:

(defvar a 42)

(equal ``(,,a)  '(list 42)) ; T
(equal ``(,a)   '(list a))  ; T
(equal ``(,',a) ''(42))     ; T
(equal ``(a)    ''(a))      ; T (obviously)

So:

• Doubly-unquoted, form is fully expanded.
• Singly-unquoted, form is not expanded.
• Unquoted with a comma, form is fully expanded and the result quoted.

Answer 1

The ,',X trick is used to shield the X from another evaluation.

See how:

     (setq a 'fn)
     (let ((x 'a)) ``(,,x ,',x)) ==>  `(,a a) ==> (fn a)

     ;; ``,',X ==> `,(quote "the value of X") ==> "the value of X"

     ;; ``,,X  ==> `,"the value of X" ==> "the value of the value of X"

Answer 2

During evaluation of a doubly backquoted form, the inner backquote is handled first, and the result is a singly backquoted form. During evaluation of the inner backquoted form, only elements preceeded by two commas are evaluated. However, the result of evaluating these doubly unquoted elements are still (singly) unquoted, and are thus evaluated again when the resulting singly backquoted form are evaluated. To achieve evaluation only in the inner backquoted form, an ordinary quote has to be inserted, resulting in ,',.

See how

(let ((tmp (gensym)))
    ``(lambda (,tmp ,,tmp ,',tmp) ()))

evaluates to

`(LAMBDA (,TMP ,#:G42 #:G42) nil)