Macros do not allow definition of lexical variable

Question

This code that uses (experimental) macros:

use experimental :macros; 
macro new-var() { 
  quasi { 
    my $a = 42
  }
};
new-var; 
say $a

Fails with Variable '$a' is not declared, although the macro passes through without an error. If that's a correct macro declaration, what does it do? If it's not, is there a way to define new variables from within a macro?

Answer 1

The answer from moritz is correct about the state of macros, though from what I know of the work being done in 007, I don't think the program as written would be correct even with a working implementation of Perl 6 macros.

Perl 6 macros will not be textual in nature (C macros are an example of textual ones). A quasi is a quote construct, much like we have quotes for strings and regexes, except that it quotes Perl 6 code, representing it as something AST-ish. (I once would have said that it produces AST, but it's been realized that if an infix were to be interpolated inside of a quasi, then it comes with a precedence and associativity, and we we can't actually form the correct tree for the expression until after interpolation.)

There's a macro concept of "hygiene", whereby symbols declared in the macro body should not, by default, leak out to the place that the macro is applied, since they may well just be implementation details. One would have to explicitly ask to put a symbol into the compiling context where the macro is applied. So I expect the program would have to look like this:

macro new-var() { 
    quasi { 
        my COMPILING::<$a> = 42
    }
};
new-var; 
say $a

Note that this won't work today in Rakudo, although you might find something like it can be made to work in 007.

Answer 2

This might not be the answer you are looking for, but macros in Rakudo are currently really broken. At this point in time I can't even remember if it's supposed to work, or if it's a bug in Rakudo -- it's mostly not worth it figuring it out, because most macro things barely work at all.

This is why Carl Mäsak created 007 to experiment with Macro design outside of Rakudo core, with the goal of eventually bringing the lessons learned back to Rakudo and the Perl 6 language design.

Answer 3

(As I go to publish this I see Jonathan has answered as well. I'll publish anyway.)

You've got Moritz' succinct answer to this question. But I'm thinking it's worth emphasizing and clarifying a few things.

Macro issues

In my answer to your recent somewhat related question I closed with:

So, bottom line, lexical declarations currently fail to stick. The experiment hasn't gotten that far. Carl Mäsak, the creator of the experimental macros in Rakudo, is continuing the experiment at 007. If you leave an issue at its GH repo Carl will answer.

I don't think Carl Mäsak has an SO account. And I note that there's still no issue at the 007 repo. So Carl may not be aware of your question.

If anyone wants to know something about macros in P6 that isn't covered by the official docs, please search existing issues of the 007 project and/or post a new one with a title that starts with [P6].

A nanswer

A NaN is a number that's Not a Number. This answer is a nanswer designed to elicit an answer from Carl.

I'm nanswering to reduce the scope that Carl's authoritative answer would need to cover. It would just need to confirm, clarify, or correct what Moritz and I have written and link to pertinent issues and other elements of the 007 repo.

Once I've published this nanswer I'll post an issue on the 007 repo pointing to your question and this nanswer. Given that it's Christmas and Carl has a family with young children I'd allow him a week or two, perhaps a month or two, to answer.

In the meantime, this nanswer is my very brief introduction to the project of designing and implementing macros in P6; and to "hygienic macros", one of the key issues relevant to your question.

Macros progress

The macros available in Rakudo are limited, buggy, and unpolished. Carl knows that. For the back story, see his blog that's mostly about P6 and P6 macros.

To move things along without P6 macros causing the rest of the P6 language undue distractions, and vice-versa, he went underground 4 years ago as the secret agent behind 007, a "Small experimental language with a license to macro".

Carl has been working diligently on it ever since (albeit with suitable pauses related to fatherhood and other elements IRL). He has had some help from a couple others, most notably vendethiel (who does have an SO account, @ven; perhaps he'll answer?).

He consistently expresses appreciation for any help, and treats questions raised in issues on the repo as help, so any questions you care to leave there will be appreciated.

For example, in response to an 007 issue I filed, Carl has recently confirmed that 007's primary goal remains being a vehicle for improving P6 macros.

Hygienic macros

macro new-var { quasi { my $a = 42 } }

If that's a correct macro declaration, what does it do?

Here's a simple description of what's going on that Carl will hopefully improve upon.

Consider this code (assume two of the three declarations are commented out):

macro new-var {           my $a = 42 }
macro new-var { quasi   { my $a = 42 } }
macro new-var { quasi { { my $a = 42 } } }
...
new-var;
...
say $a;
...
my $a;

Leaving the first line uncommented won't do what someone might expect for the reasons covered in my answer to your earlier SO question (linked above); macros must return an AST (and in most cases that means use of the quasi construct).

It hopefully makes sense that the third wouldn't leave any trace of $a for the say line because the macro returns (an AST corresponding to) a block which encloses the $a declaration; so the declaration expires before the compiler arrives at the say. So that code will yield a compile-time error.

This leaves the second macro definition, the one you used in your question.

The braces of the macro new-var { ... } construct define a lexical scope, just like they do with many similar constructs such as sub foo { ... }. So, hopefully, it's not too shocking that the lifetime of a my $a declaration directly inside those braces would end at the end of the macro call, so $a would be no longer in scope by the time the compiler reaches the say in the above code.

It turns out that the braces of the quasi { ... } construct have the same effect even though the AST returned by the quasi is not a block, but rather just the statement or list of statements inside the braces. Any lexical declarations inside the braces fail to stick just as they would fail to stick if they were defined outside the quasi or inside another pair of braces.

The simple explanation is that this is a bug.

The deeper story is that there's a tension between having hygienic macros, which is the best default, and unhygienic macros, ones where lexical declarations stick, which is what you're looking for.

The good news is that Carl provided hygiene by default in the current experimental macros implementation in Rakudo. This is as it should be. He also, I think, has unhygienic macros working in 007.

The bad news is that he hasn't backported the unhygienic escape hatch from 007 (presuming it exists) to the current experimental macros in Rakudo. And while Carl understands this problem well, he's focusing on solving many others too before updating the P6 macros implementation in Rakudo.

The ugly news is that the reality appears to be that any improvement to P6 macros, including enabling unhygienic macros, is likely several years away. Folk could perhaps work with Carl to more quickly sort out declaring lexical variables in the current experimental macros implementation. And/or, if folk go help Carl with 007, the big leap forward that will come from 007 would likely arrive more quickly.

is there a way to define new variables from within a macro?

I'm pretty sure the answer is no, but perhaps Carl has figured out a way to fake this.