Flex and Bison: Beginning a sentence with a specif

Question

I am working on a program using Flex and Bison. My task can be done using only Flex(using start conditions etc.), but I have understood that using Bison might make my life easier. My task is to design a program which recognizes a programming language declaration part. Its syntax and logic can be understood through my code below. My problem is that I want my program to recognize as an acceptable declaration part every part of code which begins only with the "var" keyword! Until now, I have not managed to do that. How can I succeed it?

I present below my .l (flex) and .y (bison) files.

exercise4.l

%{
#include <stdio.h>
%}
%%
[ \t\n]+                { /* Ignore all whitespaces */ }
";"         { /* Ignore all semicolons */ }
":"         { /* Ignore all colons */ }
var                     { printf("A keyword: %s\n",yytext); return VAR; }
real                    { printf("A variable type: %s\n",yytext); return REAL; }
boolean                 { printf("A variable type: %s\n",yytext); return BOOLEAN; }
integer                 { printf("A variable type: %s\n",yytext); return INTEGER; }
char                    { printf("A variable type: %s\n",yytext); return CHAR; }
[a-zA-Z][a-zA-Z0-9_]*   { printf("A variable name: %s\n",yytext); return VAR_NAME; }
.                       { printf("Unrecognized character!\n"); return yytext[0]; }
%%

exercise4.y

%{
#include <stdio.h>    
%}
%token VAR VAR_NAME REAL BOOLEAN INTEGER CHAR
%%
program : VAR typedecls ;
typedecls : typedecl | typedecls typedecl ;
typedecl : varlist ':' var_type ';' ;
varlist : VAR_NAME | varlist ',' VAR_NAME ;
var_type : REAL | BOOLEAN | INTEGER | CHAR ;
%%
main( argc, argv )
int argc;
char **argv;
{
extern FILE *yyin;
++argv, --argc; /* skip over program name */
if ( argc > 0 )
yyin = fopen( argv[0], "r" );
else
yyin = stdin;
//yylex();
yyparse();
}
yyerror(char *s)  
{  
 printf("\nError\n");  
}
#include "lex.yy.c"

Answer 1

Your scanner never returns : or ; tokens (and complains about , being unrecognized). But your grammar includes those tokens.

Consequently,

typedecl : varlist ':' var_type ';' ;

can never match anything. It doesn't see the : it is expecting, and the var_type is thus unexpected. (And it wouldn't ever match the ; if it got that far.)

---

I don't know what ancient tutorial you're following, but the pre-ansi prototype for main suggests it was composed in the last century (and not in its final years, either). As a rough guide, I slightly updated your files.

Here's exercise4.l; the most important modification is that I removed the handbuilt trace logs, since I intend to use flex's built-in debugging feature. I also added some options to reduce compiler warnings, and enabled line number tracking for use in error messages. I left in the bug.

/* File: exercise4.l */
%{
  /* The bison-generated header file includes token declarations */
  #include "exercise4.tab.h"
%}

%option noinput nounput noyywrap nodefault
%option yylineno

%%
[ \t\n]+                { /* Ignore all whitespaces */ }
";"                     { /* Ignore all semicolons. BUG */ }
":"                     { /* Ignore all colons. BUG */ }
var                     { return VAR; }
real                    { return REAL; }
boolean                 { return BOOLEAN; }
integer                 { return INTEGER; }
char                    { return CHAR; }
[a-zA-Z][a-zA-Z0-9_]*   { return VAR_NAME; }
.                       { return yytext[0]; }

And here's the parser, with a few fixes (like using the standard C prototype syntax from the 1989 C standard):

/* File: exercise4.y */
%{
  /* Used in this file */
  #include <stdio.h>

  /* Forward and external declarations */
  extern int yylineno;
  int yylex();
  void yyerror(const char* msg);
%}

 /* These two options make error messages more informative */    
%define parse.lac full
%error-verbose

%token VAR VAR_NAME REAL BOOLEAN INTEGER CHAR

%%

program   : VAR typedecls
typedecls : typedecl
          | typedecls typedecl
typedecl  : varlist ':' var_type ';'
varlist   : VAR_NAME
          | varlist ',' VAR_NAME ;
var_type  : REAL | BOOLEAN | INTEGER | CHAR ;

%%

/* Welcome to 1990 */
int main( int argc, char** argv ) {
  extern FILE *yyin;
  if ( argc > 1 )
    yyin = fopen( argv[1], "r" );
  else
    yyin = stdin;
  if (!yyin) {
    /* If you don't check, you'll end up segfaulting when
     * yylex tries to read from NULL. Checking lets us print
     * a hopefully meaningful error message.
     */
    perror("Could not open file for reading");
    return 1;
  }
  return yyparse();
}

/* Now that error messages have some content, it's worthwhile
 * actually using the argument passed to yyerror */
void yyerror(const char* msg) {
  fprintf(stderr, "At line %d: %s\n", yylineno, msg);
}

Now, I run the files through flex, bison and gcc, in that order, to produce an executable:

# The `-d` option to flex causes it to insert debugging traces. That's
# a lot less effort and a lot more useful than rolling your own trace
# logs.
flex -d -o exercise4.scan.c exercise4.l
# The `-d` option to bison causes it to output a header file, whose name
# is the same as the output file name with `.c` changed to `.h`. That's
# the file which we need to `#include` in the scanner, so that token
# names are available to scanner actions. Bison also has a debugging
# feature, but you need to use `-t` to enable it, plus add a line at
# runtime. For now, we'll leave it out.
bison -d -o exercise4.tab.c exercise4.y
# Now compile and link an executable:
gcc -Wall -g -o exercise4 exercise4.tab.c exercise4.scan.c

Normally, the three shell commands would go into a Makefile so that you could just type make exercise4 to create the executable. But there's nothing really complicated there.

Now, we can try it out. Since I didn't fix the bugs, you'll easily see the problem:

var a, b: integer;
--accepting rule at line 14 ("var")
--accepting rule at line 11 (" ")
--accepting rule at line 19 ("a")
--accepting rule at line 20 (",")
--accepting rule at line 11 (" ")
--accepting rule at line 19 ("b")
--accepting rule at line 13 (":")
--accepting rule at line 11 (" ")
--accepting rule at line 17 ("integer")
At line 1: syntax error, unexpected INTEGER, expecting ':' or ','

The lines starting -- are flex's trace. You can see each token as it is recognized. So when a : is encountered in the input, the rule at line 13 is executed. That rule is { /* Ignore all colons */ }. And that's what happens.

The error message at the end was printed by yyerror. Thanks to the %errors-verbose directive, it tells us what it found (INTEGER) and what it was expecting.

Hope that all helps some.