Perl Regex syntax

Question

I would like to use Perl to take a previously generated SPSS syntax file and format it for use in an R environment.

This is probably a very simple task for those familiar with Perl and regex, but I am stumbling.

The steps as I've laid them out for this Perl script are as follows:

1. Read in SPSS file
2. Find appropriate chunks of SPSS file (regex) for further processing and formatting
3. Further processing noted above (more regex)
4. Return R syntax to command line or preferably a file.

The basic format of SPSS value labels syntax is:

...A bunch of nonsense I do not care about...
...
 Value Labels
/gender
1 "M"
2 "F"
/purpose
1 "business"
2 "vacation"
3 "tiddlywinks"

execute . 
...Resume nonsense...

And the desired R syntax I am after looks like:

gender <- as.factor(gender
    , levels= c(1,2)
    , labels= c("M","F")
    )
...

Here is the Perl script I have written thus far. I have successfully read each line into the appropriate array. I have the general flow of what I need for the final print function, but I need to figure out how to ONLY print the appropriate @levels and @labels arrays for each @vars array.

#!/usr/bin/perl

#Need to change to read from argument in command line
open(VARVAL, "append.txt");
@lines = <VARVAL>;
close(VARVAL);

#Read through each line and put into a variable, a value, or a reject
#I really only want to read in everything between "value labels" and "execute ."
#That probably requires more regex...
foreach  (@lines){
    if ($_ =~ /\//){        #Anything with a / is a variable, remove the / and push
        $_ =~ tr/\///d;
        push(@vars, $_)
    } elsif ($_ =~/\d/) {
        push(@vals, $_)    #Anything that has a number in the line is a value
        }
}
#Splitting each @vals array into levels or labels arrays
foreach (@vals){
    @values = split(/\s+/, $_); #Splitting on a space, vunerable...better to split on first non digit character?
    foreach (@values) {
        if ($_ =~/\d/){
            push(@levels, $_);
        } else {
            push(@labels, $_)
        }
    }
}

#Get rid of newline
#I should provavly do this somewhere else?
chomp(@vars);
chomp(@levels);
chomp(@labels);

#Need to tell it when to stop adding in @levels & @labels. While loop? Hash lookup?
#Need to get rid of final comma
#Need to redirect output to a file
foreach (@vars){
    print $_ ." <- as.factor(" . $_ . "\n\t, levels = c(" ;
         foreach (@levels){
            print $_ . ",";
         }
    print ")\n\t, labels = c(";
    foreach(@labels){
            print $_ . ",";
        }
    print ")\n\t)\n";
}

And finally, here is sample output from the script as it currently runs:

gender <- as.factor(gender
    , levels = c(1,2,1,2,3,)
    , labels = c("M","F","biz","action","tiddlywinks",)
    )

I need this to only include levels 1,2 and labels M and F.

Thanks for the help!

Answer 1

This seems to work for me:

#!/usr/bin/env perl
use strict;
use warnings;

my @lines = <DATA>;

my $current_label = '';
my @ordered_labels;
my %data;
for my $line (@lines) {
    if ( $line =~ /^\/(.*)$/ ) { # starts with slash
        $current_label = $1;
        push @ordered_labels, $current_label;
        next;
    }
    if ( length $current_label ) {
        if ( $line =~ /^(\d) "(.*)"$/ ) {
            $data{$current_label}{$1} = $2;
            next;
        }
    }
}

for my $label ( @ordered_labels ) {
    print "$label <- as.factor($label\n";
    print "    , levels= c(";
    print join(',',map { $_ } sort keys %{$data{$label}} );
    print ")\n";
    print "    , labels= c(";
    print join(',',
        map { '"' . $data{$label}{$_} . '"'  }
        sort keys %{$data{$label}} );
    print ")\n";
    print "    )\n";
}

__DATA__
...A bunch of nonsense I do not care about...
...
 Value Labels
/gender
1 "M"
2 "F"
/purpose
1 "business"
2 "vacation"
3 "tiddlywinks"

execute . 

And yields:

gender <- as.factor(gender
    , levels= c(1,2)
    , labels= c("M","F")
    )
purpose <- as.factor(purpose
    , levels= c(1,2,3)
    , labels= c("business","vacation","tiddlywinks")
    )