Perl command line multi-line replace

Question

I'm trying to replace text in a multi-line file using command-line perl. I'm using Ubuntu Natty.

Below is the content of my text file (called test.txt):

[mysqld]
#
# * Basic Settings
#

#
# * IMPORTANT
#   If you make changes to these settings and your system uses apparmor, you may
#   also need to also adjust /etc/apparmor.d/usr.sbin.mysqld.
#

user            = mysql
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
skip-external-locking

Below is my perl command:

perl -i -pe "s/(\[mysqld\][^\^]+)/\1\nsometext/g" test.txt

However, instead of replacing all the text in the file, below is what I end up with:

[mysqld]

sometext#
# * Basic Settings
#

#
# * IMPORTANT
#   If you make changes to these settings and your system uses apparmor, you may
#   also need to also adjust /etc/apparmor.d/usr.sbin.mysqld.
#

user            = mysql
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
skip-external-locking
#

I tried the Regex in RegexBuddy for Perl and it matches everything in the text file, but for some reason it's not working using perl on the command line.

I'd appreciate some assistance.

Thanks in advance.

Answer 1

The -p switch causes Perl to iterate over every line of the input and execute the given code for each of them (and to print the lines afterwards). Specifically, the command

perl -p -e 'SOME_CODE_HERE;'

is exactly equivalent to running the following Perl program:

LINE: while (<>) {
    SOME_CODE_HERE;
} continue {
    print or die "-p destination: $!\n";
}

Your regexp seems to be intended to match multiple lines at once, which obviously won't work if Perl is processing the input line by line. To make it work as intended, you have (at least) two options:

1. Change Perl's notion of what constitutes a line by using the -0NNN switch. In particular, the switch -0777 causes Perl to treat every input file as a single "line".

2. Rewrite your code to e.g. use the .. flip-flop operator.

By the way, I strongly suspect that your regexp doesn't mean what you think it means. In particular, [^\^]+ matches a string of one or more characters that doesn't contain a caret (^). Since your input doesn't seem likely to contain any carets, this seems essentially equivalent to (?s:.+) (or just .+ if you use the /s modifier).

Answer 2

You are reading the file line-by-line, so only the first line matches your regex. What you'll want to do -- if you truly wish to delete most of the content -- is to slurp the file by using the -0 option, e.g. -0777. This is line ending processing, and 777 is just a number used by convention as an octal number large enough so as to cause file slurping.

perl -0777 -i -pe 's/(\[mysqld\][^\^]+)/$1\nsometext/g' test.txt

Also, I replaced your quotes. If you are in *nix, which it seems you are, single quotes are preferable. Case in point, $1 would not be interpolated by the shell.