shell script: search and replace over multiple lin

Question

I'm looking for a way to search and replace over multiple lines through a shell script. This is what I'm trying to do:

source:
[stuff before]
<!--WIERD_SPECIAL_COMMENT_BEGIN-->
  [stuff here, possibly multiple lines.
<!--WIERD_SPECIAL_COMMENT_END-->
[stuff after]    

target:
[stuff before]
[new content]
[stuff after]

In short, I want to delete the comments and everything between them and replace with some new content. Basically, I want to do a simple sed command over multiple lines, and if possible just using some basic *nix tools, no additional scripting language.

Answer 1

If you only need to match complete lines then you can do this task with awk. Something like:

    awk -v NEWTEXT=foo 'BEGIN{n=0} /COMMENT_BEGIN/ {n=1} {if (n==0) {print $0}} /COMMENT_END/ {print NEWTEXT; n=0}' < myfile.txt

If the file is not so well formatted, with comments on the same line as text you want to keep or remove, then I would use perl, read the entire file into a single string, do a regular expression match and replace on that string, then write the new string to a new file. This is not so simple and you need to write a perl script to do the work. Something like:

#!/usr/bin/perl
$newtext = "foo\nbar";
$/ = '';  # no input separator so whole file is read.
$s = <>;  # read whole file from stdin
$startPattern = quotemeta('<!--WIERD_SPECIAL_COMMENT_BEGIN-->');
$endPattern = quotemeta('<!--WIERD_SPECIAL_COMMENT_END-->');
$pattern = $startPattern . '.+' . $endPattern;
$s =~ s/$pattern/$newtext/sg;
print $s;

Answer 2

another solution... this is possible to be done in a one-liner, but using perl regular expressions, which I find easier to work with than sed or awk (which are cumbersome with multi-line match and replace):

perl -0 -i -pe 's/<!--WIERD_SPECIAL_COMMENT_BEGIN-->[\s\S]*<!--WIERD_SPECIAL_COMMENT_END-->/your new content here/gim' yourfile1.txt

please note that this will replace the file with the new, changed content.

Answer 3

sed does this just fine. The following is as simple as it gets; if you need to extract stuff from the delimiter line before the start delimiter or after the end delimiter, that's going to be a little more complex.

sed '/<!--WIERD_SPECIAL_COMMENT_BEGIN-->/,/<!--WIERD_SPECIAL_COMMENT_END-->/d' input >output

If you have any control over this, fix the spelling of "weird".