Find and replace in file and overwrite file doesn&

Question

I would like to run a find and replace on an HTML file through the command line.

My command looks something like this:

sed -e s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html > index.html

When I run this and look at the file afterward, it is empty. It deleted the contents of my file.

When I run this after restoring the file again:

sed -e s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html

The stdout is the contents of the file, and the find and replace has been executed.

Why is this happening?

Answer 1

use sed's -i option, e.g.

sed -i bak -e s/STRING_TO_REPLACE/REPLACE_WITH/g index.html

Answer 2

To change multiple files (and saving a backup of each as *.bak):

perl -p -i -e "s/\|/x/g" *  

will take all files in directory and replace | with x this is called a “Perl pie” (easy as a pie)

Answer 3

When the shell sees > index.html in the command line it opens the file index.html for writing, wiping off all its previous contents.

To fix this you need to pass the -i option to sed to make the changes inline and create a backup of the original file before it does the changes in-place:

sed -i.bak s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html

Without the .bak the command will fail on some platforms, such as Mac OSX.

Answer 4

Warning: this is a dangerous method! It abuses the i/o buffers in linux and with specific options of buffering it manages to work on small files. It is an interesting curiosity. But don't use it for a real situation!

Besides the -i option of sed you can use the tee utility.

From man:

tee - read from standard input and write to standard output and files

So, the solution would be:

sed s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html | tee | tee index.html

-- here the tee is repeated to make sure that the pipeline is buffered. Then all commands in the pipeline are blocked until they get some input to work on. Each command in the pipeline starts when the upstream commands have written 1 buffer of bytes (the size is defined somewhere) to the input of the command. So the last command tee index.html, which opens the file for writing and therefore empties it, runs after the upstream pipeline has finished and the output is in the buffer within the pipeline.

Most likely the following won't work:

sed s/STRING_TO_REPLACE/STRING_TO_REPLACE_IT/g index.html | tee index.html

-- it will run both commands of the pipeline at the same time without any blocking. (Without blocking the pipeline should pass the bytes line by line instead of buffer by buffer. Same as when you run cat | sed s/bar/GGG/. Without blocking it's more interactive and usually pipelines of just 2 commands run without buffering and blocking. Longer pipelines are buffered.) The tee index.html will open the file for writing and it will be emptied. However, if you turn the buffering always on, the second version will work too.

Answer 5

An alternative, useful, pattern is:

sed -e 'script script' index.html > index.html.tmp && mv index.html.tmp index.html

That has much the same effect, without using the -i option, and additionally means that, if the sed script fails for some reason, the input file isn't clobbered. Further, if the edit is successful, there's no backup file left lying around. This sort of idiom can be useful in Makefiles.

Quite a lot of seds have the -i option, but not all of them; the posix sed is one which doesn't. If you're aiming for portability, therefore, it's best avoided.

Answer 6

sed -i.bak "s#https.*\.com#$pub_url#g" MyHTMLFile.html

If you have a link to be added, try this. Search for the URL as above (starting with https and ending with.com here) and replace it with a URL string. I have used a variable $pub_url here. s here means search and g means global replacement.

It works !