sed command creates randomly named files

Question

I recently wrote a script that does a sed command, to replace all the occurrences of "string1" with "string2" in a file named "test.txt".

It looks like this:

sed -i 's/string1/string2/g' test.txt

The catch is, "string1" does not necessarily exist in test.txt.

I notice after executing a bunch of these sed commands, I get a number of empty files, left behind in the directory, with names that look like this:

"sed4l4DpD"

Does anyone know why this might be, and how I can correct it?

Answer 1

So after much testing last night, it turns out that sed was creating these files when trying to operate on an empty string. The way i was getting the array of "$string1" arguments was through a grep command, which seems to be malformed. What I wanted from the grep was all lines containing something of the type "Text here '.'".

For example the string, "Text here 'ABC.DEF'" in a file, should have been caught by grep, then the ABC.DEF portion of the string, would be substituted by ABC_DEF. Unfortunately the grep I was using would catch lines of the type "Text here ''" (that is, nothing between the ''). When later on, the script attempted to perform a sed replacement using this empty string, the random file was created (probably because sed died).

Thanks for all your help in understanding how sed works.

Answer 2

Its better if you do it in this way:

cat large_file | sed 's/string1/string2/g' > file_filtred

Answer 3

I wasn't able to reproduce this with a quick test (using GNU sed 4.2.1) -- but strace did show sed creating a file called sedJd9Cuy and then renaming it to tmp (the file named on the command line).

It looks like something is going wrong after sed creates the temporary file and before it's able to rename it.

My best guess is that you've run out of room in the filesystem; you're able to create a new empty file, but unable to write to it.

What does df . say?

EDIT:

I still don't know what's causing the problem, but it shouldn't be too difficult to work around it.

Rather than

sed -i 's/string1/string2/g' test.txt

try something like this:

sed 's/string1/string2/g' test.txt > test.txt.$$ && mv -f test.txt.$$ test.txt

Something is going wrong with the way sed creates and then renames a text file to replace your original file. The above command uses sed as a simple input-output filter and creates and renames the temporary file separately.

Answer 4

-i is the suffix given to the new/output file. Also, you need -e for the command.
Here's how you use it:

sed -i '2' -e 's/string1/string2/g' test.txt

This will create a file called test.txt2 that is the backup of test.txt

To replace the file (instead of creating a new copy - called an "in-place" substitution), change the -i value to '' (ie blank):

sed -i '' -e 's/string1/string2/g' test.txt

EDIT II

Here's actual command line output from a Mac (Snow Leopard) that show that my modified answer (removed space from between the -i and the suffix) is correct.
NOTE: On a linux server, there must be no space between it -i and the suffix.

> echo "this is a test" > test.txt
> cat test.txt
this is a test
> sed -i '2' -e 's/a/a good/' test.txt 
> ls test*
test.txt    test.txt2
> cat test.txt
this is a good test
> cat test.txt2
this is a test
> sed -i '' -e 's/a/a really/' test.txt 
> ls test*
test.txt    test.txt2
> cat test.txt
this is a really good test