Removing lines from multiple files with sed comman

问题:

So, disclaimer: I am pretty new to using bash and zsh, so there is a chance the answer is really simple. Nonetheless. I checked previous postings and couldn't find anything. (edit: I have tried this in both bash and zsh shells- same problem.)

I have a directory with many files and am trying to remove the first line from each file.

So say the directory contains: file1.txt file2.txt file3.txt ... etc.

I am using the sed command (non-GNU):

sed -i -e "1d" *.txt

For some reason, this is only removing the first line of the first file. I thought that the *.txt would affect all files matching the pattern in directory. Strangely, it is creating the file duplicates with -e appended, but both the duplicate and original are the same.

I tried this with other commands (e.g. ls *.txt) and it works fine. Is there something about sed I am missing?

Thank you in advance.

回答1:

Different versions of sed in differing operating systems support various parameters.

OpenBSD (5.4) sed

The -i flag is unavailable. You can use the following /bin/sh syntax:

for i in *.txt
do
    f=`mktemp -p .` 
    sed -e "1d" "${i}" > "${f}" && mv -- "${f}" "${i}"
done

FreeBSD (11-CURRENT) sed

The -i flag requires an extension, even if it's empty. Thus must be written as sed -i "" -e "1d" *.txt

GNU sed

This looks to see if the argument following -i is another option (or possibly a command). If so, it assumes an in-place modification. If it appears to be a file extension such as ".bak", it will rename the original with the ".bak" and then modify it into the original file's name.

There might be other variations on other platforms, but those are the three I have at hand.

回答2:

1. use it without -e !

for one file use:

sed -i '1d' filename

for all files use :

sed -i '1d' *.txt

or

files=/path/to/files/*.extension ; for var in $files ; do sed -i '1d' $var ; done

.for me i use ubuntu and debian based systems , this method is working for me 100% , but for other platformes i'm not sure , so this is other method :

2. replace first line with emty pattern , and remove empty lines , (double commands):

   for files in $(ls /path/to/files/*.txt); do sed -i "s/$(head -1 "$files")//g" "$files" ; sed -i '/^$/d' "$files" ; done

Note: if your files contain splash '/' , then it will give error , so in this case sed command should look like this ( sed -i "s[$(head -1 "$files")[[g" )

hope that's what you're looking for :)

回答3:

The issue here is that the line number isn't reset when sed opens a new file, so 1 only matches the first line of the first file.

One solution is to use a shell loop, calling sed once for each file. Gumnos' answer shows how to do this in the most widely compatible way, although if you have a version of sed supporting the -i flag, you could do this instead:

for i in *.txt; do
    sed -i.bak '1d' "$i"
done

It is possible to avoid creating the backup file by passing an empty suffix but personally, I don't think it's such a bad thing. One day you'll be grateful for it!

---

It appears that you're not working with GNU tools but if you were, I would recommend using GNU awk for this task. The variable FNR is useful here, as it keeps track of the record number for each file individually, allowing you to do this:

gawk -i inplace 'FNR>1' *.txt

Using the inplace extension, this allows you to remove the first line from each of your files, by only printing the lines where FNR is greater than 1.

Testing it out:

$ seq 5 > file1
$ seq 5 > file2
$ gawk -i inplace 'FNR>1' file1 file2
$ cat file1
2
3
4
5
$ cat file2
2
3
4
5

回答4:

The last argument you are passing to the Sed is the problem try something like this.

var=(`find *txt`)
for file in "${var[@]}"
do
    sed -i -e 1d $file
done

This did the trick for me.