Bash: How to tail then copy multiple files (eg usi

问题:

I've been trying various combinations of xargs and piping but I just can't get the right result. Previous questions don't quite cover exactly what I want to do:

• I have a source directory somewhere, lets say /foo/source, with a mix of different files
• I want to copy just the csv files found in source to a different destination, say /foo/dest
• But I ALSO at the same time need to remove 232 header rows (eg using tail)

I've figured out that I need to pipe the results of find into xargs, which can then run commands on each find result. But I'm struggling to tail then copy. If I pipe tail into cp, cp does not seem to receive the file (missing file operand). Here's some examples of what I've tried so far:

find /foo/source -name "*.csv" | xargs -I '{}' sh -c 'tail -n +232 | cp -t /foo/dest'

cp: missing file operand

find /foo/source -name "*.csv" | xargs -I '{}' sh -c 'tail -n +232 {} | cp -t /foo/dest'

Result:

cp: failed to access '/foo/dest': No such file or directory ...

find /foo/source -name "*.csv" | xargs -I '{}' sh -c 'tail -n +232 {} > /foo/dest/{}'

sh: /foo/dest/foo/source/0001.csv: No such file or directory ...

Any pointers would be really appreciated!

Thanks

回答1:

Your last command is close, but the problem is that {} is replaced with the full pathname, not just the filename. Use the basename command to extract the filename from it.

find /foo/source -name "*.csv" | xargs -I '{}' sh -c 'tail -n +232 {} > /foo/dest/$(basename {})'

回答2:

Just use find with exec and copy the file name in a variable:

find your_dir -name "*.csv" -exec sh -c 'f="$1"; tail -n +5 "$f" > dest_dir/$(basename "$f")' -- {} \;

See f={} makes $f hold the name of the file, with the full path. Then, it is a matter of redirecting the output of tail into the file, stripping the path from it.

Or, based on Random832's suggestion below in comments (thanks!):

find your_dir -name "*.csv" -exec sh -c 'tail -n +5 "$1" > dest_dir/$(basename "$1")' -- {} \;

回答3:

As an alternative to find and xargs you could use a for loop, and as an alternative to tail you could use sed, consider this:

source=/foo/source
dest=/foo/dest
for csv in $source/*.csv; do sed '232,$ !d' $csv > $dest/$(basename $csv); done

回答4:

Using GNU Parallel you would do:

find /foo/source -name "*.csv" | parallel tail -n +232 {} '>' /foo/dest/{/}