Merging files (cat) in each-folder Unix

Question

Inside my main folder, I have multiple sub-folders and each sub folder contains multiple files. I want to merge these files in every sub-folder.

So I am trying to do something like this:

cd ../master-folder

for file in $( find . -name "*.txt" );
do
cat "all the text files in this sub folder" > "name of the subfolder.txt"
rm  "all the previous text files excluding the merged output obviously"
    done

Appreciate the help! Thank you.

Answer 1

I would do it like this, if the order of the files doesn't matter :

for i in $(find -maxdepth 1 -mindepth 1 -type d)
do
    find $i -name '*.txt' -type f -exec cat {} >> $i-list.txt \;
    find $i -name '*.txt' -type f -exec rm {} \;
done

The first find looks for subdirectories.

The second one appends all subfile's content to a file

The third one deletes the subfiles.

This doesn't work if there are recursive subdirectories. If you want this, remove '-maxdepth 1'

Answer 2

Why not visit each directory recursively? Something along the lines of:

#!/bin/bash                                                                     

shopt -s nullglob # Make failed globs expand to nothing

function visit {
    pushd "$1"
    txts=(*.txt)
    if ((${#txts[@]} > 0))
    then
        cat "${txts[@]}" > "${PWD##*/}.txt"
        rm -f "${txts[@]}"
    fi
    for dir in */
    do
        visit "$dir"
    done
    popd
}

visit /path/to/start/dir

Caveat: If you have sym links that create cycles in your directory tree then this is a bad idea.