diff to output only the file names

Question

I'm looking to run a Linux command that will recursively compare two directories and output only the file names of what is different. This includes anything that is present in one directory and not the other or vice versa, and text differences.

Answer 1

If you want to get a list of files that are only in one directory and not their sub directories and only their file names:

diff -q /dir1 /dir2 | grep /dir1 | grep -E "^Only in*" | sed -n 's/[^:]*: //p'

If you want to recursively list all the files and directories that are different with their full paths:

diff -rq /dir1 /dir2 | grep -E "^Only in /dir1*" | sed -n 's/://p' | awk '{print $3"/"$4}'

This way you can apply different commands to all the files.

For example I could remove all the files and directories that are in dir1 but not dir2:

diff -rq /dir1 /dir2 | grep -E "^Only in /dir1*" | sed -n 's/://p' | awk '{print $3"/"$4}' xargs -I {} rm -r {}

Answer 2

On my linux system to get just the filenames

diff -q /dir1 /dir2|cut -f2 -d' '

Answer 3

From the diff man page:

-q   Report only whether the files differ, not the details of the differences.
-r   When comparing directories, recursively compare any subdirectories found.

Example command:

diff -qr dir1 dir2

Example output (depends on locale):

$ ls dir1 dir2
dir1:
same-file  different  only-1

dir2:
same-file  different  only-2
$ diff -qr dir1 dir2
Files dir1/different and dir2/different differ
Only in dir1: only-1
Only in dir2: only-2

Answer 4

rsync -rvc --delete --size-only --dry-run source dir target dir

Answer 5

You can also use rsync

rsync -rv --size-only --dry-run /my/source/ /my/dest/ > diff.out

Answer 6

The approach of running diff -qr old/ new/ has one major drawback: it may miss files in newly created directories. E.g. in the example below the file data/pages/playground/playground.txt is not in the output of diff -qr old/ new/ whereas the directory data/pages/playground/ is (search for playground.txt in your browser to quickly compare). I also posted the following solution on Unix & Linux Stack Exchange, but I'll copy it here as well:

To create a list of new or modified files programmatically the best solution I could come up with is using rsync, sort, and uniq:

(rsync -rcn --out-format="%n" old/ new/ && rsync -rcn --out-format="%n" new/ old/) | sort | uniq

Let me explain with this example: we want to compare two dokuwiki releases to see which files were changed and which ones were newly created.

We fetch the tars with wget and extract them into the directories old/ and new/:

wget http://download.dokuwiki.org/src/dokuwiki/dokuwiki-2014-09-29d.tgz
wget http://download.dokuwiki.org/src/dokuwiki/dokuwiki-2014-09-29.tgz
mkdir old && tar xzf dokuwiki-2014-09-29.tgz -C old --strip-components=1
mkdir new && tar xzf dokuwiki-2014-09-29d.tgz -C new --strip-components=1

Running rsync one way might miss newly created files as the comparison of rsync and diff shows here:

rsync -rcn --out-format="%n" old/ new/

yields the following output:

VERSION
doku.php
conf/mime.conf
inc/auth.php
inc/lang/no/lang.php
lib/plugins/acl/remote.php
lib/plugins/authplain/auth.php
lib/plugins/usermanager/admin.php

Running rsync only in one direction misses the newly created files and the other way round would miss deleted files, compare the output of diff:

diff -qr old/ new/

yields the following output:

Files old/VERSION and new/VERSION differ
Files old/conf/mime.conf and new/conf/mime.conf differ
Only in new/data/pages: playground
Files old/doku.php and new/doku.php differ
Files old/inc/auth.php and new/inc/auth.php differ
Files old/inc/lang/no/lang.php and new/inc/lang/no/lang.php differ
Files old/lib/plugins/acl/remote.php and new/lib/plugins/acl/remote.php differ
Files old/lib/plugins/authplain/auth.php and new/lib/plugins/authplain/auth.php differ
Files old/lib/plugins/usermanager/admin.php and new/lib/plugins/usermanager/admin.php differ

Running rsync both ways and sorting the output to remove duplicates reveals that the directory data/pages/playground/ and the file data/pages/playground/playground.txt were missed initially:

(rsync -rcn --out-format="%n" old/ new/ && rsync -rcn --out-format="%n" new/ old/) | sort | uniq

yields the following output:

VERSION
conf/mime.conf
data/pages/playground/
data/pages/playground/playground.txt
doku.php
inc/auth.php
inc/lang/no/lang.php
lib/plugins/acl/remote.php
lib/plugins/authplain/auth.php
lib/plugins/usermanager/admin.php

rsync is run with theses arguments:

• -r to "recurse into directories",
• -c to also compare files of identical size and only "skip based on checksum, not mod-time & size",
• -n to "perform a trial run with no changes made", and
• --out-format="%n" to "output updates using the specified FORMAT", which is "%n" here for the file name only

The output (list of files) of rsync in both directions is combined and sorted using sort, and this sorted list is then condensed by removing all duplicates with uniq