Difference between two directories in Linux [close

Question

I'm trying to find the files existing in one directory but not in the other, I tried to use this command:

diff -q dir1 dir2

The problem with the above command that it finds both the files in dir1 but not in dir2 as well as the files in dir2 but not in dir1,

I am trying to find the files in dir1 but not in dir2 only.

Here's a small sample of what my data looks like

dir1    dir2    dir3
1.txt   1.txt   1.txt
2.txt   3.txt   3.txt
5.txt   4.txt   5.txt
6.txt   7.txt   8.txt

Another question on my mind is how can I find the files in dir1 but not in dir2 or dir3 in a single command?

Answer 1

This is the bash script to print commands for syncing two directories

dir1=/tmp/path_to_dir1
dir2=/tmp/path_to_dir2
diff -rq $dir1 $dir2 | sed -e "s|Only in $dir2\(.*\): \(.*\)|cp -r $dir2\1/\2 $dir1\1|" |  sed -e "s|Only in $dir1\(.*\): \(.*\)|cp -r $dir1\1/\2 $dir2\1|" 

Answer 2

This should do the job:

diff -rq dir1 dir2

Options explained (via diff(1) man page):

• -r - Recursively compare any subdirectories found.
• -q - Output only whether files differ.

Answer 3

Unsatisfied with all the replies, since most of them work very slowly and produce unnecessarily long output for large directories, I wrote my own Python script to compare two folders.

Unlike many other solutions, it doesn't compare contents of the files. Also it doesn't go inside subdirectories which are missing in another directory. So the output is quite concise and the script works fast.

#!/usr/bin/env python3

import os, sys

def compare_dirs(d1: "old directory name", d2: "new directory name"):
    def print_local(a, msg):
        print('DIR ' if a[2] else 'FILE', a[1], msg)
    # ensure validity
    for d in [d1,d2]:
        if not os.path.isdir(d):
            raise ValueError("not a directory: " + d)
    # get relative path
    l1 = [(x,os.path.join(d1,x)) for x in os.listdir(d1)]
    l2 = [(x,os.path.join(d2,x)) for x in os.listdir(d2)]
    # determine type: directory or file?
    l1 = sorted([(x,y,os.path.isdir(y)) for x,y in l1])
    l2 = sorted([(x,y,os.path.isdir(y)) for x,y in l2])
    i1 = i2 = 0
    common_dirs = []
    while i1<len(l1) and i2<len(l2):
        if l1[i1][0] == l2[i2][0]:      # same name
            if l1[i1][2] == l2[i2][2]:  # same type
                if l1[i1][2]:           # remember this folder for recursion
                    common_dirs.append((l1[i1][1], l2[i2][1]))
            else:
                print_local(l1[i1],'type changed')
            i1 += 1
            i2 += 1
        elif l1[i1][0]<l2[i2][0]:
            print_local(l1[i1],'removed')
            i1 += 1
        elif l1[i1][0]>l2[i2][0]:
            print_local(l2[i2],'added')
            i2 += 1
    while i1<len(l1):
        print_local(l1[i1],'removed')
        i1 += 1
    while i2<len(l2):
        print_local(l2[i2],'added')
        i2 += 1
    # compare subfolders recursively
    for sd1,sd2 in common_dirs:
        compare_dirs(sd1, sd2)

if __name__=="__main__":
    compare_dirs(sys.argv[1], sys.argv[2])

Sample usage:

user@laptop:~$ python3 compare_dirs.py dir1/ dir2/
DIR  dir1/out/flavor-domino removed
DIR  dir2/out/flavor-maxim2 added
DIR  dir1/target/vendor/flavor-domino removed
DIR  dir2/target/vendor/flavor-maxim2 added
FILE dir1/tmp/.kconfig-flavor_domino removed
FILE dir2/tmp/.kconfig-flavor_maxim2 added
DIR  dir2/tools/tools/LiveSuit_For_Linux64 added

Or if you want to see only files from the first directory:

user@laptop:~$ python3 compare_dirs.py dir2/ dir1/ | grep dir1
DIR  dir1/out/flavor-domino added
DIR  dir1/target/vendor/flavor-domino added
FILE dir1/tmp/.kconfig-flavor_domino added

P.S. If you need to compare file sizes and file hashes for potential changes, I published an updated script here: https://gist.github.com/amakukha/f489cbde2afd32817f8e866cf4abe779

Answer 4

This answer optimizes one of the suggestions from @Adail-Junior by adding the -D option, which is helpful when neither of the directories being compared are git repositories:

git diff -D --no-index dir1/ dir2/

If you use -D then you won't see comparisons to /dev/null: text Binary files a/whatever and /dev/null differ

Answer 5

diff -r dir1 dir2 | grep dir1 | awk '{print $4}' > difference1.txt

Explanation:

• diff -r dir1 dir2 shows which files are only in dir1 and those only in dir2 and also the changes of the files present in both directories if any.

• diff -r dir1 dir2 | grep dir1 shows which files are only in dir1

• awk to print only filename.

Answer 6

vim's DirDiff plugin is another very useful tool for comparing directories.

vim -c "DirDiff dir1 dir2"

It not only lists which files are different between the directories, but also allows you to inspect/modify with vimdiff the files that are different.