How do I copy an entire directory of files into an

Question

Run the following code from a directory that contains a directory named bar (containing one or more files) and a directory named baz (also containing one or more files). Make sure there is not a directory named foo.

import shutil
shutil.copytree('bar', 'foo')
shutil.copytree('baz', 'foo')

It will fail with:

$ python copytree_test.py 
Traceback (most recent call last):
  File "copytree_test.py", line 5, in <module>
    shutil.copytree('baz', 'foo')
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/shutil.py", line 110, in copytree
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/os.py", line 172, in makedirs
OSError: [Errno 17] File exists: 'foo'

I want this to work the same way as if I had typed:

$ mkdir foo
$ cp bar/* foo/
$ cp baz/* foo/

Do I need to use shutil.copy() to copy each file in baz into foo? (After I've already copied the contents of 'bar' into 'foo' with shutil.copytree()?) Or is there an easier/better way?

Answer 1

The previous solution has some issue that src may overwrite dst without any notification or exception.

I add a predict_error method to predict errors before copy.copytree mainly base on Cyrille Pontvieux's version.

Using predict_error to predict all errors at first is best, unless you like to see exception raised one by another when execute copytree until fix all error.

def predict_error(src, dst):  
    if os.path.exists(dst):
        src_isdir = os.path.isdir(src)
        dst_isdir = os.path.isdir(dst)
        if src_isdir and dst_isdir:
            pass
        elif src_isdir and not dst_isdir:
            yield {dst:'src is dir but dst is file.'}
        elif not src_isdir and dst_isdir:
            yield {dst:'src is file but dst is dir.'}
        else:
            yield {dst:'already exists a file with same name in dst'}

    if os.path.isdir(src):
        for item in os.listdir(src):
            s = os.path.join(src, item)
            d = os.path.join(dst, item)
            for e in predict_error(s, d):
                yield e


def copytree(src, dst, symlinks=False, ignore=None, overwrite=False):
    '''
    would overwrite if src and dst are both file
    but would not use folder overwrite file, or viceverse
    '''
    if not overwrite:
        errors = list(predict_error(src, dst))
        if errors:
            raise Exception('copy would overwrite some file, error detail:%s' % errors)

    if not os.path.exists(dst):
        os.makedirs(dst)
        shutil.copystat(src, dst)
    lst = os.listdir(src)
    if ignore:
        excl = ignore(src, lst)
        lst = [x for x in lst if x not in excl]
    for item in lst:
        s = os.path.join(src, item)
        d = os.path.join(dst, item)
        if symlinks and os.path.islink(s):
            if os.path.lexists(d):
                os.remove(d)
            os.symlink(os.readlink(s), d)
            try:
                st = os.lstat(s)
                mode = stat.S_IMODE(st.st_mode)
                os.lchmod(d, mode)
            except:
                pass  # lchmod not available
        elif os.path.isdir(s):
            copytree(s, d, symlinks, ignore)
        else:
            if not overwrite:
                if os.path.exists(d):
                    continue
            shutil.copy2(s, d)

Answer 2

You can modify shutil and get the effect (on my version of shutil this is on line 315)

Change

os.makedirs(dst)

To

os.makedirs(dst,exist_ok=True)

Answer 3

Try This:

import os,shutil

def copydir(src, dst):
  h = os.getcwd()
  src = r"{}".format(src)
  if not os.path.isdir(dst):
     print("\n[!] No Such directory: ["+dst+"] !!!")
     exit(1)

  if not os.path.isdir(src):
     print("\n[!] No Such directory: ["+src+"] !!!")
     exit(1)
  if "\\" in src:
     c = "\\"
     tsrc = src.split("\\")[-1:][0]
  else:
    c = "/"
    tsrc = src.split("/")[-1:][0]

  os.chdir(dst)
  if os.path.isdir(tsrc):
    print("\n[!] The Directory Is already exists !!!")
    exit(1)
  try:
    os.mkdir(tsrc)
  except WindowsError:
    print("\n[!] Error: In[ {} ]\nPlease Check Your Dirctory Path !!!".format(src))
    exit(1)
  os.chdir(h)
  files = []
  for i in os.listdir(src):
    files.append(src+c+i)
  if len(files) > 0:
    for i in files:
        if not os.path.isdir(i):
            shutil.copy2(i, dst+c+tsrc)

  print("\n[*] Done ! :)")

copydir("c:\folder1", "c:\folder2")

Answer 4

A merge one inspired by atzz and Mital Vora:

#!/usr/bin/python
import os
import shutil
import stat
def copytree(src, dst, symlinks = False, ignore = None):
  if not os.path.exists(dst):
    os.makedirs(dst)
    shutil.copystat(src, dst)
  lst = os.listdir(src)
  if ignore:
    excl = ignore(src, lst)
    lst = [x for x in lst if x not in excl]
  for item in lst:
    s = os.path.join(src, item)
    d = os.path.join(dst, item)
    if symlinks and os.path.islink(s):
      if os.path.lexists(d):
        os.remove(d)
      os.symlink(os.readlink(s), d)
      try:
        st = os.lstat(s)
        mode = stat.S_IMODE(st.st_mode)
        os.lchmod(d, mode)
      except:
        pass # lchmod not available
    elif os.path.isdir(s):
      copytree(s, d, symlinks, ignore)
    else:
      shutil.copy2(s, d)

• Same behavior as shutil.copytree, with symlinks and ignore parameters
• Create directory destination structure if non existant
• Will not fail if dst already exists

Answer 5

In slight improvement on atzz's answer to the function where the above function always tries to copy the files from source to destination.

def copytree(src, dst, symlinks=False, ignore=None):
    if not os.path.exists(dst):
        os.makedirs(dst)
    for item in os.listdir(src):
        s = os.path.join(src, item)
        d = os.path.join(dst, item)
        if os.path.isdir(s):
            copytree(s, d, symlinks, ignore)
        else:
            if not os.path.exists(d) or os.stat(s).st_mtime - os.stat(d).st_mtime > 1:
                shutil.copy2(s, d)

In my above implementation

• Creating the output directory if not already exists
• Doing the copy directory by recursively calling my own method.
• When we come to actually copying the file I check if the file is modified then only we should copy.

I am using above function along with scons build. It helped me a lot as every time when I compile I may not need to copy entire set of files.. but only the files which are modified.

Answer 6

This limitation of the standard shutil.copytree seems arbitrary and annoying. Workaround:

def copytree(src, dst, symlinks=False, ignore=None):
    for item in os.listdir(src):
        s = os.path.join(src, item)
        d = os.path.join(dst, item)
        if os.path.isdir(s):
            shutil.copytree(s, d, symlinks, ignore)
        else:
            shutil.copy2(s, d)

Note that it's not entirely consistent with the standard copytree:

• it doesn't honor symlinks and ignore parameters for the root directory of the src tree;
• it doesn't raise shutil.Error for errors at the root level of src;
• in case of errors during copying of a subtree, it will raise shutil.Error for that subtree instead of trying to copy other subtrees and raising single combined shutil.Error.