Convert ls output into csv

问题:

How would I convert:

$ find . -ls > /tmp/files.txt

Which gives me something like:

908715       40 -rwxrwxr-x    1 david            staff               16542 Nov 15 14:12 ./dump_info.py
908723        0 drwxr-xr-x    2 david            staff                  68 Nov 20 17:35 ./metadata

Into a csv output? It would look like:

908715,40,-rwxrwxr-x,1,david,staff,16542,Nov 15 14:12,./dump_info.py
908723,0,drwxr-xr-x,2,david,staff,68,Nov 20 17:35,./metadata

Here is an example title with spaces in the filename:

652640,80,-rw-rw-r--,1,david,staff,40036,Nov,6,15:32,./v_all_titles/V Catalog Report 11.5.xlsx

回答1:

It's a bit long to type in at the command-line, but it properly preserves spaces in the filename (and quotes it, too!)

find . -ls | python -c '
import sys
for line in sys.stdin:
    r = line.strip("\n").split(None, 10)
    fn = r.pop()
    print ",".join(r) + ",\"" + fn.replace("\"", "\"\"") + "\""
'

回答2:

If you don't care about the spaces in the date:

$ find . -ls | tr -s ' ' ,

If you do care about those spaces:

$ find . -ls | awk '{printf( "%s,%s,%s,%s,%s,%s,%s,%s %s %s,%s\n", $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11 )}'

Neither of these will work if your filenames contain any whitespace. As a hack to deal with spaces in the filename, you could try:

 ... | sed 's/,/ /8g'

to get rid of all but the first 8 commas (assuming your sed supports the nonstandard 8g option as gnu sed does). Of course this won't deal with commas in the filename.

回答3:

And yet another variant. See section "-printf format" in man page of find to customize.

$ find . -type f -fprintf /tmp/files.txt "%i,%b,%M,%n,%u,%g,%s,%CY-%Cm-%Cd %CT,%p\n"

Example output:

$ less /tmp/files.txt

3414558,40,-rw-rw-r--,1,webwurst,webwurst,16542,2014-09-18 15:54:36.9232917780,./dump_info.py
3414559,8,-rw-rw-r--,1,webwurst,webwurst,68,2014-09-18 15:54:51.1752922580,./metadata

回答4:

Here's a python script I drafted...

#!/opt/app/python/bin/python
# Convert ls output to clean csv    Paolo Villaflores 2015-03-16
#
# Sample usage: ls -l | ls2csv.py
#
# Features:
#   accepts -d argument to change dates to yyyy-mm-dd_hhmm format
#   input is via stdin
#   separate file/directory field
#   handle -dils type input (find -ls) versus -l
#   handle space in filename, by applying quotes around filename
#   handle date - format into something excel can handle correctly, whether it is from current year or not.
#   adds a header
#   handle symlinks - type l



import sys
from datetime import datetime

b0=True

def is_f(s):
  if s == '-':
    return 'f'
  return s

for line in sys.stdin:
    if len(line) < 40:
      continue
    if b0:
      b1=line[0] in ['-', 'd', 'c', 'l'] # c is for devices e.g. /devices/pseudo/pts@0:5, l is for symbolic link
      b0=False
      if b1:  # true when shorter ls -l style 8/9 columns. 9 for symlink
        cols=7
        print "d,perms,#links,owner,group,size,modtime,name,symlink"
      else:
        cols=9
        print "inode,bsize,d,perms,#links,owner,group,size,modtime,name,symlink"
    r = line.strip("\n").split(None, cols+1)
    if len(r) < cols+1:
      continue
    if r[cols-7][0] == 'c':
       continue  # ignore c records: devices
    fn = r.pop()
    if b1:
      c = ''
    else:
      c = ",".join(r[0:2]) + ","
    z = 0
    z = r[cols].find(':')
    if z < 0:
      d = r[cols - 1] + "/" + r[cols - 2] + "/" + r[cols]
    else:
      n = str(datetime.now()  )
      d = ''
      # handle the case where the timestamp has no year field
      tm=datetime.strptime(r[cols-2]+ " " + r[cols-1]+ " " + n[:4] +" " + r[cols], "%b %d %Y %H:%M")
      if (tm-datetime.now()).days > 0:
        d = r[cols - 1] + "/" + r[cols - 2] + "/" + str((datetime.now().year-1)) + " " + r[cols]
        tm=datetime.strptime(r[cols-2]+ " " + r[cols-1]+ " " + str(int(n[:4])-1) +" " + r[cols], "%b %d %Y %H:%M")
      else:
        d = r[cols - 1] + "/" + r[cols - 2] + "/" + " ".join([n[:4], r[cols] ] )
      if len(sys.argv) > 1 and sys.argv[1] == '-d':
        d=tm.strftime("%Y-%m-%d_%H%M")

    y = fn.find(">")
    symlink=''
    if y > 0:
       symlink = ',\"' + fn[y+2:] + '"'
       fn = fn[:y-2]
    if  fn.find( " ") <0:
      if fn.find('"') <0:
        fn2=fn
      else:
        fn2="'" + fn + "'"
    else:
      fn2="'" + fn + "'"
    print c+ is_f(r[cols-7][0]) + ",\"" + r[cols-7][1:] + "\"," + ",".join(
      r[cols-6:cols-2]) + "," + d + "," + fn2 + symlink

回答5:

this should do the job

 find . -ls|awk 'BEGIN{OFS=","}$1=$1'

回答6:

ls target 

boto3-1.11.3-py2.py3-none-any.whl
engagment-states-batch-rds-loader-0.1.27.whl
mypy_extensions-0.4.3-py2.py3-none-any.whl
mysql_connector_python-8.0.15-cp36-cp36m-macosx_10_13_x86_64.whl
pandas-0.25.3-cp36-cp36m-macosx_10_9_x86_64.whl
retrying-1.3.3-py3-none-any.whl
structlog-19.2.0-py2.py3-none-any.whl
typing-3.7.4.1-py3-none-any.whl


echo $(ls target) | tr ' ' ,

boto3-1.11.3-py2.py3-none-any.whl,engagment-states-batch-rds-loader-0.1.27.whl,mypy_extensions-0.4.3-py2.py3-none-any.whl,mysql_connector_python-8.0.15-cp36-cp36m-macosx_10_13_x86_64.whl,pandas-0.25.3-cp36-cp36m-macosx_10_9_x86_64.whl,retrying-1.3.3-py3-none-any.whl,structlog-19.2.0-py2.py3-none-any.whl,typing-3.7.4.1-py3-none-any.whl

回答7:

You could use sed -r

(
_space_="\ *";
type=".";
perm="[^\ ]*";
hlinks=$perm;
user=$perm;
group=$perm;
size="[0-9]*";
modified=".{12}";
name=".*";
ls -l /etc | sed -r s/"^($type)($perm)$_space_($hlinks)$_space_($user)$_space_($group)$_space_($size)$_space_($modified)$_space_($name)"/'"\1","\2","\3","\4","\5","\6","\7","\8"'/g
)