Files number not counting properly

2019-08-18 00:26发布

问题:

I have a directory with 264420 files

$ find /Users/Username -type f | wc -l
    264420

I'm trying to count the file number recursively in Node.js with simple counter++

The issue is that counter++ gives incorrect number of files, while array.push(file) with array.length in the end working just fine and gives back correct amount of files - 264420

$ node total-files.js

114 'total_files_count' 
264420 'total_files_array_count'

The simplified script looks like the following:

const fs = require('fs')
const path = require('path')
const dir = '/Users/Username'
let totalFiles = 0
let totalFilesArray = []

const totalFilesObj = getFilesRecursively(dir, totalFiles, totalFilesArray)

console.log(totalFilesObj.total_files, 'total_files_count', totalFilesObj.total_files_array.length, 'total_files_array_count')


function getFilesRecursively(dir, totalFiles, totalFilesArray){

    totalFiles = totalFiles || 0
    totalFilesArray = totalFilesArray || []  

    try { var files = fs.readdirSync(dir) }
    catch(err) { console.error(err.message)  }

    for (var i in files) {

        var name = path.join(dir, files[i])

        try {  

            let stats = fs.lstatSync(name)  // track symlink as link, but not as file

            if (stats.isDirectory()) 
                getFilesRecursively(name, totalFiles, totalFilesArray) 

            else if (stats.isFile()) {
                totalFiles++   // why this not working? so we should use .push to array instead?
                totalFilesArray.push(name) 
            }

        } catch(err) { console.error(err.message) }     
    }

    return { "total_files" : totalFiles, "total_files_array" : totalFilesArray }
}

I'm planning to run the script on large directories with hundreds of thousands of files and want to use counter incremental which is faster than pushing to array.

Kindly help.

回答1:

Two reasons

  • You are not adding the return value of recursion to the final value.
  • And incrementing a primitive value doesn't change the value back in the caller function's scope.

Make it

if (stats.isDirectory()) 
{
     var returnVal  = getFilesRecursively(name, totalFiles, totalFilesArray);
     totalFiles +=  returnVal.total_files;
}

Also, set the totalFiles value to 0 in the beginning of the method.



回答2:

As @gurvinder372 kindly suggested in his answer

And incrementing a primitive value doesn't change the value back in the caller function's scope.

So one of the possible solutions without changing underlying code logic and not using .push is to create single value array and increment that value:

totalFiles++                  ->  totalFiles[0]++
let totalFiles = 0            ->  let totalFiles = [0]
totalFiles = totalFiles || 0  ->  totalFiles = totalFiles || [0]