Recursively scan directory and sub-directories thr

Question

I'm trying create a list of all files (and their sizes) in a directory including everything within the sub-directories.

The files are on a remote server. So my script connects through FTP, then runs a recursive function using ftp_chdir to go through each directory.

If there's another way to do this, I'm open to suggestions.

$flist = array();

function recursive_list_dir($conn_id, $dir, $parent = "false") {
 global $flist;
 ftp_chdir($conn_id, $dir) or die("Fudgeballs: ".$parent."/".$dir);
 $list = array();
 $list = ftp_rawlist($conn_id, ".");

 if($parent != "false") { $dir = $parent."/".$dir; }

 for($x = 0; $x < count($list); $x++) {
  $list_details = preg_split("/[\s]+/", $list[$x]);
  $file = $list_details[3];
  $size = $list_details[2];

  if(!strstr($file, ".")) { // if there's no dot (.), then we assume it's a directory (is there a command similar to "is_dir" for FTP? that would be more fail proof?)
   recursive_list_dir($conn_id, $file, $dir);
  }
  else { $flist[] = $dir."@".$file."@".$size; }
 }
 ftp_chdir($conn_id, "..");
}

recursive_list_dir($conn_id, ".");

The script worked fine up to a point, but now it's not working. The PHP returns an error with ftp_chdir. The only thing that changed is that we've added more files to the server. The script works if I run it on a sub-directory. But if I run it on "." it fails. So is this failing because there are too many files and sub-directories?

Answer 1

I haven't tested this out, but here's how I did it a while back:

    $hostname = 'write.your.server.here';
    $username = 'username';
    $password = 'password';
    $startdir = 'starting/directory'; // absolute path
    $suffix   = "gif,png,jpeg,pdf,php"; // suffixes to list
    $files = array();
    $conn_id = ftp_connect($hostname);
    $login = ftp_login($conn_id, $username, $password);
    if (!$conn_id) {
        echo 'Wrong server!';
        exit;
    } else if (!$login) {
        echo 'Wrong username/password!';
        exit;
    } else {

        $files = raw_list("$startdir");
    }

    ftp_close($conn_id);

    function raw_list($folder) {
        global $conn_id;
        global $suffix;
        global $files;
        $suffixes = explode(",", $suffix);
        $list     = ftp_rawlist($conn_id, $folder);
        $anzlist  = count($list);
        $i = 0;
        while ($i < $anzlist) {
            $split    = preg_split("/[\s]+/", $list[$i], 9, PREG_SPLIT_NO_EMPTY);
            $itemname = $split[8];
            $endung   = strtolower(substr(strrchr($itemname ,"."),1));
            $path     = "$folder/$itemname";
            if (substr($list[$i],0,1) === "d" AND substr($itemname,0,1) != ".") {
                raw_list($path);
            } else if(substr($itemname,0,2) != "._" AND in_array($endung,$suffixes)) {
                array_push($files, $path);
            }
            $i++;
        }
        return $files;
    }

Answer 2

Usage of globals in PHP is not good practice. See this:

function ftp_get_files_list( $conn_id, $baseDir='.' ) {
    $files = array();
    $dirs = array($baseDir);
    while( $dir = array_shift($dirs) ) {
        $list = ftp_rawlist( $conn_id, $dir);
        while( $line = array_shift($list) ) {
            $col = preg_split( "@\s+@", $line );
            if (count($col) <= 2) continue;
            $fname = implode(' ',array_slice($col,8)); // support filenames with spaces
            $isDir =($col[0][0]=='d');
            if ($isDir)
                array_push($dirs, $dir.'/'.$fname );
            else
                array_push($files, $dir.'/'.$fname );
        }
    }
    return $files;
}

Answer 3

The fact that it was working before you gave it more inputs seems to me to suggest that it might be a problem there. Try putting a set_time_limit(300); at the top which will allow it to run for 5 minutes before timing out and see if that fixes the problem.

Answer 4

A real recursive solution that does not use global variables:

function ftp_list_files_recursive($ftp_stream, $path)
{
    $lines = ftp_rawlist($ftp_stream, $path);

    $result = array();

    foreach ($lines as $line)
    {
        $tokens = explode(" ", $line);
        $name = $tokens[count($tokens) - 1];
        $type = $tokens[0][0];

        $filepath = $path . "/" . $name;
        if ($type == 'd')
        {
            $result = array_merge($result, ftp_list_files_recursive($ftp_stream, $filepath));
        }
        else
        {
            $result[] = $filepath;
        }
    }
    return $result;
}

Works for FTP servers that use a common *nix-style listing like:

-r--r--r-- 1 ftp ftp             13 Nov 09  2015 file.txt
dr-xr-xr-x 1 ftp ftp              0 Nov 10  2015 folder

Won't work for files with a space in its name.