Script that will transfer photos from phone camera

Question

Story

I take photos and record videos with my phone camera and keep all of them on my sdcard. I periodically back them up on my PC, so I keep these camera photos on PC storage in sync with phone storage.

For years, I've been backing up my phone camera photos to my PC in the following way:

1. Plug in phone into PC and allow access to phone data
2. Browse SDCard → DCIM → Camera
3. Wait several minutes for the system to load a list of ALL photos
4. Copy only several latest photos which haven't been backed up yet

I figured that waiting several minutes for all photos to load is an unnecessary drag so I downloaded adb platform tools. I've added the folder bin to my Path environment variable (i.e. %USERPROFILE%\Tools\adb-platform-tools_r28.0.3) so that I can seamlessly use adb and not write its full path each time.

The script

I wrote the following script for Git Bash for Windows. It is also compatible with Unix if you change the $userprofile variable. Essentially, the script pulls camera photos between two dates from phone storage to PC.

# Attach device and start deamon process
adb devices

# Initialize needed variables
userprofile=$(echo "$USERPROFILE" | tr "\\" "/") # Windows adjustments

srcFolder="//storage/06CB-C9CE/DCIM/Camera"    # Remote folder
dstFolder="$userprofile/Desktop/CameraPhotos"  # Local folder
lsFile="$dstFolder/camera-ls.txt"
filenameRegex="2019061[5-9]_.*"  # Date from 20190615 to 20190619

# Create dst folder if it doesn't exist
mkdir -p "$dstFolder"

# 1. List contents from src folder
# 2. Filter out file names matching regex
# 3. Write these file names line by line into a ls file
adb shell ls "$srcFolder" | grep -E "$filenameRegex" > "$lsFile"

# Pull files listed in ls file from src to dst folder
while read filename; do
  if [ -z "$filename" ]; then continue; fi
  adb pull "$srcFolder/$filename" "$dstFolder" # adb: error: ...
done < "$lsFile"

# Clean up
rm "$lsFile"

# Inform the user
echo "Done pulling files to $dstFolder"

The problem

When I run the script (bash adb-pull-camera-photos.sh), everything runs smoothly except for the adb pull command in the while-loop. It gives the following error:

': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg

I am not sure why the output is broken. Sometimes when I resize the Git Bash window some of the text goes haywire. This is the actual error text:

adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg': No such file or directory
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg': No such file or directory
adb: error: failed to stat remote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg': No such file or directory

I am sure that these files exist in the specified directory on the phone. When I manually execute the failing command in bash, it succeeds with the following output:

$ adb pull "//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg" "C:/Users/User/Desktop/CameraPhotos/"
//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg: 1 file pulled. 15.4 MB/s (1854453 bytes in 0.115s)

The question

I can't figure out what's wrong with the script. I thought the Windows system might be causing a commotion, because I don't see the reason why the same code works when entered manually, but doesn't work when run in a script. How do I fix this error?

Additional info

• Note that I had to use // in the beginning of an absolute path on Windows because Git Bash would interpret / as its own root directory (C:\Program Files\Git).
• I've echoed all variables inside the script and got all the correct paths that otherwise work via manual method.

camera-ls.txt file contents

20190618_124656.jpg
20190618_204522.jpg
20190619_225739.jpg

Additional questions

1. Is it possible to navigate to external sdcard without using its name? I had to use /storage/06CB-C9CE/ because /sdcard/ navigates to internal storage.
2. Why does tr "\\" "/" give me this error: tr: warning: an unescaped backslash at end of string is not portable?

Answer 1

The problem was with Windows line delimiters.

Easy fix

Just add the IFS=$'\r\n' above the loop so that the read command knows the actual line delimiter.

IFS=$'\r\n'
while read filename; do
  if [ -z "$filename" ]; then continue; fi
  adb pull "$srcFolder/$filename" "$dstFolder"
done < "$lsFile"

Explanation

I tried plugging the whole while-loop into the console and it failed with the same error:

$ bash adb-pull-camera-photos.sh
List of devices attached
9889db343047534336      device

tr: warning: an unescaped backslash at end of string is not portable
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_124656.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190618_204522.jpg
': No such file or directoryemote object '//storage/06CB-C9CE/DCIM/Camera/20190619_225739.jpg
Done pulling files to C:/Users/User/Desktop/CameraPhotos

This time I started investigating why the output was broken. I remembered that windows uses \r\n as newline, which means Carriage Return + Line Feed, (CR+LF), so some text must have been overwritten.

It was because of broken values stored inside the $filename variable.

This is the loop from the script:

while read filename; do
  if [ -z "$filename" ]; then continue; fi
  adb pull "$srcFolder/$filename" "$dstFolder"
done < "$lsFile"

Since each iteration of the while-loop reads a line from $lsFile in the following form:

exampleFilename.jpg\r\n

It misinterprets the newline symbols as part of the file name, so adb pull tries to read files with these whitespaces in their names, but fails and it additionally writes a broken output.

Answer 2

Windows batch script

Here's a .bat script that can be run by Windows Command Prompt or Windows PowerShell. No Git Bash required.

:: Start deamon of the device attached
adb devices

:: Pull camera files starting from date
set srcFolder=/storage/06CB-C9CE/DCIM/Camera
set dstFolder=%USERPROFILE%\Desktop\CameraPhotos
set lsFile=%USERPROFILE%\Desktop\CameraPhotos\camera-ls.txt
set dateRegex=2019061[5-9]_.*

mkdir %dstFolder%
adb shell ls %srcFolder% | adb shell grep %dateRegex% > %lsFile%
for /F "tokens=*" %%A in (%lsFile%) do adb pull %srcFolder%/%%A %dstFolder%
del %lsFile%
echo Done pulling files to %dstFolder%

• Just edit the srcFolder to point to your phone camera folder,
• plug a pattern into the dateRegex for matching the date interval and
• save it as a file with .bat extension, i.e: adb-pull-camera-photos.bat.
• Double-click the file and it will pull filtered photos into CameraPhotos folder on Desktop.

Keep in mind that you still need have adb for Windows on your PC.