How to handle closing parenthesis in path names in

Question

I have a long path name to a program I must run in a for /f loop, which includes a closing parenthesis ")", and from which I need to parse the output:

for /f "tokens=1" %%G in ('"C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe" list') do (echo Will do something with %%G)

...where 'list' is a parameter passed to my program. I get the error "'C:\Documents' is not recognized as an internal or external command, operable program or batch file."

I do know the problem is that the closing parenthesis in fact closes the "for" block, so the ending double quotes is not "seen", so the long path name is not enclosed within double quotes anymore. What I don't understand is why is this happening, since my path is enclosed within double quotes? I also tried the usebackq option:

for /f "usebackq tokens=1" %%G in (`"C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe" list`) do (echo Will do something with %%G)

...with no better results. I tried to escape like this "^)" or like this "^^)", nothing to do. Tried doubling the double quotes:

for /f "tokens=1" %%G in ('""C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe"" list') do (echo Will do something with %%G)

Still does not work.

I'm, furthermore, in fact using a variable that holds the path, which is not know in advance (built from %CD%), and EnableDelayedExpansion is activated. I tried the delayed expansion (which did fixed similar problems in other situations) to prevent the variable's expansion at read time and delay it at execution time:

setlocal EnableDelayedExpansion
set _var=%CD%\program.exe
@REM _var now contains C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe
for /f "tokens=1" %%G in ('"!_var!" list') do (echo %%G)
endlocal

Still doesn't work, don't understand why.

But, doubling the double quotes with delayed expansion in above code:

for /f "tokens=1" %%G in ('""!_var!"" list') do (echo %%G)

does work!... why... why having to do this??? What effect does it have? I don't understand. I also fear it may cause a problem in some specific circumstances...

Any idea?

Answer 1

Comments in the answers to this question indicate XP gives different behavior then newer Windows versions.

There is a known FOR /F bug in XP: http://www.dostips.com/forum/viewtopic.php?p=9062#p9062. But this problem is not related to that bug.

The actual problem stems from how FOR /F executes a command in the IN() clause. It uses CMD \C command (See How does the Windows Command Interpreter (CMD.EXE) parse scripts?)

You can observe this behavior by adding this line to Aacini's PROG.BAT example.

echo cmdcmdline=%cmdcmdline%

The next issue deals with how CMD deals with quotes that appear in the /C command, and why XP behaves differently than more recent Windows versions.

This command fails in XP, but succeeds in Vista and beyond:

for /f "delims=" %a in ('"test (this)\prog" args') do @echo %a

The command that FOR tries to execute (%cmdcmdline%) is the same in both versions (disregarding differences in %COMSPEC%):

C:\Windows\system32\cmd.exe /c "test (this)\prog" args

XP has a CMD design flaw in how it deals with the quotes. The flaw is even documented (but it is not recognized as a flaw). Vista and beyond partially fix the design flaw, but don't bother to correct the documentation.

Here is an excerpt from HELP CMD

If /C or /K is specified, then the remainder of the command line after
the switch is processed as a command line, where the following logic is
used to process quote (") characters:

    1.  If all of the following conditions are met, then quote characters
        on the command line are preserved:

        - no /S switch
        - exactly two quote characters
        - no special characters between the two quote characters,
          where special is one of: &<>()@^|
        - there are one or more whitespace characters between the
          two quote characters
        - the string between the two quote characters is the name
          of an executable file.

    2.  Otherwise, old behavior is to see if the first character is
        a quote character and if so, strip the leading character and
        remove the last quote character on the command line, preserving
        any text after the last quote character.

We want CMD to follow rule 1 so that quotes are preserved, but ( and ) violate the special character constraint on XP, so rule 2 is followed and the CMD tries to execute

test (this)\prog args

It should be fairly obvious why this fails!

I can't think of any reason why the special character constraint exists in rule 1. It defeats the whole purpose of what MS is attempting to do.

Apparently the design flaw is partially fixed in Vista and beyond, but they haven't updated the HELP documentation. Vista ignores the special characters ( and ) and processes the command using rule 1, the quotes are preserved, and everything works.

Update 2015-05-17: Unfortunately, Vista and beyond still treat @, ^, and & as special characters, even though they are valid characters within file names. Of course <, >, and | are treated as special characters, but they are not valid in a file name anyway. So for Vista and beyond, the documentation for rule 1 should read where special is one of: &<>@^|.

I've traced the behavior that everyone has documented, and it is all consistent with the above.

There is a way to execute the command on XP without using a delayed expansion variable, and it is compatible with Vista and beyond.

for /f "delims=" %a in ('^""test (this)\prog" args^"') do @echo %a

The opening and closing quotes are escaped so that the ) does not interfere with the FOR parser. The command that is executed for the IN() clause is

C:\Windows\system32\cmd.exe /c ""test (this)\prog" args"

Both XP and Vista follow rule 2 because there is more than two quotes, so CMD executes

"test (this)\prog" args

and everything works!

The rest of this answer is outdated but preserved to give context to existing comments.

---

Your very 1st code example should work; it cannot (make that should not) give the error message you describe. The error message breaks off the path at the first space, which implies that the path was not quoted or escaped. But you are "sure" it was quoted.

The key to the problem is three pieces of information near the end of your post:

1. You are actually using a variable with delayed expansion

2. this doesn't work: for /f "tokens=1" %%G in ('"!_var!" list') do (echo %%G)

3. this works: for /f "tokens=1" %%G in ('""!_var!"" list') do (echo %%G)

If the value of var is already quoted, you will get the behavior you are describing.

The value of your var must be "C:\Documents and Settings\myaccount\Desktop\Test_release (x86)\program.exe", including the quotes.

To make this explanation more readable I will shorten the path to "test (this)\prog.exe"

"!var!" fails because it expands to ""test (this)\prog.exe"", which effectively unquotes the path. The string has three areas, two that are quoted, and one in the middle that is not:

"empty quoted area"unquoted path"empty quoted area"

""!var!"" works because it expands to """test (this)\prog.exe""" and the path is now quoted again. There are now five areas within the string:

"empty quoted area"empty unquoted area"quoted path"empty unquoted area"empty quoted area"

The simple answer as to how you should proceed:

If the value of var is already quoted, then simply use !var! Edit- that doesn't work on XP: "!var!" works on both

If the value of var is not quoted, then use "!var!" Edit- that doesn't work on XP: ""!var!"" works on both

Answer 2

I made some tests to find why the direct method you used at first didn't work. I first created a directory named test (this) and inside it I created a batch file named prog.bat:

>dir /b
test (this)
>dir /b "test (this)"
prog.bat
>type "test (this)\prog.bat"
@echo off
echo/One - Line one of prog
echo/Two - Line two of prog
echo/Three - Line three of prog

Then I first tried to access the contents of such file with for /f. Note that useback option is required because the file name include spaces and must be enclosed in quotes:

>for /f "usebackq tokens=1" %G in ("test (this)\prog.bat") do @echo %G
@echo
echo/One
echo/Two
echo/Three

Previous result prove that the pathname was correctly given. Now, to execute the Batch file instead of read it, just enclose its name in back quotes, right?

>for /f "usebackq tokens=1" %G in (`"test (this)\prog.bat"`) do @echo %G
'test' is not recognized as an internal or external command,
operable program or batch file.

Then, because the message said that test is the name of the command not found, I created a Batch file named test.bat:

>type test.bat
@echo off
echo Test.batFileLine1
echo Test.batFileLine2
echo Test.batFileLine2

>for /f "usebackq tokens=1" %G in (`"test (this)\prog.bat"`) do @echo %G
Test.batFileLine1
Test.batFileLine2
Test.batFileLine2

Aha! Previous result show that in this case the quotes are ignored and just test (this)\prog.bat is executed, right?

>for /f "usebackq tokens=1" %G in (`test (this)\prog.bat`) do @echo %G
\prog.bat`) was unexpected at this time.

Now what? Well, the problem NOW is related to the parentheses:

>for /f "usebackq tokens=1" %G in (`test ^(this^)\prog.bat`) do @echo %G
Test.batFileLine1
Test.batFileLine2
Test.batFileLine2

My conclusion is that the for /f command have an error when both back-quotes and quotes are used in combination with "usebackq" option and the filename have spaces, and that this bug is omitted if a delayed variable expansion is used.