PowerShell stripping double quotes from command li

Question

Recently I have been having some trouble using GnuWin32 from PowerShell whenever double quotes are involved.

Upon further investigation, it appears PowerShell is stripping double quotes from command line arguments, even when properly escaped.

PS C:\Documents and Settings\Nick> echo '"hello"'
"hello"
PS C:\Documents and Settings\Nick> echo.exe '"hello"'
hello
PS C:\Documents and Settings\Nick> echo.exe '\"hello\"'
"hello"

Notice that the double quotes are there when passed to PowerShell's echo cmdlet, but when passed as an argument to echo.exe, the double quotes are stripped unless escaped with a backslash (even though PowerShell's escape character is a backtick, not a backslash).

This seems like a bug to me. If I am passing the correct escaped strings to PowerShell, then PowerShell should take care of whatever escaping may be necessary for however it invokes the command.

What is going on here?

For now, the fix is to escape command line arguments in accordance with these rules (which seem to be used by the CreateProcess API call which PowerShell uses to invoke .exe files):

• To pass a double quote, escape with a backslash: \" -> "
• To pass a one or more backslashes followed by a double quote, escape each backslash with another backslash and escape the quote: \\\\\" -> \\"
• If not followed by a double quote, no escaping is necessary for backslashes: \\ -> \\

Note that further escaping of double quotes may be necessary to escape the double quotes in the Windows API escaped string to PowerShell.

Here are some examples, with echo.exe from GnuWin32:

PS C:\Documents and Settings\Nick> echo.exe "\`""
"
PS C:\Documents and Settings\Nick> echo.exe "\\\\\`""
\\"
PS C:\Documents and Settings\Nick> echo.exe "\\"
\\

I imagine that this can quickly become hell if you need to pass a complicated command line parameter. Of course, none of this documented in the CreateProcess() or PowerShell documentation.

Also note that this is not necessary to pass arguments with double quotes to .NET functions or PowerShell cmdlets. For that, you need only escape your double quotes to PowerShell.

Answer 1

It is a known thing:

It's FAR TOO HARD to pass parameters to applications which require quoted strings. I asked this question in IRC with a "roomful" of PowerShell experts, and it took hour for someone to figure out a way (I originally started to post here that it is simply not possible). This completely breaks PowerShell's ability to serve as a general purpose shell, because we can't do simple things like executing sqlcmd. The number one job of a command shell should be running command-line applications... As an example, trying to use SqlCmd from SQL Server 2008, there is a -v parameter which takes a series of name:value parameters. If the value has spaces in it, you must quote it...

...there is no single way to write a command line to invoke this application correctly, so even after you master all 4 or 5 different ways of quoting and escaping things, you're still guessing as to which will work when ... or, you can just shell out to cmd, and be done with it.

Answer 2

This seems to be fixed in recent versions of PowerShell at the time of this writing, so it is no longer something to worry about.

If you still think you see this issue, remember it may be related to something else, such as the program that invokes PowerShell, so if you cannot reproduce it when invoking PowerShell directly from a command prompt or the ISE, you should debug elsewhere.

For example, I found this question when investigating a problem of disappearing quotes when running a PowerShell scrip from C# code using Process.Start. The issue was actually C# Process Start needs Arguments with double quotes - they disappear

Answer 3

I personally avoid using '\' to escape things in PowerShell, because it's not technically a shell escape character. I've gotten unpredictable results with it. In double-quoted strings, you can use "" to get an embedded double-quote, or escape it with a back-tick:

PS C:\Users\Droj> "string ""with`" quotes"
string "with" quotes

The same goes for single quotes:

PS C:\Users\Droj> 'string ''with'' quotes'
string 'with' quotes

The weird thing about sending parameters to external programs is that there is additional level of quote evaluation. I don't know if this is a bug, but I'm guessing it won't be changed, because the behavior is the same when you use Start-Process and pass in arguments. Start-Process takes an array for the arguments, which makes things a bit clearer, in terms of how many arguments are actually being sent, but those arguments seem to be evaluated an extra time.

So, if I have an array, I can set the arg values to have embedded quotes:

PS C:\cygwin\home\Droj> $aa = 'arg="foo"', 'arg=""""bar""""'
PS C:\cygwin\home\Droj> echo $aa
arg="foo"
arg=""""bar""""

The 'bar' argument has enough to cover the extra hidden evaluation. It's as if I send that value to a cmdlet in double-quotes, then send that result again in double-quotes:

PS C:\cygwin\home\Droj> echo "arg=""""bar""""" # level one
arg=""bar""
PS C:\cygwin\home\Droj> echo "arg=""bar""" # hidden level 
arg="bar"

One would expect these arguments to be passed to external commands as-is, as they are to cmdlets like 'echo'/'write-output', but they are not, because of that hidden level:

PS C:\cygwin\home\Droj> $aa = 'arg="foo"', 'arg=""""bar""""'
PS C:\cygwin\home\Droj> start c:\cygwin\bin\echo $aa -nonew -wait
arg=foo arg="bar"

I don't know the exact reason for it, but the behavior is as if there is another, undocumented step being taken under the covers that re-parses the strings. For example, I get the same result if I send the array to a cmdlet, but add a parsing level by doing it through invoke-expression:

PS C:\cygwin\home\Droj> $aa = 'arg="foo"', 'arg=""""bar""""'
PS C:\cygwin\home\Droj> iex "echo $aa"
arg=foo
arg="bar"

...which is exactly what I get when I send these arguments to my external cygwin 'echo.exe':

PS C:\cygwin\home\Droj> c:\cygwin\bin\echo 'arg="foo"' 'arg=""""bar""""'
arg=foo arg="bar"

Answer 4

Relying on the CMD to shell out the issue as indicated in accepted answer didn't work for me as double quotes were still stripped out when calling the CMD executable.

The good solution for me was to structure my command line as an array of strings instead of a single full string containing all the arguments. Then simply pass that array as the arguments for the binary invocation:

$args = New-Object System.Collections.ArrayList
$args.Add("-U") | Out-Null
$args.Add($cred.UserName) | Out-Null
$args.Add("-P") | Out-Null
$args.Add("""$($cred.Password)""")
$args.Add("-i") | Out-Null
$args.Add("""$SqlScriptPath""") | Out-Null
& SQLCMD $args

In that case, double quotes surrounding arguments are properly passed to the invoked command.

If you need, you can test and debug it with EchoArgs from the PowerShell Community Extensions