How would I go about replacing all of the double quotes in my batch file's parameters with escaped double quotes? This is my current batch file, which expands all of its command line parameters inside the string:
@echo off
call bash --verbose -c "g++-linux-4.1 %*"
It then uses that string to make a call to Cygwin's bash, executing a Linux cross-compiler. Unfortunately, I'm getting parameters like these passed in to my batch file:
"launch-linux-g++.bat" -ftemplate-depth-128 -O3 -finline-functions
-Wno-inline -Wall -DNDEBUG -c
-o "C:\Users\Me\Documents\Testing\SparseLib\bin\Win32\LinuxRelease\hello.o"
"c:\Users\Me\Documents\Testing\SparseLib\SparseLib\hello.cpp"
Where the first quote around the first path passed in is prematurely ending the string being passed to GCC, and passing the rest of the parameters directly to bash (which fails spectacularly.)
I imagine if I can concatenate the parameters into a single string then escape the quotes it should work fine, but I'm having difficulty determining how to do this. Does anyone know?
As an addition to mklement0's excellent answer:
Almost all executables accept
\"
as an escaped"
. Safe usage in cmd however is almost only possible using DELAYEDEXPANSION.To explicitely send a literal
"
to some process, assign\"
to an environment variable, and then use that variable, whenever you need to pass a quote. Example:Note
SETLOCAL ENABLEDELAYEDEXPANSION
seems to work only within batch files. To get DELAYEDEXPANSION in an interactive session, startcmd /V:ON
.If your batchfile does't work with DELAYEDEXPANSION, you can enable it temporarily:
If you want to pass dynamic content from a variable that contains quotes that are escaped as
""
you can replace""
with\"
on expansion:This replacement is not safe with
%...%
style expansion!In case of OP
bash -c "g++-linux-4.1 !v_params:"=\"!"
is the safe version.If for some reason even temporarily enabling DELAYEDEXPANSION is not an option, read on:
Using
\"
from within cmd is a little bit safer if one always needs to escape special characters, instead of just sometimes. (It's less likely to forget a caret, if it's consistent...)To achieve this, one precedes any quote with a caret (
^"
), quotes that should reach the child process as literals must additionally be escaped with a backlash (\^"
). ALL shell meta characters must be escaped with^
as well, e.g.&
=>^&
;|
=>^|
;>
=>^>
; etc.Example:
Source: Everyone quotes command line arguments the wrong way, see "A better method of quoting"
To pass dynamic content, one needs to ensure the following:
The part of the command that contains the variable must be considered "quoted" by
cmd.exe
(This is impossible if the variable can contain quotes - don't write%var:""=\"%
). To achieve this, the last"
before the variable and the first"
after the variable are not^
-escaped. cmd-metacharacters between those two"
must not be escaped. Example:This isn't safe, if
%dynamic_content%
can contain unmatched quotes.eplawless's own answer simply and effectively solves his specific problem: it replaces all
"
instances in the entire argument list with\"
, which is how Bash requires double-quotes inside a double-quoted string to be represented.To generally answer the question of how to escape double-quotes inside a double-quoted string using
cmd.exe
, the Windows command-line interpreter (whether on the command line - often still mistakenly called the "DOS prompt" - or in a batch file):See bottom for a look at PowerShell.tl;dr:
You must use
""
when passing a string to a(nother) batch file and you may use""
with applications created with Microsoft's C/C++/.NET compilers (which also accept\"
), which on Windows includes Python and Node.js:Example:
foo.bat "We had 3"" of rain."
The following applies to batch files only:
""
is the only way to get the command interpreter (cmd.exe
) to treat the whole double-quoted string as a single argument.Sadly, however, not only are the enclosing double-quotes retained (as usual), but so are the doubled escaped ones, so obtaining the intended string is a two-step process; e.g., assuming that the double-quoted string is passed as the 1st argument,
%1
:set "str=%~1"
removes the enclosing double-quotes;set "str=%str:""="%"
then converts the doubled double-quotes to single ones.Be sure to use the enclosing double-quotes around the assignment parts to prevent unwanted interpretation of the values.
\"
is required - as the only option - by many other programs, (e.g., Ruby, Perl, and even Microsoft's own PowerShell(!)), but ITS USE IS NOT SAFE:\"
is what many executables and interpreters either require - including Microsoft's own PowerShell when passed strings from the outside - or, in the case of Microsoft's compilers, support as an alternative to""
- ultimately, though, it's up to the target program to parse the argument list.foo.exe "We had 3\" of rain."
\"
CAN RESULT IN UNWANTED, ARBITRARY EXECUTION OF COMMANDS and/or INPUT/OUTPUT REDIRECTIONS:& | < >
ver
command; see further below for an explanation and the next bullet point for a workaround:foo.exe "3\" of snow" "& ver."
\""
is a robust alternative.If you must use
\"
, there are only 3 safe approaches, which are, however quite cumbersome: Tip of the hat to T S for his help.Using (possibly selective) delayed variable expansion in your batch file, you can store literal
\"
in a variable and reference that variable inside a"..."
string using!var!
syntax - see T S's helpful answer.Only with LITERAL strings - ones NOT involving VARIABLES - do you get a similarly methodical approach: categorically
^
-escape allcmd.exe
metacharacters:" & | < >
and - if you also want to suppress variable expansion -%
:foo.exe ^"3\^" of snow^" ^"^& ver.^"
Otherwise, you must formulate your string based on recognizing which portions of the string
cmd.exe
considers unquoted due to misinterpreting\"
as closing delimiters:in literal portions containing shell metacharacters:
^
-escape them; using the example above, it is&
that must be^
-escaped:foo.exe "3\" of snow" "^& ver."
in portions with
%...%
-style variable references: ensure thatcmd.exe
considers them part of a"..."
string and that that the variable values do not themselves have embedded, unbalanced quotes - which is not even always possible.For background information, read on.
Background
Note: This is based on my own experiments. Do let me know if I'm wrong.
POSIX-like shells such as Bash on Unix-like systems tokenize the argument list (string) before passing arguments individually to the target program: among other expansions, they split the argument list into individual words (word splitting) and remove quoting characters from the resulting words (quote removal). What the target program is handed is conceptually an array of individual arguments with (syntax-required) quotes removed.
By contrast, the Windows command interpreter apparently does not tokenize the argument list and simply passes the single string comprising all arguments - including quoting chars. - to the target program.
However, some preprocessing takes place before the single string is passed to the target program:
^
escape chars. outside of double-quoted strings are removed (they escape the following char.), and variable references (e.g.,%USERNAME%
) are interpolated first.Thus, unlike in Unix, it is the target program's responsibility to parse to parse the arguments string and break it down into individual arguments with quotes removed. Thus, different programs can hypothetically require differing escaping methods and there's no single escaping mechanism that is guaranteed to work with all programs - https://stackoverflow.com/a/4094897/45375 contains excellent background on the anarchy that is Windows command-line parsing.
In practice,
\"
is very common, but NOT SAFE, as mentioned above:Since
cmd.exe
itself doesn't recognize\"
as an escaped double-quote, it can misconstrue later tokens on the command line as unquoted and potentially interpret them as commands and/or input/output redirections.In a nutshell: the problem surfaces, if any of the following characters follow an opening or unbalanced
\"
:& | < >
; for example:cmd.exe
sees the following tokens, resulting from misinterpreting\"
as a regular double-quote:"3\"
of
snow" "
& ver.
Since
cmd.exe
thinks that& ver.
is unquoted, it interprets it as&
(the command-sequencing operator), followed by the name of a command to execute (ver.
- the.
is ignored;ver
reportscmd.exe
's version information).The overall effect is:
foo.exe
is invoked with the first 3 tokens only.ver
is executed.Even in cases where the accidental command does no harm, your overall command won't work as designed, given that not all arguments are passed to it.
Many compilers / interpreters recognize ONLY
\"
- e.g., the GNU C/C++ compiler, Python, Perl, Ruby, even Microsoft's own PowerShell when invoked fromcmd.exe
- and, except for PowerShell with\""
, for them there is no simple solution to this problem.Essentially, you'd have to know in advance which portions of your command line are misinterpreted as unquoted, and selectively
^
-escape all instances of& | < >
in those portions.By contrast, use of
""
is SAFE, but is regrettably only supported by Microsoft-compiler-based executables and batch files (in the case of batch files, with the quirks discussed above).By contrast, PowerShell, when invoked from the outside - e.g., from
cmd.exe
, whether from the command line or a batch file - recognizes only\"
and, on Windows, the more robust\""
, even though internally PowerShell uses`
as the escape character in double-quoted strings and also accepts""
; e.g.:powershell -c " \"ab c\".length"
works (outputs4
), as does the more robustpowershell -c " \""ab c\"".length"
,but
powershell -c " ""ab c"".length"
breaks.Related information
^
can only be used as the escape character in unquoted strings - inside double-quoted strings,^
is not special and treated as a literal.^
in parameters passed to thecall
statement is broken (this applies to both uses ofcall
: invoking another batch file or binary, and calling a subroutine in the same batch file):^
instances in double-quoted values are inexplicably doubled, altering the value being passed: e.g., if variable%v%
contains literal valuea^b
,call :foo "%v%"
assigns"a^^b"
(!) to%1
(the first parameter) in subroutine:foo
.^
withcall
is broken altogether in that^
can no longer be used to escape special characters: e.g.,call foo.cmd a^&b
quietly breaks (instead of passing literala&b
toofoo.cmd
, as would be the case withoutcall
) -foo.cmd
is never even invoked(!), at least on Windows 7.Escaping a literal
%
is a special case, unfortunately, which requires distinct syntax depending on whether a string is specified on the command line vs. inside a batch file; see https://stackoverflow.com/a/31420292/45375%%
. On the command line,%
cannot be escaped, but if you place a^
at the start, end, or inside a variable name in an unquoted string (e.g.,echo %^foo%
), you can prevent variable expansion (interpolation);%
instances on the command line that are not part of a variable reference are treated as literals (e.g,100%
).Generally, to safely work with variable values that may contain spaces and special characters:
set "v=a & b"
assigns literal valuea & b
to variable%v%
(by contrast,set v="a & b"
would make the double-quotes part of the value). Escape literal%
instances as%%
(works only in batch files - see above).echo "%v%"
does not subject the value of%v%
to interpolation and prints"a & b"
(but note that the double-quotes are invariably printed too). By contrast,echo %v%
passes literala
toecho
, interprets&
as the command-sequencing operator, and therefore tries to execute a command namedb
.Also note the above caveat re use of
^
with thecall
statement.%~1
to remove enclosing double-quotes from the 1st parameter) and, sadly, there is no direct way that I know of to getecho
to print a variable value faithfully without the enclosing double-quotes.for
-based workaround that works as long as the value has no embedded double quotes; e.g.:set "var=^&')|;,%!" for /f "delims=" %%v in ("%var%") do echo %%~v
cmd.exe
does not recognize single-quotes as string delimiters - they are treated as literals and cannot generally be used to delimit strings with embedded whitespace; also, it follows that the tokens abutting the single-quotes and any tokens in between are treated as unquoted bycmd.exe
and interpreted accordingly.Even if supported by the target program, however, it is not advisable to use single-quoted strings, given that their contents are not protected from potentially unwanted interpretation by
cmd.exe
.PowerShell
Windows PowerShell is a much more advanced shell than
cmd.exe
, and it has been a part of Windows for many years now (and PowerShell Core brought the PowerShell experience to macOS and Linux as well).PowerShell works consistently internally with respect to quoting:
`"
or""
to escape double-quotes''
to escape single-quotesThis works on the PowerShell command line and when passing parameters to PowerShell scripts or functions from within PowerShell.
(As discussed above, passing an escaped double-quote to PowerShell from the outside requires
\"
or, more robustly,\""
- nothing else works).Sadly, when invoking external programs, you're faced with the need to both accommodate PowerShell's own quoting rules and to escape for the target program:
This problematic behavior is also discussed and summarized in this GitHub docs issue
Double-quotes inside double-quoted strings:
Consider string
"3`" of rain"
, which PowerShell-internally translates to literal3" of rain
.If you want to pass this string to an external program, you have to apply the target program's escaping in addition to PowerShell's; say you want to pass the string to a C program, which expects embedded double-quotes to be escaped as
\"
:Note how both
`"
- to make PowerShell happy - and the\
- to make the target program happy - must be present.The same logic applies to invoking a batch file, where
""
must be used:By contrast, embedding single-quotes in a double-quoted string requires no escaping at all.
Single-quotes inside single-quoted strings do not require extra escaping; consider
'2'' of snow'
, which is PowerShell' representation of2' of snow
.PowerShell translates single-quoted strings to double-quoted ones before passing them to the target program.
However, double-quotes inside single-quoted strings, which do not need escaping for PowerShell, do still need to be escaped for the target program:
PowerShell v3 introduced the magic
--%
option, called the stop-parsing symbol, which alleviates some of the pain, by passing anything after it uninterpreted to the target program, save forcmd.exe
-style environment-variable references (e.g.,%USERNAME%
), which are expanded; e.g.:Note how escaping the embedded
"
as\"
for the target program only (and not also for PowerShell as\`"
) is sufficient.However, this approach:
%
characters in order to avoid environment-variable expansions.Invoke-Expression
in a second.Thus, despite its many advancements, PowerShell has not made escaping much easier when calling external programs. It has, however, introduced support for single-quoted strings.
I wonder if it's fundamentally possible in the Windows world to ever switch to the Unix model of letting the shell do all the tokenization and quote removal predictably, up front, irrespective of the target program, and then invoke the target program by passing the resulting tokens.
For example for Unreal engine Automation tool run from batch file - this worked for me
eg: -cmdline=" -Messaging" -device=device -addcmdline="-SessionId=session -SessionOwner='owner' -SessionName='Build' -dataProviderMode=local -LogCmds='LogCommodity OFF' -execcmds='automation list; runtests tests+separated+by+T1+T2; quit' " -run
Hope this helps someone, worked for me.
Google eventually came up with the answer. The syntax for string replacement in batch is this:
Which produces "replicate me". My script now looks like this:
Which replaces all instances of
"
with\"
, properly escaped for bash.The escape character in batch scripts is
^
. But for double-quoted strings, double up the quotes: