How to escape single quotes within single quoted s

Question

Let's say, you have a bash alias like:

alias rxvt='urxvt'

which works fine.

However:

alias rxvt='urxvt -fg '#111111' -bg '#111111''

won't work, and neither will:

alias rxvt='urxvt -fg \'#111111\' -bg \'#111111\''

So how do you end up matching up opening and closing quotes inside a string once you have escaped quotes?

alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''

seems ungainly although it would represent the same string if you're allowed to concatenate them like that.

Answer 1

Both versions are working, either with concatenation by using the escaped single quote character (\'), or with concatenation by enclosing the single quote character within double quotes ("'").

The author of the question did not notice that there was an extra single quote (') at the end of his last escaping attempt:

alias rxvt='urxvt -fg'\''#111111'\'' -bg '\''#111111'\''
           │         │┊┊|       │┊┊│     │┊┊│       │┊┊│
           └─STRING──┘┊┊└─STRIN─┘┊┊└─STR─┘┊┊└─STRIN─┘┊┊│
                      ┊┊         ┊┊       ┊┊         ┊┊│
                      ┊┊         ┊┊       ┊┊         ┊┊│
                      └┴─────────┴┴───┰───┴┴─────────┴┘│
                          All escaped single quotes    │
                                                       │
                                                       ?

As you can see in the previous nice piece of ASCII/Unicode art, the last escaped single quote (\') is followed by an unnecessary single quote ('). Using a syntax-highlighter like the one present in Notepad++ can prove very helpful.

The same is true for another example like the following one:

alias rc='sed '"'"':a;N;$!ba;s/\n/, /g'"'"
alias rc='sed '\'':a;N;$!ba;s/\n/, /g'\'

These two beautiful instances of aliases show in a very intricate and obfuscated way how a file can be lined down. That is, from a file with a lot of lines you get only one line with commas and spaces between the contents of the previous lines. In order to make sense of the previous comment, the following is an example:

$ cat Little_Commas.TXT
201737194
201802699
201835214

$ rc Little_Commas.TXT
201737194, 201802699, 201835214

Answer 2

Here is an elaboration on The One True Answer referenced above:

Sometimes I will be downloading using rsync over ssh and have to escape a filename with a ' in it TWICE! (OMG!) Once for bash and once for ssh. The same principle of alternating quotation delimiters is at work here.

For example, let's say we want to get: Louis Theroux's LA Stories ...

1. First you enclose Louis Theroux in single quotes for bash and double quotes for ssh: '"Louis Theroux"'
2. Then you use single quotes to escape a double quote '"'
3. The use double quotes to escape the apostrophe "'"
4. Then repeat #2, using single quotes to escape a double quote '"'
5. Then enclose LA Stories in single quotes for bash and double quotes for ssh: '"LA Stories"'

And behold! You wind up with this:

rsync -ave ssh '"Louis Theroux"''"'"'"'"''"s LA Stories"'

which is an awful lot of work for one little ' -- but there you go

Answer 3

Most of these answers hit on the specific case you're asking about. There is a general approach that a friend and I have developed that allows for arbitrary quoting in case you need to quote bash commands through multiple layers of shell expansion, e.g., through ssh, su -c, bash -c, etc. There is one core primitive you need, here in native bash:

quote_args() {
    local sq="'"
    local dq='"'
    local space=""
    local arg
    for arg; do
        echo -n "$space'${arg//$sq/$sq$dq$sq$dq$sq}'"
        space=" "
    done
}

This does exactly what it says: it shell-quotes each argument individually (after bash expansion, of course):

$ quote_args foo bar
'foo' 'bar'
$ quote_args arg1 'arg2 arg2a' arg3
'arg1' 'arg2 arg2a' 'arg3'
$ quote_args dq'"'
'dq"'
$ quote_args dq'"' sq"'"
'dq"' 'sq'"'"''
$ quote_args "*"
'*'
$ quote_args /b*
'/bin' '/boot'

It does the obvious thing for one layer of expansion:

$ bash -c "$(quote_args echo a'"'b"'"c arg2)"
a"b'c arg2

(Note that the double quotes around $(quote_args ...) are necessary to make the result into a single argument to bash -c.) And it can be used more generally to quote properly through multiple layers of expansion:

$ bash -c "$(quote_args bash -c "$(quote_args echo a'"'b"'"c arg2)")"
a"b'c arg2

The above example:

1. shell-quotes each argument to the inner quote_args individually and then combines the resulting output into a single argument with the inner double quotes.
2. shell-quotes bash, -c, and the already once-quoted result from step 1, and then combines the result into a single argument with the outer double quotes.
3. sends that mess as the argument to the outer bash -c.

That's the idea in a nutshell. You can do some pretty complicated stuff with this, but you have to be careful about order of evaluation and about which substrings are quoted. For instance, the following do the wrong things (for some definition of "wrong"):

$ (cd /tmp; bash -c "$(quote_args cd /; pwd 1>&2)")
/tmp
$ (cd /tmp; bash -c "$(quote_args cd /; [ -e *sbin ] && echo success 1>&2 || echo failure 1>&2)")
failure

In the first example, bash immediately expands quote_args cd /; pwd 1>&2 into two separate commands, quote_args cd / and pwd 1>&2, so the CWD is still /tmp when the pwd command is executed. The second example illustrates a similar problem for globbing. Indeed, the same basic problem occurs with all bash expansions. The problem here is that a command substitution isn't a function call: it's literally evaluating one bash script and using its output as part of another bash script.

If you try to simply escape the shell operators, you'll fail because the resulting string passed to bash -c is just a sequence of individually-quoted strings that aren't then interpreted as operators, which is easy to see if you echo the string that would have been passed to bash:

$ (cd /tmp; echo "$(quote_args cd /\; pwd 1\>\&2)")
'cd' '/;' 'pwd' '1>&2'
$ (cd /tmp; echo "$(quote_args cd /\; \[ -e \*sbin \] \&\& echo success 1\>\&2 \|\| echo failure 1\>\&2)")
'cd' '/;' '[' '-e' '*sbin' ']' '&&' 'echo' 'success' '1>&2' '||' 'echo' 'failure' '1>&2'

The problem here is that you're over-quoting. What you need is for the operators to be unquoted as input to the enclosing bash -c, which means they need to be outside the $(quote_args ...) command substitution.

Consequently, what you need to do in the most general sense is to shell-quote each word of the command not intended to be expanded at the time of command substitution separately, and not apply any extra quoting to the shell operators:

$ (cd /tmp; echo "$(quote_args cd /); $(quote_args pwd) 1>&2")
'cd' '/'; 'pwd' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")
/
$ (cd /tmp; echo "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
'cd' '/'; [ -e *'sbin' ] && 'echo' 'success' 1>&2 || 'echo' 'failure' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
success

Once you've done this, the entire string is fair game for further quoting to arbitrary levels of evaluation:

$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")"
/
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")"
/
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")")"
/
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")"
success
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *sbin ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")"
success
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")")"
success

etc.

These examples may seem overwrought given that words like success, sbin, and pwd don't need to be shell-quoted, but the key point to remember when writing a script taking arbitrary input is that you want to quote everything you're not absolutely sure doesn't need quoting, because you never know when a user will throw in a Robert'; rm -rf /.

To better understand what is going on under the covers, you can play around with two small helper functions:

debug_args() {
    for (( I=1; $I <= $#; I++ )); do
        echo -n "$I:<${!I}> " 1>&2
    done
    echo 1>&2
}

debug_args_and_run() {
    debug_args "$@"
    "$@"
}

that will enumerate each argument to a command before executing it:

$ debug_args_and_run echo a'"'b"'"c arg2
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)"
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'> 
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''> 
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'> 
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'bash'"'"' '"'"'-c'"'"' '"'"''"'"'"'"'"'"'"'"'debug_args_and_run'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'echo'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'a"b'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'c'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'arg2'"'"'"'"'"'"'"'"''"'"''> 
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''> 
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'> 
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

Answer 4

I can confirm that using '\'' for a single quote inside a single-quoted string does work in Bash, and it can be explained in the same way as the "gluing" argument from earlier in the thread. Suppose we have a quoted string: 'A '\''B'\'' C' (all quotes here are single quotes). If it is passed to echo, it prints the following: A 'B' C. In each '\'' the first quote closes the current single-quoted string, the following \' glues a single quote to the previous string (\' is a way to specify a single quote without starting a quoted string), and the last quote opens another single-quoted string.

Answer 5

If you're generating the shell string within Python 2 or Python 3, the following may help to quote the arguments:

#!/usr/bin/env python

from __future__ import print_function

try:  # py3
    from shlex import quote as shlex_quote
except ImportError:  # py2
    from pipes import quote as shlex_quote

s = """foo ain't "bad" so there!"""

print(s)
print(" ".join([shlex_quote(t) for t in s.split()]))

This will output:

foo ain't "bad" so there!
foo 'ain'"'"'t' '"bad"' so 'there!'

Answer 6

Simple example of escaping quotes in shell:

$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc

It's done by finishing already opened one ('), placing escaped one (\'), then opening another one ('). This syntax works for all commands. It's very similar approach to the 1st answer.