how to replace a variable in shell script string

Question

I'm having problems getting this to work...

I have a variable that is holding a SQL to with a placeholder:

echo $SQL
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('$BATCH_END')

I have another variable that holds the value:

echo $BATCH_END
2012-11-14 17:06:13

I want to replace the placeholder with the value. I'm not particularly great at Unix scripting, but I've tried this:

echo $SQL | sed -e "s/'$BATCH_END/$BATCH_END/g"

but it still doesn't get replaced...

Can anyone help? I want to replace the placeholder, and keep the final string assigned to $SQL

I also need to know how to get the value of the output back into the variable, for example, I tried:

 SQL=`echo "$SQL" | echo "${SQL//\$BATCH_END/$BATCH_END}"`

Answer 1

You are missing the end of that single-quote pair in your script.

Change from:

echo $SQL | sed -e "s/'$BATCH_END/$BATCH_END/g"

To:

echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g"

Updated - as per followup comment:

To save the result of the above replacement back into $SQL, do either of the following:

# Preferred way
SQL=$(echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g")

# Old way
SQL=`echo $SQL | sed -e "s/\$BATCH_END/$BATCH_END/g"`

This is called command substitution. Either syntax ($(...) vs. enclosure by backticks) works, but the preferred one allows you to do nesting.

The preferred-preferred way: Herestring

This is probably a bit more advanced than what you care about, but doing it in the following way will save you a subprocess from having to use echo unnecessarily:

SQL=$(sed -e "s/\$BATCH_END/$BATCH_END/g" <<< $SQL)

Answer 2

In my terminal:

$ SQL="SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('\$BATCH_END')"
$ # (observe: I escaped the $ sign to have the same variable as you)
$ echo "$SQL"
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('$BATCH_END')
$ BATCH_END="2012-11-14 17:06:13"
$ echo "$BATCH_END"
2012-11-14 17:06:13
$ # Now the replacement:
$ echo "${SQL//\$BATCH_END/$BATCH_END}"
SELECT PX_PROMOTION_ID, PRIORITY, STATUS, EXCLSVE, TYPE, PERORDLMT, PERSHOPPERLMT, TOTALLMT, RSV_INT, PX_GROUP_ID, CAMPAIGN_ID, STOREENT_ID, VERSION, REVISION, EFFECTIVE, TRANSFER, CDREQUIRED, EXPIRE, LASTUPDATEBY, TO_CHAR(LASTUPDATE, 'YYYYMMDD HH24MMSS') AS LASTUPDATE, TO_CHAR(STARTDATE, 'YYYYMMDD HH24MMSS') AS STARTDATE, TO_CHAR(ENDDATE, 'YYYYMMDD HH24MMSS') AS ENDDATE, TO_CHAR(RSV_TIME, 'YYYYMMDD HH24MMSS') AS RSV_TIME, RSV_REAL, TGTSALES, NAME, CODE, RSV_VCH, OPTCOUNTER FROM PX_PROMOTION WHERE LASTUPDATE BETWEEN (SELECT MAX(BATCHSTART) FROM XRPTEBATCHCONTROL) AND TIMESTAMP('2012-11-14 17:06:13')

Done!

Answer 3

The problem is that you are using a double-quoted string in the shell. In a double quoted string, variables like $BATCH_END get interpreted as shell variables and interpolated. The ' character has no special meaning within a double quoted string; it doesn't prevent variables from being interpolated. So your $BATCH_END string is being substituted in both places; your sed invocation is equivalent to:

sed -e "s/'2012-11-14 17:06:13/2012-11-14 17:06:13/"

Which as you can see, is not very helpful (you also have a stray ' in there). You need to escape the $ sign, to prevent it from being interpreted as a shell variable:

sed -e "s/\$BATCH_END/$BATCH_END/"

Answer 4

You need to quote the first $ so that it does not get expanded as a shell variable.

echo "$SQL" | sed -e "s/'\$BATCH_END'/'$BATCH_END'/g"

… Or choose an easier placeholder, something like @BATCH_END@ for instance.

To assign the result back to $SQL you will need some more shell escaping:

SQL=`echo "$SQL" | sed -e "s/'\\\$BATCH_END'/'$BATCH_END'/g"`

Answer 5

One way to do it is with 'differential quoting' in a single argument:

echo "$SQL" | sed -e 's/$BATCH_END/'"$BATCH_END/g"

The first part of the -e option is in single quotes, so the shell does not expand the first $BATCH_END and it can match the word in the SQL statement. The second part is in double quotes, so the shell expands the second $BATCH_END and places its text into the SQL.

If you needed to worry about the single quotes around $BATCH_END, you'd have to play other tricks; probably a backslash would be simplest (and it's a viable option anyway):

echo "$SQL" | sed -e "s/'\$BATCH_END'/'$BATCH_END'/g"

The backslash stops the shell expanding the first $BATCH_END but the absence of a backslash means the second is expanded. Inside the double quotes, the single quotes lose their 'no expansion' property.