How to urlencode data for curl command?

Question

I am trying to write a bash script for testing that takes a parameter and sends it through curl to web site. I need to url encode the value to make sure that special characters are processed properly. What is the best way to do this?

Here is my basic script so far:

#!/bin/bash
host=${1:?'bad host'}
value=$2
shift
shift
curl -v -d "param=${value}" http://${host}/somepath $@

Answer 1

Use Perl's URI::Escape module and uri_escape function in the second line of your bash script:

...

value="$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$2")"
...

Edit: Fix quoting problems, as suggested by Chris Johnsen in the comments. Thanks!

Answer 2

This may be the best one:

after=$(echo -e "$before" | od -An -tx1 | tr ' ' % | xargs printf "%s")

Answer 3

Simple PHP option:

echo 'part-that-needs-encoding' | php -R 'echo urlencode($argn);'

Answer 4

One of variants, may be ugly, but simple:

urlencode() {
    local data
    if [[ $# != 1 ]]; then
        echo "Usage: $0 string-to-urlencode"
        return 1
    fi
    data="$(curl -s -o /dev/null -w %{url_effective} --get --data-urlencode "$1" "")"
    if [[ $? != 3 ]]; then
        echo "Unexpected error" 1>&2
        return 2
    fi
    echo "${data##/?}"
    return 0
}

Here is the one-liner version for example (as suggested by Bruno):

date | curl -Gso /dev/null -w %{url_effective} --data-urlencode @- "" | cut -c 3-

Answer 5

Here is a POSIX function to do that:

encodeURIComponent() {
  awk 'BEGIN {while (y++ < 125) z[sprintf("%c", y)] = y
  while (y = substr(ARGV[1], ++j, 1))
  q = y ~ /[[:alnum:]_.!~*\47()-]/ ? q y : q sprintf("%%%02X", z[y])
  print q}' "$1"
}

Example:

value=$(encodeURIComponent "$2")

Source

Answer 6

I find it more readable in python:

encoded_value=$(python -c "import urllib; print urllib.quote('''$value''')")

the triple ' ensures that single quotes in value won't hurt. urllib is in the standard library. It work for exampple for this crazy (real world) url:

"http://www.rai.it/dl/audio/" "1264165523944Ho servito il re d'Inghilterra - Puntata 7