Curl with multiline of JSON

Question

Consider the curl command below, is it possible to allow newline in JSON (without the minify) and execute directly in bash (Mac/Ubuntu)

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d \
'
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}'

When I run the command above, seems error occurred at the second { How to fix the above command?

Updated: actually I was able to run the command without issue previously, not sure why problem happen recently.

Answer 1

I remembered another way to do this with a "Here Document" as described in the Bash man page and detailed here. The @- means to read the body from STDIN, while << EOF means to pipe the script content until "EOF" as STDIN to curl. This layout may be easier to read than using separate files or the "echo a variable" approach.

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: application/json; charset=utf-8' \
--data-binary @- << EOF
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}
EOF

NOTE: Use the --trace <outfile> curl option to record exactly what goes over the wire. For some reason, this Here Document approach strips newlines. (Update: Newlines were stripped by curl -d option. Corrected!)

Answer 2

You should use outer double quotes, and the escape all inner quotes like this:

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d \
"
{
    \"field1\": \"test\",
    \"field2\": {
        \"foo\": \"bar\"
    }
}"

Answer 3

Along the lines of Martin's suggestion of putting the JSON in a variable, you could also put the JSON in a separate file, and then supply the filename to -d using curl's @ syntax:

curl -0 -v -X POST http://www.example.com/api/users \
  -H "Expect:" \
  -H 'Content-Type: text/json; charset=utf-8' \
  -d @myfile.json

The disadvantage is obvious (2 or more files where you used to have one.) But on the plus side, your script could accept a filename or directory argument and you'd never need to edit it, just run it on different JSON files. Whether that's useful depends on what you are trying to accomplish.

Answer 4

For some reason, this Here Document approach strips newlines

@eric-bolinger the reason the Heredoc strips newlines is because you need to tell your Heredoc to preserve newlines by quoting the EOF:

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d @- <<'EOF'

{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}
EOF

Notice the single-ticks surrounding EOF the first time it's defined, but not the second.

Answer 5

You could assign your json to a var:

json='
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}'

Now you can forward this to curl using stdin:

echo $json | curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d @-