Creating a POST with url elisp package in emacs: u

Question

I'm currently creating a Rest client for making blog posts much in the spirit of pastie.el. The main objective is for me to write a textile in emacs and make a post to a Rails application that will create it. It is working fine until I type anything in either spanish or japanese, then I get a 500 error. pastie.el has this same problem also by the way.

Here is the code:

(require 'url)

(defun create-post()

(interactive)

(let ((url-request-method "POST")

    (url-request-extra-headers '(("Content-Type" . "application/xml")))

    (url-request-data (concat "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
                              "<post>"
                              "<title>"
                              "Not working with spanish nor japanese"
                              "</title>"
                              "<content>"
                              ;; "日本語"   ;; not working
                              ;; "ñ"        ;; not working either
                              "h1. Textile title\n\n"
                              "*Textile bold*"
                              "</content>"
                              "</post>"))
    )                               ; end of let varlist
(url-retrieve "http://127.0.0.1:3000/posts.xml"
              ;; CALLBACK
              (lambda (status)
                (switch-to-buffer (current-buffer)))
              )))

The only way I can imagine right now that the problem could be fixed is by making emacs encode the utf-8 characters so that a 'ñ' becomes '&#241' (which works by the way).

What could be a work around for this problem?

EDIT: '*' is not equivalent to *'. What I meant was that if I encoded to utf-8 with emacs using for example 'sgml-char' it would make the whole post become utf-8 encoded. Like *Textile bold* thus making RedCloth being unable to convert it into html. Sorry, it was very bad explained.

Answer 1

Thanks to @legoscia I know now that I have to encode the data by myself. I'll post the function here for future reference:

(require 'url)

(defun create-post()
(interactive)
(let ((url-request-method "POST")
    (url-request-extra-headers '(("Content-Type" . "application/xml; charset=utf-8")))
    (url-request-data
     (encode-coding-string (concat "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"                                  
                                   "<post>"
                                   "<title>"
                                   "Not working with spanish nor japanese"
                                   "</title>"
                                   "<content>"
                                   "日本語\n\n"   ;; working!!!
                                   "ñ\n\n"        ;; working !!!
                                   "h1. Textile title\n\n"
                                   "*Textile bold*"
                                   "</content>"
                                   "</post>") 'utf-8)
     )
    )                               ; end of let varlist
(url-retrieve "http://127.0.0.1:3000/posts.xml"
              ;; CALLBACK
              (lambda (status)
                (switch-to-buffer (current-buffer))
                ))))                 ;let

Answer 2

A guess: does it work if you set url-request-data to

(encode-coding-string (concat "<?xml etc...") 'utf-8)

instead?

There's nothing really to tell url what coding system you use, so I guess you have to encode your data yourself. This should also give a correct Content-length header, as that just comes from (length url-request-data), which would obviously give the wrong result for most UTF-8 strings.