Decoding JSON with python using Appengine

Question

I have the following code which retrieves values from a simple 3 input form:

//retrieves data from a form
var $form = $( this ),
    prgname= $form.find('input[name="prg"]').val(),
    startDate = $("#startdate").datepicker({ dateFormat: 'yy-mm-dd' }).val(),
    endDate = $("#enddate").datepicker({ dateFormat: 'yy-mm-dd' }).val();

The following code sends the request to the server:

var request = $.ajax({
      url: "/prg/",
      type: "post",
  data: JSON.stringify({prg:  prgname, start:startDate, end:endDate}),
  contentType: 'application/json',
  dataType: 'json',
      success: function() {},
      error: function (jqXHR, textStatus, errorThrown){};

on the server end using python and webapp2 Im doing the following, (here is where I am unsure of things)

import json

class PrgHandler(webapp2.RequestHandler):
    def post(self):
        prg= cgi.escape(self.request.POST['prg'])
        start_date = cgi.escape(self.request.POST['start'])
        end_date = cgi.escape(self.request.POST['end'])

        #some code to write to db here
        ....
        #if successful return a success message
        if success:
            success_msg = [{'class': 'success', 'msg': 'Successfully saved to the database'}]
        else:
            success_msg = [{'class': 'error', 'msg': 'Could not saved to the database'}]

        data_string = json.dumps(success_msg)
        self.response.headers.add_header('content-type', 'application/json', charset='utf-8')   
        self.response.write(data_string)

When I get the response it is skipping the success function and going directly to the error.

Logging the error values im not getting any thing meaningful:

the error is:
The text status is:error
The jqXHR is:[object Object] 

Chrome's console is giving me the error:

Resource interpreted as Document but transferred with MIME type application/json:

I looked that up and the solutions on SO did not work, I think this is an error with the server side code:

self.response.headers.add_header('content-type', 'application/json', charset='utf-8')

If I comment out the above line I get no error in chrome and I just get back the response on a blank page with the correct values in the following format:

[{"msg": "Successfully saved to the database", "class": "success"}]

In the above case it does save to the database so I cannot seem to find anything wrong except for the header and simply don't know how to proceed!

EDIT The error it seems is from the server side I had removed the following line: event.preventDefault();

from my script and it caused all the problems now at least Im getting a clear indication of where the problem is. It's from incorrectly getting the posted data, how would I do it the correct way? I tried the following:

json_data = self.request.GET.items()
decoded = json.loads(json_data)

but Im getting a TypeError: expected string or buffer on the following line: json_data = self.request.GET.items()

Answer 1

Have a look in your debugger. You receive a JSON string in your post (webapp2 multidict). You have to decode this string using json.loads, resulting in a python object.

Here is my jquery code to send and receive json :

function gaeQuery(request) {
    var url = "/query";
    var payload = {'jsondata' : JSON.stringify(request)};
    $.post(
    url, 
    payload, 
    function(response) {
        procesResponse(response);
    },  // succes response callback 
    'json',  // response contains JSON content, and will be decoded in a js object
    {
        contentType: "application/json;charset=utf-8", // send JSON content
        timeout: 20000,
        tryCount: 0,
        retryLimit: 3, // max 3 retries             
        error: function(xhr, textStatus, errorThrown) { // error handling callback
            if (textStatus === 'timeout') {
                this.tryCount++;
                if (this.tryCount <= this.retryLimit) { //try again until retryLimit
                    $.ajax(this);
                    return;
                }
                alert('We have tried ' + this.retryLimit + ' times and it is still not working. We give in. Sorry.');
                return;
            }
            if (xhr.status === 500) { // internal server error
                alert('Oops! There seems to be a server problem, please try again later.');
            } 
            else {
                alert('Oops! There was a problem, sorry.'); // something went wrong
            }
        }
    }
    );
}

Answer 2

OK so I managed to figure this out and thought I will post the answer that worked for me to help anyone looking for this information because the webapp2 docs are not that helpful when it comes to 'getting' posted json data.

on the client side I did the following:

var request = $.ajax({
    url: "/some/url/",
    type: "POST",
    data: JSON.stringify([{someval: val1, someval2:val2, someval3:val3}]),
    contentType: "application/json",
    dataType: 'json',
    beforeSend: function() {
        $('#loading-div').show();
    },
    complete: function(){
        $('#loading-div').hide();
    },
    success: function(response, textStatus, jqXHR){}
});

The reason I couldnt figure out the problem straight away was because of the following line which I deleted along with some commented out line which prevented the page from redirecting after posting. This was the source of all the weird, unrelated and unhelpful error messages:

event.preventDefault();

on the server side to get the json data being posted to appengine do the following:

jdata = json.loads(cgi.escape(self.request.body))
    for vals in jdata:
        val1 = vals['someval']
        val2 = vals['someval2']
        val3 = vals['someval3']

The above was the root of the problem I wasn't doing it right and without the previous line on the client side there was no way to figure it out.

Anyways once you have the data do whatever processing you need to do with it and once you are done and need to send back a json response add the following lines:

//the data would look something like this 
data = {'return_value': val1, 'return_value2': val2,
        'return_value3': val3, 'return_value4': val4}

        return_data = json.dumps(data)
        self.response.headers.add_header('content-type', 'application/json', charset='utf-8')
        self.response.write(return_data)

Almost forgot on the client side again to access the variables sent back from the server with jquery its very straight forward...do something like:

 success: function(response, textStatus, jqXHR){
        console.log(response.return_value);
        console.log(response.return_value2);
        console.log(response.return_value3);
 }

Hope this will help someone seeking this information.