parsing and getting list from response of get requ

Question

I'm trying to parse a website with the requests module:

import requests
some_data = {'a':'',
             'b':''}
with requests.Session() as s:
    result = s.post('http://website.com',data=some_data)
    print(result.text)

The page is responding as below:

{
    "arrangetype":"U",
    "list": [
        {
            "product_no":43,
            "display_order":4,
            "is_selling":"T",
            "product_empty":"F",
            "fix_position":null,
            "is_auto_sort":false
        },
        {
            "product_no":44,
            "display_order":6,
            "is_selling":"T",
            "product_empty":"F",
            "fix_position":null,
            "is_auto_sort":false
        }
    ],
    "length":2
}

I found that instead of parsing full HTML, it would be better to deal with the response as all the data I want is in that response.

What I want to get is a list of the values of product_no, so the expected result is:

[43,44]

How do I do this?

Answer 1

Convert your JSON response to a dictionary with json.loads(), and collect your results in a list comprehension.

Demo:

from json import loads

data = """{
    "arrangetype":"U",
    "list": [
        {
            "product_no":43,
            "display_order":4,
            "is_selling":"T",
            "product_empty":"F",
            "fix_position":null,
            "is_auto_sort":false
        },
        {
            "product_no":44,
            "display_order":6,
            "is_selling":"T",
            "product_empty":"F",
            "fix_position":null,
            "is_auto_sort":false
        }
    ],
    "length":2 
}"""

json_dict = loads(data)

print([x['product_no'] for x in json_dict['list']])
# [43, 44]

Full Code:

import requests
from json import loads

some_data = {'a':'',
             'b':''}

with requests.Session() as s:
    result = s.post('http://website.com',data=some_data)
    json_dict = loads(result.text)
    print([x["product_no"] for x in json_dict["list"]])