Python 3 Paho-MQTT Published/Subscribed JSON messa

Question

Rookie here.

I have a simple python code that's supposed to subscribe to a topic and publish JSON payload to the same topic using MQTT protocol. But for some reason, I am unable to load the payload as JSON!

What am I doing wrong here?

# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt
import json
mqtt_broker      = '192.168.1.111'
mqtt_topic_one   = 'mqtt_topic/tops_one'
mqtt_topic_two   = 'mqtt_topic/tops_two'

json_data_1      = '''{
    "this_json": "info",
    "data": {
        "multi_keyval": {
            "1": "1",
            "5": "5",
            "15": "15"
        },
        "single_keyval": {
            "single_key": "200"
        }
    }
}'''


def pass_to_func_and_pub(data_to_pub):
    print(data_to_pub)                # --------> This PRINTS
    print(json.loads(data_to_pub))    # --------> This DOES NOT PRINT

    # The following two lines don't work either.
    unpacked_json = json.loads(data_to_pub)
    client.publish(mqtt_topic_two, unpacked_json['this_json'])


def on_connect(client, userdata, flags, rc):
    client.subscribe(mqtt_topic_one)
    client.publish(mqtt_topic_one, json_data_1)


def on_message(client, userdata, msg):
    pass_to_func_and_pub(str(msg.payload))

client = mqtt.Client()

client.on_connect = on_connect
client.on_message = on_message

client.connect(mqtt_broker)
try:
    client.loop_forever()
except KeyboardInterrupt:
    client.disconnect()
    print('MQTT client disconnected, exiting now.')

Answer 1

There are a couple of problems here.

1. Exception handling

You're not handling exceptions (and Paho effectively ignores them within handlers, to keep the client alive I guess). This means when the exception is thrown in json.loads(data_to_pub), you're never seeing this but the function is exited as there is no local except block.

Improved version

def pass_to_func_and_pub(data_to_pub):
    print("Raw data: ", data_to_pub)
    try:
        unpacked_json = json.loads(data_to_pub)
    except Exception as e:
        print("Couldn't parse raw data: %s" % data_to_pub, e)
    else:
        print("JSON:", unpacked_json)
        client.publish(mqtt_topic_two, unpacked_json['this_json'])

Hang on, what exception?

Running this improved version we can now see:

Couldn't parse raw data: b'{\n "this_json": "info",\n "data": {\n "multi_keyval": {\n "1": "1",\n "5": "5",\n "15": "15"\n },\n "single_keyval": {\n "single_key": "200"\n }\n }\n}' Expecting value: line 1 column 1 (char 0)

Hmmm, what's that b' doing there?...

2. The encoding problem

Essentially your problem comes down to one line

def on_message(client, userdata, msg):
    pass_to_func_and_pub(str(msg.payload))

By calling str on the payload of that MqttMessage, which is a bytes object in Python 3, you'll get the stringified version of those bytes, e.g. b'foobar'.

This b, of course, makes it invalid JSON now, hence Expecting value: line 1 column 1 (char 0)...

Fixed version

Don't call str! Json.loads can handle bytes too. So:

def on_message(client, userdata, msg):
    pass_to_func_and_pub(msg.payload)

Or, assuming utf-8 encoding, we can do this more explicitly (I prefer working in strings):

def on_message(client, userdata, msg):
    pass_to_func_and_pub(msg.payload.decode('utf-8'))

Hope that helps!