How to parse an infinite json array from stdin in

Question

I'm trying to write a small replacement for i3status, a small programm that comunicates with i3bar conforming this protocol. They exchange messeages via stdin and stdout.

The stream in both directions is an infinite array of json objects. The start of the stream from i3bar to i3status (which i want to replace) looks like this:

[
{"name": "some_name_1","instance": "some_inst_1","button": 1,"x": 213,"y": 35}
,{"name": "some_name_1","instance": "some_inst_2","button": 2,"x": 687,"y": 354}
,{"name": "some_name_2","instance": "some_inst","button": 1,"x": 786,"y": 637}
,{"name": "some_name_3","instance": "some_inst","button": 3,"x": 768,"y": 67}
...

This is an "array" of objects which represent clicks. The array will never close.

My question is now: What is the right way of parsing this?

Obviously I cannot use the json library because this is not a vaild json object.

Answer 1

Write a custom reader function (or Decoder) which does a "streaming array parse" like so:

1. Read and discard leading whitespace.
2. If the next character is not a [ then return an error (can't be an array).
3. While true do:
   1. Call json.Decoder.Decode into the "next" item.
   2. Yield or process the "next" item.
   3. Read and discard whitespace.
   4. If the next character is:
      1. A comma , then continue the for-loop in #3.
      2. A close bracket ] then exit the for-loop in #3.
      3. Otherwise return an error (invalid array syntax).

Answer 2

What you are looking for is a Streaming API for JSON. There are many available a quick Google search revealed this project that does list Streaming as one of it's advance features.

Answer 3

I'm writing my own handler for click events in i3 as well. That's how I stumbled upon this thread.

The Golang standard library does actually do exactly what is required (Golang 1.12). Not sure it did when you asked the question or not?

// json parser read from stdin
decoder := json.NewDecoder(os.Stdin)

// Get he first token - it should be a '['
tk, err := decoder.Token()
if err != nil {
    fmt.Fprintln(os.Stderr, "couldn't read from stdin")
    return
}
// according to the docs a json.Delim is one [, ], { or }
// Make sure the token is a delim
delim, ok := tk.(json.Delim)
if !ok {
    fmt.Fprintln(os.Stderr, "first token not a delim")
    return
}
// Make sure the value of the delim is '['
if delim != json.Delim('[') {
    fmt.Fprintln(os.Stderr, "first token not a [")
    return
}

// The parser took the [ above
// therefore decoder.More() will return
// true until a closing ] is seen - i.e never 
for decoder.More() {
    // make a Click struct with the relevant json structure tags
    click := &Click{}

    // This will block until we have a complete JSON object
    // on stdin
    err = decoder.Decode(click)
    if err != nil {
            fmt.Fprintln(os.Stderr, "couldn't decode click")
            return
    }
    // Do something with click event
}