JSONKit benchmarks

Question

I pulled the code from http://www.bonto.ch/blog/2011/12/08/json-libraries-for-ios-comparison-updated/ and tested it on my local machine.

I was surprised by the results and NSJSonSerialization gave much better performance than JSONKit.

Has NSJSonSerialization really overtaken JSONKit in terms of speed or did I do something wrong?

twitter_timeline.json

repeat.json

random.json

Answer 1

Interestingly, there are still people out there who care about JSON parser performance ;)

About three years ago, I developed my own JSON parser with the goal to make it fast and with a low memory foot-print. Since then, I was interested how it compares to others, especially NSJSONSerialization and JSONKit. I do have a number of bench marks which I run occasionally, to check how new compilers and OSs did improve the parsers. I also forked Bonto's bench, fixed a few errors, added a few pathological JSON files and added my own parser as well.

Well, I can confirm that during the last years, the performance of NSJSONSerialization has been improved greatly. It's now comparable to JSONKit for many work loads. If I remember right, a certain test took 690ms for NSJSONSerialization iOS 5.0, while it now performs at 520ms on iOS 6.1. JSONKit took about 524ms - and now with the same sources - it takes about 500ms, the latter solely due to better compilers.

Since I was curious, I gathered some results, compiled with Xcode (soon available) and a few screenshots taken from my forked Bonto's bench mark.

The test includes my own parser library (JPJson) in the test with name "JPJsonData" (the newest version which is not yet checked in into GitHub), in a fair and comparable test. I ensured, JSONKit has been compiled with all optimizations enabled, and disabled assertions, so that it is as fast as possible.

The test has been run on this hardware, an iPad 2 (the second variant), with these specs:

System Name: iPhone OS System Version: 6.1.3 Model: iPad Localized Model: iPad Platform: iPad2,2 Hardware Model: K94AP Platform Type: 14 Platform String: iPad 2G CPU Frequency: 475.113 MHz BUS Frequency: 475.113 MHz Total Memory: 502 MByte User Memory: 419 MByte

---

Some other test file, pretty much average JSON:

---

JSON with mostly numbers:

---

JSON with mostly random Unicode strings.

These JSON Strings do contain "Unicode noncharacters" and "Unicode NULL" characters. JSONKit does not allow those characters - thus is fails. JPJson won't allow them by default, but it has options which when set, allow those characters in JSON Strings:

---

Update

I ran the benchmark for an iPhone 5s on iOS 7, for both 32-bit and 64-bit.

JSONKit failed to compile for 64-bit.

System Name: iPhone OS
System Version: 7.0.4
Model: iPhone
Localized Model: iPhone
Platform: iPhone6,2
Hardware Model: N53AP
Platform Type: 18
Platform String: Unknown iPhone
CPU Frequency: 0 MHz
BUS Frequency: 0 MHz
Total Memory: 999 MByte
User Memory: 858 MByte

All timings in milli seconds.

-----------------------------------
twitter_timeline.json

| Library             |  32-bit |  64-bit |
|---------------------|--------:|--------:|
| JSONKitData         |   1.68  |       *)|
| JPJsonData          |   1.16  |    1.29 |
| NSJSONSerialization |   2.44  |    2.21 |
| SBJsonData          |   6.05  |    6.69 |



-----------------------------------
appache_builds.json

| Library             |  32-bit |  64-bit |
|---------------------|--------:|--------:|
| JSONKitData         |    2.80 |      *) |
| JPJsonData          |    3.21 |    3.52 |
| NSJSONSerialization |    3.23 |    4.17 |
| SBJsonData          |   10.41 |   11.75 |


-----------------------------------
mesh.json

| Library             |  32-bit |  64-bit |
|---------------------|--------:|--------:|
| JSONKitData         |   87.64 |      *) |
| JPJsonData          |   25.46 |   20.34 |
| NSJSONSerialization |  144.25 |   34.25 |
| SBJsonData          |  105.60 |  107.70 |



-----------------------------------
sample.json

| Library             |  32-bit |  64-bit |
|---------------------|--------:|--------:|
| JSONKitData         |  failed |      *) |
| JPJsonData          |    5.97 |    6.39 |
| NSJSONSerialization |   18.94 |   13.66 |
| SBJsonData          |  failed |   88.19 |


-----------------------------------
random.json

| Library             |  32-bit |  64-bit |
|---------------------|--------:|--------:|
| JSONKitData         |   23.01 |      *) |
| JPJsonData          |   17.69 |   20.62 |
| NSJSONSerialization |   28.83 |   28.60 |
| SBJsonData          |   82.93 |   82.74 |

*) Failed to compile

Answer 2

I run the tests on a iPhone 4 (ios7) and iPhone 4S (ios 6.1) and NSJSONSerialization is faster in both cases.

Answer 3

You have not done anything wrong.

Things have changed in iOS 6. Apple have greatly improved the performance of NSJSONSerialization. When using iOS 6 and up, it is the fastest.

JSONKit has also not been updated in a year. It is falling behind.