I have a simple Java
app that needs to traverse a large JSON
array (with around 20K items), and within each array, I parse a sub-array. Each item looks like this:
{"index":0,"f1":[2,16,16,16,16,16,32,16],"f2":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"startTime":0.0}
I am using JSONPath
to iterate over each item. What I do is first I read the length, and simply iterate over the whole array. But it is very slow (like, 1 item per second).
int length = JsonPath.read(response, "$.result.length()");
for (int i = 0; i < length; i++) {
double start_time = JsonPath.read(response, "$.result["+i+"].startTime");
ArrayList<Integer> f1= JsonPath.read(response, "$.result["+i+"].f1");
//...other things
}
Is there a way to optimise it?
Got it. Thanks to Erwin, I can parse the whole JSON at once into a
HASHMap
simply like this:And then we can simply call
get(i)
to access a specific item in the loop:You should minimise number of
read
operation. First time you scan the whole file and and next you scann
times file partially. Reading from disk is slower than from memory: Latency Numbers Every Programmer Should Know, so you should load the file to memory once and then iterate over items. Also, fromJsonPath
documentation:You can improve your
JsonPath
:$.result
and read only what you need by:$.result..['f1','startTime']
.Example app which loads only required fields: