I have been using Linked List
implemented ArrayList
for storing my data in Java
. It is quite big and i am partitioning it into several regions using subList
and sending it into different processor creating threads for processing.I have 80 cores in my system and I am creating 20 threads now. Each thread has to process at least one million of instances from millions of instances. Data is partitioned as:
List<List<Integer>> totalInstances = new ArrayList<List<Integer>>();
//fill the data in totalInstances
List<List<Integer>> instanceSet1 = totalInstances.subList(index1,index2);
List<List<Integer>> instanceSet2 = totalInstances.subList(index2,index3);
....................
and so on. I have created such 20 sets.The processing is almost unsynchronized. There are some synchronization but that seems to appear in rare case.It is only the reading operation that takes place in this data set.I am not joining the treads for reading each of the example from the sublist. Threads are only synchronized to to enter into next joined after reading all the instances in their corresponding list of instances.Even then I am getting some pause during processing.It happens when each of the thread reads one instance , they wait for sometime and read another thread.What can be the reason for it?
- is it due to using linked list ?
- I am printing the instances that every thread access.I have heard that
I/O
operation hinders the multi-threading process. - Any data structure suitable for it?
- Any debugging tools for finding the performance hinderance?
Thanks in advance