JavaScript data grid for millions of rows [closed]

Question

I need to present a large number of rows of data (ie. millions of rows) to the user in a grid using JavaScript.

The user shouldn't see pages or view only finite amounts of data at a time.

Rather, it should appear that all of the data are available.

Instead of downloading the data all at once, small chunks are downloaded as the user comes to them (ie. by scrolling through the grid).

The rows will not be edited through this front end, so read-only grids are acceptable.

What data grids, written in JavaScript, exist for this kind of seamless paging?

Answer 1

I don't mean to start a flame war, but assuming your researchers are human, you don't know them as well as you think. Just because they have petabytes of data doesn't make them capable of viewing even millions of records in any meaningful way. They might say they want to see millions of records, but that's just silly. Have your smartest researchers do some basic math: Assume they spend 1 second viewing each record. At that rate, it will take 1000000 seconds, which works out to more than six weeks (of 40 hour work-weeks with no breaks for food or lavatory).

Do they (or you) seriously think one person (the one looking at the grid) can muster that kind of concentration? Are they really getting much done in that 1 second, or are they (more likely) filtering out the stuff the don't want? I suspect that after viewing a "reasonably-sized" subset, they could describe a filter to you that would automatically filter out those records.

As paxdiablo and Sleeper Smith and Lasse V Karlsen also implied, you (and they) have not thought through the requirements. On the up side, now that you've found SlickGrid, I'm sure the need for those filters became immediately obvious.

Answer 2

I suggest sigma grid, sigma grid has embed paging features which could support millions of rows. And also, you may need a remote paging to do it. see the demo http://www.sigmawidgets.com/products/sigma_grid2/demos/example_master_details.html

Answer 3

I kind of fail to see the point, for jqGrid you can use the virtual scrolling functionality:

http://www.trirand.net/aspnetmvc/grid/performancevirtualscrolling

but then again, millions of rows with filtering can be done:

http://www.trirand.net/aspnetmvc/grid/performancelinq

I really fail to see the point of "as if there are no pages" though, I mean... there is no way to display 1,000,000 rows at once in the browser - this is 10MB of HTML raw, I kind of fail to see why users would not want to see the pages.

Anyway...

Answer 4

dojox.grid.DataGrid offers a JS abstraction for data so you can hook it up to various backends with provided dojo.data stores or write your own. You'll obviously need one that supports random access for this many records. DataGrid also provides full accessibility.

Edit so here's a link to Matthew Russell's article that should provide the example you need, viewing millions of records with dojox.grid. Note that it uses the old version of the grid, but the concepts are the same, there were just some incompatible API improvements.

Oh, and it's totally free open source.

Answer 5

I know it's an old question but still.. There is also dhtmlxGrid that can handle millions of rows. There is a demo with 50,000 rows but the number of rows that can be loaded/processed in grid is unlimited.

Disclaimer: I'm from DHTMLX team.

Answer 6

Disclaimer: i heavily use YUI DataTable without no headache for a long time. It is powerful and stable. For your needs, you can use a ScrollingDataTable wich suports

• x-scrolling
• y-scrolling
• xy-scrolling
• A powerful Event mechanism

For what you need, i think you want is a tableScrollEvent. Its API says

Fired when a fixed scrolling DataTable has a scroll.

As each DataTable uses a DataSource, you can monitoring its data through tableScrollEvent along with render loop size in order to populate your ScrollingDataTable according to your needs.

Render loop size says

In cases where your DataTable needs to display the entirety of a very large set of data, the renderLoopSize config can help manage browser DOM rendering so that the UI thread does not get locked up on very large tables. Any value greater than 0 will cause the DOM rendering to be executed in setTimeout() chains that render the specified number of rows in each loop. The ideal value should be determined per implementation since there are no hard and fast rules, only general guidelines:

• By default renderLoopSize is 0, so all rows are rendered in a single loop. A renderLoopSize > 0 adds overhead so use thoughtfully.
• If your set of data is large enough (number of rows X number of Columns X formatting complexity) that users experience latency in the visual rendering and/or it causes the script to hang, consider setting a renderLoopSize.
• A renderLoopSize under 50 probably isn't worth it. A renderLoopSize > 100 is probably better.
• A data set is probably not considered large enough unless it has hundreds and hundreds of rows.
• Having a renderLoopSize > 0 and < total rows does cause the table to be rendered in one loop (same as renderLoopSize = 0) but it also triggers functionality such as post-render row striping to be handled from a separate setTimeout thread.

For instance

// Render 100 rows per loop
 var dt = new YAHOO.widget.DataTable(<WHICH_DIV_WILL_STORE_YOUR_DATATABLE>, <HOW YOUR_TABLE_IS STRUCTURED>, <WHERE_DOES_THE_DATA_COME_FROM>, {
     renderLoopSize:100
 });

<WHERE_DOES_THE_DATA_COME_FROM> is just a single DataSource. It can be a JSON, JSFunction, XML and even a single HTML element

Here you can see a Simple tutorial, provided by me. Be aware no other DATA_TABLE pluglin supports single and dual click at the same time. YUI DataTable allows you. And more, you can use it even with JQuery without no headache

Some examples, you can see

• List item

Feel free to question about anything else you want about YUI DataTable.

regards,