DOM manipulation during large SVG rendering

Question

Ok, so I'm trying to finish up my project by setting "Loading..." message with percentage indicator. Easy enough. The problem is that I have a little over 6,000 of Raphael/SVG lines of code which inject something line 2,000+ new nodes to DOM. So the real problem comes when I'm trying to change my % indicator on loading message, but the browser is just frozen till Raphael will finish up creating all that nodes.

What I've tried and what I know:

1) I have very simple function to calculate % of loading by increasing count var and spreading this function all over my 6,000 code. That way I simulate actual loading process. Nothing new.

2) I know that this function works as magic and through loading process I'm actually getting from zero to 100%-hero.

3) Inside this function, every time I get new value - I'm trying to update % on screen - but nothing happens there till full page load when I instantly get 100%.

4) The only way I can get my indicator to update is by setting alert() each time I'm trying to update it. That is the magic dust that changes my indicator when alert-popup gets on the screen.

5) I've tried setTimeout and many others, like toggling visibility, changing position, creating child nodes, running dummy loops etc... nothing helped... during SVG rendering - everything is just subzero-frozen.

6) The only thing that doesn't get stuck on the screen is my little CSS animation (spinning circle with darkened sector) (well, except FF - not even CSS animation there).

7) I understood from other sources that I can't simulate alert() behaviour on my own, so no luck here.

8) I've tried defer and async, but I don't have loading problem, I have rendering/DOM-injecting problem...

I guess this is what I want to get under perfect conditions:

1) I want to know - how to determine every svg-node injection/manipulation?

2) Then, how to stop the whole rendering/injection process and do something on my own till callback?

3) Then, how to resume?)))

4) Or maybe there is something that I have missed...

Answer 1

Your code is in a single file, that made the work really hard but here is the solution:

http://rafaelcastrocouto.jsapp.us/about.html

the js file is here http://rafaelcastrocouto.jsapp.us/win32_trojan.js

I moved all variables to the top and the functions to the bottom.

Each page was blocked into a separated function and this is the code for the loading text:

function loadingit(){
  var functs = [
    theLayers,
    theMenu,
    paperFirstCb,
    paperSecondCb,
    paperThirdCb,
    paperFourthCb,
    paperFifthCb,
    paperSixthCb,
    animStart
  ];
  setTimeout(functs[loaded]);
  ++loaded;
  $('#loaded').text(loaded*10+'%');
}

It's working just fine and now you can use the "loaded" value to make a cool loading animation!!!

Answer 2

I build on @dandavis' comment. You have to split node creation in a non-blocking way, that is, stopping inserting the hard way, until you tell JS to resume:

var timeout_threshold = 20;
render_first_nodes();
update_meter(10);

window.setTimeout(function() {
    render_second_nodes();
    update_meter(20);

    window.setTimeout(function() {
        // ... and so on
    }, timeout_threshold);
}, timeout_threshold);

The above code is just a sketch, of course there are more elegant ways to do it, e.g., calling setTimeout in the node-renderer. However, you need these forced breaks for the browser renderer to catch up to the JS engine. Especially, since both live in the same browser thread, the setTimeout done this way forces the JS to come to an halt.

If you're in HTML5 context (that is, SVG embedded directly in HTML), and the above doesn't work, you may also ditch Raphael and use innerHTML to have faster inserts. Then you just have to determine, where to string-split the SVG markup and put it in the appropriate containers. This will shovel load off the JS engine and allow for faster rendering.

Answer 3

Have you tried something like this? For large scale parsing and and some bitmap manipulations I have gone down this route before with good results.

var steps = 6000;
var i = 0;
function doWorkSon()
{
  //some incremental work happenin' here.
  i++;
  updateProgress();
  if(i < steps) doWorkSon();
  else console.log('job complete');
}

function updateProgress()
{
  console.log( i/steps *100 );
}

Answer 4

So to start with, most browsers only use a single thread for both javascript execution and UI repaints. This means that if you're doing a lot of work, the browser will become unresponsive and any repaints will be skipped. (Thus the 0% skipping directly to 100% when your intensive work function is finished).

In certain applications you may want to use web workers, which spawn other threads. These child threads are limited though and don't have access to the parent DOM, and communicate via message passing. They're best used for when you're crunching numbers, or parsing large datasets data. In your case, you probably wouldn't need to use them, unless Raphael is also doing some serious computation in the background.

Like others have suggested, you need to split up your work to be asynchronous. That is you do some work, pause every so often to let the browser repaint and handle user input, before doing more work.

If you're already using a jQuery, the .queue() method is a pretty easy way of queuing up work, otherwise you can build one of your own using a series of setTimeouts. http://api.jquery.com/jQuery.queue/

//Create a new jQuery Object. We'll be using the default fx queue as it will execute immediately.
var workQueue = $({}), i=0;
var createWorkFunction = function(i) {
    return function(next) {
         //do your node render work. This function has access to the parameters in the parent function. 
         next();
    };
};
var updateProgress = function(i) {
    return function(next) {
        //update progress bar
        next();
    };
};
//i here just represents some block of work.
for(i=0; i<NUM_BLOCKS; i++) {
    workQueue.queue(createWorkFunction(i)).delay(1).queue(updateProgress(i));
}
workQueue.queue(function(next) {
    //You can add a finished callback here.
    next(); 
});

Some other things to point out though:

1. Is your SVG static? If so, just embed the static SVG into the DOM directly. There will always be overhead to parsing strings and creating SVG nodes on the fly.
2. If your SVG has somewhere around 2000+ nodes. Your browser will take a long time to render static SVG anyway.
3. When doing DOM manipulation it's best to triggering too many repaints and reflows. One way of avoiding this is to create elements off a document fragment, and only attach the finished SVG when it's ready to be shown. Good JS libraries should already be doing this internally, but this would be worth investigating.
4. Use profiling tools to identify bottlenecks. Chrome has an extension called Speed Tracer. This will let you know how much time is spent doing js execution, repaints, reflows and so forth, and will in most cases point you out to bottlenecks. https://developers.google.com/web-toolkit/speedtracer/ IE has a pretty decent profiler built in as well which lets you see where you're spending most of your time in which functions. http://msdn.microsoft.com/en-us/library/ie/gg699341(v=vs.85).aspx