I need to traverse forms on a site and save intermediate results to files. I'm using phantomjs' page.evaluate
, but I'm having trouble accessing the filesystem from within page.evaluate
's sandboxed environment. I have something like this:
for (var i = 0; i<option1.length; i++){
for (var ii = 0; ii<option2.length; ii++){
for (var iii = 0; iii<option3.length; iii++){
...
//I found what I want to save
fs.write("someFileName", someData);
}
}
}
Obviously, I don't have access to nodejs' fs
from within page.evaluate
, so the above does not work. I seem to have a few options:
- Store everything I need to write to an array, and return that from the
page.evaluate
context into the outer, nodejs context, then save it from there. This would require memory I don't have. - Break up the above logic into smaller
page.evaluate
methods that return singe pieces of data to save to the filesytem. - Somehow pass into the
page.evaluate
a magic function to write to the filesystem. This seems to not be possible (if I try to pass in a function that callsfs.writeFile
for example, I get thatfs
is undefined, even iffs
is a free variable in the function I passed?) - Return an iterator which, when pulled, yields the next piece of data to be written
- Setup a trivial web server on the localhost that simply accepts POST requests and writes their contents into the filesystem. The
page.evaluate
code would then make those requests to the localhost. I almost try this but I'm not sure I'll be affected by the same-origin policy.
What are my options here?