Neo4J / py2neo — cursor-based query?

Question

If I do something like this:

from py2neo import Graph
graph = Graph()
stuff = graph.cypher.execute("""
    match (a:Article)-[p]-n return a, n, p.weight
""")

on a database with lots of articles and links, the query takes a long time and uses all my system's memory, presumably because it's copying the entire result set into memory in one go. Is there some kind of cursor-based version where I could iterate through the results one at a time without having to have them all in memory at once?

EDIT

I found the stream function:

stuff = graph.cypher.stream("""
    match (a:Article)-[p]-n return a, n, p.weight
""")

which seems to be what I want according to the documentation but now I get a timeout error (py2neo.packages.httpstream.http.SocketError: timed out), followed by the server becoming unresponsive until I kill it using kill -9.

Answer 1

Have you tried implementing a paging mechanism? Perhaps with the skip keyword: http://neo4j.com/docs/stable/query-skip.html

Similar to using limit / offset in a postgres / mysql query.

EDIT: I previously said that the entire result set was stored in memory, but it appears this is not the case when using api streaming - per Nigel's (Neo engineer) comment below.