Neo4j cannot read certain nodes. Throws NotFoundEx

Question

After aborting an import-process I wound up with a corrupt database. Reading nodes with a certain label almost always throws an error:

"Unable to load one or more relationships from Node[160147]. This usually happens when relationships are deleted by someone else just as we are about to load them."

The said node is a superconnected one with lots of relations. I cannot delete it nor it's relationships (using cypher) as it throws the same error. This happened in version 2.1.0, however öI have since updated to 2.1.2 in a futile hope the issue would be resolved.

Can I delete the node from the filesystem somehow or how can I salvage this?

Stack below.

(
    [message] => Unable to load one or more relationships from Node[160147]. This usually happens when relationships are deleted by someone else just as we are about to load them. Please try again.
    [exception] => NotFoundException
    [fullname] => org.neo4j.graphdb.NotFoundException
    [stacktrace] => Array
        (
            [0] => org.neo4j.kernel.impl.core.NodeImpl.loadMoreRelationshipsFromNodeManager(NodeImpl.java:481)
            [1] => org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:358)
            [2] => org.neo4j.kernel.impl.core.NodeImpl.loadInitialRelationships(NodeImpl.java:288)
            [3] => org.neo4j.kernel.impl.core.NodeImpl.ensureRelationshipMapNotNull(NodeImpl.java:260)
            [4] => org.neo4j.kernel.impl.core.NodeImpl.getAllRelationships(NodeImpl.java:151)
            [5] => org.neo4j.kernel.impl.core.NodeImpl.getRelationships(NodeImpl.java:235)
            [6] => org.neo4j.kernel.impl.api.store.PersistenceCache.nodeGetRelationships(PersistenceCache.java:205)
            [7] => org.neo4j.kernel.impl.api.store.CacheLayer.nodeListRelationships(CacheLayer.java:445)
            [8] => org.neo4j.kernel.impl.api.StateHandlingStatementOperations.nodeGetRelationships(StateHandlingStatementOperations.java:992)
            [9] => org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.nodeGetRelationships(ConstraintEnforcingEntityOperations.java:364)
            [10] => org.neo4j.kernel.impl.api.OperationsFacade.nodeGetRelationships(OperationsFacade.java:182)
            [11] => org.neo4j.kernel.impl.core.NodeProxy.getRelationships(NodeProxy.java:143)
            [12] => org.neo4j.kernel.impl.core.NodeProxy.getRelationships(NodeProxy.java:76)
            [13] => org.neo4j.cypher.internal.spi.v2_1.TransactionBoundQueryContext.getRelationshipsFor(TransactionBoundQueryContext.scala:112)
            [14] => org.neo4j.cypher.internal.compiler.v2_1.spi.DelegatingQueryContext.getRelationshipsFor(DelegatingQueryContext.scala:57)
            [15] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compiler$v2_1$spi$ExceptionTranslatingQueryContext$$super$getRelationshipsFor(ExceptionTranslatingQueryContext.scala:59)
            [16] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext$$anonfun$getRelationshipsFor$1.apply(ExceptionTranslatingQueryContext.scala:59)
            [17] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext$$anonfun$getRelationshipsFor$1.apply(ExceptionTranslatingQueryContext.scala:59)
            [18] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compiler$v2_1$spi$ExceptionTranslatingQueryContext$$translateException(ExceptionTranslatingQueryContext.scala:149)
            [19] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext.getRelationshipsFor(ExceptionTranslatingQueryContext.scala:59)
            [20] => org.neo4j.cypher.internal.compiler.v2_1.spi.DelegatingQueryContext.getRelationshipsFor(DelegatingQueryContext.scala:57)
            [21] => org.neo4j.cypher.internal.compiler.v2_1.pipes.matching.SingleStep$$anonfun$1.apply(SingleStep.scala:44)
            [22] => org.neo4j.cypher.internal.compiler.v2_1.pipes.matching.SingleStep$$anonfun$1.apply(SingleStep.scala:43)
            [23] => org.neo4j.cypher.internal.helpers.DynamicIterable$$anon$1.iterator(DynamicIterable.scala:28)
            [24] => org.neo4j.cypher.internal.helpers.DynamicJavaIterable$$anon$3.iterator(DynamicIterable.scala:38)
            [25] => org.neo4j.kernel.impl.traversal.TraversalBranchWithState.expandRelationshipsWithoutChecks(TraversalBranchWithState.java:71)
            [26] => org.neo4j.kernel.impl.traversal.TraversalBranchImpl.expandRelationships(TraversalBranchImpl.java:104)
            [27] => org.neo4j.kernel.impl.traversal.StartNodeTraversalBranch.next(StartNodeTraversalBranch.java:47)
            [28] => org.neo4j.kernel.impl.traversal.AsOneStartBranch.next(AsOneStartBranch.java:100)
            [29] => org.neo4j.graphdb.traversal.PreorderDepthFirstSelector.next(PreorderDepthFirstSelector.java:49)
            [30] => org.neo4j.kernel.impl.traversal.MonoDirectionalTraverserIterator.fetchNextOrNull(MonoDirectionalTraverserIterator.java:68)
            [31] => org.neo4j.kernel.impl.traversal.MonoDirectionalTraverserIterator.fetchNextOrNull(MonoDirectionalTraverserIterator.java:35)
            [32] => org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
            [33] => org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
            [34] => scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
            [35] => scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
            [36] => scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
            [37] => scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
            [38] => scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
            [39] => scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
            [40] => scala.collection.Iterator$class.foreach(Iterator.scala:727)
            [41] => scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
            [42] => scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
            [43] => scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:176)
            [44] => scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
            [45] => scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
            [46] => scala.collection.AbstractIterator.to(Iterator.scala:1157)
            [47] => scala.collection.TraversableOnce$class.toList(TraversableOnce.scala:257)
            [48] => scala.collection.AbstractIterator.toList(Iterator.scala:1157)
            [49] => org.neo4j.cypher.internal.compiler.v2_1.pipes.LegacySortPipe.internalCreateResults(LegacySortPipe.scala:32)
            [50] => org.neo4j.cypher.internal.compiler.v2_1.pipes.PipeWithSource.createResults(Pipe.scala:98)
            [51] => org.neo4j.cypher.internal.compiler.v2_1.pipes.PipeWithSource.createResults(Pipe.scala:95)
            [52] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1$$anonfun$apply$2.apply(ExecutionPlanBuilder.scala:119)
            [53] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1$$anonfun$apply$2.apply(ExecutionPlanBuilder.scala:118)
            [54] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionWorkflowBuilder.runWithQueryState(ExecutionPlanBuilder.scala:169)
            [55] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:117)
            [56] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:102)
            [57] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anon$1.execute(ExecutionPlanBuilder.scala:68)
            [58] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anon$1.execute(ExecutionPlanBuilder.scala:67)
            [59] => org.neo4j.cypher.internal.ExecutionPlanWrapperForV2_1.execute(CypherCompiler.scala:126)
            [60] => org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:75)
            [61] => org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:69)
            [62] => org.neo4j.cypher.javacompat.ExecutionEngine.execute(ExecutionEngine.java:84)
            [63] => org.neo4j.server.rest.web.CypherService.cypher(CypherService.java:114)
            [64] => java.lang.reflect.Method.invoke(Method.java:606)
            [65] => org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139)
            [66] => java.lang.Thread.run(Thread.java:724)
        )

    [cause] => Array
        (
            [message] => RelationshipRecord[2121672] not in use
            [exception] => InvalidRecordException
            [stacktrace] => Array
                (
                    [0] => org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getRecord(RelationshipStore.java:267)
                    [1] => org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:337)
                    [2] => org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.getMoreRelationships(NeoStoreTransaction.java:1522)
                    [3] => org.neo4j.kernel.impl.nioneo.xa.NeoStoreTransaction.getMoreRelationships(NeoStoreTransaction.java:1101)
                    [4] => org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:89)
                    [5] => org.neo4j.kernel.impl.core.RelationshipLoader.getMoreRelationships(RelationshipLoader.java:52)
                    [6] => org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:786)
                    [7] => org.neo4j.kernel.impl.core.NodeImpl.loadMoreRelationshipsFromNodeManager(NodeImpl.java:477)
                    [8] => org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:358)
                    [9] => org.neo4j.kernel.impl.core.NodeImpl.loadInitialRelationships(NodeImpl.java:288)
                    [10] => org.neo4j.kernel.impl.core.NodeImpl.ensureRelationshipMapNotNull(NodeImpl.java:260)
                    [11] => org.neo4j.kernel.impl.core.NodeImpl.getAllRelationships(NodeImpl.java:151)
                    [12] => org.neo4j.kernel.impl.core.NodeImpl.getRelationships(NodeImpl.java:235)
                    [13] => org.neo4j.kernel.impl.api.store.PersistenceCache.nodeGetRelationships(PersistenceCache.java:205)
                    [14] => org.neo4j.kernel.impl.api.store.CacheLayer.nodeListRelationships(CacheLayer.java:445)
                    [15] => org.neo4j.kernel.impl.api.StateHandlingStatementOperations.nodeGetRelationships(StateHandlingStatementOperations.java:992)
                    [16] => org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.nodeGetRelationships(ConstraintEnforcingEntityOperations.java:364)
                    [17] => org.neo4j.kernel.impl.api.OperationsFacade.nodeGetRelationships(OperationsFacade.java:182)
                    [18] => org.neo4j.kernel.impl.core.NodeProxy.getRelationships(NodeProxy.java:143)
                    [19] => org.neo4j.kernel.impl.core.NodeProxy.getRelationships(NodeProxy.java:76)
                    [20] => org.neo4j.cypher.internal.spi.v2_1.TransactionBoundQueryContext.getRelationshipsFor(TransactionBoundQueryContext.scala:112)
                    [21] => org.neo4j.cypher.internal.compiler.v2_1.spi.DelegatingQueryContext.getRelationshipsFor(DelegatingQueryContext.scala:57)
                    [22] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compiler$v2_1$spi$ExceptionTranslatingQueryContext$$super$getRelationshipsFor(ExceptionTranslatingQueryContext.scala:59)
                    [23] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext$$anonfun$getRelationshipsFor$1.apply(ExceptionTranslatingQueryContext.scala:59)
                    [24] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext$$anonfun$getRelationshipsFor$1.apply(ExceptionTranslatingQueryContext.scala:59)
                    [25] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext.org$neo4j$cypher$internal$compiler$v2_1$spi$ExceptionTranslatingQueryContext$$translateException(ExceptionTranslatingQueryContext.scala:149)
                    [26] => org.neo4j.cypher.internal.compiler.v2_1.spi.ExceptionTranslatingQueryContext.getRelationshipsFor(ExceptionTranslatingQueryContext.scala:59)
                    [27] => org.neo4j.cypher.internal.compiler.v2_1.spi.DelegatingQueryContext.getRelationshipsFor(DelegatingQueryContext.scala:57)
                    [28] => org.neo4j.cypher.internal.compiler.v2_1.pipes.matching.SingleStep$$anonfun$1.apply(SingleStep.scala:44)
                    [29] => org.neo4j.cypher.internal.compiler.v2_1.pipes.matching.SingleStep$$anonfun$1.apply(SingleStep.scala:43)
                    [30] => org.neo4j.cypher.internal.helpers.DynamicIterable$$anon$1.iterator(DynamicIterable.scala:28)
                    [31] => org.neo4j.cypher.internal.helpers.DynamicJavaIterable$$anon$3.iterator(DynamicIterable.scala:38)
                    [32] => org.neo4j.kernel.impl.traversal.TraversalBranchWithState.expandRelationshipsWithoutChecks(TraversalBranchWithState.java:71)
                    [33] => org.neo4j.kernel.impl.traversal.TraversalBranchImpl.expandRelationships(TraversalBranchImpl.java:104)
                    [34] => org.neo4j.kernel.impl.traversal.StartNodeTraversalBranch.next(StartNodeTraversalBranch.java:47)
                    [35] => org.neo4j.kernel.impl.traversal.AsOneStartBranch.next(AsOneStartBranch.java:100)
                    [36] => org.neo4j.graphdb.traversal.PreorderDepthFirstSelector.next(PreorderDepthFirstSelector.java:49)
                    [37] => org.neo4j.kernel.impl.traversal.MonoDirectionalTraverserIterator.fetchNextOrNull(MonoDirectionalTraverserIterator.java:68)
                    [38] => org.neo4j.kernel.impl.traversal.MonoDirectionalTraverserIterator.fetchNextOrNull(MonoDirectionalTraverserIterator.java:35)
                    [39] => org.neo4j.helpers.collection.PrefetchingIterator.peek(PrefetchingIterator.java:60)
                    [40] => org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:46)
                    [41] => scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
                    [42] => scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
                    [43] => scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
                    [44] => scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
                    [45] => scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
                    [46] => scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
                    [47] => scala.collection.Iterator$class.foreach(Iterator.scala:727)
                    [48] => scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
                    [49] => scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
                    [50] => scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:176)
                    [51] => scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45)
                    [52] => scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
                    [53] => scala.collection.AbstractIterator.to(Iterator.scala:1157)
                    [54] => scala.collection.TraversableOnce$class.toList(TraversableOnce.scala:257)
                    [55] => scala.collection.AbstractIterator.toList(Iterator.scala:1157)
                    [56] => org.neo4j.cypher.internal.compiler.v2_1.pipes.LegacySortPipe.internalCreateResults(LegacySortPipe.scala:32)
                    [57] => org.neo4j.cypher.internal.compiler.v2_1.pipes.PipeWithSource.createResults(Pipe.scala:98)
                    [58] => org.neo4j.cypher.internal.compiler.v2_1.pipes.PipeWithSource.createResults(Pipe.scala:95)
                    [59] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1$$anonfun$apply$2.apply(ExecutionPlanBuilder.scala:119)
                    [60] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1$$anonfun$apply$2.apply(ExecutionPlanBuilder.scala:118)
                    [61] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionWorkflowBuilder.runWithQueryState(ExecutionPlanBuilder.scala:169)
                    [62] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:117)
                    [63] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anonfun$getExecutionPlanFunction$1.apply(ExecutionPlanBuilder.scala:102)
                    [64] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anon$1.execute(ExecutionPlanBuilder.scala:68)
                    [65] => org.neo4j.cypher.internal.compiler.v2_1.executionplan.ExecutionPlanBuilder$$anon$1.execute(ExecutionPlanBuilder.scala:67)
                    [66] => org.neo4j.cypher.internal.ExecutionPlanWrapperForV2_1.execute(CypherCompiler.scala:126)
                    [67] => org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:75)
                    [68] => org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:69)
                    [69] => org.neo4j.cypher.javacompat.ExecutionEngine.execute(ExecutionEngine.java:84)
                    [70] => org.neo4j.server.rest.web.CypherService.cypher(CypherService.java:114)
                    [71] => java.lang.reflect.Method.invoke(Method.java:606)
                    [72] => org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139)
                    [73] => java.lang.Thread.run(Thread.java:724)
                )

            [fullname] => org.neo4j.kernel.impl.nioneo.store.InvalidRecordException
        )
)

Answer 1

For consistency checking your datastore check out Mark's great blog post.

Answer 2

What kind of import process did you abort how? If it was batch-inserter, then there are no guarantees on consistent stores if you abort it and you'll probably safer redoing your import.

If it was transactional there shouldn't be inconsistencies.

Please also note that 2.1.0 had some issues and shouldn't be used, so you might run into double issues there.