Neo4j Embedded database hangs after abnormal appli

2020-04-16 18:53发布

问题:

I'm going to make my first application based on Neo4j Embedded database public and ran into a serious blocking issue -

This my previous question with database deleting where I'm faced this issue Effectively clear Neo4j database

I use:

OS: Windows 8.1
jdk1.8.0_51 
neo4j version 2.2.3
lucene version 3.6.2
spring-data-neo4j version 3.4.0.M1

I noticed that everything works properly if I use graphDatabaseService.shutdown(); method before terminating of my application.. Otherwise database is destroyed(not only my application but even Neo4j server also hangs on this corrupted db when I try to load my embedded db with Neo4j Web interface).

Is any way to make Neo4j Embedded database more fault tolerant ? I will lose all my data after the first error (for example blackout event) in the production environment..

I spent a few days on this issue but still have no ideas how to proceed with it. Please help.

UPDATED

info from message.log

2015-07-27 08:51:59.232+0000 INFO  [o.n.k.i.DiagnosticsManager]:   index:
2015-07-27 08:51:59.233+0000 INFO  [o.n.k.i.DiagnosticsManager]:     lucene:
2015-07-27 08:51:59.233+0000 INFO  [o.n.k.i.DiagnosticsManager]:       node:
2015-07-27 08:51:59.233+0000 INFO  [o.n.k.i.DiagnosticsManager]:         node_auto_index:
2015-07-27 08:51:59.234+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.fdt: 2015-07-27T11:50:33+0300 - 44,78 MB
2015-07-27 08:51:59.234+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.fdx: 2015-07-27T11:50:33+0300 - 1,78 MB
2015-07-27 08:51:59.234+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.fnm: 2015-07-27T11:50:33+0300 - 54,00 B
2015-07-27 08:51:59.235+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.frq: 2015-07-27T11:50:34+0300 - 3,99 MB
2015-07-27 08:51:59.235+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.nrm: 2015-07-27T11:50:34+0300 - 684,50 kB
2015-07-27 08:51:59.235+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.prx: 2015-07-27T11:50:34+0300 - 4,12 MB
2015-07-27 08:51:59.236+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.tii: 2015-07-27T11:50:34+0300 - 212,98 kB
2015-07-27 08:51:59.236+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2p.tis: 2015-07-27T11:50:34+0300 - 21,83 MB
2015-07-27 08:51:59.237+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2s.cfs: 2015-07-27T11:50:37+0300 - 1,48 MB
2015-07-27 08:51:59.237+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2t.cfs: 2015-07-27T11:50:38+0300 - 1,55 MB
2015-07-27 08:51:59.237+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2u.cfs: 2015-07-27T11:50:39+0300 - 1,57 MB
2015-07-27 08:51:59.238+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _2z.cfs: 2015-07-27T11:50:43+0300 - 1,46 MB
2015-07-27 08:51:59.238+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _30.cfs: 2015-07-27T11:50:44+0300 - 12,49 MB
2015-07-27 08:51:59.238+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _31.cfs: 2015-07-27T11:50:45+0300 - 1,22 MB
2015-07-27 08:51:59.238+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _32.cfs: 2015-07-27T11:51:56+0300 - 3,36 MB
2015-07-27 08:51:59.239+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.fdt: 2015-07-27T11:49:35+0300 - 19,12 MB
2015-07-27 08:51:59.239+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.fdx: 2015-07-27T11:49:35+0300 - 773,77 kB
2015-07-27 08:51:59.239+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.fnm: 2015-07-27T11:49:35+0300 - 54,00 B
2015-07-27 08:51:59.240+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.frq: 2015-07-27T11:49:35+0300 - 1,72 MB
2015-07-27 08:51:59.240+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.nrm: 2015-07-27T11:49:35+0300 - 290,17 kB
2015-07-27 08:51:59.240+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.prx: 2015-07-27T11:49:35+0300 - 1,77 MB
2015-07-27 08:51:59.240+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.tii: 2015-07-27T11:49:35+0300 - 72,69 kB
2015-07-27 08:51:59.241+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _x.tis: 2015-07-27T11:49:35+0300 - 9,29 MB
2015-07-27 08:51:59.241+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments.gen: 2015-07-27T11:51:57+0300 - 20,00 B
2015-07-27 08:51:59.241+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments_2: 2015-07-27T11:51:57+0300 - 2,00 kB
2015-07-27 08:51:59.242+0000 INFO  [o.n.k.i.DiagnosticsManager]:           write.lock: 2015-07-27T11:51:55+0300 - 0,00 B
2015-07-27 08:51:59.242+0000 INFO  [o.n.k.i.DiagnosticsManager]:         - Total: 2015-07-27T11:51:57+0300 - 133,52 MB
2015-07-27 08:51:59.242+0000 INFO  [o.n.k.i.DiagnosticsManager]:       - Total: 2015-07-27T11:48:44+0300 - 133,52 MB
2015-07-27 08:51:59.242+0000 INFO  [o.n.k.i.DiagnosticsManager]:       relationship:
2015-07-27 08:51:59.242+0000 INFO  [o.n.k.i.DiagnosticsManager]:         __rel_types__:
2015-07-27 08:51:59.243+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments.gen: 2015-07-27T11:50:46+0300 - 20,00 B
2015-07-27 08:51:59.243+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments_1: 2015-07-27T11:50:46+0300 - 32,00 B
2015-07-27 08:51:59.243+0000 INFO  [o.n.k.i.DiagnosticsManager]:         - Total: 2015-07-27T11:51:53+0300 - 52,00 B
2015-07-27 08:51:59.244+0000 INFO  [o.n.k.i.DiagnosticsManager]:       - Total: 2015-07-27T11:48:44+0300 - 52,00 B
2015-07-27 08:51:59.244+0000 INFO  [o.n.k.i.DiagnosticsManager]:     - Total: 2015-07-27T11:48:44+0300 - 133,52 MB
2015-07-27 08:51:59.244+0000 INFO  [o.n.k.i.DiagnosticsManager]:   - Total: 2015-07-27T11:48:44+0300 - 133,52 MB
2015-07-27 08:51:59.244+0000 INFO  [o.n.k.i.DiagnosticsManager]:   index.db: 2015-07-27T11:48:33+0300 - 411,00 B
2015-07-27 08:51:59.245+0000 INFO  [o.n.k.i.DiagnosticsManager]:   lock: 2015-07-27T11:48:16+0300 - 0,00 B
2015-07-27 08:51:59.249+0000 INFO  [o.n.k.i.DiagnosticsManager]:   messages.log: 2015-07-27T11:51:54+0300 - 47,16 kB
2015-07-27 08:51:59.250+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore: 2015-07-27T11:51:57+0300 - 8,00 kB
2015-07-27 08:51:59.251+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.counts.db.a: 2015-07-27T11:51:57+0300 - 2,80 kB
2015-07-27 08:51:59.251+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.counts.db.b: 2015-07-27T11:50:46+0300 - 2,80 kB
2015-07-27 08:51:59.252+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.id: 2015-07-27T11:51:58+0300 - 9,00 B
2015-07-27 08:51:59.253+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.labeltokenstore.db: 2015-07-27T11:50:46+0300 - 8,00 kB
2015-07-27 08:51:59.254+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.labeltokenstore.db.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.255+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.labeltokenstore.db.names: 2015-07-27T11:50:46+0300 - 7,98 kB
2015-07-27 08:51:59.256+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.labeltokenstore.db.names.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.259+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.nodestore.db: 2015-07-27T11:51:57+0300 - 5,69 MB
2015-07-27 08:51:59.260+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.nodestore.db.id: 2015-07-27T11:51:58+0300 - 9,00 B
2015-07-27 08:51:59.260+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.nodestore.db.labels: 2015-07-27T11:48:22+0300 - 68,00 B
2015-07-27 08:51:59.261+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.nodestore.db.labels.id: 2015-07-27T11:51:58+0300 - 9,00 B
2015-07-27 08:51:59.265+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db: 2015-07-27T11:51:57+0300 - 31,17 MB
2015-07-27 08:51:59.265+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.arrays: 2015-07-27T11:48:22+0300 - 128,00 B
2015-07-27 08:51:59.266+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.arrays.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.267+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.268+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.index: 2015-07-27T11:50:46+0300 - 8,00 kB
2015-07-27 08:51:59.269+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.index.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.269+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.index.keys: 2015-07-27T11:50:46+0300 - 7,98 kB
2015-07-27 08:51:59.270+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.index.keys.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.274+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.strings: 2015-07-27T11:51:57+0300 - 41,28 MB
2015-07-27 08:51:59.274+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.propertystore.db.strings.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.278+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshipgroupstore.db: 2015-07-27T11:51:57+0300 - 151,68 kB
2015-07-27 08:51:59.280+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshipgroupstore.db.id: 2015-07-27T11:51:58+0300 - 9,00 B
2015-07-27 08:51:59.283+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshipstore.db: 2015-07-27T11:51:57+0300 - 37,01 MB
2015-07-27 08:51:59.284+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshipstore.db.id: 2015-07-27T11:51:58+0300 - 9,00 B
2015-07-27 08:51:59.285+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshiptypestore.db: 2015-07-27T11:50:46+0300 - 8,00 kB
2015-07-27 08:51:59.286+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshiptypestore.db.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.286+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshiptypestore.db.names: 2015-07-27T11:50:46+0300 - 7,98 kB
2015-07-27 08:51:59.287+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.relationshiptypestore.db.names.id: 2015-07-27T11:51:57+0300 - 9,00 B
2015-07-27 08:51:59.287+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.schemastore.db: 2015-07-27T11:50:46+0300 - 8,00 kB
2015-07-27 08:51:59.288+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.schemastore.db.id: 2015-07-27T11:51:58+0300 - 9,00 B
2015-07-27 08:51:59.288+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.transaction.db.0: 2015-07-27T11:50:43+0300 - 251,72 MB
2015-07-27 08:51:59.289+0000 INFO  [o.n.k.i.DiagnosticsManager]:   neostore.transaction.db.1: 2015-07-27T11:50:48+0300 - 6,12 MB
2015-07-27 08:51:59.289+0000 INFO  [o.n.k.i.DiagnosticsManager]:   schema:
2015-07-27 08:51:59.289+0000 INFO  [o.n.k.i.DiagnosticsManager]:     index:
2015-07-27 08:51:59.289+0000 INFO  [o.n.k.i.DiagnosticsManager]:       lucene:
2015-07-27 08:51:59.289+0000 INFO  [o.n.k.i.DiagnosticsManager]:         1:
2015-07-27 08:51:59.290+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _4v.cfs: 2015-07-27T11:50:35+0300 - 395,64 kB
2015-07-27 08:51:59.290+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5c.cfs: 2015-07-27T11:50:44+0300 - 49,47 kB
2015-07-27 08:51:59.290+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5d.fdt: 2015-07-27T11:51:58+0300 - 0,00 B
2015-07-27 08:51:59.291+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5d.fdx: 2015-07-27T11:51:58+0300 - 0,00 B
2015-07-27 08:51:59.291+0000 INFO  [o.n.k.i.DiagnosticsManager]:           failure-message: 2015-07-27T11:48:27+0300 - 16,00 kB
2015-07-27 08:51:59.291+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments.gen: 2015-07-27T11:50:44+0300 - 20,00 B
2015-07-27 08:51:59.292+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments_2: 2015-07-27T11:50:44+0300 - 542,00 B
2015-07-27 08:51:59.292+0000 INFO  [o.n.k.i.DiagnosticsManager]:           write.lock: 2015-07-27T11:48:28+0300 - 0,00 B
2015-07-27 08:51:59.292+0000 INFO  [o.n.k.i.DiagnosticsManager]:         - Total: 2015-07-27T11:51:58+0300 - 461,66 kB
2015-07-27 08:51:59.292+0000 INFO  [o.n.k.i.DiagnosticsManager]:         2:
2015-07-27 08:51:59.293+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _36.cfs: 2015-07-27T11:50:03+0300 - 103,38 kB
2015-07-27 08:51:59.293+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5b.cfs: 2015-07-27T11:50:44+0300 - 73,52 kB
2015-07-27 08:51:59.293+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5c.fdt: 2015-07-27T11:51:58+0300 - 0,00 B
2015-07-27 08:51:59.294+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5c.fdx: 2015-07-27T11:51:58+0300 - 0,00 B
2015-07-27 08:51:59.294+0000 INFO  [o.n.k.i.DiagnosticsManager]:           failure-message: 2015-07-27T11:48:28+0300 - 16,00 kB
2015-07-27 08:51:59.294+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments.gen: 2015-07-27T11:50:44+0300 - 20,00 B
2015-07-27 08:51:59.295+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments_2: 2015-07-27T11:50:44+0300 - 542,00 B
2015-07-27 08:51:59.295+0000 INFO  [o.n.k.i.DiagnosticsManager]:           write.lock: 2015-07-27T11:48:28+0300 - 0,00 B
2015-07-27 08:51:59.295+0000 INFO  [o.n.k.i.DiagnosticsManager]:         - Total: 2015-07-27T11:51:58+0300 - 193,45 kB
2015-07-27 08:51:59.295+0000 INFO  [o.n.k.i.DiagnosticsManager]:         3:
2015-07-27 08:51:59.296+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _37.cfs: 2015-07-27T11:50:04+0300 - 789,69 kB
2015-07-27 08:51:59.296+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _4q.cfs: 2015-07-27T11:50:33+0300 - 427,07 kB
2015-07-27 08:51:59.296+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _55.cfs: 2015-07-27T11:50:41+0300 - 104,29 kB
2015-07-27 08:51:59.297+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5a.cfs: 2015-07-27T11:50:44+0300 - 42,41 kB
2015-07-27 08:51:59.297+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5b.fdt: 2015-07-27T11:51:58+0300 - 0,00 B
2015-07-27 08:51:59.297+0000 INFO  [o.n.k.i.DiagnosticsManager]:           _5b.fdx: 2015-07-27T11:51:58+0300 - 0,00 B
2015-07-27 08:51:59.297+0000 INFO  [o.n.k.i.DiagnosticsManager]:           failure-message: 2015-07-27T11:48:28+0300 - 16,00 kB
2015-07-27 08:51:59.298+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments.gen: 2015-07-27T11:50:44+0300 - 20,00 B
2015-07-27 08:51:59.298+0000 INFO  [o.n.k.i.DiagnosticsManager]:           segments_2: 2015-07-27T11:50:44+0300 - 1,01 kB
2015-07-27 08:51:59.298+0000 INFO  [o.n.k.i.DiagnosticsManager]:           write.lock: 2015-07-27T11:48:28+0300 - 0,00 B
2015-07-27 08:51:59.299+0000 INFO  [o.n.k.i.DiagnosticsManager]:         - Total: 2015-07-27T11:51:58+0300 - 1,35 MB
2015-07-27 08:51:59.300+0000 INFO  [o.n.k.i.DiagnosticsManager]:       - Total: 2015-07-27T11:48:28+0300 - 1,99 MB
2015-07-27 08:51:59.300+0000 INFO  [o.n.k.i.DiagnosticsManager]:     - Total: 2015-07-27T11:48:27+0300 - 1,99 MB
2015-07-27 08:51:59.300+0000 INFO  [o.n.k.i.DiagnosticsManager]:     label:
2015-07-27 08:51:59.301+0000 INFO  [o.n.k.i.DiagnosticsManager]:       lucene:
2015-07-27 08:51:59.301+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _3d.cfs: 2015-07-27T11:50:04+0300 - 427,97 kB
2015-07-27 08:51:59.302+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _3d_1.del: 2015-07-27T11:50:44+0300 - 37,00 B
2015-07-27 08:51:59.302+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _4q.cfs: 2015-07-27T11:50:31+0300 - 209,44 kB
2015-07-27 08:51:59.302+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _4q_1.del: 2015-07-27T11:50:44+0300 - 37,00 B
2015-07-27 08:51:59.302+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _5f.cfs: 2015-07-27T11:50:44+0300 - 90,51 kB
2015-07-27 08:51:59.303+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _5g.cfs: 2015-07-27T11:51:56+0300 - 18,07 kB
2015-07-27 08:51:59.303+0000 INFO  [o.n.k.i.DiagnosticsManager]:         _5h.cfs: 2015-07-27T11:51:56+0300 - 108,35 kB
2015-07-27 08:51:59.303+0000 INFO  [o.n.k.i.DiagnosticsManager]:         segments.gen: 2015-07-27T11:51:56+0300 - 20,00 B
2015-07-27 08:51:59.303+0000 INFO  [o.n.k.i.DiagnosticsManager]:         segments_2: 2015-07-27T11:51:56+0300 - 776,00 B
2015-07-27 08:51:59.304+0000 INFO  [o.n.k.i.DiagnosticsManager]:         write.lock: 2015-07-27T11:48:22+0300 - 0,00 B
2015-07-27 08:51:59.304+0000 INFO  [o.n.k.i.DiagnosticsManager]:       - Total: 2015-07-27T11:51:56+0300 - 855,19 kB
2015-07-27 08:51:59.304+0000 INFO  [o.n.k.i.DiagnosticsManager]:     - Total: 2015-07-27T11:48:22+0300 - 855,19 kB
2015-07-27 08:51:59.304+0000 INFO  [o.n.k.i.DiagnosticsManager]:   - Total: 2015-07-27T11:48:27+0300 - 2,82 MB
2015-07-27 08:51:59.305+0000 INFO  [o.n.k.i.DiagnosticsManager]:   store_lock: 2015-07-27T11:48:16+0300 - 0,00 B
2015-07-27 08:51:59.305+0000 INFO  [o.n.k.i.DiagnosticsManager]: --- STARTED diagnostics for KernelDiagnostics:StoreFiles END ---
2015-07-27 08:51:59.378+0000 INFO  [o.n.k.EmbeddedGraphDatabase]: Database is now ready
2015-07-27 08:52:02.592+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 399ms.
2015-07-27 08:52:08.474+0000 INFO  [o.n.k.i.a.i.s.OnlineIndexSamplingJob]: Sampled index :Criterion(name) with 18159 unique values in sample of avg size 18159 taken from index containing 18159 entries
2015-07-27 08:52:11.514+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 354ms.
2015-07-27 08:52:13.692+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 336ms.
2015-07-27 08:52:18.760+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 3728ms.
2015-07-27 08:52:18.764+0000 INFO  [o.n.k.i.a.i.s.OnlineIndexSamplingJob]: Sampled index :CriterionGroup(name) with 1691 unique values in sample of avg size 1691 taken from index containing 1691 entries
2015-07-27 08:52:19.951+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 490ms.
2015-07-27 08:52:25.130+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 4476ms.
2015-07-27 08:52:26.492+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 761ms.
2015-07-27 08:52:33.603+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 5408ms.
2015-07-27 08:52:35.734+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 809ms.
2015-07-27 08:52:43.564+0000 WARN  [o.n.k.EmbeddedGraphDatabase]: GC Monitor: Application threads blocked for 5601ms.

During delete operation application hangs on tx.close(); line:

public class Neo4jUtils {

    final static Logger logger = LoggerFactory.getLogger(Neo4jUtils.class);

    private static final int BATCH_SIZE = 10;

    public static void cleanDb(Neo4jTemplate template) {
        logger.info("Cleaning database");
        long deletedNodesCount = 0;
        do {
            GraphDatabaseService graphDatabaseService = template.getGraphDatabaseService();
            Transaction tx = graphDatabaseService.beginTx();
            try {
                Result<Map<String, Object>> result = template.query("MATCH (n) WITH n LIMIT " + BATCH_SIZE + " OPTIONAL MATCH (n)-[r]-() DELETE n, r RETURN count(n) as count", null);
                deletedNodesCount = (long) result.single().get("count");
                tx.success();
                logger.info("Deleted " + deletedNodesCount + " nodes...");
            } catch (Throwable th) {
                logger.error("Error while deleting database", th);
                throw th;
            } finally {
                tx.close();
            }
        } while (deletedNodesCount > 0);
    }

}

UPDATED

I found the general source of this issue but have no idea right now how to fix it.

This is my WebSecurityConfig:

@Configuration
@EnableWebSecurity
@EnableWebMvcSecurity
public class WebSecurityConfig extends WebSecurityConfigurerAdapter {


    @Autowired
    private SocialAuthenticationSuccessHandler socialAuthenticationSuccessHandler;

    @Autowired
    private TokenAuthenticationService tokenAuthenticationService;

    @Autowired
    private TokenAuthenticationFilter tokenAuthenticationFilter;

    @Autowired
    private DBUserDetailsService userDetailsService;

.....

}

I autowired 4 fields that internally use Neo4j(DAO, repositories and so on). Neo4j hangs on the restart after unclean shutdown when these fields are present inside of my WebSecurityConfig. When I comment these fields the issue disappears.

What can be wrong ?

回答1:

Neo4j Enterprise allows you to run multiple Neo4j instances in a cluster providing you high availability and scale out for reads. To survive n failures at the same time you need 2n+1 instances.

In case of a unclean shutdown, the expected behaviour is an automatic recovering operation upon next startup (which might take some time). In the last few years I've never experienced Neo4j in a non-recoverable state after a unclean shutdown.

For embedded use cases be sure to call gdb.shutdown() in a JVM shutdown hook.



回答2:

I can confirm that this issue related to the following fix - https://github.com/neo4j/neo4j/commit/c9596af0d990d7dde9a9053799b7295985ae3c05

I have applied the changes from this commit and the issue is gone.

So, right now I'm waiting for Neo4j 2.2.4 release.

Do you know the planned release date for Neo4j 2.2.4 ?