In Kyoto Cabinet Database using File Hash Database

Question

I am using the follow to open:

db.open("db.kch#tune_defrag=10000", DB.OWRITER | DB.OCREATE)

I am putting and removing elements. At the end of execution, the database is "empty", the count() function returns 0, because I remove all elements. Why the file size always increase when I repeat the test? Is it possible to run something like "garbage collector" to clean the removed registers? If I execute the same test 100 times, I have a 500 MB size database, even I have only 2 records.

I tried to put the "tune_defrag=10000" but I think it didn't work fine.

Obs.: a only register is less than 1K, I don't understand why the registers take so much space in disk.

Thanks for any help

Answer 1

I haven't used this particular db but in some other ones a hack to resolve this issue is to copy the db into a new one, and then delete the old one. After making sure it copied well :).

I've implemented this process into production systems, as long as it is coded really really well it should work.

Answer 2

running:

kchashmgr defrag path_to_kcabinet_file

is what i do to get the db file 'resized'. I didn't find the api access to this, that's why i do it with a shell command using kchashmgr utility (obviosuly this can be called from inside a program).

Answer 3

From a cursory look through the kyoto documentation it doesn't appear that you have any way to resize or otherwise clean the database of deleted records... or really to manage it in any way shape or form.

That project looks like it is a long way off from being "production ready". If you really want to implement it, I'd suggest contacting the project owners (http://fallabs.com/) and seeing if they have any plans for some much needed utility functions.

Otherwise, I'd suggest moving to a different nosql style database that is a bit more mature.

Answer 4

Try this:

db.kch#dfunit=8

That means the KC runs defrag for every 8 fragmentation detected, and 8 is actually recommended by Mikio.

Available options are listed here, however it could use some polish.

http://fallabs.com/kyotocabinet/command.html