Yaml to database

Question

For my c++ program, I use jobs.yml which contains information about jobs in yaml format like:

141647523:
  JobType: Turbo
  LocalJobID: 16773779.0
  Status: Done
  SystemPriority: 0.0
  UserPriority: 8.0

Then I use yaml-cpp library for parsing file and casting jobs to c++ Job objects. Due to the big size of jobs.yml (~900Mb) I can't read all file at once at the beginning of the program and store in RAM.

How can I overcome this obstacle? I guess that I should convert jobs.yml to database and in the program get desired data through sql requests? In this case what I should do?

Or maybe there are available more simple solutions?

Answer 1

yaml-cpp has an event-based API. Just derive from YAML::EventHandler and use YAML::Parser.HandleNextDocument. Then process the events in your event handler one-by-one. This should save you from loading the whole file into RAM.