Way to store large amount of experiments and corre

问题:

I have an experiment setup that produces a few csv and text files every run. In my research I run a number of these in parallel each day and trying to store and organized them in a directory heir-achy is getting rather inefficient and tedious.

One idea I thought of doing was using MySQL or something to store the the run description along with the associated output. This however has the inefficiency in that I am storing a number of gigantic Strings in the table fields.

Basically I would like to know if a system exists that can automatically organize the output of runs. Such that it would give a list of the run descriptions, and upon choosing this you can request the corresponding outputs. This wouldn't have to be in any particular language as I can hopefully just add the step to a shell script.

回答1:

I assume that the csv and txt files are identically named for each run minus the extensions. Have you considered a simple java or python script that could take all of the csv and txt files in a folder and put them into a single XML file where all of the information would be saved and with the data from the linked csv file and txt file together, meaning that for hundreds of file pairs (csv and txt) you could end up with a single XML file that could be opened to see the separate run results?