How to store data inside the executable file

Question

I need to find a way to store 250 KB of plain text numbers inside my program's executable file.

Usually, I would put the data in a separate file and let the program read it while it is running, but that's not an option here. Instead, the program and the data need to be in one executable file.

I have absolutely no idea how to do it (except writing 250.000 #defines :-) and I'd appreciate any suggestions.

Answer 1

You can use the xxd command with the -i option to convert any file to a char vector in C. If you are on Windows you can look into using it in Cygwin.

Answer 2

It sounds like you're trying to avoid putting it in a source file, but that's exactly what I'd do:

int numbers[250000] = {1, 2, ...};

It's technically possible to keep them as a plain file and write a linker directive file that creates a new data section of the proper size and combines them, but there's really no reason. Put that definition in a separate file and #include it into the file that needs it

Answer 3

You could just generate an array definition. For example, suppose you have numbers.txt:

$ head -5 numbers.txt
0.99043748698114
0.0243802034269436
0.887296518349228
0.0644020236531517
0.474582201929554

I've generated it for the example using:

$ perl -E'say rand() for (1..250_000)' >numbers.txt

Then to convert it to C array definition you could use a script:

$ perl -lpE'BEGIN{ say "double data[] = {"; }; 
>     END{ say "};" }; 
>     s/$/,/' > data.h < numbers.txt 

It produces:

$ head -5 data.h
double data[] = {
0.99043748698114,
0.0243802034269436,
0.887296518349228,
0.0644020236531517,

$ tail -5 data.h
0.697015237317363,
0.642250552146166,
0.00577098769553785,
0.249176256744811,
};

It could be used in your program as follows:

#include <stdio.h>    
#include "data.h"

int main(void) {
  // print first and last numbers
  printf("%g %g\n", data[0], data[sizeof(data)/sizeof(*data)-1]);
  return 0;
}

Run it:

$ gcc *.c && ./a.out
0.990437 0.249176

Answer 4

Not the solution (this was given before) but: don't put it in a header file. Write a header, which defines a function that returns an array. Then implement this in a .c file. Otherwise, you will end up in a compilation mess...

Answer 5

I agree with the previous answers. The best way is to simply store it in the code and then compile it into the program. For the sake of argument you could look at the format for an executable and add some data/code in there (This is how a lot of viruses work) and simply read from the executable and get the data. http://refspecs.freestandards.org/elf/elf.pdf has the format for an executable. Once again this is for the sake of argument and is not recommended.

Answer 6

You could adapt this solution to numbers:

static const wchar_t *systemList[] = {
    L"actskin4.ocx",
    L"advpack.dll",
    L"asuninst.exe",
    L"aswBoot.exe",
    L"AvastSS.scr",
    L"avsda.dll",
    L"bassmod.dll",
    L"browseui.dll",
    L"CanonIJ Uninstaller Information",
    L"capicom.dll",
    L"cdfview.dll",
    L"cdm.dll",
    L"d3dx9_24.dll",
    L"d3dx9_25.dll",
    L"d3dx9_27.dll",
    L"d3dx9_28.dll",
    L"d3dx9_29.dll",
    L"d3dx9_30.dll",
    L"danim.dll",
    L"dfrgntfs.exe",
    L"dhcpcsvc.dll",
    L"dllhost.exe",
    L"dnsapi.dll",
    L"drivers\\aavmker4.sys",
    L"drivers\\apt.sys",
    L"drivers\\aswFsBlk.sys",
    L"drivers\\aswmon.sys",
    L"drivers\\aswmon2.sys",
    L"drivers\\aswRdr.sys",
    L"drivers\\aswSP.sys",
    L"drivers\\aswTdi.sys",
    L"drivers\\avg7core.sys",
    L"drivers\\avg7rsw.sys",
    L"drivers\\avg7rsxp.sys",
    L"drivers\\avgclean.sys",
    L"drivers\\avgmfx86.sys",
    L"drivers\\avgntdd.sys",
    L"drivers\\avgntmgr.sys",
    L"drivers\\avgtdi.sys",
    L"drivers\\avipbb.sys",
    L"drivers\\cmdmon.sys",
    L"drivers\\gmer.sys",
    L"drivers\\inspect.sys",
    L"drivers\\klick.sys",
    L"drivers\\klif.sys",
    L"drivers\\klin.sys",
    L"drivers\\pxcom.sys",
    L"drivers\\pxemu.sys",
    L"drivers\\pxfsf.sys",
    L"drivers\\pxrd.sys",
    L"drivers\\pxscrmbl.sys",
    L"drivers\\pxtdi.sys",
    L"drivers\\rrspy.sys",
    L"drivers\\rrspy64.sys",
    L"drivers\\ssmdrv.sys",
    L"drivers\\UMDF",
    L"drivers\\USBSTOR.SYS",
    L"DRVSTORE",
    L"dxtmsft.dll",
    L"dxtrans.dll",
    L"en-us",
    L"extmgr.dll",
    L"fntcache.dat",
    L"hal.dll",
    L"icardie.dll",
    L"ie4uinit.exe",
    L"ieakeng.dll",
    L"ieaksie.dll",
    L"ieakui.dll",
    L"ieapfltr.dat",
    L"ieapfltr.dll",
    L"iedkcs32.dll",
    L"ieframe.dll",
    L"iepeers.dll",
    L"iernonce.dll",
    L"iertutil.dll",
    L"ieudinit.exe",
    L"ieui.dll",
    L"imon1.dat",
    L"inseng.dll",
    L"iphlpapi.dll",
    L"java.exe",
    L"javaw.exe",
    L"javaws.exe",
    L"jgdw400.dll",
    L"jgpl400.dll",
    L"jscript.dll",
    L"jsproxy.dll",
    L"kbdaze.dll",
    L"kbdblr.dll",
    L"kbdbu.dll",
    L"kbdkaz.dll",
    L"kbdru.dll",
    L"kbdru1.dll",
    L"kbdtat.dll",
    L"kbdur.dll",
    L"kbduzb.dll",
    L"kbdycc.dll",
    L"kernel32.dll",
    L"legitcheckcontrol.dll",
    L"libeay32_0.9.6l.dll",
    L"Macromed",
    L"mapi32.dll",
    L"mrt.exe",
    L"msfeeds.dll",
    L"msfeedsbs.dll",
    L"msfeedssync.exe",
    L"msftedit.dll",
    L"mshtml.dll",
    L"mshtmled.dll",
    L"msrating.dll",
    L"mstime.dll",
    L"netapi32.dll",
    L"occache.dll",
    L"perfc009.dat",
    L"perfh009.dat",
    L"pncrt.dll",
    L"pndx5016.dll",
    L"pndx5032.dll",
    L"pngfilt.dll",
    L"px.dll",
    L"pxcpya64.exe",
    L"pxdrv.dll",
    L"pxhpinst.exe",
    L"pxinsa64.exe",
    L"pxinst.dll",
    L"pxmas.dll",
    L"pxsfs.dll",
    L"pxwave.dll",
    L"rasadhlp.dll",
    L"rasmans.dll",
    L"riched20.dll",
    L"rmoc3260.dll",
    L"rrsec.dll",
    L"rrsec2k.exe",
    L"shdocvw.dll",
    L"shell32.dll",
    L"shlwapi.dll",
    L"shsvcs.dll",
    L"sp2res.dll",
    L"spmsg.dll",
    L"ssiefr.EXE",
    L"STKIT432.DLL",
    L"streamhlp.dll",
    L"SWSC.exe",
    L"tzchange.exe",
    L"url.dll",
    L"urlmon.dll",
    L"vsdata.dll",
    L"vsdatant.sys",
    L"vsinit.dll",
    L"vsmonapi.dll",
    L"vspubapi.dll",
    L"vsregexp.dll",
    L"vsutil.dll",
    L"vswmi.dll",
    L"vsxml.dll",
    L"vxblock.dll",
    L"webcheck.dll",
    L"WgaLogon.dll",
    L"wgatray.exe",
    L"wiaservc.dll",
    L"windowspowershell",
    L"winfxdocobj.exe",
    L"wmp.dll",
    L"wmvcore.dll",
    L"WREGS.EXE",
    L"WRLogonNtf.dll",
    L"wrlzma.dll",
    L"wuapi.dll",
    L"wuauclt.exe",
    L"wuaueng.dll",
    L"wucltui.dll",
    L"wups.dll",
    L"wups2.dll",
    L"wuweb.dll",
    L"x3daudio1_0.dll",
    L"xactengine2_0.dll",
    L"xactengine2_1.dll",
    L"xactengine2_2.dll",
    L"xinput1_1.dll",
    L"xinput9_1_0.dll",
    L"xmllite.dll",
    L"xpsp3res.dll",
    L"zlcomm.dll",
    L"zlcommdb.dll",
    L"ZPORT4AS.dll"
};