Can I replace a call to open_memstream with a mall

Question

All,

I have a program that prints to a stream. I need to buffer this stream in memory, and then print each line as necessary to an actual file later.

Since the fprintf() function calls must have a FILE * pointer, I need to have said pointer addressing space in memory. I had used the open_memstream() function, but this is not supported on windows.

Since malloc() returns a void * pointer that magically casts to the necessary pointer as needed, could I use that as my FILE * pointer? If so, what caveats are there? Do I need to watch out for running out of space?

Update:

After finding the source for open_memstream(), which was harder than it should have been, it looks like they are doing a file stream to malloc'd space.

Since that is the case, and I've got their source, I'm going to se if I can't get a working version to cross compile for windows with mingw.

Answer 1

The function you want is sprintf() or the related snprintf(), which will format a stream to a string for later use.

Answer 2

No. malloc() just gives you a block of (probably uninitialized) memory. There's no "magical" casting going on; when you do int * buf = malloc(10*sizeof(int); you are pointing buf at, effectively, 10 uninitialized ints.

The corresponding thing with a FILE would be FILE * f = malloc(10*sizeof(FILE)); which points f at 10 uninitialized FILE structures, which doesn't make any sense. Furthermore, writing to an uninitialized FILE is likely to result in a crash if you're lucky.

It's easier to help if you tell us what platforms you're targeting and what you actually want to achieve. On POSIX, you can use shm_open() to get a file descriptor pointing to "shared memory" and fdopen() to turn the file descriptor to a FILE*. Yes, it might run out of space.

Answer 3

For those who come after me, have hope! There is a solution. As noted in my question, I was using open_memstream(), which is unsupported on windows.

Since I have a File * pointer (this cannot be changed to char *), I needed to redirect it to memory until later. Since I'm dealing with a file in memory, I looked into mmap(). It handily solves the problem, but again, it is linux only.

But, windows includes a corollary to mmap() called MapViewOfFile(). Through the magic of #ifdef I've got it using whichever is necessary:

#ifdef WIN32
#include <windows.h>
#else
#include <sys/mman.h>
#endif

Later on, in the main method, I call tmpfile() which is supported on both platforms. This opens a stream to a guaranteed unique temporary file for me. Now that I have my FILE * pointer, I need to mmap() the space. But mmap() needs a file descriptor, not a stream, so I used the fileno() function to get the new file descriptor.

/* create tmp file and get file descriptor */
int fd;
yyout = tmpfile();
fd = fileno(yyout);

Now I have some more #ifdef code to determine which memory mapping codeset needs to be used. Note the difference in the mapped space between the two versions. Windows maps 16384 bytes and linux maps 4096 bytes. This is because the smaller value segfaults on windows, as noted in my question here.

#ifdef WIN32
    HANDLE fm;
    HANDLE h = (HANDLE) _get_osfhandle (fd);

    fm = CreateFileMapping(
             h,
             NULL,
             PAGE_READWRITE|SEC_RESERVE,
             0,
             16384,
             NULL);
    if (fm == NULL) { 
            fprintf (stderr, "%s: Couldn't access memory space! %s\n", argv[0],  strerror (GetLastError()));
            exit(GetLastError());
    }
    bp = (char*)MapViewOfFile(
              fm,
              FILE_MAP_ALL_ACCESS,
              0,
              0,
              0);
    if (bp == NULL) { 
            fprintf (stderr, "%s: Couldn't fill memory space! %s\n", argv[0],  strerror (GetLastError()));
            exit(GetLastError());
    }
#else
    bp = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_FILE|MAP_PRIVATE, fd, 0);
    if (bp == MAP_FAILED) {
            fprintf (stderr, "%s: Couldn't access memory space! %s\n", argv[0], FileName, strerror (errno));
            exit(errno);
    }
#endif

A bunch of work now happens, wherin data is sent to the yyout stream. Eventually the flushData() method gets called. It finalizes the stream with a null terminated character, flushes it, and then rewinds it. Then the pointer to the memory space is passed through a function pointer, along with the proper stream to print to.

void flushData(void) {
    /* write out data in the stream and reset */ 
    while (currFields < headerFields) { fprintf(yyout, ",\"\""); currFields++; } 
    currFields = 0;
    fprintf(yyout, "%c%c%c", 13, 10, '\0');
    fflush(yyout);
    rewind(yyout);
    if (faqLine == 1) {
        faqLine = 0; /* don't print faq's to the data file */
    }
    else {
        (*printString)(outfile, bp);
        fflush(outfile);
    }
    fflush(yyout);
    rewind(yyout);
}

This is one of the functions that could be pointed to for printing. It walks the memory space and prints each char until it hits the null printed earlier.

int printAnsi( FILE *outstream, char *string) {
    /* loop over the chars in string and print them to the outputstream as ansi */
    char * ps = string;
    while (*ps != '\0') {
        fprintf(outstream, "%c", *ps);
        ps++;
    }
    return 0;
}

The end result of all this being that I have a stream to memory space just like open_memstream(), up to also having a char pointer I can use to walk through the memory space if necessary. It is cross platform, and (seemingly) fully functional.

If anyone wants more details or have notes about problems I should fix, please add a comment.