Is there any way to determine how many characters

Question

I'm working in C++.

I want to write a potentially very long formatted string using sprintf (specifically a secure counted version like _snprintf_s, but the idea is the same). The approximate length is unknown at compile time so I'll have to use some dynamically allocated memory rather than relying on a big static buffer. Is there any way to determine how many characters will be needed for a particular sprintf call so I can always be sure I've got a big enough buffer?

My fallback is I'll just take the length of the format string, double it, and try that. If it works, great, if it doesn't I'll just double the size of the buffer and try again. Repeat until it fits. Not exactly the cleverest solution.

It looks like C99 supports passing NULL to snprintf to get the length. I suppose I could create a module to wrap that functionality if nothing else, but I'm not crazy about that idea.

Maybe an fprintf to "/dev/null"/"nul" might work instead? Any other ideas?

EDIT: Alternatively, is there any way to "chunk" the sprintf so it picks up mid-write? If that's possible it could fill the buffer, process it, then start refilling from where it left off.

Answer 1

I would use a two-stage approach. Generally, a large percentage of output strings will be under a certain threshold and only a few will be larger.

Stage 1, use a reasonable sized static buffer such as 4K. Since snprintf() can restrict how many characters are written, you won't get a buffer overflow. What you will get returned from snprintf() is the number of characters it would have written if your buffer had been big enough.

If your call to snprintf() returns less than 4K, then use the buffer and exit. As stated, the vast majority of calls should just do that.

Some will not and that's when you enter stage 2. If the call to snprintf() won't fit in the 4K buffer, you at least now know how big a buffer you need.

Allocate, with malloc(), a buffer big enough to hold it then snprintf() it again to that new buffer. When you're done with the buffer, free it.

We worked on a system in the days before snprintf() and we acheived the same result by having a file handle connected to /dev/null and using fprintf() with that. /dev/null was always guaranteed to take as much data as you give it so we would actually get the size from that, then allocate a buffer if necessary.

Keep in kind that not all systems have snprintf() (for example, I understand it's _snprintf() in Microsoft C) so you may have to find the function that does the same job, or revert to the fprintf /dev/null solution.

Also be careful if the data can be changed between the size-checking snprintf() and the actual snprintf() to the buffer (i.e., wathch out for threads). If the sizes increase, you'll get buffer overflow corruption.

If you follow the rule that data, once handed to a function, belongs to that function exclusively until handed back, this won't be a problem.

Answer 2

As others have mentioned, snprintf() will return the number of characters required in a buffer to prevent the output from being truncated. You can simply call it with a 0 buffer length parameter to get the required size then use an appropriately sized buffer.

For a slight improvement in efficiency, you can call it with a buffer that's large enough for the normal case and only do a second call to snprintf() if the output is truncated. In order to make sure the buffer(s) are properly freed in that case, I'll often use an auto_buffer<> object that handles the dynamic memory for me (and has the default buffer on the stack to avoid a heap allocation in the normal case).

If you're using a Microsoft compiler, MS has a non-standard _snprintf() that has serious limitations of not always null terminating the buffer and not indicating how big the buffer should be.

To work around Microsoft's non-support, I use a nearly public domain snprintf() from Holger Weiss.

Of course if your non-MS C or C++ compiler is missing snprintf(), the code from the above link should work just as well.

Answer 3

Since you're using C++, there's really no need to use any version of sprintf. The simplest thing to do is use a std::ostringstream.

std::ostringstream oss;
oss << a << " " << b << std::endl;

oss.str() returns a std::string with the contents of what you've written to oss. Use oss.str().c_str() to get a const char *. It's going to be a lot easier to deal with in the long run and eliminates memory leaks or buffer overruns. Generally, if you're worrying about memory issues like that in C++, you're not using the language to its full potential, and you should rethink your design.

Answer 4

I've looked for the same functionality you're talking about, but as far as I know, something as simple as the C99 method is not available in C++, because C++ does not currently incorporate the features added in C99 (such as snprintf).

Your best bet is probably to use a stringstream object. It's a bit more cumbersome than a clearly written sprintf call, but it will work.

Answer 5

Take a look at CodeProject: CString-clone Using Standard C++. It uses solution you suggested with enlarging buffer size.

// -------------------------------------------------------------------------
    // FUNCTION:  FormatV
    //      void FormatV(PCSTR szFormat, va_list, argList);
    //

    // DESCRIPTION:
    //      This function formats the string with sprintf style format-specs. 
    //      It makes a general guess at required buffer size and then tries
    //      successively larger buffers until it finds one big enough or a
    //      threshold (MAX_FMT_TRIES) is exceeded.
    //
    // PARAMETERS: 
    //      szFormat - a PCSTR holding the format of the output
    //      argList - a Microsoft specific va_list for variable argument lists
    //
    // RETURN VALUE: 
    // -------------------------------------------------------------------------


void FormatV(const CT* szFormat, va_list argList)
{
#ifdef SS_ANSI

    int nLen    = sslen(szFormat) + STD_BUF_SIZE;
    ssvsprintf(GetBuffer(nLen), nLen-1, szFormat, argList);
    ReleaseBuffer();
#else
    CT* pBuf            = NULL;
    int nChars          = 1;
    int nUsed           = 0;
    size_type nActual   = 0;
    int nTry            = 0;

    do  
    {
        // Grow more than linearly (e.g. 512, 1536, 3072, etc)

        nChars          += ((nTry+1) * FMT_BLOCK_SIZE);
        pBuf            = reinterpret_cast<CT*>(_alloca(sizeof(CT)*nChars));
        nUsed           = ssnprintf(pBuf, nChars-1, szFormat, argList);

        // Ensure proper NULL termination.
        nActual         = nUsed == -1 ? nChars-1 : SSMIN(nUsed, nChars-1);
        pBuf[nActual+1]= '\0';


    } while ( nUsed < 0 && nTry++ < MAX_FMT_TRIES );

    // assign whatever we managed to format

    this->assign(pBuf, nActual);
#endif
}

Answer 6

For what it's worth, asprintf is a GNU extension that manages this functionality. It accepts a pointer as an output argument, along with a format string and a variable number of arguments, and writes back to the pointer the address of a properly-allocated buffer containing the result.

You can use it like so:

#define _GNU_SOURCE
#include <stdio.h>

int main(int argc, char const *argv[])
{
    char *hi = "hello"; // these could be really long
    char *everyone = "world";
    char *message;
    asprintf(&message, "%s %s", hi, everyone);
    puts(message);
    free(message);
    return 0;
}

Hope this helps someone!