Read unknown number of lines from stdin, C

Question

i have a problem with reading stdin of unknown size. In fact its a table in .txt file, which i get to stdin by calling parameter '<'table.txt. My code should look like this:

#include <stdio.h>
#include <string.h>


int main(int argc,char *argv[])
{

   char words[10][1024];
   int i=0;
   while(feof(stdin)==0)
   {
      fgets(words[i],100,stdin); 
      printf("%s", words[i]);
      i++;
   }
   return 0;
}

but there is the problem i dont know the nuber of lines, which in this case is 10(we know the number of characters in line - 1024). It would be great if someone know the solution. Thanks in advance.

Answer 1

I suggest that you use malloc and realloc to manage your memory. Keep track of how big your array is or how many entries it has, and call realloc to double its size whenever the array is not big enough.

Answer 2

Op appears to need to store the data somewhere

#define N 100000u
char BABuffer[N];

int main(int argc, char *argv[]) {
  size_t lcount = 0;
  size_t ccount = 0;
  char words[1024 + 2];

  while(fgets(words, sizeof words, stdin) != NULL) {
    size_t len = strlen(words);
    if (ccount + len >= N - 1) {
      fputs("Too much!\n", stderr);
      break;
    }
    memcpy(&BABuffer[ccount], words, len);
    ccount += len;
    lcount++;
  }
  BABuffer[ccount] = '\0';

  printf("Read %zu lines.\n", lcount);
  printf("Read %zu char.\n", ccount);
  fputs(BABuffer, stdout);
  return 0;
}

Note: ccount includes the end-of-line character(s).

Answer 3

You have hit on one of the issues that plagues all new C-programmers. How do I dynamically allocate all memory I need to free myself from static limits while still keeping track of my collection of 'stuff' in memory. This problem usually presents itself when you need to read an unknown number of 'things' from an input. The initial options are (1) declare some limit big enough to work (defeating the purpose), or (2) dynamically allocate a pointers as needed.

Obviously, the goal is (2). However, you then run into the problem of "How do I keep track of what I've allocated?" This in itself is an issue that dogs beginners. The problem being, If I dynamically allocate using a bunch of pointers, **How do I iterate over the list to get my 'stuff' back out? Also, you have to initialize some initial number of pointers (unless using an advanced data structure like a linked-list), so the next question is "what do I do when I run out?"

The usual solution is to allocate an initial set of pointers, then when the limit is reached, reallocate to twice as many as original, and keep going. (as Grayson indicated in his answer).

However, there is one more trick to iterate over the list to get your 'stuff' back out that is worth understanding. Yes, you can allocate with malloc and keep track of the number of pointers used, but you can free yourself from tying a counter to your list of pointers by initially allocating with calloc. That not only allocates space, but also sets the allocated pointers to NULL (or 0). This allows you to iterate over your list with a simple while (pointer != NULL). This provides many benefits when it comes to passing your collection of pointers to functions, etc.. The downside (a minimal one) is that you get to write a reallocation scheme that uses calloc to allocate new space when needed. (bummer, I get to get smarter -- but I have to work to do it...)

You can evaluate whether to use malloc/realloc off-the-shelf, or whether to reallocate using calloc and a custom reallocate function depending on what your requirements are. Regardless, understanding both, just adds more tools to your programming toolbox.

OK, enough jabber, where is the example in all this blather?

Both of the following examples simply read all lines from any text file and print the lines (with pointer index numbers) back to stdout. Both expect that you will provide the filename to read as the first argument on the command line. The only difference between the two is the second has the reallocation with calloc done is a custom reallocation function. They both allocate 255 pointers initially and double the number of pointers each time the limit is hit. (for fun, you can set MAXLINES to something small like 10 and force repeated reallocations to test).

first example with reallocation in main()

# include <stdio.h>
# include <stdlib.h>
# include <string.h>

#define MAXLINES 255

void free_buffer (char **buffer)
{
    register int i = 0;

    while (buffer[i])
    {
        free (buffer[i]);
        i++;
    }

    free (buffer);

}

int main (int argc, char **argv) {

    if (argc < 2) {
        fprintf (stderr, "Error: insufficient input. Usage: %s input_file\n", argv[0]);
        return 1;
    }

    char *line = NULL;      /* forces getline to allocate space for buf */
    ssize_t read = 0;       /* number of characters read by getline     */
    size_t n = 0;           /* limit number of chars to 'n', 0 no limit */
    char **filebuf = NULL;
    char **rtmp = NULL;
    int linecnt = 0;
    size_t limit = MAXLINES;
    size_t newlim = 0;

    FILE *ifp = fopen(argv[1],"r");
    if (!ifp)
    {
        fprintf(stderr, "\nerror: failed to open file: '%s'\n\n", argv[1]);
        return 1;
    }

    filebuf = calloc (MAXLINES, sizeof (*filebuf)); /* allocate MAXLINES pointers           */

    while ((read = getline (&line, &n, ifp)) != -1) /* read each line in file with getline  */
    {
        if (line[read - 1] == 0xa) { line[read - 1] = 0; read--; } /* strip newline         */

        if (linecnt >= (limit - 1))                 /* test if linecnt at limit, reallocate */
        {
            newlim = limit * 2;                     /* set new number of pointers to 2X old */
            if ((rtmp = calloc (newlim, sizeof (*filebuf))))    /* calloc to set to NULL    */
            {
                /* copy original filebuf to newly allocated rtmp */
                if (memcpy (rtmp, filebuf, linecnt * sizeof (*filebuf)) == rtmp)
                {
                    free (filebuf);                 /* free original filebuf                */
                    filebuf = rtmp;                 /* set filebuf equal to new rtmp        */
                }
                else
                {
                    fprintf (stderr, "error: memcpy failed, exiting\n");
                    return 1;
                }
            }
            else
            {
                fprintf (stderr, "error: rtmp allocation failed, exiting\n");
                return 1;
            }
            limit = newlim;                         /* update limit to new limit    */
        }

        filebuf[linecnt] = strdup (line);           /* copy line (strdup allocates) */

        linecnt++;                                  /* increment linecnt            */
    }

    fclose(ifp);

    if (line) free (line);                          /* free memory allocated to line    */

    linecnt = 0;                                    /* reset linecnt to iterate filebuf */

    printf ("\nLines read in filebuf buffer:\n\n"); /* output all lines read            */
    while (filebuf[linecnt])
    {
        printf (" line[%d]: %s\n", linecnt, filebuf[linecnt]);
        linecnt++;
    }

    printf ("\n");

    free_buffer (filebuf);                          /* free memory allocated to filebuf */

    return 0;
}

second example with reallocation in custom function

# include <stdio.h>
# include <stdlib.h>
# include <string.h>

#define MAXLINES 255

/* function to free allocated memory */
void free_buffer (char **buffer)
{
    register int i = 0;

    while (buffer[i]) 
    { 
        free (buffer[i]); 
        i++; 
    }

    free (buffer);

}

/* custom realloc using calloc/memcpy */
char **recalloc (size_t *lim, char **buf)
{
    int newlim = *lim * 2;
    char **tmp = NULL;

    if ((tmp = calloc (newlim, sizeof (*buf))))
    {
        if (memcpy (tmp, buf, *lim * sizeof (*buf)) == tmp)
        {
            free (buf);
            buf = tmp;
        }
        else
        {
            fprintf (stderr, "%s(): error, memcpy failed, exiting\n", __func__);
            return NULL;
        }
    }
    else
    {
        fprintf (stderr, "%s(): error, tmp allocation failed, exiting\n", __func__);
        return NULL;
    }

    *lim = newlim;

    return tmp;
}

int main (int argc, char **argv) {

    if (argc < 2) {
        fprintf (stderr, "Error: insufficient input. Usage: %s input_file\n", argv[0]);
        return 1;
    }

    char *line = NULL;      /* forces getline to allocate space for buf */
    ssize_t read = 0;       /* number of characters read by getline     */
    size_t n = 0;           /* limit number of chars to 'n', 0 no limit */
    char **filebuf = NULL;
    int linecnt = 0;
    size_t limit = MAXLINES;

    FILE *ifp = fopen(argv[1],"r");
    if (!ifp)
    {
        fprintf(stderr, "\nerror: failed to open file: '%s'\n\n", argv[1]);
        return 1;
    }

    filebuf = calloc (MAXLINES, sizeof (*filebuf)); /* allocate MAXLINES pointers           */

    while ((read = getline (&line, &n, ifp)) != -1) /* read each line in file with getline  */
    {
        if (line[read - 1] == 0xa) { line[read - 1] = 0; read--; } /* strip newline         */

        if (linecnt >= (limit - 1))                 /* test if linecnt at limit, reallocate */
        {
            filebuf = recalloc (&limit, filebuf);   /* reallocate filebuf to 2X size        */
            if (!filebuf)
            {
                fprintf (stderr, "error: recalloc failed, exiting.\n");
                return 1;
            }
        }

        filebuf[linecnt] = strdup (line);           /* copy line (strdup allocates)     */

        linecnt++;                                  /* increment linecnt                */
    }

    fclose(ifp);

    if (line) free (line);                          /* free memory allocated to line    */

    linecnt = 0;                                    /* reset linecnt to iterate filebuf */

    printf ("\nLines read in filebuf buffer:\n\n"); /* output all lines read            */
    while (filebuf[linecnt])
    {
        printf (" line[%d]: %s\n", linecnt, filebuf[linecnt]);
        linecnt++;
    }

    printf ("\n");

    free_buffer (filebuf);                          /* free memory allocated to filebuf */

    return 0;
}

Take a look at both examples. Know that there are many, many ways to do this. These examples just give one approach that provide example of using a few extra tricks than you will normally find. Give them a try. Drop a comment if you need more help.