C++ CSV line with commas and strings within double

Question

I'm reading a CSV file in C++ and the row format is as such:

"Primary, Secondary, Third", "Primary", , "Secondary", 18, 4, 0, 0, 0

(notice the empty value)

When I do:

while (std::getline(ss, csvElement, ',')) {
   csvColumn.push_back(csvElement);
}

This splits up the first string into pieces which isn't correct.

How do I preserve the string when iterating? I tried to do a combination of the above and while also grabbing the lines separated by double quote but I got wild results.

Answer 1

You need to interpret the comma depending on whether you're betwwen the quote or not. This is too complexfor getline().

The solution would be to read the full line with getline(), and parse the line by iterating through the string character by character, and maintaing an indicator whether you're between double quotes or not.

Here is a first "raw" example (double quotes are not removed in the fields and escape characters are not interpreted):

string line; 
while (std::getline(cin, line)) {        // read full line
    const char *mystart=line.c_str();    // prepare to parse the line - start is position of begin of field
    bool instring{false};                
    for (const char* p=mystart; *p; p++) {  // iterate through the string
        if (*p=='"')                        // toggle flag if we're btw double quote
            instring = !instring;     
        else if (*p==',' && !instring) {    // if comma OUTSIDE double quote
            csvColumn.push_back(string(mystart,p-mystart));  // keep the field
            mystart=p+1;                    // and start parsing next one
        }
    }
csvColumn.push_back(string(mystart));   // last field delimited by end of line instead of comma
}

Online demo

Answer 2

How do I preserve the string when iterating?

Here is the C++ approach I have used.

I noticed that you have only 3 field types: string, null, and int.

The following approach uses these field types (in method "void init()"), in the order each each row presents the fields, sometimes using string::find() ( instead of getline() ) to locate field end.

Each of the 3 methods consumes characters from the string with erase. I know erase is slow, but I made this choice for my convenience. (erasing is easier to test, just add a cout after each extract). The erase's can be removed / replaced by appropriate handling (where needed) of start-of-search index.

#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>
#include <vector>

#include <cassert>


class CSV_t
{
   typedef std::vector<int>  IntVec_t;

   // private nested class -- holds contents of 1 csv record
   class CSVRec_t
   {
   public:
      std::string primary;
      std::string secondary;
      std::string nullary;
      std::string thirdary;
      IntVec_t    i5;

      std::string show()
         {
            std::stringstream ss;
            ss <<            std::setw(25) << primary
               << "     " << std::setw(10) << secondary
               << "     " << std::setw(12)<< thirdary << "     ";

            for (size_t i=0;
                 i<i5.size(); ++i) ss << std::setw(5) << i5[i];

            ss << std::endl;
            return (ss.str());
         }

   }; // class CSVRec_t


   typedef std::vector<CSVRec_t> CSVRecVec_t;

   CSVRecVec_t csvRecVec;  // holds all csv record

public:

   CSV_t() { };

   void init(std::istream& ss)
      {
         do  // read all rows of file
         {
            CSVRec_t csvRec;

            std::string s;
            (void)std::getline(ss, s);

            if(0 == s.size()) break;

            assert(s.size()); extractQuotedField(s, csvRec.primary);   // 1st quoted substring
            assert(s.size()); extractQuotedField(s, csvRec.secondary); // 2nd quoted substring
            assert(s.size()); confirmEmptyField(s, csvRec.nullary);    // null field
            assert(s.size()); extractQuotedField(s, csvRec.thirdary);  // 3rd quoted substring
            assert(s.size()); extract5ints(s, csvRec.i5);              // handle 5 int fields

            csvRecVec.push_back(csvRec);  // capture

            if(ss.eof()) break;

         }while(1);
      }

   void show()
      {
         std::cout << std::endl;

         for (size_t i = 0; i < csvRecVec.size(); ++i)
            std::cout << std::setw(5) << i+1 << "   " << csvRecVec[i].show();

         std::cout << std::endl;
      }

private:

   void extractQuotedField(std::string& s, std::string& s2)
      {
         size_t indx1 = s.find('"', 0);
         assert(indx1 != std::string::npos);

         size_t indx2 = s.find('"', indx1+1);
         assert(indx2 != std::string::npos);

         size_t rng1 = indx2 - indx1 + 1;

         s2 = s.substr(indx1, rng1);

         s.erase(indx1, rng1+1);
      }

   void confirmEmptyField(std::string& s, std::string nullary)
      {
         size_t indx1 = s.find('"');

         nullary = s.substr(0, indx1);

         // tbd - confirm only spaces and comma's in this substr()

         s.erase(0, indx1); 
      }

   void extract5ints(std::string& s, IntVec_t& i5)
      {
         std::stringstream ss(s);

         int t = 0;
         for (int i=0; i<5; ++i)
         {
            ss >> t;
            ss.ignore(1); // skip ','
            assert(!ss.bad()); // confirm ok
            i5.push_back(t);
         }
         s.erase(0, std::string::npos);
      }

};  // class CSV_t



int t288(void) // test 288
{
   std::stringstream ss;
   ss << "\"Primary, Secondary, Third\", \"Primary\", , \"Secondary\", 18, 4, 0, 0, 0\n"
      << "\"Pramiry, Secandory, Thrid\", \"Pramiry\", , \"Secandory\", 19, 5, 1, 1, 1\n"
      << "\"Pri-mary, Sec-ondary, Trd\", \"Pri-mary\", , \"Sec-ondary\", 20, 6, 2, 3, 4\n"
      << std::endl;

   CSV_t csv;

   csv.init(ss);

   csv.show(); // results

   return (0);
}

Answer 3

Using std::quoted allows you to read quoted strings from input streams.

#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>

int main() {
    std::stringstream ss;
    ss << "\"Primary, Secondary, Third\", \"Primary\", , \"Secondary\", 18, 4, 0, 0, 0";

    while (ss >> std::ws) {
        std::string csvElement;

        if (ss.peek() == '"') {
            ss >> std::quoted(csvElement);
            std::string discard;
            std::getline(ss, discard, ',');
        }
        else {
            std::getline(ss, csvElement, ',');
        }

        std::cout << csvElement << "\n";
    }
}

Live Example

The caveat is that quoted strings are only extracted if the first non-whitespace character of a value is a double-quote. Additionally, any characters after the quoted strings will be discarded up until the next comma.