How to avoid reading comments in text file?

Question

Currently I've successfully write to my file some random numbers between 0 to 10 with this code(below is just some sample code to demonstrate the problem):

for (int i = 1; i <= size; i++)
{
    type = rand () % 3;
    switch (type)
    {
    case 0:  afile << rand () % 10;
             break;
    case 1:  afile << rand () % 10;
             afile << "\t\t";
             afile << rand () % 10;
             break;
    case 2:  afile << rand () % 10;
             afile << "\t\t";
             afile << rand () % 10;
             afile << "\t\t";
             afile << rand () % 10;
             /*afile << "8";
             afile << "\t";
             afile << "7";
             afile << "\t";
             afile << "2";*/
    }

    afile << "\t\t" << "// Possible type " << i << endl;
}

then my afile look like this after executing the code:

8       // Possible type 1
1       7       // Possible type 2
4       0       3       // Possible type 3

The problem occur when I read the data from this file and output to another file:

int type;

while (afile >> type)
{
    if(type == 0) 
    {
        afile >> .........;
        ..........
    } 
        else if(type == 1) {.........}
        else if(type == 2) {.........}
    }       
}
...................................... 

My output file stop reading after the first line since it's also read the comment which is invalid data to process, if I delete the comment then everything is working fine. So how can I overcome this situation? Thank you.

Answer 1

Tony D already provided a reasonable answer as I see, but I thought I'd add a code of my own too, since I already wrote and tested it.

The below is pretty much self-explanatory for anybody speaking C++, it's basically what Tony proposed, but with a twist - getting the data line by line, making use of std::stringstream, but then also making use of the binary nature of the data OP uses. The data there is either a valid integer, or a comment. Or in other terms, either a valid integer, or not. So in the code below when a valid conversion of the data from the stream to integer can't be made - the rest of the line is treated as a comment. edit: ...actually, while it was a somewhat valid solution, I modified the code to incorporate a bit saner approach - one that skips a comment (denoted by # or // to show two ways to do it) but still lets us decide what to do on a malformed value. This does not allow for a 45fubar to pass as 45 and then bad fubar, which was a problem with the previous code, but allows for 45//comment to be interpreted correctly.

I still think just outright cutting out \/\/.*? is a better approach though. The point of this answer though is to be a bit different. ;)

#include <ctime>
#include <cmath>
#include <string>
#include <iostream>
#include <sstream>
#include <fstream>
#include <vector>

void write(std::ostream& output, int lines) {
    for (int i = 0; i < lines; i++) {  // for i lines
        int n = rand() % 10 + 1;       // generate n numbers per line
        for (int j = 0; j < n; j++) {  // loop over line
            output << rand() % 99;     // output a random number
            if (j + 1 < n) {           // if not last?
                output << "\t\t";      // then add tabs
            }
        }
        output << " // " << n << " numbers\n"; // I'm not using std::endl here because it actually *flushes* the stream - flushing every iteration isn't advisable
    }
}

std::vector<std::vector<int>> read(std::istream& input) {
    std::vector<std::vector<int>> v;           // a vector of vectors of ints
    std::string line;               
    while (std::getline(input, line)) {        // getline returns the stream by reference, so this handles EOF
        std::stringstream ss(line);            // create a stringstream out of line
        int n = 0;                          
        std::vector<int> numbers_in_line;
        while (ss) {                           // while the stream is good
            std::string word;
            if (ss >> word) {                  // if there's still data to get
                std::stringstream tester(word);
                tester >> n;
                if (tester && tester.peek() == std::char_traits<char>::eof()) { // conversion went well, no data was left in stream
                    numbers_in_line.push_back(n);   // push it to the vector
                } else {         // conversion didn't go well, or went well but data was left in the stream
                    bool conversion_went_well = tester.good();
                    tester.clear();
                    char c = tester.get();
                    if (c == '#' || (c == '/' && tester.peek() == '/')) { // if it's a comment
                        if (conversion_went_well) {
                            numbers_in_line.push_back(n);                 // push it to the vector
                        }
                        break;                                            // and ignore the rest of the line
                    } else {
                        std::cerr << "Unexpected symbol: '" << tester.str() << "'\n"; // report unexpected data
                        //  so how do we handle a malformed value?
                        //  error out? ignore following values in this line? accept following values in this line?
                        // if you leave it as is now - it will accept following values from this line
                    }

                } 
            }
        }
        v.push_back(numbers_in_line);       
    }

    return v;
}

int main(int argc, char** argv) {
    std::srand(std::time(nullptr));
    write(std::cout, 4);                                    // write random data
    std::vector<std::vector<int>> numbers = read(std::cin); // read the data

    for (std::vector<int> line: numbers) {  // loop over vector via C++11 features
        for (int n: line) {
            std::cerr << n << " ";
        }
        std::cerr << "\n";
    }

    return 0;
}

An example run:

$ ./test.exe < data > data
50 44 92 43 97
26 32 54
30 91
93 4
$ cat data
50      44      92      43      97 // 5 numbers
26      32      54 // 3 numbers
30      91 // 2 numbers
93      4 // 2 numbers


$ ./test.exe < data2 > dump
Unexpected symbol: 'i91'
Unexpected symbol: '4i'
Unexpected symbol: 'lol'
Unexpected symbol: 'numbers'
50 44 92 43 97
26 32 54
30
93 3 2
7337
7337
$ cat data2
50      44      92      43      97 // 5 numbers
26      32      54 # 3 numbers
30      i91 // 2 numbers
93      4i      lol 3 2 numbers
7337//test comment
7337#test comment 2

Answer 2

You've a couple reasonable options:

• read the entire line into a std::string, scan for and remove any comment, then create a std::istringstream from whatever's left and stream from that to extract the non-comment values

• before reading values, use >> std::ws and afile.peek() to see if the next character is a '/': if so skip until you reach a newline.

The former is a useful technique to get used to in C++ (helps when you want to report line numbers with data issues), and looks like this:

if (std::ifstream in(filename))
{
    std::string line;
    while (getline(in, line))
    {
        std::string::size_type n = line.find("//");
        if (n != std::string::npos)
            line.erase(n);

        std::istringstream iss(line);

        int atype;
        while (iss >> atype)
            ...etc...
    }

Answer 3

There are a couple of ways to do this.

Skipping the rest of a line after you find a quote (Faster)

Basically what you would do here, is read the file line by line in a loop. When you hit two characters "//". You'd call "break;" and skip to the next line.'

Some untested dummy code:

while(line = file.getLine()){
  loopChars = sizeof(line);
  for(x = 0; x < loopChars; x++) {
    char currentChar = line[x];
    if(x+1 < loopChars){
       char nextChar = line[x+1];
    } else {
       char nextChar = '';
    }

    if(nextChar == "/" && currentChar == "/"){
      // Go to next line 
      break; 
    } else {
      // Do your normal processing here
    }
  }
}

Removing Quotes First (slower)

Here is a solution for removing quotes (one liners "//" and multi-liners "/**/") from a file. Basically, you would run this against the file you are processing before you start reading it for the number data.

http://www.cplusplus.com/forum/beginner/80380/

#include <iostream>
#include <fstream>
using namespace std;

int main (){
    ifstream infile;
    string filename;
    ofstream outfile;
    char c1, c2;
    bool isInsideComment = false;

    cout << "Please input file name (to remove comments from): ";
    cin >> filename;

    infile.open (filename.c_str());
    if (infile.fail()) {
        cout << "nInvaild file name.n";
        return 1;
    }
    outfile.open(("out_"+filename).c_str());

    infile.get (c1);
    while (!infile.eof()) {
        if ((c1 == '/') && (!isInsideComment)) {
            infile.get (c2);
            if (c2 == '*')
            isInsideComment = true;
            else if ((c1 == '/') && (c2 == '/'))
            isInsideComment = true;
            else {
                outfile.put (c1);
                outfile.put (c2);
            }
        }

        else if ( (c1 == '*') && isInsideComment) {
            infile.get (c2);
            if (c2 == '/')
            isInsideComment = false;
            else if ((c1 == 'n') && isInsideComment)
            isInsideComment = false;
        }
        else if (!isInsideComment)
        outfile.put (c1);
        infile.get (c1);
    }
    infile.close();
    outfile.close();
}