CSVParser processes LF as CRLF

问题:

I am trying to parse a CSV file as below

String NEW_LINE_SEPARATOR = "\r\n"; CSVFormat csvFileFormat = CSVFormat.DEFAULT.withRecordSeparator(NEW_LINE_SEPARATOR); FileReader fr = new FileReader("201404051539.csv"); CSVParser csvParser = csvFileFormat.withHeader().parse(fr); List<CSVRecord> recordsList = csvParser.getRecords();

Now the file got normal lines ending with CRLF characters however for few lines there is additional LF character appearing in middle. i.e.

    a,b,c,dCRLF --line1
    e,fLF,g,h,iCRLF --line2

Due to this, the parse operation creates three records whereas actually they are only two.

Is there a way I can get the LF character appearing in middle of second line not treated as line break and get two records only upon parsing?

Thanks

回答1:

I think uniVocity-parsers is the only parser you will find that will work with line endings as you expect.

The equivalent code using univocity-parsers will be:

    CsvParserSettings settings = new CsvParserSettings(); //many options here, check the tutorial
    settings.getFormat().setLineSeparator("\r\n");
    settings.getFormat().setNormalizedNewline('\u0001'); //uses a special character to represent a new record instead of \n.
    settings.setNormalizeLineEndingsWithinQuotes(false); //does not replace \r\n by the normalized new line when reading quoted values.
    settings.setHeaderExtractionEnabled(true); //extract headers from file
    settings.trimValues(false); //does not remove whitespaces around values 
    CsvParser parser = new CsvParser(settings);

    List<Record> recordsList = parser.parseAllRecords(new File("201404051539.csv"));

If you define a line separator to be \r\n then this is the ONLY sequence of characters that should identify a new record (when outside quotes). All values can have either \r or \n without being enclosed in quotes because that's NOT the line separator sequence.

When parsing the input sample you gave:

String input = "a,b,c,d\r\ne,f\n,g,h,i\r\n";
parser.parseAll(new StringReader(input));

The result will be:

LINE1 = [a, b, c, d]
LINE2 = [e, f
, g, h, i]

Disclosure: I'm the author of this library. It's open-source and free (Apache 2.0 license)