read file(not utf-8) line by line?

Question

Is it possible to read file line by line, if it is not in utf-8 encoding with std::io::File and std::io::BufReader?

I look at std::io::Lines and it return Result<String>, so I worry, have I implement my own BufReader that do the same, but return Vec<u8> instead, or I can reuse std::io::BufReader in some way?

Answer 1

You do not have to re-implement BufReader itself, it provides exactly the method you need for your usecase read_until:

fn read_until(&mut self, byte: u8, buf: &mut Vec<u8>) -> Result<usize>

You supply your own Vec<u8> and the content of the file will be appended until byte is encountered (0x0A being LF).

There are several potential gotchas:

• the buffer may end not only with a LF byte, but with a CR LF sequence,
• it is up to you to clear buf between subsequent calls.

A simple while let Ok(_) = reader.read_until(0x0A as u8, buffer) should let you read your file easily enough.

You may consider implement a std::io::Lines equivalent which converts from the encoding to UTF-8 to provide a nice API, though it will have a performance cost.