Read a file/URL line-by-line in Swift

Question

I am trying to read a file given in an NSURL and load it into an array, with items separated by a newline character \n.

Here is the way I've done it so far:

var possList: NSString? = NSString.stringWithContentsOfURL(filePath.URL) as? NSString
if var list = possList {
    list = list.componentsSeparatedByString("\n") as NSString[]
    return list
}
else {
    //return empty list
}

I'm not very happy with this for a couple of reasons. One, I'm working with files that range from a few kilobytes to hundreds of MB in size. As you can imagine, working with strings this large is slow and unwieldy. Secondly, this freezes up the UI when it's executing--again, not good.

I've looked into running this code in a separate thread, but I've been having trouble with that, and besides, it still doesn't solve the problem of dealing with huge strings.

What I'd like to do is something along the lines of the following pseudocode:

var aStreamReader = new StreamReader(from_file_or_url)
while aStreamReader.hasNextLine == true {
    currentline = aStreamReader.nextLine()
    list.addItem(currentline)
}

How would I accomplish this in Swift?

A few notes about the files I'm reading from: All files consist of short (<255 chars) strings separated by either \n or \r\n. The length of the files range from ~100 lines to over 50 million lines. They may contain European characters, and/or characters with accents.

Answer 1

This function takes a file stream and returns an AnyGenerator that returns every line of the file:

func lineGenerator(file:UnsafeMutablePointer<FILE>) -> AnyGenerator<String>
{
  return AnyGenerator { () -> String? in
    var line:UnsafeMutablePointer<CChar> = nil
    var linecap:Int = 0
    defer { free(line) }
    return getline(&line, &linecap, file) > 0 ? String.fromCString(line) : nil
  }
}

So for instance, here's how you would use it to print every line of a file named "foo" in your app bundle:

let path = NSBundle.mainBundle().pathForResource("foo", ofType: nil)!
let file = fopen(path,"r") // open the file stream
for line in lineGenerator(file) {
  // suppress print's automatically inserted line ending, since
  // lineGenerator captures each line's own new line character.
  print(line, separator: "", terminator: "")
}
fclose(file) // cleanup the file stream

I developed this answer by modifying Alex Brown's answer to remove a memory leak mentioned by Martin R's comment, and by updating it to work with Swift 2.2 (Xcode 7.3).

Answer 2

It turns out good old-fasioned C API is pretty comfortable in Swift once you grok UnsafePointer. Here is a simple cat that reads from stdin and prints to stdout line-by-line. You don't even need Foundation. Darwin suffices:

import Darwin
let bufsize = 4096
// let stdin = fdopen(STDIN_FILENO, "r") it is now predefined in Darwin
var buf = UnsafePointer<Int8>.alloc(bufsize)
while fgets(buf, Int32(bufsize-1), stdin) {
    print(String.fromCString(CString(buf)))
}
buf.destroy()

Answer 3

Try this answer, or read the Mac OS Stream Programming Guide.

You may find that performance will actually be better using the stringWithContentsOfURL, though, as it will be quicker to work with memory-based (or memory-mapped) data than disc-based data.

Executing it on another thread is well documented, also, for example here.

Update

If you don't want to read it all at once, and you don't want to use NSStreams, then you'll probably have to use C-level file I/O. There are many reasons not to do this - blocking, character encoding, handling I/O errors, speed to name but a few - this is what the Foundation libraries are for. I've sketched a simple answer below that just deals with ACSII data:

class StreamReader {

    var eofReached = false
    let fileHandle: UnsafePointer<FILE>

    init (path: String) {
        self.fileHandle = fopen(path.bridgeToObjectiveC().UTF8String, "rb".bridgeToObjectiveC().UTF8String)
    }

    deinit {
        fclose(self.fileHandle)
    }

    func nextLine() -> String {
        var nextChar: UInt8 = 0
        var stringSoFar = ""
        var eolReached = false
        while (self.eofReached == false) && (eolReached == false) {
            if fread(&nextChar, 1, 1, self.fileHandle) == 1 {
                switch nextChar & 0xFF {
                case 13, 10 : // CR, LF
                    eolReached = true
                case 0...127 : // Keep it in ASCII
                    stringSoFar += NSString(bytes:&nextChar, length:1, encoding: NSASCIIStringEncoding)
                default :
                    stringSoFar += "<\(nextChar)>"
                }
            } else { // EOF or error
                self.eofReached = true
            }
        }
        return stringSoFar
    }
}

// OP's original request follows:
var aStreamReader = StreamReader(path: "~/Desktop/Test.text".stringByStandardizingPath)

while aStreamReader.eofReached == false { // Changed property name for more accurate meaning
    let currentline = aStreamReader.nextLine()
    //list.addItem(currentline)
    println(currentline)
}