Is the ios iPhone simulator causing memory usage a

I am trying to process a large text file in my app. I know that I want to be careful with the amount of memory being consumed while I read the data. Once a piece of data is read the app does not need to keep the data around.

Thanks to “Martin R” and the post Read a file/URL line-by-line for helping me jump start my effort.

I am trying to monitor the memory consumption of my app as it reads in the large data file so that I can be sure it is behaving as expected. Here’s where I am running into a problem.

When I run Instruments using Command-I from within Xcode and I monitor allocations I see that during the read of the file the app peeks at ~15MB and then drops back down. This is fairly repeatable +/- 0.5MB.

When I run the app using Command-R from within Xcode and then let it finish reading through the file, and then press record within Instruments, the memory consumption now swells to ~360MB.

So to clarify, the two ways I have done measurement of memory allocations are:
Profile:
1. Xcode Command-I.
2. Instruments Record Allocations. Observe ~15MB
Simulate and Profile:
1. Xcode Command-R.
2. Let app run to “IDLE”.
3. Instruments Record. Observe ~360MB.

I have been trying to figure out a few things here.
Q1. Why the difference? (This may answer all my questions)

Q2. Do I have a real problem or is this only a problem because of how debug code is annotated on to the simulator?

Q3. Similar to Q2, if I run a debug build on a real device, will it have the same issue?

Q4. For my app, ~15MB is acceptable when parsing the file, but ~360MB will not be. Is there another way I can continue to debug on my device without taking this 360MB hit?

Version 8.1 (8B62)
Sierra
2.7Ghz i5
MacBook Pro Circa 2015

Sample Code attached. The first part of the file is merely a copy of the code from the referenced post for reader convenience. One can take this code as is and run it in Xcode. At the bottom is the ViewController ViewDidLoad() method where things "run". The memory “swell” is after “File opened”.

//
//

import UIKit

/* Originally from 
 * stackoverflow:
 * https://stackoverflow.com/questions/24581517/read-a-file-url-line-by-line-in-swift 
 * posted by Martin R.
 * Much thanks!
*/
class StreamReader  {

  let encoding : String.Encoding
  let chunkSize : Int
  var fileHandle : FileHandle!
  let delimData : Data
  var buffer : Data
  var atEof : Bool

  init?(path: String, delimiter: String = "\n", encoding: String.Encoding = .utf8,
        chunkSize: Int = 4096) {

    guard let fileHandle = FileHandle(forReadingAtPath: path),
      let delimData = delimiter.data(using: encoding) else {
        return nil
    }
    self.encoding = encoding
    self.chunkSize = chunkSize
    self.fileHandle = fileHandle
    self.delimData = delimData
    self.buffer = Data(capacity: chunkSize)
    self.atEof = false
  }

  deinit {
    self.close()
  }

  /// Return next line, or nil on EOF.
  func nextLine() -> String? {
    precondition(fileHandle != nil, "Attempt to read from closed file")

    // Read data chunks from file until a line delimiter is found:
    while !atEof {
      if let range = buffer.range(of: delimData) {
        // Convert complete line (excluding the delimiter) to a string:
        let line = String(data: buffer.subdata(in: 0..<range.lowerBound), encoding: encoding)
        // Remove line (and the delimiter) from the buffer:
        buffer.removeSubrange(0..<range.upperBound)
        return line
      }
      let tmpData = fileHandle.readData(ofLength: chunkSize)
      if tmpData.count > 0 {
        buffer.append(tmpData)
      } else {
        // EOF or read error.
        atEof = true
        if buffer.count > 0 {
          // Buffer contains last line in file (not terminated by delimiter).
          let line = String(data: buffer as Data, encoding: encoding)
          buffer.count = 0
          return line
        }
      }
    }
    return nil
  }

  /// Start reading from the beginning of file.
  func rewind() -> Void {
    fileHandle.seek(toFileOffset: 0)
    buffer.count = 0
    atEof = false
  }

  /// Close the underlying file. No reading must be done after calling this method.
  func close() -> Void {
    fileHandle?.closeFile()
    fileHandle = nil
  }
}

extension StreamReader : Sequence {
  func makeIterator() -> AnyIterator<String> {
    return AnyIterator {
      return self.nextLine()
    }
  }
}



class ViewController: UIViewController {

  override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view, typically from a nib.

    let path2WordList = Bundle.main.path(forResource: "large_text_file", ofType: "txt")
    var wordCnt: Int = 0

    if nil != path2WordList {
      if let aStreamReader = StreamReader(path: path2WordList!) {
        defer {  aStreamReader.close() }
        print("File openned")

        /* Read and discard */
        while aStreamReader.nextLine() != nil {
          wordCnt += 1
        }

      } // if let ...
    } // if nil ...

    print ("Final wordCnt := \(wordCnt)")
  } // viewDidLoad


  override func didReceiveMemoryWarning() {
    super.didReceiveMemoryWarning()
    // Dispose of any resources that can be recreated.
  }


}

I've encountered problems like this when using long running while loops. The problem is that anything that is allocated into the current autorelease pool won't get deallocated until the loop exits.

To guard against this, you can wrap the contents of your while loop in autoreleasepool(invoking:). This will cause each iteration of your loop to have its own autorelease pool that is drained each time.

It would look something like this:

/// Return next line, or nil on EOF.
func nextLine() -> String? {
  precondition(fileHandle != nil, "Attempt to read from closed file")

  var result: String? = nil

  // Read data chunks from file until a line delimiter is found:
  while !atEof, result == nil {
    result = autoreleasepool {
      if let range = buffer.range(of: delimData) {
        // Convert complete line (excluding the delimiter) to a string:
        let line = String(data: buffer.subdata(in: 0..<range.lowerBound), encoding: encoding)
        // Remove line (and the delimiter) from the buffer:
        buffer.removeSubrange(0..<range.upperBound)
        return line
      }
      let tmpData = fileHandle.readData(ofLength: chunkSize)
      if tmpData.count > 0 {
        buffer.append(tmpData)
      } else {
        // EOF or read error.
        atEof = true
        if buffer.count > 0 {
          // Buffer contains last line in file (not terminated by delimiter).
          let line = String(data: buffer as Data, encoding: encoding)
          buffer.count = 0
          return line
        }
      }
      return nil
    }
  }
  return result
}

As to whether your memory growth is a side effect of the debug environment, it's hard to say. But it would probably be wise to guard against this kind of growth regardless.