Swift vs Objective C pointer manipulation issue

I have this code in Objective C which works fine:

list = controller->audioBufferList;
list->mBuffers[0].mDataByteSize = inNumberFrames*kSampleWordSize;
list->mBuffers[1].mDataByteSize = inNumberFrames*kSampleWordSize;

And it works fantastic, it updates mDataByteSize field of mBuffers[0] & mBuffers[1]. I tried translating the same in Swift but it doesn't work:

public var audioBufferList:UnsafeMutableAudioBufferListPointer

In function,

let listPtr = controller.audioBufferList.unsafeMutablePointer

let buffers = UnsafeBufferPointer<AudioBuffer>(start: &listPtr.pointee.mBuffers, count: Int(listPtr.pointee.mNumberBuffers))

for var buf in buffers {
    buf.mDataByteSize = inNumberFrames * UInt32(sampleWordSize)
    NSLog("Data byte size \(buf.mDataByteSize)")
}

for buf in buffers {
    NSLog("Data byte size \(buf.mDataByteSize)")
}

The mDataByteSize is not updated. The NSLog on reading back in second for loop points to original values, not updated ones. It seems var buf is referring to another buf by making a copy. How do I fix it? It's a pure Swift language issue that I am not able to understand.

EDIT: As pointed out by Martin, I fixed the issue by modifying the for loop as

 for i in 0..<Int(listPtr.pointee.mNumberBuffers) {
    buffers[i].mDataByteSize = inNumberFrames * UInt32(sampleWordSize)
}

Now it works. But that has further aroused my curiosity in Swift Language, how non intuitive it is and how irritating it is for developers who use pointers to manipulate stuff. Why do the following loops fail? Are the var buffers copy by value?

     for buf in buffers {
         var buffer = buf
         buffer.mDataByteSize = inNumberFrames * UInt32(sampleWordSize)
     }

    for var buf in buffers {
        buf.mDataByteSize = inNumberFrames * UInt32(sampleWordSize)
    }

EDIT 2: Hamish's answer raises doubts about validity of using listPtr anywhere. I was using listPtr in a number of calls, such as:

let status = AudioUnitRender(controller.audioUnit!, ioActionFlags, inTimeStamp, inBusNumber, inNumberFrames, listPtr)

I now need to know where can we use listPtr and where we can not!

For the call:

let buffers = UnsafeBufferPointer<AudioBuffer>(start: &listPtr.pointee.mBuffers, count: Int(listPtr.pointee.mNumberBuffers))

&listPtr.pointee.mBuffers produces a temporary pointer valid only for the duration of the call to UnsafeBufferPointer's initialiser. Therefore attempting to use the buffer pointer results in undefined behaviour (the compiler will hopefully warn on such cases in Swift 5.1).

Instead, you can iterate directly over UnsafeMutableAudioBufferListPointer, as it conforms to MutableCollection.

For example:

for index in audioBufferList.indices {
  audioBufferList[index].mDataByteSize = inNumberFrames * UInt32(sampleWordSize)
  print("Data byte size \(audioBufferList[index].mDataByteSize)")
}

for buffer in audioBufferList {
  print("Data byte size \(buffer.mDataByteSize)")
}