Suppose my Azure role stores files to the role local filesystem and forgets to delete them. Can another application that uses that host in future possibly get access to those files?
I've read the whitepaper and it's full of marketing style statements but I can't find a definitive statement about how thoroughly the host machine is "cleaned" before a new role is started on it.
Can I be completely sure that another application won't see changes my application does to the filesystem?
No, it cannot be used by another VM, even accidentally. The 'scratch' drive that is mounted by your Windows Azure instance is another VHD - data is not written natively to disk. So, in order for another instance to read your data, it would have to mount your VHD, which is not possible.
Local storage is ephemeral and it is part of specific compute instance. When the instance is recycled\deleted, the local storage is gone.
Although I am not sure it's properly documented, I am fairly certain that Windows Azure recycles thoroughly whatever local files haven been dumped by a Role into a VM before letting any other Role access the VM. That's a fairly basic security policy.
Then, considering the deployment time on Windows Azure for a new VM (between 6min to 8min) I am pretty sure that the VM itself is pretty much wiped-out clean, not just the local storage, so that whatever by-product files which might have been generated by the OS itself during the execution of the VM does not become available afterward.