Write and read a file with utf-8 encoding

Question

I've been reading up on all the UTF-8 related questions and blog posts, and I've got the following example in a test.rb file:

# encoding: UTF-8
File.open("test.txt", "w") do |f|
  f.write "test © foo"
end

File.open("test.txt", "r") do |f|
  puts f.read
end

this works perfectly. is produces the © symbol correctly in the file, and it reads the © back to me and prints it on the screen.

but when I use this same code in my actual project, i get this written to the file instead of the © symbol: \u00A9

FWIW: I'm getting this result when running an rspec (v1.2.9) test against my code. the spec produces a file with a © symbol in it, and then reads the file back in to check the contents.

I'm running this in Ruby 1.9.2 at the moment, but I also need to support all the way back to Ruby 1.8.6. This is a Windows environment with RubyInstaller.org versions of Ruby.

Answer 1

On which OS does your application run? It could be that the default encoding for the file is ASCII. Does it help if you add w:utf-8 and r:utf-8 to the open parameters?

Answer 2

Read the file with less code:

# encoding: UTF-8
file_content = File.open("test.txt", "r:UTF-8", &:read)

Answer 3

If i execute your code i get an error on the special character. Can you try this code ?

# encoding: UTF-8
File.open("test.txt", "w:UTF-8") do |f| 
  f.write "test \u00A9 foo" 
end 

#Encoding.filesystem = "UTF-8"
p Encoding.find("filesystem") 
File.open("test.txt", "r:UTF-8") do |f| 
  puts f.read 
end 

On my windows box i then get

#<Encoding:Windows-1252>
test © foo

I have no idea why the  is there..