Is there a way in ruby 1.9 to remove invalid byte

2019-01-17 14:57发布

Suppose you have a string like "€foo\xA0", encoded UTF-8, Is there a way to remove invalid byte sequences from this string? ( so you get "€foo" )

In ruby-1.8 you could use Iconv.iconv('UTF-8//IGNORE', 'UTF-8', "€foo\xA0") but that is now deprecated. "€foo\xA0".encode('UTF-8') doesn't do anything, since it is already UTF-8. I tried:

"€foo\xA0".force_encoding('BINARY').encode('UTF-8', :undef => :replace, :replace => '')

which yields

"foo"

But that also loses the valid multibyte character €

4条回答
冷血范
2楼-- · 2019-01-17 15:29

Ruby 2.0 and 1.9.3

"€foo\xA0".encode(Encoding::UTF_8, Encoding::UTF_8, :invalid => :replace)

Ruby 2.1+

"€foo\xA0".scrub
查看更多
【Aperson】
3楼-- · 2019-01-17 15:35
"€foo\xA0".chars.select(&:valid_encoding?).join
查看更多
孤傲高冷的网名
4楼-- · 2019-01-17 15:41
"€foo\xA0".encode('UTF-16le', invalid: :replace, replace: '').encode('UTF-8')
查看更多
女痞
5楼-- · 2019-01-17 15:49
    data = '' if not (data.force_encoding("UTF-8").valid_encoding?)
查看更多
登录 后发表回答