Getting “incompatible character encodings: UTF-8 a

A user saves text in a form with the word "café," (note the "é"). When loading the #show page displaying this text, I get an Encoding::CompatibilityError with message incompatible character encodings: UTF-8 and ASCII-8BIT.

Error can duplicated in the console:

> r = Report.find(123) # load the record
> r.update(:comments => "café")
 => true 
> r.comments
 => "caf\xC3\xA9" 
> r.comments.encoding
 => #<Encoding:ASCII-8BIT> 
> r.comments.encode("UTF-8")
Encoding::UndefinedConversionError: "\xC3" from ASCII-8BIT to UTF-8
from (irb):11:in `encode'

Displaying the field on the #show page (<%= r.comments %>) produces the error.

That the encoding is not UTF-8, but ASCII-8BIT, seems wrong since the database column encoding is set to UTF-8, and a number of application configurations define encoding as "UTF-8."

Ideally, the "é" gets displayed the way it was typed in without any errors, but at this point, I'd settle for just a way to avoid the error. It seems all of my forms (dozens) are susceptible to this.

Is there a way to allow ASCII-8BIT characters to display?

Forcing encoding (<%= string.force_encoding("UTF-8") %>) produces the desired result, avoiding an error and displaying characters correctly, which led to the following workaround:

WORKAROUND

# /config/initializers/active_record_extension_force_encode_read_attribute.rb
module ActiveRecordExtensionForceEncodeReadAttribute

  def read_attribute(attr_name)
    attribute = super
    attribute.is_a?(String) ? attribute.force_encoding("UTF-8") : attribute
  end

end

# include the extension
ActiveRecord::Base.send(:include, ActiveRecordExtensionForceEncodeReadAttribute)

This feels wrong for several reasons, but works.

My setup

I'm using the mysql2 gem. The database columns are all UTF-8, TEXT datatype. I've followed suggestions to declare UTF-8 the encoding for my app (via How to fix ActionView::Template::Error (incompatible character encodings: ASCII-8BIT and UTF-8)).

Parameters look like this

Parameters: {"utf8"=>"✓", "authenticity_token"=>"my_auth_token=", "commit"=>"Save", "report"=>{"comments"=>"café "}}

The error is Encoding::CompatibilityError in WeekendManagerReports#show (the view) with message: incompatible character encodings: UTF-8 and ASCII-8BIT and gets marked in the view where I print the attribute:

<%= @report.comments %>

Error stack trace:

vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/core_ext/string/output_safety.rb:180:in `concat'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/core_ext/string/output_safety.rb:180:in `concat'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/buffers.rb:12:in `<<'
app/views/weekend_manager_reports/_show_fields.html.erb:68:in `_app_views_weekend_manager_reports__show_fields_html_erb___2906173005587511585_48245020'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:145:in `block in render'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:161:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:339:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:143:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/partial_renderer.rb:306:in `render_partial'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/partial_renderer.rb:279:in `block in render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/abstract_renderer.rb:38:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/partial_renderer.rb:278:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/renderer.rb:47:in `render_partial'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/helpers/rendering_helper.rb:35:in `render'
app/views/weekend_manager_reports/_show.html.erb:1:in `_app_views_weekend_manager_reports__show_html_erb__1009321416738415989_48308580'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:145:in `block in render'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:161:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:339:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:143:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/partial_renderer.rb:306:in `render_partial'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/partial_renderer.rb:279:in `block in render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/abstract_renderer.rb:38:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/partial_renderer.rb:278:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/renderer.rb:47:in `render_partial'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/helpers/rendering_helper.rb:35:in `render'
app/views/reports/show.html.erb:56:in `_app_views_reports_show_html_erb__2577918850424361413_47390860'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:145:in `block in render'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:161:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:339:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/template.rb:143:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/template_renderer.rb:55:in `block (2 levels) in render_template'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/abstract_renderer.rb:38:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/template_renderer.rb:54:in `block in render_template'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/template_renderer.rb:62:in `render_with_layout'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/template_renderer.rb:53:in `render_template'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/template_renderer.rb:17:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/renderer.rb:42:in `render_template'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/renderer/renderer.rb:23:in `render'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/rendering.rb:99:in `_render_template'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/streaming.rb:217:in `_render_template'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/rendering.rb:82:in `render_to_body'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/rendering.rb:32:in `render_to_body'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/renderers.rb:32:in `render_to_body'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/abstract_controller/rendering.rb:25:in `render'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/rendering.rb:16:in `render'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/instrumentation.rb:41:in `block (2 levels) in render'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/core_ext/benchmark.rb:12:in `block in ms'
/home/vhc3248/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/benchmark.rb:303:in `realtime'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/core_ext/benchmark.rb:12:in `ms'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/instrumentation.rb:41:in `block in render'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/instrumentation.rb:84:in `cleanup_view_runtime'
vendor/bundle/ruby/2.2.0/gems/activerecord-4.1.9/lib/active_record/railties/controller_runtime.rb:25:in `cleanup_view_runtime'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/instrumentation.rb:40:in `render'
vendor/bundle/ruby/2.2.0/bundler/gems/wicked_pdf-8f970f29c4de/lib/wicked_pdf/pdf_helper.rb:22:in `render_with_wicked_pdf'
app/controllers/concerns/common_report_controller_functions.rb:88:in `block (2 levels) in show'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/mime_responds.rb:258:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/mime_responds.rb:258:in `respond_to'
app/controllers/concerns/common_report_controller_functions.rb:86:in `show'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/abstract_controller/base.rb:189:in `process_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/rendering.rb:10:in `process_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:113:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:113:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:229:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:229:in `block in halting'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `block in halting'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `block in halting'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:149:in `block in halting_and_conditional'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:229:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:229:in `block in halting'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `block in halting'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:166:in `block in halting'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:86:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:86:in `run_callbacks'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/abstract_controller/callbacks.rb:19:in `process_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/rescue.rb:29:in `process_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `block in instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/notifications.rb:159:in `instrument'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
vendor/bundle/ruby/2.2.0/gems/activerecord-4.1.9/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/abstract_controller/base.rb:136:in `process'
vendor/bundle/ruby/2.2.0/gems/actionview-4.1.9/lib/action_view/rendering.rb:30:in `process'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal.rb:196:in `dispatch'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_controller/metal.rb:232:in `block in action'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/routing/route_set.rb:82:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/routing/route_set.rb:50:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/journey/router.rb:73:in `block in call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/journey/router.rb:59:in `each'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/journey/router.rb:59:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/routing/route_set.rb:685:in `call'
vendor/bundle/ruby/2.2.0/gems/exception_notification-4.0.1/lib/exception_notification/rack.rb:28:in `call'
vendor/bundle/ruby/2.2.0/gems/warden-1.2.3/lib/warden/manager.rb:35:in `block in call'
vendor/bundle/ruby/2.2.0/gems/warden-1.2.3/lib/warden/manager.rb:34:in `catch'
vendor/bundle/ruby/2.2.0/gems/warden-1.2.3/lib/warden/manager.rb:34:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/etag.rb:23:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/conditionalget.rb:25:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/head.rb:11:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/flash.rb:254:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/session/abstract/id.rb:225:in `context'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/session/abstract/id.rb:220:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/cookies.rb:562:in `call'
vendor/bundle/ruby/2.2.0/gems/activerecord-4.1.9/lib/active_record/query_cache.rb:36:in `call'
vendor/bundle/ruby/2.2.0/gems/activerecord-4.1.9/lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/callbacks.rb:82:in `run_callbacks'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
vendor/bundle/ruby/2.2.0/gems/railties-4.1.9/lib/rails/rack/logger.rb:38:in `call_app'
vendor/bundle/ruby/2.2.0/gems/railties-4.1.9/lib/rails/rack/logger.rb:22:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/request_id.rb:21:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/methodoverride.rb:21:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/runtime.rb:17:in `call'
vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.9/lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.9/lib/action_dispatch/middleware/static.rb:84:in `call'
vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/sendfile.rb:112:in `call'
vendor/bundle/ruby/2.2.0/gems/railties-4.1.9/lib/rails/engine.rb:514:in `call'
vendor/bundle/ruby/2.2.0/gems/railties-4.1.9/lib/rails/application.rb:144:in `call'

You can display the text on the show page with: