Character with encoding UTF8 has no equivalent in

Question

I am getting the following exception:

Caused by: org.postgresql.util.PSQLException: ERROR: character 0xefbfbd of encoding "UTF8" has no equivalent in "WIN1252"

Is there a way to eradicate such characters, either via SQL or programmatically?
(SQL solution should be preferred).

I was thinking of connecting to the DB using WIN1252, but it will give the same problem.

Answer 1

This problem appeared for us around 19/11/2016 with our old Access 97 app accessing a postgresql 9.1 DB.

This was solved by changing the driver to UNICODE instead of ANSI (see plang comment).

Answer 2

That looks like the byte sequence 0xBD, 0xBF, 0xEF as a little-endian integer. This is the UTF8-encoded form of the Unicode byte-order-mark (BOM) character 0xFEFF.

I'm not sure what Postgre's normal behaviour is, but the BOM is normally used only for encoding detection at the beginning of an input stream, and is usually not returned as part of the result.

In any case, your exception is due to this code point not having a mapping in the Win1252 code page. This will occur with most other non-Latin characters too, such as those used in Asian scripts.

Can you change the database encoding to be UTF8 instead of 1252? This will allow your columns to contain almost any character.