JSON Invalid UTF-8 middle byte

问题:

This error happens when the (Jackson, this case) JSON engine tries to parse some JSON that is not encoded in UTF-8.

How to tell the engine that it should expect something different from UTF-8, such as UTF-16?

HttpHeaders requestHeaders = createSomeHeader();
RestTemplate restTemplate = new RestTemplate();
HttpEntity<?> requestEntity = new HttpEntity<Object>(requestHeaders);
String url = "someurl"
ResponseEntity<MyObject[]> arrayResponseEntity = restTemplate.exchange(url, HttpMethod.GET, requestEntity, MyObject[].class);

error log:

Caused by: org.springframework.http.converter.HttpMessageNotReadableException: Could not read JSON: Invalid UTF-8 middle byte 0x20
at [Source: org.apache.http.conn.EofSensorInputStream@44d397b0; line: 92, column: 42]; nested exception is org.codehaus.jackson.JsonParseException: Invalid UTF-8 middle byte 0x20
at [Source: org.apache.http.conn.EofSensorInputStream@44d397b0; line: 92, column: 42]
at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.readInternal(MappingJacksonHttpMessageConverter.java:138)
at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:154)
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:74)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:622)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:608)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:449)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:404)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:380)
... 4 more
Caused by: org.codehaus.jackson.JsonParseException: Invalid UTF-8 middle byte 0x20
at [Source: org.apache.http.conn.EofSensorInputStream@44d397b0; line: 92, column: 42]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1213)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:375)
at org.codehaus.jackson.impl.Utf8StreamParser._reportInvalidOther(Utf8StreamParser.java:2132)
at org.codehaus.jackson.impl.Utf8StreamParser._reportInvalidOther(Utf8StreamParser.java:2139)
at org.codehaus.jackson.impl.Utf8StreamParser._decodeUtf8_3fast(Utf8StreamParser.java:1962)

回答1:

JSON data must be encoded as UTF-8, UTF-16 or UTF-32. The JSON decoder can determine the encoding by examining the first four octets of the byte stream:

       00 00 00 xx  UTF-32BE
       00 xx 00 xx  UTF-16BE
       xx 00 00 00  UTF-32LE
       xx 00 xx 00  UTF-16LE
       xx xx xx xx  UTF-8

It sounds like the server is encoding data in some illegal encoding (ISO-8859-1, windows-1252, etc.)

回答2:

I got this exception when in the Java Client Application I was serializing a JSON like this

String json = mapper.writeValueAsString(contentBean);

and on the Server Side I was using Spring Boot as REST Endpoint. Exception was:

nested exception is com.fasterxml.jackson.databind.JsonMappingException: Invalid UTF-8 start byte 0xaa

My problem was, that I was not setting the correct encoding on the HTTP Client. This solved my problem:

updateRequest.setHeader("Content-Type", "application/json;charset=UTF-8");
StringEntity entity= new StringEntity(json, "UTF-8");
updateRequest.setEntity(entity);

Android set content type HttpPost

回答3:

I got this after saving the JSON file using Notepad2, so I had to open it with Notepad++ and then say "Convert to UTF-8". Then it worked.

回答4:

I had this problem inconsistently between different platforms, as I got JSON as String from Mapper and did the writing myself. Sometimes it went into file as ansi and other times correctly as UTF8. I switched to

mapper.writeValue(file, data);

letting Mapper do the file operations, and it started working fine.

回答5:

On the off chance it may help others I'll share a related anecdote.

I encountered this exact error (Invalid UTF-8 middle byte 0x3f) running a PowerShell script via the PowerShell Integrated Script Environment (ISE). The identical script, executed outside the ISE, works fine. The code uses the Confluence v3 and v5.x REST APIs and this error is logged on the Confluence v5.x server - presumably because the ISE somehow mucks with the request.