What encoding to use when interpreting HTTP/1.1 he

Question

In HTTP/1.1 specs I get this when it comes to define headers:

message-header = field-name ":" [ field-value ]

[...]

field-value = *( field-content | LWS )

field-contet = <the OCTETs making up the field-value and consisting of either *TEXT or combinations of token, separators, and quoted-string>

and the definition for OCTET and TEXT is:

OCTET = <any 8-bit sequence of data>

TEXT = <any OCTET except CTLs, but including LWS> ; where CTL refers to control characters from US-ASCII charset.

Question: Now, when it comes to header names (called field-names in definition), the encoding used is US-ASCII (specified in HTTP/1.1 specs), but how would a server application know what encoding to use for header values?

Note: I think it's normal to be US-ASCII encoded, but the definition lets enough room for different situation.

Answer 1

The semantics of non-ASCII code points is essentially undefined. Avoid them.

Recipients usually decode using ISO-8859-1, which at least allows recovery later on (because it'll preserve all octets).

(Also: you're looking at the wrong spec; RFC 2616 is obsoleted by RFC 7230)