unknown bytes is returned by method getBytes()

Question

import java.io.UnsupportedEncodingException;
import java.util.Arrays;

public class Main {
 public static void main(String[] args)
 {
  try 
  {
   String s = "s";
   System.out.println( Arrays.toString( s.getBytes("utf8") ) );
   System.out.println( Arrays.toString( s.getBytes("utf16") ) );
   System.out.println( Arrays.toString( s.getBytes("utf32") ) );
  }  
  catch (UnsupportedEncodingException e) 
  {
   e.printStackTrace();
  }
 }
}

Console:

[115]
[-2, -1, 0, 115]
[0, 0, 0, 115]

What is it?

[-2, -1] - ???

Also, i noted, that if i do that:

String s = new String(new char[]{'\u1251'});
System.out.println( Arrays.toString( s.getBytes("utf8") ) );
System.out.println( Arrays.toString( s.getBytes("utf16") ) );
System.out.println( Arrays.toString( s.getBytes("utf32") ) );

Console:

[-31, -119, -111]
[-2, -1, 18, 81]
[0, 0, 18, 81]

Answer 1

The -2, -1 is a Byte Order Mark (BOM - U+FEFF) that indcates that the following text is encoded in UTF-16 format.

You are probably getting this because, while there is only one UTF8 and UTF32 encoding, there are two UTF16 encodings UTF16LE and UTF16BE, where the 2 bytes in the 16-bit value are stored in Big-Endian or Little Endian format.

As the values that come back are 0xFE xFF, this suggests that the encoding is UTF16BE

Answer 2

The mysterious -2, -1 is a UTF-16 Byte Order Mark (BOM). The other negative values are simply bytes. In Java, the byte type is signed, and ranges from -128 to +127.

Answer 3

Don't forget that bytes are unsigned in Java. So -2, -1 really means 0xfe 0xff... and U+FEFF is the Unicode byte order mark (BOM)... that's what you're seeing here in the UTF-16 version.

To avoid getting the BOM when encoding, use UTF-16BE or UTF-16LE explicitly. (I would also suggest using the names which are guaranteed by the platform rather than just "utf8" etc. Admittedly the name is guaranteed to be found case-insensitively, but the lack of a hyphen makes it less reliable, and there are no downsides to using the canonical name.)

Answer 4

A byte in java is a signed type, so it is entirely normal for it to have negative values.