Node.js data parsing raw bytes with Buffer

Question

I'm trying to use Buffer to parse 29 bytes of data that is formatted in odd ways. I've been using the slice() method to carve up the data on these odd boundaries. A sample stream looks like the following in hex (spaces added for clarity)...

01 1d 00 00 01 0a 0a 0b 0b 0c 0c 00 00 04 d2 00 00 00 0e c8 00 00 00 00 00 00 00 cc c4

var raw = '011d0000010a0a0b0b0c0c000004d20000000ec800000000000000ccc4';
buff = new Buffer(raw, 'utf8');
var position = 2;

// message type
var msg_type = buff.slice(position,(position+1)).toString();
position += 1;
console.log('... message type is ' + msg_type);

// event type
var event_type = buff.slice(position,(position+1)).toString();
position += 1;
console.log('... event type is ' + event_type);

// grab more data...

This produces an msg_type=1 and event_type=0. It should be 0 and 1, respectively. All of the slices that follow are wrong.

I'm clearly misunderstanding something with the encoding here. Either during Buffer instantiation or in the toString() extraction.

How can I treat this input stream as a series of hex strings and convert them into binary data that I can operate on?

Thanks!

Answer 1

I think the answer is that your Buffer doesn't represent the object you think it does:

> var raw = '01 02 03'
> buff = new Buffer(raw, 'utf8')
<Buffer 30 31 20 30 32 20 30 33>
> buff.slice(0,0)
<Buffer >
> buff.slice(0,1)
<Buffer 30>
> buff.slice(0,2)
<Buffer 30 31>
> buff.slice(0,3)
<Buffer 30 31 20>
> buff.slice(0,0).toString()
''
> buff.slice(0,1).toString()
'0'
> buff.slice(0,2).toString()
'01'
> buff.slice(0,3).toString()
'01 '
> buff.slice(0,4).toString()
'01 0'
> buff.slice(0,5).toString()
'01 02'
> buff.slice(0,6).toString()
'01 02 '
> buff.slice(0,7).toString()
'01 02 0'
> buff.slice(0,8).toString()
'01 02 03'
> buff.length
8

Instead of representing a buffer that is three bytes long, this represents a buffer that is 8 bytes long. As you slice into it, you're getting the individual hexadecimal values of the characters. (Note the space is 20 -- just like the %20 so ubiquitous in URLs.)

But I have a feeling your var = '01 1d 00...' is just an attempt to populate the buffer with some binary data rather than what's actually happening in your program -- it might be easier to work with a simplified version of how you're actually filling the buffer. Maybe read it out of a file?

Answer 2

Node's Buffer supports this directly with the hex encoding:

var raw = '011d0000010a0a0b0b0c0c000004d20000000ec800000000000000ccc4';
var buff = new Buffer(raw, 'hex');