Code golf - hex to (raw) binary conversion

Question

In response to this question asking about hex to (raw) binary conversion, a comment suggested that it could be solved in "5-10 lines of C, or any other language."

I'm sure that for (some) scripting languages that could be achieved, and would like to see how. Can we prove that comment true, for C, too?

NB: this doesn't mean hex to ASCII binary - specifically the output should be a raw octet stream corresponding to the input ASCII hex. Also, the input parser should skip/ignore white space.

edit (by Brian Campbell) May I propose the following rules, for consistency? Feel free to edit or delete these if you don't think these are helpful, but I think that since there has been some discussion of how certain cases should work, some clarification would be helpful.

1. The program must read from stdin and write to stdout (we could also allow reading from and writing to files passed in on the command line, but I can't imagine that would be shorter in any language than stdin and stdout)
2. The program must use only packages included with your base, standard language distribution. In the case of C/C++, this means their respective standard libraries, and not POSIX.
3. The program must compile or run without any special options passed to the compiler or interpreter (so, 'gcc myprog.c' or 'python myprog.py' or 'ruby myprog.rb' are OK, while 'ruby -rscanf myprog.rb' is not allowed; requiring/importing modules counts against your character count).
4. The program should read integer bytes represented by pairs of adjacent hexadecimal digits (upper, lower, or mixed case), optionally separated by whitespace, and write the corresponding bytes to output. Each pair of hexadecimal digits is written with most significant nibble first.
5. The behavior of the program on invalid input (characters besides [a-fA-F \t\r\n], spaces separating the two characters in an individual byte, an odd number of hex digits in the input) is undefined; any behavior (other than actively damaging the user's computer or something) on bad input is acceptable (throwing an error, stopping output, ignoring bad characters, treating a single character as the value of one byte, are all OK)
6. The program may write no additional bytes to output.
7. Code is scored by fewest total bytes in the source file. (Or, if we wanted to be more true to the original challenge, the score would be based on lowest number of lines of code; I would impose an 80 character limit per line in that case, since otherwise you'd get a bunch of ties for 1 line).

Answer 1

EDIT: This code was written a long time before the question edit which fleshed out the requirements.

Given that a single line of C can contain a huge number of statements, it's almost certainly true without being useful.

In C# I'd almost certainly write it in more than 10 lines, even though it would be feasible in 10. I'd separate out the "parse nybble" part from the "convert a string to a byte array" part.

Of course, if you don't care about spotting incorrect lengths etc, it becomes a bit easier. Your original text also contained spaces - should those be skipped, validated, etc? Are they part of the required input format?

I rather suspect that the comment was made without consideration as to what a pleasant, readable solution would look like.

Having said that, here's a hideous version in C#. For bonus points, it uses LINQ completely inappropriately in an effort to save a line or two of code. The lines could be longer, of course...

using System;
using System.Linq;

public class Test
{
    static void Main(string[] args)
    {
        byte[] data = ParseHex(args[0]);
        Console.WriteLine(BitConverter.ToString(data));

    }

    static byte[] ParseHex(string text)
    {
        Func<char, int> parseNybble = c => (c >= '0' && c <= '9') ? c-'0' : char.ToLower(c)-'a'+10;
        return Enumerable.Range(0, text.Length/2)
            .Select(x => (byte) ((parseNybble(text[x*2]) << 4) | parseNybble(text[x*2+1])))
            .ToArray();
    }
}

(This is avoiding "cheating" by using any built-in hex parsing code, such as Convert.ToByte(string, 16). Aside from anything else, that would mean losing the use of the word nybble, which is always a bonus.)

Answer 2

PHP, 28 symbols:

<?=pack(I,hexdec($argv[1]));

Answer 3

Gah.

You aren't allowed to call me on my off-the-cuff estimates! ;-P

Here's a 9 line C version with no odd formatting (Well, I'll grant you that the tohex array would be better split into 16 lines so you can see which character codes map to which values...), and only 2 shortcuts that I wouldn't deploy in anything other than a one-off script:

#include <stdio.h>
char hextonum[256] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 0, 0, 0, 0, 0, 0,10,11,12,13,14,15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,10,11,12,13,14,15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
char input[81]="8b1f0008023149f60300f1f375f40c72f77508507676720c560d75f002e5ce000861130200000000";
void main(void){
   int i = 0;
   FILE *fd = fopen("outfile.bin", "wb");
   while((input[i] != 0) && (input[i+1] != 0))
      fputc(hextonum[input[i++]] * 16 + hextonum[input[i++]], fd);
}

No combined lines (each statement is given its own line), it's perfectly readable, etc. An obfuscated version could undoubtedly be shorter, one could cheat and put the close braces on the same line as the preceding statement, etc, etc, etc.

The two things I don't like about it is that I don't have a close(fd) in there, and main shouldn't be void and should return an int. Arguably they're not needed - the OS will release every resource the program used, the file will close without any problems, and the compiler will take care of the program exit value. Given that it's a one-time use script, it's acceptable, but don't deploy this.

It becomes eleven lines with both, so it's not a huge increase anyway, and a ten line version would include one or the other depending on which one might feel is the lessor of two evils.

It doesn't do any error checking, and it doesn't allow whitespace - assuming, again, that it's a one time program then it's faster to do search/replace and get rid of spaces and other whitespace before running the script, however it shouldn't need more than another few lines to eat whitespace as well.

There are, of course, ways to make it shorter but they would likely decrease readability significantly...

Hmph. Just read the comment about line length, so here's a newer version with an uglier hextonum macro, rather than the array:

#include <stdio.h>
#define hextonum(x) (((x)<'A')?((x)-'0'):(((x)<'a')?((x)+10-'A'):((x)+10-'a')))
char input[81]="8b1f0008023149f60300f1f375f40c72f77508507676720c560d75f002e5ce000861130200000000";
void main(void){
   int i = 0;
   FILE *fd = fopen("outfile.bin", "wb");
   for(i=0;(input[i] != 0) && (input[i+1] != 0);i+=2)
      fputc(hextonum(input[i]) * 16 + hextonum(input[i+1]), fd);
}

It isn't horribly unreadable, but I know many people have issues with the ternary operator, but the appropriate naming of the macro and some analysis should readily yield how it works to the average C programmer. Due to side effects in the macro I had to move to a for loop so I didn't have to have another line for i+=2 (hextonum(i++) will increment i by 5 each time it's called, macro side effects are not for the faint of heart!).

Also, the input parser should skip/ignore white space.

grumble, grumble, grumble.

I had to add a few lines to take care of this requirement, now up to 14 lines for a reasonably formatted version. It will ignore everything that's not a hexadecimal character:

#include <stdio.h>
int hextonum[] = {-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,1,2,3,4,5,6,7,8,9,-1,-1,-1,-1,-1,-1,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1};
char input[]="8b1f 0008 0231 49f6 0300 f1f3 75f4 0c72 f775 0850 7676 720c 560d 75f0 02e5 ce00 0861 1302 0000 0000";
void main(void){
   unsigned char i = 0, nibble = 1, byte = 0;
   FILE *fd = fopen("outfile.bin", "wb");
   for(i=0;input[i] != 0;i++){
      if(hextonum[input[i]] == -1)
         continue;
      byte = (byte << 4) + hextonum[input[i]];
      if((nibble ^= 0x01) == 0x01)
         fputc(byte, fd);
   }
}

I didn't bother with the 80 character line length because the input isn't even less than 80 characters, but a 3 level ternary macro could replace the first 256 entry array. If one didn't mind a bit of "alternative formatting" then the following 10 line version isn't completely unreadable:

#include <stdio.h>
int hextonum[] = {-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,1,2,3,4,5,6,7,8,9,-1,-1,-1,-1,-1,-1,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1};
char input[]="8b1f 0008 0231 49f6 0300 f1f3 75f4 0c72 f775 0850 7676 720c 560d 75f0 02e5 ce00 0861 1302 0000 0000";
void main(void){
   unsigned char i = 0, nibble = 1, byte = 0;
   FILE *fd = fopen("outfile.bin", "wb");
   for(i=0;input[i] != 0;i++){
      if(hextonum[input[i]] == -1) continue;
      byte = (byte << 4) + hextonum[input[i]];
      if((nibble ^= 0x01) == 0x01) fputc(byte, fd);}}

And, again, further obfuscation and bit twiddling could result in an even shorter example.

Answer 4

45 byte executable (base64 encoded):

6BQAitjoDwDA4AQI2LQCitDNIevrWMOy/7QGzSF09jLkBMAa5YDkByrEJA/D

(paste into a file with a .com extension)

EDIT: Ok, here's the code. Open a Window's console, create a file with 45 bytes called 'hex.com', type "debug hex.com" then 'a' and enter. Copy and paste these lines:

db e8,14,00,8a,d8,e8,0f,00,c0,e0,04,08,d8,b4,02,8a,d0,cd,21,eb,eb,cd,20
db b2,ff,b4,06,cd,21,74,f6,32,e4,04,c0,1a,e5,80,e4,07,2a,c4,24,0f,c3

Press enter, 'w' and then enter again, 'q' and enter. You can now run 'hex.com'

EDIT2: Made it two bytes smaller!

db e8, 11, 00, 8a, d8, e8, 0c, 00, b4, 02, 02, c0, 67, 8d, 14, c3
db cd, 21, eb, ec, ba, ff, 00, b4, 06, cd, 21, 74, 0c, 04, c0, 18
db ee, 80, e6, 07, 28, f0, 24, 0f, c3, cd, 20

That was tricky. I can't believe I spent time doing that.

Answer 5

Late to the game, but here's some Python{2,3} one-liner (100 chars, needs import sys, re):

sys.stdout.write(''.join([chr(int(x,16)) for x in re.findall(r'[A-Fa-f0-9]{2}', sys.stdin.read())]))

Answer 6

I can't code this off the top of my head, but for every two characters, output (byte)((AsciiValueChar1-(AsciiValueChar1>64?48:55)*16)+(AsciiValueChar1-(AsciiValueChar1>64?48:55))) to get a hex string changed into raw binary. This would break horribly if your input string has anything other than 0 to 9 or A to B, so I can't say how useful it would be to you.