How do I get Rust's openssl crate to decrypt d

Question

I'm decrypting legacy data created by a Ruby on Rails application using the symmetric-encryption gem in Rust. See my earlier question How do I decrypt data encrypted by Ruby's `symmetric-encryption` gem in another language?.

I've already implemented this in Node where the crypto library seems to know how to strip out gibberish in an encrypted string that Rust's openssl crate isn't stripping out (at least, the way I'm using it). I have already stripped out the PKCS7 padding and a header, yet it's still there. What is that gobbledygook, and how do I get Rust to remove it?

The encrypted data began as plaintext with a fixed-size header, was encrypted using AES-128-CBC with PKCS7 padding, and was then Base64 encoded. Using Node, I can decrypt it correctly using the following code:

const crypto = require("crypto");

const KEY = Buffer.from("1234567890ABCDEF");
const IV = Buffer.from("1234567890ABCDEF");
const CIPHERTEXT = Buffer.from("QEVuQwBAEACuPUPByDkk5jyNzQ3Wd3xTy2Isihz62XTLe1M5qKQrvw==", "base64");
const HEADER_SIZE = 8;
const ALGO = "aes-128-cbc";

const decipher = crypto.createDecipheriv(ALGO, KEY, IV);

decipher.update(CIPHERTEXT.slice(HEADER_SIZE));
const result = decipher.final();

console.log([...result]);
console.log(result.toString());

The result is

[ 72, 97, 108, 102 ]
Half

I'd prefer to use Rust for the application I'm writing. Using the openssl crate, I can decode the encrypted data, but there's a bunch of junk that Node's library knows how to strip but Rust isn't stripping automatically the way I'm using it:

extern crate base64;
extern crate openssl;

use openssl::symm::*;

const KEY: &'static [u8] = b"1234567890ABCDEF";
const IV: &'static [u8] = b"1234567890ABCDEF";
const CIPHERTEXT: &'static str = "QEVuQwBAEACuPUPByDkk5jyNzQ3Wd3xTy2Isihz62XTLe1M5qKQrvw==";
const HEADER_SIZE: usize = 8;

fn main() {
    let decoded = base64::decode(&CIPHERTEXT).unwrap();
    let ciphertext = &decoded[HEADER_SIZE..];

    let result = decrypt(Cipher::aes_128_cbc(), KEY, Some(IV), ciphertext).unwrap();
    println!("{:?}", result);
    println!("{:?}", String::from_utf8_lossy(&result));
}

Here the result is

[221, 75, 14, 215, 54, 120, 246, 222, 194, 208, 53, 68, 127, 190, 124, 8, 72, 97, 108, 102]
"�K\u{e}�6x����5D\u{7f}�|\u{8}Half"

You can see that the last four bytes are correct, but that Node stripped out the preceding 16 bytes, as it appears it should have. I don't know what those bytes are.

At first I thought the 16 bytes of gibberish were from attempting to decrypt the PKCS7 padding. But I can verify that the padding was already stripped out: if I create a Crypter following the example in the openssl crate's documentation, but don't include the truncate() step, the resulting Vec has 12 consecutive 12s—the PKCS7 padding—that were automatically removed.

So the gibberish is not PKCS7 padding, but I don't know what it is or how to get rid of it using Rust.

And to anticipate the Miranda warning about not futzing with crypto if I'm not an expert: this is not going to be used in production or to create data to put into production.

Answer 1

Your Node decryption is incorrect and it only works because you are ignoring exactly the correct number of bytes before the data begins, as sfackler explains.

symmetric-encryption pads the data with a header. The header has flexible length based on the options you are using. This code properly parses the entire header and then uses the IV embedded in the header:

extern crate base64;
extern crate openssl;
extern crate byteorder;

use openssl::symm::*;
use byteorder::{LittleEndian, ReadBytesExt};

use std::io::{self, Cursor, BufRead};

const KEY: &'static [u8] = b"1234567890ABCDEF";
//const IV: &'static [u8] = b"1234567890ABCDEF";
const CIPHERTEXT: &'static str = "QEVuQwBAEACuPUPByDkk5jyNzQ3Wd3xTy2Isihz62XTLe1M5qKQrvw==";

const FLAG_COMPRESSED: u8  = 0b1000_0000;
const FLAG_IV: u8          = 0b0100_0000;
const FLAG_KEY: u8         = 0b0010_0000;
const FLAG_CIPHER_NAME: u8 = 0b0001_0000;
const FLAG_AUTH_TAG: u8    = 0b0000_1000;

#[derive(Debug)]
struct EncryptedData<'a> {
    version: u8,
    compressed: bool,
    iv: Option<&'a [u8]>,
    key: Option<&'a [u8]>,
    cipher_name: Option<&'a [u8]>,
    auth_tag: Option<&'a [u8]>,
    data: &'a [u8],
}

impl<'a> EncryptedData<'a> {
    fn decode(decoded: &'a [u8]) -> Result<Self, io::Error> {
        let mut cursor = Cursor::new(decoded);

        let magic = {
            let raw = cursor.get_ref();
            let current = cursor.position() as usize;
            &raw[current..][..4]
        };
        cursor.consume(4);
        assert_eq!(b"@EnC", magic);

        let version = cursor.read_u8()?;
        let flags = cursor.read_u8()?;
        let compressed = flags & FLAG_COMPRESSED != 0;

        fn maybe_read_string<'b>(cursor: &mut Cursor<&'b [u8]>, flag: bool) -> Result<Option<&'b [u8]>, io::Error> {
            if flag {
                let len = cursor.read_u16::<LittleEndian>()? as usize;
                let data = {
                    let raw = cursor.get_ref();
                    let current = cursor.position() as usize;
                    Some(&raw[current..][..len])
                };
                cursor.consume(len);
                Ok(data)
            } else {
                Ok(None)
            }
        }

        let iv = maybe_read_string(&mut cursor, flags & FLAG_IV != 0)?;
        let key = maybe_read_string(&mut cursor, flags & FLAG_KEY != 0)?;
        let cipher_name = maybe_read_string(&mut cursor, flags & FLAG_CIPHER_NAME != 0)?;
        let auth_tag = maybe_read_string(&mut cursor, flags & FLAG_AUTH_TAG != 0)?;
        let data = {
            let raw = cursor.get_ref();
            let current = cursor.position() as usize;
            &raw[current..]
        };

        Ok(EncryptedData {
            version, compressed, iv, key, cipher_name, auth_tag, data,
        })
    }
}

fn main() {
    let decoded = base64::decode(&CIPHERTEXT).unwrap();
    let data = EncryptedData::decode(&decoded).unwrap();

    match decrypt(Cipher::aes_128_cbc(), KEY, data.iv, &data.data) {
        Ok(result) => {
            println!("{:?}", result);
            println!("{:?}", String::from_utf8_lossy(&result));
        }
        Err(e) => {
            println!("{}", e);
        }
    }
}

Answer 2

decipher.update returns a Buffer which is ignored in the Node implementation. If you change the Node implementation to print the buffer returned from the update call, you can see that it's the same garbage data:

const buffer = decipher.update(CIPHERTEXT.slice(HEADER_SIZE));
console.log(buffer.toString());

Outputs:

�K�6x����5D�|

If you change the Rust version to ignore that portion of the output as well, you see the expected output:

extern crate base64;
extern crate openssl;

use openssl::symm::*;

const KEY: &'static [u8] = b"1234567890ABCDEF";
const IV: &'static [u8] = b"1234567890ABCDEF";
const CIPHERTEXT: &'static str = "QEVuQwBAEACuPUPByDkk5jyNzQ3Wd3xTy2Isihz62XTLe1M5qKQrvw==";
const HEADER_SIZE: usize = 8;

fn main() {
    let decoded = base64::decode(&CIPHERTEXT).unwrap();
    let ciphertext = &decoded[HEADER_SIZE..];

    let t = Cipher::aes_128_cbc();
    let mut d = Crypter::new(t, Mode::Decrypt, KEY, Some(IV)).unwrap();
    let mut result = vec![0; CIPHERTEXT.len() + t.block_size()];
    d.update(&ciphertext, &mut result).unwrap();
    let len = d.finalize(&mut result).unwrap();
    result.truncate(len);
    println!("{:?}", result);
    println!("{:?}", String::from_utf8_lossy(&result));
}

Outputs:

[72, 97, 108, 102]
"Half"