How to create an IPFS compatible multihash

问题:

I'm trying to create an IPFS compatible mutihash but it is not matching. I am asking here because I have not yet found an example that takes this from hashing to the end result.

echo -n multihash > multihash.txt


ipfs add multihash.txt
added QmZLXzjiZU39eN8QirMZ2CGXjMLiuEkQriRu7a7FeSB4fg multihash.txt


sha256sum multihash.txt
9cbc07c3f991725836a3aa2a581ca2029198aa420b9d99bc0e131d9f3e2cbe47  multihash.txt

node

> var bs58=require('bs58')
bs58.encode(new Buffer('9cbc07c3f991725836a3aa2a581ca2029198aa420b9d99bc0e131d9f3e2cbe47','hex'))
'BYptxaTgpcBrqZx9tghNCWFfUuYBcGfLydEvDjXqBV7k'

> var mh=require('multihashes')
mh.toB58String(mh.encode(new Buffer('9cbc07c3f991725836a3aa2a581ca2029198aa420b9d99bc0e131d9f3e2cbe47','hex'), 'sha2-256'))
'QmYtUc4iTCbbfVSDNKvtQqrfyezPPnFvE33wFmutw9PBBk'

The intent is to re-create the IPFS path QmZLXzjiZU39eN8QirMZ2CGXjMLiuEkQriRu7a7FeSB4fg using the multihashes package.

I'm able to create the same hash QmYtUc...9PBBk as shown in the example here: https://github.com/multiformats/multihash#example

回答1:

IPFS uses multihash where the format is the following:

base58(<varint hash function code><varint digest size in bytes><hash function output>)

The list of hash function codes can be found in this table.

Here's some pseudocode of the process using SHA2-256 as the hashing function.

sha2-256   size  sha2-256("hello world")
0x12       0x20  0xb94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Concatenating those three items will produce

1220b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Which then you encode it to base58

QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4

---

Here's an example of how to essentially implement multihash in JavaScript:

const crypto = require('crypto')
const bs58 = require('bs58')

const data = 'hello world'

const hashFunction = Buffer.from('12', 'hex') // 0x20

const digest = crypto.createHash('sha256').update(data).digest()

console.log(digest.toString('hex')) // b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

const digestSize = Buffer.from(digest.byteLength.toString(16), 'hex')

console.log(digestSize.toString('hex')) // 20

const combined = Buffer.concat([hashFunction, digestSize, digest])

console.log(combined.toString('hex')) // 1220b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

const multihash = bs58.encode(combined)

console.log(multihash.toString()) // QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4

---

There's a CLI you can use to generate multihashes:

$ go get github.com/multiformats/go-multihash/multihash
$ echo -n "hello world" | multihash -a sha2-256
QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4

---

As @David stated, A file in IPFS is "transformed" into a Unixfs "file", which is a representation of files in a DAG. So when you use add to upload a file to IPFS, the data has metadata wrapper which will give you a different result when you multihash it.

For example:

$ echo -n "hello world" | ipfs add -Q
Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD

---

Here's an example in Node.js of how to generate the exact same multihash as ipfs add:

const Unixfs = require('ipfs-unixfs')
const {DAGNode} = require('ipld-dag-pb')

const data = Buffer.from('hello world', 'ascii')
const unixFs = new Unixfs('file', data)

DAGNode.create(unixFs.marshal(), (err, dagNode) => {
  if (err) return console.error(err)

  console.log(dagNode.toJSON().multihash) // Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD
})

---

Hope this helps

回答2:

A file in IPFS is 'transformed' into a Unixfs file, which is a representation of files in a DAG, in your example, you are hashing directly your multihash.txt with sha2-256, but what happens inside IPFS is:

• file gets chunked into 256KiB pieces
• each chunk goes into a DAG node inside a Unixfs protobuf https://github.com/ipfs/js-ipfs-unixfs
• a dag is created with links to all the chunks.

回答3:

const { randomBytes } = require('crypto')
const multihash = require('multihashes')

const buffer = Buffer.from(randomBytes(32), 'hex')
const encoded = multihash.encode(buffer, 'sha2-256')
const hash = multihash.toB58String(encoded)
console.log(hash)

回答4:

The simplest way currently is to simply use ipfs-http-client. All above previous solutions in this thread don't work for me anymore.

import ipfsClient from "ipfs-http-client";
const ipfs = ipfsClient(IPFS_PROVIDER, "5001", { protocol: "https" });

// onlyHash: true only generates the hash and doesn't upload the file
const hash = (await ipfs.add(new Buffer(vuln), { onlyHash: true }))[0].hash

回答5:

Some of the other answers are outdated. This is what worked for me:

    const Unixfs = require('ipfs-unixfs'); // @0.1.16
    const { DAGNode } = require('ipld-dag-pb'); // @0.18.1
    const bs58 = require('bs58'); // @4.0.1

    // ...

    const data = Buffer.from('hello world');
    const unixFs = new Unixfs('file', data)
    const dagNode = new DAGNode(unixFs.marshal());
    const link = await dagNode.toDAGLink();
    const multihash = bs58.encode(link._cid.multihash).toString();
    console.log(multihash); // Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD