how exactly blocks are created in hyperledger fabr

Question

I am going through hyperledger fabric V1.0. Need to know the internals of hyper-ledger.

1) How and where the blocks and transactions are saved in hyperledger internally.

2) Is it in the NoSQL DB (Level DB, Couch DB) ??. If yes, what is the structure, in the way it is getting saved ??.

3) Where can I find the end to end setup doc for V1.0 (which includes CA, fabric, orderer setup. Currently I am following the link, which is getting updated.

https://hyperledger-fabric.readthedocs.io/

Answer 1

The BatchSize and BatchTimeout are the defining criteria for creation of blocks by the Ordering service. The BatchTimeout is pretty straightforward, but BatchSize, however, has three subparameters - MaxMessageCount, AbsoluteMaxBytes and PreferredMaxBytes. So, if the BatchTimeout is set to a pretty high value, BatchSize would be instrumental in deciding when a block will get created. If the messages are really not that big, a new block gets created whenever the number of messages become equal to MaxMessageCount first while the cumulative block size <= PreferredMaxBytes or in the other case, where cumulative size hits the max bytes threshold, the block will end up having arbitrary number of messages such that cumulative size doesn't cross the PreferredMaxBytes. On the other hand, if the transaction size is big (>PreferredMaxBytes), then a block will get created with fewer no. of transaction with a restriction of the cumulative size do not cross AbsoluteMaxBytes.

Answer 2

For all the code references below, use the fabric git repo as your guide. (S/O prevents me from posting more than 2 links, so I'm unable to link to the code fragments directly.)

Transactions are collected into blocks/batches on the ordering service first. Blocks are cut either when the BatchSize is met, or when BatchTimeout elapses (provided a non-empty block).

Refer to:

1. configtx.yaml in the common/configtx/tool/ directory for more info on the block-cutting criteria.
2. The Block type definition in protos/common/common.proto.

These blocks are stored locally to disk on every ordering service node along with a LevelDB to index these blocks by number -- see orderer/ledger/file.

These blocks are then delivered (via the ordering service's Deliver RPC) to committing peers. They store them locally, and maintain a LevelDB-based block index (similar to the one for the orderers), a LevelDB-based history index to track the history of each key on the blockchain, and a state database that maintains the latest values for all the keys on the blockchain. This state DB can be either LevelDB or CouchDB-based. See the ledger doc for more info.

PS: Question #3 is irrelevant and should be broken off to a separate thread.

Answer 3

Kostas answer for one and two are phenomenal. For question 3, the documentation has come along pretty far since March and the best setup I have found is here https://github.com/hyperledger/fabric/blob/master/docs/source/getting_started.rst It gives explicit instructions on switching to couchdb for the endorsers, a good explanation of the configuration files and tools and the script for the chaincode deployments serves as a great scaffolding to add your own deployments to. Another hidden gem in there is the docker-compose-e2e-template file which includes the certificates authorities. Pay close attention to the fact you need to edit that file, not the docker-compose-e2e.yaml file to be included when you do a./network_setup up. Also, if you look at the top of ./network_setup.sh it has the reference to the "copied" docker-compose-e2e.yaml that is commented.