Hedera Technical Insights: Hedera Hashgraph Transaction and Query Processing Model
1516208829798
Apr 28, 2020
by Paul Madsen
Head of Identity, The HBAR Foundation

Hedera Hashgraph is a public ledger. Nodes on the network collectively establish and maintain a consensus state for a set of application data.

Clients interact with that state through API calls sent to the nodes of the network.

To write to state, clients send transactions to a chosen node for subsequent gossip to the rest of the network, assignment of a consensus timestamp and a place in consensus order, and then application to state.

To read from state, clients send queries to a chosen node which, by looking at its local copy of state, can respond with the requested information.

Transaction sequence

Transactions cause the consensus state maintained by the mainnet nodes to change.

All transactions have an associated fee that reflects the burden it places on the mainnet nodes.

Below is the typical sequence for a transaction submitted to Hedera:

  1. A client constructs a transaction by:
    • Specifying the appropriate parameters, for instance, from and to which accounts hbars should be transferred, the file to be stored, etc
    • Specifying a node
    • Stipulating a maximum fee that it will pay
    • Applying appropriate signatures
  2. Client submits signed transaction to the specified mainnet node
  3. The chosen mainnet node:
    • Confirms the client can pay the transaction fee
    • Submits the transaction to the network
  4. All mainnet nodes receive the transaction via gossip
  5. All mainnet nodes use the hashgraph algorithm to establish a consensus timestamp and order for that transaction
  6. All mainnet nodes:
    • Reject duplicate transactions (but still charge a small fee)
    • Validate transactions against relevant application rules
    • Apply transactions to state in the consensus order. For an hbar transfer this would mean all nodes simply incrementing the balance of the receiving account and decrementing the sender’s balance
    • Process the fee associated with each transaction – allocating the different portions of the transaction fee between the submitting node and the Hedera treasury
    • Create and store receipts and records for transactions
  7. To obtain confirmation of a transaction or its result, clients can:
    • Query any node to confirm that the transaction was successfully assigned a consensus timestamp and applied to state by:
      • Retrieving the transaction receipt (free)
      • Retrieving the transaction record
      • Querying the corresponding aspect of state
    • Retrieve from a mirror node either by:
      • An API query
      • gRPC subscription

Query sequence

Queries do not require consensus processing and do not impact state.

Some queries are free, but most require that the client pay the node that responds to the query a fee for that service.

Below is the typical sequence for a query submitted to Hedera:

  1. Clients construct a query by:
    • Specifying the appropriate parameters, for instance, the account whose balance is being queried
    • Creating a CryptoTransfer payment transaction. This transaction will serve to compensate the node that responds to the query. Set the value of the transfer to be sufficient for the particular query
    • Signing the payment transaction
    • Embedding the payment transaction within the query
  2. Clients submit query to a mainnet node
  3. The mainnet node will calculate the fee for the query based on its specifics, e.g. how large is the file returned, how much gas was used to run a smart contract locally, etc
  4. The mainnet node will compare the calculated fee to the value of the embedded payment transaction:
    • If the payment is less than the calculated fee, reject the transaction
    • If the payment is equal or greater than the calculated fee:
      • Return the requested data
      • Submit the payment transaction to the rest of the network (as for any transaction)

Variants and optimizations

The above sequences for transactions and queries have certain variants and optimizations.

State proofs

When querying a node for some aspect of state or the status of a transaction, a client will be able to stipulate that it desires a state proof for the response be included. A state proof is a cryptographic assertion from the network as a whole as to the veracity of the response. Consequently, even if the response comes from a single particular node, the client can be confident that it does indeed accurately reflect the true Hedera state at a given moment in time.

When implemented, state proofs can be created by main or a mirror node.

Once obtained by a client, a state proof can be archived in case of future arbitration.

State proofs will impose an additional fee so clients may choose to request a state proof only occasionally.

A large component of the size of a state proof (and so the contributor to the associated fee) will be due to the address book history. An optimization is possible where, if the requesting client already has the latest address book, that address book need not be included with the returned state proof - making it smaller and cheaper.

Consensus Service

In the Hedera Consensus Service (HCS) model, the nodes of the mainnet do not apply the transaction to a copy of state they maintain. Instead, the transactions logically flow through the mainnet and, in so doing, are assigned a consensus timestamp and order in the corresponding topic. It is the nodes of some other application network (appnet) that, based on those timestamps, process the transaction and apply it to their copy of appnet state with application-specific rules.

The HCS model separates the roles of calculating consensus timestamps and order from that of validating transactions and maintaining appnet state and consequently offers advantages in performance and data confidentiality.

Choosing a node for transaction submission

When submitting a transaction, a client can choose which node to use and so avoid malicious or slow nodes.

Unless there is some sort of relationship between a client and a particular node, a client should pick a node randomly.

A client can also protect itself from a node censoring its transactions or a node being offline by sending different versions (but with the same transaction identifier) of a transaction to multiple nodes. Even if both transactions are assigned a consensus timestamp by the network, only the first will be applied to state. The second will still have an associated fee, though smaller than the fee of the first.

If, after sending a transaction to a node and not being able to retrieve a record in some small period of time, a client could resend the transaction to another node, and keep trying until success. Even if almost 1/3 of nodes are maliciously not submitting the client’s transactions, then it would take on average x = 2/3(1) + 1/3(x+1) which gives x = 1.5 tries to succeed.

If the client maintains a list in memory of which nodes have successfully submitted its transactions in the past, or a list of those that have not, then the client can avoid malicious nodes. Such nodes will lose node fees if clients are avoiding it.

A client might occasionally try submitting to a node on the black list and thereby determine if the previous failed transaction submission was malicious or merely that the node was offline at the time.

Querying multiple nodes

If unsure about the honesty of a particular node, a client can choose to not query that node when asking about some aspect of state.

If unsure about the honesty of all nodes (as a general policy), a client can query multiple nodes and compare their answers.

State proofs provide an alternative mechanism by which clients can reassure themselves of the veracity of a node’s responses to queries.

Both models will have associated additional fees and a client should use those fees to determine the appropriate choice.

Cost answer query

To help it stipulate an appropriate value for the payment for a query, a Client can first ask a node what would be the cost of a particular query, if actually sent. This sort of query is free to the client.

Subscribing to a mirror

Mirrors are a separate network from the mainnet. Mirrors listen to the mainnet and provide value-add services that, if provided by the mainnet nodes, might impact their performance towards consensus. The fundamental role of mirrors is to facilitate retrieval and analysis of transaction history and the subsequent state.

Mirrors are separate business and will make different decisions about the services they wish to offer. For instance, a particular mirror node might choose to persist all transactions from genesis and build a business around making that history available. Another mirror might choose to store only crypto transfer transactions. A given enterprise might run its own mirror and store only transactions and state relevant to its business.

A mirror node may make its Hedera data available to external clients. It will be through the mirror that those clients would be able to examine the transaction history and corresponding state of some aspect of Hedera (whatever portion the mirror choose to support).

Mirror nodes can support a subscription mechanism by which clients can be sent any transaction records that meet certain criteria. For instance, a client could stipulate to a mirror that it wants to be notified of any and all transactions that impact a particular crypto account.

HCS appnets are a particularly important category of mirror clients. Members of an appnet will subscribe to a mirror (or perhaps multiple mirrors) to be notified of transactions submitted to a particular HCS topic.