Distributed ledger technologies (DLT) aren’t just built on code; they’re about community too. And while we might wish for universal agreement, any community worth the name will have its share of disagreement and dispute. The DLT community is a healthy one, and so we’ve received some good-faith inquiries about Hedera Hashgraph, its consensus mechanisms, and the manner that we ensure fairness to all network participants.

Recently, we have been engaged in a dialogue with a Medium blogger. You can read the latest response and feedback here.

The back-and-forth has afforded us the opportunity to better define and explain various concepts and features of the Hedera architecture. To continue in the spirit of transparency that we’ve provided ahead of Open Access on the 16th, we’re looking to address any questions or misconceptions relating to Hedera Hashgraph through an upcoming regular “chat with the co-founder” video series. We believe this series will address questions in a timely, consistent fashion for our community. These videos will serve as a repository for answers about Hedera. You can check them out on our YouTube channel shortly after OA.

We’d like to make one last contribution to this dialogue blog series before Open Access.

Delaying transactions

Every DLT, be it Bitcoin, Ethereum, or Hedera, processes transactions in three high level steps:

• A client (like a wallet software) hands their transactions to a node
• That node submits the transaction to the rest of the network
• The network places the transaction into a consensus order

For some use cases a small delay in the above sequence would likely not be critical – for example, digital purchase of software, or a micropayment purchasing a single play. For others, like stock markets and auctions, a delay could be extremely important and damaging to the application. For such use cases, a delayed transaction could mean a missed opportunity, or a higher price paid on a Black Friday deal.

There are two distinct opportunities in the above sequence for a particular transaction to be treated unfairly; that is, to be delayed and so receive a later place in the consensus sequence than would be the case if there were no bias.

The first opportunity for delay is that the node that the client chooses (in Step 1) can postpone sending the transaction to the rest of the network (in Step 2). The node might delay a given transaction to punish whichever client sent the transaction (perhaps it is a competitor), to favor some other client by moving their transaction forward (perhaps the node has a hidden relationship with that other client), or perhaps to themselves front-run the transaction in question.

Let’s call this first sort of delay a compromise of “fair access”. The particular node chosen by the client is preventing that client from fairly accessing the network by delaying the submission of that client’s transactions relative to the transactions of other clients. An analogy would be a race track official that prevents a particular race car from entering the track for a particular race, and forces them to wait until the next race to compete.

The second opportunity for a transaction to be delayed is during the process by which the transaction is assigned a place in consensus order (in Step 3). Continuing the analogy, this would be like a race car competing and crossing the finish line first, but being officially recorded as coming in second place.

Let’s call this second sort of delay a compromise of “fair order”; that is, the consensus algorithm itself is being inappropriately manipulated or skewed to treat the transaction as if it arrived to all the nodes later than when it actually arrived. The consensus order doesn’t reflect reality.

A given transaction could receive fair access but suffer unfair order. Or vice versa. Both access and order could be fair. Or neither could be.

Fair access

All DLTs (that allow for the possibility of non-code clients) must confront the risk of a node inappropriately delaying Step 2. In other words, every DLTs faces the risk of a bad node compromising fair access. There is an inherent potential for abuse in the client-node relationship, since the chosen node serves as the client’s proxy to the rest of the network.

For violating fair access, if a malicious node wants to delete a transaction and ignore it, then that is easy to do. Or it can delay sending the transaction in many different ways. It can send it in a later group of transactions that it gossips out (a Hedera “event”), or send it later within a single group, or send the group itself later. In all those cases, the effect is the same: the transaction is transmitted to the network later than it should have been. So the transaction is received later than it should have been. So the transaction will end up later in the consensus order in which all the transactions were received. This is a lack of fair access.

Clients of all DLTs have in theory the same five choices for protecting themselves against this risk:

• Run a node. Completely cut out a middleman proxy to the network by acting as your own
• Submit a transaction to a node run by people you know and trust
• Submit a transaction to a node that is run by anonymous people, but has provided fair access for you in the past
• Submit a transaction to a node that is run by anonymous people, but is well rated by a rating system that shows that other people have found it to be fast in the past
• Submit a transaction to multiple nodes. If a bad node delays one of the copies, then the copy sent to even a single honest node will ensure fair access

For Hedera:

• #1 will only be viable once we transition to a permissionless network (more on our path to permissionless here)
• #2 is an option, but is not mandated
• #3 and #4 reflect the fact that anyone who submits a transaction will see how long it took from when they submitted it until it reached consensus
• #5 is presented as an option in the Hedera API documentation (https://github.com/hashgraph/hedera-protobuf/blob/master/src/main/proto/BasicTypes.proto). It does imply extra cost to the client as those duplicate transactions have fees. Though the fees are lower for the duplicates than for the first one. The magnitude of this effect would depend on a) how many nodes were used and b) for how many transactions the mechanism was leveraged.

We expect clients to factor the cost of option five into their security and business models – they will decide how critical fair access is for each type of transaction they make – and plan accordingly.

For transactions for which order is not important, there would less value in sending multiple copies of a transaction. Even if a node were to unfairly delete a transaction entirely, and refuse to forward it to the network, then this would be visible within seconds, because consensus normally happens within seconds. So for applications where a delay of a few seconds is acceptable, the client could simply resubmit it a few seconds later to a second node, if it didn’t get through the first node by then.

Fair order

Even if a transaction receives fair access, so the node to which it was sent promptly submitted it to the network without bias or delay, a DLT might have the possibility that the transaction will be inappropriately and unfairly recorded as being received later than it actually arrived, through the determination of the consensus order in Step 3 of the basic transaction process.

DLTs differ in whether or not they allow for a transaction to be inappropriately counted as being delayed through explicit bias in this step. In other words - not all DLTs can guarantee fair order.

If a DLT’s consensus algorithm elects a leader (regardless of whether they are elected by POW, picked by a random number generator, or just because it is their turn), then, even if only for a short period of time, that leader will have the opportunity and ability to show bias and delay the transactions of a particular client. In a blockchain, that delay could manifest as a transaction being pushed to the next block, or to a later position within the current block. The key takeaway is this: any such leader-based DLT cannot guarantee fair order.

Hashgraph elects no leader and therefore removes the possibility of tampering with fair ordering. The place of a transaction within the consensus order is calculated by the nodes as a network, not assigned by a leader. It reflects the order in which the transaction was received by all the nodes.

Hedera Hashgraph guarantees fair order and allows clients to protect themselves against the compromise of fair access that plagues all DLTs. It is because we can guarantee fair ordering that clients may be willing to pay extra fees to ensure they have fair access.

Other DLTs may be unable to guarantee fair order but nevertheless do allow clients to protect themselves against compromise of fair access – with the mechanisms described above. That clients do not generally avail themselves of these protections perhaps reflects the reality of the potentially unfair ordering to follow – why go to any effort protecting your transactions’ fair access if the leader-based consensus to follow can compromise their fair ordering?

What TPS to report?

Any citation or claim of throughput or performance in general by a DLT should be clear on any caveats or qualifications attached to those numbers.

Hedera first published throughput numbers in our whitepaper – where we made clear that the large values of transactions per second (TPS) were for consensus only and did not reflect transaction signature verification or processing into state.

More recently we shared our expected performance numbers at Open Access in our announcement of OA last week. In our release, we highlight expected TPS for each and every Hedera service.

At OA, the Hedera Hashgraph network will offer three services: 1) cryptocurrency, throttled to 10,000 transactions per second, 2) smart contracts, and 3) file service, both throttled to 10 transactions per second to start and increasing systematically throughout the remainder of 2019. While Hedera is seeing faster speeds in testing, the throttling is designed to similarly roll out network services in a measured, responsible way, and Hedera plans to increase speed methodically in the coming months. The recently announced Hedera Consensus Service is expected to deliver similar performance as the cryptocurrency service and will be publicly available later this year.

For Hedera, relevant caveats for our predicted 10k TPS at Open Access are that this throughput value:

• Is for hbar transfers, as opposed to smart contract or file service transactions. We optimize for fast hbar transfers by having nodes maintain the hbar balances of all accounts in memory, and not the disk. Some DLTs optimize for smart contract performance by keeping smart contract state in RAM. We do expect the throughput of both hbar transfers and file and smart contract transactions to increase beyond their throttled values at OA, but hbar transfer TPS will always be significantly greater than smart contract TPS
• Presumes that receipts are requested from mainnet, and state proofs are requested from mirror nodes, for these hbar transfer transactions. Receipts are generated as part of the normal transaction processing and so place minimal burden on the node that responds to a request for one. The components of a state proof are also created as part of normal transaction processing – these components are assembled into the state proof only when requested by a client. It is this additional assembling, plus the larger size of a state proof relative to a receipt, that places a slightly larger burden on the node responding to a query

In practice, the burden of responding to state proof requests would be unlikely to significantly strain a node and impact its ability to process transactions into consensus – the throughput for which we predict to be 10k TPS at OA. Nor would adding the time to assemble and return a state proof significantly add to the end-to-end latency for a transaction – it is a matter of additional microseconds. Also, the additional bandwidth needed per node is reduced as the number of nodes grows. The requests are spread out over more nodes, so there is less bandwidth needed per node.

Consequently, even if state proofs were requested on each and every transaction (which we do not expect clients to request), and even if many of them are requested from mainnet nodes rather than mirror nodes, we do not anticipate that significantly impacting the throughput. The caveat we make about receipts versus state proofs when citing throughput is arguably our being overly cautious.

The above would change if the nodes neared their maximum capacity – which we do not anticipate to be the case at or soon after OA. By the time throughput demand (as opposed to capacity) pushes nodes near their limits, we will look to the mirror network to remove from mainnet nodes the burden of state proofs. With the mirror network responding to state proof requests, and setting their own fees for them, the mainnet can fully focus on consensus processing.

And to be clear, neither state proofs nor full mirror network are supported at OA so at least when the network is still in beta, the question of impact of state proofs on throughput is moot.

We have endeavoured to be transparent about our anticipated throughput, any qualifications on that performance, and the roadmap by which we will get to our peak. We fully recognize the importance of such openness and will continue to address questions raised by developers building on our platform, and other members of the community. Thank you in advance for listening to our livestream next week and we look forward to continuing this community discourse after OA!