The Hedera Consensus Service: Beyond Smart Contracts
Jun 20, 2019
by Mance Harmon
CEO and Co-founder

The digital landscape is dominated by centralized application services and centralized data servers. We hope the organizations that operate these services are trustworthy, and that they will be honest and treat their users fairly. For example, we trust the service providers to process the transactions in the order received, that transactions are not dropped or delayed to the detriment of some and benefit of others, and that the order of transactions is not manipulated or changed after the fact.

Distributed ledger technology improves the situation by eliminating the need to trust a single organization. Distributed applications operate on a collection of computers, each administered by a different organization. The order of transactions submitted by application users is determined by the group of computers that vote on the order, rather than a single computer operator that decides the order. Modifying transaction order in a distributed application requires collusion by a large group of computer operators, and is far less likely than with a single organization.

Distributed ledgers are often categorized in two ways: Public vs Private, and Open vs Permissioned. Public / Private refers to who can use the network services. If users must first register and be approved to use the network, then the network is considered Private. If not, then the network is Public. Open / Permissioned refers to who has the ability to run a computer in the network. If computer operators can join the network anonymously, then the network is Open. Otherwise, the network is a Permissioned network.

For many years a debate has raged about the relative value of Public ledgers versus Private ledgers.

Generally, a Public ledger is considered to be more decentralized, and therefore is more trustworthy. Public ledger networks comprise a relatively larger number of nodes, each administered by a different entity, and each node running in a different location.

Private ledgers often have a small number of nodes that vote on the order of transactions, and many times these nodes might all be in a single datacenter or operated by the same cloud provider. As a result, Private ledgers are considered less trustworthy and less resilient to attack than Public ledgers.

  • Summary: Public ledgers are more distributed, and therefore more trustworthy than Private ledgers.

When developers build distributed applications, they often want to execute business logic on multiple computers on the network, and have each computer verify the outcome of the calculations. This elevates the trust in the results of the calculations made by network nodes. In public networks, the execution of this software, normally referred to as a smart contract, is very expensive and inefficient. Ledgers try to fix this problem by putting some of the operations into native services, which run much faster than smart contracts. But if the public ledger doesn’t choose to put a function into a native service, then developers are forced to live with the inefficiencies of smart contracts. Developers are often restricted in their use of programming languages and tools, and they must share server resources with all other users of the Public network. On the other hand, developers building distributed applications on Private networks can use any programming language and toolset they desire, and they may have the full resources of the Private network for their exclusive use, and their logic runs at the speed of a native service, not of a smart contract.

  • Summary: Private networks provide a much better set of options for developers, and can be fully dedicated to a single application if desired. Public networks are restrictive, costly, and inefficient when executing smart contracts.

One potential downside of a Public ledger is that the information in the ledger is public. That means that anyone, without permission, can download a copy of the ledger and read its contents. There are many cases where that is not acceptable. The information could be encrypted, but then the smart contracts could not operate on the information. Of course, this is not a problem for Private ledgers where only those that have been given explicit permission may read the contents of the ledger.

  • Summary: Private ledgers protect data privacy better than public ledgers.

When developers decide to build a distributed application, they must first decide on what network the application will run. Public networks provide a set of services that enable developers to build arbitrarily complex applications, and these services are immediately available on the existing network. Developers can simply begin development of the application. Private networks require the collaboration of multiple parties to install ledger software on multiple computers in order to create the network. This requires significant resources, coordination, and additional costs on the part of all members of the network.

  • Summary: Public networks provide an existing trusted platform for developers. Private networks require substantial up-front and ongoing costs to establish and maintain the servers in the network.

Because of these differences, there is a schism in the DLT market, between two distinct sets of market participants and technologies. Examples on the public side include Libra from Facebook, Ethereum, EOS, Stellar, and others. On the private side are Hyperledger, Corda from R3, EEA, Quorum, and others.

The Hedera Consensus Service (HCS) in combination with Hedera Mirror Nodes and State Proofs unifies the public and private DLT markets by providing the advantages of both categories without the disadvantages. Many developers have built centralized applications running natively on dedicated computers. HCS allows them to use a public distributed network for fast, fair, low-cost transaction sequencing that is immutable. In addition, HCS provides distributed trust in code execution without the need for smart contracts. Essentially, HCS allows developers to add their own native services to the public Hedera ledger. The state of the application is not stored on the public ledger, it’s stored in a private computer. The computation of the native service is not performed on the public ledger. The computation is performed on a private computer. Therefore, neither of these functions require the resources of the public ledger.

Both Public ledgers and Private ledgers need more than 2/3 of the computer operators to be honest for the network to be trusted. But when using HCS you only need more than 2/3 honest on the Public side (the Hedera network). On the Private side, none of the computers have to be honest because they can prove their calculations are correct based on the state proof from the Public network. Any computer can cryptographically prove that it acted correctly, and therefore it achieves as much trust as the Public network.

HCS makes it possible for existing centralized platforms (like existing auction sites, stock exchanges, and MMO games) to achieve the trust of being decentralized while still having the privacy and performance of being centralized. Also, applications using DLT platforms like Hyperledger can use HCS with all the benefits of a public network.

Hedera Consensus Service:

1. Provides the distributed trust of a public network
2. Enables developers to build native applications on dedicated central servers that are highly efficient and low cost
3. Enables use of a public network for trust while protecting private information in private servers
4. Eliminates the up-front and ongoing costs associated with creating a private network

Diagram 1: Current architecture for smart contracts. Smart contracts and application state are executed and stored directly on the nodes of the distributed ledger. As a result, all Dapps must share compute resources and file storage on the nodes, and all information in the smart contracts is public.

Diagram 2: Hedera Consensus Service with Mirror Nodes. Distributed applications can execute on dedicated servers, and private information remains private.

1. Application clients submit transactions with messages and topic to mainnet nodes
2. Hedera network puts messages in consensus order
3. Mirror nodes receive all information from the mainnet
4. Mirror node sends messages for the topic for the application
5. Application performs the business logic, and sends the results to the Client software

The HCS puts messages into consensus order. To ensure that the application is processing the transactions appropriately (e.g., the application can be trusted with the code execution), there are multiple options available.

Validating Correct Application Code Execution

Single-Node Solutions

1. If the application code is available, one can elect to run a copy of the application and process the same transactions from any mirror node. This enables anyone to check the accuracy of the application code. Only a single honest actor is needed to ensure the integrity of the calculations
2. If the application code is not publicly available, but an auditor is allowed to run a copy of the application, then the auditor can verify the correct processing of the transactions

Multi-Node Solution

One could create a network of applications nodes receiving a message stream from local mirror nodes. The collection of application nodes and mirror nodes we call an application network: an appnet. As long as a single application node operator is honest, then incorrect processing of the message stream can be detected. If a dispute is raised, it is possible to execute an audit of the processing of the message stream. The message stream is guaranteed to be tamper proof because state proofs are generated based on signatures from the mainnet nodes.

Appnets are easier to create when using Hyperledger Fabric. Fabric can aid in managing the set of computers, how they connect, and the logic of how they work together.

Interested in learning more?

Dr. Leemon Baird (CTO / Co-Founder of Hedera) and Bryan Gross (Principle Product Manager, IBM Blockchain Platform) will discuss the new service in more detail during a webinar today, June 20th, at 10am PT/1pm ET. To register, visit:
https://hedera.zoom.us/webinar/register/4715573586804/WN_xUlH6FS7TwKqrPL_-g9-Gg
A replay will also be made available at:
https://www.youtube.com/c/HederaHashgraph
To read the full white paper, visit:
https://www.hedera.com/hh-consensus-service-whitepaper.pdf