Evaluating Blockchains for IoT

论文翻译

201802 26-28 Evaluating Blockchains for IoT P5 R17

这是一篇关于区块链和物联网的论文，第一次写，先适应一下这个文本编辑器。第一天比较粗糙，只学会了标题，以后会慢慢自己画图，制表等……

Abstract:

As proof-of-work blockchains are inherently energy greedy and offer probabilistic guarantees, blockchains based on Byzantine consensus appear as a promising technology to track billions of connected devices. In this paper, we evaluate the performance of prominent blockchains that solve the classic Byzantine consensus problem. Our results show that while offering reasonable throughput their performance usually do not scale to tens of devices and drops dramatically as the number of devices increases. This study motivates the need for solutions that solves the Blockchain consensus problem, a scalable variant of the classic Byzantine consensus problem but dedicated to blockchains.
由于工作证明区块链本身具有能量贪婪并提供概率保证，基于拜占庭共识的区块链似乎是一种有前途的技术，可以跟踪数十亿个连接设备。 在本文中，我们评估了解决经典拜占庭共识问题的突出区块链的表现。 我们的结果表明，虽然提供合理的吞吐量，但它们的性能通常不会扩展到数十个设备，并且随着设备数量的增加而急剧下降。 这项研究激发了解决区块链共识问题的解决方案的需求，这是经典拜占庭共识问题的可扩展变体，但致力于区块链。

Key words

Evaluating, Byzantine Consensus, Blockchains

1 Introducation

Blockchain is promising but enterprises have been reluctant at adopting proof-of-work blockchains [1] because they offer probabilistic guarantees [2], can be subject to double-spending [3], especially in a consortium or private context [4]. Instead, consortium and private blockchains, which often solve deterministically the classic Byzantine consensus problem, are typically more secure when some assumptions are met. As classic Byzantine consensus protocols [5] were originally designed for small networks, it remains unclear whether one can adapt them to scale to numerous blockchain devices as required by the Internet of Things (IoT). 1
区块链是有希望的，但企业一直不愿意采用工作证明区块链[1]，因为它们提供概率保证[2]，可能会受到双重支出[3]，特别是在财团或私人环境中[4]。相反，联盟链和私链通常可以确定地解决经典的拜占庭共识问题，在满足某些假设时通常更安全。由于经典的拜占庭共识协议[5]最初是为小型网络设计的，因此尚不清楚是否可以根据物联网（IoT）的要求对其进行调整以扩展到众多区块链设备。 1

To address this problem, we evaluate prominent consortium blockchains, including Hyperledger Fabric (HLF) v0.6 with the Practical Byzantine Fault Tolerance (PBFT) consensus protocol [5], HLF v1.0 with the Byzantine Fault-Tolerant State Machine Replication (BFT-SMaRt) consensus protocol [6], and Ripple with the Ripple consensus protocol [7]. Some efforts were recently devoted to evaluate blockchains [8], however, they aim at comparing the performance of inherently slow proof-of-work blockchains to faster blockchains based on Byzantine consensus but we are not aware of experimental evaluation focused on energy-efficient technologies. 23
为了解决这个问题，我们评估了显着的联盟区块链，包括Hyperledger Fabric（HLF）v0.6与实际拜占庭容错（PBFT）共识协议[5]，HLF v1.0与拜占庭容错状态机复制（BFT） -SMaRt）共识协议[6]和Ripple与Ripple共识协议[7]。最近有一些努力用于评估区块链[8]，但是，它们的目的是将基于拜占庭共识的固有慢速工作证明区块链的性能与更快的区块链进行比较，但我们并未意识到以能效技术为重点的实验评估。 23

Our results show that these blockchains are reasonably fast but their performance degrade significantly with the increase in the number of devices.
我们的结果表明，这些区块链相当快，但随着设备数量的增加，它们的性能显着下降。

2 Related work

Various blockchains have been proposed for different purposes, but only few proposals aim at targeting the Internet of Things (IoT). Applying blockchains to IoT will likely require to adapt some of the mechanisms of already existing blockchains, especially their consensus implementation.
已经提出了各种区块链用于不同目的，但是只有少数提议旨在针对物联网（IoT）。将区块链应用于物联网可能需要调整现有区块链的一些机制，特别是它们的共识实施。

R3 explored various blockchain technologies, including Ethereum [4], before proposing a distributed ledger, called Corda [9], mainly to process and record transactions between financial institutions. Corda aims at supporting two consensus protocols, Raft [10] and BFT-SMaRt [6]. As the former is insecure, we focused our attention on the latter that tolerates less than n/3 malicious behaviours: BFT-SMaRt is a high performance Byzantine Fault Tolerant State Machine Replication that relies on a leader to solve the classic Byzantine consensus problem that predates blockchains. At the time of writing its implementation in Corda was not ready, so we focused on the variant of Hyperledger Fabric that implements BFT-SMaRt as well.
在提出一个名为Corda [9]的分布式分类账之前，R3探索了各种区块链技术，包括以太坊[4]，主要用于处理和记录金融机构之间的交易。 Corda旨在支持两个共识协议，Raft [10]和BFT-SMaRt [6]。由于前者不安全，我们将注意力集中在容忍少于n / 3恶意行为的后者上：BFT-SMaRt是一种高性能的拜占庭容错状态机复制，它依赖于领导者来解决经典的拜占庭共识问题。 blockchains。在撰写本文时，它在Corda中的实现尚未准备就绪，因此我们专注于实现BFT-SMaRt的Hyperledger Fabric变体。

Hyperledger Fabric v0.6 is a customizable consortium blockchain platform [11]that supports smart contracts called “chaincodes”, Chaincode supports existing programming languages like Go and Java. Hyperledger Fabric v0.6 relies on PBFT, a Byzantine consensus protocol that was designed 16 years ago for local area network and is still prominent today [5]. As Hyperledger Fabric v1.0 is known to be insecure in the presence of malicious behaviors [12], a fork of v1.0 master branch was changed to invoke a BFT ordering service based on BFT-SMaRt. This branch uses gRPC and supports only timestamp broadcast rather than the message types offered by the main branch but copes with malicious behaviors.
Hyperledger Fabric v0.6是一个可定制的联盟区块链平台[11]，支持称为“链码”的智能合约，Chaincode支持现有的编程语言，如Go和Java。 Hyperledger Fabric v0.6依赖于PBFT，这是16年前为局域网设计的拜占庭共识协议，至今仍然很突出[5]。由于已知Hyperledger Fabric v1.0在存在恶意行为时不安全[12]，因此更改了v1.0主分支的分支以基于BFT-SMaRt调用BFT订购服务。此分支使用gRPC并仅支持时间戳广播，而不支持主分支提供的消息类型，但应对恶意行为。

Ripple is a blockchain platform with a strong focus on financial applications. It sets a wide range of transactions for accounts and their XRP currency exchanges, but it lacks the support of smart contracts. It runs its own Ripple consensus algorithm [7]. Although the correctness of the Ripple consensus algorithm has been debated [13], it is out of the scope of this paper to prove whether the code is correct.
Ripple是一个区块链平台，专注于金融应用。它为账户及其XRP货币兑换设置了广泛的交易，但它缺乏智能合约的支持。它运行自己的Ripple共识算法[7]。虽然Ripple一致性算法的正确性已经引起争论[13]，但证明代码是否正确超出了本文的范围。

IOTA [14] is a cryptocurrency for IoT. As opposed to the classic Byzantine consensus technologies, it does not totally order transactions but builds a directed acyclic graph (DAG) of transactions, called Tangle. To commit new transactions, IOTA requires currently a single coordinator, which may become a single point of failure, as the failure of this coordinator prevents transactions from being executed1. IOTA is still under development and expected to be decentralised in the future. Commercial products also rely on DAG of transactions [15], unfortunately their code is not available.
IOTA [14]是物联网的加密货币。与经典的拜占庭共识技术相反，它并不完全对交易进行排序，而是构建了一个有向交易的非循环图（DAG），称为Tangle。为了提交新事务，IOTA当前需要一个协调器，这可能会成为单点故障，因为此协调器的故障会阻止事务被执行1。 IOTA仍处于开发阶段，预计将来会分散。商业产品也依赖交易的DAG [15]，遗憾的是他们的代码不可用。

The Red Belly Blockchain2 relies on recent variants of consensus, called Blockchain Consensus [16], especially designed for scalability. Its analysis is part of future work.
Red Belly Blockchain2依赖于最近的共识变体，称为Blockchain Consensus [16]，专为可扩展性而设计。它的分析是未来工作的一部分。

3 Evaluating Byzantine Consensus Blockchains

The main focus of this study is the comparison and analysis of mainstream secure blockchains. We selected the three following blockchain platforms:
Hyperledger Fabric v0.6 with PBFT consensus [5]. In particular, the reason why we complemented our study with the prototypical HLF version using BFT-SMaRT is because the consensus of HLF v0.6 is known to have unresolved bugs.3
Ripple with the XRP consensus algorithm [7]: In Ripple, a validator is usually a trusted node that is authenticated by Ripple Inc. in the public network. To speed up the execution, we created our own local validator rather than a public validator.
Hyperledger Fabric, based on v1.0, with BFT-SMaRt consensus [6]. Because this branch uses gRPC, which supports only timestamp broadcast rather than the message types offered by the main branch, we had to adapt our benchmark when using BFT-SMaRt.
本研究的主要焦点是主流安全区块链的比较和分析。 我们选择了以下三个区块链平台：
具有PBFT共识的Hyperledger Fabric v0.6 [5]。 特别是，我们使用BFT-SMaRT用原型HLF版本补充我们的研究的原因是因为已知HLF v0.6的共识具有未解决的错误。
使用XRP一致性算法的Ripple [7]：在Ripple中，验证器通常是由Ripple Inc.在公共网络中验证的可信节点。 为了加快执行速度，我们创建了自己的本地验证器而不是公共验证器。
Hyperledger Fabric，基于v1.0，具有BFT-SMaRt共识[6]。 因为这个分支使用gRPC，它只支持时间戳广播而不是主分支提供的消息类型，所以我们在使用BFT-SMaRt时必须调整我们的基准。

A. Distributed Experiments

We ran some experiments on a distributed system of 32 machines using Emulab, an environment providing interconnected heterogeneous and physical machines for distributed system experiments. To this end, clients ran Node.js v6.11.4 on Ubuntu 16.04 to generate the load with 2 types of machines equiped with:
Two 64-bit E5-2630 Haswell processors with 8 cores running at 2.4GHz, 64 GB 2133 MT/s DDR4 RAM (8 x 8GB modules), one Intel SATA SSD with 200 GB and 2 x 1TB 7200 rpm SATA disks.
One 64-bit E5-2630 Nehalem processor with 4 cores running at 2.4 GHz, with 12 GB 1066 MHz DDR2 RAM (6 x 2 Gb modules), one Seagate SATA disk with 250 GB and one 500 GB 7200 rpm Western Digital SATA disk.
A.分布式实验
我们使用Emulab在32台机器的分布式系统上进行了一些实验，Emulab是一个为分布式系统实验提供互连的异构和物理机器的环境。 为此，客户端在Ubuntu 16.04上运行Node.js v6.11.4以使用以下两种类型的机器生成负载：
两个64位E5-2630 Haswell处理器，8核运行2.4GHz，64 GB 2133 MT / s DDR4 RAM（8 x 8GB模块），一个Intel SATA SSD 200 GB和2 x 1TB 7200 rpm SATA磁盘。
一个64位E5-2630 Nehalem处理器，4核运行，2.4 GHz，12 GB 1066 MHz DDR2 RAM（6 x 2 Gb模块），一个希捷SATA磁盘250 GB和一个500 GB 7200 rpm Western Digital SATA磁盘。

B. Adapting the Workload to the Blockchain Apis

The difficulty in evaluating different blockchains lies in the lack of interface standards. We constructed a generic workload that performs the same functions on the different blockchains interface. Because, smart contracts are not supported by Ripple, we designed the workload to perform a transfer from an account A to another account B with the amount 1.
Table I Parameters of the workload

1. The HLF v0.6 API
   It provides operations that maintain key-value pairs, also called states. Therefore, to create an account, the client can just put a key as the username as well as a corresponding value standing for the balance with an initial value. The transfer of money is represented as an atomic operation that the balance of A decreases and the balance of B increases.
   Operations which change the state by the function PutState() are non-blocking operations called invocations. Therefore, the client can know whether the transaction has been executed only if it queries the blockchain, which is a blocking operation.
2. The Ripple API
   It specifies the format of an account as a hash value, which is created by a passphrase, included in the network with the consent of the root account, and activated by a transfer from another existing account to it.
   The transfer transaction is called payment in Ripple. Therefore, we designed a workload for Ripple as a payment transaction from an account to another account activated in this network with amount 1, which is identical to the transaction in HLF. Similarly to HLF, the Ripple interface is non-blocking in that a transaction commitment is not acknowledged.
3. The HLF with Bft-Smart Api
   It is an experimental branch of the BFT ordering service for HLF. The current version only supports the timestamp broadcast message type, and was tested in order to make a comparison with PBFT ordering in HLF v0.6. As opposed to HLF v0.6, this newer version uses gRPC.
   B.使工作量适应区块链Apis
   评估不同区块链的困难在于缺乏界面标准。我们构建了一个通用工作负载，它在不同的区块链接口上执行相同的功能。因为Ripple不支持智能合约，我们设计工作负载以执行从帐户A到另一个帐户B的转帐，金额为1。
   表1 工作负载的参数
   表一

1）HLF v0.6 API
它提供维护键值对的操作，也称为状态。因此，要创建一个帐户，客户端只需将一个密钥作为用户名以及一个代表余额的相应值和一个初始值。货币转移表示为原子操作，A的余额减少，B的余额增加。
通过函数PutState（）更改状态的操作是称为调用的非阻塞操作。因此，客户端只有在查询区块链（即阻塞操作）时才能知道事务是否已被执行。
2）Ripple API
它将帐户的格式指定为哈希值，该哈希值由密码创建，在root帐户同意的情况下包含在网络中，并通过从另一个现有帐户转移到该帐户来激活。
转账交易在Ripple中称为付款。因此，我们将Ripple的工作负载设计为从帐户到此网络中激活的另一个帐户的支付交易，金额为1，与HLF中的交易相同。与HLF类似，Ripple接口是非阻塞的，因为不确认事务约定。
3）具有Bft-Smart Api的HLF
它是HLF BFT订购服务的实验分支。当前版本仅支持时间戳广播消息类型，并进行了测试，以便与HLF v0.6中的PBFT排序进行比较。与HLF v0.6相反，这个较新的版本使用gRPC。

C. Applying a Workload on the Blockchains

All experiments were based on the loadtest library of Node.js, which is a tool for testing the performance of a backend service by sending requests. This homogenises the interfaces of the blockchains as both HLF and Ripple supports it through HTTP.
A single HTTP request is defined as a JSON object, which guarantees that the transaction submissions to Ripple and HLF are identical. Again, the transaction invocations are nonblocking. In other words, when a transaction is submitted by HTTP, an HTTP response about the status of the transaction is sent from the server, while the transaction is being processed. The client cannot know if the transaction has committed unless it queries the server again.
The loadtest framework provides the control over the requests sent to the cluster. It offers the possibility to test both platforms by specifying the workload and the scheme of sending workloads. Critical parameters of the workload are listed in Table I, which correspond to characteristics of the HTTP protocol. We increased the workload in terms of request rate step by step and finally gathered related metrics under different request rates. The parameters of a single load testing are listed in Table II.
Table II Parameters of the experimental settings

Table III Parameters of the workloads for hlf v0.6 and ripple

1. Requests
   As for HLF v0.6, to enable a customized transaction we deployed a corresponding chaincode, coded in the high-level programming language Go. The official example includes a chaincode supporting a simple money transfer transaction, which is deployed on the cluster first, then the cluster accepts multiple requests invoking this transaction.
   Ripple has a set of well-defined transactions but lacks the support of customized transactions. The defined transactions are related to financial systems, including the transfer. However, accounts are inactive at first. The account address is derived from the public key of it, which is generated by a set of cryptographic steps. To activate an account, another account should transfer an amount of money to its address. The paid address will be activated if the amount reaches the threshold, which is decided by the whole network dynamically. Therefore, to experiment on a private Ripple network two accounts should be activated via the approach above.
   However, HLF with BFT-SMaRt ordering only supports the timestamp broadcasting with gRPC rather than other transaction types like the smart contract invocation. Therefore, the testing of BFT-SMaRt ordering was done by its inherited performance testing module. For all the experiments, we use a single orderer. While their role is unclear, changing the number of orderers does not seem to impact performance significantly [12].
2. Testing Steps
   Based on the aforementioned principles, the testing consists of multiple automated steps: (1) Setting up the cluster, (2) deploying the chaincode (for HLF), (3) activating accounts (for Ripple), (4) setting parameters of workloads, (5) setting parameters of sending workloads, (6) launching the load testing for 10 times, (7) analyzing the results. Parameters of Step (4) and Step (5) for both platforms are listed in Table IIIand Table IV, respectively.
   Table IV Parameters of the experiments for hlf v0.6 and ripple

Fig. 1.The throughput of a node with the increase of the request rate in HLF v0.6. The QPS first increases but then drops sharply, finally becomes 0.

C.在区块链上应用工作量
所有实验都基于Node.js的loadtest库，这是一个通过发送请求来测试后端服务性能的工具。这可以使区块链的接口同质化，因为HLF和Ripple都通过HTTP支持它。
单个HTTP请求被定义为JSON对象，这保证了对Ripple和HLF的事务提交是相同的。同样，事务调用是非阻塞的。换句话说，当HTTP提交事务时，在处理事务时，将从服务器发送有关事务状态的HTTP响应。除非再次查询服务器，否则客户端无法知道事务是否已提交。
loadtest框架提供对发送到集群的请求的控制。它提供了通过指定工作负载和发送工作负载方案来测试两个平台的可能性。表I中列出了工作负载的关键参数，它们对应于HTTP协议的特性。我们逐步增加了请求率方面的工作量，最终在不同的请求率下收集了相关的指标。表II列出了单次负载测试的参数。
表II实验设置的参数

表III hlf v0.6和ripple的工作负载参数

1）请求
至于HLF v0.6，为了启用自定义事务，我们部署了一个相应的链代码，用高级编程语言Go编码。官方示例包括支持简单汇款交易的链码，首先在集群上部署，然后集群接受调用此交易的多个请求。
Ripple有一组明确定义的交易，但缺乏定制交易的支持。已定义的交易与财务系统相关，包括转移。但是，帐户最初处于非活动状态。帐户地址是从其公钥导出的，该公钥由一组加密步骤生成。要激活帐户，另一个帐户应将一笔金额转移到其地址。如果金额达到阈值，则将激活付费地址，该阈值由整个网络动态决定。因此，要在私有Ripple网络上进行试验，应通过上述方法激活两个帐户。
但是，具有BFT-SMaRt排序的HLF仅支持使用gRPC进行广播的时间戳，而不支持智能合约调用等其他事务类型。因此，BFT-SMaRt排序的测试是由其继承的性能测试模块完成的。对于所有实验，我们使用单个订货人。虽然他们的角色不明确，但改变订购者的数量似乎并没有显着影响绩效[12]。
2）测试步骤
基于上述原则，测试包括多个自动化步骤：（1）设置集群，（2）部署链代码（用于HLF），（3）激活帐户（用于Ripple），（4）设置工作负载的参数，（5）设置发送工作负载的参数，（6）启动负载测试10次，（7）分析结果。两种平台的步骤（4）和步骤（5）的参数分别列于表III和表IV中。
表IV hlf v0.6和波纹的实验参数

图1. HLF v0.6中请求率增加的节点吞吐量。 QPS首先增加但随后急剧下降，最终变为0。

4 Results and Analysis

The load was applied over 4, 8, 12, 16, 20, 24, 28 and 32 independent physical machines. In this section, we describe the observed results and we analyze the performances of the chosen platforms. HLF with BFT-SMaRt consensus is evaluated separately because of its differences.
A. Metrics of the Load Testing

1. Query Per Second
   The number of queries per second (QPS) is a common indicator of throughput of a system. It translates into the number of requests the server can treat within a second [17]. The request usually refers to the HTTP request. Obviously, the more requests the system processes in a second, the better it performs.
2. Latency
   The latency is the system delay. In other words, it means the delay of a request or a task which the system processes [17]. A system usually processes multiple requests, so the latency is usually the average value of latencies in a period. As the invocations are non-blocking, we did not compute the latency as the time between a transaction is invoked and its commit is acknowledged.
   B. Blockchain Throughput as the Request Rate Increases
   The result of HLF v0.6 is shown in Fig. 1 whereas the result of Ripple is shown in Fig 2. In both figures, we observe that for each cluster, the throughput in either system increases with the growth of the request rate first, and then reaches a peak. However, after reaching the peak the throughput decreases sharply with the increase of the request rate, because the system cannot handle the demand.
   Fig. 2.The throughput of a node with the increase of the request rate in ripple. The trend is similar to the HLF v0.6.

Fig. 3.The peak throughput of a node with the increase of the number of nodes in HLF v0.6 and ripple.

Moreover, for either system, the trends of throughputs with different cluster sizes are similar, which means that the performance has no trend of increasing with the increase of nodes. In other words, consensus algorithms of Ripple, HLF v0.6 consensus, do not scale. Note that this limitation is well-known for classic BFT consensus algorithms because they solve a slightly more restrictive problem than the blockchain consensus [16]. In particular, both PBFT and BFT-SMaRt rely on a leader, which may act as a bottleneck [16].
C. Peak Throughput with the Increase of Nodes
The results of HLF v0.6 and Ripple consensus are depicted in Fig. 3, whereas Fig 4 depicts the results of HLF v1.0. As expected, the peak throughput decreases with the increase of the cluster size in Ripple. The more nodes involved in a cluster, the more messages are exchanged. For a Ripple private network with only one validator, transactions are batched and sent to the validator, so that the validator should process numerous requests. Therefore, if the number of validators and machines running validators remains the same, the consensus does not scale.
Fig. 4.The peak throughput of a node with the increase of the number of nodes in HLF with BFT-smart consensus.

Meanwhile, the HLF v0.6 cluster’s performance fluctuates with the increase of nodes, as well as the HLF with BFT-SMaRt consensus. Traditional PBFT consensus is used in HLF v0.6 so that the broadcast operation is always involved in the consensus, which creates multiple requests at a time in every node. The bottleneck effect at the leader prevents the consensus protocol from scaling.
As for BFT-SMaRt consensus in HLF, all traffic should go through the centralized nodes called orderers. Though the performance can scale by adding more orderers, it is impossible for the cluster to only have orderers but not have simple clients because of its nature. With the same number of orderers, the performance will not scale if the number of clients increases.
Meanwhile, the HLF v0.6 cluster’s performance fluctuates with the increase of nodes, as well as the HLF with BFT-SMaRt consensus. Traditional PBFT consensus is used in HLF v0.6 so that the broadcast operation is always involved in the consensus, which creates multiple requests at a time in every node. The bottleneck effect at the leader prevents the consensus protocol from scaling.
As for BFT-SMaRt consensus in HLF, all traffic should go through the centralized nodes called orderers. Though the performance can scale by adding more orderers, it is impossible for the cluster to only have orderers but not have simple clients because of its nature. With the same number of orderers, the performance will not scale if the number of clients increases.
D. The Relation Between Throughput and Latency
With the increase of the request rate, the latencies of both HLF v0.6 and Ripple first increase then decrease sharply, as shown in Fig 5 and Fig 6. The increase of the latency is because of the congestion control of the TCP protocol which prevents the computer from serving overwhelming requests simultaneously. The followed decrease is because systems are out of work. With TCP’s underlying congestion control, if too much data comes to the server suddenly, the “slow start” mechanism will limit the reception rate on the server side and the “congestion avoidance” will half the congestion window if the network congestion is detected. Therefore, the high request rate will contribute to increasing latency because of triggering the network congestion.
Fig. 5.The throughput with the increase of the latency in HLF v0.6.

Fig. 6.The throughput with the increase of the latency in ripple.

4 结果与分析
负载应用于4个，8个，12个，16个，20个，24个，28个和32个独立的物理机器上。在本节中，我们描述了观察到的结果，并分析了所选平台的性能。具有BFT-SMaRt共识的HLF由于其差异而单独评估。
A.负载测试的度量标准
1）每秒查询
每秒查询数（QPS）是系统吞吐量的常见指标。它转化为服务器在一秒钟内可以处理的请求数[17]。请求通常是指HTTP请求。显然，系统在一秒钟内处理的请求越多，它的执行就越好。
2）延迟
延迟是系统延迟。换句话说，它意味着系统处理的请求或任务的延迟[17]。系统通常处理多个请求，因此延迟通常是一段时间内的延迟的平均值。由于调用是非阻塞的，因此我们没有计算延迟，因为调用事务和确认其提交之间的时间。
B.随着请求率的增加，区块链吞吐量
HLF v0.6的结果如图1所示，而Ripple的结果如图2所示。在两个图中，我们观察到，对于每个簇，任一系统的吞吐量都随着请求率的增长而增加，然后达到顶峰。然而，在达到峰值后，吞吐量随着请求率的增加而急剧下降，因为系统无法处理需求。
图2.节点吞吐量随着纹波请求率的增加而增加。趋势类似于HLF v0.6。

图3.节点的峰值吞吐量随着HLF v0.6和纹波节点数量的增加而增加。

此外，对于任一系统，具有不同簇大小的吞吐量趋势是相似的，这意味着性能没有随着节点的增加而增加的趋势。换句话说，Ripple的共识算法，HLF v0.6共识，不能扩展。请注意，这种限制对于经典BFT一致性算法而言是众所周知的，因为它们解决了比区块链共识更严格的问题[16]。特别是，PBFT和BFT-SMaRt都依赖于领导者，这可能成为瓶颈[16]。
C.随着节点的增加峰值吞吐量
HLF v0.6和Ripple共识的结果如图3所示，而图4描述了HLF v1.0的结果。正如预期的那样，峰值吞吐量随着Ripple中簇大小的增加而降低。群集中涉及的节点越多，交换的消息就越多。对于只有一个验证器的Ripple专用网络，事务被批处理并发送到验证器，以便验证器应处理大量请求。因此，如果验证程序和运行验证程序的计算机的数量保持不变，则共识不会扩展。
图4. 随着BFT智能共识，HLF中节点数量增加的节点峰值吞吐量。

同时，HLF v0.6集群的性能随着节点的增加而波动，以及具有BFT-SMaRt共识的HLF。在HLF v0.6中使用传统的PBFT共识，以便广播操作始终参与共识，这在每个节点中一次创建多个请求。领导者的瓶颈效应阻止了共识协议的扩展。
至于BFT-SMaRt在HLF中的共识，所有流量都应该通过称为orderers的集中式节点。尽管通过添加更多订购者可以扩展性能，但是由于集群的性质，集群不可能只有订购者而不能拥有简单的客户。如果订货人的数量相同，则如果客户数量增加，性能将无法扩展。
同时，HLF v0.6集群的性能随着节点的增加而波动，以及具有BFT-SMaRt共识的HLF。在HLF v0.6中使用传统的PBFT共识，以便广播操作始终参与共识，这在每个节点中一次创建多个请求。领导者的瓶颈效应阻止了共识协议的扩展。
至于BFT-SMaRt在HLF中的共识，所有流量都应该通过称为orderers的集中式节点。尽管通过添加更多订购者可以扩展性能，但是由于集群的性质，集群不可能只有订购者而不能拥有简单的客户。如果订货人的数量相同，则如果客户数量增加，性能将无法扩展。

D.吞吐量和延迟之间的关系

随着请求速率的增加，HLF v0.6和Ripple的延迟首先增加然后急剧下降，如图5和图6所示。延迟的增加是由于TCP协议的拥塞控制阻止了计算机同时服务于压倒性的请求。随后的减少是因为系统失灵了。使用TCP的基础拥塞控制，如果突然有太多数据到达服务器，“慢启动”机制将限制服务器端的接收速率，并且如果检测到网络拥塞，则“拥塞避免”将是拥塞窗口的一半。因此，由于触发网络拥塞，高请求率将导致延迟增加。
图5. HLF v0.6中延迟增加的吞吐量。

图6.随着纹波延迟增加的吞吐量。

5 Limitations When Setting Up HLF v0.6 设置HLF v0.6时的限制

HLF v0.6 depends on Docker containers, where chaincodes are encapsulated and invoked by external requests. HLF clients communicate with chaincodes in Docker containers using Unix sockets or HTTP within the machine or endpoint. The experiment was launched on Ubuntu 16.04 OS. However, Ubuntu 16.04 adopts the systemd service manager rather than the previous initd, by which Docker starts as a service rather than applying configurations in /etc/default/docker. This causes the Unix socket endpoint to remain closed by default. Therefore, starting an HLF client with the same step as on previous versions of Ubuntu will fail on Ubuntu 16.04.
HLF v0.6依赖于Docker容器，其中链代码由外部请求封装和调用。 HLF客户端使用机器或端点内的Unix套接字或HTTP与Docker容器中的链代码进行通信。 该实验在Ubuntu 16.04 OS上启动。 但是，Ubuntu 16.04采用systemd服务管理器而不是之前的initd，Docker作为服务启动而不是在/ etc / default / docker中应用配置。 这会导致Unix套接字端点默认保持关闭状态。 因此，在Ubuntu 16.04上使用与以前版本的Ubuntu相同的步骤启动HLF客户端将失败。
7 Conclusion 结论
This paper presents the first evaluation of Byzantine-tolerant blockchains that could be adapted for IoT. Our results are preliminary but show that more work is necessary for main secure blockchains to scale. This seems to confirm the well-known inadequacy of Byzantine consensus to large-scale blockchains, mainly because Byzantine consensus requires one proposal to be decided. A natural future work is thus the comparison of these blockchain solving the Byzantine consensus problem to recent scalable solutions that solve the Blockchain Consensus problem [16], like the Red Belly Blockchain.
本文介绍了可以适应物联网的拜占庭耐受性区块链的首次评估。 我们的结果是初步的，但表明主要安全区块链需要做更多工作才能扩展。 这似乎证实了众所周知的拜占庭共识对大规模区块链的不足，主要是因为拜占庭共识需要一个提案来决定。 因此，自然未来的工作是将这些解决拜占庭共识问题的区块链与解决区块链共识问题[16]的最近可扩展解决方案进行比较，如Red Belly Blockchain。

ACKNOWLEDGMENT 感谢

We are grateful to Dinh Tien Tuan Anh, Mike Hibler, David Schwartz and Baohua Yang for their help. This research is in part supported under Australian Research Council’s Discovery Projects funding scheme (project number 180104030) entitled “Taipan: A Blockchain with Democratic Consensus and Validated Contracts”.
我们感谢Dinh Tien Tuan Anh，Mike Hibler，David Schwartz和Baohua Yang的帮助。 该研究部分得到了澳大利亚研究理事会发现项目资助计划（项目编号180104030）的支持，该计划名为“Taipan：具有民主共识和验证合同的区块链”。

reference

[1] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” 2008.
[2] V. Gramoli, “From blockchain consensus back to byzantine consensus,”Future Generation of Computer Systems, 2017.
[3] C. Natoli and V. Gramoli, “The blockchain anomaly,” in Proceedings of the 15th IEEE International Symposium on Network Computing and Applications (NCA’16), 2016, pp. 310–317.
[4] “The balance attack or why forkable blockchains are ill-suited for consortium,” in 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2017, Denver, CO, USA,June 26-29, 2017, 2017, pp. 579–590. [Online]. Available: https://doi.org/10.1109/DSN.2017.44
[5] M. Castro and B. Liskov, “Practical byzantine fault tolerance,” inProceedings of the Third Symposium on Operating Systems Designand Implementation, ser. OSDI ’99. Berkeley, CA, USA: USENIXAssociation, 1999, pp. 173–186. [Online]. Available: http://dl.acm.org/citation.cfm?id=296806.296824
[6] A. Bessani, J. Sousa, and E. E. Alchieri, “State machine replication forthe masses with bft-smart,” in Dependable Systems and Networks (DSN),2014 44th Annual IEEE/IFIP International Conference on. IEEE, 2014,pp. 355–362.
[7] D. Schwartz, N. Youngs, and A. Britto, “The ripple protocol consensus algorithm,” Ripple Labs Inc. White Paper, vol. 5, 2014.
[8] T. T. A. Dinh, J. Wang, G. Chen, R. Liu, B. C. Ooi, and K.-L.Tan, “Blockbench: A framework for analyzing private blockchains,” inProceedings of the 2017 ACM International Conference on Managementof Data, pp. 1085–1100.
[9] R. G. Brown, J. Carlyle, I. Grigg, and M. Hearn, “Corda: An introduction,”R3 CEV, August, 2016.
[10] D. Ongaro and J. K. Ousterhout, “In search of an understandableconsensus algorithm.” in USENIX Annual Technical Conference, 2014,pp. 305–319.
[11] C. Cachin, “Architecture of the hyperledger blockchain fabric,” inWorkshop on Distributed Cryptocurrencies and Consensus Ledgers,2016.
[12] J. Sousa, A. Bessani, and M. Vukoli´c, “A byzantine fault-tolerant ordering service for the hyperledger fabric blockchain platform,” 2017.
[13] F. Armknecht, G. O. Karame, A. Mandal, F. Youssef, and E. Zenner,“Ripple: Overview and outlook,” in International Conference on Trustand Trustworthy Computing. Springer, 2015, pp. 163–180.
[14] S. Popov, “The tangle,” oct 2017.
[15] L. Baird, “The Swirlds Hashgraph Consensus Algorithm: Fair, Fast,Byzantine Fault Tolerance,” pp. 1–27, 2016. [Online]. Available: http://www.swirlds.com/downloads/SWIRLDS-TR-2016-01.pdf
[16] T. Crain, V. Gramoli, M. Larrea, and M. Raynal, “Blockchain consensus,”in ALGOTEL 2017-19`emes Rencontres Francophones sur lesAspects Algorithmiques des T´el´ecommunications, 2017.
[17] S. Souders, “High-performance web sites,” Communications of the ACM,vol. 51, no. 12, pp. 36–41, 2008.