广东工业大学学报 ›› 2021, Vol. 38 ›› Issue (06): 77-83.doi: 10.12052/gdutxb.210104

• • 上一篇    下一篇

去中心化的数据处理方案设计

李光程1, 赵庆林1, 谢侃2   

  1. 1. 澳门科技大学 资讯科技学院，澳门 999078;
    2. 广东工业大学 自动化学院，广东 广州 510006
  • 收稿日期:2021-07-08 出版日期:2021-11-10 发布日期:2021-11-09
  • 通信作者: 赵庆林(1974–)，男，教授，博士，主要研究方向为区块链和去中心化计算、机器学习及其应用、物联网、无线通信和网络、云/雾计算、软件定义的无线网络，E-mail：zqlict@hotmail.com E-mail:zqlict@hotmail.com
  • 作者简介:李光程(1994–)，男，博士研究生，主要研究方向为区块链和去中心化计算、机器学习及其应用、云/边缘计算以及数据处理，E-mail：guangcheng.li@hotmail.com
  • 基金资助:
    国家自然科学基金资助项目(61872451，61872452)；澳门科学技术发展基金资助项目(0098/2018/A3，0076/2019/A2，0062/2020/A2)

A Design of Decentralized Data Processing Scheme

Li Guang-cheng1, Zhao Qing-lin1, Xie Kan2   

  1. 1. Faculty of Information Technology, Macau University of Science and Technology, Macau SAR 999078, China;
    2. School of Automation, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2021-07-08 Online:2021-11-10 Published:2021-11-09

HTML

PDF (PC)

2619

摘要： 传统的基于主/从的数据处理框架容易受到主节点的单点故障和性能瓶颈的影响。相比之下, 区块链系统采用去中心化的框架, 能够聚合海量的计算资源。提出了一种基于区块链的数据处理框架, 利用区块链的优点来解决中心化框架的缺点。在所提出的框架中, 区块链存储任务信息, 采用的有用工作共识证明共识机制使节点能够使用其计算资源处理任务, 同时竞争领导者(将待处理的任务分派到区块链)。模拟表明, 所提出的框架在吞吐量和任务响应时间方面优于集中式框架。

关键词: 区块链, 去中心化框架, 有用工作证明, 区块链应用

Abstract: Conventional master/slave-based data processing frameworks are vulnerable to single point of failure and performance bottlenecks of the master node. In contrast, blockchain systems adopt a decentralized framework and are capable of aggregating enormous computing resources. A blockchain-based data processing framework is proposed that utilizes the advantages of the blockchain for solving the drawbacks of the centralized framework. In this framework, the blockchain stores the task information and the adopted proof of useful work consensus enables nodes to process tasks using their computing resources, while competing for the leader (who dispatches pending tasks to the blockchain). Extensive simulations show that the proposed framework is better than the centralized framework in terms of the throughput and the task response time.

Key words: blockchain, decentralized framework, proof of useful work, blockchain application

中图分类号: 

  • TP391
[1] DEAN J, GHEMAWAT S. MapReduce: simplified data processing on large clusters [J]. Communications of the ACM, 2008, 51(1): 107-113.
[2] TOSHNIWAL A, TANEJA S, SHUKLA A, et al. Storm@ Twitter[C]//Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. [S.l.]: ACM, 2014: 147-156.
[3] CARBONE P, KATSIFODIMOS A, EWEN S, et al. Apache flink: stream and batch processing in a single engine [J]. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015, 36(4): 28-38.
[4] REN Z, XU X, WAN J, et al. Workload characterization on a production hadoop cluster: a case study on Taobao[C]//2012 IEEE International Symposium on Workload Characterization (ⅡSWC). La Jolla, USA: IEEE, 2012: 3-13.
[5] WAN J, LIU M, HU X, et al. Dual-JT: toward the high availability of Job Tracker in Hadoop[C]//4th IEEE International Conference on Cloud Computing Technology and Science Proceedings. Taipei: IEEE, 2012: 263-268.
[6] SWAN M. Blockchain: blueprint for a new economy[M]. USA: O’Reilly Media, Inc., 2015.
[7] VUKOLIĆ M. The quest for scalable blockchain fabric: Proof-of-Work vs. BFT replication[C]//International Workshop on Open Problems in Network Security. Cham: Springer, 2015.
[8] MIMS C. The Bitcoin network is now more powerful than the top 500 supercomputers, Combined[EB/OL]. (2013-05-13) [2021-07-01]. https://finance.yahoo.com/news/bitcoin-network-row-more-powerful-153313393.html.
[9] WOOD G, BUTERIN V. Ethereum: a secure decentralised generalised transaction ledger [J]. Ethereum Project Yellow Paper, 2014: 1-32.
[10] ZHANG F, EYAL I, ESCRIVA R, et al. REM: resource-efficient mining for blockchains[C]//26th USENIX Conference on Security Symposium. [S.l.]: ACM, 1427-1444.
[11] BENDJOUDI A, MELAB N, TALBI E G. An adaptive hierarchical master-worker (AHMW) framework for grids—application to B&B algorithms [J]. Journal of Parallel and Distributed Computing, 2012, 72(2): 120-131.
[12] LI M, WENG J, YANG A, et al. Crowdbc: a blockchain-based decentralized framework for crowdsourcing [J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 30(6): 1251-1266.
[13] LI Z, YANG Z, XIE S, et al. Credit-based payments for fast computing resource trading in edge-assisted internet of things [J]. IEEE Internet of Things Journal, 2019, 6(4): 6606-6617.
[14] YANG Z, YANG K, LEI L, et al. Blockchain-based decentralized trust management in vehicular networks [J]. IEEE Internet of Things Journal, 2018, 6(2): 1495-1505.
[15] ZYSKIND G, NATHAN O, PENTLAND A. Enigma: decentralized computation platform with guaranteed privacy[EB/OL]. ArXiv Preprint ArXiv: 1506.03471 (2015-06-10) [2021-07-01]. https://arxiv.org/abs/1506.03471.
[16] HOEKSTRA M, LAL R, PAPPACHAN P, et al. Using innovative instructions to create trustworthy software solutions[C]//Proceedings of the 2nd International Workshop on Hardware and Architectural Support for Security and Privacy. [S.l.]: ACM, 2013.
[17] HUNT T, ZHU Z, XU Y, et al. Ryoan: a distributed sandbox for untrusted computation on secret data[J]. Proceedings of the 12th USENIX conference on Operating Systems Design and ImplementationNovember. [S.l.]: ACM, 2016: 533-549.
[18] MCKEEN F, ALEXANDROVICH I, ANATI I, et al. Intel® software guard extensions (Intel® SGX) support for dynamic memory management inside an enclave[C]//Proceedings of the Hardware and Architectural Support for Security and Privacy. [S.l.]: ACM, 2016.
[19] EYAL I, GENCER A E, SIRER E G, et al. Bitcoin-NG: a scalable blockchain protocol[C]// Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation. [S.l.]: ACM, 2016.
[20] LUU L, NARAYANAN V, ZHENG C, et al. A secure sharding protocol for open blockchains[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. [S.l.]: ACM, 2016.
[21] ZAMANI M, MOVAHEDI M, RAYKOVA M. RapidChain: scaling blockchain via full sharding[C]//Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. [S.l.]: ACM, 2018.
[1] 陈冰儿, 王帮海, 劳南新. 基于DPoS扩展的量子加密区块链[J]. 广东工业大学学报, 2021, 38(02): 34-38.
[2] 聂敏航, 欧毓毅. 一种可自定义金额的数字货币去中心化混淆方案[J]. 广东工业大学学报, 2021, 38(01): 64-68.
[3] 魏生, 戴科冕. 区块链金融场景应用分析及企业级架构探讨[J]. 广东工业大学学报, 2020, 37(02): 1-10.
[4] 魏生, 戴科冕. 基于区块链技术的私募股权众筹平台变革及展望[J]. 广东工业大学学报, 2019, 36(02): 37-46.
[5] 徐恪, 姚文兵. 赛博智能经济与区块链[J]. 广东工业大学学报, 2018, 35(03): 1-9.
[6] 冷杰武, 江平宇, 刘加军, 陈庆新, 刘强. 区块链技术驱动的产消者自组织产品制造社群构建[J]. 广东工业大学学报, 2017, 34(05): 1-9.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发