去中心化的数据处理方案设计

doi:10.12052/gdutxb.210104

Abstract

Abstract: Conventional master/slave-based data processing frameworks are vulnerable to single point of failure and performance bottlenecks of the master node. In contrast, blockchain systems adopt a decentralized framework and are capable of aggregating enormous computing resources. A blockchain-based data processing framework is proposed that utilizes the advantages of the blockchain for solving the drawbacks of the centralized framework. In this framework, the blockchain stores the task information and the adopted proof of useful work consensus enables nodes to process tasks using their computing resources, while competing for the leader (who dispatches pending tasks to the blockchain). Extensive simulations show that the proposed framework is better than the centralized framework in terms of the throughput and the task response time.

Key words: blockchain, decentralized framework, proof of useful work, blockchain application

CLC Number:

TP391

Li Guang-cheng, Zhao Qing-lin, Xie Kan. A Design of Decentralized Data Processing Scheme[J].Journal of Guangdong University of Technology, 2021, 38(06): 77-83.

References

[1] DEAN J, GHEMAWAT S. MapReduce: simplified data processing on large clusters [J]. Communications of the ACM, 2008, 51(1): 107-113.
[2] TOSHNIWAL A, TANEJA S, SHUKLA A, et al. Storm@ Twitter[C]//Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. [S.l.]: ACM, 2014: 147-156.
[3] CARBONE P, KATSIFODIMOS A, EWEN S, et al. Apache flink: stream and batch processing in a single engine [J]. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015, 36(4): 28-38.
[4] REN Z, XU X, WAN J, et al. Workload characterization on a production hadoop cluster: a case study on Taobao[C]//2012 IEEE International Symposium on Workload Characterization (ⅡSWC). La Jolla, USA: IEEE, 2012: 3-13.
[5] WAN J, LIU M, HU X, et al. Dual-JT: toward the high availability of Job Tracker in Hadoop[C]//4th IEEE International Conference on Cloud Computing Technology and Science Proceedings. Taipei: IEEE, 2012: 263-268.
[6] SWAN M. Blockchain: blueprint for a new economy[M]. USA: O’Reilly Media, Inc., 2015.
[7] VUKOLIĆ M. The quest for scalable blockchain fabric: Proof-of-Work vs. BFT replication[C]//International Workshop on Open Problems in Network Security. Cham: Springer, 2015.
[8] MIMS C. The Bitcoin network is now more powerful than the top 500 supercomputers, Combined[EB/OL]. (2013-05-13) [2021-07-01]. https://finance.yahoo.com/news/bitcoin-network-row-more-powerful-153313393.html.
[9] WOOD G, BUTERIN V. Ethereum: a secure decentralised generalised transaction ledger [J]. Ethereum Project Yellow Paper, 2014: 1-32.
[10] ZHANG F, EYAL I, ESCRIVA R, et al. REM: resource-efficient mining for blockchains[C]//26th USENIX Conference on Security Symposium. [S.l.]: ACM, 1427-1444.
[11] BENDJOUDI A, MELAB N, TALBI E G. An adaptive hierarchical master-worker (AHMW) framework for grids—application to B&B algorithms [J]. Journal of Parallel and Distributed Computing, 2012, 72(2): 120-131.
[12] LI M, WENG J, YANG A, et al. Crowdbc: a blockchain-based decentralized framework for crowdsourcing [J]. IEEE Transactions on Parallel and Distributed Systems, 2018, 30(6): 1251-1266.
[13] LI Z, YANG Z, XIE S, et al. Credit-based payments for fast computing resource trading in edge-assisted internet of things [J]. IEEE Internet of Things Journal, 2019, 6(4): 6606-6617.
[14] YANG Z, YANG K, LEI L, et al. Blockchain-based decentralized trust management in vehicular networks [J]. IEEE Internet of Things Journal, 2018, 6(2): 1495-1505.
[15] ZYSKIND G, NATHAN O, PENTLAND A. Enigma: decentralized computation platform with guaranteed privacy[EB/OL]. ArXiv Preprint ArXiv: 1506.03471 (2015-06-10) [2021-07-01]. https://arxiv.org/abs/1506.03471.
[16] HOEKSTRA M, LAL R, PAPPACHAN P, et al. Using innovative instructions to create trustworthy software solutions[C]//Proceedings of the 2nd International Workshop on Hardware and Architectural Support for Security and Privacy. [S.l.]: ACM, 2013.
[17] HUNT T, ZHU Z, XU Y, et al. Ryoan: a distributed sandbox for untrusted computation on secret data[J]. Proceedings of the 12th USENIX conference on Operating Systems Design and ImplementationNovember. [S.l.]: ACM, 2016: 533-549.
[18] MCKEEN F, ALEXANDROVICH I, ANATI I, et al. Intel^{^®} software guard extensions (Intel^® SGX) support for dynamic memory management inside an enclave[C]//Proceedings of the Hardware and Architectural Support for Security and Privacy. [S.l.]: ACM, 2016.
[19] EYAL I, GENCER A E, SIRER E G, et al. Bitcoin-NG: a scalable blockchain protocol[C]// Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation. [S.l.]: ACM, 2016.
[20] LUU L, NARAYANAN V, ZHENG C, et al. A secure sharding protocol for open blockchains[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. [S.l.]: ACM, 2016.
[21] ZAMANI M, MOVAHEDI M, RAYKOVA M. RapidChain: scaling blockchain via full sharding[C]//Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. [S.l.]: ACM, 2018.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

A Design of Decentralized Data Processing Scheme

HTML

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 5

Metrics

Comments

Recommended 0

[1]	Chen Bing-er, Wang Bang-hai, Lao Nan-xin. A Quantum-encrypted Blockchain Based on Delegated Proof of Stake (DPoS) Extension [J]. Journal of Guangdong University of Technology, 2021, 38(02): 34-38.
[2]	Wei Sheng, Dai Ke-mian. An Analysis of Blockchain Applications in Financial Scenarios and an Exploration of Enterprise Software Architecture of Blockchain as a Service(BaaS) [J]. Journal of Guangdong University of Technology, 2020, 37(02): 1-10.
[3]	Wei Sheng, Dai Ke-mian. Revolution and Prospect of Private Equity Crowd-Funding Platform Based on Blockchain Technology [J]. Journal of Guangdong University of Technology, 2019, 36(02): 37-46.
[4]	Xu Ke, Yao Wen-bing. Cyber Intelligent Economy and Blockchain [J]. Journal of Guangdong University of Technology, 2018, 35(03): 1-9.
[5]	Leng Jie-wu, Jiang Ping-yu, Liu Jia-jun, Chen Qing-xin, Liu Qiang. Self-organizing of Product-oriented Prosumers Manufacturing Community Driven by the Blockchain Technique [J]. Journal of Guangdong University of Technology, 2017, 34(05): 1-9.