International Journal of Frontiers in Engineering Technology, 2023, 5(2); doi: 10.25236/IJFET.2023.050209.
Linxiangyi Li1, Haiyan Wang2, Yanjin Wu3
1Distributed Storage, Shaanxi University of Science & Technology, Xi’an, China
2Signal and Information Processing, Shaanxi University of Science & Technology, Xi’an, China
3Embedded and Simulator, Shaanxi University of Science & Technology, Xi’an, China
With the development of the information age, the volume of data is exploding and the operational and performance requirements of massive data are difficult to balance. The high cost of distributed storage systems with high data reliability and excellent random read and write performance, often consisting of multiple racks, multiple nodes and hundreds of disks, is a thorny issue for researchers. A tool to predict the performance of distributed storage architectures has been built based on a study of the cost of distributed storage architectures. Without hardware, enabling low cost and fast performance prediction. It also provides performance data and latency density to compare the performance of different workloads and architectures more intuitively, providing ideas for optimising the performance of distributed systems.
Low cost, No hardware, Performance evaluation, Distributed systems
Linxiangyi Li, Haiyan Wang, Yanjin Wu. A virtualisation performance evaluation approach for distributed block device. International Journal of Frontiers in Engineering Technology (2023), Vol. 5, Issue 2: 50-57. https://doi.org/10.25236/IJFET.2023.050209.
[1] Raja F R, Akram A. A Highly Scalable and Efficient Distributed File Storage System [J]. 2022.
[2] Weil S A. Ceph: reliable, scalable, and high-performance distributed storage [J]. Santa Cruz, 2007.
[3] Zhangling W U, Wei M. Distributed storage system, data processing method, and storage node, US11262916B2 [P]. 2022.
[4] Mak R, Kalyanakrishnan A, Song G Y, et al. Object Tiering in A Distributed Storage System, US2022121364A1 [P]. 2022.
[5] Inggs G, Thomas D B, Luk W. An Efficient, Automatic Approach to High Performance Heterogeneous Computing [J]. Computer Science, 2015.
[6] Li Z, Shen H. Measuring Scale-Up and Scale-Out Hadoop with Remote and Local File Systems and Selecting the Best Platform [J]. IEEE Transactions on Parallel and Distributed Systems, 2017, 28(11): 3201-3214.
[7] Aung, Mi K M, Zhu, et al. Building a large-scale object-based active storage platform for data analytics in the internet of things [J]. Journal of supercomputing, 2016.
[8] Zheng S, Chen R, Jin Y, et al. NeoFlow: A Flexible Framework for Enabling Efficient Compilation for High Performance DNN Training [J]. IEEE Transactions on Parallel and Distributed Systems, 2021.
[9] Batchu R K, Seetha H. A Hybrid Detection System for DDoS Attacks Based on Deep Sparse Autoencoder and Light Gradient Boost Machine[J]. Journal of Information & Knowledge Management, 2023, 22(01).
[10] Katkar R, Kulkarni K. Study on Block Device Driver and NVMe their Implementation Impacts on Performance. 2014.
[11] Shen Z, Chen F, Yadgar G, et al. One Size Never Fits All: A Flexible Storage Interface for SSDs [C]// 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2019.
[12] Wadhwa B, Byna S, Butt A. Toward Transparent Data Management in Multi-layer Storage Hierarchy for HPC Systems. 2018.
[13] Office of Law Enforcement Standards (OLES)United States of America, America U. Test Results for Hardware Write Block Device: FastBloc IDE (Firmware Version 16) [J]. Bureau of Justice Statistics.2006
[14] M Norell. Test Results for Hardware Write Block Device: Fastbloc Fe (Firewire Interface) [J]. 2012.
[15] Yang Y, Huang C, Zhang Y, et al. Processable Potassium–Carbon Nanotube Film with a Three-Dimensional Structure for Ultrastable Metallic Potassium Anodes[J]. 2022.
[16] Nguyen L D, Leyva-Mayorga I, Popovski P. Witness-based Approach for Scaling Distributed Ledgers to Massive IoT Scenarios[J]. 2020.
[17] Combs V T, Hurley P, Schultz C B, et al. DISTRIBUTED SYSTEM EVALUATION [J]. 2022.
[18] axboe/fio: Flexible I/O Tester [EB/OL]. [2022-12-23]. https://github.com/axboe/fio
[19] Iozone Filesystem Benchmark [EB/OL]. [2022-12-23]. www.iozone.org/
[20] BCC-Tools for BPF-based Linux IO analysis, networking, monitoring, and more [EB/OL]. [2022-12-23]. https://github.com/iovisor/bcc
[21] TPC benchmarks [EB/OL]. [2022-12-23]. www.tpc.org/
[22] HammerDB open-source software with source code hosted by the TPC [EB/OL]. [2022-12-23]. https://www.hammerdb.con