Fusion: Practice and Applications
FPA
2692-4048
2770-0070
10.54216/FPA
https://www.americaspg.com/journals/show/3453
2018
2018
Efficient Data Processing Techniques for Structured Data Analysis Using Stream Pipeline Parallelism
Sampath Kini K, Assistant Professor, Computer Science and Engineering, NITTE Deemed to be University, Karnataka, India
Sampath
Sampath
Professor, Computer Science and Engineering, NMAM Institute of Technology, NITTE Deemed to be University, Karnataka, India
D. K.
Sreekantha
This research illustrates how dynamic task balancing and data sharing may improve distributed data processing. The technology handles parallel processing system difficulties with huge datasets by minimizing resource utilization, time complexity, and output. We modify the workload on the fly after splitting to ensure that all processing units receive equal work. One last optimization phase optimizes job distribution to maximize system efficiency. We test the solution for latency, speed, scalability, resource utilization, fault tolerance, and synchronization overhead. Results reveal that the new strategy outperforms existing ones in every regard. It features the lowest latency, quickest production, and highest growth potential. The approach handles mistakes well, divides data effectively, and syncs everything at a cheap cost. These properties make it ideal for real-time data processing and fast-growing applications. Future study will concentrate on flexible splitting strategies, fault tolerance mechanisms, and predictive analytics machine learning models. These modifications will improve real-time data handling.
2025
2025
104
115
10.54216/FPA.180109
https://www.americaspg.com/articleinfo/3/show/3453