Academic
Publications
A generic parallel processing model for facilitating data mining and integration

A generic parallel processing model for facilitating data mining and integration,10.1016/j.parco.2011.02.006,Parallel Computing,Liangxiu Han,Chee Sun

A generic parallel processing model for facilitating data mining and integration  
BibTex | RIS | RefWorks Download
To facilitate data mining and integration (DMI) processes in a generic way, we investigate a parallel pipeline streaming model. We model a DMI task as a streaming data-flow graph: a directed acyclic graph (DAG) of Processing Elements (PEs). The composition mechanism links PEs via data streams, which may be in memory, buffered via disks or inter-computer data-flows. This makes it possible to build arbitrary DAGs with pipelining and both data and task parallelisms, which provide room for performance enhancement. We have applied this approach to a real DMI case in the life sciences and implemented a prototype. To demonstrate feasibility of the modelled DMI task and assess the efficiency of the prototype, we have also built a performance evaluation model. The experimental evaluation results show that a linear speedup has been achieved with the increase of the number of distributed computing nodes in this case study.
Journal: Parallel Computing - PC , vol. 37, no. 3, pp. 157-171, 2011
Cumulative Annual
View Publication
The following links allow you to view full publications. These links are maintained by other sources not affiliated with Microsoft Academic Search.