A distributed architecture for data mining and integration

• DOI: 10.1145/1552280.1552282
• Publication Date: 2009-06-09T12:44:24Z
• Authors: Malcolm P. Atkinson; Jano I. van Hemert; Liangxiu Han; Ally Hume; Chee Sun Liew
• Keywords: 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

This paper presents the rationale for a new architecture to support a significant increase in the scale of data integration and data mining. It proposes the composition into one framework of (1) data mining and (2) data access and integration. We name the combined activity DMI. It supports enactment of DMI processes across heterogeneous and distributed data resources and data mining services. It posits that a useful division can be made between the facilities established to support the definition of DMI processes and the computational infrastructure provided to enact DMI processes. Communication between those two divisions is restricted to requests submitted to gateway services in a canonical DMI language. Larger-scale processes are enabled by incremental refinement of DMI-process definitions often by recomposition of lower-level definitions. Autonomous evolution of data resources and services is supported by types and descriptions which will support detection of inconsistencies and semi-automatic insertion of adaptations. These architectural ideas are being evaluated in a feasibility study that involves an application scenario and representatives of the community.