ETL based data integration scheduling

Qianqian Zheng

doi:10.1117/12.2655919

12 January 2023 ETL based data integration scheduling

Qianqian Zheng

Proceedings Volume 12509, Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022); 125091K (2023) https://doi.org/10.1117/12.2655919
Event: Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022), 2022, Guangzhou, China

Abstract

As the open source ETL tool with the largest number of users, Kettle's powerful and concise functions are welcomed by the majority of ETL practitioners. ETL is the process of extracting, converting and loading data from the data source to the target. Based on the enterprise data model, ETL builds a reasonable data storage mode, establishes an enterprise data exchange platform, meets the data exchange needs between various application systems, provides all-round data services, and meets the data support needs of enterprise decision-making [1]. In addition to formulating the corresponding extraction, transformation and loading tasks, ETL tools are also particularly important for the formulation of scheduling monitoring. For industrial economic data, benefit data and production data are generated periodically (such as daily, weekly and daily). Through the scheduler, users can more easily realize ETL task scheduling management, task execution performance and other data monitoring and analysis. However, Kettle's scheduling and monitoring function is very weak. This paper will use the scheduling system to monitor and schedule tasks to improve the efficiency of data integration.

Citation Download Citation

Qianqian Zheng "ETL based data integration scheduling", Proc. SPIE 12509, Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022), 125091K (12 January 2023); https://doi.org/10.1117/12.2655919

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available