Functional Code Development for Existing Data Transformations of Retail Industry Data Warehouse

Several ETL tools with graphical user interfaces and additional built-in functions have been developed to simplify the creation and maintenance of ETL processes. However, a drawback of using GUI-based ETL solutions is that they limit the flexibility of the software for customization by customers, as it remains closed-source. To overcome this issue, our project adopts a different approach that utilizes functional code written in the Python scripting language. ETL activities are defined by Python functional code, enabling users to easily and efficiently create complex ETL tasks with multiple resources and parallel tasks. Furthermore, our approach simplifies ETL development and surpasses GUI techniques in terms of testing, cost and flexibility.