Selected Works
Data Pipeline for Airflow
This repository showcases an ETL (Extract, Transform, Load) process using Apache Airflow, aimed at demonstrating my capabilities as a data engineer. The project includes custom Docker configurations to set up the environment, and an Airflow DAG that extracts data from an XML file, transforms it into structured data, and loads it into an MS SQL database.
Docker
Python
Airflow
SQL
Real-time IOT Data Dashboard
Built a real-time analytics dashboard using Python and C# to visualize streaming data from socket connections. Using a Raspberry Pi with a ToF sensor to count how many people have entered a room. That data is send from the Pi to a PC that will display that information on our website. The data is transferred via socket connections. With the Pi as the server and the PC as the client.
Python
C#
Socket Connection
TCP