- I have demonstrated expertise in implementing advanced data transformation processes,
designing efficient data pipelines, and migrating data between cloud platforms.
In the first project, I led the successful migration with 95% accuracy of data from AWS to GCP, combination
of technologies including Airflow, Impala, SQL & Google BigQuery, along with custom pipeline development
to ensure seamless data transfer.
In the current project, I'm playing a role of architecting and implementing transformer jobs designed to process data from diverse sources,
including EventHub with various formats such as JSON and CSV. This involves storing the data in its raw format and applying transformations to
populate the primary data layer effectively. Additionally, I conduct comprehensive data profiling using tools like Pandas & Python, JSON parsing.
I've also contributed to the development of custom connector codes using Scala for an in-house data engineering framework.
Technologies such as Spark, Scala, Databricks, SQL and Azure. Tools JIRA, IntelliJ, Visual Studio.
My passion for technology extends to a keen interest in cybersecurity, where I actively seek opportunities to integrate security best
practices into data engineering solutions.
With a keenness to always learn and innovate.
.svg)
