>>
Diego. López
Data Engineer based in Madrid, Spain. Curious, self-teaching, proactive. Fluent in Software Development and Machine Learning. You can also check my LinkedIn Profile.
Diego López
Data Engineer & Software Developer
I was born in Madrid and studied on Sagrado Corazón de Madrid. I enroled in the Material Science and Engineering Bachelor in Universidad Politécnica de Madrid.
On my fourth year I got an Erasmus scolarship to study in Vilnius Tech. I completed my studies in four years.
During my Final Grade Paper I started to get into programming and Data, analyzing laboratory results using Python. The project, financed by Huawei, consisted on improving the electromechanical coupling factor in SAW devices.
My first job was already software related, as an Automation Developer in Prosegur. I jumped to Data Engineering in the consultancy firm Alten, and I am currently working as a Data Engineer/Software Developer in the insurtech firm Allianz Direct.
Focused in the areas of Data, ML/AI and software development, I am also very interested in (cyber)security, DevOps and all things software related!
Experience
Job Experiences
Data Engineer & Software Developer
Allianz Direct (November 2023 - today)
Data Engineer and software developer in charge of building, maintaining and scaling a unified Data Platform as a core component of a data-centric company.
This data platform is a self-service data lakehouse, in which different domain users can autonomously generate new data pipelines by filling templates, which are automatically deployed on Databricks, and which custom logic is developed in Scala and Python.
Our goal is a data platform in which employees all across the company can include data as a core component of their workflows and decision processes, making this as agile and easy for them as possible, while still following software and governance best practices.
My responsabilities include:
Develop an ecosystem (custom libraries, CI/CD pipelines, IaC) in which users can automatically ingest data (Kafka, APIs, files) and transform it (Python, dbt) by just filling templates
Build custom pipelines for data analytics and machine learning use cases
Data governance (IAM, GDPR, data quality)
Platform security (access controls, security by depth)
Cost optimization (storage and compute)
Platform monitoring (both internal and user-facing dashboards)
Tech stack: Data Platform based on Azure Databricks. Code developed in Scala, Python and SQL. GitHub used for version control and CI/CD. Terraform for IaC. For ingestion and transformation: Kafka, Spark, pandas, dbt.
Data Engineer
Alten (March 2022 - November 2023)
Data Engineer in charge of building data pipelines to ingest, transform and load data from different business applications into a unified cloud-based data lakehouse, serving the business needs all across the process. The data lakehouse was based on Azure Databricks, and the pipelines were bulit using Scala, Python or SQL within the Spark framework. Other tools used: Azure Synapse, Azure Data Factory, Azure DevOps.
Automation Developer
Prosegur (October 2019 - March 2022)
Automation of manual business processes for departments within the company, using tools such as Python, SQL, VBA withing Blue Prism RPA workflows.
Skills
Technologies and soft skills
Programming languages
Proficient in Scala, Python and SQL. Some limited experience with web development related languages (this website!)
Frameworks
Proficient in Spark, SQL Databases and REST APIs. Experience with pandas and Kafka
Cloud
Proficient with Azure services. Also have some specific knowledge of Google Cloud and AWS
Machine Learning
Experience with preprocessing, feature engineering and decision tree based models (XGBoost). Theoretical knowledge of state-of-the-art Deep Learning architectures (convolutions, transformers, residual networks). Made an end-to-end ML pipeline as a side project
Linux
Experience handling Linux/UNIX systems (Ubuntu, WSL)
$ echo Nice to meet you $(whoami)!
Soft skills
I work well in teams and I try to be proactive and innovative. I am very curious by nature, both in work and non-work related topics (I am completing a Philosophy Bachelor in my free time). I always get along with my colleagues, I joke a lot and I like fun and friendly environments!
Personal Projects
Projects I have worked on in my free time
Botwenger
End-to-end ML workflowEnd-to-end machine learning workflow to make daily recommendations for the football fantasy game Biwenger. Including data scraping, preprocessing, feature engineering, model training, daily inference, post-processing and serving. Check the README for more info!
Curriculum Vitae website
You are already here!Development of this static website containing my CV using Django for development and GitHub Pages for deployment
More to come!
Diego López ©