Senior Data Engineer (Python and Spark)
Digital
Our client is a leading global provider of digital platform engineering and development services. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. We are seeking a Senior Data Engineer to join our client team and drive the performance, reliability and scalability of our data platform. This role focuses on Snowflake optimization, pipeline automation and mentoring junior engineers while contributing to key architectural decisions. |
- 5+ years of experience as a Data Engineer, building scalable data pipelines and working with cloud-based data ecosystems
- Expertise in SQL and hands-on experience building performant datasets in BigQuery or similar cloud data warehouses
- Proficiency in Python and PySpark for scalable data processing in distributed environments
- Understanding of data modeling, ELT/ETL patterns, and data quality best practices
- Familiarity with Google Cloud Platform, particularly BigQuery, Dataflow, and Cloud Composer, GCS, or equivalent cloud data services
- Background in building scalable data pipelines, both batch and near real-time, in a cloud-native environment
- Proficiency with version control, CI/CD pipelines, and automated testing frameworks
- Capability to troubleshoot and optimize performance across compute, storage, and processing layers
- Design and deliver batch and real-time ETL/ELT pipelines across cloud environments to support analytics and reporting
- Write and optimize advanced SQL transformations and build performant, cost-efficient BigQuery data models
- Implement scalable data processing solutions using Python and PySpark, ensuring maintainable and high-quality code
- Build robust data models and apply validation practices to maintain accuracy and reliability
- Use GCP services such as BigQuery, Dataflow, Cloud Composer, Pub/Sub, and GCS to build and operate modern data platforms
- Troubleshoot complex pipeline issues and continuously improve compute, storage, and processing performance
- Collaborate with engineering, analytics, and business teams while contributing to CI/CD, code reviews, and testing standards
Ref: JN-092026-1160160