Overview of the role
The Data Engineer position focuses on developing, analyzing, maintaining, and evolving data processing systems in operational and analytical environments.
This role involves working with modern, cloud-native data stacks including AWS Glue, Apache Spark, Delta Lake, and Databricks to solve complex, large-scale data challenges that directly impact global decision-making and contribute to core value streams.
Key responsibilities
- Designing and implementing scalable and secure data pipelines using AWS Glue or Databricks.
- Developing and optimizing distributed processes in Apache Spark for large volumes of data.
- Managing and maintaining Delta Lake tables, ensuring data integrity, versioning, and schema evolution.
- Writing and maintaining clean, modular Python code for ETL processes and automation.
- Collaborating with cross-functional groups to translate business needs into reliable data solutions.
Requirements
- Data Engineering: 5+ years of comprehensive experience in Data Engineering.
- Cloud Platforms: 3+ years of hands-on experience with AWS Glue or Databricks.
- Distributed Computing: 3+ years of proven experience with Apache Spark.
- Data Lake Technology: 2+ years of practical experience with Delta Lake.
- Python Expertise: 5+ years of Python programming focused on ETL and automation.
- Monitoring Systems: Familiarity with monitoring, observability, and data pipeline alerting systems.
- Cloud Knowledge: Experience with cloud environments, preferably AWS ecosystem.
- Language Proficiency: Advanced proficiency in English.
What to expect from us
- Fully Remote: Work from anywhere in the world.
- Competitive Compensation: Excellent payment in USD or your preferred local currency.
- Flexible Hours: Design your own work schedule for optimal work-life balance.
- Paid Leave: Parental leave, vacation, and national holidays.
- Dynamic Work Culture: Innovative and multicultural environment.
- Elite Collaboration: Work with the global top 1% of talent.
- Growth and Support: Mentorship and career development opportunities.
Seniority level
Employment type
Job function
Industries
#J-18808-Ljbffr