Tech Lead Data Platform

Job description

Company Overview

Lean Tech is a rapidly expanding organization situated in Medellín, Colombia.

We pride ourselves on possessing one of the most influential networks within software development and IT services for the entertainment, financial, and logistics sectors.

Our corporate projections offer a multitude of opportunities for professionals to elevate their careers and experience substantial growth.

Joining our team means engaging with expansive engineering teams across Latin America and the United States, contributing to cutting‑edge developments in multiple industries.

Position Overview

As a Senior Data Engineer and Technical Lead, you will take complete ownership of Lean Tech's cutting‑edge iPaaS data platform, driving its strategic evolution and operational excellence.

This senior role encompasses the entire data ecosystem, from microservices deployed on AWS ECS and robust Linux virtual machines (including Kubernetes administration) to our expansive AWS S3 data lake, sophisticated Airflow orchestration, and optimized Snowflake environments.

You will be instrumental in defining and implementing engineering standards, mentoring a talented team on best practices in dbt, Python (leveraging libraries like Pandas, SQL Alchemy, boto3), and SQL, and championing continuous architectural improvements.

Your responsibilities will span the full lifecycle, including scoping new integrations, standing up infrastructure with advanced DevOps practices (Docker, CI/CD), enforcing stringent security and access controls (including hands‑on Snowflake RBAC and DDL as code with Liquibase), and administering our DataHub data catalog.

This position offers a unique opportunity to combine deep technical leadership with hands‑on expertise in production ownership, cost and capacity management, and cross‑functional collaboration, ensuring our data platform is scalable, secure, and empowers data‑driven decisions across the organization.

Key Responsibilities

• Lead the end‑to‑end ownership and evolution of the iPaaS data platform, encompassing microservices deployed on AWS ECS, Linux VMs with Kubernetes administration, the AWS S3 data lake, orchestration, Snowflake layers, and comprehensive data governance.
  
  
• Drive the intake and scoping of new data integrations, collaborating with requesters to assess technical feasibility, define clear milestones and SLAs, and provide expert implementation guidelines.
  
  
• Architect and evolve robust infrastructure with a strong DevOps mindset, provisioning and hardening Linux VMs, configuring secure networking and secrets, and containerizing services using Docker for reliable deployments.
  
  
• Operate and optimize the data lake on AWS S3, managing raw, staged, and curated zones, defining folder/partitioning conventions, lifecycle policies, and cost‑efficient storage patterns.
  
  
• Design and execute scalable data orchestration workflows using Apache Airflow, running complex Python (leveraging Pandas, SQLAlchemy, requests, boto3) and dbt pipelines at scale.
  
  
• Develop and maintain secure and efficient container images for data workloads (Python, dbt, Airflow workers), focusing on base images, dependency pinning, multi‑stage builds, vulnerability scanning, and image registry management.
  
  
• Champion data modeling and transformation best practices with dbt, designing new models, enforcing rigorous testing and documentation, conducting thorough PR reviews, and mentoring engineers.
  
  
• Manage and optimize Snowflake environments (Ingestion → Transformations → Consumption), ensuring peak performance through tuning, appropriate warehouse sizing, and environment parity.
  
  
• Govern Snowflake resources effectively using Liquibase (DDL as code), safely creating, modifying, and dropping warehouses, databases, and schemas, while managing users, roles, and grants.
  
  
• Enforce stringent security and access controls, implementing least‑privilege RBAC, establishing owner approvals, conducting periodic user‑permission matrix reviews, and ensuring audit readiness.
  
  
• Implement and administer the data catalog (DataHub), facilitating metadata ingestion, enhancing data lineage and discoverability, managing access, and onboarding new users.
  
  
• Oversee production monitoring and troubleshooting, including health checks.
  
  

Required Skills & Experience

• 6+ years of hands‑on experience in data engineering or platform roles, including at least 2 years in a technical leadership capacity, guiding data teams and projects.
  
  
• Deep expertise with the modern data stack, including AWS S3 for data lake operations, Apache Airflow for robust orchestration, dbt for advanced data modeling and transformation, and Snowflake for scalable data warehousing, encompassing performance tuning and warehouse sizing.
  
  
• Proven ability to govern Snowflake environments using Liquibase (DDL as code) for resource management, user/role administration, and secure change promotion.
  
  
• Strong hands‑on experience with Snowflake RBAC, implementing least‑privilege access controls, masking policies, and ensuring environment parity.
  
  
• Advanced proficiency in Docker, encompassing multi‑stage builds, slim images, caching, SBOM/vulnerability scanning, and private registry management.
  
  
• Experience with container orchestration platforms like AWS ECS for deploying and scaling microservices.
  
  
• Strong background in DevOps practices, including provisioning and hardening Linux VMs, configuring networking/secrets, and performing Kubernetes administration on Linux.
  
  
• Experience with CI/CD for containers and data, including Git workflows, automated build/test (unit + dbt tests), image publishing, and environment promotion.
  
  
• Expertise in Python, including libraries such as Pandas, SQL Alchemy, requests, Airflow, and boto3, for data engineering tasks and API development.
  
  
• Strong SQL skills for optimizing transformations and storage layouts.
  
  
• Extensive experience owning production data platforms, encompassing observability (logs, metrics, traces), proactive alerting, incident response, Root Cause Analysis (RCA), postmortems, and driving continuous reliability improvements.
  
  
• Hands‑on experience implementing and administering data catalogs like DataHub, focusing on metadata ingestion, lineage, discoverability, and access management.
  
  
• Demonstrated ability to set and enforce engineering standards, coach and mentor data engineers, drive architectural improvements, and partner cross‑functionally with clear stakeholder communication for requirements intake, timeline setting, and decision records.
  
  
• Experience managing cost and capacity for data platforms, including S3.
  
  

Nice to Have Skills

• Experience with advanced DataHub features, such as lineage customizations and metadata policies.
  
  
• Familiarity with iPaaS tooling like Workato or similar platforms.
  
  
• Exposure to security and compliance frameworks, including IAM, encryption, audit trails, and disaster recovery concepts (DR/RTO/RPO).
  
  

Soft Skills

• Leadership & Mentorship: Inspire and guide your team, fostering growth through mentorship, rigorous code reviews, and by setting high engineering standards that drive continuous architectural improvements.
  
  
• Cross‑functional Collaboration & Communication: Excel at partnering with diverse stakeholders, translating complex technical requirements into clear plans, defining data contracts, and ensuring transparent communication throughout the project lifecycle.
  
  
• End‑to‑End Ownership & Problem‑Solving: Embrace full ownership of the data platform, from proactive monitoring and troubleshooting to leading incident response, conducting thorough RCA, and implementing robust reliability improvements.
  
  
• Strategic Thinking & Proactive Optimization: Demonstrate a keen ability to manage cost and capacity, optimize resources, and continuously seek out innovative solutions to enhance platform performance and efficiency.
  
  
• Quality & Standards Advocate: Champion best practices in data modeling, testing, and documentation, ensuring high‑quality code and robust data pipelines through meticulous PR reviews and adherence to stringent CI checks.
  
  

Why You Will Love Working with Us

Join a powerful tech workforce and help us change the world through technology.

Professional development opportunities with international customers, a collaborative work environment, a clear career path, and mentorship programs that will lead to new levels.

Lean Tech is an equal‑opportunity employer and we celebrate diversity and are committed to creating an inclusive environment for all employees.

Required Skill Profession

Other General