Data Engineer Associate

Job description

Requisition ID: 226352

Thanks for your interest in ScotiaTech, Scotiabank's new and innovative Technology hub in Bogota.

Join a purpose driven winning team that promotes creativity and innovation in a fast-paced environment, where we’re always committed to results, in an inclusive, diverse, and high-performing culture.

Purpose The DataOps Engineer is responsible for building, maintaining, and optimizing scalable and reliable data pipelines and infrastructure in a cloud environment, under the guidance of the Senior DataOps Engineer.

This role collaborates with a global team of DataOps Engineers and cross-functional stakeholders to ensure high-quality data products and services that support the organization’s data-driven goals.

By implementing DataOps best practices and leveraging cloud technologies, the DataOps Engineer contributes to efficient data workflows, operational reliability, and seamless delivery across multiple countries.

Accountabilities  • Data Pipeline Development and Maintenance:
- Assist in the design, development, and maintenance of cloud-based data pipelines using platforms such as Azure, GCP, or AWS (e.g., Dataflow, BigQuery, GCS, S3, Lambda functions, Cloud Functions, Composer), following established best practices and under the guidance of senior engineers.
- Implement and support automated data workflows, including ETL processes, data quality checks, and orchestration using tools like Python, Spark, Airflow, or Cloud Composer.
- Support efforts to meet performance and reliability targets as defined by senior team members.

• Cloud Infrastructure Support:
- Support the deployment and management of cloud data infrastructure components, ensuring scalability, security, and cost efficiency as directed by the Senior DataOps Engineer or other senior staff.
- Apply infrastructure-as-code (IaC) practices using tools like Terraform or CloudFormation to provision and manage resources.
- Monitor and assist in optimizing cloud resource usage (e.g., storage, compute) using tools like Azure Cost Management or GCP Billing.
ent, or GCP Billing.

• Automation and CI/CD Implementation:
- Build and maintain CI/CD pipelines for data workflows using tools such as Jenkins, GitHub Actions, Cloud Build and Jfrog to enable rapid and reliable deployments.
- Automate repetitive tasks, such as data validation and monitoring, to improve operational efficiency.

• Collaboration and Global Teamwork:
- Work closely with Data Engineers, Data Scientists, and business teams to deliver data products that meet business requirements.
- Collaborate with global DataOps team members across multiple time zones, contributing to shared goals and knowledge sharing.
- Provide regular updates on task progress and escalate issues to the Senior DataOps Engineer as needed.

• Monitoring and Incident Response:
- Monitor data pipelines and infrastructure using tools like Cloud Monitoring, Prometheus, or Cloud Logging to ensure operational health.
- Participate in incident response and troubleshooting, conducting root cause analysis to resolve issues within defined SLAs (e.g., <4 hours for critical incidents).

• Compliance and Security:
- Implement data security measures, such as encryption to Personal Identifiable Information and role-based access control (RBAC), in line with global regulations (e.g., GDPR, CCPA).
- Support audits and documentation to ensure compliance with organizational and regulatory standards.

• Control versioning:
- Deploy and manage repositories using git to keep and track changes on time
- Apply best practices on versioning to create branches and modifications around the different tool that allows systems of control versioning • Continuous Improvement:
- Contribute to the development of reusable frameworks, scripts, and documentation to enhance team efficiency.
- Stay informed about DataOps and cloud technology trends, proposing improvements to processes and tools.

Education / Experience / Other Information (include only those that are specific to the role) • Education:
- Bachelor’s degree in computer science, Data Engineering, Information Technology, or a related field.

• Experience:
- Minimum of 1 years of experience in data engineering, DevOps, or DataOps roles, with hands-on experience in cloud environments (e.g., AWS, Azure, GCP).
- Demonstrated experience supporting the building and maintenance of data pipelines using tools like Dataflow, Spark, or BigQuery, preferably as part of a team or under the guidance of senior engineers.
- Familiarity with CI/CD pipelines and automation tools (e.g., Jenkins, GitHub Actions, Cloud Build).
- Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation) and monitoring solutions (e.g., Cloud Monitoring, Prometheus).
- Exposure to working in distributed teams or cross-functional environments is preferred.
- 1-3 years using Python and SQL • Certifications (Preferred):
- Cloud certifications such as Google Cloud Associate Data Engineer, Azure Data Fundamentals, AWS Certified Developer.
- Entry-level DevOps or Data Ops certifications are a plus.

• Skills:
- Proficiency in programming languages like Python, SQL, or Scala for data pipeline development.
- Proficiency in orchestration tools like Airflow, Rancher, Kubernetes, Terraform.
- Solid understanding of Data Ops principles, including automation, ETL and ELT, version control, and observability.
- Strong problem-solving skills and attention to detail in managing complex data workflows.
- Effective communication skills to collaborate with global teams and document technical processes.
- Ability to work independently and manage tasks in a fast-paced, distributed environment.

Working Conditions
• Work in a standard office-based environment; non-standard hours are a common occurrence.



#LI-hybrid

Required Skill Profession

Computer Occupations