Senior Data Platform Engineer CR 12/11/26

2 hours, 40 minutes ago
Contract
Senior
DevOps and Infrastructure
Margo Bank

Margo Bank

Unlock excellence with MARGO Consulting: where ambition, expertise, and innovation drive the most complex tech challenges.

Professional Services
$8M raised

Description

  • Own and evolve the Apache Airflow platform, including environments, deployments, upgrades, configuration, connections, pools, and DAG standards.
  • Lead the migration from Azure Data Factory to Airflow and enhance the metadata-driven orchestration framework.
  • Ensure reliable, observable orchestration through SLAs, alerting, retries, backfills, health monitoring, and clear failure ownership.
  • Manage Azure platform resources using Terraform, including Databricks, Data Lake Storage, Data Factory, Key Vault, networking, and supporting services.
  • Own secrets, identity, and access management across Key Vault, managed identities, Azure RBAC, Entra ID, and Unity Catalog.
  • Manage Azure DevOps repositories, CI/CD pipelines, branch policies, environment promotion, deployment validation, and Terraform state.
  • Drive Azure resource governance and cost efficiency through tagging, budgets, right-sizing, and cost attribution.
  • Operate and optimize Databricks, including Unity Catalog, cluster and job policies, Auto Loader ingestion patterns, and workload performance.
  • Contribute to data catalog, data quality, lineage, observability, developer tooling, and AI-assisted engineering workflows.
  • Provide second-line production support, lead incident resolution and root-cause analysis, and improve runbooks and operational processes.

Requirements

  • Hands-on experience managing Apache Airflow platforms and orchestration standards.
  • Experience migrating or modernizing orchestration workloads from Azure Data Factory to Apache Airflow.
  • Experience managing Azure resources with Terraform and infrastructure as code.
  • Experience with Azure services including Databricks, Data Lake Storage, Key Vault, Azure Data Factory, networking, and identity management.
  • Knowledge of Azure RBAC, Entra ID groups, managed identities, service principals, and least-privilege access controls.
  • Experience with Azure DevOps repositories, CI/CD pipelines, branch policies, and multi-environment deployments.
  • Experience operating Databricks and Unity Catalog, including governance, cluster or job policies, and workload optimization.
  • Experience designing observability, monitoring, alerting, and operational processes for data platforms.
  • Ability to assess and communicate architectural trade-offs across performance, cost, reliability, and complexity.
  • Experience troubleshooting production platform incidents, conducting root-cause analyses, and maintaining operational documentation.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Freelance Senior Quantexa Data Engineer

HITCONTRACT 11-50 Professional Services

Join Danske Bank as a fully remote Senior Quantexa Data Engineer on a 2027 contract, building scalable and reliable software solutions in a complex banking environment.

2 days, 2 hours ago

Contract Senior Data Engineer

YLD 51-200 information technology & services

Join YLD as a remote Data Engineer to build and scale secure data-platform infrastructure for a client’s internal and external stakeholders.

Apache Airflow Apache Spark AWS Datadog dbt Kafka Python SQL
2 days, 5 hours ago

Senior Data Engineer

Massive Rocket 51-200 Media

Massive Rocket is hiring a Senior Data Engineer to design and optimise Snowflake-based customer data foundations that enable CRM, personalisation, and audience activation for clients.

CRM dbt Segment Snowflake SQL
1 week ago

Data Engineer - Web Scraping (3 months contract)

Join CXG as a remote Data Engineer - Web Scraping on a 3-month contract to maintain web-scraping operations and build data pipelines that support insights for global luxury brands.

Apache Airflow Azure dbt Git Python SQL Terraform
1 week, 3 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers