Job Summary:
As an Azure DevOps Engineer, you will be responsible for designing, implementing, and managing CI/CD pipelines, as well as maintaining our Azure environments. A strong background in Azure Data Factory and Azure Databricks is essential to support our data engineering efforts and drive data integration solutions.
Key Responsibilities:
Design, implement, and maintain CI/CD pipelines using Azure DevOps.
Collaborate with development and operations teams to automate deployments and improve system reliability.
Utilize Azure Data Factory for data ingestion, transformation, and orchestration.
Develop and manage Azure Databricks notebooks and workflows for data processing and analytics.
Monitor and optimize performance of Azure services and data processing workflows.
Implement infrastructure as code using tools like Azure Resource Manager (ARM) templates or Terraform.
Ensure compliance with security and governance policies in the Azure environment.
Troubleshoot and resolve issues related to Azure DevOps, ADF, and ADB.
Collaborate with data engineers and analysts to understand data requirements and provide technical solutions.
Document processes, configurations, and architectures for future reference.
Qualifications:
Bachelor’s degree in computer science, Information Technology, or related field.
Proven experience as a DevOps Engineer, with a strong focus on Azure.
In-depth knowledge of Azure Data Factory and Azure Databricks.
Experience with CI/CD tools such as Azure Pipelines, Jenkins, or GitHub Actions.
Strong scripting skills in languages like PowerShell, Python, or Bash.
Familiarity with containerization technologies like Docker and orchestration tools like Kubernetes.
Understanding of networking, security, and cloud architecture best practices.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills.