Purpose of the Job You have already moved from a ETL developer, Data Engineer or Systems Engineer into a Big Data Devops Engineer role in a linux based environment.
Maybe you are working at a smaller organisation where you are a bit of a jack of all trades in the business intelligence / data warehousing space but you yearn for something more challenging?
You want to work at scale with a large diverse team of specialist supporting a 24X7 operation. You will be used to taking responsibility for deploying data transformation code into production and, with the assistance of appropriate monitoring and proactively troubleshooting any issues that arise.
Job Category IT Job Objectives 1. Work with specialist in operations to ensure data applications and the jobs running on them are secure and available in production : - Work with the ETL development teams and Data Engineers to deploy and ensure the deployability of infrastructure, packaged applications and data transformation jobs.
2. Work with specialist in operations to ensure data applications and the jobs running on them are secure and available in production : - Implement appropriate monitoring and logging to ensure potential failure can be detected, diagnosed and remediate before services are impacted.
4. Work with application owners to manage the cost infrastructure deployed for applications both in development and production : - Tag resource appropriately so that their cost can be monitored by system.
Qualifications Essential : - Grade 12.- 3 year degree / diploma in technology field- Azure Administrator / Developer Associate Certification- Terraform certification (desirable) Experience - 3 years experience experience in technology related field either as ETL developer, Data Engineer or Systems Engineer for Linux hosted data management application managed through scripted automated deployment : - 1 years experience as Big Data DevOps Engineer responsible for the deployment and availability of data transformation jobs- Retail and / or Ecommerce (Desirable).
Knowledge and Skills - Deploying and managing applications and databases in the cloud- Big data / data warehousing / business intelligence application patterns- DevOps- Agile- Application troubleshooting and root cause of failure analysis- Ability to implement SDLC automation and Testing for data transformation or data query jobs- Ability to configure and deploy infrastructure using Terraform- Ability to implement logging and monitor data application using environment and application specific logging- Ability to implement mechanisms to detect and respond to incidents- Ability to implement high availability, and disaster recovery for big data jobs