Want to work in an environment with buzzing, energetic, smart, and motivated people?
An innovative firm leading the way in health, finance and insurance is currently seeking the skills of a highly motivated and experienced Senior Data Engineer to join their dynamic development team.
The team is responsible for building complex data solutions as well as pipelines used for extraction, cleaning, transformation, and loading into various target databases such as PostgreSQL, Redshift, MongoDB and / or Document DB.
The team is also looking at event-based streaming solutions to stream data from on premise to the cloud.
IT / Mathematics qualification or equivalent industry experience
AWS certification or other formal IT certification beneficial
7+ years experience in software engineering role with at least 3+ years in a data engineering role.
Experience in AWS services, including Glue, S3, Athena, RDS, EMR, Redshift, DocumentDB
Experience in working with RDBMS and NoSQL databases including Amazon Redshift, PostgreSQL, MongoDB, Amazon DocumentDB
Scala or Java
Spark and / or pandas (Development and debugging)
Design, develop and operationalize large scale data solutions using various approved technologies that are fit for purpose.
Design, develop and deploy complex data pipelines from ingestion to consumption for both on premise and in the cloud.
Develop robust, scalable, and high performant solutions.
Write unit, integration, and performance tests for the solutions.
Deliver high quality solutions within the set timelines.
Improve data quality and efficiency.
Ensure code is committed into source control and deployed using CI / CD pipelines.
Adhere to technical best practices including internally defined architectural guidelines and coding standards.
Understand business products to be able to translate business requirements into technical solutions.
Liaise with relevant parties where clarification of business requirements or resolution to technical issues is needed.
Provide support and maintenance of new and existing data solutions.
Reference Number for this position is LL53991 which is a Contract position based in Sandton offering a contract rate of up to R850 per hour