Data Engineering Lead
Progressive Edge Consulting
Cape Town, Western Cape
11h ago
source : findojobs-za

A largeestablished enterprise is looking to employ a Technical Lead : Data Engineeringwith deep technical knowledge the ability to lead and mentor a team of Data engineers.

You will require substantial expertise in abroad range of software development & programming fields.Your knowledge of data analysis, end user requirementsand business requirements analysis will help you develop a clear understandingof the business need and incorporate these needs into technical solutions.

You will also need a solid understanding of physicaldatabase design and the SDLC and have the ability to work well with and lead ateam.

Responsibilities : Architects Data analytics framework.Translates complex functional and technical requirements into detailedarchitecture, design, and high performing software.

Leads Data and batch / real-time analytical solutions leveragingtransformational technologies.Works on multiple projects as a technical lead driving user storyanalysis and elaboration, design and development of software applications,testing, and builds automation tools.

Development and OperationsDatabase Development and OperationsPolicies, Standards and ProceduresCommunicationsBusiness Continuity & Disaster RecoveryResearch and EvaluationCoaching / MentoringQualificationsEssential : 4 years Bachelors degree in computerscience, computer engineering, or equivalent work experience AWS Certificationat least to associate levelExperience5+ years Data engineering or software engineering3-5 years demonstrated experience leading teams ofengineers2+ years Big Data experience5+ years experience with Extract Transform and Load(ETL) processes2+ years Cloud AWS experienceAt least 2 years demonstrated experience with agileor other rapid application development methods - Agile exposure, Kanban orScrum5 years demonstrated experience with objectoriented design, coding and testing patterns as well as experience inengineering (commercial or open source) software platforms and large scale datainfrastructures.

Knowledge & Skills*Creating data feeds from on-premise to AWS Cloud(2 years)*Support data feeds in production on break fixbasis (2 years)*Creating data marts using Talend or similar ETLdevelopment tool (4 years)*Manipulating data using python and pyspark (2years)*Processing data using the Hadoop paradigmparticularly using EMR, AWSs distribution of Hadoop (2 years)*Devops for Big Data and Business Intelligenceincluding automated testing and deployment (2 years)* Expert knowledge of data modeling andunderstanding of different data structures and their benefits and limitationsunder particular use cases.

Further technical skills required : *Capability to architect highly scalabledistributed systems, using different open source tools.

  • Big Data batch and streaming tools* Talend (1 year)* AWS : EMR, EC2, S3 (1 year)* Python (1 year)* PySpark or Spark (1 year) - Desirable* Business Intelligence Data modelling (3 years)
  • Report this job

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    My Email
    By clicking on "Continue", I give neuvoo consent to process my data and to send me email alerts, as detailed in neuvoo's Privacy Policy . I may withdraw my consent or unsubscribe at any time.
    Application form