Intermediate and Senior Big Data / AWS Data Engineers
PBT Group
Johannesburg, Gauteng, ZA
5d ago

PBT Group has a requirement for a Big Data / AWS Data Engineer Developer to interpret requirements provided by business and produce effective Big Data solutions and to design and build AWS Cloud ETL pipelines for ingesting and processing of Big Data.

Programming exposure for code transformations and integrating the big data solution with existing systems. Develop information solutions from a variety of sources for both structured and unstructured data.

Technical ownership of Big Data solutions for structured and unstructured data.

Your duties will include working with the AWS data lake development team, developing and implementing solutions to facilitate movement of data, testing the solutions and performing third line support for developed solutions.

DUTIES :

  • Consult with business teams to understand data ingestion and processing requirements
  • Develop and implement big data models and solutions
  • Design and implement ETL methodologies and technologies and the integration with big data
  • Conduct root cause analysis on production issues
  • Technical leadership of entire information management process of both structured and unstructured data
  • Provide ongoing support and enhancement to ETL system
  • Optimization and the information solutions
  • Implementing machine learning algorithms
  • Configuration of the Hadoop infrastructure and environment for optimal performance
  • Integrate with statistical and actuarial analysts to build models
  • Producing relevant technical documentation and specifications
  • Estimate time and resource requirements for business requirement
  • Integration of big data solutions with existing reporting and analytical solutions
  • Develop data processing functions (DPF’s) using Java and Python
  • Experience :

  • Data Warehouse principles and practises
  • Big Data using Hadoop
  • Kimball Methodology
  • ETL development
  • Data Security and Protection Policies
  • Qualifications :

  • Matric Essential
  • National Diploma in IT (BTech)
  • Bachelor of Science (Information Systems, Computer Science, Mathematics) Advantageous
  • Report this job
    checkmark

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Apply
    My Email
    By clicking on "Continue", I give neuvoo consent to process my data and to send me email alerts, as detailed in neuvoo's Privacy Policy . I may withdraw my consent or unsubscribe at any time.
    Continue
    Application form