Standard Bank is a firm believer in technical innovation, to help us guarantee exceptional client service and leading edge financial solutions.
Our growing global success reflects our commitment to the latest solutions, the best people, and a uniquely flexible and vibrant working culture.
To help us drive our success into the future, we are looking for a Data Engineer, Elasticsearch with the ability to execute on our Data Journey End to End.
The purpose of this role is to develop solutions around data acquisition, ingestion and reporting using the full capabilities of the Elastic Stack (elastic.
co). The role is expected to provide technical ETL solutions in response to the needs of stakeholders (making use of Logstash, Beats and Kafka) by interpreting business requirements;
defining solution; defining build and test tasks; performing testing; participating in the deployment of solutions; provide third tier support and understanding of Elasticsearch.
The role requires good knowledge of DevOps practices in an agile team with T-roles; clearly define tasks with delivery effort estimates;
contribute effectively to lean management practices and iteration ceremonies in the Data and AI : Big Data, Fast Data Feature Team portfolio
Key Responsibilities / Accountabilities
Define tasks and define the technology components that best deliver to user stories, in the context of epics, by conducting the following activities :
Ensure tasks are clearly scoped, with acceptance criteria, to ensure clarity of what is required
Define and performing appropriate unit and integration tests to ensure no errors are produced during and after deployments
Define tasks with effective sizing of the effort required to deliver to contribute to predictable delivery and velocity management
Provide documentation to reflect the solution design and capture data management artefacts
Incorporate standard requirements such as data quality control and future proofing into task definition and delivery with assistance from the rest of the DevOps team
Ensure applicable standards are addressed in the solution design
Have a good understanding of JSON field mapping and templates in the context of Elasticsearch
Construct end to end data service solutions : Outputs and measures
Identify data requirements and construct / extend data solutions (including ETL) to meet business requirements by conducting the following activities :
Make use of existing mechanisms, tools and agents provided by the Elastic Stack to ingest data from a range of sources with controls that deliver to required data quality
Apply appropriate continuous integration practices to prevent defects
Have a good understanding of the Logstash language and some Python / Ruby concepts
Apply DevOps testing practices as defined for the feature team
Test the solution in terms of the master test plan to ensure no defects are identified post deployment
Meet all governance requirements
Document the solution sufficiently
Engage with the other team members and other teams across organisation to achieve delivery objectives
Deliver solutions that are robust and future-proofed to reduce waste and manage technology deficit
Perform work aligned with the team process; meeting governance requirements; and utilising the Continuous Integration toolset
Team based delivery is the focus of feature team, as opposed to individual performance. This requires the DE to participate in team practices and deliver to team standards :
Participate in team ceremonies
Perform work which is traditionally performed by non-engineering roles which requires knowledge of other traditional roles (such as business analysis and testing)
Contribute to team performance by identifying and implementing team improvements
Attend and participate in Elasticsearch Meet-ups
Keep up-to-date with online webinars and features of the Elastic Stack
Provide third tier support and enhancement services : Outputs and measures
Provide third tier support and guidance on Elasticsearch semantics, queries, analytics and visualizations
Maintain in-depth knowledge of the technology design of the Elastic Stack
Understand accurately and completely the impact of an incident on system functionality and business
Achieve quick response times to incidents through effective cause diagnosis using the Elasticsearch API and Kibana
Prevent recurrence of incidents by identifying and effectively responding to root causes through effective cause diagnosis
Clear and specific communication to users and other role players regarding the impact and resolution of an incident
Ensure that all incident, problem and change management requirements are met
Deliver enhancements to the system as required by business or in response to opportunities to address technology deficit or enhance solution performance
Preferred Qualification and Experience
First Degree in IT and Computer Sciences
Developed solutions using one of the following languages : Python, Java, Ruby
Deployed solutions on Linux using Docker, Ansbile / Terraform / Other
Elasticsearch, Logstash, Kibana
Operating within an agile team
Working with security data in Financial Services industry
Knowledge / Technical Skills / Expertise
The design, creation, testing and documenting of new and amended programs from supplied specifications in accordance with agreed standards.
The ability to analyse the behaviour of code to diagnose a problem and find the underlying cause. This includes but is not limited to using a debugger.
The ability to revise existing code without impacting its functional behaviour.
The ability to use script builders as well as other related automation like continuous integration, automated deployments, and static code analysis tools.