Big Data Engineers (Scala / Spark) - Senior And Lead
Johannesburg, Gauteng
6d ago
source : findojobs-za

Quantium Founded in , Quantium combines the best of human and artificial intelligence to power possibilities for individuals, organisations and society.

Our solutions make sense of what has happened and what will, could or should be done to re-shape industries and societies around the needs of the people they serve.

Headquartered in Australia, Quantium has been expanding internationally for the last few years. Based on this momentum, weve made the decision to build one of our global Product & Technology Hubs in South Africa to compliment our thriving data science and AI consulting business.

We are setting-up a cross-functional team of product, technology, engineering specialists in Johannesburg and Cape Town. This new team will work closely with our clients locally (supporting both build and ongoing operations), work on projects for global clients and also directly contribute to our global product roadmaps for our flagship products.

The RoleWe're expanding Q.Refinery, our flagship transaction enrichment and analytics product, processing billions of transactions for leading banks in the US, Europe, Asia, South Africa and Australia.

Working with Scala, Spark and the rest of the Quantium tech ecosystem, youll be building applications to work with large, complex transactional data sets that provide meaningful insights to some of the worlds largest banks.

This is NOT a simple Spark ETL, it requires critically applied software engineering principles to scale. Nor are we functional purists, we write pragmatic functional code to solve unique complex problems that require out of the box thinking.

If you enjoy solving challenging engineering problems first and foremost then this could be the role for you.We're a cross-functional team of product managers, designers, analysts, testers, and of course engineers.

We're multi-lingual and don't do silos. You'll be an expert in your field, but keen to grow your "T" shape across our stack and other disciplines.

Apart from writing well-tested code, your day to day will involve : influencing what we build and how we build it,building up the big data engineering skill of the team, and in turn picking up other areas of our stack,participating in our agile ceremonies,performing code reviews,wrangling CI, CD, and local development to "sharpen the saw",supporting our customer engagement team, and keeping our production environment hummingFrom a stack perspective, Scala / Spark analytics pipelines, Snowflake query engine, .

NET back-end services, querying multi-tens-of-billion-row datasets in CosmosDB and Snowflake, populated by Scala and Spark.

We've adopted Kubernetes for orchestrating our core services and products, with a model driving builds, configuration and deployment.

There's ample problems for you to solve at every layer of our stack.The role will be a great fit if...You have experience developing applications using Apache Spark or similar big data technologiesYou have built Scala applications, preferably in distributed contexts (if you have a real love of Scala and are itching to move from Java, well also consider it)A solid foundation in functional and object-oriented programming with data structuresA passion for solving problems and writing efficient algorithmsAn awareness of considerations around structuring data on distributed systems to support analytic use casesA passion for delivering high-quality, peer-reviewed, well-tested codeA love for knowledge sharing, you know what works, but youre also happy to learn new methods and technologyYou're in-the-loop with industry trends, but focus on solving the right problem the right wayWhen it comes to architecture and solution design discussions, you can advocate your ideas and critique other's constructivelyYou have a passion for mentoring others, and in your own learning and developmentWhy should you be excited about this opportunity?

This is a unique opportunity to join a newly formed team, where you will have the opportunity to shape the culture and accelerate your career as we grow.

We have a flat structure that means YOU will have the opportunity to own solutions. The team will be operating on our latest tech stack and will work collaboratively with our global teams and in particular with our teams in Sydney and Manchester.

As a global and cross-industry business, mobility is core to our employee value proposition. We have defined pathways for our people to explore short and long term career opportunities in the Europe, Australia, New Zealand, India and the US.

Our benefits and perksCompetitive salaryAnnual bonus schemeGlobal work and development opportunitiesLots of social, health and wellbeing activities and events throughout the yearMore about QuantiumAs one of the worlds fully diversified data science and AI leaders we operate across every sector of the economy and were growing fast - with growth comes opportunity! Were passionate about building out our team of smart, fun, diverse and motivated people.

We combine a team of experts that spans data scientists, actuaries, statisticians, business analysts, strategy consultants, engineers, technologists, programmers, product developers, and futurists all dedicated to harnessing the power of data to drive transformational outcomes for our clients.

We actively foster a culture where our people can stretch themselves to reach their full potential. We also know that work has to work for you, and modern life is fast paced and balance can be tricky.

You want to work where you are respected and valued as an individual, not a number. Quantium embraces a flexible and supportive environment dedicated to powering possibilities for our team members, clients and partners.

Quantium are an equal opportunity employer and value diversity. We are a multicultural team of various beliefs, sexual orientations and gender identities.

We celebrate our diversity of race, religion, colour, national origin, gender, sexual orientation, age, marital status and disability status.

Please note, this role is not suitable for Graduates. If you are a graduate and youd like information about our Graduate Academy please visit If you submit an application or expression of interest to The Quantium Group Pty Ltd, we may disclose your application or expression of interest on a confidential basis to our related companies, and joint venture companies, including Quantium Health Pty Limited.

We will only make such disclosures for the purpose of considering your application or expression of interest and opportunities in those organisations.

Report this job

Thank you for reporting this job!

Your feedback will help us improve the quality of our services.

My Email
By clicking on "Continue", I give neuvoo consent to process my data and to send me email alerts, as detailed in neuvoo's Privacy Policy . I may withdraw my consent or unsubscribe at any time.
Application form