Senior Data Engineer
Black Swan Data
Gauteng, South Africa
3d ago
source : findojobs-za

We are Black Swan Data In a turbulent and fast-changing marketplace, the old world of traditional consumer research is no longer enough.

Where there was slow and pre-emptive questionnaires and surveys, now there’s big data, machine learning and predictive analytics.

For some, it’s unknown territory. For Black Swan Data, it’s unprecedented possibilities.Black Swan Data is on a mission to revolutionise Insights & Innovation through the power of Social Prediction.

Using sophisticated data science and AI techniques, we unlock social data’s potential to understand, connect and predict consumer behaviour more accurately and further ahead than before.

Our TRENDSCOPE SaaS platform enables brands to apply Social Prediction to their business and deliver faster, better, more successful new product innovation launches.

It puts the power of trend prediction at our user’s fingertips and is central to how we turn trend intelligence into a competitive advantage for global giants like PepsiCo, Johnson & Johnson, P&G, Danone and Coty.

We’re fast movers. Since 2011, we’ve spread our wings across the UK, Hungary, USA and South Africa. Today our team of around 150 Swans (and counting) includes data scientists, developers, insights consultants, analysts, designers, engineers and marketeers.

But first, we’re entrepreneurs and champions of change who thrive on moving fast and breaking new ground. In 2015 we were awarded the Sunday Times #1 Fastest Growing Start-Up.

In 2016 we were named the UK’s Fastest Growing Exporter. And in both 2017 and 2018 we ranked high in the Financial Times 1000 Europe’s Fastest Growing Companies.

Since, we’ve been recognised by Deloitte Fast 50 and the London Stock Exchange for our cutting-edge technology and innovative approach.

Now is a great opportunity to join our rapidly growing data and technology SaaS business which is led by an experienced and inspiring management team.

We’re passionate about what we do and committed to disrupt and drive industry change, so if you’re an enthusiastic team player and go-getter we want to hear from you.

The best way to predict your future is to invent it. Peter Drucker What you’ll be doingWe are looking for a talented developer to join our team.

The role involves building and operating high performance big data pipelines to facilitate all our SaaS product for some of the world’s leading brands.

You'll be working closely within a team of developers and data scientists distributed remotely across the UK, South Africa and Hungary.

The role will involve building and operating an ingestion and analytics platform that collects data from social and internet sources through streaming or batch mechanisms.

Further to this we will expect the candidate to facilitate data processing needs incorporating automated data science to expose the enriched, normalised and aggregated data via APIs to our clients.

This includes data cleansing, aggregation and machine learning and deep learning.As well as working with the world’s leading big data technologies you will also work with data streaming architectures and micro-services to ingest data.

What you will learn from the role We try and automate as much as possible to streamline our processes. This includes continuous integration and deployment.

Quality is very important to us, and through automated testing we strive to keep the quality of our systems as high as possible.

Working with large sets of data and using distributed databases.Agile delivery process within squads made up from multiple skillsets.

The potential for learning and working in another language like Java, Scala or Python.End to end exposure to a cloud platform.

About youTo apply for the role, you’ll need : To be the type of developer that can pick up a new technology quickly and deliver features in a highly agile manner.

Experience writing testable functional Scala in a production grade system.To have used Apache Spark in a production system using ScalaHaving experience in Spark orchestration tools such as Apache AirflowExperience of using a cloud platform to architect and build data pipelines.

To easily navigate the administration of a Hadoop cluster on a cloud platform such as DatabricksTo have used Docker containers to deploy your systemsHaving a strong JVM development background including use of SpringHaving worked with data streaming for example Kafka You’ll be part of a dynamic and growing global team As we continue to grow across the world, you’ll find every day brings with it fresh challenges and opportunities to try new things.

You’ll be part of a diverse, smart, agile but laid-back team passionate about technology, who are also open-minded and always open to new ideas.

Because we’re serious about finding ways to do things better. Which means you’ll find serious hardware. And no red tape.

Or unnecessary process. Just less noise, less disruption and more time to focus.So you’ll get the freedom to be yourself, to do the work you want to do and the freedom to grow.

You’ll be trusted to get on with things and won’t have someone checking up on you 24 / 7. But you’ll also be supported, coached, advised and mentored.

With constant opportunities to learn.You’ll be well looked afterNot only will you get to work with a great team, you’ll also enjoy flexible hours, the flexibility to work from home on a remote contract.

We recognise and reward hard work and we’re looking for someone to grow and go places with usYou know you’ll fit right inThings move quickly at Black Swan Data.

And so do you. You’re bright, curious and optimistic, quick to pick up new concepts and open to learning new things. You love a challenge and getting stuck right in, ready to step up and run with anything.

You’re happy to fly solo and also enjoy being part of a team. Which means you’re adaptable and easy-going. And a good communicator.

You have a great eye for detail but can also see the bigger picture. And you have that magical ability to balance perfection and delivery.

You get things done. Because you give 110%. As do we. What makes a Swan?We want people fanatical about creating innovative solutions inspired by data.

We know Black Swan’s DNA is different, it’s what helps us find new perspectives on our clients’ problems. Working in a fast paced, energized environment, our bar is set high, and an ability to build long lasting partnerships both internally and externally is vital.

We have made a difference at our core and that is embedded in our values. We believe in long-term partnerships through transparency and honestyWe believe in leaving the world in a better place than we found itWe believe anyone can achieve great things.

We do not put people in boxesWe listen more than we talkWe think 10% crazy is just about right!Most importantly, and because you spend 60% of your waking time at work, we believe you should enjoy it

Report this job

Thank you for reporting this job!

Your feedback will help us improve the quality of our services.

My Email
By clicking on "Continue", I give neuvoo consent to process my data and to send me email alerts, as detailed in neuvoo's Privacy Policy . I may withdraw my consent or unsubscribe at any time.
Application form