Data Engineering Lead

2 months ago

Cape Town, South Africa Intelligent Debt Management Full time

**Data Engineering Lead** **(Cape Town)**

The Intelligent Debt Management (IDM) Group, South Africa’s largest and multi-award-winning debt management company, has a position available for a Data Engineering Lead.

This individual will join the Data Team and will be accountable for mostly (but not limited to) data engineering design and implementation across on premise and cloud based platforms.

**Qualifications**:

- Bachelor's degree in Computer Science, Engineering, or a related field.
- Certifications in Azure, Azure Databricks Data Engineering or similar or equivalent practical experience.
- Extensive proven experience as a data engineer or in a similar role, designing and implementing data architectures, data integration and ETL processes. Practical experience in Azure, Azure Data Factory and Azure Databricks is preferred.
- Proven experience in Python/PySpark and SQL/SparkSQL programming skills.
- At least 4 years of experience in Information Technology - specifically in data management.
- Strong knowledge of data management and storage solutions, including SQL Server Database, Azure Data Lake Storage or similar.
- Ability to lead and innovate in a fast-paced and evolving technology landscape.
- Strong communication and leadership abilities.
- Excellent attention to detail and problem solving skills related to data, data structures and algorithms.

**Key responsibilities will include**:

- Lead the design and implementation of scalable data solutions on the Microsoft Azure cloud platform, including Azure Databricks.
- Develop, maintain and optimise robust data pipelines for data ingestion, processing, and transformation to support analytics, machine learning and other data-driven services.
- Ensure data security, quality and integrity across all of the pipelines and related data processes.
- Define the mechanism through which skilled decentralized analytics teams can contribute to the engineering processes.
- Work closely with the Head of Data and enterprise architecture to ensure that the data engineering vision is aligned with the systems architecture and the data strategy.
- Collaborate with the data product owner, analysts, data scientists, and business stakeholders to understand data requirements and implement effective data workflows.
- Assist in data initiatives and support technical teams including architects, software developers, and data scientists.
- Provide leadership and mentoring to a small, skilled team of data engineers.

**What’s in it for you?**

Being South Africa’s largest company of its kind - IDM is continuously growing, with no signs of stopping.

One of our core values is ‘growing people’ and we take that very seriously. We have internal learning opportunities to better our people, and leaders who are experts in their fields and freely impart knowledge and are open to new ideas. As one of our people said, _“you can speak to anyone here. Your rank doesn’t matter. You are listened to._”

Our culture is one of collaboration, diversity, and helpfulness. That goes for our people internally and how we deal with our clients.

At IDM, you can play a part in educating and assisting real South Africans get out of tough financial situations. You can make a difference with the work you do. Our people believe this too - one said, _“I wake up each day knowing I am going to make a difference.”_

Other key benefits:

- We make sure our employees are mentally and physically healthy with Healthy Together campaigns throughout the year.
- We offer various rewards and awards as tokens of recognition for hard work.
- We offer company benefits - like a half day off on your birthday, medical aid, retirement annuity, and further study opportunities should you meet relevant requirements.
- We have a culture of growth, meaning you have access to move both upwards or across in the business if you want to.

  • Electrical Lead Engineer

    2 weeks ago

    Cape Town, South Africa Lead Discovery GmbH Full time

    Triz Engineering Solutions is looking for an Electrical Lead Engineer to form part of our engineering team, with the ability to fit into the DNA of our company culture that is dynamic and embraces change. Apart from a contained and well-rounded person with a high EQ level, he/she will be required to deal with complexity in a fast-paced environment. The...

  • Data Engineer

    7 days ago

    Cape Town, South Africa Axis Data Full time

    Axis Data is a US-based advanced data engineering and analytics consulting firm, and a certified partner of Palantir. We help clients implement and support the most advanced data operations platform: Palantir Foundry Role Overview: As a Senior Engineer at Axis Data, you will lead the development and deployment of data solutions, leveraging your expertise in...

  • Data Engineer

    5 months ago

    Cape Town, South Africa Dimension Data Full time

    As a Data Engineer, you'll design and build the data architecture of our products. You will use your technical knowledge to help us develop new products by taking an idea from concept to reality. You'll work on projects across multiple teams in order to achieve company goals and objectives. Your role will involve: - Analyzing business requirements; -...

  • Lead Data Engineer

    2 months ago

    Cape Town, South Africa Adzuna ZA B C2 Full time

    We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team in Cape Town. The ideal candidate will play a critical role in designing, developing, and optimizing our data infrastructure, ensuring it meets the needs of our growing business. As a Lead Data Engineer, you will lead a team of data professionals, drive data...

  • Lead Data Engineer

    2 months ago

    Cape Town, South Africa IOCO Full time

    We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team in Cape Town. The ideal candidate will play a critical role in designing, developing, and optimizing our data infrastructure, ensuring it meets the needs of our growing business. As a Lead Data Engineer, you will lead a team of data professionals, drive data...

  • Lead Data Engineer

    2 months ago

    Cape Town, South Africa IOCO Full time

    We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team in Cape Town. The ideal candidate will play a critical role in designing, developing, and optimizing our data infrastructure, ensuring it meets the needs of our growing business. As a Lead Data Engineer, you will lead a team of data professionals, drive data...

  • Data Engineering Lead

    4 weeks ago

    Cape Town, South Africa iLaunch Full time

    Build a configurable data ingestion platform to collect and protect data Monitor team success Identify the capabilities and capacities from data science and research to design and engineering Matric/ Grade 12 National Diploma in an Information Technology / Bachelor's Degree in computer science, Statistics, Informatics, Information Systems Min 7+ years'...

  • Lead Data Engineer AWS

    1 month ago

    Cape Town, Western Cape, South Africa Parvana Full time

    About ParvanaParvana is a leading payments company dedicated to transforming the payment landscape by providing a risk-free and empowering payment solution. Our mission is to alleviate debt burdens, enabling shoppers to pay as they earn, free from the threat of accumulating charges. We're committed to providing a flexible environment with ample opportunity...

  • Avaya Engineer

    2 months ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job Title: Avaya Engineer - Dimension DataWe are seeking a highly skilled Avaya Engineer to join our team at Dimension Data. As an Avaya Engineer, you will be responsible for providing technical support and expertise to our clients and internal teams.Key Responsibilities:Proactively monitor and resolve technical incidents and problems to ensure minimal...

  • Data Engineering Team Lead

    2 weeks ago

    Cape Town, Western Cape, South Africa wePlace Full time

    About the Role We are seeking a seasoned Data Engineering Team Lead to oversee the direction and lead a data warehouse team responsible for the full product lifecycle of software components in our central warehouse. The successful candidate will ensure the successful delivery of required functionality and maintenance-related tasks within the Bank Data...

  • Data Engineering Team Lead

    3 weeks ago

    Cape Town, Western Cape, South Africa wePlace Full time

    About the RoleWe are seeking a highly skilled Data Engineering Team Lead to join our team at wePlace. As a key member of our data warehouse team, you will be responsible for providing direction and leading a team of engineers responsible for the full product lifecycle of software components in our central warehouse.Your primary objective will be to ensure...

  • Data Engineering Lead

    1 day ago

    Cape Town, Western Cape, South Africa Acuity Consultants Full time

    About the RoleAcuity Consultants is seeking a highly skilled Data Engineer to join our team in Cape Town. As a Senior Data Engineer, you will play a key role in designing and building scalable data pipelines for our global music streaming services.Key ResponsibilitiesDesign and develop high-quality data pipelines using Apache Spark, Scala, and...

  • Technical Lead

    4 weeks ago

    Cape Town, Western Cape, South Africa Sanlam Full time

    About the RoleWe are seeking a highly skilled and experienced Technical Lead to join our Data and AI Engineering team at Sanlam Group. As a Technical Lead, you will be responsible for leading the development and implementation of data and AI solutions that drive business growth and transformation.Key ResponsibilitiesLead the development and implementation of...

  • Lead Data Engineer AWS

    1 month ago

    Cape Town, South Africa Parvana Full time

    About our client:Our client is a leading payments company dedicated to transforming the payment landscape by providing a riskfree and empowering payment solution. Their payment solution aims to alleviate debt burdens enabling shoppers to pay as they earn free from the threat of accumulating charges. You will be working on the latest tech with ample...

  • Lead Data Engineer AWS

    1 month ago

    Cape Town, Western Cape, South Africa Parvana Full time

    About Parvana:Parvana is a leading payments company dedicated to transforming the payment landscape by providing a risk-free and empowering payment solution. Our payment solution aims to alleviate debt burdens, enabling shoppers to pay as they earn, free from the threat of accumulating charges. You will be working on the latest tech with ample opportunity...

  • Lead Data Engineer AWS

    3 weeks ago

    Cape Town, Western Cape, South Africa Parvana Full time

    About Parvana:Parvana is a leading payments company dedicated to transforming the payment landscape by providing a risk-free and empowering payment solution. Our payment solution aims to alleviate debt burdens, enabling shoppers to pay as they earn, free from the threat of accumulating charges. You will be working on the latest tech with ample opportunity...

  • Senior Data Engineering Lead

    2 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    Senior DataOps EngineerWe are seeking a highly skilled Senior DataOps Engineer to lead the development of DataOps practices within our organisation. As a seasoned DevOps professional with a strong background in software and data, you will be responsible for setting the direction for DataOps within the team, building scalable data pipelines, and ensuring...

  • Lead Data Scientist

    2 weeks ago

    Cape Town, Western Cape, South Africa ShopriteX Full time

    Job Title: Lead Data ScientistAt ShopriteX, we are seeking a highly skilled and experienced Lead Data Scientist to join our Data & Insights Monetisation team. As a key member of our team, you will be responsible for leading the delivery of complex data science and engineering solutions that drive business value and provide a competitive advantage for our...

  • Data Lead

    4 months ago

    Cape Town, South Africa e-Merge IT Recruitment Full time

    A dynamic startup leveraging alternative data to provide unique insights into consumer behaviour for investors and corporate clients with a mission to deliver differentiated and in-depth consumer research, shaping investment decisions and strategic moves in the market. As a small yet ambitious team, they are seeking a talented individual to join as a **Data...

  • Lead Data Scientist

    2 weeks ago

    Cape Town, Western Cape, South Africa ShopriteX Full time

    Job Title: Lead Data ScientistShopriteX is seeking a highly skilled Lead Data Scientist to join our Data & Insights Monetisation team. As a key member of our team, you will be responsible for leading the delivery of complex data science and engineering solutions that drive business value and provide a competitive advantage for Shoprite.Key...