Data Engineering Lead

3 months ago

Cape Town, South Africa Intelligent Debt Management Full time

**Data Engineering Lead** **(Cape Town)**

The Intelligent Debt Management (IDM) Group, South Africa’s largest and multi-award-winning debt management company, has a position available for a Data Engineering Lead.

This individual will join the Data Team and will be accountable for mostly (but not limited to) data engineering design and implementation across on premise and cloud based platforms.

**Qualifications**:

- Bachelor's degree in Computer Science, Engineering, or a related field.
- Certifications in Azure, Azure Databricks Data Engineering or similar or equivalent practical experience.
- Extensive proven experience as a data engineer or in a similar role, designing and implementing data architectures, data integration and ETL processes. Practical experience in Azure, Azure Data Factory and Azure Databricks is preferred.
- Proven experience in Python/PySpark and SQL/SparkSQL programming skills.
- At least 4 years of experience in Information Technology - specifically in data management.
- Strong knowledge of data management and storage solutions, including SQL Server Database, Azure Data Lake Storage or similar.
- Ability to lead and innovate in a fast-paced and evolving technology landscape.
- Strong communication and leadership abilities.
- Excellent attention to detail and problem solving skills related to data, data structures and algorithms.

**Key responsibilities will include**:

- Lead the design and implementation of scalable data solutions on the Microsoft Azure cloud platform, including Azure Databricks.
- Develop, maintain and optimise robust data pipelines for data ingestion, processing, and transformation to support analytics, machine learning and other data-driven services.
- Ensure data security, quality and integrity across all of the pipelines and related data processes.
- Define the mechanism through which skilled decentralized analytics teams can contribute to the engineering processes.
- Work closely with the Head of Data and enterprise architecture to ensure that the data engineering vision is aligned with the systems architecture and the data strategy.
- Collaborate with the data product owner, analysts, data scientists, and business stakeholders to understand data requirements and implement effective data workflows.
- Assist in data initiatives and support technical teams including architects, software developers, and data scientists.
- Provide leadership and mentoring to a small, skilled team of data engineers.

**What’s in it for you?**

Being South Africa’s largest company of its kind - IDM is continuously growing, with no signs of stopping.

One of our core values is ‘growing people’ and we take that very seriously. We have internal learning opportunities to better our people, and leaders who are experts in their fields and freely impart knowledge and are open to new ideas. As one of our people said, _“you can speak to anyone here. Your rank doesn’t matter. You are listened to._”

Our culture is one of collaboration, diversity, and helpfulness. That goes for our people internally and how we deal with our clients.

At IDM, you can play a part in educating and assisting real South Africans get out of tough financial situations. You can make a difference with the work you do. Our people believe this too - one said, _“I wake up each day knowing I am going to make a difference.”_

Other key benefits:

- We make sure our employees are mentally and physically healthy with Healthy Together campaigns throughout the year.
- We offer various rewards and awards as tokens of recognition for hard work.
- We offer company benefits - like a half day off on your birthday, medical aid, retirement annuity, and further study opportunities should you meet relevant requirements.
- We have a culture of growth, meaning you have access to move both upwards or across in the business if you want to.

  • Engineering Lead

    3 weeks ago

    Cape Town, South Africa NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Engineering Lead

    2 weeks ago

    Cape Town, Western Cape, South Africa NTT DATA Full time

    About the RoleWe are seeking an exceptional Engineering Lead to join our team at NTT DATA. As a Platform Engineering Lead, you will play a critical role in designing, implementing, and maintaining our organization's technology platforms.Key ResponsibilitiesTechnical Leadership and Strategy: Lead the development and execution of the organization's platform...

  • Data Engineer

    4 weeks ago

    Cape Town, South Africa Axis Data Full time

    Axis Data is a US-based advanced data engineering and analytics consulting firm, and a certified partner of Palantir. We help clients implement and support the most advanced data operations platform: Palantir Foundry Role Overview: As a Senior Engineer at Axis Data, you will lead the development and deployment of data solutions, leveraging your expertise in...

  • Data Engineer

    6 months ago

    Cape Town, South Africa Dimension Data Full time

    As a Data Engineer, you'll design and build the data architecture of our products. You will use your technical knowledge to help us develop new products by taking an idea from concept to reality. You'll work on projects across multiple teams in order to achieve company goals and objectives. Your role will involve: - Analyzing business requirements; -...

  • Lead Data Engineer

    3 months ago

    Cape Town, South Africa IOCO Full time

    We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team in Cape Town. The ideal candidate will play a critical role in designing, developing, and optimizing our data infrastructure, ensuring it meets the needs of our growing business. As a Lead Data Engineer, you will lead a team of data professionals, drive data...

  • Lead Data Engineer

    3 months ago

    Cape Town, South Africa IOCO Full time

    We are seeking a highly skilled and experienced Lead Data Engineer to join our dynamic team in Cape Town. The ideal candidate will play a critical role in designing, developing, and optimizing our data infrastructure, ensuring it meets the needs of our growing business. As a Lead Data Engineer, you will lead a team of data professionals, drive data...

  • Data Engineering Lead

    2 weeks ago

    Cape Town, Western Cape, South Africa Acuity Consultants Full time

    About the RoleAcuity Consultants is seeking a highly skilled Senior Data Engineer to lead the design and development of scalable data pipelines for our music streaming service algorithmic music recommendation feature.This is a Cape Town-based position with a CTC of R1.4 – R1.6 million per annum.As Senior Data Engineer, you will work with cutting-edge data...

  • Data Engineering Lead

    2 months ago

    Cape Town, South Africa iLaunch Full time

    Build a configurable data ingestion platform to collect and protect data Monitor team success Identify the capabilities and capacities from data science and research to design and engineering Matric/ Grade 12 National Diploma in an Information Technology / Bachelor's Degree in computer science, Statistics, Informatics, Information Systems Min 7+ years'...

  • Data Engineering Lead

    5 days ago

    Cape Town, Western Cape, South Africa Planet10 Full time

    Job SummaryWe are seeking an experienced Data Engineering Lead to join our team at Planet10. As a key member of our marketing technology and data solutions team, you will play a pivotal role in the development and delivery of innovative solutions for our clients.

  • Technical Lead

    3 weeks ago

    Cape Town, Western Cape, South Africa IOCO Full time

    **Job Title:** Technical Lead - Data Engineering**About IOCO:**iOCO is a consultancy that solves complex real-world problems with innovative software solutions. We believe that anything is possible with modern technology, software, and development expertise. We are continuously pushing the boundaries of innovative solutions across multiple industries using...

  • Data Engineering Team Lead

    1 month ago

    Cape Town, Western Cape, South Africa wePlace Full time

    About the Role We are seeking a seasoned Data Engineering Team Lead to oversee the direction and lead a data warehouse team responsible for the full product lifecycle of software components in our central warehouse. The successful candidate will ensure the successful delivery of required functionality and maintenance-related tasks within the Bank Data...

  • Software Development Lead

    2 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    About the RoleWe are seeking a highly skilled Software Development Lead to join our team at Dimension Data. As a Technical Lead, you will play a key role in supporting the development and delivery of new features and functionality within our current projects.In this role, you will be responsible for taking ownership of assigned technical tasks, leading or...

  • Data Engineering Team Lead

    3 days ago

    Cape Town, Western Cape, South Africa wePlace Full time

    About the RoleWe are seeking an experienced Data Engineering Team Lead to lead our data warehouse team responsible for the full product lifecycle of software components in our central warehouse. The successful candidate will be responsible for ensuring the successful delivery of required functionality and maintenance-related tasks within Bank Data...

  • Data Engineering Lead

    4 weeks ago

    Cape Town, Western Cape, South Africa Acuity Consultants Full time

    About the RoleAcuity Consultants is seeking a highly skilled Data Engineer to join our team in Cape Town. As a Senior Data Engineer, you will play a key role in designing and building scalable data pipelines for our global music streaming services.Key ResponsibilitiesDesign and develop high-quality data pipelines using Apache Spark, Scala, and...

  • "Data Engineering Team Lead"

    3 weeks ago

    Cape Town, Western Cape, South Africa Progressive IT Resourcing Full time

    Job DetailsCompany Group Technology is responsible for the provision of digitally enabled technology services as a group COE, driving business and transformation and providing group-wide digital and data architecture.We operate various technology platforms and shared services, ensuring cyber and information security resilience and acting as technology...

  • Senior Data Engineering Lead

    1 month ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    Senior DataOps EngineerWe are seeking a highly skilled Senior DataOps Engineer to lead the development of DataOps practices within our organisation. As a seasoned DevOps professional with a strong background in software and data, you will be responsible for setting the direction for DataOps within the team, building scalable data pipelines, and ensuring...

  • Lead Data Architect

    3 weeks ago

    Cape Town, Western Cape, South Africa IOCO Full time

    Data Engineering Leadership RoleWe are seeking an experienced Data Engineer to lead our data engineering team and drive data initiatives that align with our business objectives.Key Responsibilities:Lead Data Strategy: Develop and implement data engineering strategies that support data-driven decision-making.Design and Architecture: Design and architect...

  • Lead Data Scientist

    1 month ago

    Cape Town, Western Cape, South Africa ShopriteX Full time

    Job Title: Lead Data ScientistAt ShopriteX, we are seeking a highly skilled and experienced Lead Data Scientist to join our Data & Insights Monetisation team. As a key member of our team, you will be responsible for leading the delivery of complex data science and engineering solutions that drive business value and provide a competitive advantage for our...

  • Data Lead

    5 months ago

    Cape Town, South Africa e-Merge IT Recruitment Full time

    A dynamic startup leveraging alternative data to provide unique insights into consumer behaviour for investors and corporate clients with a mission to deliver differentiated and in-depth consumer research, shaping investment decisions and strategic moves in the market. As a small yet ambitious team, they are seeking a talented individual to join as a **Data...

  • Delivery Lead: Data Engineering

    2 months ago

    Cape Town, South Africa wePlace Full time

    About the roleTo provide direction and lead a data warehouse team responsible for the full product lifecycle of the software components in the central warehouse, to ensure successful delivery of the required functionality and maintenance related tasks within the Bank Data Operations. Oversee the ongoing drive to increase efficiency, optimisation, innovation...