Lead Data Engineer for Large-Scale Projects

2 weeks ago


Cape Town, Western Cape, South Africa Network Recruitment Full time
Job Summary

We are seeking an experienced Senior Data Engineer to join our team at Network Recruitment. The successful candidate will have a proven track record of delivering complex data engineering projects, working with large-scale data sets, and maintaining high-quality data infrastructure.

About the Role

This is a fantastic opportunity for a motivated and skilled data engineer to take on new challenges, develop their skills, and contribute to the success of our clients. As a key member of our team, you will work closely with our technical experts to design, build, and maintain efficient data pipelines and architectures that meet the needs of our clients.

Key Responsibilities
  • Automate Workflows and Optimize Data Delivery: You will be responsible for designing, building, and maintaining automated workflows that optimize data delivery, ensuring timely and accurate data transfer.
  • Build and Maintain Efficient CI/CD Processes: You will work with our development teams to design, build, and maintain efficient Continuous Integration and Continuous Deployment (CI/CD) processes that enable seamless deployment of code changes.
  • Design Large, Complex Data Sets for Business Needs: You will collaborate with our business stakeholders to design and build large, complex data sets that meet business requirements, leveraging your expertise in data modeling and architecture.
  • Maintain Data Quality, Governance, and Infrastructure: You will be responsible for maintaining high-quality data infrastructure, including data warehousing, ETL, and data governance, ensuring data accuracy, consistency, and security.
Tech Stack

We use a range of technologies to deliver our solutions, including:

  • Python, PySpark, Palantir/Palantir Foundry
  • SQL, AWS, Airflow, Docker, Terraform
Requirements

To succeed in this role, you will need:

  • A Bachelor's degree in Computer Science or related field, or equivalent experience
  • At least 8 years of experience in data engineering, with 2+ years in big data pipelines
  • Hands-on experience with Python, PySpark, and Palantir/Palantir Foundry, including development on Palantir
  • Proficiency in SQL and cloud platforms (AWS/Azure/GCP)
  • Strong analytical skills and data management expertise
Estimated Salary Range:

$140,000 - $180,000 per year, depending on location and experience



  • Cape Town, Western Cape, South Africa The Building Company Full time

    About the JobWe are seeking a highly skilled Software Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale software applications.Main Responsibilities:Design and develop software applications using various programming languages.Collaborate with cross-functional...


  • Cape Town, Western Cape, South Africa Turner & Townsend Full time

    Job OverviewWe are seeking an experienced Strategic Project Director to lead our team in delivering large-scale commercial developments across Africa. As a key member of our leadership team, you will be responsible for ensuring the successful delivery of projects, exceeding client expectations, and driving business growth.About the RoleThe Strategic Project...


  • Cape Town, Western Cape, South Africa Capital Recruit Full time

    We are a leading renewable energy company in Cape Town, focused on delivering innovative solar power solutions for buildings. Our goal is to provide sustainable energy alternatives that reduce carbon emissions and lower energy costs for our clients.As a Senior Electrical Engineer, you will be responsible for leading the development and implementation of...


  • Cape Town, Western Cape, South Africa Dimension Data Full time

    About the RoleWe are seeking a highly skilled Software Development Lead to join our team at Dimension Data. As a Technical Lead, you will play a key role in supporting the development and delivery of new features and functionality within our current projects.In this role, you will be responsible for taking ownership of assigned technical tasks, leading or...


  • Cape Town, Western Cape, South Africa IOCO Full time

    We are seeking an experienced and dynamic project lead to manage complex transitions, particularly in the Financial Services/Investment experience and Technology sectors.Our ideal candidate will have a strong understanding of operating model design, insourcing strategies, and organizational restructuring.Please note that this is a 6-month renewable contract...


  • Cape Town, Western Cape, South Africa Dimension Data Full time

    Job DescriptionWe are seeking a highly skilled network engineer to join our team at Dimension Data. As a network engineer, you will be responsible for designing, implementing and supporting new IT projects in modern networks, security solutions and cloud technologies.The ideal candidate will have at least 2 years' experience in networking field and possess...


  • Cape Town, Western Cape, South Africa Dimension Data Full time

    Job OverviewThe Senior Project Manager role at Dimension Data requires a seasoned professional to lead and direct concurrent client projects classified as standard or complex. This position may also involve larger scale programmes and responsible for delivery across domains, business areas, geographies and technologies.


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    About the RoleNetwork Recruitment is seeking a highly skilled Senior Mechanical Project Lead to join their team.Job Description:We are looking for an experienced Mechanical Engineer with strong leadership skills to lead complex mechanical projects from conception to delivery. The ideal candidate will have a background in HVAC, Fire, and Wet services, as well...


  • Cape Town, Western Cape, South Africa OfferZen Ltd Full time

    At OfferZen Ltd, we're looking for a seasoned Senior Data Engineering Specialist to join our team. As a key member of our engineering team, you'll play a crucial role in designing, building, and maintaining our large-scale data systems.Key Responsibilities:Design, develop, and deploy large-scale data processing systemsCollaborate with cross-functional teams...


  • Cape Town, Western Cape, South Africa Impact tech Inc Full time

    Our CompanyAt Impact.tech Inc, we are passionate about creating technology that makes a difference. We strive to be at the forefront of innovation, using cutting-edge tools to solve real-world problems for our clients. Our teams work together to drive rapid growth and success, serving the largest brands in the world.We aim to create a culture where ambition...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    We are seeking a highly experienced Senior Mechanical Engineering Project Manager to lead our large-scale projects in the HVAC, Fire, and Wet services sector.About the RoleAs a Senior Mechanical Engineering Project Manager at Network Recruitment, you will be responsible for managing complex mechanical engineering projects from start to finish. This includes...


  • Cape Town, Western Cape, South Africa Shoprite Full time

    Job Title: Network Engineering LeadAbout the Role:The Network Engineering Lead will play a key role in the development, implementation, and maintenance of Shoprite's computer networks. This position requires a deep understanding of technical support and the ability to manage and lead user support, troubleshoot and resolve network problems, and configure,...


  • Cape Town, Western Cape, South Africa Planet10 Full time

    Job DescriptionWe are seeking an experienced Data Engineer Specialist to join our team at Planet10. As a key member of our data engineering practice, you will play a crucial role in designing and implementing scalable data solutions for our clients.Company OverviewAt Planet10, we strive to deliver innovative marketing technology and data solutions that drive...


  • Cape Town, Western Cape, South Africa Vantage Data Centers Full time

    Vantage Data Centres is a leading provider of hyperscale data centre infrastructure, operating across North America, EMEA and Asia Pacific. We power, cool, protect and connect the technology of the world's well-known hyperscalers, cloud providers and large enterprises.We are committed to being a workplace of inclusion, equity, respect and acceptance. Our...


  • Cape Town, Western Cape, South Africa DLK Group (PTY) Ltd Full time

    Senior Project Manager OpportunityWe are seeking a highly skilled Senior Project Manager to join our team at DLK Group (PTY) Ltd. The ideal candidate will have a strong understanding of project management principles and experience in managing large-scale projects in multi-disciplinary environments.Key Responsibilities:Lead and manage large-scale projects...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    Network RecruitmentWe are seeking a seasoned Civil Engineer to join our team as a High-Profile Civil Engineering Project Manager.Job Description:About the Role:The ideal candidate will possess a postgraduate degree in engineering and at least 8 years of experience in managing large-scale civil engineering projects. They should be proficient in using civil...

  • Project Coordinator

    1 month ago


    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job SummaryWe are seeking a skilled Project Coordinator to join our team at Dimension Data. As a Project Coordinator, you will be responsible for the day-to-day management of projects and teams. You will manage all aspects of project management including budgeting, timelines, and resource allocation.The ideal candidate will have an excellent understanding of...

  • Data Engineer

    4 days ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    Job Description:We are seeking an experienced Data Engineer to join our team at PBT Group. As a Data Engineer, you will play a key role in designing, building, and operationalizing large-scale enterprise data solutions and applications using AWS data and analytics services.The ideal candidate will have a strong background in data engineering, experience...


  • Cape Town, Western Cape, South Africa Lula Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at Lula. As a key member of our Data team, you will be responsible for implementing and maintaining data models, ensuring data reliability and quality, and developing scalable data pipelines.Key Responsibilities:Data Architecture: Design and develop data architecture...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    About the Role:Network Recruitment is seeking an experienced Senior Civil Infrastructure Project Manager to lead large-scale projects in the design and delivery of civil engineering infrastructure.About You:We are looking for a highly skilled project manager with a postgraduate degree in engineering and minimum 8 years of experience in managing civil...