Data Engineering Lead

4 hours ago

Cape Town, South Africa Progressive Edge Full time

**Data Engineering Lead**

**Permanent**

**Cape Town**

A large established enterprise is looking to employ a Technical Lead : Data Engineering with deep technical knowledge the ability to lead and mentor a team of Data engineers. You will require substantial expertise in a broad range of software development & programming fields. Your knowledge of data analysis, end user requirements and business requirements analysis will help you develop a clear understanding of the business need and incorporate these needs into technical solutions. You will also need a solid understanding of physical database design and the SDLC and have the ability to work well with and lead a team.

**Responsibilities**:

- Architects Data analytics framework.
- Translates complex functional and technical requirements into detailed architecture, design, and high performing software.
- Leads Data and batch/real-time analytical solutions leveraging transformational technologies.
- Development and Operations
- Database Development and Operations
- Policies, Standards and Procedures
- Communications
- Business Continuity & Disaster Recovery
- Research and Evaluation
- Coaching/ Mentoring

**Qualifications**:

- Essential: 4 years Bachelors degree in computer science, computer engineering, or equivalent work experience AWS Certification at least to associate level

**Experience**:
Essential:

- 5+ years Data engineering or software engineering
- 3-5 years demonstrated experience leading teams of engineers
- 2+ years Big Data experience
- 5+ years experience with Extract Transform and Load (ETL) processes
- 2+ years Cloud AWS experience
- 5 years demonstrated experience with object oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and large scale data infrastructures.

**Knowledge & Skills**:
Essential:

- Creating data feeds from on-premise to AWS Cloud (2 years)
- Support data feeds in production on break fix basis (2 years)
- Creating data marts using Talend or similar ETL development tool (4 years)
- Manipulating data using python and pyspark (2 years)
- Processing data using the Hadoop paradigm particularly using EMR, AWSs distribution of Hadoop (2 years)
- DevOps for Big Data and Business Intelligence including automated testing and deployment (2 years)
- Extensive knowledge in different programming or scripting languages
- Expert knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases.

**Further technical skills required**:

- Capability to architect highly scalable distributed systems, using different open source tools.
- Big Data batch and streaming tools
- Talend (1 year)
- AWS: EMR, EC2, S3 (1 year)
- Python (1 year)
- PySpark or Spark (1 year) - Desirable
- Business Intelligence Data modelling (3 years)
- SQL (3 years)

  • Data Engineering Lead

    3 weeks ago

    Cape Town, South Africa Old Mutual Limited Full time

    **Let's Write Africa's Story Together!**Old Mutual is a firm believer in the African opportunity and our diverse talent reflects this.**Job Description**Collaborate in cross-functional teams to architect, design, build and maintain scalable data capabilities via data platform and data products. This entails taking a leading role in the end-to-end data...

  • Data Engineering Lead

    3 weeks ago

    Cape Town, South Africa Old Mutual Full time

    Let's Write Africa's Story Together! Old Mutual is a firm believer in the African opportunity and our diverse talent reflects this. Job Description Collaborate in cross‑functional teams to architect, design, build and maintain scalable data capabilities via data platform and data products. This entails taking a leading role in the end‑to‑end data...

  • Data Engineering Lead

    2 weeks ago

    Cape Town, South Africa Intelligent Debt Management Full time

    **Data Engineering Lead** **(Cape Town)** The Intelligent Debt Management (IDM) Group, South Africa’s largest and multi-award-winning debt management company, has a position available for a Data Engineering Lead. This individual will join the Data Team and will be accountable for mostly (but not limited to) data engineering design and implementation...

  • Lead Data Engineer

    5 days ago

    Cape Town, South Africa Ioco Full time

    In this role you will be responsible for developing and optimizing scalable data pipelines, ensuring high data quality, and managing robust data infrastructure to support reliable, efficient and cost-effective data delivery. It plays a key role in maintaining system performance, data integrity, and compliance with governance standards.This position supports...

  • Data Engineering Lead

    4 days ago

    Cape Town, South Africa iLaunch Full time

    Build a configurable data ingestion platform to collect and protect data Monitor team success Identify the capabilities and capacities from data science and research to design and engineering Matric/ Grade 12 National Diploma in an Information Technology / Bachelor's Degree in computer science, Statistics, Informatics, Information Systems Min 7+ years'...

  • Data Engineering Lead

    4 days ago

    Cape Town, South Africa Decision Inc Full time

    We are currently looking for someone based in Cape Town. a very knowledgeable in on-prem to cloud (snowflake) migration and data warehousing. **Who are we?** Decision Inc. is a global management and technology consulting organization. We help companies become Data Driven Businesses that are fit to take on the challenges of a digital world. Transforming...

  • Senior Team Lead, Software Engineering/Architecture

    1 week ago

    Cape Town, South Africa NTT DATA, Inc. Full time

    Senior Team Lead, Software Engineering/Architecture Join to apply for the Senior Team Lead, Software Engineering/Architecture role at NTT DATA, Inc. Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients...

  • Senior Team Lead, Software Engineering/Architecture

    1 week ago

    Cape Town, South Africa NTT DATA, Inc. Full time

    Senior Team Lead, Software Engineering/Architecture Join to apply for the Senior Team Lead, Software Engineering/Architecture role at NTT DATA, Inc. Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients...

  • Data Center Engineering Lead — Operations

    5 days ago

    Cape Town, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    A leading cloud services provider in Cape Town seeks an experienced Technical Operations Leader to manage maintenance and drive innovation in data center operations. The ideal candidate holds a Bachelor's degree in Electrical Engineering and has proven leadership experience. Key responsibilities include overseeing technical operations and leading teams....

  • Lead Electrical Engineer

    2 weeks ago

    Cape Town, South Africa Triz Engineering Solutions Full time

    Overview We’re growing fast! Join our expanding team as a Lead Electrical Engineer in Centurion. Be part of exciting new projects—apply today or share with your network. Triz Engineering Solutions is looking for a Electrical Lead Engineer to form part of our engineering team, with the ability to fit into the DNA of our company culture that is dynamic and...