Data Engineering Lead

2 weeks ago


Cape Town, South Africa Progressive Edge Full time

**Data Engineering Lead**

**Permanent**

**Cape Town**

A large established enterprise is looking to employ a Technical Lead : Data Engineering with deep technical knowledge the ability to lead and mentor a team of Data engineers. You will require substantial expertise in a broad range of software development & programming fields. Your knowledge of data analysis, end user requirements and business requirements analysis will help you develop a clear understanding of the business need and incorporate these needs into technical solutions. You will also need a solid understanding of physical database design and the SDLC and have the ability to work well with and lead a team.

**Responsibilities**:

- Architects Data analytics framework.
- Translates complex functional and technical requirements into detailed architecture, design, and high performing software.
- Leads Data and batch/real-time analytical solutions leveraging transformational technologies.
- Development and Operations
- Database Development and Operations
- Policies, Standards and Procedures
- Communications
- Business Continuity & Disaster Recovery
- Research and Evaluation
- Coaching/ Mentoring

**Qualifications**:

- Essential: 4 years Bachelors degree in computer science, computer engineering, or equivalent work experience AWS Certification at least to associate level

**Experience**:
Essential:

- 5+ years Data engineering or software engineering
- 3-5 years demonstrated experience leading teams of engineers
- 2+ years Big Data experience
- 5+ years experience with Extract Transform and Load (ETL) processes
- 2+ years Cloud AWS experience
- 5 years demonstrated experience with object oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and large scale data infrastructures.

**Knowledge & Skills**:
Essential:

- Creating data feeds from on-premise to AWS Cloud (2 years)
- Support data feeds in production on break fix basis (2 years)
- Creating data marts using Talend or similar ETL development tool (4 years)
- Manipulating data using python and pyspark (2 years)
- Processing data using the Hadoop paradigm particularly using EMR, AWSs distribution of Hadoop (2 years)
- DevOps for Big Data and Business Intelligence including automated testing and deployment (2 years)
- Extensive knowledge in different programming or scripting languages
- Expert knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases.

**Further technical skills required**:

- Capability to architect highly scalable distributed systems, using different open source tools.
- Big Data batch and streaming tools
- Talend (1 year)
- AWS: EMR, EC2, S3 (1 year)
- Python (1 year)
- PySpark or Spark (1 year) - Desirable
- Business Intelligence Data modelling (3 years)
- SQL (3 years)


  • Data Engineer

    4 weeks ago


    Cape Town, South Africa Dimension Data Full time

    As a Data Engineer, you'll design and build the data architecture of our products. You will use your technical knowledge to help us develop new products by taking an idea from concept to reality. You'll work on projects across multiple teams in order to achieve company goals and objectives. Your role will involve: - Analyzing business requirements; -...


  • Cape Town, South Africa Dimension Data Full time

    As a Lead Application Developer, you'll be responsible for the development of high-level applications using Java and .Net Framework. You will also provide technical support to other members of the team by helping them troubleshoot issues with their application software. You should have experience developing web based applications that use JavaScript, JQuery...

  • Lead Data Architect

    5 days ago


    Cape Town, South Africa iLaunch Full time

    Architect cloud solutions for data integration and storage Establishing methods and procedures for tracking data quality, completeness, redundancy, compliance and improvement. Creating plans for data capacity planning, data security, life cycle data management, scalability, backup, disaster recovery, business continuity, and archiving. Oversee the mapping...

  • Data Solutions Lead

    1 week ago


    Cape Town, South Africa Incubeta Full time

    Founded in 2001, we are digital marketing experts who help brands to use digital to upgrade their growth. We are Google's longest serving European partner with a 20 year long relationship and have worked with them to deliver market-leading strategies for some of the world's biggest companies. Our work has been recognised through several awards, achieving 11...

  • IT Data Engineer

    4 weeks ago


    Cape Town, South Africa Armstrong Appointments Full time

    Our client based in Cape Town are currently seeking the skills of an IT Data Engineer to assist in designing and implementing scalable and robust processes to support the data engineering capability. This role will be responsible for extracting and transforming massive amounts of data at scale and consolidating this data into a bigger data ecosystem. Main...


  • Cape Town, South Africa Boardroom Appointments Full time

    Key Roles and Responsibilities:Technical Leadership and Strategy: Lead the development and execution of the organization's platform engineering strategy. Provide technical leadership and guidance to the platform engineering team, ensuring alignment with business goals.Team Management: Manage and mentor a team of platform engineers, fostering a collaborative...

  • AWS Data Engineer

    4 weeks ago


    Cape Town, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • Lead Engineer

    2 days ago


    Cape Town, South Africa Pronel Personnel Full time

    Our client based in Cape Town (CBD) is currently seeking a Lead Engineer to join their innovative team. The Lead Engineer will be responsible for leading technical initiatives, driving engineering excellence and mentoring team members to deliver high-quality solutions. The role requires a strong technical background, exceptional leadership skills and a...

  • Lead Engineer

    6 days ago


    Cape Town, South Africa Pronel Personnel Full time

    Our client based in Cape Town (CBD) is currently seeking a Lead Engineer to join their innovative team. The Lead Engineer will be responsible for leading technical initiatives, driving engineering excellence and mentoring team members to deliver high-quality solutions. The role requires a strong technical background, exceptional leadership skills and a...

  • Lead Engineer

    5 days ago


    Cape Town, South Africa Adzuna ZA B C2 Full time

    Our client based in Cape Town (CBD) is currently seeking a Lead Engineer to join their innovative team. The Lead Engineer will be responsible for leading technical initiatives, driving engineering excellence and mentoring team members to deliver high-quality solutions. The role requires a strong technical background, exceptional leadership skills and a...

  • Lead Engineer

    5 days ago


    Cape Town, South Africa Pronel Personnel Full time

    Our client based in Cape Town (CBD) is currently seeking a Lead Engineer to join their innovative team. The Lead Engineer will be responsible for leading technical initiatives, driving engineering excellence and mentoring team members to deliver high-quality solutions. The role requires a strong technical background, exceptional leadership skills and a...

  • MS Engineer

    4 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking for a new challenge in your current role? Are you looking to join a team of highly talented colleagues who love challenges and helping customers succeed? Do you want to work in one of the most dynamic environments in the region? This is your opportunity!The role of a Technical Lead involves supporting the development and...

  • Data Architect

    3 days ago


    Cape Town, South Africa Progressive Edge Full time

    **Data Architect & Team Lead** **Permanent** **Cape Town** Job description: We are looking for a data expert who is able to confidently move from architectural definition and technical guidance to hands-on implementation and troubleshooting. The right person for this role will be a people Leader, who leads by technical example and has excellent management...

  • Data Engineer

    4 weeks ago


    Cape Town, South Africa 60degrees Full time

    Job Description Data Engineer (Gauteng) Compensation: R 450 000 – R 550 000 per annum Working model: Hybrid The opportunity that awaits you: Our client, a solutions energy engineering solution provider with projects mainly in Sub-Saharan Africa seeks to expand their team. Our client takes pride in not just ticking a box but...

  • Data Engineer

    3 weeks ago


    Cape Town, South Africa DLK Group (PTY) Ltd Full time

    ·        Modifying the database structure, as necessary, from information given by application developers.·        Enrolling users and maintaining system security.·        Controlling and monitoring user access to the database.·        Monitoring and optimizing the performance of the database.·        Maintaining...

  • Data Engineer

    4 weeks ago


    Cape Town, South Africa DLK Group Full time

    Job Description ·Modifying the database structure, asnecessary, from information given by application developers. ·Enrolling users and maintaining systemsecurity. ·Controlling and monitoring user accessto the database. ·Monitoring and optimizing the performanceof the database. ·Maintaining archived data. ·Contacting database vendor...

  • L2 Network Engineer

    4 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of micro-managing your work? Do you want to be part of a team where it is easy to make a difference? My client instructs me to find the right people for this L2 Network Engineer role!As a L2 Network Engineer, you'll be responsible for helping to maintain and upgrade the company's networks. You will perform regular network...

  • Engineering Manager

    4 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingThe Engineering Manager provides operational management and guidance to a team of Engineers / Technicians who are accountable for installation, configuration and fault management. This position provides technical support to NTT (Ltd) clients.Manages teams of incumbents responsible for setup and installation of technical systems,...

  • Network Engineer L1

    4 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingNetwork Engineer L1LOCATION : Cape Town, South AfricaTYPE : TemporaryDo you want to join a company that values your opinion? Do you want to work on the same project for years, becoming more and more involved in the process? Do you enjoy working with people from all over the world? Check out this Senior Network Engineer L1 role!As a Network...

  • Data Engineer

    2 weeks ago


    Cape Town, South Africa IOCO Full time

    We are seeking an experienced Data Migration Engineer to join our team on a contract basis to facilitate this migration process. The ideal candidates will have strong expertise in data engineering, particularly with Azure Synapse services, SSIS, and API integration.   What you’ll do: Collaborate with the existing team to understand the current data...