Data Engineering

5 days ago


Cape Town, Western Cape, South Africa PBT Group Full time

The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued growth through efficient data handling. The role also includes collaborating with multiple teams to understand data needs and translate them into scalable platforms and services.

Duties & Responsibilities

Operational Delivery

  • Build, enhance, and maintain real-time data pipelines.
  • Work with infrastructure and operations teams to maintain data infrastructure.
  • Identify, document, and resolve feature gaps while implementing solutions.
  • Modernize and maintain services and tooling to ensure resilience, fix data discrepancies, and enhance customer experience.
  • Monitor daily execution, diagnose and log issues, and fix pipelines to ensure SLAs are met.
  • Mentor junior engineers through code reviews, best practices, and knowledge sharing.

Technical Leadership

  • Participate in the engineering community of practice.
  • Share AWS knowledge and practical experience within the community.
  • Challenge and contribute to the development of architectural principles and patterns.

Compliance

  • Ensure solutions adhere to Olympus patterns, guidelines, and standards.
  • Operate within project environments and participate in continuous improvement efforts.

Delivery Management

  • Follow and participate in Agile methodologies such as sprint planning, backlog grooming, retrospectives, demos, and PI planning.

Skills & Experience Required

Essential:

  • Strong AWS Data Engineering experience.
  • Hands-on experience with Kafka and Confluent.
  • Strong expertise in real-time data streaming tools such as Kafka & Flink.
  • Proficiency in:
    • Apache Spark
    • Apache Storm
    • Cloudera Data Flow
    • Databricks
    • Hadoop
    • AWS Kinesis Streaming
  • Experience in developing cloud-based solutions.
  • 2-3 years of experience designing and developing streaming data pipelines for ingestion or transformation using AWS technologies.
  • Experience with distributed log systems such as Kafka and AWS Kinesis.

Qualifications & Certifications

  • A degree in Computer Science, Information Systems, Business Administration, Commerce, or equivalent.
  • Bachelor's Degree in Computer Science, Big Data, or similar fields.
  • AWS Data Engineer Certification (Advantageous).
  • Relevant technical certifications (Preferred).

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

** If you have not received any feedback after 2 weeks, please consider you application as unsuccessful.


  • Data Engineer

    4 weeks ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    ? Join PBT Group as a Senior Data Engineer ? Are you a dynamic data engineer passionate about optimising large-scale data ecosystems? PBT Group is looking for a Data Engineer to design and implement robust, scalable processes that support our growing data engineering capability. This role is pivotal in advancing our cloud engineering and data management...

  • Data Engineer

    3 weeks ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    ? Join PBT Group as a Senior Data Engineer ?Are you a dynamic data engineer passionate about optimising large-scale data ecosystems? PBT Group is looking for a Data Engineer to design and implement robust, scalable processes that support our growing data engineering capability. This role is pivotal in advancing our cloud engineering and data management...

  • Data Engineer

    1 week ago


    Cape Town, Western Cape, South Africa Datonomy Solutions Full time

    Job Overview:We are seeking an experienced Data Engineer with strong Solution Architecture expertise to lead the data engineering efforts for a financial data migration project. The role focuses on designing, building, and optimizing data pipelines to support the transition from an on-premise Hyperion system to OneStream in the cloud. The ideal candidate...


  • Cape Town, Western Cape, South Africa Data Centrix Full time

    Data Centrix is committed to delivering exceptional customer service, and we are seeking a talented Technical Support Engineer to join our team. As a key member of our technical operations group, you will be responsible for providing high-level support to our customers, ensuring that their technical queries are resolved efficiently and effectively.Key...

  • Data Engineer

    2 weeks ago


    Cape Town, Western Cape, South Africa Goldman Tech Resourcing Full time

    A company that is built around innovative work, great people, and a singular focus on the world of commerce is seeking a Data Engineer. Candidates to be based in Cape Town, however there is a hybrid working model. Skills Required · BSc in Computer Science, Computer Engineering, or a related field. · 3 years of experience in data engineering or software...

  • Data Engineer

    3 weeks ago


    Cape Town, Western Cape, South Africa Goldman Tech Resourcing Full time

    An innovative retail media solutions provider is looking for a Data Engineer to join their team. This company specializes in dynamic digital display networks and strategic in-store advertising, empowering brands to maximize visibility and customer engagement through cutting-edge technology and impactful campaigns. Skills Required · BSc in Computer Science,...

  • Data Engineer

    2 weeks ago


    Cape Town, Western Cape, South Africa Progressive IT Resourcing Full time

    Job DetailsWho are we?Company Group Technology is responsible for the provision of a digitally enabled technology service as a group COE, drive business and transformation and provide group-wide digital and data architecture. We operate the various technology platforms and shared services, ensure Cyber and Information Security resilience, and act as...


  • Cape Town, Western Cape, South Africa Data Centrix Full time

    Qualifications:Matric qualificationTertiary qualification (Computer Science or IT related)Certifications CCNA, CCNP, JNCIAValid Drivers License and own vehicleExperience: Must have professional OME certification, working knowledge and experience of 5+ yearsSolid background in network administration and architectureIn-depth understanding of TCPIP and the...


  • Cape Town, Western Cape, South Africa BADRI Management Consultancy Full time

    Data Architecture and Engineering LeadershipWe are looking for a Director of Data Engineering who can champion data mesh principles and enable decentralized data ownership across the organization. The successful candidate will have strong leadership skills and experience in modern data architectures, cloud infrastructure, and automation.


  • Cape Town, Western Cape, South Africa Lycopodium Full time

    Job Description:Lycopodium is seeking a highly skilled Data Engineer to join its team in Cape Town. The successful candidate will be responsible for designing, developing, and maintaining data structures, as well as developing and maintaining data pipelines to collect, transform, store, and share/integrate data across platforms.Responsibilities:Design and...

  • Cloud Data Engineer

    2 days ago


    Cape Town, Western Cape, South Africa Goodwall SA Full time

    Data Engineering Role at Goodwall SA: A Mission-Driven OpportunityAre you passionate about using data to drive social impact? Do you want to work on a mission-driven project that empowers youth globally? Look no further than Goodwall SA's Data Engineer role.In this position, you will work closely with our Programs and Product teams to develop and execute...


  • Cape Town, Western Cape, South Africa Communicate Recruitment Full time

    The successful candidate will be responsible for building, optimizing, and maintaining complex data pipelines and databases. They will collaborate with data scientists and analysts to ensure seamless data accessibility and quality while developing ETL processes and scalable storage solutions. The role involves implementing best practices for data security,...

  • Data Engineer

    7 days ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    The Data Engineer will be responsible for designing, building, and optimising data pipelines and ensuring the efficient movement and transformation of data across various systems. The ideal candidate will have expertise in SQL, data modelling, and experience with cloud-based or on-premises data solutions. Additionally, knowledge of ML and statistical...


  • Cape Town, Western Cape, South Africa Intellinexus Full time

    Company OverviewIntellinexus is a leading business-led data and business intelligence consultancy that empowers organisations with data-driven decision-making capabilities. Our team combines innovation and expertise to help organisations become smarter, more agile, and more resilient to the changes in their operating environments.Duties &...


  • Cape Town, Western Cape, South Africa BigTalent Full time

    BigTalent, a leading organization in the financial services sector, is seeking an exceptional Data Engineering Lead to drive their technical vision and strategy.Job OverviewThe successful candidate will lead the design and implementation of scalable data pipelines, platforms, and infrastructure to support business objectives.The ideal candidate will be a...


  • Cape Town, Western Cape, South Africa Top Recruitment Full time

    Data Insights Engineer Job SummaryWe are seeking a highly skilled Data Insights Engineer to join our team in Cape Town. As a key member of our data analytics team, you will be responsible for designing and developing data sets and reports for business analytic requirements using advanced technologies such as Apache Airflow, Google BigQuery, and Google Looker...

  • Data Engineer Lead

    4 hours ago


    Cape Town, Western Cape, South Africa Gomag Full time

    We are looking for a seasoned Data Engineer Lead to join our team at Gomag. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Lead the design and development of data pipelines and architectureCollaborate with cross-functional teams to...

  • AWS Data Engineer

    3 weeks ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 4+ years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • AWS Data Engineer

    2 weeks ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 4+ years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...


  • Cape Town, Western Cape, South Africa PBT Group Full time

    Are you a skilled Data Streaming Engineer passionate about real-time data processing and cutting-edge technology? Join our PBT team and play a pivotal role in shaping data streaming architecture. This position offers the opportunity to work with industry-leading tools like Kafka and Flink, collaborating with talented development and data engineering teams to...