Data Streaming Engineer

7 months ago


Johannesburg, South Africa Golden Rule Full time
The Role

An opportunity exists for a Senior Data Streaming Engineer who will Build, enhance, and maintain our real-time data pipeline. The role will work with various infrastructure and operations teams to maintain our data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth of the bank. You will collaborate with many teams across the bank to understand data needs and turn them into platforms and services, monitor and maintain the health of the data streams, and have an impact on the data ecosystem within Square.

Skills and Experience

Experience

Experience of developing solutions in the cloud At least 2-3 years' experience with designing and developing streaming Data Pipelines for Data Ingestion or Transformation using AWS technologies Experience with distributed log systems, such as Kafka and AWS Kinesis redPanda or Confluence experience advantageous Experience in developing data warehouses and data marts Experience in Data Vault and Dimensional modelling techniques Experience working with automated data warehousing solutions would be advantageous

Qualifications

Bachelor’s Degree in Computer Science or similar fields like Information Systems, Big Data, etc. AWS Data Engineer Certification would be advantageous Related Technical certifications

Other

Operational Delivery

Build, enhance, and maintain our real-time data pipeline Work with various infrastructure and operations teams to maintain our data infrastructure Be self-driven in identifying and documenting feature gaps, and designing and implementing solutions to them Help build, modernize, and maintain services and tooling to ensure resiliency, fix data discrepancies, and enhance the customer experience Monitor daily execution, diagnose and log issues, and fix pipelines to ensure SLAs are met with internal stakeholders Mentor other engineers and help them grow; code reviews, guidance on best practices, leveraging your experience in the field

Technical Leadership

Participate in the engineering and other discipline’s community of practice Share AWS knowledge and practical experience with community Challenge and contribute to development of architectural principals and patterns

Compliance

Ensure solutions adhere to Olympus patterns, guidelines and standards Operate within project environments and participate in continuous improvement efforts

Delivery Management

Follow and participate in defined ways of work including, but not limited to, sprint planning, backlog grooming, retrospectives, demos and PI planning 
  • Data Engineer

    4 weeks ago


    Johannesburg, South Africa Nedbank Full time

    Job title : Data EngineerJob Location : Gauteng, JohannesburgDeadline : January 02, 2025Quick Recommended Links Jobs by Location Job by industries Job Purpose The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Nedbank Data Architecture Roadmap, to advance technical thought leadership for the...

  • Data Engineer

    3 weeks ago


    Johannesburg, South Africa Nedbank Full time

    Job title : Data Engineer Job Location : Gauteng, Johannesburg Deadline : January 02, 2025 Quick Recommended Links Jobs by Location Job by industries Job Purpose The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Nedbank Data Architecture Roadmap, to advance technical...

  • Data Engineer

    6 months ago


    Johannesburg, South Africa Planet10 Full time

    A leader in the Banking and Financial space is on the lookout for a Data Engineer to join their team. The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company’s Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data...

  • Cloud Data Engineer

    7 months ago


    Johannesburg, South Africa Dimension Data Full time

    The Cloud Data Engineer, AWS Data Pipeline, AWS Storage Gateway and Digital Assets will work on the Amazon Web Services (AWS) infrastructure team. The engineer's primary focus is to provide data services that enhance our clients' businesses. You'll be a key member of the data pipeline team helping us deliver these solutions quickly and efficiently while...

  • Business Analyst

    1 week ago


    Johannesburg, South Africa Data Centrix Full time

    Qualifications / Knowledge / Experience:Bachelors degree in computer science, Information Systems and/ or;Diploma in Business Analysis from an IIBA endorsed education provider and/or;Valid CBAP or CCBA certification.Advanced computer literacy.Financial services knowledge.Advanced knowledge of the BABOK.SDLC methodology knowledge.Minimum 5 years experience in...

  • Business Analyst

    1 week ago


    Johannesburg, South Africa Data Centrix Full time

    Qualifications / Knowledge / Experience: Bachelors degree in computer science, Information Systems and/ or; Diploma in Business Analysis from an IIBA endorsed education provider and/or; Valid CBAP or CCBA certification. Advanced computer literacy. Financial services knowledge. Advanced knowledge of the BABOK. SDLC methodology knowledge. Minimum 5 years...


  • Johannesburg, South Africa Nintex Full time

    Job title : Principal Data Engineer Job Location : Gauteng, Johannesburg Deadline : January 04, 2025 Quick Recommended Links Jobs by Location Job by industries About the role:  You are an expert in data acquisition and processing not only in your team’s domain, but also across Nintex. Your technical leadership demonstrates a passion...

  • Data Engineer Lead

    4 weeks ago


    Johannesburg, Gauteng, South Africa Nedbank Full time

    Job Title : Data EngineerA highly skilled Data Engineer is required by Nedbank to lead the design, implementation, and maintenance of data systems. The successful candidate will have a deep understanding of data technologies and be able to leverage their expertise to advance technical thought leadership within the Enterprise.Job DescriptionThe primary...


  • Johannesburg, Gauteng, South Africa E-Merge Full time

    About the Role:We are seeking an experienced Azure Data Engineer to join our team in Johannesburg. As a key member of our team, you will be responsible for designing and implementing data solutions using Microsoft Azure cloud technologies.Key Responsibilities:Design and implement data pipelines and data streams using Azure cloud data technologies.Analyse...


  • Johannesburg, Gauteng, South Africa Dimension Data Full time

    At Dimension Data, we are seeking an experienced Cloud Data Architect to lead our data engineering solutions. The ideal candidate will have a deep understanding of AWS technologies and experience in designing scalable data pipelines.Key ResponsibilitiesDesign and implement data pipelines using AWS services such as AWS Data Pipeline and AWS Storage...


  • Johannesburg, Gauteng, South Africa Entelect Software Ltd Full time

    Exciting opportunity at Entelect to take your career to the next level. As an Enterprise Data Strategist, you will be working in a dynamic environment that prioritizes growth and collaboration.You will work with Solution Architects to design and implement cloud, hybrid, and on-prem solutions that deliver impactful advanced analytics using machine learning,...


  • Johannesburg, South Africa Promapp Full time

    About Nintex: At Nintex, we are transforming the way people work, everywhere. As the global standard for process intelligence and automation, we're trusted by over 10,000 public and private sector organizations across 90 countries. Our customers, from industry giants like Amazon, Coca-Cola, and Microsoft, rely on the Nintex Platform to accelerate their...


  • Johannesburg, Gauteng, South Africa Golden Rule Full time

    Job SummaryWe are seeking a skilled Data Streaming Engineer to join our team, focusing on building and maintaining our real-time data pipeline. This role will involve collaboration with various infrastructure and operations teams to ensure the reliability and scalability of our data infrastructure.Key Responsibilities:Design and develop streaming data...

  • Data Engineer

    1 month ago


    Johannesburg, South Africa Network Recruitment Full time

    As a Data Engineer, you'll be at the heart of transforming complex, unstructured data into actionable insights. You'll be responsible for implementing process improvements, enhancing data systems, and ensuring the integrity and scalability of our data infrastructure. If you're excited to work with large data sets, optimize ETL processes, and ensure data...


  • Johannesburg, South Africa Nedbank Full time

    **Requisition Details & Talent Acquisition Specialist** - REQ 136186 - Keabetswe Modise**Job Family** - Information Technology**Career Stream** - Data- Leadership Pipeline- Manage Managers (MM)**Job Purpose** - To manage the architecture, and data integration engineering team, driving design, and architecting detailed data solutions aligned with the Groups...

  • Head: Data Integration

    3 months ago


    Johannesburg, South Africa Nedbank Full time

    **Requisition Details & Talent Acquisition Specialist** - REQ 136182 - Keabetswe Modise**Job Family** - Information Technology **Career Stream** - Data**Leadership Pipeline** - Manage Managers (MM)**Job Purpose** - To lead and manage Nedbank’s data integration and data warehousing initiatives, ensuring the effective and efficient flow of data across the...

  • AWS Data Engineer

    4 weeks ago


    Johannesburg, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...


  • Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you'll be at the heart of transforming complex, unstructured data into actionable insights. You'll be responsible for implementing process improvements, enhancing data systems, and ensuring the integrity and scalability of our data infrastructure. If you're excited to work with large data sets, optimize ETL processes, and ensure data...


  • Johannesburg, South Africa LexisNexis South Africa Shared Services (Pty) Ltd Company Full time

    About the Role At LexisNexis we develop the legal profession’s most innovative products for data analysis, visualization, and research. We use the latest techniques in AI, machine learning, and data visualization to uncover insights about judges’ rulings, build forecasts of likely outcomes, and reveal critical connections in massive datasets spanning...


  • Johannesburg, Gauteng, South Africa PBT Group Full time

    We are seeking a highly skilled AWS Data Engineer to join our team at the PBT Group. This role involves designing and implementing scalable data systems, ensuring seamless integration of data from various sources, and empowering business solutions through efficient data management.The ideal candidate will possess a strong grasp of software development and...