Data Streaming Engineer

2 weeks ago

Johannesburg, Gauteng, South Africa Golden Rule Full time
The Role

An opportunity exists for a Senior Data Streaming Engineer who will Build, enhance, and maintain our real-time data pipeline. The role will work with various infrastructure and operations teams to maintain our data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth of the bank. You will collaborate with many teams across the bank to understand data needs and turn them into platforms and services, monitor and maintain the health of the data streams, and have an impact on the data ecosystem within Square.

Skills and Experience

Experience

Experience of developing solutions in the cloud At least 2-3 years' experience with designing and developing streaming Data Pipelines for Data Ingestion or Transformation using AWS technologies Experience with distributed log systems, such as Kafka and AWS Kinesis redPanda or Confluence experience advantageous Experience in developing data warehouses and data marts Experience in Data Vault and Dimensional modelling techniques Experience working with automated data warehousing solutions would be advantageous

Qualifications

Bachelor's Degree in Computer Science or similar fields like Information Systems, Big Data, etc. AWS Data Engineer Certification would be advantageous Related Technical certifications

Other

Operational Delivery

Build, enhance, and maintain our real-time data pipeline Work with various infrastructure and operations teams to maintain our data infrastructure Be self-driven in identifying and documenting feature gaps, and designing and implementing solutions to them Help build, modernize, and maintain services and tooling to ensure resiliency, fix data discrepancies, and enhance the customer experience Monitor daily execution, diagnose and log issues, and fix pipelines to ensure SLAs are met with internal stakeholders Mentor other engineers and help them grow; code reviews, guidance on best practices, leveraging your experience in the field

Technical Leadership

Participate in the engineering and other discipline's community of practice Share AWS knowledge and practical experience with community Challenge and contribute to development of architectural principals and patterns

Compliance

Ensure solutions adhere to Olympus patterns, guidelines and standards Operate within project environments and participate in continuous improvement efforts

Delivery Management

Follow and participate in defined ways of work including, but not limited to, sprint planning, backlog grooming, retrospectives, demos and PI planning

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Planet10 Full time

    A leader in the Banking and Financial space is on the lookout for a Data Engineer to join their team. The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company's Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Nedbank Full time

    Requisition Details & Talent Acquisition ConsultantREQ Keabetswe ModiseJob FamilyInformation TechnologyCareer StreamData Leadership PipelineManager of Self ProfessionalJob PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Nedbank Data Architecture Roadmap, to advance technical thought...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Nedbank Full time

    Requisition Details & Talent Acquisition Consultant REQ Thembi MtshaliJob FamilyRisk, Audit and ComplianceCareer StreamAuditingLeadership PipelineManage Self: ProfessionalJob PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Nedbank Data Architecture Roadmap, to advance technical...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Recru-it Full time

    Job Purpose: The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of...

  • Cloud Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    The Cloud Data Engineer, AWS Data Pipeline, AWS Storage Gateway and Digital Assets will work on the Amazon Web Services (AWS) infrastructure team. The engineer's primary focus is to provide data services that enhance our clients' businesses. You'll be a key member of the data pipeline team helping us deliver these solutions quickly and efficiently while...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Welcome to our Data Engineering team at XYZ Company!Qualifications: Bachelor's degree in Computer Science, Information Technology, or a related field (Master's degree preferred)Requirements: 5+ years of solid experience in data engineeringStrong background in data warehousing and data lake architectureProficiency in Data Vault and dimensional modeling...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Qualifications:Bachelor's degree in Computer Science, Information Technology, or a related field (Master's degree preferred)Requirements: 5+ years' experience in data engineeringStrong data warehousing and data lake architecture experienceData Vault and dimensional modelling techniquesSQL experienceBig data experience such as Hadoop, Spark, AWS EMRExperience...

  • Junior Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Planet10 Full time

    A Leader in the Financial and Banking sector is seeking a Junior Data Engineer.The main role of the Data Engineer is to utilize their data knowledge and related technologies, following the Company's Data Architecture Roadmap, to enhance technical leadership for the Enterprise, deliver appropriate data products, and back data initiatives. Moreover, Data...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa NTT Full time

    JOB DESCRIPTION NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa NTT Full time

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our clients solve some of...

  • Data Engineer

    4 weeks ago

    Johannesburg, Gauteng, South Africa NTT Full time

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our clients solve some of...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Valid8 Financial, Inc. Full time

    Help us sort through our massive data set and answer the questions we don't even know to ask. We have undertaken a large initiative to bring everything into a data lake and want to start surfacing dashboards for our clients/publishers, our business development teams, and our sales teams. What your day will look like We have a new set of dashboards we are...

  • Junior Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Recru-it Full time

    Job Responsibilities:Responsible for the day-to-day maintenance of data pipelines, providing support to the data squads including performing data related tasks (such as data profiling, data cleaning, data configurations, data support, data validation, data quality assurance) in a Data Epic and assisting in basic data pipelines, data ingestion and supporting...

  • Data Scientist

    2 weeks ago

    Johannesburg, Gauteng, South Africa Planet10 Full time

    Purpose of the Position Extract and gather data from various sources, both real time event streams and batch, develop technical architectural solutions in the cloud utilizing this data for intelligent automation and data driven insights.Qualifications Essential Degree in Mathematics/Engineering/Data Science/Data Analysis Post Graduate qualification in...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Energy At Work Full time

    Job Title: Data Engineer (SAS)Location: RemoteAbout Our Client: Our client is a leading player in the insurance industry, dedicated to providing innovative solutions to meet the needs of our clients. They leverage cutting-edge technology to drive their business forward and are seeking a skilled Data Engineer to join their dynamic team. If you are passionate...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Iolap Full time

    Do you like building data systems and pipelines?Do you enjoy interpreting trends and patterns?Are you able to recognize the deeper meaning of data?Join iOLAP, an Elixirr Company, as a Data Engineer and help us analyze and organize raw data?to provide valuable business insights to our clients and stakeholdersAs Data Engineer, you will be responsible to ensure...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Iolap Full time

    Do you like building data systems and pipelines? Do you enjoy interpreting trends and patterns? Are you able to recognize the deeper meaning of data?Join iOLAP, an Elixirr Company, as a Data Engineer and help us analyze and organize raw datato provide valuable business insights to our clients and stakeholdersAs Data Engineer, you will be responsible to...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa iOLAP Full time

    Do you like building data systems and pipelines? Do you enjoy interpreting trends and patterns? Are you able to recognize the deeper meaning of data? Join iOLAP, an Elixirr Company, as a Data Engineer and help us analyze and organize raw data to provide valuable business insights to our clients and stakeholdersAs Data Engineer, you will be responsible to...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Puma Energy Full time

    Main Purpose: Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. Ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists to develop insightful analytics and business stakeholders to make well-informed...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Mimecast Limited Full time

    Data Engineer (SQL experience essential) page is loaded Data Engineer (SQL experience essential) Apply locations South Africa - Johannesburg time type Full time posted on Posted 3 Days Ago job requisition id R4508 Analytics Engineer We're your dream business intelligence team The Business Intelligence & Analytics team at Mimecast is an exciting blend of...