Data Streaming Engineer

Found in: Talent ZA C2 - 3 weeks ago

Johannesburg, South Africa Golden Rule Full time
The Role

An opportunity exists for a Senior Data Streaming Engineer who will Build, enhance, and maintain our real-time data pipeline. The role will work with various infrastructure and operations teams to maintain our data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth of the bank. You will collaborate with many teams across the bank to understand data needs and turn them into platforms and services, monitor and maintain the health of the data streams, and have an impact on the data ecosystem within Square.

Skills and Experience

Experience

Experience of developing solutions in the cloud At least 2-3 years' experience with designing and developing streaming Data Pipelines for Data Ingestion or Transformation using AWS technologies Experience with distributed log systems, such as Kafka and AWS Kinesis redPanda or Confluence experience advantageous Experience in developing data warehouses and data marts Experience in Data Vault and Dimensional modelling techniques Experience working with automated data warehousing solutions would be advantageous

Qualifications

Bachelor’s Degree in Computer Science or similar fields like Information Systems, Big Data, etc. AWS Data Engineer Certification would be advantageous Related Technical certifications

Other

Operational Delivery

Build, enhance, and maintain our real-time data pipeline Work with various infrastructure and operations teams to maintain our data infrastructure Be self-driven in identifying and documenting feature gaps, and designing and implementing solutions to them Help build, modernize, and maintain services and tooling to ensure resiliency, fix data discrepancies, and enhance the customer experience Monitor daily execution, diagnose and log issues, and fix pipelines to ensure SLAs are met with internal stakeholders Mentor other engineers and help them grow; code reviews, guidance on best practices, leveraging your experience in the field

Technical Leadership

Participate in the engineering and other discipline’s community of practice Share AWS knowledge and practical experience with community Challenge and contribute to development of architectural principals and patterns

Compliance

Ensure solutions adhere to Olympus patterns, guidelines and standards Operate within project environments and participate in continuous improvement efforts

Delivery Management

Follow and participate in defined ways of work including, but not limited to, sprint planning, backlog grooming, retrospectives, demos and PI planning 

  • Senior Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa The Individual Touch Full time

    You will take responsibility for managing our master data set, developing reports, and troubleshooting data issues. To do well in this role you need a very fine eye for detail, experience as a data analyst, and deep understanding of the popular data analysis tools and databases: Full job description available for short-listed candidates.Experience/Skills:5+...

  • AWS Data Engineer with DataBricks

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.Analyze, re-architect and...

  • AWS Data Engineer with DataBricks

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.Analyze, re-architect and...

  • AWS Data Engineer with DataBricks

    Found in: Whatjobs ZA C2 - 1 week ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience.   Duties: Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake. Analyze, re-architect and...

  • AWS Data Engineer with DataBricks

    Found in: Adzuna ZA B C2 - 2 weeks ago

    Johannesburg, South Africa Adzuna ZA B C2 Full time

    PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.Analyze, re-architect and...

  • AWS Data Engineer/JHB

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa Rooftop Recruitment Full time

    AWS Data Engineer/JHBContract Type: 12 monthsOur client is seeking a highly motivated and experienced AWS Data Engineer to join their GlobalMarkets division. The role is responsible for closely working the traders, quantitative analysts, andData Scientists to design, develop, and maintain data infrastructure and pipelines for our global markets business.The...

  • Data Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Johannesburg, South Africa Network Recruitment Full time

    Our client, a leading company within the Financial Services industry, is seeking a highly skilled Senior Data Engineer to join their dynamic team. In this role, you will play a critical part in designing, developing, and maintaining robust data pipelines, ensuring the availability, reliability, and accessibility of our data assets. Your expertise in data...

  • Data Engineer

    Found in: Adzuna ZA B C2 - 4 days ago

    Johannesburg, South Africa Adzuna ZA B C2 Full time

    Our client, a leading company within the Financial Services industry, is seeking a highly skilled Senior Data Engineer to join their dynamic team. In this role, you will play a critical part in designing, developing, and maintaining robust data pipelines, ensuring the availability, reliability, and accessibility of our data assets. Your expertise in data...

  • Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa Network Recruitment Full time

    Our client, a leading company within the Financial Services industry, is seeking a highly skilled Senior Data Engineer to join their dynamic team. In this role, you will play a critical part in designing, developing, and maintaining robust data pipelines, ensuring the availability, reliability, and accessibility of our data assets. Your expertise in data...

  • Data Center Technician

    2 days ago

    Johannesburg, South Africa Dimension Data Full time

    **Requirements**: - The Data Center Technician (DCT) responsibility is the maintenance of all Data Centre White Space, ensuring a 24/7 service offering within Internet Solutions Data Centers.The DCT provides a service to clients to ensure that their IT infrastructure and systems remain operational through proactively identifying, investigating and resolving...

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Puma Energy Full time

    Main Purpose: Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. Ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists to develop insightful analytics and business stakeholders to make well-informed...

  • Senior ISP Network Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Dimension Data Full time

    FunctieomschrijvingThe Senior ISP Network Engineer, DSL & Telephone Cable Networks, Data Center (DC), Integrated services provider (ISP), MPLS, ACI, SDA, CORE, IP Networks will be responsible for designing and implementing complex data networks across multiple locations. The role involves working closely with business teams to provide solutions to their...

  • Senior Software Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of being micro-managed? Are you looking for a company where no-one tells you how to do your job? Do you want to work in a team, but not be told what to do and when to do it? If so, we believe this Senior Software Engineer role is perfect for you!As a Senior Software Engineer, you'll be responsible for building and maintaining...

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PRR Recruitment Services Full time

    This well known company is looking for a Data Engineer. will be based in Rivonia The ideal candidate will have strong experience in the following: Data cleaning SQL ExportBig data Sets BSc Degree in Maths and Stats will be required a min of 3 years experience. Stable track record is required. Send your Cv to marinda@prrrecruitment.co.za

  • Data Visualisation Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources.     Duties: A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Visualisation Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources.  Duties:A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Visualisation Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources.  Duties:A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Visualisation Engineer

    Found in: Adzuna ZA B C2 - 2 weeks ago

    Johannesburg, South Africa Adzuna ZA B C2 Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources. Duties:A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Johannesburg, South Africa Psybergate Full time

    What you will be doing / What we are looking for:  Data Engineer skill is required to enable the data lifecycle within EDS, namely the ability to integrate data between source system (golden/trusted) and the target database (LOB), hence provide good quality data by applying the necessary framework/s and governance to whomever may require it (BI and...

  • Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa Psybergate Full time

    What you will be doing / What we are looking for:  Data Engineer skill is required to enable the data lifecycle within EDS, namely the ability to integrate data between source system (golden/trusted) and the target database (LOB), hence provide good quality data by applying the necessary framework/s and governance to whomever may require it (BI and...