Data Streaming Engineer

Found in: Talent ZA C2 - 3 weeks ago

Cape Town, South Africa PBT Group Full time

PBT Group has an opportunity for a Data Streaming Engineer to build, enhance, and maintain real-time data pipeline. Work with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth. You will collaborate with many teams to understand data needs and turn them into platforms and services, monitor and maintain the health of the data streams, and have an impact on the data ecosystem.

 

Duties:

  1. Operational Delivery
  • Build, enhance, and maintain our real-time data pipeline.
  • Work with various infrastructure and operations teams to maintain our data infrastructure.
  • Be self-driven in identifying and documenting feature gaps and designing and implementing solutions to them.
  • Help build, modernize, and maintain services and tooling to ensure resiliency, fix data discrepancies, and enhance the customer experience.
  • Monitor daily execution, diagnose and log issues, and fix pipelines to ensure SLAs are met with internal stakeholders.
  • Mentor other engineers and help them grow; code reviews, guidance on best practices, leveraging your experience in the field.

 

  1. Technical Leadership
  • Participate in the engineering and other discipline’s community of practice.
  • Share AWS knowledge and practical experience with community.
  • Challenge and contribute to development of architectural principals and patterns.

 

  1. Compliance
  • Ensure solutions adhere to Olympus patterns, guidelines and standards.
  • Operate within project environments and participate in continuous improvement efforts.

 

  1. Delivery Management
  • Follow and participate in defined ways of work including, but not limited to, sprint planning, backlog grooming, retrospectives, demos and PI planning.

 

Skills and Experience Required:

  • Experience of developing solutions in the cloud
  • At least 2-3 years' experience with designing and developing streaming Data Pipelines for Data Ingestion or Transformation using AWS technologies.
  • Experience with distributed log systems, such as Kafka and AWS Kinesis
  • redPanda or Confluence experience advantageous
  • Experience in developing data warehouses and data marts
  • Experience in Data Vault and Dimensional modelling techniques
  • Experience working with automated data warehousing solutions would be advantageous.

 

Qualifications/ Certification:

  • Bachelor’s degree in computer science or similar fields like Information Systems, Big Data, etc.
  • AWS Data Engineer Certification would be advantageous.
  • Related Technical certifications.

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

 

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa Dimension Data Full time

    As a Data Engineer, you'll design and build the data architecture of our products. You will use your technical knowledge to help us develop new products by taking an idea from concept to reality. You'll work on projects across multiple teams in order to achieve company goals and objectives. Your role will involve: - Analyzing business requirements; -...

  • Kafka Big Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for a Kafka Big Data Engineer. As a Kafka Engineer, you will be responsible for the building, improvement and scaling of our streaming data platform. This role requires a combination of strong technical skills, deep understanding of distributed systems as well as excellent communication abilities. Kafka engineer is a big data...

  • AWS Data Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Data Engineer, preferably with Hadoop experience.   Duties: Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake. Analyze,...

  • AWS Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Data Engineer, preferably with Hadoop experience. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.Analyze, re-architect...

  • AWS Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Data Engineer, preferably with Hadoop experience. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties - Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.Analyze, re-architect...

  • Data Engineer

    Found in: beBee jobs ZA - 2 weeks ago

    Cape Town, Western Cape, South Africa Creative CFO Full time

    Creative CFO's job vacancy, Career and RecruitmentJob title : Data Engineer – Cape Town jobs in Western CapeJob Location : Western Cape, Cape TownDeadline : April 28, 2024Quick Recommended LinksJobs by LocationJob by industries The position:As a Data Engineer at Creative CFO, you will be at the forefront of architecting, building, and maintaining our data...

  • Kafka Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    Kafka Engineer Job Description As a Kafka Engineer, you will be responsible for the building, improvement and scaling of our streaming data platform. This role requires a combination of strong technical skills, deep understanding of distributed systems as well as excellent communication abilities.Kafka Engineer Job ProfileKafka engineer is a big data...

  • DevOps Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake,...

  • DevOps Data Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer.   Duties: Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake, Python,....

  • AWS Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • DevOps Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Strong experience with AWS, Terraform, CI/CD, GitDesign, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB,...

  • DevOps Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Strong experience with AWS, Terraform, CI/CD, GitDesign, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB,...

  • DevOps Data Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer.   Duties: Strong experience with AWS, Terraform, CI/CD, Git Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB,...

  • IT Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Cape Town, South Africa Armstrong Appointments Full time

    Our client based in Cape Town are currently seeking the skills of an IT Data Engineer to assist in designing and implementing scalable and robust processes to support the data engineering capability. This role will be responsible for extracting and transforming massive amounts of data at scale and consolidating this data into a bigger data ecosystem. Main...

  • Data Flow Analytics

    Found in: beBee S ZA - 3 weeks ago

    Western Cape, Cape Town, South Africa Amazon Full time

    Amazon’s job vacancy, Career and Recruitment Job title : Data Flow Analytics – Cloud Support Engineer, Support Engineering jobs in Western Cape Job Location : Western Cape, Cape Town Deadline : April 08, 2024 Quick Recommended Links Jobs by Location Job by industries WHAT CAN YOU EXPECT FROM A LIFE AT AWS? Every day will bring new and exciting...

  • Automation Test Analyst

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa Dimension Data Full time

    As an Automation Test Analyst, you will be responsible for the implementation of software testing strategies to help increase quality and productivity. You will also provide test automation support for new or existing applications in a range of environments including Web, Desktop and Mobile. Requirements Evaluates the quality of test automation...

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa 60degrees Full time

    Job Description Data Engineer (Gauteng) Compensation: R 450 000 – R 550 000 per annum Working model: Hybrid The opportunity that awaits you: Our client, a solutions energy engineering solution provider with projects mainly in Sub-Saharan Africa seeks to expand their team. Our client takes pride in not just ticking a box but...

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Cape Town, South Africa DLK Group Full time

    Job Description ·Modifying the database structure, asnecessary, from information given by application developers. ·Enrolling users and maintaining systemsecurity. ·Controlling and monitoring user accessto the database. ·Monitoring and optimizing the performanceof the database. ·Maintaining archived data. ·Contacting database vendor...

  • Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Cape Town, South Africa IOCO Full time

    We are seeking an experienced Data Migration Engineer to join our team on a contract basis to facilitate this migration process. The ideal candidates will have strong expertise in data engineering, particularly with Azure Synapse services, SSIS, and API integration. What you’ll do:Collaborate with the existing team to understand the current data...

  • Data Engineer

    Found in: Whatjobs ZA C2 - 1 week ago

    Cape Town, South Africa IOCO Full time

    We are seeking an experienced Data Migration Engineer to join our team on a contract basis to facilitate this migration process. The ideal candidates will have strong expertise in data engineering, particularly with Azure Synapse services, SSIS, and API integration.   What you’ll do: Collaborate with the existing team to understand the current data...