Data Pipeline Specialist

2 days ago


Johannesburg, Gauteng, South Africa Nagarro Inc Full time

We're a dynamic and non-hierarchical company that builds innovative products, services, and experiences. Our work culture is employee-centric, and we're looking for great new colleagues to join our team.

Job Overview

We're seeking a senior big data expert to lead our cloud data pipeline efforts on Google Cloud Platform (GCP). The ideal candidate will have a strong background in designing, developing, and maintaining scalable data pipelines and ETL processes.

Key Responsibilities
  • Design and implement efficient data storage solutions using BigQuery, Cloud Storage, and other GCP services.
  • Collaborate with data scientists, machine learning engineers, and other stakeholders to integrate and deploy machine learning models into production environments.
  • Develop and maintain custom deployment solutions for machine learning models using Kubeflow, AI Platform, and Docker.
  • Write clean, efficient, and maintainable code in Python and PySpark for data processing and transformation tasks.
  • Ensure data quality, integrity, and consistency through data validation and monitoring processes.
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
  • Proven experience as a data engineer with a focus on GCP.
  • Strong proficiency in Python and PySpark for data processing and transformation.
  • Familiarity with GCP services such as BigQuery, Cloud Storage, Dataflow, and AI Platform.
  • Experience with Terraform for infrastructure as code.
  • Excellent communication and collaboration skills.


  • Johannesburg, Gauteng, South Africa Communicate Recruitment Full time

    About the RoleThis is an exciting opportunity to work as a Data Engineering Specialist. You will be involved in designing and developing efficient ETL pipelines, collecting and integrating data from various sources, and aggregating unstructured data into a structured format. If you have hands-on experience with cloud-based data platforms and a solid...


  • Johannesburg, Gauteng, South Africa Standard Bank of South Africa Limited Full time

    We are seeking a Data Pipeline Engineer to join our team at Standard Bank of South Africa Limited.Job DescriptionThis role involves designing and implementing scalable data pipelines to support business insights from raw data sources.The successful candidate will develop and maintain data pipelines for collecting, managing, and analysing large datasets. They...


  • Johannesburg, Gauteng, South Africa Liberty Group Limited Full time

    About the RoleWe are seeking an experienced Data Pipeline Developer to join our team at Liberty Group Limited. As a key member of our technology team, you will be responsible for designing, building, and deploying production-ready data pipelines from ingestion to integration within big data architectures.


  • Johannesburg, Gauteng, South Africa Ananzi Full time

    Data Engineer - Real-Time Data SolutionsWe need a seasoned expert to lead our efforts in building and maintaining a real-time data pipeline that meets our business needs.Job Description:The ideal candidate will possess a deep understanding of data streaming concepts and experience with tools like Apache Kafka and Apache Flink.Responsibilities:Design and...


  • Johannesburg, Gauteng, South Africa FROSTBYTE Full time

    FROSTBYTE is a leading provider of custom software development solutions, quality control solutions, and consulting services.About the OpportunityJob Summary:As a Senior Software Engineer, you will be responsible for developing and maintaining software applications and systems.Key ResponsibilitiesDesign, develop, and deploy software applications and...


  • Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Data Engineering Position OverviewWe are looking for a highly skilled and experienced Data Engineer to join our team at Network Recruitment. The successful candidate will be responsible for developing and maintaining scalable, secure, and repeatable data pipelines that meet business needs.Main Areas of Responsibility:Design, implement, and maintain scalable...

  • Data Science Expert

    4 days ago


    Johannesburg, Gauteng, South Africa Data Bene Full time

    Job DescriptionWe are seeking a highly skilled Data Analyst to join our team at Data Bene. As a key member of our organization, you will play a vital role in collecting, analyzing, and interpreting data sets to inform business decisions.Your primary focus will be on ensuring the accuracy and quality of data, while also developing ad hoc analyses from start...


  • Johannesburg, Gauteng, South Africa Nedbank Full time

    We are seeking an experienced Data Engineer Specialist to join our team at Nedbank. The successful candidate will be responsible for designing, building, and maintaining scalable data infrastructure using various technologies.Key Responsibilities:Maintain and improve data quality in operational and analytics databases.Design and implement data pipelines for...


  • Johannesburg, Gauteng, South Africa DataFin Full time

    Job DescriptionWe are seeking a highly skilled Data Architecture Specialist to join our team at DataFin in Johannesburg. As a key member of our IT department, you will be responsible for designing and developing scalable ETL systems for our Big Data warehouse.The ideal candidate will have 5-10 years' experience in designing and developing data warehouses...

  • Data Scientist

    5 days ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    As a Specialist Risk Analytics & Data Recruiter working with prestigious corporate concerns in Gauteng, I am dedicated to exploring new commercial opportunities within the market. This allows me to enhance your current skills within the FMCG, Manufacturing and Logistics industries.About this RoleThis is an opportunity for you to implement and maintain...


  • Johannesburg, Gauteng, South Africa E-Merge Full time

    We are seeking a highly skilled AWS Data Engineering Specialist to join our team at E-Merge. In this role, you will be responsible for designing, building, and maintaining Big Data pipelines using AWS services.About the RoleThe ideal candidate will have experience working with cloud-based data engineering tools and technologies, including AWS EMR, Redshift,...


  • Johannesburg, Gauteng, South Africa Eqplus Technologies (Pty) Ltd Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer Specialist to join our team at Eqplus Technologies (Pty) Ltd. In this role, you will be responsible for designing and implementing ETL/ELT methodologies and technologies to maintain our data warehouse. You will also provide maintenance and support to our ETL and ML environments.Duties &...


  • Johannesburg, Gauteng, South Africa Standard Bank of South Africa Limited Full time

    We are seeking a Technical Data Specialist to join our team and provide technical expertise in data warehousing and data modeling concepts. You will work closely with the development team to design, build, test, and deploy large-scale data processing systems.About the RoleWe are seeking a highly skilled professional with extensive experience in data analysis...


  • Johannesburg, Gauteng, South Africa Puma Energy Full time

    We are seeking a Cloud Data Specialist to join our team at Puma Energy.Role SummaryThe successful candidate will be responsible for designing, developing, and maintaining data architecture and pipelines that adhere to ETL principles and business goals.Main Responsibilities:Design and implement efficient data architectures that meet business needs.Develop...


  • Johannesburg, Gauteng, South Africa Ananzi Full time

    Data Streaming SpecialistWe are looking for a highly motivated professional to join our team as a Data Streaming Specialist. The successful candidate will be responsible for designing, developing, and maintaining our real-time data pipeline.Main Responsibilities:Build and maintain real-time data ingestion pipelines using Apache Kafka and Apache Flink.Work...


  • Johannesburg, Gauteng, South Africa Network Recruitment Full time

    **Job Description:**We are seeking a highly skilled AWS Cloud Data Specialist to join our team at Network Recruitment. As a Cloud Data Specialist, you will be responsible for designing and implementing large-scale data solutions on the AWS cloud platform.In this role, you will lead the design, implementation, and successful delivery of complex data pipelines...


  • Johannesburg, Gauteng, South Africa Network Recruitment Full time

    We are looking for a talented Data Engineer to join our team. As a Specialist Risk Analytics & Data Recruiter working with prestigious corporate concerns in Gauteng, I am passionate about delivering innovative solutions. In this role, you will be responsible for designing and implementing data pipelines.About this OpportunityThis is an exciting opportunity...

  • Data Engineer

    2 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    We are seeking a highly skilled and experienced Data Engineer to join our team. In this role, you will be responsible for developing and maintaining scalable, secure, and repeatable data pipelines. You will manage data platforms, ensuring the accuracy and accessibility of data for reporting and analysis across the business. Key Responsibilities: Implement...

  • Data Engineer

    2 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    An industry requiring specialist skills, deserves specialist recruitment. I am a Specialist Risk Analytics & Data Recruiter working with very prestigious corporate concerns in Gauteng. Assisting you in exploring new commercial opportunities within the market is my main priority. This will allow me to enhance your current skills within the FMCG, Manufacturing...


  • Johannesburg, Gauteng, South Africa Network Recruitment Full time

    **Job Summary:**We are seeking an experienced Data Engineer to design, develop, and implement efficient data pipelines using Python, SQL, and cloud-based technologies.About the Job:This is a key role within our organization, requiring a highly skilled professional who can work independently and collaboratively to deliver high-quality results.Bachelor's...