Senior Data Engineer

6 days ago

Johannesburg, Gauteng, South Africa Hire Resolve Full time R800 000 - R1 200 000 per year
Hire Resolve's client is looking for a Senior Data Engineer to join their team in Johannesburg, GP. The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines to support telecommunications CDR processing, real-time data ingestion, and analytical workloads.

This role requires expertise in data modeling, ETL development, stream processing, and distributed data systems. The ideal candidate will work closely with developers, DevOps, and analytics teams to transform raw network data into clean, structured, and query-ready datasets that power dashboards, machine learning models, and business logic. The candidate will provide technical leadership, optimize data workflows for performance and reliability, and drive best practices in data engineering methodologies

Responsibilities: 
  • Build ETL/ELT pipelines for ingesting, cleansing, and transforming CDRs and telecommunications logs
    from multiple network elements (5G/4G/3G/2G).
  • Design and maintain real-time data flows using Kafka, Apache NiFi and Apache Flink.
  • Work with large-scale distributed file systems for batch and streaming ingestion.
  • Integrate and structure data for analytics platform like Apache Druid, Hudi and Superset.
  • Develop CI/CD pipelines for deploying data workflows and transformation logic.
  • Ensure data quality, schema validation and compliance with retention and security policies.
  • Monitor data pipeline health and optimize performance throughput and cost efficiency.
  • Write complex and performant queries for data validation, transformation, aggregation, and analytics
    across relational and distributed platforms.
  • Develop and optimize big data processing workflows in platforms like Apache Spark, Hive and Druid.
  • Establish efficient issue tracking and workflow processes, enhancing productivity and collaboration
    across engineering teams.
  • Implement security best practices and compliance frameworks to safeguard infrastructure, data, and
    applications from vulnerabilities and threats.
  • Maintain secure role-based access control mechanisms, encryption strategies, and identity management
    solutions to protect sensitive data and ensure regulatory compliance.
  • Mapping data flows from source to transformation to consumption.
  • Design and implement full-text search and indexing solutions for querying and retrieval of structured and unstructured telecommunications data using Apache tools or similar search engines.
  • Analyze and estimate and implement storage requirements and strategies for large-scale CDR datasets
    and real-time data streams, ensuring optimal resource allocation and scalability across environments.
  • Ensure data integrity and consistency across ingestion, transformation and storage layers through
    validation checks, schema enforcement and robust error handling mechanisms.
  • Develop and maintain quality monitoring tools to proactively detect anomalies, missing records or data
    corruption across pipelines.
  • Perform other duties as assigned
Requirements: 
  • Bachelor's degree in Computer Engineering, Software Engineering, Computer Science, or a related field.
  • Strong experience in building data pipelines using tools like Apache NiFi, Kafka, Airflow or similar.
  • Proficiency in SQL, Python, database administration and management like PostgreSQL, MySQL.
  • Solid understanding of distributed data systems like Hive and Hudi and Spark.
  • Experience with streaming frameworks like Kafka Streams, Apache Flink and Apache Beam.
  • Familiarity with data serialization formats like JSON.
  • Knowledge of SFTP and secure data transfer mechanisms for ingesting remote files.
  • Proficient with Linux environments, shell scripting and storage systems like Ceph.
  • Experience with data governance, including data privacy and regulatory compliance like GDPR and
    implementing access control, auditing and data usage policies.
  • Experience in maintaining central inventory of data assets, managing metadata and enabling searchable
    discovery across structured and unstructured datasets.
  • Experience in data lineage tracking to map data flows, visualize and track dependencies.
  • Experience with OLAP systems, analytical modelling and columnar databases and designing and
    querying multidimensional cubes.
  • Strong problem-solving skills, ability to work in a fast-paced environment, and manage multiple projects
    efficiently.
  • Strong collaboration skills, adaptability, and a commitment to continuous learning 
How to Apply: 

If you would like to apply for this role, kindly forward your CV to Gaby Turner at [email protected] or you may forward your CV to [email protected] 

  • Senior Data Scientist

    3 hours ago

    Johannesburg, Gauteng, South Africa NTT Data Full time US$80 000 - US$160 000 per year

    Job SummaryThe Senior Data Scientist will lead the design, development, and deployment of advanced analytics and machine learning solutions that drive strategic decision-making and operational efficiency. This role requires a deep understanding of data science, data engineering, and AI concepts, and will play a pivotal role in embedding intelligent...

  • Senior Data Analyst

    3 hours ago

    Johannesburg, Gauteng, South Africa NTT Data Full time US$60 000 - US$120 000 per year

    Role Summary:The Senior Data Analyst will play a pivotal role in transforming data into actionable insights that drive strategic decision-making across the organisation. This role requires a strong analytical mindset and proficiency in data visualisation. The ideal candidate will bridge the gap between data and business.Responsibilities:Develop and maintain...

  • Senior Data Management Security Engineer

    6 days ago

    Johannesburg, Gauteng, South Africa NTT DATA, Inc. Full time R800 000 - R1 200 000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.The Data Management Security Engineer...

  • Senior Azure Engineer

    2 days ago

    Johannesburg, Gauteng, South Africa NTT DATA, Inc. Full time

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.We are looking for a highly...

  • Senior Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Harambee Youth Employment Accelerator Full time R120 000 - R240 000 per year

    We're Hiring: Senior Data Engineer – JohannesburgAre you ready to build technology that changes lives?Harambee Youth Employment Accelerator is looking for an exceptionalSenior Data Engineerto join our Software Development team. You'll work on collecting, storing, and processing large datasets to deliver optimal solutions—while partnering with our Head of...

  • Senior Data Engineer

    1 week ago

    Johannesburg, Gauteng, South Africa Harambee Full time R800 000 - R1 200 000 per year

    Senior Data Engineer - JohannesburgHelp build technology that changes lives.About the RoleThe Harambee Software Development team is looking for an exceptional Senior Data Engineer to join our dynamic team based in Johannesburg.This role is focused on collecting, storing, and processing large sets of data to deliver optimal solutions-along with their...

  • Senior Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Harambee Youth Employment Accelerator Full time R600 000 - R1 200 000 per year

    Senior Data Engineer – JohannesburgHelp build technology that changes lives.About the RoleThe Harambee Software Development team is looking for an exceptional Senior Data Engineer to join our dynamic team based in Johannesburg.This role is focused on collecting, storing, and processing large sets of data to deliver optimal solutions—along with their...

  • Senior Data Integration Engineer

    1 week ago

    Johannesburg, Gauteng, South Africa Indsafri Full time R2 000 000 - R2 500 000 per year

    Job Description:Job Title: Senior Data Integration Engineer (Salesforce, Databricks & MuleSoft)Location: Johannesburg (Hybrid)Employment Type: ContractContract Tenure: 6 to 12 monthsJob SummaryWe are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer...

  • Senior Azure Integrations Engineer

    2 days ago

    Johannesburg, Gauteng, South Africa NTT DATA, Inc. Full time

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.We are looking for a highly...

  • Senior Data Engineer

    6 days ago

    Johannesburg, Gauteng, South Africa Hire Resolve Full time R1 200 000 - R2 400 000 per year

    Hire Resolve's client is seeking a highly skilled Senior Data Engineer to join their dynamic team within the telecommunications domain. The ideal candidate will be responsible for designing, developing, and maintaining scalable and reliable data pipelines to support the ingestion, processing, and analysis of Call Detail Records (CDRs) and real-time data...