Data Engineer

2 days ago

Pretoria, Gauteng, South Africa Cls Full time
Introduction:
Our client, a fast-growing data solutions start-up based in Pretoria East, is looking for a motivated and detail-oriented Data Engineer to join their dynamic team. This is an exciting opportunity for someone passionate about data, technology, and innovation to contribute to building scalable data systems and play a key role in shaping the company's data engineering capability.
 
Job Purpose:
To design, develop and maintain robust, scalable and secure data engineering solutions across the full data lifecycle, from data ingestion and transformation to warehousing and integration.

REQUIREMENTS
Minimum education (essential):
  • BSc in Computer Science, Engineering or relevant field 
Minimum applicable experience (years):
  • 2-4 years
Required nature of experience:
  • Experience with SQL Server and Azure Synapse Analytics/Microsoft Fabric for query writing, indexing, performance tuning and schema design.
  • Hands-on experience developing ETL pipelines, including data extraction from REST/SOAP APIs, databases and flat files.
  • Proficiency in data transformation using Python and Azure-native tools.
  • Experience with data warehousing.
  • Background in data modelling, including dimensional modelling, schema evolution and versioning.
  • Practical knowledge of cloud-based data storage and processing using Azure Blob Storage.
  • Familiarity with pipeline optimisation, fault tolerance, monitoring and security best practices.
  • Experience developing web applications using C# and the .NET platform.
  • Experience with front-end development using Blazor, , JavaScript/TypeScript, HTML, CSS/SCSS. 
Skills and Knowledge (essential):
  • SQL Server, Azure Synapse Analytics, Azure Blob Storage, Microsoft Fabric
  • Python
  • REST/SOAP APIs, Data Extraction, Transformation, Loading (ETL)
  • Azure Data Factory, Pipeline Orchestration
  • Dimensional Modelling, Schema Evolution, Data Warehousing
  • Power BI
  • Performance Optimisation, Indexing, Query Tuning
  • Cloud Data Processing, Backups
  • C#, .NET, Blazor
  • JavaScript/TypeScript, HTML, CSS/SCS
Other:
  • Proficient in Afrikaans and English
  • Own transport and license
KEY PERFORMANCE AREAS AND OBJECTIVES
ETL and Pipeline Development
  • Design, build, and orchestrate efficient ETL pipelines using Azure Synapse for both batch and near-real-time data ingestion.
  • Extract data from a variety of structured and unstructured sources including REST APIs, SOAP APIs, databases, and flat files.
  • Apply robust data transformation logic using Python and native Azure Synapse transformation tools.
  • Optimise data flows for performance, scalability, and cost-effectiveness.
  • Implement retry mechanisms, logging and monitoring within pipelines to ensure data integrity and fault tolerance.
 Data Architecture and Management
  • Design and manage scalable and efficient data architectures using Microsoft SQL Server and Azure services, including Synapse Analytics/Microsoft Fabric and Blob Storage.
  • Develop robust schema designs, indexes and query strategies to support analytical and operational workloads.
  • Support schema evolution and version control, ensuring long-term maintainability and consistency across datasets.
  • Implement and maintain metadata repositories and data dictionaries for improved data governance and transparency.
  • Define and maintain role-based access control to ensure data security and compliance.
Data Warehousing and BI Integration
  • Architect and manage enterprise data warehouses using Azure Synapse Analytics.
  • Apply best practices for data loading, partitioning strategies, and storage optimisation.
  • Integrate warehousing solutions with Power BI and other analytics platforms for seamless business intelligence consumption.
Data Modelling & Standards
  • Develop and maintain conceptual, logical and physical data models.
  • Implement dimensional modelling techniques (e.g., star/snowflake schemas) to support advanced analytics and reporting.
  • Apply normalisation standards and relational modelling techniques to support OLTP and OLAP workloads.
  • Ensure consistency of data models across systems and support schema versioning and evolution.
 Reporting and Communication
  • Provide clear, timely updates on task status and progress to senior developers / management.
  • Contribute to reports, manuals, and other documentation related to software status, operation, and maintenance.
  • Collaborate effectively with team members and stakeholders using the appropriate communication channels.
  • Maintain system and product change logs and release notes according to company standards.
 Automation, Monitoring and Optimisation
  • Automate recurring data engineering tasks and deploy solutions with CI/CD best practices.
  • Implement monitoring and alerting mechanisms to detect data quality issues and pipeline failures.
  • Analyse and optimise query performance across platforms (SQL Server, Azure Synapse).
  • Support scalability planning and cost control by monitoring pipeline execution and resource usage
Security and Best Practices
  • Enforce security best practices for data access, including encryption and secure authentication.
  • Ensure compliance with data governance policies and applicable regulatory standards.
  • Document processes, architectural decisions and technical implementations in alignment with organisational standards
 Contribution to The Team
  • Collaborate with developers, data analysts, data scientists and business teams to understand data requirements and deliver scalable solutions.
  • Work with the team to integrate pipelines with source control and deployment workflows
Quality Management and Compliance
  • Document data processes, transformations and architectural decisions.
  • Maintain high standards of software quality within the team by adhering to good processes, practices and habits.
  • Ensure compliance to the established processes and standards for the development lifecycle, including but not limited to data archival.
  • Safeguard confidential information and data.


  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Boardroom Appointments Full time R600 000 - R1 200 000 per year

    This roles responsibility is to design, develop and maintain data-based solutions including ensuring that the operationalization of data pipelines and data stores are high-performing, efficient, organized, and reliable, given a set of business requirements and constraints.The Data Engineer will build and maintain secure and compliant data processing...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Boardroom Appointments Full time R600 000 - R1 200 000 per year

    REQUIRED MINIMUM EDUCATION/TRAINING:BCom or Bachelors degree in Information Technology / Computer Science / Engineering (IT) or similarData Engineering tools (Azure Data Engineer) certification will be an added advantageITIL or COBIT certification will be an added advantageREQUIRED MINIMUM WORK EXPERIENCE:2 - 4 years of working experience in an IT...

  • Data Process Engineer

    4 days ago

    Pretoria, Gauteng, South Africa BICRM Full time

    *** Our Johannesburg Office is Hiring *** 24 months renewable contract ***At BICRM, we are at the forefront of innovation in the financial services industry. We are seeking a Data / Process Engineer to join our team and support the Financial Surveillance department at a leading bank. This role involves conducting business process modelling and engineering to...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Werkie Full time R300 000 - R500 000 per year

    (eng= Data EngineerMore About the Company: An innovative company based in Faerie Glen, Pretoria, is looking for a skilled and motivated Data Engineer to join their growing team. This role offers the opportunity to work on large-scale data projects, build advanced data pipelines, and contribute to creating scalable data solutions that drive business success....

  • Data Engineer

    6 days ago

    Pretoria, Gauteng, South Africa Werkie Full time R360 000 - R600 000 per year

    (eng= Data EngineerMore About the Company: An innovative company based in Faerie Glen, Pretoria, is looking for a skilled and motivated Data Engineer to join their growing team. This role offers the opportunity to work on large-scale data projects, build advanced data pipelines, and contribute to creating scalable data solutions that drive business success....

  • Data Engineering and Architecture Lead

    2 weeks ago

    Pretoria, Gauteng, South Africa X, bigly labs Full time R1 200 000 - R2 400 000 per year

    AtX, bigly labs, we're Dis-Chem's high-performance innovation hub, where bold ideas meet data, design, and radical customer focus. Our mission is simple:power the future of healthcareby lowering costs, improving outcomes, and unlocking new possibilities. We're driven by one big question:How do we use data + technology today to create healthier lives...

  • Data Engineer EvdB

    1 week ago

    Pretoria, Gauteng, South Africa Mediro ICT Full time

    Design and implement effective database solutions and models to store and retrieve company data.Develop database architecture, data pipelines, and ETL processes.Define data governance policies and ensure data quality and security.Collaborate with IT teams and business units to understand data requirements.Optimize new and existing database systems for...

  • Data Scientist

    2 days ago

    Pretoria, Gauteng, South Africa On Point HR & Talent Solutions Full time

    Location – BedfordviewWe're looking for a curious, high-energyData Scientist whose love for SAS analyticssets them apart.In this hands-on role, you'll join a team that thrives on making sense of massive datasets, solving chunky business problems, and turning insights into impact. Here, you won't just crunch numbers; you'll drive innovation, shape...

  • Data Analyst- Project Performance

    1 week ago

    Pretoria, Gauteng, South Africa Zutari Full time

    DescriptionZutari: Co-creating an engineered impact.We co-create engineering solutions that have a positive impact and improve people's lives. Zutari values inclusion and recognises the importance of a diverse, talented workforce, believing that people need other people to succeed.What kind of talent do we pursue?We employ people with the right attitude and...

  • Data Analyst

    2 weeks ago

    Pretoria, Gauteng, South Africa Boardroom Appointments Full time R250 000 - R500 000 per year

    As a Data Analyst, you will be responsible for data models, analysing, and interpreting data to provide valuable insights and drive informed business decisions. The Data Analyst designs, create and deploy enterprise-scale data analytics solutions.The Data Analysts responsibility is to clean and transform data, design and build enterprise data models and...