Data Engineer

1 week ago

Pretoria, South Africa 7087Caea-55C1-4426-B223-F722720D8411 Full time

We are seeking a hands‑on Data Engineer with strong experience in building scalable data pipelines and analytics solutions on Databricks. You will design, implement, and maintain end‑to‑end data flows, optimize performance, and collaborate with data scientists, analytics, and business stakeholders to turn raw data into trusted insights. Key Responsibilities Design, develop, test, and maintain robust data pipelines and ETL/ELT processes on Databricks (Delta Lake, Spark, SQL, Python / Scala / SQL notebooks). Architect scalable data models and data vault / dimensional schemas to support reporting, BI, and advanced analytics. Implement data quality, lineage, and governance practices; monitor data quality metrics and resolve data issues proactively. Collaborate with Data Platform Engineers to optimize cluster configuration, performance tuning, and cost management in cloud environments (Azure Databricks). Build and maintain data ingestion from multiple sources (RDBMS, SaaS apps, files, streaming queues) using modern data engineering patterns (CDC, event‑driven pipelines, change streams, Lakeflow Declarative Pipelines). Ensure data security and compliance (encryption, access controls) in all data pipelines. Develop and maintain CI / CD pipelines for data workflows; implement versioning, testing, and automated deployments. Partner with data scientists and analysts to provision clean data, notebooks, and reusable data products; support feature stores and model deployment pipelines where applicable. Optimize Spark jobs for speed and cost; implement job scheduling, monitoring, and alerting. Document data lineage, architecture, and operational runbooks; participate in architectural reviews and best‑practice governance. Qualifications / Experience Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field. 3+ years of hands‑on data engineering experience. Expertise with Apache Spark (PySpark), Databricks notebooks, Delta Lake, and SQL. Strong programming skills in Python for data processing. Experience with cloud data platforms (Azure) and their Databricks offerings; familiarity with object storage (ADLS). Proficient in building and maintaining ETL/ELT pipelines, data modeling, and performance optimization. Knowledge of data governance, data quality, and data lineage concepts. Experience with CI / CD for data pipelines, and orchestration tools (GitHub Actions, Asset Bundles or Databricks' jobs). Strong problem‑solving skills, attention to detail, and ability to work in a collaborative, cross‑functional team. Advantageous Skills Experience with streaming data (Structured Streaming, Kafka, Delta Live Tables). Familiarity with materialized views, streaming tables, data catalogs and metadata management. Knowledge of data visualization and BI tools (Splunk, Power BI, Grafana). Experience with data security frameworks and compliance standards relevant to the industry. Certifications in Databricks or cloud provider platforms. #J-18808-Ljbffr

  • Data Engineer

    2 weeks ago

    Pretoria, South Africa Kamo Placements Cc Full time

    Reference : JHB G-1Data Engineer Duties & Responsibilities JOB DESCRIPTION Data Engineering & Pipeline ManagementDesign, build, and optimize T-SQL stored procedures, functions, and scripts for high-volume data processing and ECM scoring.Develop, deploy, and monitor end-to-end ETL / ELT workflows (e.g., SQL Server Agent, SSIS, Azure Data Factory, or Airflow)...

  • Data Engineer

    2 weeks ago

    Pretoria, South Africa Bizmod Full time

    A leading financial institution is seeking a Senior Data Engineer / Analyst to join a multi‑year IT transformation programme aimed at renewing and modernising its financial surveillance systems. Key Responsibilities Define and optimise business processes and value chains relevant to the Financial Surveillance function. Identify, analyse, and improve...

  • Data Engineer

    1 week ago

    Pretoria, Gauteng, South Africa Interfront SOC Ltd. Full time R500 000 - R1 200 000 per year

    Key Responsibilities:Data Engineering & Pipeline ManagementDesign, build, and optimize T-SQL stored procedures, functions, and scripts for high-volume data processing and ECM scoring.Develop, deploy, and monitor end-to-end ETL/ELT workflows (e.g., SQL Server Agent, SSIS, Azure Data Factory, or Airflow) with checkpoint/rollback, job tracking, and recovery...

  • Data Engineer

    4 days ago

    Pretoria, Gauteng, South Africa Boardroom Appointments Full time R600 000 - R1 200 000 per year

    This roles responsibility is to design, develop and maintain data-based solutions including ensuring that the operationalization of data pipelines and data stores are high-performing, efficient, organized, and reliable, given a set of business requirements and constraints.The Data Engineer will build and maintain secure and compliant data processing...

  • Data Engineer

    1 week ago

    Pretoria, Gauteng, South Africa Interfront Full time R800 000 - R1 200 000 per year

    Job Reference Number: Recruit198-199Department: TaxBusiness Unit:Industry: Information TechnologyJob Type: ContractPositions Available: 2Salary: Market RelatedPerform data cleansing, preparation & analysis to support business intelligence & machine learning. Design, build & maintain robust data pipelines, feature stores & scheduled workflows that power the...

  • Data Engineer

    4 days ago

    Pretoria, Gauteng, South Africa Boardroom Appointments Full time R600 000 - R1 200 000 per year

    REQUIRED MINIMUM EDUCATION/TRAINING:BCom or Bachelors degree in Information Technology / Computer Science / Engineering (IT) or similarData Engineering tools (Azure Data Engineer) certification will be an added advantageITIL or COBIT certification will be an added advantageREQUIRED MINIMUM WORK EXPERIENCE:2 - 4 years of working experience in an IT...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Mediro ICT Full time R900 000 - R1 200 000 per year

    Design, develop, and maintain scalable data pipelines to ingest, process, and store structured and unstructured data from multiple sources.Develop ETL/ELT processes to transform raw data into clean, analysable formats for reporting, analytics and dashboards.Work with cloud data platformsEnsure data security and complianceIntegrate and consolidate data from...

  • Data Engineer

    3 weeks ago

    Pretoria, South Africa Mi-C3 International Ltd Full time

    As a Data Integration Engineer, you will be responsible for designing, implementing, and maintaining data integration solutions to handle real‑time streaming data from various sources like IoT / IIoT protocols, third‑party APIs, or even raw files. Your main objective will be to process data in real‑time and provide valuable insights for our...

  • Data Process Engineer

    3 weeks ago

    Pretoria, South Africa Bicrm Full time

    Our Johannesburg Office is Hiring 24 months renewable contract At BICRM, we are at the forefront of innovation in the financial services industry. We are seeking a Data / Process Engineer to join our team and support the Financial Surveillance department at a leading bank. This role involves conducting business process modelling and engineering to enhance...

  • Senior Data Engineer

    3 weeks ago

    Pretoria, South Africa Moyo Full time

    WELCOME TO MOYO Senior Data Engineer We are searching for an experienced Senior Data Engineer to join us in driving significance together. Primary Duties and Responsibilities In this consulting role, you will play a key role in understanding our clients' business objectives and translating them into robust, scalable data solutions. You will lead the...