Senior Data Engineer | Data Integration Developer
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
Introduction
Centurion | Gauteng | Permanent
7+ years experience required - Senior
Hands on Engineer | Data Integration Developer
This role is for someone that is not afraid to get their hands dirty.
Hands-on role that owns the full data engineering lifecycle:
- from ingestion and transformation through to platform architecture and production deployment – designing, building and running complex data pipelines on AWS.
Technical Skills Required:
- Python (advanced) – data pipeline development, automation and scripting.
- SQL (advanced) – complex query writing, performance optimisation and stored procedures.
- AWS (mid–advanced) – S3, Glue, Lambda, Redshift, RDS, IAM, CloudWatch, Athena.
- Data warehousing and data modelling – star/snowflake schemas, dimensional and analytics-oriented design.
- ETL/ELT pipeline development – batch, incremental and near-real-time patterns.
- Data lakehouse architecture on cloud-native platforms.
- CI/CD and Git – version control, automated testing and deployment pipelines.
- Power BI – supporting BI teams, data model integration and semantic layer understanding.
- Experience ingesting data from relational databases (e.g. SQL Server, Oracle), cloud storage, APIs, SFTP, MongoDB and file-based sources.
Highly Valued (Differentiating Skills):
- Apache Spark / PySpark for large-scale distributed processing.
- dbt for analytics engineering and transformation layer management.
- Apache Airflow or AWS Step Functions for orchestration.
- Terraform or CloudFormation (infrastructure-as-code).
- Real-time streaming – Kafka, Kinesis or similar.
- Machine learning workflow support – feature engineering and model data pipelines.
- Microsoft Azure or GCP as secondary cloud exposure.
Senior Data Engineer, AWS, Python, SQL, Data Engineering, ETL/ELT, Data Lakehouse, Data Warehousing, Power BI, CI/CD, Agile, AI-enabled engineering, Financial Services, Centurion, Hybrid
Duties & Responsibilities
Delivery Ownership within Agile Execution
- Contribute to data engineering outcomes aligned to sprint goals and programme-level commitments.
- Take part in Agile ceremonies – planning, estimation, prioritisation and continuous improvement.
- Decompose data features into implementable tasks with reliable effort estimates.
- Ensure outputs meet agreed functional, performance and data quality expectations.
Data Engineering & Pipeline Development
- Design and implement scalable ingestion and transformation pipelines across multiple systems and data domains, supporting batch and incremental processing.
- Build in error handling, monitoring and alerting to keep pipelines robust.
- Implement data validation and reconciliation mechanisms to maintain confidence in data assets.
Platform Design & Architecture (AWS)
- Design solutions aligned to the data platform strategy and target architecture, and contribute to the evolution of the cloud-based data lakehouse environment.
- Assess the impact of design choices on security, performance, cost and supportability.
- Identify integration points, upstream/downstream dependencies and risks early in the delivery lifecycle.
Technical Leadership, Coaching & Standards
- Provide guidance and technical oversight to less experienced data engineers.
- Support analytics, BI and data science teams with clarity on data structures, availability and pipeline behaviour.
- Review code and configuration, and help define shared standards, patterns and best practices.
- Optimise and simplify existing pipelines and data models.
AI-Enabled Engineering (Governance-Led)
- Promote responsible use of AI to improve engineering productivity and solution quality, within group and business-unit governance frameworks.
- Use only enterprise-approved AI tooling, and ensure no sensitive, proprietary or client information is exposed to public or unapproved AI platforms.
- Validate all AI-assisted outputs and retain accountability for correctness, compliance and production readiness.
- Apply AI to understand data flows and lineage, improve pipeline logic, support query formulation and test scenario creation, and accelerate documentation.
DevOps, Operational Support & Incident Response
- Apply version control, automated testing, CI/CD and structured releases.
- Contribute to operational readiness, deployment pipelines and monitoring.
- Diagnose and resolve data incidents, perform root cause analysis and implement corrective actions.
Security, Risk, Compliance & Governance
- Build solutions with privacy, access control and regulatory considerations embedded by design.
- Identify and mitigate data risks, and support remediation of audit findings and compliance gaps.
- Maintain accurate documentation of pipelines, data models and operational considerations, and keep work tracking up to date in Jira.
- Adhere to delivery governance, change management and release processes.
Stakeholder Engagement
- Communicate technical progress, constraints and decisions clearly to business and technical stakeholders.
- Collaborate across technology and analytics teams and contribute positively to team culture.
Desired Experience & Qualification
Skills & Experience:
- 7+ years’ professional experience in data engineering or related roles.
- Demonstrated delivery of production-ready, end-to-end data pipelines and cloud data platforms.
- Strong exposure to AWS-based data architectures.
- Experience across the full stack – ingestion, transformation, modelling and serving – not just a single layer.
- Hands-on experience working within Agile or SAFe delivery environments.
- Experience in financial services, insurance, investment management or other regulated data environments (preferred).
- Practical experience applying AI tooling in engineering workflows – including using AI responsibly and validating its outputs – to support pipeline understanding, optimisation, testing or documentation.
- Experience coaching or guiding other engineers.
Interested?
-
Pretoria, Gauteng, South Africa Placements24 Full-timePlacements24 is seeking a Senior Clinical Data Manager for a remote role supporting global clinical trials. You will oversee data quality, integrity, and timelines, collaborating with international teams to ensure compliant data management across studies.The position requires 6+ years in clinical data management, strong experience with EDC systems, and...
-
Senior Data Engineer
2 days ago
Pretoria, Gauteng, South Africa Hire Resolve Full-timeJob Title: Senior Data Engineer/Developer Location: Remote Industry : IT Salary: R 780 000 - R1200 000 Hire Resolve’s Client is seeking a talented and experienced Senior Data Engineer/Developer. The successful candidate will build and optimize the Microsoft-based data infrastructure powering real-time financial analytics, fraud detection, and business...
-
pretoria, gauteng, South Africa Placements24 Full-timePlacements24 is seeking a Senior Clinical Data Manager for a world-leading pharmaceutical company. This remote-first role oversees data management across global trials, ensuring data integrity and regulatory readiness for submissions. You will design and validate databases using EDC systems, develop data management plans, manage CRO relationships, mentor...
-
Senior Data Engineer
2 days ago
Pretoria, Gauteng, South Africa Skywaves Rise Remote Full-timeAbout the Role Our client is seeking a highly skilled Senior Data Engineer to join their fully remote team. The ideal candidate possesses a strong background in data warehousing, ETL/ELT processes, big data technologies, and cloud data platforms.Key ResponsibilitiesDesign, build, and maintain scalable data pipelines using big data technologies (e.g., Spark,...
-
Senior Data Engineer
3 days ago
Pretoria, South Africa Network Recruitment Full-timeAn innovative and data-driven organisation is seeking a talented Senior Data Engineer to join its growing team. This opportunity is ideal for a technically strong professional who enjoys solving complex data challenges and building scalable solutions that support analytics, reporting, and operational systems. In this role, you will be responsible for...
-
Senior Data Engineer
14 hours ago
Pretoria, Gauteng, South Africa Kohort Full-timeAbout Kohort Kohort is the leading machine learning platform for mobile gaming user acquisition, optimisation, and financial planning. We provide game studios with investment-grade LTV predictions using petabyte-scale data and machine learning. We are a group of gaming and data experts from King, Sensor Tower, Tripledot, Mistplay, Bloomberg and Football...
-
Senior Data Engineer
2 days ago
Pretoria, Gauteng, South Africa Placements24 Full-timeAbout the RoleOur client, WhatJobs Direct, is seeking a highly experienced Senior Data Engineer to join their innovative team based in Pretoria, Gauteng. This role is instrumental in building and maintaining robust, scalable data pipelines and infrastructure that support advanced analytics and business intelligence initiatives. You will work with large,...
-
Senior Cloudera Data Engineer
2 days ago
Pretoria, Gauteng, South Africa Indsafri India Private Limited Full-timeWe are seeking a highly experienced Senior Cloudera Data Engineer to join our dynamic data team. The ideal candidate will have extensive experience in designing, developing, and optimizing complex data pipelines and solutions within the Cloudera ecosystem. This role requires a deep understanding of big data technologies and a proven ability to translate...
-
Data Engineer
2 days ago
Pretoria, Gauteng, South Africa Huzzle.Com Remote Full-time R 1 - R 2 ContractAt Huzzle, we connect exceptional talent with exciting opportunities at leading companies across the UK, US, Canada, Europe, and Australia. We work with a diverse range of clients, including startups, digital agencies, and technology-driven businesses across industries such as SaaS, MarTech, FinTech, and AI. Unlike traditional outsourcing agencies, we place...
-
Senior Data Engineer
2 days ago
Pretoria, Gauteng, South Africa Ats Client Full-timeJob Description We are seeking a highly skilled and experienced Senior Data Engineer to design, build, and scale our modern data platform. This role is ideal for a hands‑on technical leader who can architect robust data pipelines, drive best practices in data engineering, and mentor a growing team. You will play a critical role in enabling analytics, data...
Similar jobs in Pretoria
This job listing is provided by a partner site
Get alerts for similar jobs
Receive job offers for Senior Data Engineer | Data Integration Developer
Get alerts for similar jobs
Receive job offers for Senior Data Engineer | Data Integration Developer