Data Engineer

2 weeks ago

Cape Town, Western Cape, South Africa Sanlam Group Full time
Job title : Data Engineer - Group Data & ML Platform TribeJob Location : Western Cape, BellvilleDeadline : May 03, 2025Quick Recommended Links
  • Jobs by Location
  • Job by industries

What will make you successful in this role?

What success will look like for you 12 months into the role.

(Ingestions Squad) Built configurable data ingestion platforms to collect and protect data from any source within two weeks or less.

  • The team has bought and built to assemble a tool soup of persistent and secure ingestion tools, optimized for cost efficiency, with reliability as a non-negotiable standard. Success will be measured by the consistent achievement of the two-week ingestion target while maintaining high reliability and security.

(Data products empowerment squad) Simplified data transformation and product development for the platforms SQL only users:

  • Developed solutions that allow SQL-only users to easily schedule and orchestrate complex data transformation processes, such as through Directed Acyclic Graphs (DAGs). Success will be measured by the 100% adoption of your tools by platform users, the full deprecation of custom solutions and stored procedures, and the reliable orchestration of multi-stage transformation processes underlying our company wide data products. 

(Insights & ML tooling squad) Accelerated the development of insights and ML models by analysts and data scientists across the company:

  • Built systems that empower analysts and data scientists to produce the insights and ML models needed to drive customer and commercial success with minimal engineering skills and platform friction. Success will be measured by the number of successful deployments of insights and models per month that directly contribute to improving our business success metrics.

Engineered the platform for observability and performance.

  • Made your systems observable from the outside, taking ownership of their performance. Facilitated real-time tracking of the system's internal state through logs, metrics, and traces. Defined clear and measurable Service Level Indicators (SLIs) and Service Level Objectives (SLOs), and integrate these into team OKRs to drive excellence. Success will be measured by meeting or exceeding defined performance and reliability benchmarks, and the quick identification and resolution of issues.

Engineer the platform with privacy and security by design. 

  • Secured the platform through compliance engineering initiatives that reduced the time and cognitive load related to compliance. Implemented our Data Governance and Privacy policies effectively, leveraging encryption, firewalls, VPCs, Role Based Access Control and IAM integrations such as Active Directory. Success will be measured by the platform's adherence to security and privacy standards, with minimal compliance overhead for users.

Leveraged containerization and infrastructure as code. 

  • Implemented containerization strategies and infrastructure as code (IaC) practices to ensure scalable, repeatable, and efficient deployment of data services. Utilized tools like Docker and Terraform to automate infrastructure management and streamline the development pipeline. Success will be measured by the consistency and reliability of deployments, reduced deployment times, and increased system scalability.

What will make your successful in this role (continue)

What success will look like for you 12 months into the role.

Contributed to technology decisions. 

  • Engaged in technology spikes to clarify needs and articulate solution criteria. Evaluated different options through hands-on experimentation, leading to documented evaluations and recommendations that were understood by technology leaders or users across the company. Success will be measured by the quality and clarity of your contributions to technology decisions and the adoption of your recommendations.

Invested in your own development.

  • Executed on a culture of excellence and continuous improvement for both your part of the platform and your own abilities. Lived our culture of ownership, curiosity, and continuous learning. Participated in pre and post-mortems, sharing lessons learned across the organization along with how you are improving based on those lessons. Success is measured by your active participation in continuous learning initiatives and the tangible improvements in platform performance and personal growth.

Technical leadership and coaching. 

  • Enable a culture of continuous learning and assist other engineers in their development journey.

Qualification

  • Matric
  • National Diploma in an Information Technology related discipline or a bachelor's degree in computer science, Statistics, Informatics, Information Systems, or a quantitative field will be recommended.

Experience

  • 5+ years' experience as a data engineer in data platform or service engineering team with at least 2 in a modern context with cloud technologies
  • Track record of high impact and investing in your own development
  • Research / Data Analysis  jobs

  • Data Center Solutions Expert

    19 hours ago

    Cape Town, Western Cape, South Africa Data Centrix Full time

    About the Role:We are seeking a highly experienced Data Center Solutions Expert to lead our data center operations team. As a key member of our team, you will be responsible for designing, implementing, and maintaining complex data center solutions using Hyper-V, Exchange, and Microsoft 365 technologies.Responsibilities:Design and implement data center...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa WatersEdge Solutions Full time

    Data Engineer – Microsoft Fabric (Team Lead)  We are looking for an experienced Data Engineer (Team Lead) with expertise in Microsoft Fabric to spearhead the development and optimisation of a next-generation data warehouse. This hybrid role requires flexibility for 3 days in-office per week in Cape Town and offers an exciting opportunity to build...

  • Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Watersedge Solutions Full time

    Data Engineer – Microsoft Fabric (Team Lead)We are looking for an experienced Data Engineer (Team Lead) with expertise in Microsoft Fabric to spearhead the development and optimisation of a next-generation data warehouse.This hybrid role requires flexibility for 3 days in-office per week in Cape Town and offers an exciting opportunity to build and lead a...

  • Data Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    FinTech IndustryData Engineer RoleSouth Africa (Remote/Hybrid) Python, AWS & PySpark Experience a Must An innovative FinTech firm is hiring a Data Engineer to drive data infrastructure initiatives and streamline data processes. If you thrive in a fast-paced, cloud-first environment and enjoy designing efficient data systems from scratch, this is your chance...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Elixirr Digital Full time

    Do you like building data systems and pipelines? Do you enjoy interpreting trends and patterns? Are you able to recognize the deeper meaning of data?Join Elixirr Digital as a Data Engineer and help us analyze and organize raw data to provide valuable business insights to our clients and stakeholdersAs a Data Engineer, you will be responsible for ensuring the...

  • Data Engineering

    2 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    Data Engineering & Streaming Lead –Cape Town.The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines.The role involves working closely with various infrastructure and operations teams to maintain data infrastructure.As a senior engineer, the candidate will develop and maintain data...

  • Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Datonomy Solutions Full time

    Job Overview: We are seeking an experienced Data Engineer with strong Solution Architecture expertise to lead the data engineering efforts for a financial data migration project. The role focuses on designing, building, and optimizing data pipelines to support the transition from an on-premise Hyperion system to OneStream in the cloud. The ideal candidate...

  • Data Engineer

    4 days ago

    Cape Town, Western Cape, South Africa DLK Group Full time

    Data EngineerWe require a Data Engineer and this is an office-based position.Duties & ResponsibilitiesModifying the database structure, as necessary, from information given by application developers.Enrolling users and maintaining system security.Controlling and monitoring user access to the database.Monitoring and optimizing the performance of the...

  • Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Oxyon Human Capital Solutions Full time

    Job Summary:The Data Engineer will manage and analyze the company's data from battery systems, microgrids, and energy management solutions. This role includes designing databases, developing analytics/reporting systems, and applying machine learning (ML) for actionable insights. The Data Engineer will also integrate AI/ML into the EMS product to drive...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Oxyon Human Capital Solutions Full time

    Job Summary: The Data Engineer will manage and analyze the company's data from battery systems, microgrids, and energy management solutions. This role includes designing databases, developing analytics/reporting systems, and applying machine learning (ML) for actionable insights. The Data Engineer will also integrate AI/ML into the EMS product to drive...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Goldman Tech Resourcing Full time

    A company dedicated to partnering with underserved communities to co-develop products and services that promote dignity, safety, and financial stability is seeking a Data Engineer. This role is based in Cape Town and follows a hybrid working model. Skills Required · 4 years of relevant work experience in data engineering or a similar role · Experience with...

  • Data Engineer –

    3 weeks ago

    Cape Town, Western Cape, South Africa 5th Dimension Consulting Full time

    Data Engineer: Ingestions Squad (Group Data & ML Platform)We're a newly created engineering organisation on a mission to empower and accelerate autonomous value creation through insights and ML across the group. Our main users are the data scientists and analysts who sit inside the business units and support them in developing and re-developing their...

  • Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Surgo HR & Training Full time

    Surgo (PTY) Ltd. has partnered with a global analytics and digital solutions company serving industries including insurance, healthcare, banking and financial services, media, retail, and others. They aim to bridge the gap between digital expectations and real outcomes for international companies with Digital Intelligence.Our client is recruiting for a Data...

  • Data Engineer

    7 days ago

    Cape Town, Western Cape, South Africa EQ Plus Full time

    Key Responsibilities:Designing and implementing scalable processes to manage large data sets, ensuring quality and consistency across sources. This involves developing and maintaining data pipelines, creating optimal extraction patterns, and aligning cloud strategy with data architecture. As the Data Engineer, you will also engineer data formats for...

  • Data Engineering

    4 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued...

  • Data Engineering

    2 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    Data Engineer position currently available in Cape Town.The Data Engineer will be responsible for designing, building, and optimising data pipelines and ensuring the efficient movement and transformation of data across various systems.The ideal candidate will have expertise in SQL, data modelling, and experience with cloud-based or on-premises data...

  • Network Operations Engineer

    15 hours ago

    Cape Town, Western Cape, South Africa Data Centrix Full time

    As a Senior Networks Engineer at Data Centrix, you will be responsible for designing, implementing, and managing complex network architectures to meet the evolving needs of our clients.Primary ResponsibilitiesDesign and implement network architectures that meet the specific requirements of our clients, including scalability, security, and reliability.Manage...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Network It Full time

    Reference : BIT-CdP-1We are looking for a skilled Data Engineer to join our clients' investment team to oversee their database architecture, data flow, and data processes.The role involves efficiently managing the reception, transfer, processing, analysis, and integration of third-party systems for investment research and business operational...

  • Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Afrizan Personnel Full time

    A leading organisation is looking for a Data Engineer to join their team. The main purpose of the role is to prepare and manage large volumes of data for analytical or operational uses. This role involves designing, building, and maintaining the architecture used for data processing and storage. This includes tasks such as:Data Modeling and Design:...