Data Engineer –

5 days ago

Cape Town, Western Cape, South Africa 5th Dimension Consulting Full time

Data Engineer: Ingestions Squad (Group Data & ML Platform)

We're a newly created engineering organisation on a mission to empower and accelerate autonomous value creation through insights and ML across the group. Our main users are the data scientists and analysts who sit inside the business units and support them in developing and re-developing their financial and digital products to better meet the needs of our clients and our shareholders. Our other users are the data management and data product teams who produce the data sets analysts and scientists need to launch straight into their work.

The tribe is made up of 3 different engineering squads covering everything from how we ingest and store data, to how we develop and share ML artefacts and insights. We have one for ingestions, one for data storage and processing and one for insights & ML tooling. We have one data engineering talent pipeline for the whole tribe, where placements into specific squads are explored later in the interview process based on squad needs and candidate abilities. Data engineers report to the tech leads of the squad they are embedded into.

Responsibilities

  • Build configurable data ingestion platforms to collect and protect data from any source within two weeks or less. The team will build and assemble a tool suite of persistent and secure ingestion tools, optimized for cost efficiency, with reliability as a non-negotiable standard. Success will be measured by the consistent achievement of the two-week ingestion target while maintaining high reliability and security.
  • Engineer the platform for observability and performance. Make your systems observable from the outside, taking ownership of their performance. Facilitate real-time tracking of the system's internal state through logs, metrics, and traces. Define clear and measurable Service Level Indicators (SLIs) and Service Level Objectives (SLOs), and integrate these into team OKRs to drive excellence. Success will be measured by meeting or exceeding defined performance and reliability benchmarks, and the quick identification and resolution of issues.
  • Engineer the platform with privacy and security by design. Secure the platform through compliance engineering initiatives that reduce the time and cognitive load related to compliance. Implement our Data Governance and Privacy policies effectively, leveraging encryption, firewalls, VPCs, Role Based Access Control and IAM integrations such as Active Directory. Success will be measured by the platform's adherence to security and privacy standards, with minimal compliance overhead for users.
  • Leverage containerization and infrastructure as code. Implement containerization strategies and infrastructure as code (IaC) practices to ensure scalable, repeatable, and efficient deployment of data services. Utilize tools like Docker and Terraform to automate infrastructure management and streamline the development pipeline. Success will be measured by the consistency and reliability of deployments, reduced deployment times, and increased system scalability.
  • Contribute to technology decisions. Engage in technology spikes to clarify needs and articulate solution criteria. Evaluate different options through hands-on experimentation, leading to documented evaluations and recommendations that are understood by technology leaders or users across the company. Success will be measured by the quality and clarity of your contributions to technology decisions and the adoption of your recommendations.

Requirements

Qualifications

  • Matric
  • National Diploma in an Information Technology related discipline or a Bachelor's degree in Computer Science, Statistics, Informatics, Information Systems, or a quantitative field is recommended.

Experience

  • 5+ years experience as a Data Engineer in a data platform or service engineering team with at least 2 in a modern context with cloud technologies.
  • Track record of high impact and investing in your own development.

Skills/Attributes

  • You look for ways to do, instead of reasons not to.
  • Where some see impossible, you see a way. You have the strong conviction in our ability to innovate and you turn obstacles into opportunity, chance into change.
  • You move fast and fix things. You publish early and often; when you fail, you learn and go again. You know that speed gives you the license to fail, because you're moving fast enough to correct course.
  • You love creating more with less.
  • You seek out and value different perspectives.

If you would like to submit your CV for consideration, please fill out this form and we will get in touch with you shortly.

#J-18808-Ljbffr

  • Enterprise Cloud Engineering Leader

    1 day ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Dimension Data is seeking an Enterprise Cloud Engineering Leader to spearhead the development and delivery of cloud-based solutions. As a senior technical expert, you will be responsible for leading a team of engineers in designing and implementing complex cloud architectures that meet the needs of our clients.

  • Data Engineer

    4 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    ? Join PBT Group as a Senior Data Engineer ?Are you a dynamic data engineer passionate about optimising large-scale data ecosystems? PBT Group is looking for a Data Engineer to design and implement robust, scalable processes that support our growing data engineering capability. This role is pivotal in advancing our cloud engineering and data management...

  • Data Engineer

    2 days ago

    Cape Town, Western Cape, South Africa Watersedge Solutions Full time

    Data Engineer – Microsoft Fabric (Team Lead)We are looking for an experienced Data Engineer (Team Lead) with expertise in Microsoft Fabric to spearhead the development and optimisation of a next-generation data warehouse.This hybrid role requires flexibility for 3 days in-office per week in Cape Town and offers an exciting opportunity to build and lead a...

  • Data Engineer

    2 days ago

    Cape Town, Western Cape, South Africa Boardroom Appointments Full time

    At Boardroom Appointments, we are seeking a highly skilled Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing scalable data pipelines and architectures using PySpark, Python, and SQL.Key ResponsibilitiesDesign and implement distributed data pipelines and compute tiers operating on...

  • Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Datonomy Solutions Full time

    Job Overview:We are seeking an experienced Data Engineer with strong Solution Architecture expertise to lead the data engineering efforts for a financial data migration project. The role focuses on designing, building, and optimizing data pipelines to support the transition from an on-premise Hyperion system to OneStream in the cloud. The ideal candidate...

  • Data Engineer

    3 days ago

    Cape Town, Western Cape, South Africa Datonomy Solutions Full time

    Job Overview: We are seeking an experienced Data Engineer with strong Solution Architecture expertise to lead the data engineering efforts for a financial data migration project. The role focuses on designing, building, and optimizing data pipelines to support the transition from an on-premise Hyperion system to OneStream in the cloud. The ideal candidate...

  • Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Goldman Tech Resourcing Full time

    A company that is built around innovative work, great people, and a singular focus on the world of commerce is seeking a Data Engineer. Candidates to be based in Cape Town, however there is a hybrid working model. Skills Required · BSc in Computer Science, Computer Engineering, or a related field. · 3 years of experience in data engineering or software...

  • Data Engineer

    4 days ago

    Cape Town, Western Cape, South Africa Fact Full time

    About Fact:We are a data-driven organization seeking a skilled Analytics Engineer to join our team.This role is key in building, optimizing, and maintaining data pipelines that drive our business intelligence and analytics efforts.Key Performance OutputsData Pipeline Development: Design, build, and maintain robust ETL (Extract, Transform, Load) pipelines to...

  • Data Engineer

    6 days ago

    Cape Town, Western Cape, South Africa Oxyon Human Capital Solutions Full time

    Job Summary:The Data Engineer will manage and analyze the company's data from battery systems, microgrids, and energy management solutions. This role includes designing databases, developing analytics/reporting systems, and applying machine learning (ML) for actionable insights. The Data Engineer will also integrate AI/ML into the EMS product to drive...

  • Data Engineer

    2 days ago

    Cape Town, Western Cape, South Africa Surgo HR & Training Full time

    Surgo (PTY) Ltd. has partnered with a global analytics and digital solutions company serving industries including insurance, healthcare, banking and financial services, media, retail, and others. They aim to bridge the gap between digital expectations and real outcomes for international companies with Digital Intelligence.Our client is recruiting for a Data...

  • Data Engineering

    4 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued...

  • Data Engineering

    4 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued...

  • Data Engineering

    2 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    The Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued...

  • Data Engineer

    4 weeks ago

    Cape Town, Western Cape, South Africa Goldman Tech Resourcing Full time

    An innovative retail media solutions provider is looking for a Data Engineer to join their team. This company specializes in dynamic digital display networks and strategic in-store advertising, empowering brands to maximize visibility and customer engagement through cutting-edge technology and impactful campaigns. Skills Required · BSc in Computer Science,...

  • Data Engineer

    2 days ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Network Recruitment is seeking a skilled Data Engineer to join their team. As an experienced professional in building robust data warehouses and coding, you will thrive in dynamic environments and make valuable contributions to the company and community.Key Responsibilities:Write efficient and complex SQL Queries to optimize database performance.Experience...

  • Senior Electrical Engineer

    5 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    About Dimension Data:Dimension Data is a global technology company that enables its clients to achieve their business outcomes by using innovative technology solutions.Job Description:We are seeking an experienced Electrical Projects Manager to lead and manage electrical construction projects. The ideal candidate will have a strong background in electrical...

  • Systems Engineering Expert

    1 day ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Dimension Data seeks an experienced IT professional to join our team as a Systems Engineering Expert. This key role will be responsible for implementing and maintaining various technology systems, including Azure, AWS, Oracle Databases, SAP applications, and more.In this position, you will work closely with cross-functional teams to design and develop ETL...

  • Software Quality Assurance Engineer

    1 day ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    We are seeking a skilled Software Quality Assurance Engineer to join our team at Dimension Data. In this role, you will be responsible for designing and running tests on software usability.As a Software Quality Assurance Engineer, you will conduct thorough testing, analyze results, and report observations to the design team. You may also interact with...

  • Data Engineer

    9 hours ago

    Cape Town, Western Cape, South Africa Network It Full time

    Reference : BIT-CdP-1We are looking for a skilled Data Engineer to join our clients' investment team to oversee their database architecture, data flow, and data processes.The role involves efficiently managing the reception, transfer, processing, analysis, and integration of third-party systems for investment research and business operational...

  • Data Engineering Specialist

    1 day ago

    Cape Town, Western Cape, South Africa Capitalh Full time

    We're looking for a Data Engineering Specialist to join our team at CAPITALH. As a key member of our data engineering team, you'll design, develop, and maintain data pipelines and architectures.A Bachelor's degree in Computer Science or Software Engineering is required, along with 5+ years of experience in data engineering, preferably with Google Cloud...