Senior Data Engineer

1 month ago

Cape Town, Western Cape, South Africa Aculocity Full time

Job Title: Senior Data Engineer

Job Summary:

Aculocity is seeking a highly skilled Senior Data Engineer to lead the design, development, and optimization of complex data infrastructure and pipelines. As a key member of our agile Data Engineering team, you will play a pivotal role in delivering high-impact reporting, analytics, and machine learning solutions that drive business success.

Key Responsibilities:

  • Develop and maintain scalable data pipelines and build out new API integrations to support continuing increases in data volume and complexity.
  • Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.
  • Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
  • Develop end-to-end ML pipelines encompassing the ML lifecycle from data ingestion, data transformation, model training, model validation, model serving, and model evaluation over time.
  • Collaborate closely with AI scientists to accelerate deployments of ML algorithms to production.
  • Setup CI/CD/CT pipelines, model repository for ML algorithms.
  • Deploy models as a service.
  • Contributes to engineering wiki, and documents work.
  • Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
  • Works closely with cross-functional teams of frontend and backend engineers, product managers, and analyst to enhance data models and support advanced BI and analytics.
  • Defines company data assets (data models), ETL jobs to populate data models.
  • Designs data integrations and data quality framework.
  • Designs and evaluates open source and vendor tools for data lineage.
  • Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.
  • Mentor junior data engineers, lead code reviews, and promote best practices and skill development.

Requirements:

  • BSc or MSc degree in Computer Science or a related technical field.
  • 5+ years of Python or R development experience.
  • 5+ years of MS SQL experience (PostgreSQL experience is a plus).
  • 5+ years of experience with Warehouse Architecture, schema design and dimensional data modeling.
  • 5+ years of experience in Data Analytics and Business Intelligence tools such as Power BI.
  • Ability in managing and communicating data warehouse plans to internal clients.
  • Experience designing, building, and maintaining data processing systems on multiple platforms both Cloud (Azure, AWS, MS Fabric is a plus) and On-Premises (MS SQL Server, SSIS).
  • Experience in ML Model deployment, ML frameworks and libraries.
  • Good experience in Apache Spark.
  • Experience debugging and reasoning about production issues is desirable.
  • Experience presenting demos and training of technical, non-technical and analytical resources.

Advantages:

  • Experience in data streaming is advantageous. Kafka and/or AWS Kinesis.
  • IoT device and systems integration.

  • Senior Data Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Goldman Tech Resourcing Full time

    Senior Data EngineerWe are seeking an experienced Senior Data Engineer to join our team at Goldman Tech Resourcing. The ideal candidate will have a strong background in designing and implementing data pipelines, with expertise in managing and optimizing Elasticsearch clusters. Additionally, they should have strong proficiency with data extraction,...

  • Senior Data Center Electrical Engineer

    1 month ago

    Cape Town, Western Cape, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    About the RoleAs a Senior Data Center Electrical Engineer at Amazon Data Services South Africa (Pty) Ltd, you will play a critical role in ensuring the design, construction, and operation of our data centers meet the highest standards of safety, security, and efficiency. You will be responsible for providing technical expertise and guidance to our teams,...

  • Senior Network Engineer

    1 month ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job Title: Senior Network EngineerWe are seeking a highly skilled Senior Network Engineer to join our team at Dimension Data. As a key member of our network engineering team, you will be responsible for designing, implementing, and maintaining complex network infrastructures for our clients.Key Responsibilities:Design and implement network architectures to...

  • Senior Data Center Electrical Engineer

    2 weeks ago

    Cape Town, Western Cape, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    About the Role:As a Senior Data Center Electrical Engineer at Amazon Data Services South Africa (Pty) Ltd, you will play a critical role in ensuring the high availability and reliability of our data centers. Your expertise will be instrumental in designing, implementing, and maintaining electrical systems that meet the stringent requirements of our...

  • Senior Data Engineer

    4 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Job Title: Senior Data Engineer - Data ArchitectJob Summary:We are seeking a highly skilled Senior Data Engineer - Data Architect to join our team at Network Recruitment. As a Senior Data Engineer - Data Architect, you will be responsible for designing and implementing efficient data systems, ensuring data quality, and maintaining data infrastructure...

  • Senior Data Engineer

    1 month ago

    Cape Town, Western Cape, South Africa Datonomy Solutions Full time

    We are seeking a highly skilled Senior Data Engineer to join our team at Datonomy Solutions. In this role, you will be responsible for designing and implementing data solutions using industry best practices. You should have a strong background in data and business intelligence, with experience in SQL technology, reporting technology, and cloud technology.Key...

  • Senior Data Engineer

    1 month ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Senior Data Engineer - Data ArchitectFully remote opportunity for a Senior Data Engineer - Data Architect to join our team at Network Recruitment.Key Responsibilities:Design and implement data architectures to support business growth and scalabilityDevelop and maintain large-scale data pipelines and systemsCollaborate with cross-functional teams to ensure...

  • Senior Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Datonomy Solutions Full time

    We are looking for a Senior Data Engineer at Datonomy Solutions to support and manage BI analytics solutions. In this role, you should have a background in data and business intelligence. You should be analytical; an excellent communicator and a problem-solving aptitude is needed in this role. Ultimately, you will enhance our business intelligence systems to...

  • Senior Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa E-Merge Full time

    A Fintech department, delivering financial information to the company, seeks a Senior Data Engineer to design and develop robust data frameworks and create data services that deliver actionable insights to their global customer base.E-Merge is a globally recognised brand that rewards technical prowess.Responsibilities:Improve processes and...

  • Senior Avaya Telephony Engineer

    4 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job OverviewWe are seeking an experienced Senior Avaya Telephony Engineer to join our team at Dimension Data. This role will be responsible for providing technical support and solutions to clients, ensuring that their IT infrastructure and systems remain operational.The ideal candidate will have a strong background in Avaya telephony administration, with...

  • Senior Data Center Electrical Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    About the Job:We are seeking a highly skilled Senior Data Center Electrical Engineer to join our team at Amazon Data Services South Africa (Pty) Ltd. As a key member of our team, you will be responsible for providing full life-cycle support to our data centers from design inception through site improvement and maintenance.Key Responsibilities:• Perform...

  • Senior Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Aculocity Full time

    About the Role:The Senior Data Engineer will play a pivotal role in our agile Data Engineering team, leading the design, development, and optimization of complex data infrastructure and pipelines.Key Responsibilities:Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and...

  • Senior Network Engineer

    1 month ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job SummaryWe are seeking a highly skilled Senior Network Engineer to join our team at Dimension Data. As a key member of our network engineering team, you will be responsible for designing, implementing, and supporting new IT projects in the areas of modern networks, security solutions, and cloud technologies.You will work on a variety of complex...

  • Senior .NET Software Engineer

    4 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    **Job Description for Senior .NET Software Engineer at Dimension Data**We are seeking a highly skilled and experienced .NET Developer to join our team as a Senior .NET Software Engineer.The successful candidate will be responsible for designing, developing, testing, and implementing viable IT solutions using the latest .NET technologies.Main...

  • Senior Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team at Network Recruitment. As a Senior Data Engineer - Data Pipeline Architect, you will play a key role in designing and implementing efficient data pipelines for our clients in the financial services sector.Key ResponsibilitiesCollaborate with data scientists and analysts to develop...

  • Senior Data Engineering Specialist

    2 weeks ago

    Cape Town, Western Cape, South Africa OfferZen Ltd Full time

    At OfferZen Ltd, we're looking for a seasoned Senior Data Engineering Specialist to join our team. As a key member of our engineering team, you'll play a crucial role in designing, building, and maintaining our large-scale data systems.Key Responsibilities:Design, develop, and deploy large-scale data processing systemsCollaborate with cross-functional teams...

  • Senior Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Progressive IT Resourcing Full time

    Job Title: Senior Data Engineer - Cloud ExpertJob Summary: We are seeking a highly skilled Senior Data Engineer - Cloud Expert to join our team at Progressive IT Resourcing. As a key member of our engineering organization, you will be responsible for designing, building, and maintaining scalable data platforms on cloud technologies.Key...

  • Senior Data Engineer

    1 month ago

    Cape Town, Western Cape, South Africa LexisNexis South Africa Shared Services (Pty) Ltd Company Full time

    About the RoleWe're seeking a highly skilled Senior Data Engineer to join our team at LexisNexis. As a key member of our data engineering team, you will play a critical role in designing, building, and maintaining our data infrastructure.ResponsibilitiesDesign and implement scalable data pipelines to process large datasets in real-time.Collaborate with...

  • Senior Cloud Infrastructure Engineer

    4 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job SummaryWe are seeking a highly skilled Senior Cloud Infrastructure Engineer to join our team at Dimension Data. As a key member of our engineering team, you will be responsible for designing, deploying, and managing scalable cloud infrastructure on AWS.Key ResponsibilitiesDesign and implement efficient and reliable automated scripts to automate...

  • Senior Performance Test Engineer

    4 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job SummaryWe are seeking a skilled Senior Performance Test Engineer to join our team at Dimension Data. This role involves delivering high-quality software products through performance testing, ensuring seamless system integration and user acceptance.Key ResponsibilitiesThe successful candidate will be responsible for creating and maintaining performance...