Head of Data Engineering

2 weeks ago

Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time
Job Title: Head of Data Engineering

We are seeking a highly skilled and experienced Head of Data Engineering to join our team at Carlysle Human Capital. As a key member of our IT department, you will be responsible for designing, developing, and maintaining scalable data pipelines and related systems in a research-focused environment.

Key Responsibilities:
  • Design and develop end-to-end technical aspects of high-performance compute and storage platforms for data pipelines in a research data-intensive environment.
  • Design and develop end-to-end data pipelines, including software, databases, and processes, to support research scientists and data managers.
  • Support ETL processes, including data ingestion, transformation, validation, and integration, using various tools and frameworks.
  • Optimize data performance, scalability, and security.
  • Provide technical guidance and support to data analysts and research scientists.
  • Design data integrations and data quality frameworks.
  • Collaborate with the IT department to develop the strategy for long-term scientific Big Data platform architecture.
  • Document and communicate data engineering processes and solutions effectively.
Requirements:
  • Bachelor's degree or higher in Computer Science, IT, Engineering, Mathematics, or a related field.
  • Industry-recognized IT-related certification and technology qualification, such as databases and data-related certifications.
  • 7+ years of experience in Data Engineering, High Performance Computing, Data Warehousing, or Big Data Processing.
  • Strong experience with technologies such as Hadoop, Kafka, Nifi, or Spark, or Cloud-based Big Data processing environments like Amazon Redshift, Google BigQuery, and Azure Synapse Analytics.
  • At least 5 years of advanced experience and strong proficiency in UNIX, Linux, Windows Operating Systems, and preferably containerization technologies like Docker, Kubernetes, etc.
  • Working knowledge of various data-related programming, scripting, or data engineering tools, such as Python, R, Julia, T-SQL, PowerShell, etc.
  • Strong working experience with software compute and virtualization platforms, such as VMware, Hyper-V, OpenStack, KVM, etc.
  • Strong working experience with hardware compute platforms, including high-performance compute cluster hardware and related technologies.
Preferred Qualifications:
  • Strong experience working with various relational database technologies, such as MS SQL, MySQL, PostgreSQL, as well as NoSQL databases, such as MongoDB, Cassandra, etc.
  • Experience with Big Data technologies, such as Hadoop, Spark, and Hive.
  • Experience with data pipeline tools, such as Airflow, Spark, Kafka, or Dataflow.
  • Experience working with containerization is advantageous.
  • Experience with data quality and testing tools, such as Great Expectations, dbt, or DataGrip, is advantageous.
  • Experience working with Big Data Cloud-based technologies, such as AWS, Azure, etc., is advantageous.
  • Experience with data warehouse and data lake technologies, such as BigQuery, Redshift, or Snowflake, is advantageous.
  • Strong experience designing end-to-end data pipelines, including compute hardware infrastructure.
  • Strong knowledge of data modeling, architecture, and governance principles.
  • Strong Linux Administration skills.
  • Programming skills in various languages are advantageous.
  • Strong data security and compliance experience.
  • Excellent communication, collaboration, and problem-solving skills.
  • Ability to work independently and as part of a cross-functional team.
  • Interest and enthusiasm for medical scientific research and its applications.

  • Data Engineering Lead

    4 weeks ago

    Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time

    Job Title: Head of Data EngineeringWe are seeking a highly skilled and experienced Head of Data Engineering to join our team at Carlysle Human Capital. As a key member of our IT department, you will be responsible for designing, developing, and maintaining scalable data pipelines and related systems in a research-focused environment.Key...

  • Senior Data Engineer

    4 weeks ago

    Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time

    Job Title: Head of Data EngineeringWe are seeking a highly skilled and experienced Head of Data Engineering to join our team at Carlysle Human Capital. As a key member of our IT department, you will be responsible for designing, developing, and maintaining scalable data pipelines and related systems in a research-focused environment.Key...

  • MS Engineer

    2 weeks ago

    Durban, KwaZulu-Natal, South Africa NTT DATA Full time

    About the RoleWe are seeking a highly skilled MS Engineer to join our team at NTT DATA. As a key member of our team, you will be responsible for providing a managed service to clients to ensure that their IT infrastructure and systems remain operational.Key ResponsibilitiesMonitor client infrastructure and solutions to identify potential issues and...

  • Data Engineering Lead

    4 weeks ago

    Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time

    Job SummaryWe are seeking a highly skilled Data Engineering professional to join our team at Carlysle Human Capital. As a key member of our IT department, you will be responsible for designing, developing, and maintaining scalable data pipelines and related systems in a research-focused environment.Key ResponsibilitiesDesign and develop end-to-end technical...

  • Specialisation Head

    1 month ago

    Durban, KwaZulu-Natal, South Africa Hire Resolve Full time

    Job SummaryHire Resolve is seeking a highly skilled and experienced Specialisation Head in Engineering to join their team. The successful candidate will be responsible for overseeing a team of engineers in their specific area of expertise and ensuring the successful completion of projects in accordance with company standards and objectives.Key...

  • Clinical Unit Head

    1 month ago

    Durban, KwaZulu-Natal, South Africa KZN Department of Health Full time

    Job Title: Head Clinical UnitWe are seeking a highly skilled and experienced medical professional to lead our Internal Medicine unit as a Head Clinical Unit. The successful candidate will be responsible for coordinating services, developing tertiary services, and providing expert advice to aid diagnosis, management, and treatment of patients.Key...

  • Server Engineer

    1 month ago

    Durban, KwaZulu-Natal, South Africa NTT DATA Full time

    About the RoleThe Managed Services Cross Technology Engineer (L2) is a key member of our team, responsible for providing a managed service to clients to ensure that their IT infrastructure and systems remain operational. This role focuses on second-line support for incidents and requests with a medium level of complexity, across two or more technology...

  • Head Clinical Unit Leader

    1 month ago

    Durban, KwaZulu-Natal, South Africa KZN Department of Health Full time

    Job Title: Head Clinical UnitWe are seeking a highly skilled and experienced Specialist Anaesthesiologist to lead our Anaesthesia Department as a Head Clinical Unit. The successful candidate will be responsible for managing the department's clinical services, providing consultant cover in theatre, and participating in pre-operative assessment and...

  • Senior Data Engineer I

    1 week ago

    Durban, KwaZulu-Natal, South Africa LexisNexis (Proprietary) Limited Company Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team at LexisNexis (Proprietary) Limited Company. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining analytical solutions using Amazon Redshift and Tableau.Key ResponsibilitiesDesign and develop scalable data models and analytical...

  • Data Management Director

    4 days ago

    Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time

    Job Title: Head of Data ManagementCarlysle Human Capital is seeking a highly skilled and experienced Head of Data Management to join our team. As a key member of our organization, you will be responsible for providing strategic leadership and direction to our data management department.Key Responsibilities:Develop and implement data management strategies to...

  • Data Management Director

    3 weeks ago

    Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time

    Job Title: Head of Data ManagementCAPRISA is seeking a highly skilled and experienced Head of Data Management to lead our data management team. As a key member of our organization, you will be responsible for ensuring the efficient and effective management of our clinical data.Key Responsibilities:Provide strategic direction and technical expertise to the...

  • Head of Business Intelligence

    3 weeks ago

    Durban, KwaZulu-Natal, South Africa Hollywoodbets Full time

    Head of Business IntelligenceHollywoodbets is seeking a highly skilled and experienced Head of Business Intelligence to join our Purple Team. As a key member of our leadership team, you will be responsible for driving the development and implementation of our business intelligence strategy, ensuring that we make informed decisions that drive business growth...

  • Head of Department

    3 days ago

    Durban, KwaZulu-Natal, South Africa Meondoholdings Full time

    Job Title: Head of DepartmentA highly skilled and experienced professional is required to lead our department and drive business growth. As the Head of Department, you will be responsible for overseeing the day-to-day operations, managing a team of employees, and ensuring the department meets its targets.Key Responsibilities:Manage and motivate a team of...

  • Data Analyst

    7 days ago

    Durban, KwaZulu-Natal, South Africa Stratogo Full time

    Job Title: Data AnalystStratogo is seeking a highly skilled Data Analyst to join our team. As a Data Analyst, you will play a critical role in analyzing consumer data, market trends, and campaign effectiveness to provide actionable insights that drive our marketing strategy.Key Responsibilities:Analyze consumer data and market trends to identify...

  • Engineering Leadership Position

    4 weeks ago

    Durban, KwaZulu-Natal, South Africa Hire Resolve Full time

    Job Opportunity: Specialisation Head in EngineeringHire Resolve is seeking a highly skilled and experienced Specialisation Head in Engineering to lead a team of engineers in their specific area of expertise.Key Responsibilities:Oversee a team of engineers and ensure the successful completion of projects in accordance with company standards and...

  • Head of Sourcing and Procurement

    4 weeks ago

    Durban, KwaZulu-Natal, South Africa Talented Recruitment Full time

    Job Title: Head of Sourcing and ProcurementWe are seeking a highly skilled and experienced Head of Sourcing and Procurement to join our team at Talented Recruitment. As a key member of our organization, you will be responsible for developing and implementing strategic sourcing and procurement plans that drive business growth and profitability.Key...

  • Software Engineer

    1 week ago

    Durban, KwaZulu-Natal, South Africa Dimension Data Full time

    Job Title: Web App DeveloperAt Dimension Data, we're looking for a skilled Web App Developer to join our team. As a Web App Developer, you'll be responsible for designing, developing, and maintaining web applications that meet the needs of our clients.Key Responsibilities:Design and develop web applications using Java, PHP, C#, JavaScript, CSS, XML, and...

  • Data Management Director

    3 weeks ago

    Durban, KwaZulu-Natal, South Africa Carlysle Human Capital Full time

    Job Title: Head of Data ManagementCAPRISA's mission is to conduct research on HIV Vaccines and Pathogenesis, Prevention, Epidemiology, Tuberculosis, and AIDS treatment, as well as Covid-19 connections.This role involves efficiently managing data, leading the Data Management department, setting up databases, providing technical support to clinical teams, and...

  • Server Infrastructure Specialist

    2 weeks ago

    Durban, KwaZulu-Natal, South Africa NTT DATA Full time

    Job OverviewNTT DATA is seeking a highly skilled Server Engineer to join our team. As a Server Engineer, you will be responsible for providing a managed service to clients to ensure that their IT infrastructure and systems remain operational.Key ResponsibilitiesProactively monitor the work queues and perform operational tasks to resolve all...

  • Chief Client Services Officer

    4 weeks ago

    Durban, KwaZulu-Natal, South Africa Dimension Data Full time

    Head of Client ServicesWe are seeking a seasoned professional to lead our Client Services team as the Head of Client Services. In this critical role, you will be responsible for driving client satisfaction, service delivery, and contract profitability for assigned client contracts.As the Head of Client Services, you will collaborate closely with various...