Data Engineer

4 weeks ago

Pretoria, South Africa Luminary Talent Sourcing Full time

About the Role:

 

We are seeking a highly skilled and experienced Senior Data Engineer / Architect to join our team and lead the design and development of a real-time analytics platform. The successful candidate will be part of a team responsible for architecture, designing, and building the data infrastructure and pipelines to enable real-time data streaming, transformation, and visualization. This is an exciting opportunity to work on cutting-edge technologies and make a significant impact on our analytics capabilities.

 

Key Responsibilities:

  • Design and implement scalable and fault-tolerant data streaming and ingestion pipelines using Apache Kafka.
  • Architect and develop a cloud-based data warehousing solution using technologies using such as Apache Hadoop HDFS, Snowflake. Apache Hive or similar technologies for the storage and analysis of structured and unstructured data.
  • Lead the development of data transformation and modeling processes using  DBT, Apache Spark, Apache Beam or similar technologies.
  • Lead design and implementation of Reporting / Visualization efforts using Apache Superset, Apache Zeppelin, Metabase, Looker or similar technologies.
  • Collaborate with front-end developers to embed interactive dashboards and reports using Apache Superset or similar tools into a React front-end.
  • Ensure data security, governance, and compliance requirements are met by implementing access controls and auditing mechanisms using Apache Ranger, or Apache Sentry or similar technologies.
  • Mentor and guide junior team members, provide technical leadership, and contribute to the overall technical strategy and roadmap for the analytics platform.

KINDLY TAKE NOTE THAT THE RECRUITMENT AND SELECTION PROCESS WILL INVOLVE 

PSYCHOMETRIC ASSESSMENTS.



Requirements

Required Skills and Experience:

 

  • Experience in building real-time analytics platforms.
  • 9+ Experience in data engineering and data architecture, with a focus on a cloud-based data warehousing solutions around structured (dimensional models) and unstructured data.
  • Proficiency in Apache Kafka for data streaming and ingestion.
  • Experience with cloud-based data warehousing solutions such as Apache Hadoop HDFS, Snowflake, Amazon Redshift, Amazon S3, or Google Cloud Storage.
  • Proficiency in the programming languages as relevant for the platforms involved, such as Java, SQL, Python, or Scala.
  • Hands-on experience with Apache Spark or Apache Beam for large-scale data processing and transformation.
  • Experience with data visualization and exploration tools such as Apache Superset, Apache Zeppelin, Metabase, Looker or similar platforms.
  • Familiarity with security and governance tools and practices in the Hadoop ecosystem, such as Apache Ranger or Apache Sentry.
  • Excellent communication skills and the ability to work effectively in a collaborative team environment.
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field if preferred but not mandatory.


  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Investec Full time

    We are looking for an experienced Data Engineer to join our dynamic team.This role is based in South Africa (Investec Ltd.), and the main Investec office is the 100 Grayston.The Data Engineer will design, develop, and maintain our data systems and infrastructure.This role requires strong technical skills, a deep knowledge of data engineering principles, and...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Mediro Ict Recruitment Full time

    These jobs were popular with other job seekersWe currently have an exciting opportunity for a Data Engineer to join our growing team, our offices are based in Samrand, Centurion, kindly submit all applications to Job DescriptionProfile OutlineA technology focussed individual, data engineers will need to design, configure, develop and deploy data...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Offerzen (Pty) Ltd Full time

    Data Engineer Are you a talented data engineer looking for your Dream Developer Job? OfferZen, the developer jobs marketplace, reverses the job search process. We connect developers with a vast network of top tech companies in Africa and Europe that ...Requirements: 2+ years of relevant experience Hands-on, practical experience dealing with large amounts of...

  • Data Engineer

    4 weeks ago

    Pretoria, South Africa Boardroom Appointments Full time

    REQUIRED MINIMUM EDUCATION/TRAINING:BCom or Bachelors degree in Information Technology / Computer Science / Engineering (IT) or similarData Engineering tools (Azure Data Engineer) certification will be an added advantageITIL or COBIT certification will be an added advantageREQUIRED MINIMUM WORK EXPERIENCE:2 - 4 years of working experience in an IT...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Luminary Talent Sourcing Full time

    About the Role:We are seeking a highly skilled and experienced Senior Data Engineer / Architect to join our team and lead the design and development of a real-time analytics platform. The successful candidate will be part of a team responsible for architecture, designing, and building the data infrastructure and pipelines to enable real-time data streaming,...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Network Recruitment Full time

    Qualifications: Relevant IT Degree / Diploma / Certification Requirements:Database design skills with an understanding of data warehousing techniques and approachesUnderstanding of and ability to access data from sources such as: Microsoft SQL Server, Oracle, MongoDB, Amazon S3, Window / Lunix file system, HTTP API's, SFTP sitesCreate reports and BI...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Nestlé Sa Full time

    Press Tab to Move to Skip to Content Link Select how often (in days) to receive an alert: Select how often (in days) to receive an alert: Location:BryanstonQualification:Bachelor's degree in computer science, system analysis or a related study, or equivalent experienceExperience:Minimum of 3-4 years of experience in Data engineer skills also include...

  • Junior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Progressive It Resourcing Full time

    Job Purpose The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the...

  • AWS Data Engineer

    2 weeks ago

    Pretoria, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • AWS Data Engineer

    2 weeks ago

    Pretoria, South Africa Adzuna ZA B C2 Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • Senior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Hr On Site Full time

    These jobs were popular with other job seekersThe role operates within a cross-functional, agile development team working across various technologies. The key responsibilities of this role will focus on creating and maintaining data solutions. This will include development activities such as the implementation of ETL processes, the creation of reports,...

  • Junior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Ioco Full time

    About the Role: iOCO is seeking a motivated and detail-oriented Junior Data Engineer to join our dynamic team in Kempton Park, Johannesburg. This full-time, on-site role offers an exciting opportunity for a junior professional to grow their career in data engineering. The ideal candidate will have experience in SQL scripting, web development, and stored...

  • Intermediate Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Offerzen Full time

    Intermediate Data Engineer Are you a talented intermediate data engineer looking for your Dream Developer Job? OfferZen, the developer jobs marketplace, reverses the job search process. We connect developers with a vast network of top tech companies that are eager to apply to interview you This could be the last job posting you ever have to click on Why...

  • Intermediate Data Engineer

    4 weeks ago

    Pretoria, South Africa OfferZen Full time

    Intermediate Data Engineer Are you a talented intermediate data engineer looking for your Dream Developer Job? OfferZen, the developer jobs marketplace, reverses the job search process. We connect developers with a vast network of top tech companies that are eager to apply to interview you! This could be the last job posting you ever have to click on! Why...

  • Intermediate Data Engineer

    4 weeks ago

    Pretoria, South Africa OfferZen Full time

    Intermediate Data Engineer Are you a talented intermediate data engineer looking for your Dream Developer Job? OfferZen, the developer jobs marketplace, reverses the job search process. We connect developers with a vast network of top tech companies that are eager to apply to interview you! This could be the last job posting you ever have to click on! Why...

  • Aws Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Isanqa Resourcing Full time

    Our client requires the services of a Data Engineer/Scientist (Expert) - Midrand/Menlyn/Rosslyn/Home Office rotation. Amazing brand with cutting-edge technology Excellent teams in Global team collaboration High work-life balance with Flexible hours Agile working environmentPOSITION:Contract until December 2026EXPERIENCE:8+ Years related working...

  • Aws Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Isanqa Full time

    Our client requires the services of a Technology Integrator (Chief Expert) Midrand/Menlyn/Rosslyn/Home Office Rotation Amazing brand with cutting edge technologyExcellent teams in Global team collaborationHigh work-life balance with Flexible hoursAgile working environmentPOSITION:Contract until December 2026EXPERIENCE:12+ Years related working...

  • Senior Data Engineer

    2 weeks ago

    Pretoria, South Africa OfferZen Full time

    Senior Data Engineer Are you an experienced senior data engineer searching for your ideal Developer Job? OfferZen, the developers jobs marketplace, changes the game in job hunting. We connect developers with a wide network of leading tech companies eager to reach out and schedule interviews with you. This might just be the final job post you'll ever need to...

  • Aws Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Isanqa Resourcing Full time

    Our client requires the services of a Technology Integrator (Chief Expert) Midrand/Menlyn/Rosslyn/Home Office Rotation Amazing brand with cutting edge technology Excellent teams in Global team collaboration High work-life balance with Flexible hours Agile working environmentPOSITION:Contract until December 2026EXPERIENCE:12+ Years related working...

  • Azure Data Engineer – Johannesburg

    2 weeks ago

    Pretoria, Gauteng, South Africa E-Merge Full time

    Azure Data Engineer Wanted at a Global IT CompanyA leading company with a strong focus on cutting-edge technology and innovative IT solutions, particularly in Development and Operations for SAP, JAVA, Azure, and Cloud, is in search of an Azure Data Engineer. The company's strategic positioning provides a profound understanding of compliance and IT...