Data Engineer

2 weeks ago

Pretoria, Gauteng, South Africa Luminary Talent Sourcing Full time

About the Role:

We are seeking a highly skilled and experienced Senior Data Engineer / Architect to join our team and lead the design and development of a real-time analytics platform. The successful candidate will be part of a team responsible for architecture, designing, and building the data infrastructure and pipelines to enable real-time data streaming, transformation, and visualization. This is an exciting opportunity to work on cutting-edge technologies and make a significant impact on our analytics capabilities.

Key Responsibilities:

  • Design and implement scalable and fault-tolerant data streaming and ingestion pipelines using Apache Kafka.
  • Architect and develop a cloud-based data warehousing solution using technologies using such as Apache Hadoop HDFS, Snowflake. Apache Hive or similar technologies for the storage and analysis of structured and unstructured data.
  • Lead the development of data transformation and modeling processes using DBT, Apache Spark, Apache Beam or similar technologies.
  • Lead design and implementation of Reporting / Visualization efforts using Apache Superset, Apache Zeppelin, Metabase, Looker or similar technologies.
  • Collaborate with front-end developers to embed interactive dashboards and reports using Apache Superset or similar tools into a React front-end.
  • Ensure data security, governance, and compliance requirements are met by implementing access controls and auditing mechanisms using Apache Ranger, or Apache Sentry or similar technologies.
  • Mentor and guide junior team members, provide technical leadership, and contribute to the overall technical strategy and roadmap for the analytics platform.

KINDLY TAKE NOTE THAT THE RECRUITMENT AND SELECTION PROCESS WILL INVOLVE

PSYCHOMETRIC ASSESSMENTS.



Requirements

Required Skills and Experience:

  • Experience in building real-time analytics platforms.
  • 9+ Experience in data engineering and data architecture, with a focus on a cloud-based data warehousing solutions around structured (dimensional models) and unstructured data.
  • Proficiency in Apache Kafka for data streaming and ingestion.
  • Experience with cloud-based data warehousing solutions such as Apache Hadoop HDFS, Snowflake, Amazon Redshift, Amazon S3, or Google Cloud Storage.
  • Proficiency in the programming languages as relevant for the platforms involved, such as Java, SQL, Python, or Scala.
  • Hands-on experience with Apache Spark or Apache Beam for large-scale data processing and transformation.
  • Experience with data visualization and exploration tools such as Apache Superset, Apache Zeppelin, Metabase, Looker or similar platforms.
  • Familiarity with security and governance tools and practices in the Hadoop ecosystem, such as Apache Ranger or Apache Sentry.
  • Excellent communication skills and the ability to work effectively in a collaborative team environment.
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field if preferred but not mandatory.

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Investec Full time

    We are looking for an experienced Data Engineer to join our dynamic team.This role is based in South Africa (Investec Ltd.), and the main Investec office is the 100 Grayston.The Data Engineer will design, develop, and maintain our data systems and infrastructure.This role requires strong technical skills, a deep knowledge of data engineering principles, and...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Mediro Ict Recruitment Full time

    These jobs were popular with other job seekersWe currently have an exciting opportunity for a Data Engineer to join our growing team, our offices are based in Samrand, Centurion, kindly submit all applications to Job DescriptionProfile OutlineA technology focussed individual, data engineers will need to design, configure, develop and deploy data...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Offerzen (Pty) Ltd Full time

    Data Engineer Are you a talented data engineer looking for your Dream Developer Job? OfferZen, the developer jobs marketplace, reverses the job search process. We connect developers with a vast network of top tech companies in Africa and Europe that ...Requirements: 2+ years of relevant experience Hands-on, practical experience dealing with large amounts of...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Network Recruitment Full time

    Qualifications: Relevant IT Degree / Diploma / Certification Requirements:Database design skills with an understanding of data warehousing techniques and approachesUnderstanding of and ability to access data from sources such as: Microsoft SQL Server, Oracle, MongoDB, Amazon S3, Window / Lunix file system, HTTP API's, SFTP sitesCreate reports and BI...

  • Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Nestlé Sa Full time

    Press Tab to Move to Skip to Content Link Select how often (in days) to receive an alert: Select how often (in days) to receive an alert: Location:BryanstonQualification:Bachelor's degree in computer science, system analysis or a related study, or equivalent experienceExperience:Minimum of 3-4 years of experience in Data engineer skills also include...

  • Junior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Progressive It Resourcing Full time

    Job Purpose The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the...

  • Senior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Hr On Site Full time

    These jobs were popular with other job seekersThe role operates within a cross-functional, agile development team working across various technologies. The key responsibilities of this role will focus on creating and maintaining data solutions. This will include development activities such as the implementation of ETL processes, the creation of reports,...

  • Junior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Ioco Full time

    About the Role: iOCO is seeking a motivated and detail-oriented Junior Data Engineer to join our dynamic team in Kempton Park, Johannesburg. This full-time, on-site role offers an exciting opportunity for a junior professional to grow their career in data engineering. The ideal candidate will have experience in SQL scripting, web development, and stored...

  • Intermediate Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Offerzen Full time

    Intermediate Data Engineer Are you a talented intermediate data engineer looking for your Dream Developer Job? OfferZen, the developer jobs marketplace, reverses the job search process. We connect developers with a vast network of top tech companies that are eager to apply to interview you This could be the last job posting you ever have to click on Why...

  • Aws Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Isanqa Full time

    Our client requires the services of a Technology Integrator (Chief Expert) Midrand/Menlyn/Rosslyn/Home Office Rotation Amazing brand with cutting edge technologyExcellent teams in Global team collaborationHigh work-life balance with Flexible hoursAgile working environmentPOSITION:Contract until December 2026EXPERIENCE:12+ Years related working...

  • Aws Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Isanqa Resourcing Full time

    Our client requires the services of a Data Engineer/Scientist (Expert) - Midrand/Menlyn/Rosslyn/Home Office rotation. Amazing brand with cutting-edge technology Excellent teams in Global team collaboration High work-life balance with Flexible hours Agile working environmentPOSITION:Contract until December 2026EXPERIENCE:8+ Years related working...

  • Aws Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa Isanqa Resourcing Full time

    Our client requires the services of a Technology Integrator (Chief Expert) Midrand/Menlyn/Rosslyn/Home Office Rotation Amazing brand with cutting edge technology Excellent teams in Global team collaboration High work-life balance with Flexible hours Agile working environmentPOSITION:Contract until December 2026EXPERIENCE:12+ Years related working...

  • Azure Data Engineer – Johannesburg

    2 weeks ago

    Pretoria, Gauteng, South Africa E-Merge Full time

    Azure Data Engineer Wanted at a Global IT CompanyA leading company with a strong focus on cutting-edge technology and innovative IT solutions, particularly in Development and Operations for SAP, JAVA, Azure, and Cloud, is in search of an Azure Data Engineer. The company's strategic positioning provides a profound understanding of compliance and IT...

  • AWS Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa E-Merge Full time

    Our valued client, a key player in African capital markets and a significant presence in the automotive sector, is renowned for its diverse product range and global reach. They are currently on the lookout for a talented AWS Data Engineer to join their team. This is a great opportunity to tackle complex issues and deliver substantial value across different...

  • AWS Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa E-Merge Full time

    A leader in African capital markets with a comprehensive suite of products, a vast global reach while and an acute positioning as a local bank with the expertise ability to leverage on local presence with regional and international expertise has a vacant position for an AWS Data Engineer to work within a team solving complex problems to deliver real business...

  • Mid-Level Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa JSRH Full time

    We are looking for an experienced Mid Level Data Engineer to join us in driving significance together. Primary Duties and Responsibilities:The Data Engineer will have a knack for data analysis, data manipulation and data modelling. The consultant will be responsible for understanding and driving the overall technical vision and planning of a client's...

  • Aws Data Engineers Required

    2 weeks ago

    Pretoria, Gauteng, South Africa Africonology Full time

    Job Title:AWS Data Engineers Required - Project Based - Contractual-to start asap - JHB - HybridLocation:Gauteng, Johannesburg Metro, MidrandJob Type:Contract, expected to work Full-Time hoursPrimary Industry:Banking and FinanceSecondary Industry:Banking and FinanceJob Duties:Design, develop, and implement AWS cloud solutions for data engineering...

  • Senior Data Engineer

    2 weeks ago

    Pretoria, Gauteng, South Africa E-Merge Full time

    ONE of Africa's largest entertainment businesses require a Senior Data Engineer with advanced technical skills to join their amazing team. This opportunity will allow you to develop high quality data warehouse solutions in an Azure environment using but not limited to Synapse, Data Lake, Azure Data Factory, Wherescape and PowerBI If you want to be part of...

  • AWS Data Engineer

    4 weeks ago

    Pretoria, Gauteng, South Africa SabenzaIT Full time

    Are you a proficient IT professional with a fervor for pioneering and state-of-the-art technology? Our client is in search of a skilled AWS Data Engineer to join their vibrant team located in Midrand/Menlyn. As a prominent figure in the automotive industry worldwide, they are dedicated to shaping the future of mobility through inventive solutions and...

  • AWS Data Engineer

    2 months ago

    Pretoria, Gauteng, South Africa SabenzaIT Full time

    Are you a proficient IT professional with a fervor for pioneering and state-of-the-art technology? Our client is in search of a skilled AWS Data Engineer to join their vibrant team located in Midrand/Menlyn. As a prominent figure in the automotive industry worldwide, they are dedicated to shaping the future of mobility through inventive solutions and...