[High Salary] Data Engineer

2 weeks ago


Johannesburg, South Africa Jobted ZA C2 Full time

As a Data Engineer, you'll be at the heart of transforming complex, unstructured data into actionable insights. You'll be responsible for implementing process improvements, enhancing data systems, and ensuring the integrity and scalability of our data infrastructure. If you're excited to work with large data sets, optimize ETL processes, and ensure data quality, this is the perfect opportunity for you

Responsibilities:

- Process Optimization: Suggest and implement efficiencies to automate manual processes and optimize data delivery for better scalability.
- System Enhancements: Develop and implement new features and enhancements across our data systems, improving the CI/CD process for optimal data pipelines.
- Data Pipeline Development: Assemble and manage complex data sets to meet both functional and non-functional business requirements.
- ETL Processes: Lead the design, development, and optimization of efficient ETL pipelines.
- Testing & Consistency: Develop and execute unit tests to ensure data consistency and reliability across pipelines.
- Monitoring & Reporting: Implement and maintain automated monitoring solutions to support reporting and analytics infrastructure.
- Data Governance & Quality: Ensure high standards of data governance, quality assurance, and the maintenance of data infrastructure systems (AWS, databases, security).
- Metadata & Documentation: Maintain data warehouse and data lake metadata, data catalogs, and user documentation for internal business users.
- Best Practices: Enforce best practices across our database systems (e.g., collations, indices, database engines) to ensure performance and reliability.

Technologies:

- Programming Languages: Python, SQL
- Data Engineering: PySpark, Airflow, Terraform
- Cloud Platforms: AWS (primary), Azure, GCP
- Containerization: Docker
- Version Control & CI/CD: GitHub, Jenkins, etc.

Qualifications and Experience:

- Educational Background: BSc Degree in Computer Science, Information Systems, Engineering, or a related technical field (or equivalent work experience).
- Experience: 5+ years of related work experience, including at least 2 years of experience in building and optimizing 'big data' data pipelines and maintaining large data sets.
- Technical Proficiency:

- Strong experience with Python and SQL (PostgreSQL, MS SQL).

- Hands-on experience with cloud services, particularly AWS (experience with Azure or GCP is a plus).

- Proficiency in ETL processes and data lifecycle management.

- Experience with Glue and PySpark is highly desirable.

- Skilled in version control, CI/CD pipelines, and GitHub.

- Data Management Expertise: Deep understanding of data quality assurance, data governance, and best practices for managing data transformation processes, metadata, and dependencies.
- Analytical Skills: Strong problem-solving abilities and experience working with unstructured datasets.
- Big Data: Familiarity with message queuing, stream processing, and working with highly scalable 'big data' stores.
- Attention to Detail: A meticulous eye for detail and a commitment to maintaining high-quality data systems.
- Communication: Strong communication skills and the ability to work collaboratively in a team.

Apply now



  • Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines, databases, and infrastructure that enable high-quality data processing and analytics. You will work with cross-functional teams, including Data Scientists, Analysts, and IT specialists, to ensure seamless data flow from various sources into our data...


  • Johannesburg, South Africa Jobted ZA C2 Full time

    Become part of a proudly local organisation as a Specialist Data Engineer, with over 100 years of market influence. Our client is looking for a specialist where your work will directly contribute to shaping the future of their financial landscape. As part of a dynamic team, youll play a key role in optimizing data systems that drive business success across...

  • Data Engineer

    2 months ago


    Johannesburg, South Africa Jobted ZA C2 Full time

    Duties and Responsibilities: - Develop and maintain data pipelines - Implement data integrations - Perform data transformations - Collaborate with cross-functional teams on data projects - Ensure data quality and integrity Qualifications: - Bachelor's degree in Computer Science, Statistics, Mathematics, or related field - Minimum 4 years of technical...

  • Cloud Data Engineer

    7 months ago


    Johannesburg, South Africa Dimension Data Full time

    The Cloud Data Engineer, AWS Data Pipeline, AWS Storage Gateway and Digital Assets will work on the Amazon Web Services (AWS) infrastructure team. The engineer's primary focus is to provide data services that enhance our clients' businesses. You'll be a key member of the data pipeline team helping us deliver these solutions quickly and efficiently while...


  • Johannesburg, South Africa Agrigate One Limited Full time

    The purpose of the DaaS team is to sell data as a service which will in turn create new revenue streams for the business. The DaaS team needs to understand all departments and how they operate. They will need to research ideas and produce suitable solutions to optimise client’s data needs through innovation and utilisation of existing tools in Azure. This...


  • Johannesburg, Gauteng, South Africa Dimension Data Full time

    At Dimension Data, we are seeking an experienced Cloud Data Architect to lead our data engineering solutions. The ideal candidate will have a deep understanding of AWS technologies and experience in designing scalable data pipelines.Key ResponsibilitiesDesign and implement data pipelines using AWS services such as AWS Data Pipeline and AWS Storage...

  • Software Engineer

    4 weeks ago


    Johannesburg, South Africa Jobted ZA C2 Full time

    Key requirements:- Database management, automated Process optimization Multi-skilled software engineer with experience in the public sector (advantageous) 5 - 7 years SQL development and programming 4-5 years Agile development 4 -5 years Power BI/data management Qualifications & Experience required:- Information Technology Degree Relevant...


  • Johannesburg, South Africa Jobted ZA C2 Full time

    Key Responsibilities: - Design, develop, and maintain electrical systems and components. - Collaborate with multidisciplinary teams to deliver high-quality engineering solutions. - Conduct technical assessments, feasibility studies, and risk analyses. - Ensure compliance with industry standards, codes, and safety regulations. - Oversee the installation,...


  • Johannesburg, South Africa Standard Bank of South Africa Limited Full time

    Business Segment: Personal & Private Banking Location: ZA, GP, Johannesburg, Baker Street 30 To lead the quality engineering approach for functional and non-functional requirements for complex solutions, employing scalable and reliable automated test tools and frameworks. To enable test coverage by planning and driving testing activities across all stages...


  • Johannesburg, South Africa Elucidate Full time

    Senior Data Scientist Lead - Elucidate AI Elucidate AI is looking for a superstar to join our team as a Senior Data Scientist Lead. We are looking for a machine learning engineer who has an entrepreneurial spirit with a passion for problem solving, technical solutions, ideation, creation, and business process optimization. We seek someone with an aptitude...


  • Johannesburg, South Africa Jobted ZA C2 Full time

    Responsibilities - Interpret data, analyse results using statistical techniques and provide ongoing reports. - Provide quality assurance of imported data, working with quality assurance analysts if necessary. - Commissioning and decommissioning of data sets. - Processing confidential data and information according to guidelines. - Develop and implement...


  • Johannesburg, Gauteng, South Africa Dimension Data Full time

    Senior Software EngineerWe are seeking a highly skilled Senior Software Engineer to join our team. As a key member of our development team, you will be responsible for designing and implementing software systems that meet the needs of our clients.The ideal candidate will have a strong background in software engineering, with experience in developing...


  • Johannesburg, Gauteng, South Africa Network Recruitment Full time

    At Network Recruitment, we are seeking a skilled Data Engineer to join our team. This role is ideal for an individual who enjoys working with data pipelines, databases, and infrastructure.About the RoleThis position involves designing, building, and maintaining high-quality data processing and analytics systems. You will collaborate with cross-functional...

  • Data Engineer

    3 weeks ago


    Johannesburg, South Africa Jobted ZA C2 Full time

    A reputable company in the Financial Services sector is seeking a Data Engineer to work with cloud data systems. Proficiency in the Microsoft BI Stack, T-SQL development, and cloud services is essential. Join a dynamic team and enhance your expertise while working on cutting-edge systems and data solutions. As a Data Engineer, you will play a pivotal role in...

  • Data Engineer

    4 weeks ago


    Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you will: - Design, develop, and implement robust data pipelines and ETL processes to meet client needs, focusing on data cleansing, aggregation, and enrichment. - Collaborate with clients to understand their requirements and translate them into scalable data solutions. - Aggregate, store, and manage data from various sources, ensuring...

  • Data Engineer Lead

    2 weeks ago


    Johannesburg, Gauteng, South Africa Nedbank Full time

    Job Title : Data EngineerA highly skilled Data Engineer is required by Nedbank to lead the design, implementation, and maintenance of data systems. The successful candidate will have a deep understanding of data technologies and be able to leverage their expertise to advance technical thought leadership within the Enterprise.Job DescriptionThe primary...

  • Data Engineer

    3 weeks ago


    Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you will: - Design, develop, and implement robust data pipelines and ETL processes to meet client needs, focusing on data cleansing, aggregation, and enrichment. - Collaborate with clients to understand their requirements and translate them into scalable data solutions. - Aggregate, store, and manage data from various sources, ensuring...


  • Johannesburg, South Africa Jobted ZA C2 Full time

    DUTIES and RESPONSIBILITIES - Builds, runs and supports highly available platform with security first principle aligned with the company standards, - Proactively stays up to date with new Microsoft Azure services and more optimized cost offerings, - Creates and manages platform on the company Azure infrastructure using ARM, Terraform, Ansible, - Implements...

  • Data Engineer

    4 weeks ago


    Johannesburg, South Africa Jobted ZA C2 Full time

    My client is a leading Fintech Software provider, serving a vast variety of industries, specialising in payment solutions. This Data Engineer role offers a unique chance to develop, enhance, troubleshoot and maintain services on both on-premises (Microsoft SQL Server) and cloud data systems. It involves: - Responding to issues/tickets related to the...


  • Johannesburg, Gauteng, South Africa Dimension Data Full time

    About the Role:The Lead Data Network Engineer will be responsible for designing and implementing complex data networks across multiple locations. This individual will work closely with business teams to provide solutions to their network challenges, collaborating with partners on vendor projects or in-house engineering efforts.Key Responsibilities:Analyze...