Data Engineer

6 months ago

Johannesburg, South Africa Planet10 Full time

A leader in the Banking and Financial space is on the lookout for a Data Engineer to join their team. 

The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company’s Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the bank to enable advanced analytics, machine learning and artificial intelligence by providing clean, usable data to stakeholders. They also create data pipelines, Ingestion, provisioning, streaming, self service, API and solutions around big data that support the Bank's strategy to become a data driven organisation.

  • Software Engineers.

Essential Qualifications - NQF Level

  • Advanced Diplomas/National 1st Degrees

Preferred Qualification

  • Field of Study: BCom, BSc, BEng 

Preferred Certifications

  • Cloud (Azure, AWS), DEVOPS or Data engineering certification. Any Data Science certification will be an added advantage, Coursera, Udemy, SAS Data Scientist certification, Microsoft Data Scientist. 

Minimum Experience Level

  • Total number of years of experience:3 - 6 years
  • Type of experience: Experienced at working independently within a squad and has the demonstrated knowledge and skills to deliver data outcomes without supervision.
  • Experience designing, building, and maintaining data warehouses and data lakes.
  • Experience with big data technologies such as Hadoop, Spark, and Hive.
  • Experience with programming languages such as Python, Java, and SQL.
  • Experience with relational databases and NoSQL databases.
  • Experience with cloud computing platforms such as AWS, Azure, and GCP. Experience with data visualization tools.  Result-driven, analytical creative thinker, with demonstrated ability for innovative problem solving.   

Technical / Professional Knowledge

  • Cloud Data Engineering (Azure , AWS, Google)
  • Data Warehousing
  • Databases (PostgreSQL, MS SQL, IBM DB2, HBase, MongoDB)
  • Programming (Python, Java, SQL)
  • Data Analysis and Data Modelling
  • Data Pipelines and ETL tools (Ab Initio, ADB, ADF, SAS ETL)
  • Agile Delivery
  • Problem solving skills

Job Responsibilities

  • Responsible for the maintenance, improvement, cleaning, and manipulation of data in the bank's operational and analytics databases.
  • Data Infrastructure: Build and manage scalable, optimised, supported, tested, secure, and reliable data infrastructure eg using Infrastructure and Databases (DB2, PostgreSQL, MSSQL, HBase, NoSQL, etc), Data Lakes Storage (Azure Data Lake Gen 2), Cloud-based solutions (SAS , Azure Databricks, Azure Data Factory, HDInsight), Data Platforms (SAS, Ab Initio, Denodo, Netezza, Azure Cloud). Ensure data security and privacy in collaboration with Information Security, CISO and Data Governance
  • Data Pipeline Build (Ingestion, Provisioning, Streaming and API): Build and maintain data pipelines to:
  •  create data pipelines for data integration (Data Ingestion, Data Provisioning and Data Streaming) utilising both On Premise tool sets and Cloud Data Engineering tool sets
  • efficiently extract data (Data Acquisition) from Golden Sources, Trusted sources and Writebacks with data integration from multiple sources, formats and structures
  • load the Company’s Data Warehouse (Data Reservoir, Atomic Data Warehouse, Enterprise Data Mart) 
  • provide data to the respective Lines of Business Marts, Regulatory Marts and Compliance Marts through self service data virtualisation
  • provide data to applications or Company’s Data consumers
  • transform data to a common data model for reporting and data analysis, and to provide data in a consistent, useable format to Company’s data stakeholders
  • handle big data technologies (Hadoop), streaming (KAFKA) and data Replication (IBM Inphosphere Data Replication)
  • drive utilisation of data integration tools ( Ab Initio) and Cloud data integration tools (Azure Data Factory and Azure Data Bricks)
  • Data Modelling and Schema Build: In collaboration with Data Modellers, create data models and database schemas on the Data Reservoir, Data Lake, Atomic Data Warehouse and Enterprise Data Marts.
  • Company’s Data Warehouse Automation: Automate, monitor and improve the performance of data pipelines.
  • Collaboration: Collaborate with Data Analysts, Software Engineers, Data Modelers, Data Scientistsm Scrum Masers and Data Warehouse teams as part of a squad to contribute to the data architecture detail designs and take ownership of Epics end-to-end and ensure that data solutions deliver business value.
  • Data Quality and Data Governance: Ensure that reasonable data quality checks are implemented in the data pipelines to maintain a high level of data accuracy, consistency and security.
  • Performance and Optimisation: Ensure the performance of the Company’s data warehouse, integration patterns, batch and real time jobs, streaming and API's.
  • API Development: Build API's that enable the Data Driven Organisation, ensuring that the data warehouse is optimised for API's by collaborating with Software Engineers.

  • Cloud Data Engineer

    6 months ago

    Johannesburg, South Africa Dimension Data Full time

    The Cloud Data Engineer, AWS Data Pipeline, AWS Storage Gateway and Digital Assets will work on the Amazon Web Services (AWS) infrastructure team. The engineer's primary focus is to provide data services that enhance our clients' businesses. You'll be a key member of the data pipeline team helping us deliver these solutions quickly and efficiently while...

  • Cloud Data Engineer: Designing Scalable Data Pipelines

    1 month ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    At Dimension Data, we are seeking an experienced Cloud Data Architect to lead our data engineering solutions. The ideal candidate will have a deep understanding of AWS technologies and experience in designing scalable data pipelines.Key ResponsibilitiesDesign and implement data pipelines using AWS services such as AWS Data Pipeline and AWS Storage...

  • Lead Data Network Engineer for Complex Enterprise Solutions

    3 weeks ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    About the Role:The Lead Data Network Engineer will be responsible for designing and implementing complex data networks across multiple locations. This individual will work closely with business teams to provide solutions to their network challenges, collaborating with partners on vendor projects or in-house engineering efforts.Key Responsibilities:Analyze...

  • Data Architect

    1 month ago

    Johannesburg, Gauteng, South Africa Data Centrix Full time

    Data Discipline ExpertiseData Centrix seeks a seasoned Scrum Master - Data Discipline professional to support our ever-evolving strategy in person-centred health and care. This role demands substantial expertise in software development, programming, and data analysis, as well as knowledge of data structures, algorithms, and distributed computing.Key...

  • Data Solutions Architect

    3 weeks ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    About the Role:We are seeking a skilled Data Solutions Architect to join our team at Dimension Data. As a key member of our technology group, you will play a critical role in designing and implementing data architectures that meet the needs of our clients.The ideal candidate will have a strong background in database systems engineering and experience with...

  • Data Scientist

    2 months ago

    Johannesburg, South Africa NTT DATA Full time

    **Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT DATA** The...

  • Data Engineer

    2 months ago

    Johannesburg, Gauteng, South Africa JCM Consultants Full time

    At JCM Consultants, we're on the hunt for a talented Data Engineer to join our dynamic team!As a Data Engineer, you'll be responsible for developing scripts to process structured and unstructured data, recommending ways to improve data reliability, efficiency, and quality, and supporting the translation of data business needs into technical system...

  • Senior Business Data Analyst

    4 weeks ago

    Johannesburg, South Africa Axis Data Full time

    Axis Data is a US-based advanced data engineering and analytics consulting firm, and a certified partner of Palantir. We help clients implement and support the most advanced data operations platform: Palantir Foundry Role Overview: As a Senior Business Data Analyst at Axis Data, you will lead the development and deployment of BI and data solutions,...

  • Data Insights Strategist

    2 weeks ago

    Johannesburg, Gauteng, South Africa NTT DATA Full time

    About the Role:">This is a seasoned role for a Data Insights Strategist to transform data into critical information and knowledge that can be used to make sound business decisions. The ideal candidate will have expertise in generating visual analytics, dashboards, and ad hoc reports for the business.">Responsibilities:">">Analyzes complex business problems...

  • Data Engineer

    2 weeks ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    A reputable company in the Financial Services sector is seeking a Data Engineer to work with cloud data systems. Proficiency in the Microsoft BI Stack, T-SQL development, and cloud services is essential. Join a dynamic team and enhance your expertise while working on cutting-edge systems and data solutions. As a Data Engineer, you will play a pivotal role in...

  • Data Engineer

    3 weeks ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you will: - Design, develop, and implement robust data pipelines and ETL processes to meet client needs, focusing on data cleansing, aggregation, and enrichment. - Collaborate with clients to understand their requirements and translate them into scalable data solutions. - Aggregate, store, and manage data from various sources, ensuring...

  • Data Engineer

    3 weeks ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you will: - Design, develop, and implement robust data pipelines and ETL processes to meet client needs, focusing on data cleansing, aggregation, and enrichment. - Collaborate with clients to understand their requirements and translate them into scalable data solutions. - Aggregate, store, and manage data from various sources, ensuring...

  • Data Scientist

    1 month ago

    Johannesburg, South Africa NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Data Engineer

    2 weeks ago

    Johannesburg, South Africa Network Recruitment Full time

    A reputable company in the Financial Services sector is seeking a Data Engineer to work with cloud data systems. Proficiency in the Microsoft BI Stack, T-SQL development, and cloud services is essential. Join a dynamic team and enhance your expertise while working on cutting-edge systems and data solutions. As a Data Engineer, you will play a pivotal role in...

  • Data Engineer

    2 weeks ago

    Johannesburg, South Africa Network Recruitment Full time

    As a Data Engineer, you will:Design, develop, and implement robust data pipelines and ETL processes to meet client needs, focusing on data cleansing, aggregation, and enrichment.Collaborate with clients to understand their requirements and translate them into scalable data solutions.Aggregate, store, and manage data from various sources, ensuring...

  • Data Engineer

    2 months ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    Overview Hire Resolve is seeking a talented Data Engineer on behalf of our client, a leading provider of Customer Relationship Management (CRM) software solutions. This is an exciting opportunity to join a dynamic team that specializes in developing innovative CRM systems, designed to help businesses manage and enhance customer relationships. As a Data...

  • Data Engineer

    1 month ago

    Johannesburg, Gauteng, South Africa PBT Group Full time

    We are currently seeking a highly skilled Data Engineer to join our team at PBT Group.About the Role:As a Data Engineer, you will be responsible for designing, developing, testing, and deploying ETL solutions for our ODS and data mart projects. You will also provide technical support and troubleshooting for ETL and database-related issues.Key...

  • Data Engineer

    3 weeks ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    My client is a leading Fintech Software provider, serving a vast variety of industries, specialising in payment solutions. This Data Engineer role offers a unique chance to develop, enhance, troubleshoot and maintain services on both on-premises (Microsoft SQL Server) and cloud data systems. It involves: - Responding to issues/tickets related to the...

  • Data Engineer

    4 weeks ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    Job & Company Description: My client is a leading Fin-Tech Software provider, serving a vast variety of industries, specialising in payment solutions. This Data Engineer role offers a unique chance to develop, enhance, troubleshoot and maintain services on both on-premises (Microsoft SQL Server) and cloud data systems. It involves: - Responding to...

  • Data Engineer

    2 months ago

    Johannesburg, Gauteng, South Africa Puma Energy Full time

    Main Purpose:Puma Energy seeks a highly skilled Data Engineer to collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake.The ideal candidate will ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists to develop...