Data Engineer

2 weeks ago

Johannesburg, Gauteng, South Africa Recru-it Full time
Job Purpose:
The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the company to enable advanced analytics, machine learning and artificial intelligence by providing clean, usable data to stakeholders. They also create data pipelines, Ingestion, provisioning, streaming, self-service, API and solutions around big data that support the company strategy to become a data driven organisation.

Job Responsibilities:
  • Responsible for the maintenance, improvement, cleaning, and manipulation of data in the companies operational and analytics databases.
  • Data Infrastructure: Build and manage scalable, optimised, supported, tested, secure, and reliable data infrastructure e.g. using Infrastructure and Databases (DB2, PostgreSQL, MSSQL, HBase, NoSQL, etc.), Data Lakes Storage (Azure Data Lake Gen 2), Cloud-based solutions (SAS, Azure Databricks, Azure Data Factory, HDInsight), Data Platforms (SAS, Ab Initio, Denodo, Netezza, Azure Cloud). Ensure data security and privacy in collaboration with Information Security, CISO and Data Governance
  • Data Pipeline Build (Ingestion, Provisioning, Streaming and API): Build and maintain data pipelines to:
  • create data pipelines for data integration (Data Ingestion, Data Provisioning and Data Streaming) utilising both On Premise tool sets and Cloud Data Engineering tool sets
  • efficiently extract data (Data Acquisition) from Golden Sources, Trusted sources and Writebacks with data integration from multiple sources, formats and structures
  • load the companies Data Warehouse (Data Reservoir, Atomic Data Warehouse, Enterprise Data Mart)
  • provide data to the respective Lines of Business Marts, Regulatory Marts and Compliance Marts through self-service data virtualisation
  • provide data to applications or companies Data consumers
  • transform data to a common data model for reporting and data analysis, and to provide data in a consistent, useable format to companies data stakeholders
  • handle big data technologies (Hadoop), streaming (KAFKA) and data Replication (IBM Inphosphere Data Replication)
  • drive utilisation of data integration tools (Ab Initio) and Cloud data integration tools (Azure Data Factory and Azure Data Bricks)
  • Data Modelling and Schema Build: In collaboration with Data Modellers, create data models and database schemas on the Data Reservoir, Data Lake, Atomic Data Warehouse and Enterprise Data Marts.
  • The Companies Data Warehouse Automation: Automate, monitor and improve the performance of data pipelines.
  • Collaboration: Collaborate with Data Analysts, Software Engineers, Data Modelers, Data Scientistsm Scrum Masers and Data Warehouse teams as part of a squad to contribute to the data architecture detail designs and take ownership of Epics end-to-end and ensure that data solutions deliver business value.
  • Data Quality and Data Governance: Ensure that reasonable data quality checks are implemented in the data pipelines to maintain a high level of data accuracy, consistency and security.
  • Performance and Optimisation: Ensure the performance of the companies data warehouse, integration patterns, batch and real time jobs, streaming and API's.
  • API Development: Build API's that enable the Data Driven Organisation, ensuring that the data warehouse is optimised for API's by collaborating with Software Engineers.

Essential Qualifications - NQF Level:
  • Advanced Diplomas/National 1st Degrees

Preferred Qualification:
  • Field of Study: BCom, BSc, BEng

Preferred Certifications:
  • Cloud (Azure, AWS), DEVOPS or Data engineering certification. Any Data Science certification will be an added advantage, Coursera, Udemy, SAS Data Scientist certification, Microsoft Data Scientist.

Minimum Experience Level:
  • Total number of years of experience:3 - 6 years
  • Type of experience: Experienced at working independently within a squad and has the demonstrated knowledge and skills to deliver data outcomes without supervision.
  • Experience designing, building, and maintaining data warehouses and data lakes.
  • Experience with big data technologies such as Hadoop, Spark, and Hive.
  • Experience with programming languages such as Python, Java, and SQL.
  • Experience with relational databases and NoSQL databases.
  • Experience with cloud computing platforms such as AWS, Azure, and GCP. Experience with data visualization tools. Result-driven, analytical creative thinker, with demonstrated ability for innovative problem solving.

Technical / Professional Knowledge
  • Cloud Data Engineering (Azure, AWS, Google)
  • Data Warehousing
  • Databases (PostgreSQL, MS SQL, IBM DB2, HBase, MongoDB)
  • Programming (Python, Java, SQL)
  • Data Analysis and Data Modelling
  • Data Pipelines and ETL tools (Ab Initio, ADB, ADF, SAS ETL)
  • Agile Delivery
  • Problem solving skills

Behavioral Competencies
  • Decision Making
  • Influencing
  • Communication
  • Innovation
  • Technical/Professional Knowledge and Skills
  • Building Partnerships
  • Continuous Learning

  • Cloud Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    The Cloud Data Engineer, AWS Data Pipeline, AWS Storage Gateway and Digital Assets will work on the Amazon Web Services (AWS) infrastructure team. The engineer's primary focus is to provide data services that enhance our clients' businesses. You'll be a key member of the data pipeline team helping us deliver these solutions quickly and efficiently while...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Energy At Work Full time

    Job Title: Data Engineer (SAS)Location: RemoteAbout Our Client: Our client is a leading player in the insurance industry, dedicated to providing innovative solutions to meet the needs of our clients. They leverage cutting-edge technology to drive their business forward and are seeking a skilled Data Engineer to join their dynamic team. If you are passionate...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Planet10 Full time

    A leader in the Banking and Financial space is on the lookout for a Data Engineer to join their team. The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company's Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Iolap Full time

    Do you like building data systems and pipelines?Do you enjoy interpreting trends and patterns?Are you able to recognize the deeper meaning of data?Join iOLAP, an Elixirr Company, as a Data Engineer and help us analyze and organize raw data?to provide valuable business insights to our clients and stakeholdersAs Data Engineer, you will be responsible to ensure...

  • Data Engineer

    1 month ago

    Johannesburg, Gauteng, South Africa NTT Full time

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our clients solve some of...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa NTT Full time

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our clients solve some of...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa NTT Full time

    JOB DESCRIPTION NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Data Engineer

    4 weeks ago

    Johannesburg, Gauteng, South Africa NTT Full time

    NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our clients solve some of...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Iolap Full time

    Do you like building data systems and pipelines? Do you enjoy interpreting trends and patterns? Are you able to recognize the deeper meaning of data?Join iOLAP, an Elixirr Company, as a Data Engineer and help us analyze and organize raw datato provide valuable business insights to our clients and stakeholdersAs Data Engineer, you will be responsible to...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Nedbank Full time

    Requisition Details & Talent Acquisition Consultant REQ Thembi MtshaliJob FamilyRisk, Audit and ComplianceCareer StreamAuditingLeadership PipelineManage Self: ProfessionalJob PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Nedbank Data Architecture Roadmap, to advance technical...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa iOLAP Full time

    Do you like building data systems and pipelines? Do you enjoy interpreting trends and patterns? Are you able to recognize the deeper meaning of data? Join iOLAP, an Elixirr Company, as a Data Engineer and help us analyze and organize raw data to provide valuable business insights to our clients and stakeholdersAs Data Engineer, you will be responsible to...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Nedbank Full time

    Requisition Details & Talent Acquisition ConsultantREQ Keabetswe ModiseJob FamilyInformation TechnologyCareer StreamData Leadership PipelineManager of Self ProfessionalJob PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Nedbank Data Architecture Roadmap, to advance technical thought...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Puma Energy Full time

    Main Purpose: Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake. Ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists to develop insightful analytics and business stakeholders to make well-informed...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Mimecast Limited Full time

    Data Engineer (SQL experience essential) page is loaded Data Engineer (SQL experience essential) Apply locations South Africa - Johannesburg time type Full time posted on Posted 3 Days Ago job requisition id R4508 Analytics Engineer We're your dream business intelligence team The Business Intelligence & Analytics team at Mimecast is an exciting blend of...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Welcome to our Data Engineering team at XYZ Company!Qualifications: Bachelor's degree in Computer Science, Information Technology, or a related field (Master's degree preferred)Requirements: 5+ years of solid experience in data engineeringStrong background in data warehousing and data lake architectureProficiency in Data Vault and dimensional modeling...

  • Data Center Technician

    2 weeks ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    Requirements: The Data Center Technician (DCT) responsibility is the maintenance of all Data Centre White Space, ensuring a 24/7 service offering within Internet Solutions Data Centers.The DCT provides a service to clients to ensure that their IT infrastructure and systems remain operational through proactively identifying, investigating and resolving...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Samaha Consulting Full time

    The Data Analytics: Data Engineer role is responsible for understanding the full lifecycle of a BI Project from the user, the initiation of the request to delivery of the reporting solution to business. Continuous development of Markets Data Analytics Platform is done by enhancing current functionality and building out for new business requirements.Duties...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Ad Talent Africa Full time

    Who are we: An international group of companies within the hospitality industryWho are we looking for: A data Engineer with a high degree of proficiency in SSIS, SQL, SSAS Cubes, DAX, MDX and building stored proceduresWhat will you do:The Data Engineer will be responsible for designing data models, maintaining, and optimizing data infrastructure for data...

  • Senior Software Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of being micro-managed? Are you looking for a company where no-one tells you how to do your job? Do you want to work in a team, but not be told what to do and when to do it? If so, we believe this Senior Software Engineer role is perfect for youAs a Senior Software Engineer, you'll be responsible for building and maintaining...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa PRR Recruitment Services Full time

    This well known company is looking for a Data Engineer. will be based in Rivonia The ideal candidate will have strong experience in the following: Data cleaning SQL ExportBig data Sets BSc Degree in Maths and Stats will be required a min of 3 years experience. Stable track record is required. Send your Cv to