Data Engineer

2 weeks ago

Sandton, South Africa Progressive IT Resourcing Full time

Job Purpose

The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the bank to enable advanced analytics, machine learning and artificial intelligence by providing clean, usable data to stakeholders. They also create data pipelines, Ingestion, provisioning, streaming, self service, API and solutions around big data that support the Bank's strategy to become a data driven organisation.

Job Responsibilities

  • Responsible for the maintenance, improvement, cleaning, and manipulation of data in the bank's operational and analytics databases.
  • Data Infrastructure: Build and manage scalable, optimised, supported, tested, secure, and reliable data infrastructure eg using Infrastructure and Databases (DB2, PostgreSQL, MSSQL, HBase, NoSQL, etc), Data Lakes Storage (Azure Data Lake Gen 2), Cloud-based solutions (SAS , Azure Databricks, Azure Data Factory, HDInsight), Data Platforms (SAS, Ab Initio, Denodo, Netezza, Azure Cloud). Ensure data security and privacy in collaboration with Information Security, CISO and Data Governance
  • Data Pipeline Build (Ingestion, Provisioning, Streaming and API): Build and maintain data pipelines to:
  • create data pipelines for data integration (Data Ingestion, Data Provisioning and Data Streaming) utilising both On Premise tool sets and Cloud Data Engineering tool sets.
  • efficiently extract data (Data Acquisition) from Golden Sources, Trusted sources and Writebacks with data integration from multiple sources, formats and structures.
  • load the Company Data Warehouse (Data Reservoir, Atomic Data Warehouse, Enterprise Data Mart)
  • provide data to the respective Lines of Business Marts, Regulatory Marts and Compliance Marts through self service data virtualization.
  • provide data to applications or Company Data consumers
  • transform data to a common data model for reporting and data analysis, and to provide data in a consistent, useable format to Company data stakeholders
  • handle big data technologies (Hadoop), streaming (KAFKA) and data Replication (IBM Inphosphere Data Replication)
  • drive utilization of data integration tools ( Ab Initio) and Cloud data integration tools (Azure Data Factory and Azure Data Bricks)
  • Data Modelling and Schema Build: In collaboration with Data Modelers, create data models and database schemas on the Data Reservoir, Data Lake, Atomic Data Warehouse and Enterprise Data Marts.
  • Company Data Warehouse Automation: Automate, monitor and improve the performance of data pipelines.
  • Collaboration: Collaborate with Data Analysts, Software Engineers, Data Modelers, Data Scientists' Scrum Masers and Data Warehouse teams as part of a squad to contribute to the data architecture detail designs and take ownership of Epics end-to-end and ensure that data solutions deliver business value.
  • Data Quality and Data Governance: Ensure that reasonable data quality checks are implemented in the data pipelines to maintain a high level of data accuracy, consistency and security.
  • Performance and Optimisation:
    • Ensure the performance of the Nedbank data warehouse, integration patterns, batch and real time jobs, streaming and API's.
  • API Development:
    • Build API's that enable the Data Driven Organisation, ensuring that the data warehouse is optimised for API's by collaborating with Software Engineers.

Essential Qualifications - NQF Level

  • Advanced Diplomas/National 1st Degrees

Preferred Qualification

  • Field of Study: BCom, BSc, BEng

Preferred Certifications

  • Cloud (Azure, AWS), DEVOPS or Data engineering certification. Any Data Science certification will be an added advantage, Coursera, Udemy, SAS Data Scientist certification, Microsoft Data Scientist.

Minimum Experience Level

  • Total number of years of experience:3 - 6 years
  • Type of experience: Experienced at working independently within a squad and has the demonstrated knowledge and skills to deliver data outcomes without supervision.
  • Experience designing, building, and maintaining data warehouses and data lakes.
  • Experience with big data technologies such as Hadoop, Spark, and Hive.
  • Experience with programming languages such as Python, Java, and SQL.
  • Experience with relational databases and NoSQL databases.
  • Experience with cloud computing platforms such as AWS, Azure, and GCP. Experience with data visualization tools. Result-driven, analytical creative thinker, with demonstrated ability for innovative problem solving.

Technical / Professional Knowledge

  • Cloud Data Engineering (Azure , AWS, Google)
  • Data Warehousing
  • Databases (PostgreSQL, MS SQL, IBM DB2, HBase, MongoDB)
  • Programming (Python, Java, SQL)
  • Data Analysis and Data Modelling
  • Data Pipelines and ETL tools (Ab Initio, ADB, ADF, SAS ETL)
  • Agile Delivery
  • Problem solving skills

  • Data Engineer

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingStandard Bank Anglo is looking for a Data Engineer. This will be strictly remote work. RequirementsAdvanced knowledge of T24 architecture and Data from a MOD perspective Analyze and organize raw data Build data systems and pipelinesEvaluate business needs and objectivesInterpret trends and patternsConduct complex data analysis and report...

  • Data Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingStandard Bank Anglo is looking for a Data Engineer. This will be strictly remote work. Requirements Advanced knowledge of T24 architecture and Data from a MOD perspective Analyze and organize raw data Build data systems and pipelinesEvaluate business needs and objectivesInterpret trends and patternsConduct complex data analysis and report...

  • Senior Data Engineer-

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAbout Dimension DataAt Dimension Data we empowering our people to deliver client-centric solutions. We want to see our people make the most of every opportunity, and together we can achieve the extraordinary. Our diverse and inspiring people are experts at what they do and serious about delivering measurable and sustainable value at every...

  • Intermediate Data Engineer-

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingIntermediate Data Engineer, Data Processing and Archiving, SQL, Big Data Analytics, Big Data This is a great opportunity to work with an industry leading company in the finance sector. The role will be based at our headquarters in London. The ideal candidate will have strong experience of working on large scale data processing projects...

  • Senior Data Engineer-

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAbout Dimension DataAt Dimension Data we empowering our people to deliver client-centric solutions. We want to see our people make the most of every opportunity, and together we can achieve the extraordinary. Our diverse and inspiring people are experts at what they do and serious about delivering measurable and sustainable value at every...

  • Data Centre Operations

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingDo you want greater control over your own career? We believe this Data Centre Operations role was created for you! Please read through and apply!Data Centre Operations (DCO) person: Data Centre - manages the operational day to day tasks onsite and assists the Site Manager/ Senior Operations Manager in operational tasks such as budgeting,...

  • Process Engineer II

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    To deal with and influence the more strategic and tactical aspects of discovering, validating, documenting, and communicating business process-related knowledge through modeling: simulating and analysing current and future states. The focus is on complex business outcomes and technical aspects, in line with the business Strategy. Requirements ...

  • Data Engineer

    2 weeks ago

    Sandton, South Africa Investec Full time

    Description We are looking for an experienced Data Engineer to join our dynamic team. This role is based in South Africa (Investec Ltd.), and the main Investec office is the 100 Grayston. The Data Engineer will design, develop, and maintain our data systems and infrastructure. This role requires strong technical skills, a deep knowledge of data engineering...

  • MS Engineer

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking for a career in one of the best engineering companies? Apply for this MS Engineer (L2) roleMS Engineer (L2) is responsible for providing a managed service to clients to ensure that their IT infrastructure and systems remain operational. Through the proactive monitoring, identifying, investigating and resolving of technical...

  • DevOps Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingWe are looking for talented and enthusiastic DevOps Engineers to join our “Building 20” team at Dimension Data. Building 20 is our innovation program targeting the technology platforms we use to service our clients. We’re looking for self-starters to join this team – not only to optimise and improve our current approaches, but also...

  • Security Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of working long hours, being unproductive and feeling pressured by your manager? Are you looking to work for a company where your opinions matter and you can make an impact? We're here to offer you this job as a Security Engineer!As a Security Engineer, you'll work with the engineering team to develop and implement security...

  • SecOps Engineer

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of working long hours? Is your job unfulfilling and unrewarding? Do you want to work only 8 hours a day, including a maximum of 2 am to 9 am during weekdays? Are you looking for an opportunity in which your work will have real impact on the lives of others around you? Apply for this SecOps Engineer (L3) roleAs a SecOps...

  • Data Engineer

    4 weeks ago

    Sandton, South Africa Progressive IT Resourcing Full time

    Job PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the...

  • Data Engineer

    4 weeks ago

    Sandton, South Africa Progressive IT Resourcing Full time

    Job PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the...

  • Data Engineer

    4 weeks ago

    Sandton, South Africa Progressive IT Resourcing Full time

    Job Purpose The purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the...

  • Data Engineer

    4 weeks ago

    Sandton, South Africa Adzuna ZA B C2 Full time

    Job PurposeThe purpose of the Data Engineer is to leverage their data expertise and data related technologies, in line with the Company Data Architecture Roadmap, to advance technical thought leadership for the Enterprise, deliver fit for purpose data products, and support data initiatives. In addition, Data Engineers enhance the data infrastructure of the...

  • Master Data Migration Consultant

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAs a Master Data Migration Consultant, you'll provide support to the company's engineers on issues relating to data migration. You will work closely with our teams in Engineering and Business Development.You will be responsible for researching technical solutions that would allow us to quickly transfer data from one system or database into...

  • Data Engineer

    1 week ago

    Sandton, South Africa DataTech Recruitment Full time

    Are you a passionate and experienced Data Engineer looking for an exciting opportunity in Johannesburg? We are seeking a highly skilled individual with a BSc Degree in Computer Science, Engineering, Mathematics, Statistics, Information Systems, or a related discipline, along with a minimum of 5 years' experience in building data pipelines including at least...

  • TS Implementation Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of being stuck in one role for years and not getting any promotions? Do you want to make a real difference within your company and be respected by your peers? Are you looking for an opportunity where you can collaborate with other engineers on major projects? We believe this TS Implementation Engineer was created just for...

  • SecOps Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of working long hours? Is your job unfulfilling and unrewarding? Do you want to work only 8 hours a day, including a maximum of 2 am to 9 am during weekdays? Are you looking for an opportunity in which your work will have real impact on the lives of others around you? Apply for this SecOps Engineer (L3) role!As a SecOps...