Big Data Services Lead

3 weeks ago


Sandton, South Africa Absa Bank Limited Full time

Bring your possibility to life Define your career with us
- With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary

Team Context:
Data Engineering is responsible for the central data platform that receives and distributes data across the bank. This is a multi-platform environment and leverages a blend of custom, commercial and open-source tools to manage and support thousands of critical data-related jobs. These jobs are supported and updated in line with changes across the landscape to avoid disruption to downstream data consumers.

**Job Description**:
Role Description

The Big Data Services Lead role is responsible for driving optimization of the end-to-end environment. This includes the Hadoop platform, scheduling, automation, monitoring and the various aspects of supportability and efficiency required to manage a large, complex environment.

**Responsibilities**:

- Manage an assigned team through day to day support tasks
- Oversee enhancements and new developments, providing guidance and peer review
- Build and deploy new data pipelines
- Oversee development plans for the team and provide mentorship to the team
- Identify and drive optimisation opportunities across the environment
- Drive improvements to environment supportability and maintain effective cross team relationships
- Test changes to internal tools and approve releases, with a focus on optimizing test execution
- Drive improvements to environment supportability and maintain effective cross team relationships
- Advise users on best practices and conduct training sessions where required
- Define and continuously improve the team best practice guides and standards
- Translate complex functional and technical requirements into detailed design for the team to build

Job Experience & Skills Required:

- 4 years’ experience working in Big data environment, optimising and building big data pipelines, architectures and data sets with e.g. Java, Scala, Python, Hadoop, Apache Spark and Kafka
- Cross domain knowledge
- Experience with designing and building, BI systems and complex data eco systems
- Minimum one year experience with Scala programming language
- Familiarity with Hadoop ecosystem and its components
- Solid experience in a working environment in Big Data development utilising SQL or Python
- Experience in Big Data development using Spark
- Experience in Hadoop, HDFS and MapReduce
- Experience in database design, development and data modelling

The following additional knowledge, skills and attributes are preferred:

- Good knowledge in back-end programming, specifically Java
- Understanding of Cloud technologies and migration techniques
- Understanding of data streaming and the intersection of batch and real time data
- Experience with development in a Linux environment and its basic commands
- Ability to write reliable, manageable, and high-performance code
- Should have basic knowledge of SQL, database structures, principles, and theories
- Knowledge of workflow/schedulers
- Strong collaboration and communication skills
- Strong analytical and problem solving skills
- Data architecture and security
- Data management

Education

Bachelor's Degree: Information Technology
- Absa Bank Limited reserves the right not to make an appointment to the post as advertised_



  • Sandton, South Africa Absa Bank Full time

    We seek a skilled specialist to manage Financial Data, Big Data technologies, and Data Infrastructure dependencies. As a key member of our team, you will develop and maintain the Coverage and Pricing analysis tool.Key Areas of FocusDevelop and maintain Big Data enabling technologiesSupport team members on Data Infrastructure dependenciesIdentify and mitigate...


  • Sandton, South Africa Yugantic Full time

    ResponsibilitiesAs a Senior Data Engineer at Yugantic, you will be responsible for developing and maintaining an analytical data infrastructure that provides ad-hoc access to extensive datasets and computing power. Your key responsibilities will include:Designing and Implementing Large-Scale Data Solutions: Leverage your expertise in AWS Glue, AWS Step...


  • Sandton, South Africa Absa Bank Full time

    We are an equal opportunity employer seeking a skilled specialist to join our team. As a Data Analyst Specialist, you will be responsible for managing Financial Data, Big Data technologies, and Data Infrastructure dependencies.Main ResponsibilitiesMaintain the integrity of the Coverage and Pricing analysis toolProvide technical support to team members on Big...


  • Sandton, South Africa Mimecast Services Limited Full time

    Overview: **Data Operations Analyst** Interpret trends and stop cybercriminals in their tracks. Are you methodical and detail-oriented? Then join our dynamic Digital Business Innovation team. We use our high-powered tech stack to plan, control and improve data quality throughout the business. Day to day you’ll build reporting capabilities to identify...

  • Data Engineering Lead

    18 hours ago


    Sandton, South Africa Copious Software Full time

    Drive Business Growth with Data InsightsCopious Software is dedicated to helping businesses grow by providing high-value technology services. As a Solution Engineer in the Data vertical, you will play a crucial role in designing, implementing, and managing data solutions on the Microsoft Azure cloud platform.Your responsibilities will include creating and...


  • Sandton, South Africa Innovation Group Full time

    About the JobThe Innovation Group is seeking an experienced Senior Data Engineer to lead our data engineering efforts. As a Senior Data Engineer, you will be responsible for developing and implementing data warehousing blueprints, evaluating hardware and software platforms, and integrating systems.Key ResponsibilitiesDevelop and implement data warehousing...

  • Data Scientist

    12 hours ago


    Sandton, South Africa Discovery Full time

    Discovery HealthIntermediate Data ScientistAbout DiscoveryDiscovery's core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and whose own values align with those of Discovery. Our fast-paced and dynamic environment enables smart,...


  • Sandton, South Africa Discovery Full time

    At Discovery, we strive to make a meaningful impact on people's lives by harnessing the power of data science.We are seeking an exceptional Senior Analytics Expert to join our Health Intelligence – Operations Intelligence unit. As a key member of our team, you will leverage predictive analytics, big data, and machine learning techniques to drive business...


  • Sandton, South Africa DeARX Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team in Sandton. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing data pipelines using Ab Initio.Main Responsibilities:Design and develop data pipelines using Ab Initio.Develop efficient ETL processes.Work closely with data...


  • Sandton, South Africa Brambles Group Full time

    CHEP helps move more goods to more people, in more places than any other organization on earth via our 300 million pallets, crates and containers. We employ 11,000 people and operate in more than 55 countries. Through our pioneering and sustainable share-and-reuse business model, the world’s biggest brands trust us to help them transport their goods more...


  • Sandton, South Africa Absa Bank Full time

    Empowering Africa's future requires a skilled specialist to manage the clusters requirements for Financial Data, Big Data technologies, and Data Infrastructure dependencies.Job SummaryMaintain the Coverage and Pricing analysis tool (data sharing, accessibility, enhancements)Manage Big Data enabling technologies and provide support to team membersIdentify and...


  • Sandton, South Africa Absa Bank Full time

    A career with Absa Bank offers the opportunity to be part of its exciting growth journey. We are seeking a skilled specialist to manage Financial Data, Big Data technologies, and Data Infrastructure dependencies.Key ResponsibilitiesDevelop and maintain the Coverage and Pricing analysis toolSupport team members on Big Data enabling technologiesIdentify and...

  • Data Engineer

    7 days ago


    Sandton, South Africa Brambles Full time

    CHEP helps move more goods to more people, in more places than any other organization on earth via our 347 million pallets, crates and containers. We employ approximately 13,000 people and operate in 60 countries. Through our pioneering and sustainable share-and-reuse business model, the world's biggest brands trust us to help them transport their goods more...


  • Sandton, South Africa Absa Bank Limited Full time

    Bring your possibility to life! Define your career with us - With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary **Job...


  • Sandton, South Africa DeARX Full time

    About DeARXWe are a leading bank in Sandton, and we're looking for a talented Data Engineer to join our team.You will be responsible for designing, developing, and optimizing data pipelines using Ab Initio. You will also work closely with data scientists, analysts, and stakeholders to address data needs and ensure seamless integration.Key...


  • Sandton, South Africa Guardrisk Group Full time

    Guardrisk Group is a leading insurance company seeking an experienced Lead Data Scientist to join our team.As Lead Data Scientist for Insurance, you will be responsible for developing innovative solutions to automate business processes and enhancing operational efficiency.The ideal candidate will have a strong background in data analysis, excellent...

  • Data Team Lead

    2 days ago


    Sandton, South Africa Experian Health Full time

    Experian Health is a leading provider of data and analytics solutions.Job OverviewThis position plays a critical role in managing the company's data operations. The successful candidate will oversee all aspects of data flows, ensuring that high-quality data is delivered efficiently and effectively.Data ManagementTeam LeadershipProblem SolvingTechnical...

  • Data Science Manager

    4 weeks ago


    Sandton, South Africa Team Finder Full time

    **Role Overview**: Our client is entering a new phase where operational and commercial excellence are key to success. As part of their strategic vision, they are seeking a **Data Science Manager**to lead a team of data scientists and analysts in driving data-driven decision-making. This role involves developing machine learning models, managing data...


  • Sandton, South Africa Brambles Full time

    Job DescriptionData Engineering Role OverviewThis position is based in South Africa.Key Responsibilities:Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional/non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing...


  • Sandton, South Africa Innovation Group Full time

    Responsible for developing data warehousing blueprints, evaluating hardware and software platforms and integrating systems. Manages the data environment which includes data design, database architecture, metadata and repository creation. Translates business needs into long-term architecture solutions. Defines, designs and builds dimensional databases....