Big Data Services Lead

2 weeks ago

Sandton, South Africa Absa Bank Limited Full time

Bring your possibility to life Define your career with us
- With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary

Team Context:
Data Engineering is responsible for the central data platform that receives and distributes data across the bank. This is a multi-platform environment and leverages a blend of custom, commercial and open-source tools to manage and support thousands of critical data-related jobs. These jobs are supported and updated in line with changes across the landscape to avoid disruption to downstream data consumers.

**Job Description**:
Role Description

The Big Data Services Lead role is responsible for driving optimization of the end-to-end environment. This includes the Hadoop platform, scheduling, automation, monitoring and the various aspects of supportability and efficiency required to manage a large, complex environment.

**Responsibilities**:

- Manage an assigned team through day to day support tasks
- Oversee enhancements and new developments, providing guidance and peer review
- Build and deploy new data pipelines
- Oversee development plans for the team and provide mentorship to the team
- Identify and drive optimisation opportunities across the environment
- Drive improvements to environment supportability and maintain effective cross team relationships
- Test changes to internal tools and approve releases, with a focus on optimizing test execution
- Drive improvements to environment supportability and maintain effective cross team relationships
- Advise users on best practices and conduct training sessions where required
- Define and continuously improve the team best practice guides and standards
- Translate complex functional and technical requirements into detailed design for the team to build

Job Experience & Skills Required:

- 4 years’ experience working in Big data environment, optimising and building big data pipelines, architectures and data sets with e.g. Java, Scala, Python, Hadoop, Apache Spark and Kafka
- Cross domain knowledge
- Experience with designing and building, BI systems and complex data eco systems
- Minimum one year experience with Scala programming language
- Familiarity with Hadoop ecosystem and its components
- Solid experience in a working environment in Big Data development utilising SQL or Python
- Experience in Big Data development using Spark
- Experience in Hadoop, HDFS and MapReduce
- Experience in database design, development and data modelling

The following additional knowledge, skills and attributes are preferred:

- Good knowledge in back-end programming, specifically Java
- Understanding of Cloud technologies and migration techniques
- Understanding of data streaming and the intersection of batch and real time data
- Experience with development in a Linux environment and its basic commands
- Ability to write reliable, manageable, and high-performance code
- Should have basic knowledge of SQL, database structures, principles, and theories
- Knowledge of workflow/schedulers
- Strong collaboration and communication skills
- Strong analytical and problem solving skills
- Data architecture and security
- Data management

Education

Bachelor's Degree: Information Technology
- Absa Bank Limited reserves the right not to make an appointment to the post as advertised_

  • Big Data Administrator

    7 days ago

    Sandton, South Africa Discovery Ltd. Full time

    **Business Unit**:Discovery Health**Function**:Information Technology**Date**:4 Nov 2024Discovery - Health Systems (DHS) Big Data Administrator - About Discovery Discovery’s core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and...

  • Data Operations Analyst

    2 weeks ago

    Sandton, South Africa Mimecast Services Limited Full time

    Overview: **Data Operations Analyst** Interpret trends and stop cybercriminals in their tracks. Are you methodical and detail-oriented? Then join our dynamic Digital Business Innovation team. We use our high-powered tech stack to plan, control and improve data quality throughout the business. Day to day you’ll build reporting capabilities to identify...

  • Senior Data Analyst: Data Profiling

    1 week ago

    Sandton, South Africa Dimension Data Full time

    A tech services company in Sandton, South Africa, is seeking a Data Analyst responsible for gathering and processing data from various sources. The role requires strong analytical skills, the ability to profile source data, and a minimum of 5 years of experience in data analysis or statistics. Candidates should hold a bachelor's degree in Computer Science or...

  • Data Scientist

    7 days ago

    Sandton, South Africa The Focus Group Full time

    The Focus Group Sandton, South Africa 19 February 2025 Mid 0 - 0 Full Time Cost to Company 4 years - 6 years Skills Big Data Computer Skills Data Data Analysis Data Science Digital Python Solutions Statistics Industries Management Medical Job Description **Purpose**: Developing personalized models and algorithms aimed at fostering healthy habits, achieving...

  • Data Scientist

    2 weeks ago

    Sandton, South Africa Discovery Full time

    A leading healthcare company in Sandton is seeking an Intermediate Data Scientist to leverage predictive analytics and big data to enhance client services. The ideal candidate will have experience in data science programming and machine learning, holding at least an Honours or Master’s degree and 2 years of relevant experience. This role offers an exciting...

  • Data Scientist

    1 week ago

    Sandton, South Africa Discovery Full time R800 000 - R1 200 000 per year

    Discovery HealthIntermediate Data ScientistAbout Discovery Discovery's core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and whose own values align with those of Discovery. Our fast-paced and dynamic environment enables smart,...

  • Data Strategist

    3 days ago

    Sandton, South Africa Staff Solutions Full time

    Gauteng, Sandton - Monthly Cost To Company (Market related) Our well known and much loved OEM has a position for a Data Strategist. Based in Sandton, the position is responsible to provide a holistic view of the organization’s use of data by interpreting all of its current usage and providing a future oriented, strategic plan as well as translating the...

  • Data Scientist

    2 weeks ago

    Sandton, South Africa Discovery Full time

    Discovery Health Intermediate Data ScientistAbout Discovery Discovery’s core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and whose own values align with those of Discovery. Our fast-paced and dynamic environment enables smart,...

  • Technical Lead

    7 days ago

    Sandton, South Africa The Focus Group Full time

    The Focus Group Sandton, South Africa 19 February 2025 Mid 0 - 0 Full Time Cost to Company 5 years - 6 years Skills Big Data Coaching Data Data Science Leadership Management Medical Industries Laboratory Supply Medical Job Description Lead the technical development, testing and implementation of advanced Large Language Model (LLM) based projects and related...

  • Data Science Manager

    3 days ago

    Sandton, South Africa Team Finder Full time

    **Role Overview**: Our client is entering a new phase where operational and commercial excellence are key to success. As part of their strategic vision, they are seeking a **Data Science Manager**to lead a team of data scientists and analysts in driving data-driven decision-making. This role involves developing machine learning models, managing data...