Big Data Services Lead

4 hours ago

Sandton, South Africa Absa Bank Limited Full time

Bring your possibility to life Define your career with us
- With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary

Team Context:
Data Engineering is responsible for the central data platform that receives and distributes data across the bank. This is a multi-platform environment and leverages a blend of custom, commercial and open-source tools to manage and support thousands of critical data-related jobs. These jobs are supported and updated in line with changes across the landscape to avoid disruption to downstream data consumers.

**Job Description**:
Role Description

The Big Data Services Lead role is responsible for driving optimization of the end-to-end environment. This includes the Hadoop platform, scheduling, automation, monitoring and the various aspects of supportability and efficiency required to manage a large, complex environment.

**Responsibilities**:

- Manage an assigned team through day to day support tasks
- Oversee enhancements and new developments, providing guidance and peer review
- Build and deploy new data pipelines
- Oversee development plans for the team and provide mentorship to the team
- Identify and drive optimisation opportunities across the environment
- Drive improvements to environment supportability and maintain effective cross team relationships
- Test changes to internal tools and approve releases, with a focus on optimizing test execution
- Drive improvements to environment supportability and maintain effective cross team relationships
- Advise users on best practices and conduct training sessions where required
- Define and continuously improve the team best practice guides and standards
- Translate complex functional and technical requirements into detailed design for the team to build

Job Experience & Skills Required:

- 4 years’ experience working in Big data environment, optimising and building big data pipelines, architectures and data sets with e.g. Java, Scala, Python, Hadoop, Apache Spark and Kafka
- Cross domain knowledge
- Experience with designing and building, BI systems and complex data eco systems
- Minimum one year experience with Scala programming language
- Familiarity with Hadoop ecosystem and its components
- Solid experience in a working environment in Big Data development utilising SQL or Python
- Experience in Big Data development using Spark
- Experience in Hadoop, HDFS and MapReduce
- Experience in database design, development and data modelling

The following additional knowledge, skills and attributes are preferred:

- Good knowledge in back-end programming, specifically Java
- Understanding of Cloud technologies and migration techniques
- Understanding of data streaming and the intersection of batch and real time data
- Experience with development in a Linux environment and its basic commands
- Ability to write reliable, manageable, and high-performance code
- Should have basic knowledge of SQL, database structures, principles, and theories
- Knowledge of workflow/schedulers
- Strong collaboration and communication skills
- Strong analytical and problem solving skills
- Data architecture and security
- Data management

Education

Bachelor's Degree: Information Technology
- Absa Bank Limited reserves the right not to make an appointment to the post as advertised_

  • Big Data Administrator

    2 months ago

    Sandton, South Africa Discovery Ltd. Full time

    **Business Unit**:Discovery Health**Function**:Information Technology**Date**:4 Nov 2024Discovery - Health Systems (DHS) Big Data Administrator - About Discovery Discovery’s core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and...

  • Big Data Infrastructure Specialist

    3 weeks ago

    Sandton, South Africa Datonomy Solutions Full time

    Designing and Delivering Data Engineering SolutionsWe are seeking an experienced Data Engineer to join our team at Datonomy Solutions. As a key member of our team, you will be responsible for designing, developing, and maintaining our data infrastructure.About Our TeamOur team of experts is dedicated to delivering innovative solutions that meet the evolving...

  • Lead Hadoop Administrator

    3 months ago

    Sandton, South Africa Absa Bank Limited Full time

    Empowering Africa’s tomorrow, together one story at a time. - With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group. Job...

  • Data Strategist

    2 days ago

    Sandton, South Africa Volkswagen Group Full time

    **Job Description and Responsibilities**: - As the expert on latest technology trends & products in the space of (big) data, analytics, ad-tech, machine learning and AI, be a thought-partner for managers and executives by bridging market trends to their specific challenges, dynamics, limitations & ambitions. - Guide the organization from the maze of...

  • Data Strategist

    6 days ago

    Sandton, South Africa RMG Recruitment (Pty) Ltd Full time

    Based in Sandton, the position is responsible to provide a holistic view of the organization’s use of data by interpreting all of its current usage and providing a future oriented, strategic plan as well as translating the requirements of executives, managers into concrete data-driven initiatives and specific actions. **Job Description and...

  • Team Lead Data Science

    2 months ago

    Sandton, South Africa Brambles Group Full time

    CHEP helps move more goods to more people, in more places than any other organization on earth via our 300 million pallets, crates and containers. We employ 11,000 people and operate in more than 55 countries. Through our pioneering and sustainable share-and-reuse business model, the world’s biggest brands trust us to help them transport their goods more...

  • Data Strategist

    7 days ago

    Sandton, South Africa Staff Solutions Full time

    Gauteng, Sandton - Monthly Cost To Company (Market related) Our well known and much loved OEM has a position for a Data Strategist. Based in Sandton, the position is responsible to provide a holistic view of the organization’s use of data by interpreting all of its current usage and providing a future oriented, strategic plan as well as translating the...

  • Big Data Architecture Developer

    3 weeks ago

    Sandton, South Africa Datonomy Solutions Full time

    Are you an experienced Data Engineer looking for a new challenge? We are seeking a skilled professional to join our team in Johannesburg and contribute to the design and development of our data infrastructure.You will be responsible for developing and maintaining data pipelines using Ab Initio and other technologies, as well as collaborating with...

  • Data Systems Architect

    3 weeks ago

    Sandton, South Africa Datonomy Solutions Full time

    Company OverviewDatonomy Solutions is a leading provider of data engineering services. Our team of experts is dedicated to delivering innovative solutions that meet the evolving needs of our clients.Estimated Salary: R 600,000 - R 900,000 per annumJob Description:We are seeking an experienced Data Engineer to join our team based in Johannesburg. The...

  • Data Scientist, Platform Service Management

    7 months ago

    Sandton, South Africa Intelsat Full time

    **Your role in helping us shape the future**: Join our dynamic and innovative team of Platform Service Management professionals who specialize in platform resource planning, service management and product analytics which supports the strategy to maximize our Fleets’ revenue capture capabilities and deliver a superior customer experience. You will play a...

  • Data Analyst

    2 months ago

    Sandton, South Africa Discovery Ltd. Full time

    **Business Unit**:Discovery Health**Function**:Data Analysis**Date**:25 Oct 2024 Discovery Health Data Analyst - About Discovery - Discovery’s core purpose is to enhance and protect people’s lives. It does this through breakthrough product designs that harness incentives to encourage people to make healthier lifestyle choices. Healthy behaviour leads...

  • Junior Data Scientist

    4 weeks ago

    Sandton, South Africa Discovery Ltd. Full time

    **Business Unit**:Discovery Health **Function**:Data Sciences **Date**:13 Dec 2024 Discovery Health Junior Data Scientist - About Discovery - Discovery’s core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and whose own values...

  • Senior Data Systems Engineer

    3 weeks ago

    Sandton, South Africa Datonomy Solutions Full time

    We are seeking a highly skilled Senior Data Systems Engineer to lead our data infrastructure development in Johannesburg.As a key member of our team, you will design and implement efficient data pipelines using Ab Initio and other technologies. Your expertise in SQL, big data, and cloud platforms such as Azure is crucial to the success of this role.The ideal...

  • Senior Software Developer

    3 weeks ago

    Sandton, South Africa Datonomy Solutions Full time

    Job SummaryWe are looking for an experienced Data Engineer to join our team at Datonomy Solutions. As a key member of our team, you will be responsible for designing, developing, and maintaining our data infrastructure.About UsDatonomy Solutions is a leading provider of data engineering services. Our team of experts is dedicated to delivering innovative...

  • Data Scientist

    1 week ago

    Sandton, South Africa Absa Bank Limited Full time

    Bring your possibility to life! Define your career with us - With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary Use...

  • Data Scientist

    5 days ago

    Sandton, South Africa e-Merge IT Recruitment Full time

    **Requirements**: - A minimum of 4 years’ experience in Data Science role - Experience with R, Python, Matlab is required - Experience in programming in C# or C++, Java. - Experience working with large scale data etc. - Experience With Cloud Data Engineering Technologies Is Advantageous - Implement complex data projects - Modelling and visualising large...

  • Data Engineer

    2 months ago

    Sandton, South Africa Jobted ZA C2 Full time

    We are looking for Data Engineers to join our team based in JHB- onsite. The ideal candidate will have extensive experience in data engineering, particularly with SAS, SQL and Azure Cloud Stack, and will be responsible for designing, developing, and maintaining the data infrastructure. Qualifications: Education: Bachelors or Masters degree in Computer...

  • Lead Data Scientist

    7 days ago

    Sandton, South Africa Absa Bank Limited Full time

    Bring your possibility to life! Define your career with us - With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary Work...

  • Data Engineer

    2 months ago

    Sandton, South Africa Datonomy Solutions Full time

    We are looking for Data Engineers to join our team based in JHB- onsite. The ideal candidate will have extensive experience in data engineering, particularly with SAS, SQL and Azure Cloud Stack, and will be responsible for designing, developing, and maintaining the data infrastructure. Qualifications: Education: Bachelors or Masters degree in Computer...

  • Data Engineer

    2 months ago

    Sandton, South Africa Datonomy Solutions Full time

    We are looking for Data Engineers to join our team based in JHB- onsite.The ideal candidate will have extensive experience in data engineering, particularly with SAS, SQL and Azure Cloud Stack, and will be responsible for designing, developing, and maintaining the data infrastructure.Qualifications: Education: Bachelors or Masters degree in Computer Science,...