Big Data Services Lead
3 weeks ago
Bring your possibility to life Define your career with us
- With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary
Team Context:
Data Engineering is responsible for the central data platform that receives and distributes data across the bank. This is a multi-platform environment and leverages a blend of custom, commercial and open-source tools to manage and support thousands of critical data-related jobs. These jobs are supported and updated in line with changes across the landscape to avoid disruption to downstream data consumers.
**Job Description**:
Role Description
The Big Data Services Lead role is responsible for driving optimization of the end-to-end environment. This includes the Hadoop platform, scheduling, automation, monitoring and the various aspects of supportability and efficiency required to manage a large, complex environment.
**Responsibilities**:
- Manage an assigned team through day to day support tasks
- Oversee enhancements and new developments, providing guidance and peer review
- Build and deploy new data pipelines
- Oversee development plans for the team and provide mentorship to the team
- Identify and drive optimisation opportunities across the environment
- Drive improvements to environment supportability and maintain effective cross team relationships
- Test changes to internal tools and approve releases, with a focus on optimizing test execution
- Drive improvements to environment supportability and maintain effective cross team relationships
- Advise users on best practices and conduct training sessions where required
- Define and continuously improve the team best practice guides and standards
- Translate complex functional and technical requirements into detailed design for the team to build
Job Experience & Skills Required:
- 4 years’ experience working in Big data environment, optimising and building big data pipelines, architectures and data sets with e.g. Java, Scala, Python, Hadoop, Apache Spark and Kafka
- Cross domain knowledge
- Experience with designing and building, BI systems and complex data eco systems
- Minimum one year experience with Scala programming language
- Familiarity with Hadoop ecosystem and its components
- Solid experience in a working environment in Big Data development utilising SQL or Python
- Experience in Big Data development using Spark
- Experience in Hadoop, HDFS and MapReduce
- Experience in database design, development and data modelling
The following additional knowledge, skills and attributes are preferred:
- Good knowledge in back-end programming, specifically Java
- Understanding of Cloud technologies and migration techniques
- Understanding of data streaming and the intersection of batch and real time data
- Experience with development in a Linux environment and its basic commands
- Ability to write reliable, manageable, and high-performance code
- Should have basic knowledge of SQL, database structures, principles, and theories
- Knowledge of workflow/schedulers
- Strong collaboration and communication skills
- Strong analytical and problem solving skills
- Data architecture and security
- Data management
Education
Bachelor's Degree: Information Technology
- Absa Bank Limited reserves the right not to make an appointment to the post as advertised_
-
Big Data Technologies Manager
3 days ago
Sandton, South Africa Absa Bank Full timeWe seek a skilled specialist to manage Financial Data, Big Data technologies, and Data Infrastructure dependencies. As a key member of our team, you will develop and maintain the Coverage and Pricing analysis tool.Key Areas of FocusDevelop and maintain Big Data enabling technologiesSupport team members on Data Infrastructure dependenciesIdentify and mitigate...
-
Senior Big Data Engineer
5 days ago
Sandton, South Africa Yugantic Full timeResponsibilitiesAs a Senior Data Engineer at Yugantic, you will be responsible for developing and maintaining an analytical data infrastructure that provides ad-hoc access to extensive datasets and computing power. Your key responsibilities will include:Designing and Implementing Large-Scale Data Solutions: Leverage your expertise in AWS Glue, AWS Step...
-
Lead Data Governance Expert
3 days ago
Sandton, South Africa Absa Bank Full timeWe are an equal opportunity employer seeking a skilled specialist to join our team. As a Data Analyst Specialist, you will be responsible for managing Financial Data, Big Data technologies, and Data Infrastructure dependencies.Main ResponsibilitiesMaintain the integrity of the Coverage and Pricing analysis toolProvide technical support to team members on Big...
-
Data Operations Analyst
1 week ago
Sandton, South Africa Mimecast Services Limited Full timeOverview: **Data Operations Analyst** Interpret trends and stop cybercriminals in their tracks. Are you methodical and detail-oriented? Then join our dynamic Digital Business Innovation team. We use our high-powered tech stack to plan, control and improve data quality throughout the business. Day to day you’ll build reporting capabilities to identify...
-
Data Engineering Lead
18 hours ago
Sandton, South Africa Copious Software Full timeDrive Business Growth with Data InsightsCopious Software is dedicated to helping businesses grow by providing high-value technology services. As a Solution Engineer in the Data vertical, you will play a crucial role in designing, implementing, and managing data solutions on the Microsoft Azure cloud platform.Your responsibilities will include creating and...
-
Senior Data Engineer Lead
2 days ago
Sandton, South Africa Innovation Group Full timeAbout the JobThe Innovation Group is seeking an experienced Senior Data Engineer to lead our data engineering efforts. As a Senior Data Engineer, you will be responsible for developing and implementing data warehousing blueprints, evaluating hardware and software platforms, and integrating systems.Key ResponsibilitiesDevelop and implement data warehousing...
-
Data Scientist
12 hours ago
Sandton, South Africa Discovery Full timeDiscovery HealthIntermediate Data ScientistAbout DiscoveryDiscovery's core purpose is to make people healthier and to enhance and protect their lives. We seek out and invest in exceptional individuals who understand and support our core purpose, and whose own values align with those of Discovery. Our fast-paced and dynamic environment enables smart,...
-
Data Science Professional
9 hours ago
Sandton, South Africa Discovery Full timeAt Discovery, we strive to make a meaningful impact on people's lives by harnessing the power of data science.We are seeking an exceptional Senior Analytics Expert to join our Health Intelligence – Operations Intelligence unit. As a key member of our team, you will leverage predictive analytics, big data, and machine learning techniques to drive business...
-
Big Data Expert Wanted
2 days ago
Sandton, South Africa DeARX Full timeAbout the RoleWe are seeking an experienced Data Engineer to join our team in Sandton. As a key member of our data engineering team, you will be responsible for designing, developing, and optimizing data pipelines using Ab Initio.Main Responsibilities:Design and develop data pipelines using Ab Initio.Develop efficient ETL processes.Work closely with data...
-
Team Lead Data Science
2 weeks ago
Sandton, South Africa Brambles Group Full timeCHEP helps move more goods to more people, in more places than any other organization on earth via our 300 million pallets, crates and containers. We employ 11,000 people and operate in more than 55 countries. Through our pioneering and sustainable share-and-reuse business model, the world’s biggest brands trust us to help them transport their goods more...
-
Data Analyst Specialist
3 days ago
Sandton, South Africa Absa Bank Full timeEmpowering Africa's future requires a skilled specialist to manage the clusters requirements for Financial Data, Big Data technologies, and Data Infrastructure dependencies.Job SummaryMaintain the Coverage and Pricing analysis tool (data sharing, accessibility, enhancements)Manage Big Data enabling technologies and provide support to team membersIdentify and...
-
Senior Data Solutions Professional
3 days ago
Sandton, South Africa Absa Bank Full timeA career with Absa Bank offers the opportunity to be part of its exciting growth journey. We are seeking a skilled specialist to manage Financial Data, Big Data technologies, and Data Infrastructure dependencies.Key ResponsibilitiesDevelop and maintain the Coverage and Pricing analysis toolSupport team members on Big Data enabling technologiesIdentify and...
-
Data Engineer
7 days ago
Sandton, South Africa Brambles Full timeCHEP helps move more goods to more people, in more places than any other organization on earth via our 347 million pallets, crates and containers. We employ approximately 13,000 people and operate in 60 countries. Through our pioneering and sustainable share-and-reuse business model, the world's biggest brands trust us to help them transport their goods more...
-
Specialist Data Analyst
2 weeks ago
Sandton, South Africa Absa Bank Limited Full timeBring your possibility to life! Define your career with us - With over 100 years of rich history and strongly positioned as a local bank with regional and international expertise, a career with our family offers the opportunity to be part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group.Job Summary **Job...
-
Data Engineer Position
2 days ago
Sandton, South Africa DeARX Full timeAbout DeARXWe are a leading bank in Sandton, and we're looking for a talented Data Engineer to join our team.You will be responsible for designing, developing, and optimizing data pipelines using Ab Initio. You will also work closely with data scientists, analysts, and stakeholders to address data needs and ensure seamless integration.Key...
-
Lead Data Scientist for Insurance
2 days ago
Sandton, South Africa Guardrisk Group Full timeGuardrisk Group is a leading insurance company seeking an experienced Lead Data Scientist to join our team.As Lead Data Scientist for Insurance, you will be responsible for developing innovative solutions to automate business processes and enhancing operational efficiency.The ideal candidate will have a strong background in data analysis, excellent...
-
Data Team Lead
2 days ago
Sandton, South Africa Experian Health Full timeExperian Health is a leading provider of data and analytics solutions.Job OverviewThis position plays a critical role in managing the company's data operations. The successful candidate will oversee all aspects of data flows, ensuring that high-quality data is delivered efficiently and effectively.Data ManagementTeam LeadershipProblem SolvingTechnical...
-
Data Science Manager
4 weeks ago
Sandton, South Africa Team Finder Full time**Role Overview**: Our client is entering a new phase where operational and commercial excellence are key to success. As part of their strategic vision, they are seeking a **Data Science Manager**to lead a team of data scientists and analysts in driving data-driven decision-making. This role involves developing machine learning models, managing data...
-
Data Architecture Specialist
7 days ago
Sandton, South Africa Brambles Full timeJob DescriptionData Engineering Role OverviewThis position is based in South Africa.Key Responsibilities:Create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet functional/non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing...
-
Senior Data Engineer
2 days ago
Sandton, South Africa Innovation Group Full timeResponsible for developing data warehousing blueprints, evaluating hardware and software platforms and integrating systems. Manages the data environment which includes data design, database architecture, metadata and repository creation. Translates business needs into long-term architecture solutions. Defines, designs and builds dimensional databases....