Data Engineering Lead(Big Data)

7 days ago


Johannesburg, South Africa Adcorp Holdings Full time

**Roles and Responsibilities** Internal Process**
- Responsible for the end-to-end technical solution. This means that key decisions, once discussed and debated, are appropriately understood, and implemented.
- Collaborate with the Product Development Lead and the Business Enablement Lead to ensure that all stakeholder expectations are met.
- Ensure that the company’s Edge platform can cater to all varieties of data, including batch and real-time, and is safe, secure, scalable, and stable.
- Manage the work items on behalf of the team, breaking down and assigning work items per sprint. Managing elements of sprint planning.
- DevOps CI/CD pipeline management, control, and oversight.
- Engage other platform owners and agree on optimal integration paths and methods.
- Outstanding communication and presentation skills, with the ability to convey complex concepts to non-technical stakeholders.

**Client Services**
- Ongoing relationship building with internal and external stakeholders to ensure smooth business processing.
- Ability to develop new improved processes and to facilitate discussions with external and internal stakeholders.
- Committed to Client experience.
- Enable business communications between business areas.
- Participate and contribute to a culture that builds rewarding relationships, facilitates feedback, and provides exceptional client service.
- Build trust and creditability with respect to Data in the financial market.

**People Management**
- Develop and maintain productive and collaborative working relationships with colleagues, peers, and stakeholders.
- Continuously develop own expertise in terms of professional, industry, and legislation knowledge.
- Contribute to continuous innovation through the development, sharing, and implementation of new ideas.
- Ability to work proactively and independently to deliver precise and accurate work.
- Take ownership of driving personal career development.
- Comfortable working with a strictly agile project methodology. This includes breaking down features and user stories for the team of data engineers into appropriately sized deliverables and when required, defining standards, performing code reviews, and validating for quality.
- Display an inquisitive, detailed-orientated nature, and be unafraid to ask difficult questions, or challenge the status quo when the need arises.
- Responsible for leveraging financial industry experience to build a modern, secure, scalable solution that is appropriate for the clients and partners that the company services.
- Makes use of, and develops cost-effective, right-sized solutions and services within Microsoft Azure’s Cloud Platform to meet the core platform needs.
- Displays consideration of the increasing risks faced as a society and as an organisation and suggests appropriate responses with regard to the solutions being built.

**Roles and Responsibilities (cont.)** Finance**
- Identify solutions to enhance cost -effectiveness and increase operational efficiency.
- Provide input into the risk identification processes and communicate recommendations in the appropriate forum.

**Job Requirements**
- Grade 12 or equivalent
- A bachelor’s degree in computer science or another closely related field
- 5-8 years of working experience in a related field.
- Proven work experience as a senior data engineer, or as a Data Engineering Lead
- Strong Data Bricks and Python skills are a must. SQL skills as an alternative to Python will also be considered
- Understand how big data solutions / NoSQL solutions differ from standard data solutions and explain why choices made might be different.
- Strong DevOps understanding, including CI/CD

Job Reference: TCN01680



  • Johannesburg, Gauteng, South Africa Data Centrix Full time

    Inherent RequirementsAt least 10 years of hands-on involvement in data science projects, including managing large-scale initiatives and leading multidisciplinary teams.Degree (Honours, Masters or PHD) in quantitative fields such as Actuarial Science, Statistics, Computer Science, Engineering, Mathematics and/or combination of these.Proven expertise in data...


  • Johannesburg, Gauteng, South Africa Data Centrix Full time

    Key Responsibilities:Define and implement data classification, data loss prevention (DLP), and encryption standards.Work with business units to identify sensitive data flows and implement appropriate controls.Oversee data privacy and compliance efforts (GDPR, local data protection regulations, IP protection requirements).Collaborate with IT, OT, and legal...

  • Big Data Specialist

    5 days ago


    Johannesburg, Gauteng, South Africa Tower Group (Pty) Ltd Full time

    Tower Group (Pty) Ltd is committed to delivering high-quality services to our clients. We are now seeking a Big Data Specialist to join our team and help us achieve this goal.The ideal candidate will have a strong background in big data technologies, including Hadoop, Spark, and NoSQL databases. They will be responsible for designing and implementing data...

  • Big Data Architect

    1 week ago


    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for a Big Data Architect. In this position you have the opportunity to help lead the change on operational strategy and your responsibilities will include to validate the reference architecture and provide details for the architecture of platforms, leading and working hands on towards implementation and delivery to production of...

  • Big Data Architect

    3 days ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Job Description:A skilled Data Engineer is required by Network Recruitment to design, build, and optimize data pipelines for real-time and batch processing. The ideal candidate will have a strong passion for big data, cloud technologies, and scalable solutions.Key Qualifications:Bachelor's Degree in Computer Science, Computer Engineering, or a related...

  • Data Analyst

    1 week ago


    Johannesburg, Gauteng, South Africa Data Bene Full time

    A vacancy exists for a Data Analyst within the Micro Merchant Division – Touchsides, in Rosebank, Johannesburg. The primary focus as Data Analyst is to collect, analyze, clean and interpret data sets across the business to make informed decisions. Utilize technical expertise to ensure data is accurate and high-quality.Key Responsibilities include, but are...

  • Data Engineer

    5 days ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology-driven company is seeking a skilled Data Engineer to design, build, and optimize data pipelines for real-time and batch processing. If you're passionate about big data, cloud technologies, and scalable solutions, this is your chance to make an impactQualifications:Bachelors Degree in Computer Science, Computer Engineering, or related...


  • Johannesburg, Gauteng, South Africa Paracon Full time

    About the RoleWe are looking for an experienced Cloud Data Architect to join our team in Johannesburg. As a key member of our data platform, you will be responsible for designing and implementing large-scale enterprise data solutions on the AWS cloud.The ideal candidate will have hands-on experience with AWS cloud services, proficiency in SQL, Python, and...

  • Data Engineer

    3 days ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology-driven company is seeking a skilled Data Engineer to design, build, and optimize data pipelines for real-time and batch processing. If you're passionate about big data, cloud technologies, and scalable solutions, this is your chance to make an impactQualifications: Bachelors Degree in Computer Science, Computer Engineering, or a related...


  • Johannesburg, Gauteng, South Africa Ananzi Full time

    Join Our Team as a Data Streaming EngineerWe are seeking a skilled Data Streaming Engineer to join our team at Ananzi. As a member of our team, you will be responsible for building and maintaining our real-time data pipeline, working closely with infrastructure teams to ensure seamless data systems integration, and identifying opportunities for process...

  • Data Specialist

    2 weeks ago


    Johannesburg, South Africa Dimension Data Full time

    **Role Description** The IT Data Specialist applies data analysis, data modelling and quality assurance techniques based on a detailed understanding of business processes to establish, modify or maintain internal data structures and associated components (entity, relationship and attribute descriptions). They develop and maintain of a formal description of...


  • Johannesburg, Gauteng, South Africa Golden Rule Full time

    About the Role:We are seeking an experienced Big Data Streaming Architect to lead the design and implementation of our real-time data pipeline. As a key member of our team, you will be responsible for architecting and implementing scalable data solutions that meet the needs of our business.Responsibilities:Data Pipeline Design: Design and develop scalable...


  • Johannesburg, South Africa Data Centrix Full time

    Requirements - Minimum 5 years in a data-centric environment working in either SQL Data-Warehousing, - Analytics or as a dedicated Data-Modeller. - Above average TSQL and SQL server skills. - Logical, and Physical Data-Modelling Expertise. - Experience with Forward and Reverse Engineering of Existing Databases. - Good understanding of Star Schemas, Snowflake...


  • Johannesburg, South Africa Dimension Data Full time

    As a Financial Data Analyst, you'll analyse data and develop analytical models to provide financial reporting, forecasts and risk management. You will have experience in either an analyst or senior role within the finance department of a large organisation. You should also be fluent in SQL and familiar with R programming language. **Requirements - Acquire...

  • Data Engineer

    4 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology firm is seeking a highly skilled Data Engineer to join their team in a hybrid/remote capacity. If you have experience in Python, SQL, AWS, and ETL processes , and are passionate about building and optimizing data pipelines , this role is for you Qualifications: BSc in Computer Science, Information Systems, Engineering, or a related...

  • Data Engineer

    4 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology firm is seeking a highly skilled Data Engineer to join their team in a hybrid/remote capacity. If you have experience in Python, SQL, AWS, and ETL processes , and are passionate about building and optimizing data pipelines , this role is for youQualifications: BSc in Computer Science, Information Systems, Engineering, or a related...


  • Johannesburg, South Africa NTT DATA Full time

    **Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT DATA** This...


  • Johannesburg, Gauteng, South Africa Data Bene Full time

    About the Role:The successful candidate will possess strong analytical skills, the ability to communicate complex ideas simply, and a passion for data-driven decision-making.Key Responsibilities:Collect, analyze, and interpret data setsTranslate business questions into actionable insightsDevelop ad hoc analyses from start to finishParticipate in data...

  • Data Engineer

    4 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    We are seeking a highly skilled and experienced Data Engineer to join our team. In this role, you will be responsible for developing and maintaining scalable, secure, and repeatable data pipelines. You will manage data platforms, ensuring the accuracy and accessibility of data for reporting and analysis across the business. Key Responsibilities: Implement...


  • Johannesburg, South Africa e-Merge IT Recruitment Full time

    This industry leading insurance company is urgently seeking a Data Governance Lead to join their ranks. The main goal of this position is to implement a Data Governance Operating Framework for the organisation, emphasizing the enhancement of data governance and data quality by adjusting organizational behaviour, policies, standards, principles, governance...