Big Data Engineer Leader

3 days ago


Cape Town, Western Cape, South Africa Network It Full time

We are looking for a skilled Big Data Engineer Leader to join our clients' investment team. The role involves designing and developing scalable data pipelines and ETL processes to collect, process, and integrate data from various sources into a centralized data warehouse or data lake.

The successful candidate will be responsible for efficiently managing the reception, transfer, processing, analysis, and integration of third-party systems for investment research and business operational processes.

About the Role
  • Data Pipeline Development: Design and develop efficient data pipelines using Python and SQL.
  • Collaboration with Quantitative Analysts: Work closely with quantitative analysts to understand their data requirements and provide necessary infrastructure and tools for data exploration, analysis, and visualization.

Required Skills and Qualifications: Up to 3 years' experience with database management, data analysis, and producing ETL processes at scale is essential. Exceptional skill and experience in Python, SQL, Linux server administration, and Git version control are also required.



  • Cape Town, Western Cape, South Africa Amazon Development Centre (South Africa) (Proprietary) Limited - A78 Full time

    About the RoleWe are looking for an experienced Big Data Engineer Specialist to join our team at Amazon Development Centre (South Africa) (Proprietary) Limited - A78. As a key member of our engineering team, you will be responsible for designing, developing, and implementing large-scale big data solutions using Hadoop and other relevant technologies.Your...

  • Big Data Engineer

    7 days ago


    Cape Town, Western Cape, South Africa ENGINEERINGUK Full time

    Job OverviewAt ENGINEERINGUK, we're seeking a skilled Big Data Engineer to join our innovative cloud support team. As a key member of our global organization, you'll play a vital role in shaping the future of big data solutions.About the RoleSupport customers with big data services including Apache Spark, Hive, Presto, and other Hadoop ecosystem...

  • Big Data Specialist

    4 days ago


    Cape Town, Western Cape, South Africa Afrizan Personnel Full time

    Afrizan Personnel is looking for an experienced Big Data Specialist to join their team. The main responsibility of this role is to design and implement big data solutions that support business needs.This involves:Big Data Technologies: Utilizing big data technologies like Hadoop, Spark, and Kafka to handle large-scale data processing.Database Management:...

  • Data Engineer

    5 days ago


    Cape Town, Western Cape, South Africa OfferZen Ltd Full time

    Company OverviewOfferZen Ltd is a developer jobs marketplace that reverses the traditional job search process. We connect talented data engineers with top tech companies in Africa and Europe, facilitating a seamless interview experience.Job DescriptionWe are seeking an experienced Data Engineer to join our team. The successful candidate will have hands-on...


  • Cape Town, Western Cape, South Africa Sanlam Limited Full time

    Data Engineer RoleWe are seeking an experienced Data Engineer to join our team. As a key member of our engineering organization, you will be responsible for designing, building, and maintaining large-scale data systems that meet the needs of our business units.Key ResponsibilitiesDesign and implement data ingestion pipelines to collect and process large...


  • Cape Town, Western Cape, South Africa Dimension Data Full time

    Dimension Data is seeking an Enterprise Cloud Engineering Leader to spearhead the development and delivery of cloud-based solutions. As a senior technical expert, you will be responsible for leading a team of engineers in designing and implementing complex cloud architectures that meet the needs of our clients.


  • Cape Town, Western Cape, South Africa Amazon Full time

    Big Data Support Technician: Join our talented team of experts who are shaping the future of cloud computing and big data analytics. As a Big Data Support Technician at Amazon Web Services, you will be responsible for resolving complex technical issues related to Hadoop administration, distributed systems, and cloud infrastructure. Your expertise will help...


  • Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group is currently seeking an experienced AWS Data Engineer to join their agile team. The ideal candidate will have 2 to 5 years of experience in constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security.Data Engineers are responsible for establishing the infrastructure required to derive insights...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering department, you will be responsible for designing and implementing high-performance data systems using open-source tools.About Us:We are Network Recruitment, a leading provider of specialized staffing...


  • Cape Town, Western Cape, South Africa BADRI Management Consultancy Full time

    Job OverviewWe are looking for a highly skilled Principal Lead: Data Engineering to lead our data engineering team and drive large-scale data initiatives. The successful candidate will have experience in implementing data mesh architectures at scale, AWS technologies, and certification in cloud data platforms.The ideal candidate will be a proven leader, able...

  • Big Data Specialist

    2 days ago


    Cape Town, Western Cape, South Africa iqbusiness Full time

    Required Skills and QualificationsProficiency in AWS and Big Data technologies such as EC2, S3, Lambda, and Spark/SparkSQL.Strong programming skills in languages like Java, C#, Node.js, Python, and SQL.Experience with DevOps tools like GitHub, GitLab, Jenkins, CodeBuild, CodePipeline, and CodeDeploy.

  • Big Data Engineer

    7 days ago


    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group is a leading provider of innovative data streaming solutions. We're seeking an experienced Real-Time Data Architect to join our team and contribute to the development of cutting-edge data processing systems.About the Position:The successful candidate will design and implement robust streaming pipelines using Kafka and Flink, ensuring seamless data...


  • Cape Town, Western Cape, South Africa Capitalh Full time

    We're looking for a Data Engineering Specialist to join our team at CAPITALH. As a key member of our data engineering team, you'll design, develop, and maintain data pipelines and architectures.A Bachelor's degree in Computer Science or Software Engineering is required, along with 5+ years of experience in data engineering, preferably with Google Cloud...


  • Cape Town, Western Cape, South Africa Ozow group. Full time

    Job Description:The Ozow group is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams.You will work closely with our development team to design, build, and deploy...


  • Cape Town, Western Cape, South Africa CodeConnect Staffing (Pty) Ltd Full time

    About the JobAs a Senior Data Engineer, you will be responsible for leading the design and maintenance of scalable data pipelines and architectures. Your role will involve working closely with cross-functional teams to ensure the successful implementation of cloud-based solutions.Lead the design and maintenance of scalable data pipelinesArchitect efficient...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    Key Responsibilities: Architect and build highly scalable distributed systems using open-source tools. Design and implement efficient data models, with a deep understanding of various data structures and their benefits and limitations. Work with big data batch and streaming tools to process and analyze data at scale. Develop and optimize Extract,...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    Key Responsibilities: Architect and build highly scalable distributed systems using open-source tools. Design and implement efficient data models, with a deep understanding of various data structures and their benefits and limitations. Work with big data batch and streaming tools to process and analyze data at scale. Develop and optimize Extract, Transform,...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    Key Responsibilities: Architect and build highly scalable distributed systems using open-source tools. Design and implement efficient data models, with a deep understanding of various data structures and their benefits and limitations. Work with big data batch and streaming tools to process and analyze data at scale. Develop and optimize Extract, Transform,...


  • Cape Town, Western Cape, South Africa Network Recruitment Full time

    Key Responsibilities: Architect and build highly scalable distributed systems using open-source tools. Design and implement efficient data models, with a deep understanding of various data structures and their benefits and limitations. Work with big data batch and streaming tools to process and analyze data at scale. Develop and optimize Extract, Transform,...


  • Cape Town, Western Cape, South Africa Workskuad (formerly Goodtalent) Full time

    Workskuad, a leading business platform, seeks an exceptional Senior Engineering Leader to drive its engineering team's success in Cape Town, South Africa.This is a high-profile opportunity to lead improvements in architectural and application design, engineering best practices, project planning, and DevOps. The ideal candidate will bridge technology gaps and...