Senior AWS Data Engineer

3 weeks ago


Johannesburg, Gauteng, South Africa Golden Rule Full time

Our client who is a leader in the Financial Services space has an exciting opportunity for a Senior AWS Data Engineer. 

 Minimum Qualifications

  • Bachelor's Degree in Computer Science, Information Technology, or other relevant fields.
  • Has experience in any of the following AWS Athena, AWS Glue, Pyspark, AWS DynamoDB, AWS Redshift, AWS Lambda and AWS Step Functions.
  • Proficient in SQL, Python and PySpark
  • Proficient in utilizing a cloud platform and services
  • Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, devops and operations.

Additional Experience

  • A successful history of manipulating, processing and extracting value from large, disconnected data sets.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Experience in a Data Engineering roles.
  • Experience with the following is a must: AWS Glue, PySpark, SQL
  • Experience with data pipeline and workflow management tools like AWS StepFunctions and Control-M would be beneficial but not required.
  • Experience with AWS cloud services: EC2, EMR, RDS, DynamoDB would be beneficial but not required.

Experience

  • Advanced Data Engineering knowledge and experience working with modern data practices.
  • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
  • Experience working with distributed systems as it pertains to data storage and computing.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets.
  • Build processes supporting data transformation, data structures, meta data, dependency, and workload management.
  • Proficiency in SQL with expertise in writing and optimizing complex queries

Responsibilities

  • Design and implement large scale enterprise data solutions by using a combination of the of the following technologies – AWS Glue, AWS Step-functions, AWS RedShift, AWS Lambda, AWS Athena, AWS Lakeformation, Spark, Python.
  • Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS and 3rd party services.
  • Design and build production data pipelines from ingestion to integration within a big data architecture, using PySpark, Python and SQL.
  • Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
  • Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
  • Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency.
  • Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning.
  • Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.


  • Johannesburg, Gauteng, South Africa Eqplus Technologies (Pty) Ltd Full time

    **Job Description:**Senior Data Engineer for AWSWe are looking for an experienced Senior Data Engineer to join our team at Eqplus Technologies (Pty) Ltd. The successful candidate will be responsible for designing and implementing data warehousing solutions using AWS services.Key Responsibilities:Design and develop data pipelines using AWS Lambda, Step...

  • AWS Data Engineer

    2 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience: 5 years of experience in Python/C# development 3 years of experience in AWS data engineering Education Requirements: Bachelors Degree in Computer Science, Information Systems, or a related field Advantageous: AWS Certified Machine Learning Specialty Certificate Responsibilities: Design and create data models that can extract information...

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency.Key Responsibilities:Build data lakes and APIs using AWS (S3, Quicksight).Design data models and implement CI/CD workflows.Collaborate...

  • AWS Data Engineer

    4 days ago


    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency. Key Responsibilities: Build data lakes and APIs using AWS (S3, Quicksight). Design data models and implement CI/CD workflows. ...

  • AWS Data Engineer

    6 days ago


    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency.Key Responsibilities:Build data lakes and APIs using AWS (S3, Quicksight).Design data models and implement CI/CD workflows.Collaborate...

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience:5 years of experience in Python/C# development3 years of experience in AWS Data EngineeringEducation Requirements:Bachelors degree in Computer Science, Information Systems, or a related fieldAdvantageous:AWS Certified Machine Learning Specialty CertificateResponsibilities:Design and create data models that can extract information from...

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience:5 years of experience in Python/C# development3 years of experience in AWS data engineeringEducation Requirements:Bachelors Degree in Computer Science, Information Systems, or a related fieldAdvantageous:AWS Certified Machine Learning Specialty CertificateResponsibilities:Design and create data models that can extract information from...


  • Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    Pioneering the future of data-driven innovation, we invite you to join our team as an AWS Data Platform Developer. In this role, you will leverage your expertise in data engineering to design and develop cutting-edge data platforms that drive business growth and efficiency.Responsibilities:Architect and implement data platforms using AWS services, focusing...


  • Johannesburg, Gauteng, South Africa Indsafri Full time

    Indsafri City of Johannesburg, Gauteng, South AfricaExp Required: Atleast 6+ Years of relevant work experience as Data engineer - Python & cloudJob Location: Johannesburg - Hybrid (need to go to office like 3 or 2 days in a week, rest of the days remote work)Company Name: IndsafriNotice Period: Immediate to 30 days of notice (More than 30 days of notice...

  • AWS Data Engineer

    2 weeks ago


    Johannesburg, Gauteng, South Africa E-Merge Full time

    A leader in African capital markets with a comprehensive suite of products, a vast global reach while and an acute positioning as a local bank with the expertise ability to leverage on local presence with regional and international expertise has a vacant position for an AWS Data Engineer to work within a team solving complex problems to deliver real business...

  • Data Engineer

    18 hours ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology-driven company is seeking a skilled Data Engineer to design, build, and optimize data pipelines for real-time and batch processing. If you're passionate about big data, cloud technologies, and scalable solutions, this is your chance to make an impactQualifications:Bachelors Degree in Computer Science, Computer Engineering, or related...


  • Johannesburg, Gauteng, South Africa Betway Group Full time

    Kick-start your career in the online gaming world and experience the very latest in technology and innovation.Data Pipeline Design and DevelopmentSenior Data Engineers are proficient in designing and implementing scalable and efficient data pipelines. They use streaming technologies such as Apache Spark, Apache Kafka, or equivalent to extract, transform, and...

  • Data Engineer

    4 weeks ago


    Johannesburg, Gauteng, South Africa Edge Executives Full time

    We are seeking a skilled Data Engineer to design and optimize scalable data infrastructure for AI, machine learning, and generative AI initiatives. This role involves building ETL/ELT pipelines, managing AWS-based data systems, and collaborating with cross-functional teams to develop cutting-edge solutions. Ideal candidates will have experience in AWS...


  • Johannesburg, Gauteng, South Africa Betway Group Full time

    About the RoleWe are looking for an experienced Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining large-scale data systems and infrastructure.As a Senior Data Engineer at Betway Group, you will work closely with cross-functional teams to understand data...


  • Johannesburg, Gauteng, South Africa Golden Rule Full time

    Build, enhance, and maintain our real-time data pipeline. Work with various infrastructure and operations teams to maintain our data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth of the bank. You will collaborate with many...

  • Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Optim-G Sourcing Full time

    Experience Required: Professional Experience: Minimum of 3-5 years of experience in data modelling and related fields.- AWS Certification: AWS Certified Solutions Architect or AWS Certified Data Analytics Specialty certification is a plus.- Project Management: Experience in managing data-related projects and working in an Agile environment.- Industry...

  • Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology firm is seeking a highly skilled Data Engineer to join their team in a hybrid/remote capacity. If you have experience in Python, SQL, AWS, and ETL processes , and are passionate about building and optimizing data pipelines , this role is for you Qualifications: BSc in Computer Science, Information Systems, Engineering, or a related...

  • Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology firm is seeking a highly skilled Data Engineer to join their team in a hybrid/remote capacity. If you have experience in Python, SQL, AWS, and ETL processes , and are passionate about building and optimizing data pipelines , this role is for youQualifications: BSc in Computer Science, Information Systems, Engineering, or a related...


  • Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking a highly skilled Data Products Engineer to join our team at PRR Recruitment. As a key member of our data engineering group, you will be responsible for designing, developing, and deploying innovative data products and pipelines that meet the needs of our clients.Responsibilities:Develop and implement data products using AWS services, with a...


  • Johannesburg, Gauteng, South Africa E-Merge Full time

    Technology consulting firm that empowers businesses with innovative cloud and data-driven solutions. We work alongside global organizations to create scalable, reliable, and secure solutions that drive their digital transformation.As a Cloud Data Engineer, you will be responsible for designing, developing, and implementing cloud-based data architectures to...