Senior AWS Data Engineer

2 weeks ago

Johannesburg, Gauteng, South Africa Golden Rule Full time

Our client who is a leader in the Financial Services space has an exciting opportunity for a Senior AWS Data Engineer. 

 Minimum Qualifications

  • Bachelor's Degree in Computer Science, Information Technology, or other relevant fields.
  • Has experience in any of the following AWS Athena, AWS Glue, Pyspark, AWS DynamoDB, AWS Redshift, AWS Lambda and AWS Step Functions.
  • Proficient in SQL, Python and PySpark
  • Proficient in utilizing a cloud platform and services
  • Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, devops and operations.

Additional Experience

  • A successful history of manipulating, processing and extracting value from large, disconnected data sets.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Experience in a Data Engineering roles.
  • Experience with the following is a must: AWS Glue, PySpark, SQL
  • Experience with data pipeline and workflow management tools like AWS StepFunctions and Control-M would be beneficial but not required.
  • Experience with AWS cloud services: EC2, EMR, RDS, DynamoDB would be beneficial but not required.

Experience

  • Advanced Data Engineering knowledge and experience working with modern data practices.
  • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
  • Experience working with distributed systems as it pertains to data storage and computing.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets.
  • Build processes supporting data transformation, data structures, meta data, dependency, and workload management.
  • Proficiency in SQL with expertise in writing and optimizing complex queries

Responsibilities

  • Design and implement large scale enterprise data solutions by using a combination of the of the following technologies – AWS Glue, AWS Step-functions, AWS RedShift, AWS Lambda, AWS Athena, AWS Lakeformation, Spark, Python.
  • Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS and 3rd party services.
  • Design and build production data pipelines from ingestion to integration within a big data architecture, using PySpark, Python and SQL.
  • Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
  • Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
  • Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency.
  • Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning.
  • Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.

  • AWS Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience: 5 years of experience in Python/C# development 3 years of experience in AWS data engineering Education Requirements: Bachelors Degree in Computer Science, Information Systems, or a related field Advantageous: AWS Certified Machine Learning Specialty Certificate Responsibilities: Design and create data models that can extract information...

  • AWS Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency.Key Responsibilities:Build data lakes and APIs using AWS (S3, Quicksight).Design data models and implement CI/CD workflows.Collaborate...

  • AWS Data Engineer

    4 hours ago

    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency.Key Responsibilities:Build data lakes and APIs using AWS (S3, Quicksight).Design data models and implement CI/CD workflows.Collaborate...

  • AWS Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience:5 years of experience in Python/C# development3 years of experience in AWS Data EngineeringEducation Requirements:Bachelors degree in Computer Science, Information Systems, or a related fieldAdvantageous:AWS Certified Machine Learning Specialty CertificateResponsibilities:Design and create data models that can extract information from...

  • AWS Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience:5 years of experience in Python/C# development3 years of experience in AWS data engineeringEducation Requirements:Bachelors Degree in Computer Science, Information Systems, or a related fieldAdvantageous:AWS Certified Machine Learning Specialty CertificateResponsibilities:Design and create data models that can extract information from...

  • AWS Data Engineering Specialist

    5 days ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    We are seeking a highly skilled AWS Data Engineering Specialist to join our team at E-Merge. In this role, you will be responsible for designing, building, and maintaining Big Data pipelines using AWS services.About the RoleThe ideal candidate will have experience working with cloud-based data engineering tools and technologies, including AWS EMR, Redshift,...

  • Senior Cloud Data Engineer

    6 days ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    E-Merge, an IT South Africa Hub delivering cutting-edge IT solutions, has an opening for a Senior Cloud Data Engineer.About the RoleThis position is responsible for designing, building, and maintaining Big Data pipelines. The successful candidate will ensure data sharing follows information classification policies and need-to-know principles. Additionally,...

  • Data Science Professional

    5 days ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    E-Merge is a forward-thinking organization that fosters innovation and global collaboration. We're seeking a highly skilled data scientist to join our team and contribute to the development of our AWS data engineering architecture.This role will involve developing data science solutions, implementing analytical algorithms, and supporting business teams with...

  • AWS Cloud Data Specialist

    3 days ago

    Johannesburg, Gauteng, South Africa Zensar Technologies Full time

    Data Engineering RoleWe are seeking an experienced AWS Data Engineer to join our Global Markets division. In this role, you will work closely with our traders, quantitative analysts, and data scientists to design, develop, and maintain data infrastructure and pipelines for our global markets business.The ideal candidate should have a strong background in C#...

  • Senior Cloud Engineer – Data Transformation

    6 days ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    We are looking for an experienced Senior Cloud Engineer - Data Transformation to join our team at E-Merge. In this role, you will design, build, and deliver transformative business solutions using modern data platforms. Your responsibilities will include integrating, storing, modeling, analyzing, and reporting data while aligning solutions with business...

  • AWS Data Engineer

    1 week ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    A leader in African capital markets with a comprehensive suite of products, a vast global reach while and an acute positioning as a local bank with the expertise ability to leverage on local presence with regional and international expertise has a vacant position for an AWS Data Engineer to work within a team solving complex problems to deliver real business...

  • AWS Cloud Data Specialist

    6 days ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    **Job Description:**We are seeking a highly skilled AWS Cloud Data Specialist to join our team at Network Recruitment. As a Cloud Data Specialist, you will be responsible for designing and implementing large-scale data solutions on the AWS cloud platform.In this role, you will lead the design, implementation, and successful delivery of complex data pipelines...

  • Senior Data Solutions Architect

    6 days ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    **Job Overview:**We are looking for a Senior Data Solutions Architect to join our team at Network Recruitment. As a Senior Data Solutions Architect, you will be responsible for leading the design, implementation, and successful delivery of large-scale data solutions on the AWS cloud platform.In this role, you will work closely with cross-functional teams to...

  • Data Engineer

    3 weeks ago

    Johannesburg, Gauteng, South Africa Edge Executives Full time

    We are seeking a skilled Data Engineer to design and optimize scalable data infrastructure for AI, machine learning, and generative AI initiatives. This role involves building ETL/ELT pipelines, managing AWS-based data systems, and collaborating with cross-functional teams to develop cutting-edge solutions. Ideal candidates will have experience in AWS...

  • Senior Data Engineering Professional

    3 days ago

    Johannesburg, Gauteng, South Africa Zensar Technologies Full time

    Qualifications and CompetenciesTo be successful in this role, the candidate must have experience with AWS Glue Jobs using PySpark or AWS Glue Spark. Realtime ingestion using KAFKA is an added advantage. Strong SQL and C# or Python programming knowledge is also required.The candidate should have expert knowledge of data engineering packages and libraries and...

  • Data Streaming Engineer

    6 days ago

    Johannesburg, Gauteng, South Africa Golden Rule Full time

    Build, enhance, and maintain our real-time data pipeline. Work with various infrastructure and operations teams to maintain our data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the continued growth of the bank. You will collaborate with many...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Optim-G Sourcing Full time

    Experience Required: Professional Experience: Minimum of 3-5 years of experience in data modelling and related fields.- AWS Certification: AWS Certified Solutions Architect or AWS Certified Data Analytics Specialty certification is a plus.- Project Management: Experience in managing data-related projects and working in an Agile environment.- Industry...

  • Data Engineer

    3 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology firm is seeking a highly skilled Data Engineer to join their team in a hybrid/remote capacity. If you have experience in Python, SQL, AWS, and ETL processes , and are passionate about building and optimizing data pipelines , this role is for you Qualifications: BSc in Computer Science, Information Systems, Engineering, or a related...

  • Data Engineer

    2 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology firm is seeking a highly skilled Data Engineer to join their team in a hybrid/remote capacity. If you have experience in Python, SQL, AWS, and ETL processes , and are passionate about building and optimizing data pipelines , this role is for youQualifications: BSc in Computer Science, Information Systems, Engineering, or a related...

  • Cloud-Based Data Engineer

    6 days ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    **Job Title:** Cloud-Based Data EngineerWe are seeking a highly skilled Cloud-Based Data Engineer to join our team at Network Recruitment. As a Cloud-Based Data Engineer, you will be responsible for designing and implementing large-scale data solutions on the AWS cloud platform.In this role, you will work closely with cross-functional teams to deliver...