Senior AWS Data Engineer

2 weeks ago


Johannesburg, Gauteng, South Africa Golden Rule Full time

Our client who is a leader in the Financial Services space has an exciting opportunity for a Senior AWS Data Engineer. 

 Minimum Qualifications

  • Bachelor's Degree in Computer Science, Information Technology, or other relevant fields.
  • Has experience in any of the following AWS Athena, AWS Glue, Pyspark, AWS DynamoDB, AWS Redshift, AWS Lambda and AWS Step Functions.
  • Proficient in SQL, Python and PySpark
  • Proficient in utilizing a cloud platform and services
  • Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, devops and operations.

Additional Experience

  • A successful history of manipulating, processing and extracting value from large, disconnected data sets.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Experience in a Data Engineering roles.
  • Experience with the following is a must: AWS Glue, PySpark, SQL
  • Experience with data pipeline and workflow management tools like AWS StepFunctions and Control-M would be beneficial but not required.
  • Experience with AWS cloud services: EC2, EMR, RDS, DynamoDB would be beneficial but not required.

Experience

  • Advanced Data Engineering knowledge and experience working with modern data practices.
  • Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
  • Experience working with distributed systems as it pertains to data storage and computing.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets.
  • Build processes supporting data transformation, data structures, meta data, dependency, and workload management.
  • Proficiency in SQL with expertise in writing and optimizing complex queries

Responsibilities

  • Design and implement large scale enterprise data solutions by using a combination of the of the following technologies – AWS Glue, AWS Step-functions, AWS RedShift, AWS Lambda, AWS Athena, AWS Lakeformation, Spark, Python.
  • Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS and 3rd party services.
  • Design and build production data pipelines from ingestion to integration within a big data architecture, using PySpark, Python and SQL.
  • Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
  • Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
  • Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency.
  • Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning.
  • Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience: 5 years of experience in Python/C# development 3 years of experience in AWS data engineering Education Requirements: Bachelors Degree in Computer Science, Information Systems, or a related field Advantageous: AWS Certified Machine Learning Specialty Certificate Responsibilities: Design and create data models that can extract information...

  • AWS Data Engineer

    4 weeks ago


    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency.Key Responsibilities:Build data lakes and APIs using AWS (S3, Quicksight).Design data models and implement CI/CD workflows.Collaborate...

  • AWS Data Engineer

    1 week ago


    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency. Key Responsibilities: Build data lakes and APIs using AWS (S3, Quicksight). Design data models and implement CI/CD workflows. ...

  • AWS Data Engineer

    2 weeks ago


    Johannesburg, Gauteng, South Africa PRR Recruitment Full time

    We are seeking an AWS Data Engineer to develop data products and pipelines in cloud and on-premises environments. Utilize your skills in Python, Dbt, and Apache Airflow to enhance business insights and operational efficiency.Key Responsibilities:Build data lakes and APIs using AWS (S3, Quicksight).Design data models and implement CI/CD workflows.Collaborate...

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience:5 years of experience in Python/C# development3 years of experience in AWS Data EngineeringEducation Requirements:Bachelors degree in Computer Science, Information Systems, or a related fieldAdvantageous:AWS Certified Machine Learning Specialty CertificateResponsibilities:Design and create data models that can extract information from...

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Minimum Experience:5 years of experience in Python/C# development3 years of experience in AWS data engineeringEducation Requirements:Bachelors Degree in Computer Science, Information Systems, or a related fieldAdvantageous:AWS Certified Machine Learning Specialty CertificateResponsibilities:Design and create data models that can extract information from...


  • Johannesburg, Gauteng, South Africa E-Merge Full time

    African capital markets leader E-Merge has a comprehensive suite of products and global reach, positioning itself as a local bank with regional and international expertise. We have a vacant position for an AWS Data Engineer to work within a team solving complex problems to deliver real business value across various industries, including helping develop best...


  • Johannesburg, Gauteng, South Africa E-Merge Full time

    E-Merge OverviewE-Merge is a pioneering company specializing in cloud data solutions, aiming to revolutionize the way businesses operate through innovative technology.Job OpportunityWe invite applications from experienced professionals for the position of AWS Data Engineering Lead. As a key member of our data engineering team, you will design and implement...


  • Johannesburg, Gauteng, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Cloud Data Engineer.Duties:Coordinate / Prioritise the work that the Developers and Product OwnerAssist with cross-team projects that involve development.AWS practices are followed with sound security and governance.Coordinate with the cloud Center of Excellence team for any AWS account level security and...


  • Johannesburg, Gauteng, South Africa Planet10 Full time

    A leader in the insurance space is on the lookout for an AWS Data Cloud Engineer (Big Data Engineer).Knowledge / QualificationsHigh knowledge in AWS – security groups, permissions, VPC, roles, services.Experience developing engineering applications in corporate companies.Demonstrated project development and leadership skills.Master's degree in Computer...


  • Johannesburg, Gauteng, South Africa Golden Rule Full time

    Company OverviewOur client is a leader in the Financial Services space with an exciting opportunity for a Senior AWS Data Engineer.The ideal candidate will have experience working with large-scale enterprise data solutions and a strong understanding of cloud computing platforms.


  • Johannesburg, Gauteng, South Africa Golden Rule Full time

    About UsWe are a dynamic company looking for a talented Senior AWS Data Engineer to join our team.This role requires a strong background in designing and implementing large-scale data solutions using AWS technologies such as Glue, Step-functions, RedShift, Lambda, Athena, and Lakeformation.


  • Johannesburg, Gauteng, South Africa Planet10 Full time

    A leading insurance company seeks a skilled AWS Data Cloud Engineer to design, implement, and manage large-scale data processing systems.ResponsibilitiesCreate high-performance data pipelines using AWS services such as Glue, Lake Formation, and Redshift.Develop and maintain scalable data architectures for batch and real-time data processing.Collaborate with...


  • Johannesburg, Gauteng, South Africa E-Merge Full time

    E-Merge, a leading African capital markets company, seeks an experienced AWS Data Engineer to join our team in Johannesburg. As a key member of our data engineering team, you will be responsible for designing, implementing, and delivering large-scale, critical data solutions that drive business value across various industries.Key Skills and...


  • Johannesburg, Gauteng, South Africa Indsafri Full time

    Indsafri City of Johannesburg, Gauteng, South AfricaExp Required: Atleast 6+ Years of relevant work experience as Data engineer - Python & cloudJob Location: Johannesburg - Hybrid (need to go to office like 3 or 2 days in a week, rest of the days remote work)Company Name: IndsafriNotice Period: Immediate to 30 days of notice (More than 30 days of notice...

  • AWS Data Engineer

    3 weeks ago


    Johannesburg, Gauteng, South Africa E-Merge Full time

    A leader in African capital markets with a comprehensive suite of products, a vast global reach while and an acute positioning as a local bank with the expertise ability to leverage on local presence with regional and international expertise has a vacant position for an AWS Data Engineer to work within a team solving complex problems to deliver real business...


  • Johannesburg, Gauteng, South Africa Falcorp Resourcing Full time

    Senior Data PositionFalcorp Resourcing seeks a Senior Data Retrieval Engineer to lead data analysis initiatives and drive business insights.About the JobLead data analysis projects to improve business processesDesign and implement database solutions using SQLCommunicate findings to stakeholders through reports and presentationsKey ResponsibilitiesExtract,...

  • Data Engineer

    6 days ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology-driven company is seeking a skilled Data Engineer to design, build, and optimize data pipelines for real-time and batch processing. If you're passionate about big data, cloud technologies, and scalable solutions, this is your chance to make an impactQualifications:Bachelors Degree in Computer Science, Computer Engineering, or related...

  • Data Engineer(AWS)

    2 days ago


    Johannesburg, Gauteng, South Africa Falcorp Resourcing Full time

    Reference: JHB000028-MM-2Falcorp Technologies is seeking a mature and meticulous Data Analyst who will be responsible for retrieving and gathering data, organizing it, and using it to reach meaningful conclusions. We're looking for a Data Analyst who has strong SQL experience; telco experience will be advantageous.Duties & ResponsibilitiesExtract, translate,...

  • Data Engineer

    5 days ago


    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    A leading technology-driven company is seeking a skilled Data Engineer to design, build, and optimize data pipelines for real-time and batch processing. If you're passionate about big data, cloud technologies, and scalable solutions, this is your chance to make an impactQualifications: Bachelors Degree in Computer Science, Computer Engineering, or a related...