AWS Data Engineer EXP 0675

3 weeks ago


Midrand, South Africa Open Source (Pty) Ltd Full time
ESSENTIAL SKILLS REQUIREMENTS:
Above average experience/understanding (in order of importance):
e.g. Technical Skills / Technology
  • Terraform
  • Python 3x
  • SQL - Oracle/PostgreSQL
  • Py Spark
  • Boto3
  • ETL
  • Docker
  • Linux / Unix
  • Big Data
  • Powershell / Bash
  • BMW Cloud Data Hub (CDH)
  • BMW CDEC Blueprint

· Experience in working with Enterprise Collaboration tools such as Confluence, JIRA etc.
· Experience developing technical documentation and artefacts.
· Knowledge of data formats such as Parquet, AVRO, JSON, XML, CSV etc.
· Experience working with Data Quality Tools such as Great Expectations.
· Knowledge of the Agile Working Model.
· Any additional responsibilities assigned in the Agile Working Model (AWM) Charter
ADVANTAGEOUS SKILLS REQUIREMENTS:
· Demonstrate expertise in data modelling Oracle SQL.
·
- Exceptional analytical skills analysing large and complex data sets.
·
- Perform thorough testing and data validation to ensure the accuracy of data transformations.

- Strong written and verbal communication skills, with precise documentation.

- Self-driven team player with ability to work independently and multi-task.

- Experience building data pipeline using AWS Glue or Data Pipeline, or similar platforms.
- Familiar with data store such as AWS S3, and AWS RDS or DynamoDB.
- Experience and solid understanding of various software design patterns.

- Experience preparing specifications from which programs will be written, designed, coded, tested and debugged.

- Strong organizational skills. Experience developing and working with REST API's is a bonus.
· Basic experience in Networking and troubleshooting network issues.

Basic experience/understanding of AWS Components (in order of importance):
  • Glue
  • CloudWatch
  • SNS
  • Athena
  • S3
  • Kinesis Streams (Kinesis, Kinesis Firehose)
  • Lambda
  • DynamoDB
  • Step Function
  • Param Store
  • Secrets Manager
  • Code Build/Pipeline
  • CloudFormation
  • Business Intelligence (BI) Experience
  • Technical data modelling and schema design ("not drag and drop")
  • Kafka
· AWS EMR
· Redshift

WHICH QUALIFICATIONS/EXPERIENCE DO WE NEED FOR THE ROLE?
· Relevant IT / Business / Engineering Degree
· Certifications:
· Candidates with one or more of the certifications are preferred.
AWS Certified Cloud Practitioner, AWS Certified SysOps Associate, AWS Certified Developer Associate, AWS Certified Architect Associate, AWS Certified Architect Professional, Hashicorp Certified Terraform Associate
WHAT WILL BE YOUR ROLE AND RESPONSIBILITIES?
e.g. Core Role tasks and responsibilities
Data Engineers are responsible for building and maintaining Big Data Pipelines using Data Platforms. Data Engineers are custodians of data and must ensure that data is shared in line with the information classification requirements on a need-to-know basis.

  • Midrand, South Africa Open Source (Pty) Ltd Full time

    ESSENTIAL SKILLS REQUIREMENTS:Above average experience/understanding (in order of importance):e.g. Technical Skills / TechnologyTerraformPython 3xSQL - Oracle/PostgreSQLPy SparkBoto3ETLDockerLinux / UnixBig DataPowershell / BashBMW Cloud Data Hub (CDH)BMW CDEC Blueprint · Experience in working with Enterprise Collaboration tools such as Confluence,...


  • Midrand, South Africa Slipstream Data Full time

    **Location: Waterfall City, Midrand, Johannesburg** **Salary: R90K - R120K** **Start Date: 1 July 2024** **Hybrid work model - 2 Days per week in office** Key Responsibilities: Main objectives for the position: - Build a competent technical team with the ability to collaborate effectively in solving technical customer issues, learning together, and...


  • Midrand, South Africa Africonology Full time

    Job Title: AWS Data Engineers Required - Project Based - Contractual-to start asap-JHB - HybridLocation: Gauteng, Johannesburg Metro, MidrandJob Type: Contract, expected to work Full-Time hoursPrimary Industry: Banking and FinanceSecondary Industry: Banking and FinanceJob Duties:1. Design, develop, and implement AWS cloud solutions for data engineering...


  • Midrand, South Africa Adzuna ZA B C2 Full time

    Job Title: AWS Data Engineers Required - Project Based - Contractual-to start asap-JHB - HybridLocation: Gauteng, Johannesburg Metro, MidrandJob Type: Contract, expected to work Full-Time hoursPrimary Industry: Banking and FinanceSecondary Industry: Banking and FinanceJob Duties:1. Design, develop, and implement AWS cloud solutions for data engineering...


  • Midrand, South Africa HR Genie Full time

    Job Title: AWS Data Engineers Required - Project Based - Contractual-to start asap-JHB - HybridLocation: Gauteng, Johannesburg Metro, MidrandJob Type: Contract, expected to work Full-Time hoursPrimary Industry: Banking and FinanceSecondary Industry: Banking and FinanceJob Duties:1. Design, develop, and implement AWS cloud solutions for data engineering...


  • Midrand, South Africa E-Merge Full time

    Looking for an opportunity with one of the leading companies in the manufacturing industry in search for an AWS Data Engineer that plays a crucial role in building and maintaining the data infrastructure on AWS, ensuring that organizations can effectively and securely manage and analyze their data. Continuous learning and adaptation to evolving technologies...


  • Midrand–Semi Remote, South Africa E-Merge Full time

    Looking for an opportunity with one of the leading companies in the manufacturing industry in search for an AWS Data Engineer that plays a crucial role in building and maintaining the data infrastructure on AWS, ensuring that organizations can effectively and securely manage and analyze their data. Continuous learning and adaptation to evolving technologies...


  • Midrand, South Africa Vodafone Full time

    **Role purpose**: As a part of AWS Cloud Services team, the purpose of the role is to interact with customers to gather technical requirements, architect customer infrastructure & networks, deliver solutions based on AWS cloud platform and compile solution proposal documentation in the AWS cloud services space as part of the Sales process. **Your...


  • Midrand, South Africa Open Source (Pty) Ltd Full time

    **Essential Skills** - Technical Skills / Technology - Infrastructure as Code (Cloud formation templates or terraform), Configuration Management (e.g., Ansible) - Monitoring and log analytics - AWS Lambda (Python), Cloudwatch, AWS DBs, AWS Integration - GitHub Enterprise & GitHub Actions - Experience with Docker / Kubernetes - Experience with Databases both...


  • Midrand, South Africa Limpopo Personnel Full time

    **Minimum requirements and experience needed for this role**: - IT Degree or Diploma - Several years' experience in software development (Python and Terraform) - Familiarity with API-Management, Microservices Architecture and Container Architecture - AWS Certifications **Role and Responsibilities**: - Managing projects/processes as a Senior DevOps...


  • Midrand, South Africa Liyema Consulting Full time

    Key Responsibilities :- Manage infrastructure, monitor performance, and resolve incidents using AWS and Azure technologies.- Collaborate with internal and external teams to coordinate incident resolution and ensure seamless operations.- Implement and maintain ITIL processes, including Incident, Problem, and Configuration Management.- Actively participate in...


  • Midrand, South Africa Adzuna ZA B C2 Full time

    Key Responsibilities:- Manage infrastructure, monitor performance, and resolve incidents using AWS and Azure technologies.- Collaborate with internal and external teams to coordinate incident resolution and ensure seamless operations.- Implement and maintain ITIL processes, including Incident, Problem, and Configuration Management.- Actively participate in...


  • Midrand, South Africa Liyema Consulting Full time

    Key Responsibilities: - Oversee cloud infrastructure and ensure optimal performance- Coordinate incident resolution and communicate with stakeholders- Drive continuous improvement initiatives to enhance efficiency and reliability- Actively participate in Agile ceremonies and contribute to team successRequired Skills: - Strong proficiency in AWS and Azure...


  • Midrand, South Africa Liyema Consulting Full time

    Key Responsibilities:- Oversee cloud infrastructure and ensure optimal performance- Coordinate incident resolution and communicate with stakeholders- Drive continuous improvement initiatives to enhance efficiency and reliability- Actively participate in Agile ceremonies and contribute to team successRequired Skills:- Strong proficiency in AWS and Azure...


  • Midrand, South Africa Adzuna ZA B C2 Full time

    Key Responsibilities:- Oversee cloud infrastructure and ensure optimal performance- Coordinate incident resolution and communicate with stakeholders- Drive continuous improvement initiatives to enhance efficiency and reliability- Actively participate in Agile ceremonies and contribute to team successRequired Skills:- Strong proficiency in AWS and Azure...

  • Software Engineer

    4 weeks ago


    Midrand, South Africa Lim Personnel Full time

    Software Engineer / Developer (Cloud Data Platform) (Expert)ESSENTIAL SKILLS REQUIREMENTS:Languages:Python application development skillsTechnologies:Experience in working with AWS services (Lambda, EC2, IAM, S3, Step Functions, Fargate, Batch, Code Pipeline)Familiar with Terraform and Infrastructure-as-Code conceptsWorking Mode / Mindset:Agile Development...

  • Data Analyst

    5 days ago


    Midrand, South Africa Ikusasa Recruitment Group Full time

    **Data Scientist** Role Purpose Create and deliver data-driven solutions that add business value through the use of statistical models, machine learning algorithms, data mining and visualisation techniques. Experience & Education Requirements: - Degree in Computer Science, Computer Engineering, Actuarial Science, Statistics, Business Sciences, Business...

  • Data Architect

    4 weeks ago


    Midrand, South Africa Mediro ICT Full time

    Design Solutions, with the help of a technical team of 5 to 8 people, that integrate with an enterprise environment. Recommend best practices for the entire solution. Comply with all technical and business requirements. Infrastructure design Big Data Modelling Assisting Businesses with POCs and business value calculations Lead architecture design and...


  • Midrand, South Africa LINKFIELDS INNOVATIONS (PTY) LTD Full time

    We are seeking a skilled Data Architect to join our team and take ownership of our organization's data architecture. The ideal candidate will have a strong background in designing, implementing, and managing complex data systems. They will work closely with our engineering, data science, and business teams to understand requirements, design solutions, and...


  • Midrand, South Africa LINKFIELDS INNOVATIONS (PTY) LTD Full time

    We are seeking a skilled Data Architect to join our team and take ownership of our organization's data architecture. The ideal candidate will have a strong background in designing, implementing, and managing complex data systems. They will work closely with our engineering, data science, and business teams to understand requirements, design solutions, and...