AWS Data Engineer

4 weeks ago

Cape Town, South Africa PBT Group Full time

PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.

The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data quality standards, and enabling downstream users to access data promptly. This position is an integral part of an agile team.

 

These professionals are entrusted with the responsibility of establishing the infrastructure required to derive insights from raw data, integrating data from various sources seamlessly. They empower solutions by efficiently managing substantial volumes of data, both in batch and real-time, utilizing cutting-edge technologies from the realms of big data and cloud computing. Additional responsibilities encompass the development of proof-of-concepts and the implementation of intricate big data solutions, with a primary focus on collecting, parsing, managing, analyzing, and visualizing extensive datasets. They are adept at employing technologies to resolve challenges associated with handling vast amounts of data in diverse formats, thereby delivering innovative solutions.

 

Data Engineering is a technically demanding role that necessitates a broad spectrum of expertise in software development and programming. These professionals possess knowledge in data analysis, understanding end-user and business requirements, and have the ability to translate these needs into technical solutions. They exhibit a strong grasp of physical database design and the systems development lifecycle. Collaboration within a team environment is essential for success in this role.

 

Key Responsibilities:

  • Architecting Data analytics framework.
  • Translating complex functional and technical requirements into detailed architecture, design, and high-performance software.
  • Leading the development of data and batch/real-time analytical solutions by leveraging transformative technologies.
  • Engaging in multiple projects as a technical lead, overseeing user story analysis, design, software development, testing, and automation tool creation.

 

Duties: Primary Job Objectives:

  • Development and Operations
  • Database Development and Operations
  • Establishment and Adherence to Policies, Standards, and Procedures
  • Communication
  • Business Continuity and Disaster Recovery Planning
  • Research and Evaluation
  • Coaching and Mentoring

 

Required Skills, Knowledge, and Experience:

  • A minimum of 5 years of experience in Data Engineering or Software Engineering.
  • Demonstrated leadership experience, managing teams of engineers for 3-5 years.
  • A minimum of 2 years of experience in Big Data.
  • At least 5 years of experience with Extract, Transform, and Load (ETL) processes.
  • A minimum of 2 years of experience with AWS (Amazon Web Services).
  • Demonstrated experience with agile or other rapid application development methodologies for at least 2 years (e.g., Agile, Kanban, Scrum).
  • 5 years of proven expertise in object-oriented design, coding, testing patterns, and working with commercial or open-source software platforms and large-scale data infrastructures.
  • Proficiency in creating data feeds from on-premise to AWS Cloud (2 years).
  • Support experience for data feeds in production on a break-fix basis (2 years).
  • A minimum of 4 years of experience in creating data marts using Talend or similar ETL development tools.
  • Proficiency in data manipulation using Python and PySpark (2 years).
  • Experience in processing data using the Hadoop paradigm, particularly with EMR, AWS's distribution of Hadoop (2 years).
  • DevOps experience in Big Data and Business Intelligence, including automated testing and deployment (2 years).
  • Extensive knowledge of various programming or scripting languages.
  • Expertise in data modeling and an understanding of different data structures and their suitability for specific use cases.

 

Additional Technical Skills Required:

  • The ability to design highly scalable distributed systems using various open-source tools.
  • Proficiency in both batch and streaming Big Data tools.
  • Experience with Talend for at least 1 year.
  • Familiarity with AWS services such as EMR, EC2, and S3 for at least 1 year.
  • Proficiency in Python for at least 1 year.
  • Familiarity with PySpark or Spark (desirable for at least 1 year).
  • Experience in Business Intelligence data modeling for 3 years.
  • Proficiency in SQL for 3 years.

 

Desirable:

  • 5+ years of experience in Retail Operations.

 

Qualifications/Certifications:

  • A Bachelor's degree in computer science, computer engineering, or equivalent work experience for a minimum of 4 years.
  • AWS Certification, at least at the associate level.

 

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • Junior AWS Engineer

    4 weeks ago

    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingResponsibilities Develop reusable and reliable automated scripts and make them part of continuous integration / build systems Deploy, scale and automate AWS services across multiple regions. Participate in the research and development of the automated and replicable "infrastructure as a code" approaches. Tune system parameters and...

  • AWS Engineer

    4 weeks ago

    Cape Town, South Africa IOCO Full time

    We are seeking a skilled AWS Mid/Senior Level Engineer to join our dynamic team. The AWS Engineer will play a key role in designing, implementing, and managing AWS cloud solutions, ensuring high performance, scalability, and security.6 Month contract - renewableHybrid roleMatric / Tertiary qualification preferable.AWS CertificationMin 5 years...

  • AWS Engineer

    2 weeks ago

    Cape Town, South Africa IOCO Full time

    We are seeking a skilled AWS Mid/Senior Level Engineer to join our dynamic team. The AWS Engineer will play a key role in designing, implementing, and managing AWS cloud solutions, ensuring high performance, scalability, and security. 6 Month contract - renewable Hybrid role Matric / Tertiary qualification preferable. AWS Certification Min 5 years...

  • AWS Architect

    4 weeks ago

    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of the corporate culture? Are you passionate about technology and want to work on cutting edge infrastructure? We are looking forward to hearing from you!As an AWS Architect, you'll be responsible for designing and architecting the services that will power our cloud based applications. You'll also be responsible for ensuring...

  • AWS Cloud Engineer

    4 weeks ago

    Cape Town, South Africa Parvana Full time

    Client Details: Our client works with multiple organisations to find a better way to transact. They bring together committed people dedicated to delivering innovative enterprise solutions that help their customers contribute to economic growth. As a vibrant and innovative SaaS company, they deliver industry-leading expertise and technology to solve real...

  • DevOps Data Engineer

    4 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Strong experience with AWS, Terraform, CI/CD, GitDesign, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB,...

  • DevOps Data Engineer

    2 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer.   Duties: Strong experience with AWS, Terraform, CI/CD, Git Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB,...

  • DevOps Data Engineer

    4 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Strong experience with AWS, Terraform, CI/CD, GitDesign, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB,...

  • DevOps Data Engineer

    4 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake,...

  • DevOps Data Engineer

    2 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer.   Duties: Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake, Python,....

  • Data Engineer

    4 weeks ago

    Cape Town, South Africa Dimension Data Full time

    As a Data Engineer, you'll design and build the data architecture of our products. You will use your technical knowledge to help us develop new products by taking an idea from concept to reality. You'll work on projects across multiple teams in order to achieve company goals and objectives. Your role will involve: - Analyzing business requirements; -...

  • Aws Administrator Senior

    2 weeks ago

    Cape Town, South Africa Network Contracting Full time

    **Required**: A numerate honours degree (e.g. Computer Science, Mathematics, Engineering). Ability to develop technical solutions and functional specifications based on business requirements. Knowledge of leading technologies including AWS certifications such as AWS Certified Solutions Architect, AWS Certified DevOps Engineer, or AWS Certified SysOps...

  • API Cloud Engineer

    4 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is seeking an AWS-experienced API Cloud Engineer to fill a promising opportunity. Skills and experience required include building Cloud-based Solutions (AWS, Azure, GCP), expertise in C#/.NET 6.0 and REST APIs, communication skills, and the ability to interpret architectural designs and produce solutions that will aid data migration and topic...

  • Intermediate DevOps Engineer

    4 weeks ago

    Cape Town, South Africa Black Swan Data Full time

    Summary: We are seeking an experienced and passionate DevOps Engineer with a strong background in Kubernetes, AWS, and Linux to join our team. As an Intermediate DevOps Engineer, you will play a critical role in designing, implementing, and maintaining our cloud infrastructure, with a focus on Kubernetes-based container orchestration, AWS services, and...

  • Data Streaming Engineer

    4 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for a Data Streaming Engineer to build, enhance, and maintain real-time data pipeline. Work with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining data ingestion pipelines vital to the...

  • Senior Cloud Engineer

    4 weeks ago

    Cape Town, South Africa Progressive Edge Full time

    Senior Cloud Engineer (AWS)Location: Winelands / Cape TownA flourishing fintech and payments enterprise is rapidly expanding through innovative initiatives and state-of-the-art solutions in the payment processing sector. These innovations aim to enhance payment solutions with automation and cutting-edge technologies, spanning across e-commerce to advanced...

  • Senior Cloud Engineer

    4 weeks ago

    Cape Town, South Africa Progressive Edge Full time

    Senior Cloud Engineer (AWS)Location: Winelands / Cape TownA flourishing fintech and payments enterprise is rapidly expanding through innovative initiatives and state-of-the-art solutions in the payment processing sector. These innovations aim to enhance payment solutions with automation and cutting-edge technologies, spanning across e-commerce to advanced...

  • AWS Solutions Architect

    2 weeks ago

    Cape Town, South Africa PikUniq Full time

    Attention: Please note that the hiring team responsible for this position will be using the PikUniq platform for candidate screening and conducting one-way interviews. Each application received will be carefully evaluated and screened. Further information on how to get started will be provided. We are looking forward to receiving your...

  • IT Data Engineer

    4 weeks ago

    Cape Town, South Africa Armstrong Appointments Full time

    Our client based in Cape Town are currently seeking the skills of an IT Data Engineer to assist in designing and implementing scalable and robust processes to support the data engineering capability. This role will be responsible for extracting and transforming massive amounts of data at scale and consolidating this data into a bigger data ecosystem. Main...

  • Senior Cloud Engineer

    4 weeks ago

    Western Cape, Cape Town, South Africa Progressive Edge Full time

    Progressive Edge’s job vacancy, Career and Recruitment Job title : Senior Cloud Engineer (AWS) – Cape Town jobs in Western Cape Job Location : Western Cape, Cape Town Deadline : April 20, 2024 Quick Recommended Links Jobs by Location Job by industries Role Overview We are in search of an accomplished Senior Cloud Engineer with expertise in AWS...