AWS Data Engineer

2 months ago

Johannesburg, South Africa PBT Group Full time

PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.

 

The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data quality standards, and enabling downstream users to access data promptly. This position is an integral part of an agile team.

 

These professionals are entrusted with the responsibility of establishing the infrastructure required to derive insights from raw data, integrating data from various sources seamlessly. They empower solutions by efficiently managing substantial volumes of data, both in batch and real-time, utilizing cutting-edge technologies from the realms of big data and cloud computing. Additional responsibilities encompass the development of proof-of-concepts and the implementation of intricate big data solutions, with a primary focus on collecting, parsing, managing, analyzing, and visualizing extensive datasets. They are adept at employing technologies to resolve challenges associated with handling vast amounts of data in diverse formats, thereby delivering innovative solutions.

 

Data Engineering is a technically demanding role that necessitates a broad spectrum of expertise in software development and programming. These professionals possess knowledge in data analysis, understanding end-user and business requirements, and have the ability to translate these needs into technical solutions. They exhibit a strong grasp of physical database design and the systems development lifecycle. Collaboration within a team environment is essential for success in this role.

 

Key Responsibilities:

  • Architecting Data analytics framework.
  • Translating complex functional and technical requirements into detailed architecture, design, and high-performance software.
  • Leading the development of data and batch/real-time analytical solutions by leveraging transformative technologies.
  • Engaging in multiple projects as a technical lead, overseeing user story analysis, design, software development, testing, and automation tool creation.

 

Duties: Primary Job Objectives:

  • Development and Operations
  • Database Development and Operations
  • Establishment and Adherence to Policies, Standards, and Procedures
  • Communication
  • Business Continuity and Disaster Recovery Planning
  • Research and Evaluation
  • Coaching and Mentoring

 

Required Skills, Knowledge, and Experience:

  • A minimum of 2 years of experience in Data Engineering or Software Engineering.
  • At least 2 years of experience with Extract, Transform, and Load (ETL) processes.
  • A minimum of 2 years of experience with AWS (Amazon Web Services) and Databricks.
  • Demonstrated experience with agile or other rapid application development methodologies for at least 2 years (e.g., Agile, Kanban, Scrum).
  • 2 years of proven expertise in object-oriented design, coding, testing patterns, and working with commercial or open-source software platforms and large-scale data infrastructures.
  • Proficiency in creating data feeds from on-premise to AWS Cloud (2 years).
  • Support experience for data feeds in production on a break-fix basis (2 years).
  • Proficiency in data manipulation using Python and PySpark (2 years).
  • DevOps experience in Big Data and Business Intelligence, including automated testing and deployment (2 years).
  • Extensive knowledge of various programming or scripting languages.

 

Additional Technical Skills Required:

  • The ability to design highly scalable distributed systems using various open-source tools.
  • Proficiency in both batch and streaming Big Data tools.
  • Familiarity with AWS services such as EMR, EC2, and S3 for at least 1 year.
  • Proficiency in Python for at least 1 year.
  • Familiarity with PySpark or Spark (desirable for at least 1 year).
  • Experience in Business Intelligence data modeling for 3 years.
  • Proficiency in SQL for 3 years.

 

Qualifications/Certifications:

  • A Bachelor's degree in computer science, computer engineering, or equivalent work experience for a minimum of 4 years.
  • AWS Certification, at least at the associate level.

 

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

 

  • AWS Data Engineer

    2 months ago

    Johannesburg, South Africa MSP Staffing LTD Full time

    Our client is seeking a highly motivated and experienced AWS Data Engineer to join their Global Markets division. This role offers an exciting opportunity to collaborate with leading financial institutions, contributing to the design and implementation of data pipelines to support global markets business. Key Requirements · Bachelor's degree in computer...

  • AWS Data Engineer

    2 months ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    About E-MergeE-Merge is a pioneering company that harnesses the power of Artificial Intelligence (AI) to deliver actionable insights and drive transformative change for businesses worldwide.We are currently looking for a skilled AWS Data Engineer to join our fast-paced and dynamic team.ResponsibilitiesDevelop and Operationalize Large-Scale Data...

  • AWS Data Engineer

    3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • Senior AWS Data Engineer

    3 weeks ago

    Johannesburg, Gauteng, South Africa MSP Staffing LTD Full time

    Company OverviewMSP Staffing LTD is a leading provider of top talent in the data engineering space. With a focus on innovation and excellence, we strive to deliver exceptional results for our clients.Job DescriptionWe are seeking a highly skilled AWS Data Engineer to join our team. The successful candidate will be responsible for designing and implementing...

  • AWS Data Engineer

    2 months ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse...

  • AWS Data Engineer

    2 months ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse...

  • AWS Data Engineering Specialist

    1 month ago

    Johannesburg, Gauteng, South Africa MSP Staffing LTD Full time

    MSP Staffing LTD is seeking a highly skilled AWS Data Engineer to join their Global Markets division.About the RoleThis exciting opportunity allows you to collaborate with leading financial institutions, contributing to the design and implementation of data pipelines that support global markets business.Key RequirementsBachelor's degree in computer science,...

  • Cloud Data Engineer

    7 months ago

    Johannesburg, South Africa Dimension Data Full time

    The Cloud Data Engineer, AWS Data Pipeline, AWS Storage Gateway and Digital Assets will work on the Amazon Web Services (AWS) infrastructure team. The engineer's primary focus is to provide data services that enhance our clients' businesses. You'll be a key member of the data pipeline team helping us deliver these solutions quickly and efficiently while...

  • Cloud Data Engineer: Designing Scalable Data Pipelines

    2 months ago

    Johannesburg, Gauteng, South Africa Dimension Data Full time

    At Dimension Data, we are seeking an experienced Cloud Data Architect to lead our data engineering solutions. The ideal candidate will have a deep understanding of AWS technologies and experience in designing scalable data pipelines.Key ResponsibilitiesDesign and implement data pipelines using AWS services such as AWS Data Pipeline and AWS Storage...

  • AWS Data Solutions Engineer

    1 day ago

    Johannesburg, Gauteng, South Africa PBT Group Full time

    Job DescriptionWe are seeking an experienced AWS Data Engineer to join our team at PBT Group. The ideal candidate will have a strong background in designing and implementing scalable data pipelines using Amazon Web Services (AWS).The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability,...

  • AWS Data Scientist

    1 day ago

    Johannesburg, Gauteng, South Africa PBT Group Full time

    PBT Group is a leading provider of innovative data solutions. We are currently seeking an experienced AWS Data Engineer to join our team.The role involves designing and developing scalable data pipelines, utilizing cloud-based technologies such as AWS and Databricks. This position requires strong expertise in data engineering, including experience with ETL...

  • AWS Cloud Data Engineer

    8 months ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an AWS Cloud Data Engineer. Duties:Coordinate/Prioritise the work that the Developers and Product OwnerAssist with cross-team projects that involve development.AWS practices are followed with sound security and governance.Coordinate with the cloud Center of Excellence team for any AWS account level security and...

  • AWS Data Engineer

    2 months ago

    Johannesburg, Gauteng, South Africa PBT Group Full time

    Job OverviewPBT Group is seeking an experienced AWS Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines and architectures on AWS.Responsibilities:Design and develop data analytics frameworks using AWS services.Translate complex technical requirements into...

  • AWS Data Architect

    1 month ago

    Johannesburg, Gauteng, South Africa PBT Group Full time

    About PBT GroupPBT Group is a leading organization in the industry, known for its innovative approach to data engineering. We are currently seeking an experienced AWS Data Engineer to join our team.

  • AWS Data Engineer

    3 weeks ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    AWS Data Engineer RoleWe are seeking an experienced AWS Data Engineer to join our team at E-Merge in Johannesburg, South Africa.About the Role:As an AWS Data Engineer, you will be responsible for designing and implementing optimal data models, working with large data sources, and ensuring data quality. You will also be involved in identifying key performance...

  • Aws Infrastructure Architect

    6 months ago

    Johannesburg, South Africa NTT Data Full time

    **Job Requirements **NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we...

  • Aws Solutions Architect

    6 months ago

    Johannesburg, South Africa NTT Data Full time

    **Job Requirements **NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we...

  • AWS Cloud Data Architect

    2 months ago

    Johannesburg, Gauteng, South Africa PBT Group Full time

    PBT Group is seeking an exceptional AWS Cloud Data Engineer to join our team. About the Role:Lead the design and implementation of scalable and secure cloud-based data architectures using AWS services.Collaborate with cross-functional teams to ensure data quality, integrity, and compliance with company standards.Develop and maintain ETL processes, data...

  • AWS Cloud Data Engineer – Johannesburg

    1 month ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    Data Engineer Job DescriptionE-Merge, a leading financial services group, seeks an experienced AWS data engineer to join their team in Johannesburg. The ideal candidate will be responsible for designing and implementing optimal data models, working with large data sources, and creating robust data solutions.Key Responsibilities:Designing and Implementing...

  • AWS Infrastructure Engineer

    2 months ago

    Johannesburg, South Africa Africonology Full time

    AWS Infrastructure Engineer Location: Gauteng, Johannesburg Metro, Johannesburg Job Type: Contract, expected to work Full-Time hours Primary Industry: IT and Internet Salary: ZAR0 - ZAR0 Per hour Job Duties: - Designing, implementing, and maintaining AWS infrastructure solutions. - Managing and supporting AWS services, including EC2, S3, RDS, and...