AWS Data Engineer

6 months ago

Cape Town, South Africa PBT Group Full time

PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.

The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data quality standards, and enabling downstream users to access data promptly. This position is an integral part of an agile team.

 

These professionals are entrusted with the responsibility of establishing the infrastructure required to derive insights from raw data, integrating data from various sources seamlessly. They empower solutions by efficiently managing substantial volumes of data, both in batch and real-time, utilizing cutting-edge technologies from the realms of big data and cloud computing. Additional responsibilities encompass the development of proof-of-concepts and the implementation of intricate big data solutions, with a primary focus on collecting, parsing, managing, analyzing, and visualizing extensive datasets. They are adept at employing technologies to resolve challenges associated with handling vast amounts of data in diverse formats, thereby delivering innovative solutions.

 

Data Engineering is a technically demanding role that necessitates a broad spectrum of expertise in software development and programming. These professionals possess knowledge in data analysis, understanding end-user and business requirements, and have the ability to translate these needs into technical solutions. They exhibit a strong grasp of physical database design and the systems development lifecycle. Collaboration within a team environment is essential for success in this role.

 

Key Responsibilities:

  • Architecting Data analytics framework.
  • Translating complex functional and technical requirements into detailed architecture, design, and high-performance software.
  • Leading the development of data and batch/real-time analytical solutions by leveraging transformative technologies.
  • Engaging in multiple projects as a technical lead, overseeing user story analysis, design, software development, testing, and automation tool creation.

 

Duties: Primary Job Objectives:

  • Development and Operations
  • Database Development and Operations
  • Establishment and Adherence to Policies, Standards, and Procedures
  • Communication
  • Business Continuity and Disaster Recovery Planning
  • Research and Evaluation
  • Coaching and Mentoring

 

Required Skills, Knowledge, and Experience:

  • A minimum of 5 years of experience in Data Engineering or Software Engineering.
  • Demonstrated leadership experience, managing teams of engineers for 3-5 years.
  • A minimum of 2 years of experience in Big Data.
  • At least 5 years of experience with Extract, Transform, and Load (ETL) processes.
  • A minimum of 2 years of experience with AWS (Amazon Web Services).
  • Demonstrated experience with agile or other rapid application development methodologies for at least 2 years (e.g., Agile, Kanban, Scrum).
  • 5 years of proven expertise in object-oriented design, coding, testing patterns, and working with commercial or open-source software platforms and large-scale data infrastructures.
  • Proficiency in creating data feeds from on-premise to AWS Cloud (2 years).
  • Support experience for data feeds in production on a break-fix basis (2 years).
  • A minimum of 4 years of experience in creating data marts using Talend or similar ETL development tools.
  • Proficiency in data manipulation using Python and PySpark (2 years).
  • Experience in processing data using the Hadoop paradigm, particularly with EMR, AWS's distribution of Hadoop (2 years).
  • DevOps experience in Big Data and Business Intelligence, including automated testing and deployment (2 years).
  • Extensive knowledge of various programming or scripting languages.
  • Expertise in data modeling and an understanding of different data structures and their suitability for specific use cases.

 

Additional Technical Skills Required:

  • The ability to design highly scalable distributed systems using various open-source tools.
  • Proficiency in both batch and streaming Big Data tools.
  • Experience with Talend for at least 1 year.
  • Familiarity with AWS services such as EMR, EC2, and S3 for at least 1 year.
  • Proficiency in Python for at least 1 year.
  • Familiarity with PySpark or Spark (desirable for at least 1 year).
  • Experience in Business Intelligence data modeling for 3 years.
  • Proficiency in SQL for 3 years.

 

Desirable:

  • 5+ years of experience in Retail Operations.

 

Qualifications/Certifications:

  • A Bachelor's degree in computer science, computer engineering, or equivalent work experience for a minimum of 4 years.
  • AWS Certification, at least at the associate level.

 

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • AWS Data Engineer

    1 week ago

    Cape Town, South Africa Salt South Africa Full time

    Job title : AWS Data Engineer Job Location : Western Cape, Cape Town Deadline : January 04, 2025 Quick Recommended Links Jobs by Location Job by industries Job Summary : We are seeking a skilled and motivated AWS Data Engineer to join a leading retailer in Cape Town. This mid-level role offers the opportunity to work on exciting...

  • AWS Data Engineer

    2 weeks ago

    Cape Town, South Africa Salt South Africa Full time

    Job title : AWS Data EngineerJob Location : Western Cape, Cape TownDeadline : January 02, 2025Quick Recommended Links Jobs by Location Job by industries Key Responsibilities: Design, develop, and maintain data pipelines and ETL processes using AWS tools and services. Work closely with analytics teams to support business intelligence and reporting...

  • AWS Data Engineer

    6 months ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • AWS Data Engineer

    6 months ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • AWS Cloud Data Engineer

    2 months ago

    Cape Town, South Africa IQbusiness South Africa Full time

    Introduction We are seeking a talented AWS Cloud Data Engineer to join our dynamic team. As a Cloud Data Engineer, you will play a crucial role in creating scalable and secure data solutions for our clients. To be successful in this role, it is necessary to have a deep understanding of how to store, process, and analyze massive amounts of data, as well as...

  • AWS Data Engineer

    1 week ago

    Cape Town City Centre, South Africa Jobted ZA C2 Full time

    PBT Group is currently offering an opportunity for an AWS Data Engineer with 4+ years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...

  • AWS Data Engineering Specialist

    1 week ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group, a leading provider of innovative solutions, is currently seeking an experienced AWS Data Engineer to join their team. This role offers a competitive salary of $120,000 - $180,000 per annum.Job Description:The successful candidate will be responsible for designing and implementing scalable data pipelines, ensuring seamless data integration and...

  • Junior AWS Engineer

    6 months ago

    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingResponsibilities Develop reusable and reliable automated scripts and make them part of continuous integration / build systems Deploy, scale and automate AWS services across multiple regions. Participate in the research and development of the automated and replicable "infrastructure as a code" approaches. Tune system parameters and...

  • AWS Data Architect

    1 month ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    About the RolePBT Group is seeking a highly skilled AWS Data Engineer to join our team. As a Data Engineer, you will play a crucial role in designing, building, and maintaining large-scale data pipelines and systems on Amazon Web Services (AWS).Key ResponsibilitiesData EngineeringDesign and develop scalable data architectures using AWS services such as S3,...

  • AWS Cloud Data Engineer with DevOps

    2 weeks ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is looking for a motivated and talentedDataOps Engineerto join an industry-leading IT company specialising in financial services who primary focus on AWS, to provide scalable, secure, and efficient data-driven solutions.As a DataOps Engineer you will be responsible for designing, developing, and maintaining our cloud-based data platforms on AWS....

  • AWS Systems Engineer

    1 month ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Job Title: AWS Systems Engineer - Cloud InfrastructureAbout the Role:As an AWS Systems Engineer - Cloud Infrastructure, you will be responsible for designing, deploying, and managing scalable and secure cloud infrastructure on AWS. You will work closely with cross-functional teams to ensure the smooth operation of our cloud infrastructure and services.Key...

  • AWS Data Solutions Architect

    1 week ago

    Cape Town, Western Cape, South Africa Salt South Africa Full time

    Job Title: AWS Data Solutions ArchitectA career-defining opportunity has arisen for a skilled and motivated AWS Data Engineer to join Salt South Africa, a leading retailer in Cape Town. This mid-level role offers the chance to work on exciting analytics-driven projects within a fast-paced environment.Key Responsibilities:Design, develop, and maintain...

  • Solutions Architect Intern, Aws

    6 months ago

    Cape Town, South Africa AWS EMEA SARL (South Africa) - H01 Full time

    Currently in your final year of a degree in Computer Science / Engineering / Mathematics / Technology / Related science/technical field. - Interest and aptitude to learn about and deliver cloud-based solutions to customers. - Knowledge of one of the programming languages like Java, Python, Ruby, Node.js, C#, or C++ OR the interest and technical ability to...

  • AWS Architect

    6 months ago

    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of the corporate culture? Are you passionate about technology and want to work on cutting edge infrastructure? We are looking forward to hearing from you!As an AWS Architect, you'll be responsible for designing and architecting the services that will power our cloud based applications. You'll also be responsible for ensuring...

  • AWS Cloud DataOps Engineer

    4 weeks ago

    Cape Town City Centre, South Africa Jobted ZA C2 Full time

    PBT Group is looking for a motivated and talented DataOps Engineer to join an industry-leading IT company specialising in financial services who primary focus on AWS, to provide scalable, secure, and efficient data-driven solutions. As a DataOps Engineer you will be responsible for designing, developing, and maintaining our cloud-based data platforms on...

  • AWS Cloud DataOps Engineer

    1 month ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is looking for a motivated and talented DataOps Engineer to join an industry-leading IT company specialising in financial services who primary focus on AWS, to provide scalable, secure, and efficient data-driven solutions.As a DataOps Engineer you will be responsible for designing, developing, and maintaining our cloud-based data platforms on AWS....

  • AWS Cloud DataOps Engineer

    1 month ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is looking for a motivated and talented DataOps Engineer to join an industry-leading IT company specialising in financial services who primary focus on AWS, to provide scalable, secure, and efficient data-driven solutions. As a DataOps Engineer you will be responsible for designing, developing, and maintaining our cloud-based data platforms on...

  • AWS Cloud DataOps Engineer

    1 month ago

    Cape Town, South Africa PBT Group Full time

    PBT Group is looking for a motivated and talented DataOps Engineer to join an industry-leading IT company specialising in financial services who primary focus on AWS, to provide scalable, secure, and efficient data-driven solutions.As a DataOps Engineer you will be responsible for designing, developing, and maintaining our cloud-based data platforms on AWS....

  • Data Center Electrical Engineer

    5 months ago

    Cape Town, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    As an Amazon Field Engineer, you will provide full life-cycle support to AWS Data Centers from design inception through site improvement and maintenance. You will be the ‘go to’ engineering resource for your region when technical advice is needed, and will use your subject matter expertise and engage with diverse teams to:• Perform design and equipment...

  • Data Center Engineering Operations Internship

    1 month ago

    Cape Town, South Africa Amazon Data Services South Africa (Pty) Ltd - E35 Full time

    In pursuit of Bachelor’s Degree or Diploma in Mechanical, Electrical, or Electronic Engineering, or has already graduated in those fields. - Fluent English **This is a 12-month full-time internship role based in Cape Town. Start date ASAP** At Amazon Web Services (AWS), we are working to be the most customer-centric company on earth. To get there, we need...