AWS Data Engineer

1 week ago

Cape Town, Western Cape, South Africa PBT Group Full time

PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience.

The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data quality standards, and enabling downstream users to access data promptly. This position is an integral part of an agile team.

These professionals are entrusted with the responsibility of establishing the infrastructure required to derive insights from raw data, integrating data from various sources seamlessly. They empower solutions by efficiently managing substantial volumes of data, both in batch and real-time, utilizing cutting-edge technologies from the realms of big data and cloud computing. Additional responsibilities encompass the development of proof-of-concepts and the implementation of intricate big data solutions, with a primary focus on collecting, parsing, managing, analyzing, and visualizing extensive datasets. They are adept at employing technologies to resolve challenges associated with handling vast amounts of data in diverse formats, thereby delivering innovative solutions.

Data Engineering is a technically demanding role that necessitates a broad spectrum of expertise in software development and programming. These professionals possess knowledge in data analysis, understanding end-user and business requirements, and have the ability to translate these needs into technical solutions. They exhibit a strong grasp of physical database design and the systems development lifecycle. Collaboration within a team environment is essential for success in this role.

Key Responsibilities:

  • Architecting Data analytics framework.
  • Translating complex functional and technical requirements into detailed architecture, design, and high-performance software.
  • Leading the development of data and batch/real-time analytical solutions by leveraging transformative technologies.
  • Engaging in multiple projects as a technical lead, overseeing user story analysis, design, software development, testing, and automation tool creation.

Duties: Primary Job Objectives:

  • Development and Operations
  • Database Development and Operations
  • Establishment and Adherence to Policies, Standards, and Procedures
  • Communication
  • Business Continuity and Disaster Recovery Planning
  • Research and Evaluation
  • Coaching and Mentoring

Required Skills, Knowledge, and Experience:

  • A minimum of 5 years of experience in Data Engineering or Software Engineering.
  • Demonstrated leadership experience, managing teams of engineers for 3-5 years.
  • A minimum of 2 years of experience in Big Data.
  • At least 5 years of experience with Extract, Transform, and Load (ETL) processes.
  • A minimum of 2 years of experience with AWS (Amazon Web Services).
  • Demonstrated experience with agile or other rapid application development methodologies for at least 2 years (e.g., Agile, Kanban, Scrum).
  • 5 years of proven expertise in object-oriented design, coding, testing patterns, and working with commercial or open-source software platforms and large-scale data infrastructures.
  • Proficiency in creating data feeds from on-premise to AWS Cloud (2 years).
  • Support experience for data feeds in production on a break-fix basis (2 years).
  • A minimum of 4 years of experience in creating data marts using Talend or similar ETL development tools.
  • Proficiency in data manipulation using Python and PySpark (2 years).
  • Experience in processing data using the Hadoop paradigm, particularly with EMR, AWS's distribution of Hadoop (2 years).
  • DevOps experience in Big Data and Business Intelligence, including automated testing and deployment (2 years).
  • Extensive knowledge of various programming or scripting languages.
  • Expertise in data modeling and an understanding of different data structures and their suitability for specific use cases.

Additional Technical Skills Required:

  • The ability to design highly scalable distributed systems using various open-source tools.
  • Proficiency in both batch and streaming Big Data tools.
  • Experience with Talend for at least 1 year.
  • Familiarity with AWS services such as EMR, EC2, and S3 for at least 1 year.
  • Proficiency in Python for at least 1 year.
  • Familiarity with PySpark or Spark (desirable for at least 1 year).
  • Experience in Business Intelligence data modeling for 3 years.
  • Proficiency in SQL for 3 years.

Desirable:

  • 5+ years of experience in Retail Operations.

Qualifications/Certifications:

  • A Bachelor's degree in computer science, computer engineering, or equivalent work experience for a minimum of 4 years.
  • AWS Certification, at least at the associate level.

* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent

  • AWS Architect

    1 week ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of the corporate culture? Are you passionate about technology and want to work on cutting edge infrastructure? We are looking forward to hearing from youAs an AWS Architect, you'll be responsible for designing and architecting the services that will power our cloud based applications. You'll also be responsible for ensuring...

  • Solutions Architect Intern, Aws

    1 week ago

    Cape Town, Western Cape, South Africa AWS EMEA SARL (South Africa) - H01 Full time

    Currently in your final year of a degree in Computer Science / Engineering / Mathematics / Technology / Related science/technical field. Interest and aptitude to learn about and deliver cloudbased solutions to customers. Knowledge of one of the programming languages like Java, Python, Ruby, , C#, or C++ OR the interest and technical ability to learn a...

  • Senior Data Center Electrical Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    As an Amazon Field Engineer, you will provide full life-cycle support to AWS Data Centers from design inception through site improvement and maintenance.You will be the 'go to' engineering resource for your region when technical advice is needed, and will use your subject matter expertise and engage with diverse teams to:Perform design and equipment...

  • Data Center Engineering Operations Internship

    1 week ago

    Cape Town, Western Cape, South Africa Amazon Data Services South Africa (Pty) Ltd - E35 Full time

    In pursuit of Bachelor's Degree in IT, Mechanical, Electrical, or Electronic Engineering, or has already graduated in those fields. Fluent EnglishRoles starting from June 2024 in South Africa.At Amazon Web Services (AWS), we are working to be the most customer-centric company on earth. To get there, we need exceptionally talented, bright, and driven people....

  • Senior Data Center Electrical Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Amazon Data Services South Africa (Pty) Ltd Full time

    As an Amazon Field Engineer, you will provide full life-cycle support to AWS Data Centers from design inception through site improvement and maintenance.You will be the 'go to' engineering resource for your region when technical advice is needed, and will use your subject matter expertise and engage with diverse teams to:Perform design and equipment...

  • 2024 Aws Sales Intern

    1 week ago

    Cape Town, Western Cape, South Africa AWS EMEA SARL (South Africa) - H01 Full time

    Undergraduate currently in their final year enrolled in an accredited University/College pursuing a Bachelor's Degree or higher in IT, Information Systems, Computer Science, Engineering, or related technical field.This internship will start in June 2024 and will run for 12 monthsAmazon Web Services is innovating new ways of building massively scalable...

  • Data Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Constructeducation Full time

    Location:Cape Town Reporting Line: Engineering Lead (AU)Working Practices: Minimum two days a week in the office with option to work remotely for remaining three days Employment type: Permanent Working hours: 07H00 TO 15H30 Salary: Competitive Role summary The?Data and Cloud Engineer?is responsible for supporting the evolution of the technology solution and...

  • AWS Developer

    1 week ago

    Cape Town, Western Cape, South Africa Level-Up Full time

    Overview Seeking an experienced AWS Developer to join the Dev Team and assist in the migration of our infrastructure from DigitalOcean to AWS. The ideal candidate will have a strong background in AWS services and infrastructure management, with a focus on scalability, reliability, and security. Our client is a dynamic and innovative company specializing in...

  • DevOps Data Engineer

    1 week ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, , RedShift, Lambda, DeltaLake, Python,.Analyze,...

  • Data Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Construct Education Full time

    Location: Cape TownReporting Line: Engineering Lead (AU)Working Practices: Minimum two days a week in the office with option to work remotely for remaining three daysEmployment type: PermanentWorking hours: 07H00 TO 15H30Salary: CompetitiveRole summaryThe Data and Cloud Engineer is responsible for supporting the evolution of the technology solution and...

  • Business Intelligence Engineer, AWS Support BI

    1 week ago

    Cape Town, Western Cape, South Africa Amazon Development Centre (South Africa) (Proprietary) Limited - A78 Full time

    AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success....

  • Business Intelligence Engineer, Aws Support Bi

    1 week ago

    Cape Town, Western Cape, South Africa Amazon Full time

    Business Intelligence Engineer, AWS Support BI & AnalyticsAWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes...

  • Intermediate DevOps Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Black Swan Data Full time

    Summary: We are seeking an experienced and passionate DevOps Engineer with a strong background in Kubernetes, AWS, and Linux to join our team. As an Intermediate DevOps Engineer, you will play a critical role in designing, implementing, and maintaining our cloud infrastructure, with a focus on Kubernetes-based container orchestration, AWS services, and...

  • Aws Cloud Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Electrum Full time

    About Electrum Electrum is an exciting B2B tech company. We partner with some of South Africa's biggest household names, enabling them to open up their customers' access to payments and digital goods and services. We love that the projects we work on touch the lives of millions of South Africans every day, making a real difference.We hire the best of the...

  • Aws Solutions Architect

    1 week ago

    Cape Town, Western Cape, South Africa DataTechRecruit Full time

    AWS Solutions Architect ArchitectCape Town - Western Cape - South AfricaIn this role, you'll be a key player in:Guiding Clients to the Cloud: Conduct Migration Readiness Assessments to ensure smooth transitions to AWS. Optimizing Existing Solutions: Lead Well-Architected Reviews (WARs) for existing workloads, maximizing performance and efficiency....

  • Aws Solutions Architect

    1 week ago

    Cape Town, Western Cape, South Africa DataTech Recruitment Full time

    In this role, you'll be a key player in:Guiding Clients to the Cloud: Conduct Migration Readiness Assessments to ensure smooth transitions to AWS.- Optimizing Existing Solutions: Lead Well-Architected Reviews (WARs) for existing workloads, maximizing performance and efficiency.- Collaborative Problem-Solving: Partner with clients through technical discovery...

  • AWS Solutions Architect

    1 week ago

    Cape Town, Western Cape, South Africa DataTech Recruitment Full time

    Calling all AWS enthusiasts Design, architect, and deliver secure, efficient cloud solutions as a Senior AWS Solutions Architect remote within South Africa. Collaborate with clients, lead Well-Architected Reviews, and guide migrations. AWS Solutions Architect Professional + 5+ years' experience required with 2+ years specifically in AWS Solutions...

  • AWS Cloud Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Electrum Payments Full time

    About ElectrumElectrum is an exciting B2B tech company. We partner with some of South Africa's biggest household names, enabling them to open up their customers' access to payments and digital goods and services. We love that the projects we work on touch the lives of millions of South Africans every day, making a real difference.We hire the best of the best...

  • AWS Solutions Architect

    1 week ago

    Cape Town, Western Cape, South Africa DataTech Recruitment Full time

    Calling all AWS enthusiasts Design, architect, and deliver secure, efficient cloud solutions as a Senior AWS Solutions Architect remote within South Africa. Collaborate with clients, lead Well-Architected Reviews, and guide migrations. AWS Solutions Architect Professional + 5+ years' experience required with 2+ years specifically in AWS Solutions...

  • Data Engineer

    1 week ago

    Cape Town, Western Cape, South Africa Creative CFO Full time

    Become part of a vibrant, quality-focused team that leverages trust and autonomy to deliver exceptional services to diverse, high-growth clients. Receive recognition for your committed, results-producing approach to problem-solving, and opportunities for learning to realise your own passion for personal growth. All while working with some of the country's...