AWS Data Engineer
3 weeks ago
PBT Group is currently offering an opportunity for an AWS Data Engineer with 4+ years of relevant experience.
The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources, ensuring its conformity to data quality standards, and enabling downstream users to access data promptly. This position is an integral part of an agile team.
These professionals are entrusted with the responsibility of establishing the infrastructure required to derive insights from raw data, integrating data from various sources seamlessly. They empower solutions by efficiently managing substantial volumes of data, both in batch and real-time, utilizing cutting-edge technologies from the realms of big data and cloud computing. Additional responsibilities encompass the development of proof-of-concepts and the implementation of intricate big data solutions, with a primary focus on collecting, parsing, managing, analyzing, and visualizing extensive datasets. They are adept at employing technologies to resolve challenges associated with handling vast amounts of data in diverse formats, thereby delivering innovative solutions.
Data Engineering is a technically demanding role that necessitates a broad spectrum of expertise in software development and programming. These professionals possess knowledge in data analysis, understanding end-user and business requirements, and have the ability to translate these needs into technical solutions. They exhibit a strong grasp of physical database design and the systems development lifecycle. Collaboration within a team environment is essential for success in this role.
Key Responsibilities:
- Architecting Data analytics framework.
- Translating complex functional and technical requirements into detailed architecture, design, and high-performance software.
- Leading the development of data and batch/real-time analytical solutions by leveraging transformative technologies.
- Engaging in multiple projects as a technical lead, overseeing user story analysis, design, software development, testing, and automation tool creation.
Duties: Primary Job Objectives:
- Development and Operations
- Database Development and Operations
- Establishment and Adherence to Policies, Standards, and Procedures
- Communication
- Business Continuity and Disaster Recovery Planning
- Research and Evaluation
- Coaching and Mentoring
Required Skills, Knowledge, and Experience:
- A minimum of 5 years of experience in Data Engineering or Software Engineering.
- Demonstrated leadership experience, managing teams of engineers for 3-5 years.
- A minimum of 2 years of experience in Big Data.
- At least 5 years of experience with Extract, Transform, and Load (ETL) processes.
- A minimum of 2 years of experience with AWS (Amazon Web Services).
- Demonstrated experience with agile or other rapid application development methodologies for at least 2 years (e.g., Agile, Kanban, Scrum).
- 5 years of proven expertise in object-oriented design, coding, testing patterns, and working with commercial or open-source software platforms and large-scale data infrastructures.
- Proficiency in creating data feeds from on-premise to AWS Cloud (2 years).
- Support experience for data feeds in production on a break-fix basis (2 years).
- A minimum of 4 years of experience in creating data marts using Talend or similar ETL development tools.
- Proficiency in data manipulation using Python and PySpark (2 years).
- Experience in processing data using the Hadoop paradigm, particularly with EMR, AWS's distribution of Hadoop (2 years).
- DevOps experience in Big Data and Business Intelligence, including automated testing and deployment (2 years).
- Extensive knowledge of various programming or scripting languages.
- Expertise in data modeling and an understanding of different data structures and their suitability for specific use cases.
Additional Technical Skills Required:
- The ability to design highly scalable distributed systems using various open-source tools.
- Proficiency in both batch and streaming Big Data tools.
- Experience with Talend for at least 1 year.
- Familiarity with AWS services such as EMR, EC2, and S3 for at least 1 year.
- Proficiency in Python for at least 1 year.
- Familiarity with PySpark or Spark (desirable for at least 1 year).
- Experience in Business Intelligence data modeling for 3 years.
- Proficiency in SQL for 3 years.
Qualifications/Certifications:
- A Bachelor's degree in computer science, computer engineering, or equivalent work experience for a minimum of 4 years.
- AWS Certification, at least at the associate level.
* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent
-
AWS Data Engineer
3 weeks ago
Cape Town, South Africa PBT Group Full timePBT Group is currently offering an opportunity for an AWS Data Engineer with 4+ years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and security. Data Engineers play a pivotal role in facilitating the acquisition of data from diverse sources,...
-
Data Engineer for AWS Services
3 days ago
Cape Town, Western Cape, South Africa ENGINEERINGUK Full timeAbout the PositionWe are seeking a highly skilled Cloud Support Engineer to join our team at ENGINEERINGUK. The successful candidate will have extensive experience in supporting customers who utilize AWS services for ETL workloads and data analysis.Main ResponsibilitiesDesign and implement ETL pipelines for extracting and ingesting data into data warehouses...
-
AWS Professional
2 days ago
Cape Town, Western Cape, South Africa Xcede Group Full timeData Engineer - Cloud ArchitectureXcede Group is looking for a highly skilled Data Engineer - Cloud Architecture to design and implement scalable and reliable cloud-based data processing systems.You will be responsible for leading the development of our cloud architecture, collaborating with cross-functional teams to drive business outcomes, and staying...
-
Solutions Architect Intern, Aws
2 weeks ago
Cape Town, South Africa AWS EMEA SARL (South Africa) - H01 Full timeCurrently in your final year of a degree in Computer Science / Engineering / Mathematics / Technology / Related science/technical field. - Interest and aptitude to learn about and deliver cloud-based solutions to customers. - Knowledge of one of the programming languages like Java, Python, Ruby, Node.js, C#, or C++ OR the interest and technical ability to...
-
Data Center Engineering Operations Internship
3 weeks ago
Cape Town, South Africa Amazon Data Services South Africa (Pty) Ltd - E35 Full timeIn pursuit of Bachelor’s Degree or Diploma in Mechanical, Electrical, or Electronic Engineering, or has already graduated in those fields. - Fluent English **This is a 12-month full-time internship role based in Cape Town. Start date ASAP** At Amazon Web Services (AWS), we are working to be the most customer-centric company on earth. To get there, we need...
-
Data Engineer Leader
3 days ago
Cape Town, Western Cape, South Africa Xcede Group Full timeSenior Data EngineerXcede Group, a leading organization in the Financial Services industry, is seeking a highly skilled and experienced Senior Data Engineer with extensive AWS knowledge to join their team in Cape Town.This exciting opportunity involves working on innovative projects in a fast-paced and dynamic environment. As a Senior Data Engineer, you will...
-
AWS Engineer Position
8 hours ago
Cape Town, Western Cape, South Africa Amazon Full timeA day in the life of a Cloud Support Engineer at AWS is filled with exciting challenges and opportunities. You will work on solving customer cases through various customer contact channels, including telephone, email, and web/live chat. As a member of our team, you will have the chance to learn and use groundbreaking technologies, interact with top-notch...
-
Data Engineering Lead
2 weeks ago
Cape Town, South Africa Progressive Edge Full time**Data Engineering Lead** **Permanent** **Cape Town** A large established enterprise is looking to employ a Technical Lead : Data Engineering with deep technical knowledge the ability to lead and mentor a team of Data engineers. You will require substantial expertise in a broad range of software development & programming fields. Your knowledge of data...
-
Data Engineering
2 weeks ago
Cape Town, South Africa PBT Group Full time**Duties & Responsibilities** **Operational Delivery** - Build, enhance, and maintain real-time data pipelines. - Work with infrastructure and operations teams to maintain data infrastructure. - Identify, document, and resolve feature gaps while implementing solutions. - Modernize and maintain services and tooling to ensure resilience, fix data...
-
Data Engineer
3 weeks ago
Cape Town, South Africa PBT Group Full time? Join PBT Group as a Senior Data Engineer ! ? Are you a dynamic data engineer passionate about optimising large-scale data ecosystems? PBT Group is looking for a Data Engineer to design and implement robust, scalable processes that support our growing data engineering capability. This role is pivotal in advancing our cloud engineering and data...
-
Data Engineer
3 weeks ago
Cape Town, South Africa PBT Group Full time? Join PBT Group as a Senior Data Engineer! ?Are you a dynamic data engineer passionate about optimising large-scale data ecosystems? PBT Group is looking for a Data Engineer to design and implement robust, scalable processes that support our growing data engineering capability. This role is pivotal in advancing our cloud engineering and data management...
-
Cloud Data Engineering Expert
3 days ago
Cape Town, Western Cape, South Africa Woolworths Full timeWe are seeking a Cloud Data Engineering Expert to lead our data engineering efforts and drive innovation in cloud-based data solutions.Cloud Data Engineering ObjectivesSupport the organisation's cloud strategy and alignment to data architecture and data governance.Design and implement data pipelines using AWS services.Engineer data in appropriate formats for...
-
Aws Administrator Senior
2 weeks ago
Cape Town, South Africa Network Contracting Full time**Required**: A numerate honours degree (e.g. Computer Science, Mathematics, Engineering). Ability to develop technical solutions and functional specifications based on business requirements. Knowledge of leading technologies including AWS certifications such as AWS Certified Solutions Architect, AWS Certified DevOps Engineer, or AWS Certified SysOps...
-
Data Engineer Position
3 days ago
Cape Town, Western Cape, South Africa Xcede Group Full timeJob DescriptionWe are seeking a Senior Data Engineer to join our team at Xcede Group. The successful candidate will have extensive experience working with AWS technologies, including AWS Glue, AWS EMR, and AWS S3.In this role, you will have the opportunity to work on exciting and innovative projects, utilizing your technical skills and expertise to deliver...
-
Aws Solutions Architect
1 week ago
Cape Town, South Africa DataTech Recruitment Full timeIn this role, you'll be a key player in: - Guiding Clients to the Cloud: Conduct Migration Readiness Assessments to ensure smooth transitions to AWS. - Optimizing Existing Solutions: Lead Well-Architected Reviews (WARs) for existing workloads, maximizing performance and efficiency. - Collaborative Problem-Solving: Partner with clients through technical...
-
Software Development Engineer, AWS Security
6 days ago
Cape Town, South Africa Amazon Full timeSoftware Development Engineer, AWS Security Job ID: 2800704 | Amazon Data Services France SAS - F17 Join us to drive high-impact innovation that secures our cloud by building solutions that enable an ecosystem of services to protect against sophisticated threats. The Informatics team owns the security telemetry mission in AWS. We collect, enrich and vend...
-
Cape Town, Western Cape, South Africa Amazon Full timeThe AWS Infrastructure Sustainability Tech team is seeking a passionate, results-oriented Engineer to own the delivery of customer-facing sustainability products.About the TeamWe support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely...
-
Aws Solutions Architect
1 week ago
Cape Town, South Africa DataTechRecruit Full timeAWS Solutions Architect Architect Cape Town - Western Cape - South Africa In this role, you'll be a key player in: - Guiding Clients to the Cloud: Conduct Migration Readiness Assessments to ensure smooth transitions to AWS. - Optimizing Existing Solutions: Lead Well-Architected Reviews (WARs) for existing workloads, maximizing performance and...
-
Cape Town, South Africa Amazon Data Services South Africa (Pty) Ltd Full time2+ years of computer hardware troubleshooting and repair experience - 1+ years of computer networking experience - High school or equivalent diploma We are looking for Data Center Technicians to join our expanding Infrastructure Operations team. Data Center Technicians are the primary resolver and troubleshoot and solve a range of datacenter infrastructure...
-
Data Engineering
2 weeks ago
Cape Town, South Africa PBT Group Full timeThe Data Engineering & Streaming Lead will be responsible for building, enhancing, and maintaining real-time data pipelines. The role involves working closely with various infrastructure and operations teams to maintain data infrastructure. As a senior engineer, the candidate will develop and maintain data ingestion pipelines, ensuring the bank's continued...