Data Engineer

About

At , we believe great work starts with great people. Our culture is built on curiosity, collaboration, and a deep respect for our craft. We operate with integrity, take ownership, and bring a relentless focus to solving meaningful problems.

We don't just build technology, we build trust, teams, and long-term partnerships. From our senior engineers to our newest joiners, we support one another, challenge each other, and celebrate the wins, big and small. It's a place where high standards meet real impact, and where everyone has the space to grow, lead, and do work they're proud of.

We're on the lookout for motivated and experienced Data Engineers across all levels (Juniors, Intermediates, Seniors, Leads, and Principals) to be part of our growing team.

Overview

As a Data Engineer, you will be responsible for designing, developing, and maintaining the data infrastructure, platforms, pipelines, and applications for our clients. You will collaborate with cross-functional teams to ensure efficient data processing, integration, and analysis.

Key Responsibilities

While the list below is long, an ideal candidate should have working knowledge and experience covering many of the tools and services. The requirements for each project differ over time, and these skills provide an overview of what may typically be required of a Data Engineer.

Software Engineering – Fundamentals

• A fundamental software engineering skill set underpins all engineering work at
• Experience with modern operating systems, particularly Linux.
• Experience with version control software, particularly Git.
• Software fundamentals, such as: Problem solving, data structures and algorithms, software development methodologies, common design patterns and best practices.
• Experience with at least one relevant language, preferably more. For example, Python, SQL, NodeJS, Terraform.

Cloud

• Ability to identify serverless options, managed options, and roll-your-own options, strengths and weaknesses.
• Development experience working with Terraform IaC to provision and maintain data infrastructure.
• Familiarity with AWS Well-Architected principles and experience implementing them.

Data Engineering – General

• Working knowledge of Big Data – Volume, Variety, Velocity, etc.

Data Engineering – Collection

• Good experience collecting data in hybrid environments (on-premise to cloud, and cloud to on-premise).
• Real-time: AWS Kinesis Data Streams (KDS), Kafka / MSK.
• Near Real-time: AWS Kinesis Data Firehose (KDF).
• Batch: AWS DataSync, Storage Gateway, Transfer Family (FTP / SFTP / MFT), Snowball.
• Databases: ODBC/JDBC, database replicas and replication tools, migration tools such as Database Migration Service (DMS) and SCT.

Data Engineering – Storage

• Basic experience working with on-premise storage solutions (NFS / SMB, NAS / DAS, etc.).
• Cloud Storage: Amazon S3.
• Data Formats (Parquet, CSV, Avro, JSON etc.), compression, partitioning.
• NoSQL Databases: AWS DynamoDB, MongoDB, etc.
• Relational Databases: AWS RDS or similar, MySQL / PostgreSQL, Aurora.
• Search Databases: AWS Elasticsearch / OpenSearch.
• Caching: Redis / Memcached.

Data Engineering – Processing

• Strong experience developing ETL processes, and integrating with source and destination systems.
• Strong experience developing using Python, Spark (e.g. PySpark), and SQL to work with data.
• Basic experience with Lakehouse technologies such as Apache Hudi, Apache Iceberg or Databricks Delta Lake.
• AWS Lambda for file/stream/event processing, ETL, and triggers.
• General ETL and cataloging of data, access control: AWS Glue ETL, Glue catalog, Lake Formation.
• Hadoop-like processing: Mainly Spark and Hive, Instance types and cluster and job sizing; AWS Elastic MapReduce (EMR).

Data Engineering – Analysis

• Basic understanding of cloud data warehouse architecture and data integration: AWS Redshift and Redshift spectrum.
• Data modelling skills, normalization, facts and dimensions.
• Experience in data quality.
• On-object-store querying: AWS Athena, Glue Crawlers.

Data Engineering – Security

• Basic experience with authentication and identity federation, authorisation and RBAC pertaining to data.
• Basic knowledge of cloud network security: AWS VPC, VPC endpoints, Subnets, DirectConnect.
• Identity and Access Management: AWS IAM, STS and cross-account access.
• Encryption for data at rest, and data in motion for all services used: AWS KMS / SSE, TLS, etc.

Data Engineering – Operations

• Orchestration of data pipelines: AWS Step Functions, Managed Apache Airflow, Glue, etc.
• Basic knowledge of good architecture pillars and how to apply them:
• Operational excellence, Security, and Reliability.
• Performance Efficiency, Cost Optimization, and Sustainability.

Requirements

• Proven track record of designing and implementing data solutions.
• Knowledge of and experience with AWS Cloud infrastructure and services.
• Bachelor's degree in Engineering, Computer Science, or related field.
• Certifications, such as:
• AWS Solutions Architect Associate
• AWS Data Analytics Specialty
• AWS Database Specialty.
• AWS Data Engineer Associate
• Databricks Certified: Data Engineer Associate
• Databricks Certified: Data Engineer Professional
• Any other data-related experience, e.g. working with Hadoop, databases, analytics software, etc.
• Experience with a second cloud vendor (e.g. both AWS and Azure).
• Experience with Docker/Containers/Kubernetes/CICD pipelines for data.
• Knowledge of data security and compliance standards.
• Understanding of Cloud Security best practices.
• Willingness to learn and expand knowledge related to Cloud and Data Technologies.
• Strong problem-solving and analytical skills.
• Self-organizing with the ability to prioritize and manage multiple tasks simultaneously.
• Excellent verbal and written communication skills.
• Ability to work collaboratively with clients and team members.
• Willingness to travel to clients as and when required.

What We Offer

• A culture of engineering and an environment where ideas are heard, and builders can build.
• Competitive compensation and bonus structure.
• A flexible and supportive work environment that values diversity, work-life balance, and personal growth.
• Opportunities for career advancement and ongoing professional development.
• Ongoing learning and development opportunities to enhance your skills.
• Engaging with cutting-edge technologies and awesome client projects.
• Access to a talented team of professionals and mentors.

*If you have not heard back from us within 30 days, please consider your application unsuccessful. However, we'd love for you to keep an eye out for future opportunities and please continue to apply.