Data Engineering Specialist

2 days ago

Johannesburg, South Africa Express Employment Professionals Full time

Support data driven solutions by leveraging your software and data engineering experience across a wide set of use cases within the company Group and throughout its customer base. In this role, you will become a member of company Strategic Insights (TSI), a new team within company Group’s Strategic Division; focused on maturing company’s data landscape by building data driven solutions for large-scale problems. To succeed, you should be highly skilled in batch and real-time data ingestion, and data transformation with the ability to design, build and scale data pipelines for efficient data integration and processing. TSI is one of the biggest data orientated teams in the country, with over 50 data professionals spread over 4 smaller teams: Data Management, Data Insights, AI/ML, and Engineering. This team structure enables us to achieve a holistic balance between start-up and corporate culture

**Responsibilities**:
**Data Engineering**:

- Architect, develop and maintain a declarative data integration solution with the ability to move data between databases, file systems and other locations while tracking data quality according to measures set by data owner.
- Enhance our big data and real-time data capabilities by using both open-source and internally developed tools, designed to efficiently move data while ensuring integrity.
- Oversee data support for specialized data centric services we provide, including ETL/ELT, data modelling, database design, data quality assessments, data format conversion, data transformation and data tool selection.
- Maintain up-to-date knowledge of data engineering tools, related technologies, data security and data privacy laws.
- Guide standards and best practices across the data stack

**Processes, Automation, and DevOps**:

- Contribute to the development and maintenance of our toolsets used to automate various jobs and scheduled tasks.
- Identify systemic inefficiencies, conceptualise possible solutions and drive their development.
- Ensure consistent documentation of all implemented tools, systems, and processes.

**Business and Leadership**:

- Use data engineering techniques to innovate and solve problems, translating business requirements into system designs.
- Engage with stakeholders to support the design and delivery of data science projects and solutions.
- Lead and develop a team of junior data engineers.
- Contribute to our agile way of work and innovation culture.

**Required Technical Skills & Experience**:

- Any data or cloud platform (GCP, Azure, AWS) certification is required, other relevant certifications will be highly advantageous.
- A formal qualification of at least NQF level 6 in Computer Science, Mathematics, Statistics, Software and/or Data Engineering or a related field. Any relevant specialised certifications or a post-graduate degree will be especially advantageous.
- 3-5 years relevant experience, of which at least 2 years must have been in a data engineering environment.
- Extensive knowledge and experience with data engineering and integration concepts and terms.
- Strong relational and non-relational database foundational knowledge.
- Able to create and maintain robust, automated data pipelines and DAG workflows.
- Experienced with analysis and improvement of business and software systems and processes.
- Deep knowledge and understanding of best practices, standards and deployment options enabling data transfer in batches or real time, on-prem and in the cloud.
- Cloud platform management (GCP, Azure, AWS, etc.)
- Experience with Linux, Python 3, Java, Scala, SQL, JavaScript, and tools in our stack including: PySpark, Hadoop, Cloudera, Kafka, Airflow, Argo, Prefect, Dagster, Redis, RabbitMQ, Beam, Arrow, Arrow Flight, Iceberg, Dremio, Ab Initio, Tableau, Alteryx, PowerBI, GitLab, Terraform, GCP, PowerPoint, Word & Excel.
- Experience with PySpark and other big data technologies
- Conscious of the various laws and privacy concerns surrounding personal and corporate data.
- Experience with task and workflow automation, DAGs, and relevant tooling.
- Experience with OOP and automated testing in Python.
- Experience with Google Cloud Platform and its products.
- Comfortable with mono-repo architecture and the relevant tooling.
- Experience with git workflows and interfaces.
- Experience with semantic versioning
- Communication (written and verbal)
- Stakeholder management
- Skilled planner
- Takes initiative
- Growth mindset
- Can do attitude
- Systems orientated
- Innovative thinker
- Works well in teams
- Works well under pressure
- Relishes dynamic/changing environment
- Hard working and conscientious
- Highly skilled in multitasking and context switching
- Excellent time management
- Can solve difficult problems
Strives to achieve high quality output

COVID-19 considerations:
Always remember to wear a face mask, keep a safe social distance and always sanitize when visiting any of our offices

Ability to commute/relocate:

- Johannesburg, G

  • Specialist Data Engineer

    1 month ago

    Johannesburg, South Africa Salix Recruitment Full time

    Become part of a proudly local organisation as a Specialist Data Engineer, with over 100 years of market influence. Our client is looking for a specialist where your work will directly contribute to shaping the future of their financial landscape. As part of a dynamic team, youll play a key role in optimizing data systems that drive business success across...

  • Specialist Data Engineer

    1 month ago

    Johannesburg, South Africa Salix Recruitment Full time

    Become part of a proudly local organisation as a Specialist Data Engineer, with over 100 years of market influence. Our client is looking for a specialist where your work will directly contribute to shaping the future of their financial landscape. As part of a dynamic team, youll play a key role in optimizing data systems that drive business success across...

  • Data Engineering Specialist

    3 weeks ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    As a Data Engineering Specialist at E-Merge, you will work with our team of experts to design and implement robust data engineering solutions. We are looking for a skilled professional with a strong background in Azure Cloud.You will be responsible for analyzing and organizing raw data, designing scalable data pipelines, and building clean, transformed, and...

  • Data Engineering Specialist

    3 weeks ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    Job DescriptionAt E-Merge, we are seeking an experienced Data Engineering Specialist to join our team. In this role, you will be responsible for designing and implementing state-of-the-art cloud, hybrid, and on-prem solutions.Key ResponsibilitiesData Wrangling: Wrangle big data from multiple sources into a reliable asset aimed at evolution instead of...

  • Data Engineering Specialist

    4 weeks ago

    Johannesburg, Gauteng, South Africa Network Recruitment Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at Network Recruitment. This is a unique opportunity to design, build, and maintain cutting-edge data pipelines, databases, and infrastructure.Key Responsibilities:Design and develop scalable data architectures that enable high-quality data processing and analytics.Collaborate with...

  • Data Engineering Specialist

    3 weeks ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    Data Development Engineer RoleWe're seeking an experienced Data Development Engineer to join our team at E-Merge.The successful candidate will be responsible for designing, developing and implementing ETL pipelines using cloud-based data storage solutions like AWS and Azure.Key Responsibilities:Design and develop ETL pipelines using cloud-based data storage...

  • Data Engineer

    1 month ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines, databases, and infrastructure that enable high-quality data processing and analytics. You will work with cross-functional teams, including Data Scientists, Analysts, and IT Specialists, to ensure seamless data flow from various sources into our data...

  • Data Engineer and Insights Specialist

    2 weeks ago

    Johannesburg, Gauteng, South Africa Sci Tech Placements Full time

    Job DescriptionWe are seeking a highly skilled Data Scientist/Specialist to join our team at Sci Tech Placements.Key Responsibilities:Design, validate, and enhance machine learning modelsEngineer and extract valuable insights from complex data structuresPredict future outcomes based on current trends in dataQualifications:A minimum of 6 years' experience in...

  • ▷ [High Salary] Specialist Data Engineer

    1 month ago

    Johannesburg, South Africa Jobted ZA C2 Full time

    Become part of a proudly local organisation as a Specialist Data Engineer, with over 100 years of market influence. Our client is looking for a specialist where your work will directly contribute to shaping the future of their financial landscape. As part of a dynamic team, youll play a key role in optimizing data systems that drive business success across...

  • Data Engineer

    1 month ago

    Johannesburg, South Africa Network Recruitment Full time

    As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines, databases, and infrastructure that enable high-quality data processing and analytics. You will work with cross-functional teams, including Data Scientists, Analysts, and IT Specialists, to ensure seamless data flow from various sources into our data...

  • Data Engineer

    1 month ago

    Johannesburg, South Africa Network Recruitment Full time

    As a Data Engineer, you will be responsible for designing, building, and maintaining data pipelines, databases, and infrastructure that enable high-quality data processing and analytics. You will work with cross-functional teams, including Data Scientists, Analysts, and IT Specialists, to ensure seamless data flow from various sources into our data...

  • Data Lake Engineer Specialist

    3 weeks ago

    Johannesburg, Gauteng, South Africa People Source Full time

    About the RoleAs a Data Lake Engineer Specialist, you will be responsible for ensuring the seamless integration of data from various sources into our IoT Platform Solution. This role requires strong technical skills, attention to detail, and excellent problem-solving abilities.You will collaborate with cross-functional teams to design and develop data...

  • Cloud-Based Data Engineer

    3 weeks ago

    Johannesburg, Gauteng, South Africa E-Merge Full time

    Data Engineer - Cloud Platform Specialist Job DescriptionWe are seeking a highly skilled Data Engineer - Cloud Platform Specialist to join our team at E-Merge. As a key member of our development team, you will be responsible for building and enhancing our cloud-based platform, focusing on data systems and pipelines.The ideal candidate will have experience...

  • Data Specialist

    5 days ago

    Johannesburg, South Africa Afrique Connection Consultancy Full time

    The Data Specialist will work collectively with the SI-MER team to provide expertise in the field of data processing and engineering. The role will lead and implement data engineering projects, support, and maintain data pipelines, and provide expertise and best practices regarding data engineering for staff across the organisation. As needed, the Data...

  • Cloud Infrastructure Specialist

    2 weeks ago

    Johannesburg, Gauteng, South Africa Data Centrix Full time

    About Data CentrixData Centrix is a leading organization in the field of cloud computing, striving to provide cutting-edge solutions for businesses worldwide.Salary and BenefitsThe successful candidate will receive an attractive salary ranging from $140,000 to $200,000 per annum, commensurate with experience, plus a comprehensive benefits package including...

  • Storage and Backup Specialist

    1 week ago

    Johannesburg, South Africa Data Centrix Full time

    **Qualifications and Experience** - Grade 12 - OEM (Dell) Accreditation in Storage and Back-up in some of these certs: - Associate - Information Storage and Management Version 5.0 - Dell Unity XT Unified Storage Solution - Dell EMC Integrated Data Protection Appliance - Dell EMC PowerStore - Technology Architect, Midrange Storage Solutions - Veeam Certified...

  • Senior Specialist: Data Engineering and Analytics

    2 hours ago

    Johannesburg, South Africa Only the Best Full time

    Gauteng , JHB - West (Market related) Our client in the Telecommunications industry has an EE opportunity available for a Senior Specialist: Data Engineering and Analytics to be based in the Johannesburg area. **Requirements**: - 3+ year Degree / Diploma in Computer Science, Engineering, Mathematics, Statistics or related quantitative field -...

  • Data Insights Specialist

    2 weeks ago

    Johannesburg, Gauteng, South Africa Data Centrix Full time

    Data Centrix: Data Analyst PositionWe are seeking a highly skilled Data Analyst to join our team at Data Centrix. The successful candidate will have a strong background in data analysis and be able to extract valuable insights from large datasets.Job Description:In this role, you will be responsible for collecting, cleaning, and organizing data from various...

  • Data Governance Manager

    5 days ago

    Johannesburg, South Africa Dimension Data Full time

    **Data Governance Manager** The primary responsibility of Data Governance Manager is to coordinate, design, develop, direct, and oversee the implementation of the Data Governance Framework and do analysis and presentation on various area to ensure that Governance is adhered to, and Data Quality is positively affected by this. This position will manage the...

  • Data Anayst

    3 weeks ago

    Johannesburg, South Africa Data Centrix Full time

    Qualifications / Knowledge / Experience: Bachelors degree in computer science, Information Systems and/ or; Advanced computer literacy. Financial services knowledge. SDLC methodology knowledge. Minimum 5 years experience in financial services sector. Minimum 5 years experience in IT. Key Responsibilities:Collecting, cleaning, and organizing data from...