Data Engineer

Found in: Talent ZA C2 - 3 weeks ago

Johannesburg, South Africa Puma Energy Full time

Main Purpose:

Collaborate with data scientists and business stakeholders to design, develop, and maintain efficient data pipelines feeding into the organization's data lake.
Ensure the data lake contains accurate, up-to-date, and high-quality data, enabling data scientists to develop insightful analytics and business stakeholders to make well-informed decisions.
Utilize expertise in data engineering and cloud technologies to contribute to the overall success of the organization by providing the necessary data infrastructure and fostering a data-driven culture.
Demonstrate a strong architectural sense in defining data models, leveraging the Poly-base concept to optimize data storage and access.
Facilitate seamless data integration and management across the organization, ensuring a robust and scalable data architecture.
Take responsibility for defining and designing the data catalogue, effectively modelling all data within the organization, to enable efficient data discovery, access, and management for various stakeholders.

Knowledge Skills and Abilities, Key Responsibilities:

KEY RESPONSIBILTIES:

Design, develop, optimize, and maintain data architecture and pipelines that adhere to ETL principles and business goals. Develop complex queries and solutions using Scala, .NET, Python/PySpark languages. Implement and maintain data solutions on Azure Data Factory, Azure Data Lake, and Databricks Create data products for analytics and data scientist team members to improve their productivity. Advise, consult, mentor, and coach other data and analytic professionals on data standards and practices. Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions. Lead the evaluation, implementation, and deployment of emerging tools and processes for analytic data engineering in order to improve our productivity as a team. Develop and deliver communication and education plans on analytic data engineering capabilities, standards, and processes. Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives. Collaborate with other team members and effectively influence, direct, and monitor project work. Develop strong understanding of the business and support decision making.

REQUIREMENTS:

Experience:

10 years of overall experience & at least 5 years of relevant experience 5 years of experience working with Azure data factory & Databricks in a retail environment

Skills :

Bachelor's degree required; Computer Science Engineering. 5+ years of experience working in data engineering or architecture role. Expertise in SQL and data analysis and experience with at least one programming language (Scala and .NET preferred). Experience developing and maintaining data warehouses in big data solutions. Experience with, Azure Data Lake, Azure Data Factory, and Databricks) in the data and analytics space is a must Database development experience using Hadoop or Big Query and experience with a variety of relational, NoSQL, and cloud data lake technologies. Worked with BI tools such as Tableau, Power BI, Looker, Shiny. Conceptual knowledge of data and analytics, such as dimensional modelling, ETL, reporting tools, data governance, data warehousing, and structured and unstructured data. Big Data Development experience using Hive, Impala, Spark, and familiarity with Kafka. Exposure to machine learning, data science, computer vision, artificial intelligence, statistics, and/or applied mathematics.

Languages:

Fluency in verbal and written English mandatory. Fluency in Spanish & French is useful.

Key Relationships and Department Overview:

Internal: CEO & COO of Africa, Managers across various departments, Senior Management, Head of Departments in other regional hubs of Puma Energy External: External Consultants

  • Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa Network Recruitment Full time

    Our client, a leading company within the Financial Services industry, is seeking a highly skilled Senior Data Engineer to join their dynamic team. In this role, you will play a critical part in designing, developing, and maintaining robust data pipelines, ensuring the availability, reliability, and accessibility of our data assets. Your expertise in data...

  • Data Engineer

    Found in: Adzuna ZA B C2 - 4 days ago

    Johannesburg, South Africa Adzuna ZA B C2 Full time

    Our client, a leading company within the Financial Services industry, is seeking a highly skilled Senior Data Engineer to join their dynamic team. In this role, you will play a critical part in designing, developing, and maintaining robust data pipelines, ensuring the availability, reliability, and accessibility of our data assets. Your expertise in data...

  • Data Engineer

    Found in: Whatjobs ZA C2 - 7 days ago

    Johannesburg, South Africa Network Recruitment Full time

    Our client, a leading company within the Financial Services industry, is seeking a highly skilled Senior Data Engineer to join their dynamic team. In this role, you will play a critical part in designing, developing, and maintaining robust data pipelines, ensuring the availability, reliability, and accessibility of our data assets. Your expertise in data...

  • Data Center Technician

    1 day ago

    Johannesburg, South Africa Dimension Data Full time

    **Requirements**: - The Data Center Technician (DCT) responsibility is the maintenance of all Data Centre White Space, ensuring a 24/7 service offering within Internet Solutions Data Centers.The DCT provides a service to clients to ensure that their IT infrastructure and systems remain operational through proactively identifying, investigating and resolving...

  • Senior ISP Network Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Dimension Data Full time

    FunctieomschrijvingThe Senior ISP Network Engineer, DSL & Telephone Cable Networks, Data Center (DC), Integrated services provider (ISP), MPLS, ACI, SDA, CORE, IP Networks will be responsible for designing and implementing complex data networks across multiple locations. The role involves working closely with business teams to provide solutions to their...

  • Senior Software Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of being micro-managed? Are you looking for a company where no-one tells you how to do your job? Do you want to work in a team, but not be told what to do and when to do it? If so, we believe this Senior Software Engineer role is perfect for you!As a Senior Software Engineer, you'll be responsible for building and maintaining...

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PRR Recruitment Services Full time

    This well known company is looking for a Data Engineer. will be based in Rivonia The ideal candidate will have strong experience in the following: Data cleaning SQL ExportBig data Sets BSc Degree in Maths and Stats will be required a min of 3 years experience. Stable track record is required. Send your Cv to marinda@prrrecruitment.co.za

  • Data Visualisation Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources.  Duties:A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Visualisation Engineer

    Found in: Whatjobs ZA C2 - 7 days ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources.     Duties: A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Visualisation Engineer

    Found in: Adzuna ZA B C2 - 2 weeks ago

    Johannesburg, South Africa Adzuna ZA B C2 Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources. Duties:A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Visualisation Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Data Visualisation Engineer. The visualisation specialist will be responsible for building customer (internal) facing visualisation and virtualisation assets from the data marts designed by the hybrid resources.  Duties:A Data Visualisation Engineer is responsible for presenting data in a manner that...

  • Data Engineer

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa Psybergate Full time

    What you will be doing / What we are looking for:  Data Engineer skill is required to enable the data lifecycle within EDS, namely the ability to integrate data between source system (golden/trusted) and the target database (LOB), hence provide good quality data by applying the necessary framework/s and governance to whomever may require it (BI and...

  • Data Engineer

    Found in: Adzuna ZA B C2 - 2 weeks ago

    Johannesburg, South Africa Adzuna ZA B C2 Full time

    What you will be doing / What we are looking for:Â Data Engineer skill is required to enable the data lifecycle within EDS, namely the ability to integrate data between source system (golden/trusted) and the target database (LOB), hence provide good quality data by applying the necessary framework/s and governance to whomever may require it (BI and...

  • Data Engineer

    Found in: Whatjobs ZA C2 - 7 days ago

    Johannesburg, South Africa Psybergate Full time

    What you will be doing / What we are looking for:  Data Engineer skill is required to enable the data lifecycle within EDS, namely the ability to integrate data between source system (golden/trusted) and the target database (LOB), hence provide good quality data by applying the necessary framework/s and governance to whomever may require it (BI and...

  • Head of Data Engineering

    Found in: Talent ZA 2A C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    Define and lead the execution of the data engineering strategy with the intent to significantly evolve the enterprise data warehouse, ensuring seamless data pipelines and maintain data for availability and usability by the insights division. Lead the transformation of the data human capital to de-risk the environment and ensure adaptability to the changing...

  • Head of Data Engineering

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    Define and lead the execution of the data engineering strategy with the intent to significantly evolve the enterprise data warehouse, ensuring seamless data pipelines and maintain data for availability and usability by the insights division. Lead the transformation of the data human capital to de-risk the environment and ensure adaptability to the changing...

  • Data Streaming Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Golden Rule Full time

    The Role An opportunity exists for a Senior Data Streaming Engineer who will Build, enhance, and maintain our real-time data pipeline. The role will work with various infrastructure and operations teams to maintain our data infrastructure. As a senior engineer on the Data Streaming Platform team, you will be responsible for developing and maintaining...

  • Senior Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Salt Employee Benefits Full time

    Salt Employee Benefits is looking for a savvy Snr Data Engineer to join our IT team. This role requires the incumbent to have  a significant set of technical skills, including a deep knowledge of SQL database design and multiple programming languages. The successful incumbent should have stellar communication skills as she/he will work across departments...

  • Jnr Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa Salt Employee Benefits Full time

    Data is one of the most valuable resources a company owns, and savvy data engineers are required to build an infrastructure to keep it organized. This role requires a significant set of technical skills, including a deep knowledge of SQL and multiple programming languages.  The incumbent should have stellar communication skills as you will work across...

  • Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago

    Johannesburg, South Africa PBT Group Full time

    We currently have numerous vacancies for Data Engineers skilled in tools such as the Microsoft Stack (SSIS, SSRS, SSAS), Power BI, AWS, Azure, DataStage, SAS etc.Duties:Design, develop, test and deploy ETL for ODS and data mart projects, as well as application and management reports.Provide technical support, troubleshooting and upgrade setup or support on...