See more Collapse

Intermediate Data Engineer-

3 months ago

Sandton, South Africa Dimension Data Full time
Functieomschrijving

Intermediate Data Engineer, Data Processing and Archiving, SQL, Big Data Analytics, Big Data This is a great opportunity to work with an industry leading company in the finance sector. The role will be based at our headquarters in London. The ideal candidate will have strong experience of working on large scale data processing projects within the financial services sector. You'll need to be comfortable using Python and Java programming languages for both developing new products as well as supporting existing ones through enhancements or upgrades. Experience of MongoDB is also desirable but not essential. You should ideally have gained substantial experience building scalable solutions that leverage Apache Hadoop & Spark technology for managing massive volumes of data across different platforms (e.g., AWS). This includes your own original

Requirements

  • Extensive understanding and application of Python Development and building complex solutions and applications.
  • Extensive understanding and application of Python and data processing and transformation
  • Strong understanding of Microsoft SQL Server and T-SQL Development
  • Strong Understanding of Code Optimization
  • Understanding of Real-Time Data Processing and Streaming
  • Understanding of Apache Beam
  • Advantageous
  • Understanding of Apache NiFi and building complex workflows.
  • Understanding of opensource streaming technologies (Apache Spark Streaming, Flink)
  • Understanding of Apache Kafka
  • Understanding of Apache Spark
  • Understanding of Apache Hadoop
  • Competency in Python and developing Spark models.
  • Fundamental understanding of MongoDB
  • Fundamental understanding of Redis
  • Fundamental understanding of Databricks

Fundamental understanding and optimisation of Linux and cloud environments.

Key Performance Areas:

  • Effectively conceptualise, design and create high quality, custom workflows and analytics solutions.
  • Develop, test, and implement big data solution designs.
  • Understand client requirements and establish knowledge of data for accurate design, analysis and
  • retrieval.
  • Pull data from various data sources and combine it to store it in a datastore for analysis and
  • retrieval.
  • Collaborating with end users on standardized and best practice approaches.
  • Making suggestions and enhancements on existing solutions.
  • Providing regular and timely feedback to clients on design and build status
  • Educating requestors as to appropriate and desirable parameters to ensure they get the information
  • they need
  • Ensure all tasks are updated on agile boards in a timely manner
  • Assist Project Managers and Change and Training Managers with any project and training related
  • administration tasks
  • Actively Upskill in relevant technologies as prescribed by team leadership
  • Integration and execution with Machine-learning models and AI in-flow.
  • Documentation and Design of Solutions

Job Related Experience:

  • Minimum 4+ years work experience with exposure to data pipeline development and solutions
  • architecture as well as project management/coordination experience.

Behavioural Competencies:

  • Good Communication Skills
  • Good Presentation Skills
  • Good Adaptability
  • Must take Initiative
  • Good at Planning and organising
  • Good at Teamwork
  • Good at Influencing
  • Good at Problem Solving
  • Must have Attention to Detail
  • Must be good at Analytical Thinking
  • Must have a desire for Innovation
  • Must be able to Conceptualise ideas

Qualifications

B Degree – Computer Science / Engineering

Hortonworks Certified

We have other current jobs related to this field that you can find below

  • Intermediate BI Support Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking for a way to make a difference in your career and that of your team? Do you want to work in an environment where people care about their careers, and where they take time out to help you with your development? We believe we're looking at the right person for this role.As an Intermediate BI Support Engineer, you'll be part...

  • Intermediate Data and Analytics solution specialist

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingDo you want to get out of your current role and take on a new challenge? Do you want to be part of a fast-paced, innovative company that values your input? We believe this Intermediate Data and Analytics solution specialist was created for you! Please read through and apply!As an Intermediate Data and Analytics solution specialist, you...

  • Data Engineer

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingStandard Bank Anglo is looking for a Data Engineer. This will be strictly remote work. RequirementsAdvanced knowledge of T24 architecture and Data from a MOD perspective Analyze and organize raw data Build data systems and pipelinesEvaluate business needs and objectivesInterpret trends and patternsConduct complex data analysis and report...

  • Network Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking to work in a small, but highly motivated team? Do you want to be part of a culture where innovation and creativity are encouraged at every level? Please read through the job description!As a Network Engineer, you'll be responsible for supporting and maintaining the network infrastructure. You will help to ensure that the...

  • Intermediate Project Manager

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    As an intermediate project manager, you'll help manage projects of all types, ensuring that they're completed on time and within budget. You will have a broad understanding of all aspects of the software development lifecycle (SDLC) and how to effectively utilize different tools for each stage. You'll also be able to troubleshoot problems in order to ensure...

  • Intermediate Security Automation Engineer

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingDo you want to work in a fast-paced environment, where your skills and knowledge are constantly being challenged? Do you have an interest in system administration and networking? Are you looking for a challenging role that allows you to build systems with security in mind? Then apply nowAs an Intermediate Security Automation Engineer,...

  • Intermediate Security Automation Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingDo you want to work in a fast-paced environment, where your skills and knowledge are constantly being challenged? Do you have an interest in system administration and networking? Are you looking for a challenging role that allows you to build systems with security in mind? Then apply now!As an Intermediate Security Automation Engineer,...

  • Intermediate Project Manager

    3 days ago

    Sandton, South Africa Dimension Data Full time

    As an intermediate project manager, you'll help manage projects of all types, ensuring that they're completed on time and within budget. You will have a broad understanding of all aspects of the software development lifecycle (SDLC) and how to effectively utilize different tools for each stage. You'll also be able to troubleshoot problems in order to ensure...

  • Data Engineer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingStandard Bank Anglo is looking for a Data Engineer. This will be strictly remote work. Requirements Advanced knowledge of T24 architecture and Data from a MOD perspective Analyze and organize raw data Build data systems and pipelinesEvaluate business needs and objectivesInterpret trends and patternsConduct complex data analysis and report...

  • Intermediate Developer

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingThe Intermediate Frontend Developer is responsible for creating client specific solutions by developing, implementing, and maintaining applications. They will have the ability to design and customise applications for specific client use with the aim of optimising operational efficiency. This includes being responsible for developing and...

  • Intermediate Developer

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingThe Intermediate Frontend Developer is responsible for creating client specific solutions by developing, implementing, and maintaining applications. They will have the ability to design and customise applications for specific client use with the aim of optimising operational efficiency. This includes being responsible for developing and...

  • Senior Data Engineer-

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAbout Dimension DataAt Dimension Data we empowering our people to deliver client-centric solutions. We want to see our people make the most of every opportunity, and together we can achieve the extraordinary. Our diverse and inspiring people are experts at what they do and serious about delivering measurable and sustainable value at every...

  • SCOM Engineer _ L2

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingThe SCOM Engineer (L2) is an intermediate to high level position in the EUC engineer job family and is proactive in identifying, investigating and resolving technical incidents and problems and restoring service to clients by managing incidents to resolution. Their primary objective is to ensure zero missed service level agreement...

  • Intermediate Infrastructure Project Manager

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking for a role that is directly related to your skills, but where you are not being micro managed? Are you looking for an environment where your opinions matter and there is a strong focus on culture? My client instructs me to find the right person for this roleThe role of an Intermediate Infrastructure Project Manager is to...

  • Intermediate Infrastructure Project Manager

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking for a role that is directly related to your skills, but where you are not being micro managed? Are you looking for an environment where your opinions matter and there is a strong focus on culture? My client instructs me to find the right person for this role!The role of an Intermediate Infrastructure Project Manager is to...

  • Data Centre Technician

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingThe Data Center Technician (DCT) responsibility is the maintenance of all Data Centre Facilities and management of 3rd party vendors ensuring a 24/7 service offering within Internet Solutions Data Centers .The DCT provides a service to clients to ensure that their IT infrastructure and systems remain operational through proactively...

  • Data Centre Technician

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingThe Data Center Technician (DCT) responsibility is the maintenance of all Data Centre Facilities and management of 3rd party vendors ensuring a 24/7 service offering within Internet Solutions Data Centers .The DCT provides a service to clients to ensure that their IT infrastructure and systems remain operational through proactively...

  • Senior Data Engineer-

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAbout Dimension DataAt Dimension Data we empowering our people to deliver client-centric solutions. We want to see our people make the most of every opportunity, and together we can achieve the extraordinary. Our diverse and inspiring people are experts at what they do and serious about delivering measurable and sustainable value at every...

  • Intermediate Project Manager-

    2 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking to work in an environment where your opinion is valued? Do you want to work for a company that gives you full autonomy, including the ability to make decisions and have them implemented? We believe this Intermediate Project Manager was created for youAs an Intermediate Project Manager, you'll be responsible for delivering...

  • Intermediate Project Manager-

    4 weeks ago

    Sandton, South Africa Dimension Data Full time

    FunctieomschrijvingAre you looking to work in an environment where your opinion is valued? Do you want to work for a company that gives you full autonomy, including the ability to make decisions and have them implemented? We believe this Intermediate Project Manager was created for you!As an Intermediate Project Manager, you'll be responsible for delivering...