Big Data Architect

2 weeks ago


Cape Town, South Africa Parvana Strategic Sourcing Full time

**Client Details**:
**Role Responsibilities**:
Collecting, storing, processing, and analysing huge sets of data.

Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.

Implementing ETL process.

Monitoring performance and advising any necessary infrastructure changes.

Defining data retention policies.

**Preferred Qualifications**:
Relevant tertiary qualification

**Relevant Skills / Experience**:
Proficient understanding of distributed computing principles

Management of Hadoop cluster, with all included services

Ability to solve any ongoing issues with operating the cluster

Proficiency with Hadoop v2, MapReduce, HDFS (Hadoop Distributed File System)

Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming

Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala

**Experience with the following**:
Integration of data from multiple data sources

NoSQL databases, such as HBase, Cassandra, MongoDB

Various messaging systems, such as Kafka or RabbitMQ

Big Data ML toolkits, such as Mahout, SparkML, or H2O

Cloudera / MapR / Hortonworks

**YARN**: Yet Another Resource Negotiator

**Spark**: In-Memory data processing

**Mahout, Spark MLLib**: Machine Learning algorithm libraries

**Solar, Lucene**: Searching and Indexing

**Zookeeper**: Managing cluster

**Oozie**: Job Scheduling

Knowledge of various ETL techniques and frameworks, such as Flume

Good understanding of Lambda Architecture, along with its advantages and drawbacks


  • Big Data Developer

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa PRR Recruitment Services Full time

     Big Data Developer - Fully remote role. This international software company is looking for a Big Data Developer. The ideal candidate will have strong experience in Scala and Apache Spark will need to write testable functional Scala in Production grade systemsstrong Cloud platform to architect and build data pipelines.  Will need strong data...


  • Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for an experienced Big Data Administrator. Big Data team is responsible for the operational and technical support of the Big Data platform. The platform includes Hadoop (Cloudera), ingestion and streaming (Nifi and Kafka) and is deployed both on premise (currently) and as hybrid cloud platform (planned). Project Deliverables,...

  • Kafka Big Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa PBT Group Full time

    PBT Group has an opportunity for a Kafka Big Data Engineer. As a Kafka Engineer, you will be responsible for the building, improvement and scaling of our streaming data platform. This role requires a combination of strong technical skills, deep understanding of distributed systems as well as excellent communication abilities. Kafka engineer is a big data...

  • Solutions Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    The Solutions Architect will be responsible for architecting solutions that take advantage of our technologies and enabling the team to deploy new software in a quick, reliable and scalable way. Your primary responsibility will be to drive innovation within the company by developing cutting-edge technology solutions. You'll work closely with other...

  • Data Architect

    5 days ago


    Cape Town, South Africa Games Global Full time

    **The Role**: The Data Architect will collaborate with stakeholders & developers on strategic data initiatives and products, ensuring data is optimally and consistently delivered per agreed requirements. The Data Architect also defines and provides direction for the ongoing maintenance, support and optimisation of existing data assets and delivery...

  • AWS Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingAre you tired of the corporate culture? Are you passionate about technology and want to work on cutting edge infrastructure? We are looking forward to hearing from you!As an AWS Architect, you'll be responsible for designing and architecting the services that will power our cloud based applications. You'll also be responsible for ensuring...

  • Solution Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingThe Software Architect is responsible for the design, modification, and integration of software systems to translate business requirements into technology terms. Specifically, this role develops software architecture and frameworks for software system according to the required specifications. They take responsibility of the various aspects...

  • Solutions Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingThe primary responsibility of the Solutions Architect is to consult with the client and work with internal teams to create strategies, transformational designs and the architectural vision for complex solutions that address the clients’ needs. They provide multi-technology consulting services on all aspects of application software,...

  • Data Architect

    4 days ago


    Cape Town, South Africa PBT Group Full time

    Job Title: Data Architect **Responsibilities**: - Lead the design and development of data models and data architecture solutions - Analyze complex data systems and design strategies that optimize data integration and management, minimizing data redundancy and ensuring data consistency and accuracy across multiple systems - Define data governance policies,...

  • Solutions Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    Functieomschrijving1.1.                 ROLE OVERVIEW  The Senior Solution Architect is an industry respected pre-sales, technology and solution focused specialist who creates excellent client relationships with his technical expertise and consultative approach. The Senior Solutions Architect is being viewed as a trusted technical advisor...

  • Solutions Architect Hybrid Networks WAN

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingDo you want to work in a company where your voice counts? Do you want to help create new and innovative products for our customers? Do you want to be part of an innovative team that is constantly improving its processes and delivery? Apply for this Solutions Architect Hybrid Networks WAN (CPT) role!The Senior Solution Architect is an...

  • Senior Big Data Engineer III

    Found in: Talent ZA C2 - 2 weeks ago


    Cape Town, South Africa Impact tech Inc Full time

    The Company :  At impact.com we are passionate about our people, our technology, and are obsessed with customer success. Working together enables us to grow rapidly, win, and serve the largest brands in the world. We use cutting edge technology to solve real-world problems for our clients and continue to pull ahead of the pack as the leading SaaS...

  • Technical Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa Dimension Data Full time

    FunctieomschrijvingDo you want to be a part of one of the best technology companies in the region? Do you have a passion for solving complex technical problems and working with sophisticated hardware? If so, we are looking for you!As a Technical Architect, you'll be responsible for delivering complex projects within the business. Your work will include...

  • Lead Data Architect

    4 days ago


    Cape Town, South Africa Full Circle Resourcing Full time

    **Permanent role, Hybrid working conditions** **Cape Town** **MINIMUM QUALIFICATION** - Relevant formal B-degree qualification - 8-10 years experience in IT systems and analysis **EXPERIENCE REQUIRED** - Experience leading and managing a team of Data Engineers. - Architect **cloud solutions** for data integration and storage for analytical and...

  • Senior Data Enterprise Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa DLK Group (PTY) Ltd Full time

    The Ce-I continues to provide Enterprise Architect strategic planning, innovation, and good governance support services to the province.Ce-I follows the approach below in order to achieve its strategic goal enablement (top down), this approach should be followed by the successful bidder in delivery the services:•                 ...


  • Cape Town, South Africa PBT Group Full time

    PBT is seeking a highly skilled and motivated Data Management Specialist to join our dynamic team. Your role could also be seen as a Data Steward. DMS or DS is knowledgeable about the organisation's data landscape, business processes, and data requirements. They possess a deep understanding of data governance principles, data management best practices, and...

  • Solutions Architect

    4 days ago


    Cape Town, South Africa WePlace Full time

    An industry leading Bank has an exciting opportunity available for a Solutions Architect to join their dynamic team. **Purpose Statement** Let me tell you a bit about yourself. You are a tech-savvy individual who understands the importance of staying ahead of the game when it comes to technology. You know that DevOps culture is the key to solving problems...

  • Senior Data Enterprise Architect

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa DLK Group Full time

    Job Description The Ce-I continues to provide Enterprise Architectstrategic planning, innovation, and good governance support services to theprovince. Ce-I follows the approach below in order toachieve its strategic goal enablement (top down), this approach should befollowed by the successful bidder in delivery the services: •Understanding the...

  • DevOps Data Engineer

    Found in: Talent ZA C2 - 3 weeks ago


    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake,...

  • DevOps Data Engineer

    Found in: Whatjobs ZA C2 - 7 days ago


    Cape Town, South Africa PBT Group Full time

    PBT Group has a requirement for an intermediate DevOps Data Engineer.   Duties: Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Glue, Step-functions, Kafka CC, PySpark, DynamoDB, Delta.io, RedShift, Lambda, DeltaLake, Python,....