Big Data Architect

3 weeks ago

Cape Town, Western Cape, South Africa IOCO Full time

About the Role:

We're seeking a seasoned Data Engineer to join our team and contribute to the development of scalable data solutions. As a Data Engineer, you'll design, implement, and maintain data architectures using cutting-edge technologies like Spark, collaborating with a dynamic team to deliver high-quality solutions that meet business needs.

Key Responsibilities:

  • Design and implement scalable data architectures using Spark and other technologies.
  • Evaluate and recommend new technologies and approaches to improve the performance, scalability, and reliability of our software systems.
  • Build and maintain data pipelines and compute tiers, primarily operating on Spark.
  • Provide technical guidance and mentor junior engineers.
  • Collaborate with the team to deliver high-quality solutions that meet business requirements.
  • Ensure that code is well-designed, maintainable, and adheres to best practices and standards.
  • Contribute to the development of engineering practices by working on a scrum-sized team empowered to organize and manage the team's operations.
  • Utilize project development tools like JIRA, Confluence, and GIT.
  • Write Python/SQL scripts to automate ETL processes.
  • Build and maintain operations tools for monitoring, notifications, trending, and analysis.
  • Define, create, test, and execute operations procedures.
  • Assist the DevOps Engineer in automating CI/CD practices.

Requirements:

  • 4 to 8 years of experience in Data Engineering/Development.
  • Strong experience in Python, SQL, and Data Flow.
  • Hands-on experience with Spark development and a background in big data.
  • Experience with Linux is preferred.
  • Experience in AWS (AWS certification is a plus).
  • Solid understanding of data modeling and data architecture design.

Personal Attributes:

  • Proven ability to work creatively and analytically in a problem-solving environment.
  • Confidence to express ideas as part of a team.
  • Excellent communication (written, oral) and interpersonal skills.
  • A self-starter capable of working under pressure.
  • Curious and engaged in continuous learning and professional development.

About IOCO:

At IOCO, we believe anything is possible with modern technology, software, and development expertise. We are continuously pushing the boundaries of innovative solutions across multiple industries using an array of technologies. Our culture of continuous learning will ensure that you have all the opportunities, tools, and support to hone and grow your craft. By joining IOCO, you will have an open invitation to our inspiring developer forums, where you can connect and learn from and with your peers by sharing ideas, experiences, practices, and solutions.

  • Data Architect

    10 hours ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Are you a skilled Data Engineer looking to take your career to the next level?We're seeking an experienced Data Architect to join our team at Dimension Data!About the Role:We're looking for a highly skilled Data Architect to design and build the data architecture of our products. As a key member of our team, you will use your technical knowledge to help us...

  • Data Architect

    3 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Job Title: Data ArchitectNetwork Recruitment is seeking a skilled Data Architect to help us build robust data pipelines and drive data-driven decision-making processes.Key Responsibilities:Design and implement scalable data architectures.Collaborate with cross-functional teams to gather requirements and deliver solutions.Optimize data processing and storage...

  • Senior Data Engineer

    4 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Job Title: Senior Data Engineer - Data ArchitectJob Summary:We are seeking a highly skilled Senior Data Engineer - Data Architect to join our team at Network Recruitment. As a Senior Data Engineer - Data Architect, you will be responsible for designing and implementing efficient data systems, ensuring data quality, and maintaining data infrastructure...

  • Data Architect

    4 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    We are seeking a skilled Data Architect to help us design and implement scalable data architectures. Key Responsibilities:Design and implement data pipelines and architectures.Collaborate with cross-functional teams to gather requirements and deliver solutions.Optimize data processing and storage for efficiency and performance.Ensure data quality and...

  • Big Data Specialist

    5 days ago

    Cape Town, Western Cape, South Africa OfferZen Ltd Full time

    We are seeking a highly skilled Big Data Specialist to join our team at OfferZen Ltd. As a key member of our organization, you will play a critical role in shaping the future of our company.About UsOfferZen Ltd is a leading developer jobs marketplace that reverses the traditional job search process. We connect talented developers with top tech companies in...

  • Big Data Pipeline Developer

    3 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group: Kafka Data EngineerKafka Data Engineer at PBT Group is a critical role that requires a strong understanding of distributed systems and big data technologies. The successful candidate will design, develop, and manage Kafka-based data pipelines, ensuring seamless integration with other big data technologies such as Hadoop, Spark, and...

  • Senior Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team at Network Recruitment. As a Senior Data Engineer - Data Pipeline Architect, you will play a key role in designing and implementing efficient data pipelines for our clients in the financial services sector.Key ResponsibilitiesCollaborate with data scientists and analysts to develop...

  • Big Data Streaming Engineer

    6 days ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    Kafka Engineer Job OverviewThe PBT Group is seeking a skilled Big Data Streaming Engineer to build, improve, and scale our real-time data platform using Kafka.Responsibilities:We are looking for a Big Data Streaming Engineer with expertise in developing and managing Kafka-based data pipelines.This role involves working with other big data technologies such...

  • Product Data Architect

    3 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Network Recruitment is seeking a skilled Product Data Engineer to contribute to our data-driven future. This role offers the opportunity to build efficient data pipelines, manage vast datasets, and architect scalable data solutions in a dynamic environment.Key Responsibilities:Collaborate with data scientists, analysts, and cross-functional teams to gather...

  • Enterprise Data Architect

    4 days ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    At Dimension Data, we are seeking an exceptional Enterprise Data Architect to join our team. We are looking for someone with a strong background in data architecture and experience in designing and implementing data exploitation systems.The successful candidate will be responsible for coordinating and managing changes to the MIS database, streamlining MIS...

  • Data Architect

    1 month ago

    Cape Town, Western Cape, South Africa IOCO Full time

    We are seeking a skilled Data Architect to design and implement scalable data solutions using cutting-edge technologies like Spark. As a Data Architect, you will collaborate with a dynamic team to deliver high-quality solutions that meet business needs.Key Responsibilities:Design and implement scalable data architectures using Spark and other...

  • Kafka Big Data Engineer

    3 weeks ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    PBT Group's Kafka Big Data Engineer OpportunityPBT Group is seeking a highly skilled Kafka Big Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and managing our streaming data platform using Kafka.Main Responsibilities:Design and Develop Data Pipelines: Design, develop, and...

  • Highly Skilled Data Architect

    1 week ago

    Cape Town, Western Cape, South Africa PBT Group Full time

    At PBT Group, we are seeking a highly skilled Data Architect to play a key role in designing and implementing data pipelines that drive our data architecture.The ideal candidate will have strong experience with modern data platforms and streaming services. Key responsibilities include designing, building, and maintaining data pipelines using tools such as...

  • Data Architect

    3 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Data Engineer Job DescriptionAt Network Recruitment, we are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will play a critical role in designing and optimizing data pipelines and architectures that handle large datasets and support multiple business services and teams.We are looking for a talented individual with a strong...

  • Lead Data Architect

    7 days ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Network Recruitment is currently seeking a highly skilled Lead Data Architect to join their team in the financial services sector.About Network RecruitmentWe are a well-established recruitment agency with over 40 years of experience in connecting talented professionals with leading companies across various industries.Job OverviewThe successful candidate will...

  • Data Architect

    3 weeks ago

    Cape Town, Western Cape, South Africa Network Recruitment Full time

    Data Engineer Job DescriptionJob SummaryAs a Data Engineer at Network Recruitment, you will play a pivotal role in expanding and optimizing our clients' data architecture and data pipeline systems. You will work with cutting-edge technology to build efficient data solutions that support cross-functional teams, improve processes, and unlock powerful insights...

  • Solutions Architect

    3 weeks ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    Solution OverviewThe role of Senior Solution Architect at Dimension Data requires a deep understanding of industry trends and technological advancements. As a trusted technical advisor, you will work closely with clients to design and implement business solutions that meet their needs. This position involves pre-sales technical support, analysis of client...

  • Data Engineering Leader

    10 hours ago

    Cape Town, Western Cape, South Africa CodeConnect Staffing (Pty) Ltd Full time

    Company OverviewCodeConnect Staffing (Pty) Ltd is a dynamic and forward-thinking organization, driving innovation and shaping the future of technology in Africa.We are seeking a talented Senior Data Engineer to join our team and contribute to our exciting journey. This opportunity provides you with the chance to work with a company that is making significant...

  • Solutions Architect

    1 month ago

    Cape Town, Western Cape, South Africa Dimension Data Full time

    About the RoleThe Solutions Architect will be responsible for designing and implementing innovative technology solutions that drive business growth and improvement. This role will work closely with the development team to architect solutions that leverage our technologies and enable the team to deploy new software in a quick, reliable, and scalable way.Key...

  • Strategic Data Architect

    10 hours ago

    Cape Town, Western Cape, South Africa DLK Group Full time

    Job Title: Strategic Data ArchitectCompany Overview:The DLK Group is seeking an experienced Strategic Data Architect to join our team. This role will play a critical part in shaping the company's data strategy and ensuring its implementation across various departments.About the Role:We are looking for a highly skilled and experienced Enterprise Architect...