AI Operations Specialist

1 week ago

Johannesburg, South Africa Africonology Solutions Full time

AI Operations Specialist Responsible for ensuring the stability, reliability, and performance of AI systems in production. This role blends Machine Learning Operations (MLOps), Bot Operations, and Quality Assurance (QA) practices, supporting both backend models and customer-facing bots. The specialist will manage deployments, monitor system health, conduct testing, and validate quality to guarantee seamless AI operations and a consistent customer experience. Key Responsibilities System Monitoring & Reliability Monitor AI systems to ensure uptime, performance, and error-free operation. Implement automated monitoring and alerting systems for AI performance. Define SLOs and error budgets for AI features. Testing & Quality Assurance Design, develop, and maintain automated test scripts for web, mobile, and API testing. Execute manual test cases for exploratory and non-automatable scenarios. Identify, log, and track bugs to closure using Jira, Azure DevOps, or equivalent tools. Ensure adherence to QA methodologies, tools, and best practices. AI & Bot Operations Manage version control, deployment, and release cycles of AI models and bots. Conduct offline/online evaluations and A/B tests for prompts, models, and policies. Track and troubleshoot production issues (latency, escalations, fallback errors). Design and orchestrate conversations using Voiceflow and related tools. Observability, Security & Compliance Work with observability/evaluation tools (Langfuse, Arize/Phoenix, W&B, Prometheus, Grafana). Implement guardrails, safety red-teaming, and prompt-injection defenses. Manage PII handling, content safety filters, and data loss prevention (DLP). Document incidents, perform RCA reporting, and ensure compliance with data privacy and security policies. Collaboration & Continuous Improvement Collaborate with developers, data scientists, and business teams to resolve operational issues. Participate in incident on-call rotations, maintain runbooks, and conduct disaster recovery (DR) tests. Recommend improvements to enhance system resilience, reduce downtime, and optimize costs. Requirements Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience). 3+ years of experience in operations, DevOps, or AI/ML support roles. Proven track record managing large, complex, multi-stakeholder projects. Strong knowledge of Machine Learning practices (model monitoring, retraining, pipelines). Familiarity with conversational AI platforms (Amazon Lex, Salesforce Einstein Bots, ElevenLabs). Integration experience with Amazon Connect, Genesys Cloud CX, NICE CXone, or similar. Proficiency with test automation tools (Selenium, Playwright, Cypress, Appium, or equivalent). Experience with API testing tools (Postman, RestAssured, Karate). Strong scripting and automation skills (Python, Bash, CI/CD pipelines). C1 English proficiency. Nice to Have Experience with performance/load testing tools (JMeter, Locust, Gatling). Knowledge of cloud platforms (AWS, Azure, GCP). Familiarity with Git or other version control systems. QA certification (ISTQB or equivalent). #J-18808-Ljbffr

  • Ai Operations Specialist

    3 weeks ago

    Johannesburg, South Africa Africonology Full time

    The AI Operations Specialist is responsible for ensuring the stability, reliability, and performance of AI systems in production. This role blends Machine Learning Operations (MLOps), Bot Operations, and Quality Assurance (QA) practices, supporting both backend models and customer-facing bots. The specialist will manage deployments, monitor system health,...

  • AI Specialist

    4 days ago

    Johannesburg, Gauteng, South Africa Zealhr Full time US$90 000 - US$120 000 per year

    Job Description AI SpecialistINTRODUCTION:Our client is a leading pharmaceutical company committed to innovation, digital transformation, and global expansion. Are you ready to transform industries with cutting-edge AI solutions? We are looking for an AI Specialist to lead the way in automation, data-driven decision-making, and AI-powered innovations.Job...

  • AI Specialist

    4 days ago

    Johannesburg, Gauteng, South Africa Rosstone Professional Solutions Full time R900 000 - R1 200 000 per year

    AI Specialist / Industrial EngineerThis role is critical in identifying, developing, and deploying AI-driven solutions that improve operational efficiency, inventory management, demand forecasting, and overall supply chain performance. The ideal candidate will combine strong technical AI/ML expertise with a practical understanding of warehousing operations,...

  • AI Operations

    1 week ago

    Johannesburg, South Africa Africonology Solutions Full time

    A technology solutions provider is looking for an AI Operations Specialist in Johannesburg to ensure the stability and performance of AI systems in production. The role involves overseeing deployments, maintaining system health, conducting testing, and ensuring a seamless customer experience. Candidates should have a Bachelor's degree in Computer Science,...

  • AI specialist

    2 weeks ago

    City of Johannesburg Metropolitan Municipality, South Africa OrbitAI & Advisory Full time

    Orbit AI & Advisory focuses on solving big technical challenges in the retail, engineering, aviation and pharma sectors. We believe that AI is not going to take your job, but the person who knows how to use AI is. In addition, we train companies and individuals on how to use AI in their current daily workflow which leads to an increase in efficiency. Role...

  • B2b Specialist

    1 week ago

    Johannesburg, South Africa Undetectable AI Full time

    **About Us**: Undetectable AI is a leading provider of AI content detection and humanization solutions. Our platform enables users to transform AI-generated text into human-like content, ensuring authenticity and bypassing AI detection tools. Trusted by over 14 million users worldwide, we've been featured in Forbes, Business Insider, and other prominent...

  • AI Specialist

    1 week ago

    Johannesburg, Gauteng, South Africa Tshinwelo Innovative Business Solutions [TIBS] Full time R1 500 000 - R2 500 000 per year

    Company DescriptionTshinwelo Innovative Business Solutions (TIBS), established in 2015, leverages its expertise in business strategy advisory, supply chain management, business process management, and human capital management to offer pragmatic, innovative, and sustainable solutions. By integrating traditional business approaches with digital enabling...

  • AI Specialists

    1 week ago

    Johannesburg, Gauteng, South Africa Tshinwelo Innovative Business Solutions [TIBS] Full time R1 200 000 - R2 400 000 per year

    Company DescriptionTshinwelo Innovative Business Solutions (TIBS), established in 2015, offers pragmatic and sustainable solutions in Business strategy advisory, Supply chain management, Business process management, and Human capital management. By integrating traditional business approaches with digital enabling technologies, TIBS helps organizations...

  • Principal AI Technology

    3 weeks ago

    Johannesburg, South Africa NTT DATA, Inc. Full time

    Principal AI Technology & Innovation Specialist Make an impact with NTT DATA. Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow,...

  • Multi - Discipline Specialist(AI Engineer)

    2 weeks ago

    Johannesburg, South Africa nedbank Full time

    Multi-Discipline Specialist (AI Engineer) Details Location: Johannesburg, ZA Reference: Job Classification Requisition IDs: & Job Family Information Technology IT Technical Specialists Manage Self Job Purpose To provide guidance and advice related to the integration of solutions across technologies; ensuring all outputs are in compliance with current...