LLM Applications Engineer

3 days ago

Pretoria, Gauteng, South Africa Blacksmith Agency Remote Full-time

About The Role

Blacksmith Agency is building AI into every part of how we work. As LLM Applications Engineer at Blacksmith, you own and ship our internal automation roadmap: production LLM and agent systems that real people on our team use every day. This is a builder role, not research. You will not train models.

We hire engineers who have already shipped an LLM system to production and can walk us through it in detail: what it did, how it was built, what broke and what they changed. This is a full-time seat and it has to be your only job.

Blacksmith Agency

  • Engineering
  • Full time, your only job
  • Remote (Latin America and South Africa)

What You Will Build

  • Agents and workflows with tool use and function calling on Anthropic and OpenAI, with a framework or plain APIs.
  • Evals that prove a change made things better, and guardrails that stop a model taking a wrong or unsafe action.
  • Integrations connecting WordPress, Figma, Google Workspace, Slack and our internal systems, including MCP servers.
  • Internal tools that take measurable hours off delivery, sales operations and project management.

What You Bring

  • 2+ years building production software in Python or TypeScript / Node.js.
  • At least one LLM or agent system you built yourself that ran in production with real users.
  • Hands-on tool use, evals and failure handling: validating model output, retries, human approval.
  • REST APIs, webhooks and cloud deployment.
  • Strong spoken English. You will talk through technical decisions on live calls with a US team.

Not a fit if

  • You freelance or contract for other clients and would keep them alongside this job.
  • Your LLM work is demos, tutorials or chat prompting, or your focus is model training and research.