AI Prompt & Evaluation Specialist

Job Description Location: Tomorrows Guides Ltd, Unit 1, Station Yard, Station Road

Our engineering team builds and runs the AI pipelines — what we need is someone who owns the quality of what comes out of them. You'll write the prompts behind our LLM workflows and prove they work by testing them rigorously against grounded, real-world data. Existing workflows classify and extract structured data from care-enquiry call transcripts and emails; many more use cases planned (CV parsing, Review summaries etc.) will follow.

About Us Tomorrow's Guides is a well-established, ambitious, and successful website company, operating leading care sector websites: carehome.co.uk, homecare.co.uk, and daynurseries.co.uk. Combined, our websites feature over 40,000 business profiles, host nearly 250,000 online reviews, and attract more than 22 million visitors annually.

This is a remote role, but you will need to spend time in our offices in Hungerford from time to time when required by the business so it’s important that you live within a commutable distance.

About The Role What you'll do

• Write, version and continually refine the prompts that power our LLM features
• Work closely with subject-matter experts to understand the domain in depth, then encode that expertise as precise, unambiguous rules (our workflows turn on distinctions that carry real commercial and compliance weight)
• Build and maintain "golden" evaluation datasets from real examples, with correct expected outputs
• Run systematic evals — measuring accuracy, catching regressions, comparing prompt and model variations head-to-head
• Diagnose why outputs fail — ambiguous instructions, edge cases, messy input, model limitations — and fix them
• Define measurable criteria for what "good" looks like on each field and each use case
• Keep large, evolving prompts coherent and internally consistent as rules accumulate
• Partner with our engineers, who own the pipelines, to get your prompts shipped and monitored

What We're Looking For (essential)

• Hands-on experience writing and refining LLM prompts, with a structured rather than trial-and-error approach
• A genuine eye for evaluation: you think in test cases, edge cases and measurable criteria, not vibes
• Comfort producing and reasoning about structured outputs — JSON schemas, strict enumerated values, conditional/nullable fields, type discipline
• Skill at eliciting tacit knowledge from experts and turning fuzzy real-world judgements into rules a model can follow consistently
• An instinct for how messy real-world input breaks things — e.g. speech-to-text errors, homophones, spelled-out corrections, inconsistent formatting
• Precision with written English; you understand how small wording changes shift model behaviour
• A responsible approach to sensitive personal data — our evaluation sets are built from real records containing names, contact details and health information (special-category data under UK GDPR), and you'll handle them accordingly
• Enough technical familiarity to work alongside engineers and understand how a prompt fits into a pipeline

Nice to have

• Experience with eval tooling or frameworks, or building evaluation harnesses
• Background in a regulated or high-stakes domain (health, legal, finance, insurance)
• A background in linguistics, QA, technical writing, or data analysis
• Familiarity with LLM APIs (OpenAI/Azure OpenAI, Anthropic) and how model or version changes affect outputs

Salary And Benefits

• Salary: Dependent on experience
• £4,000 per annum discretionary company bonus scheme
• 25 days annual leave + bank holidays
• 6% employer pension contribution
• Access to free perks and discounts through Perkbox
• Long Service Awards
• Cycle to Work Scheme
• Company and Team nights out

Skills Needed About The Company We're a fun and friendly company to work for.

We are a uniquely friendly and sociable organisation with a tremendous level of cross-departmental support. The direction of the company is open and transparent to all employees and there is a huge sense of working together to achieve a common goal. We are growing at a fast pace and developing at an equal speed and have surpassed some incredible goals over the last few years.

Company Culture Our recently renovated offices in Hungerford, West Berkshire, are open-plan with a real collaborative feel.

Required Criteria Desired Criteria Closing Date Tuesday 21st July, 2026

Contract Type fulltime

Salary Based on Experience