AI Systems & MLOps
Grounded answers, agent reliability, evaluation and the operating costs of AI.
Research-design drafts identify proposed methods and limitations. They do not claim completed experiments.
AI Citation Accuracy: Confident Answers, Incorrect Evidence
A plausible answer and a supporting citation are separate claims. Evaluate whether the cited passage supports the exact statement, including its qualifiers and context.
RAG Versus Long Context: A Controlled Comparison
Retrieval and long-context input make different selection decisions. Compare them for a fixed document collection instead of assuming that either architecture always wins.
Why AI Agents Fail on Long Workflows
A multi-step task can fail after several locally correct actions. Reliability depends on preserving state, recognising mistakes and recovering, not simply producing a convincing final response.
Human Approval Policies for AI Agents
Human review helps only when the reviewer sees enough information to make the right decision. Too many low-value prompts can dilute attention at the consequential moment.
AI Agent Memory: Learning or Repeating Mistakes?
Persistent memory can retain useful context and obsolete assumptions. Evaluate correction and deletion behaviour as carefully as recall of successful facts.
LLM Drift Versus Changing Workloads
A changed output distribution may reflect new users or documents rather than a changed model. Monitoring needs to separate model version, data distribution and task mix.
Small Models, Large Models and Routing Policies
A routing system is valuable when its selection cost and occasional wrong choice are outweighed by savings on acceptable answers. Average token price alone does not describe that tradeoff.
Structured Output Does Not Prove Correctness
Matching a schema establishes format, not the truth of the values inside it. A valid date, amount or category can still be incorrect for the source document.
LLM Observability Without Sensitive Logging
Reducing sensitive content in logs changes the evidence available for debugging. The design task is to preserve diagnostic context while limiting unnecessary exposure.
The Total Cost of Successful AI Automation
Successful automation can still be expensive when review, retries and maintenance are ignored. The relevant denominator is a business task completed to an agreed standard.