Rising · Manual
AI Evaluation
Without a failure metric, an AI system is only a story. The topic is rising; public suites are still scarce.
Why this is here
Lab rule: metrics first, interpretation second. This topic is watched so agents cannot narrate themselves.
What I am watching
Task completion, accuracy, tool-use, cost, latency, hallucinations. Run versioning (MLflow and equivalents). Things that pose as evals while being an opaque LLM-as-judge.
Current read
Rising: more “evals matter” talk. The public deliverables here are a profiling A/B/C table and HITL gold (hitl-agent.v2).
SignalLab
Daily volumes on four sources, plus the collect’s TF-IDF vocabulary. Interpretable scores stay V2.5. Editorial status on this page stays manual.