🧪

LLM Eval Kit

10 quality checks. LLM-as-judge. Multi-model comparison.
Score any LLM output — zero API keys required.

10
Quality Checks
3
LLM Judges
43
Tests Passing
0
API Keys Needed

LLM Output

ScoreCard

🧪

Click Score Output

Compare Models

Prompt

Model A

Model B

All 10 Checks + 3 Judges

🔍
Hallucination
Hedging, fake citations, cutoff refs
Rule-based
📝
Placeholder
{{VAR}}, [TBD], Lorem ipsum
Rule-based
🤖
Style
AI tells: "delve", "tapestry"
Rule-based
📅
Freshness
Stale year references
Rule-based
📏
Length
Too short or too long
Rule-based
🔒
PII
Emails, SSNs, API keys, tokens
Rule-based
☠️
Toxicity
Violence, insults, profanity
Rule-based
{ }
JSON Validity
Valid JSON? Schema? Types?
Rule-based
✅
Completeness
All prompt parts addressed?
Rule-based
⚖️
Consistency
Self-contradictions detected
Rule-based
🧑‍⚖️
LLM Judge
G-Eval: criteria → CoT → score
LLM Judge
⚔️
Pairwise
Compare A vs B, pick winner
LLM Judge
📊
Rubric
Score against defined levels
LLM Judge
pip install llm-eval-kit