0

Likes

0

Saves

Back to updates

[r/ML] [Project] JudgeGPT — open-source LLM-as-judge benchmarking tool with configurable scoring rubrics, CoT reasoning, and real-time GPU telemetry

Impact: 7/10

Swipe left/right

Previous Article

Move to the previous article in the active list

Move to the next article in the active list

Summary

JudgeGPT is an open-source tool for local LLM-as-judge evaluations, designed to tackle common reliability issues like position and family bias. It features configurable scoring rubrics, Chain-of-Thought reasoning, and real-time GPU telemetry to improve benchmark accuracy. This tool enables users to evaluate models running via Ollama, enhancing the trustworthiness of local LLM performance assessments.

Continue Reading

Explore related coverage about community news and adjacent AI developments: [r/ML] [D] MYTHOS-INVERSION STRUCTURAL AUDIT, [r/LocalLLaMA] karpathy / autoresearch, [r/ML] [R] Agentic AI and Occupational Displacement: A Multi-Regional Task Exposure Analysis (236 occupations, 5 US metros), [r/ML] Building behavioural response models of public figures using Brain scan data (Predict their next move using psychological modelling) [P].

Related Articles

Comments

Sign in to leave a comment.

Loading comments...