AI Evaluation Suite

Standalone evaluation infrastructure with test harness, versioned prompts, and guardrails validation. For teams that need to test AI systems without building them.

Name	eval-suite
Version	0.1.0
Tags	eval, testing, prompts, ai, quality
Source	composites/eval-suite.yaml

Render it

# CLI
npx @nanohype/sdk render-composite eval-suite --out ./my-app

# SDK
import { LocalSource, renderComposite } from "@nanohype/sdk";
const result = await renderComposite(source, "eval-suite", variables);

# MCP — from an agent
get_composite({ name: "eval-suite" })

What it draws together

Template	Rendered at	Notes
monorepo	root	the root project
eval-harness	packages/evals
prompt-library	packages/prompts
guardrails	packages/guardrails	only when IncludeGuardrails

Variables

Required

ProjectName string

Project name Must be lowercase kebab-case.

Optional

LlmProvider string — defaults to anthropic

LLM provider for eval targets

IncludeGuardrails bool — defaults to true

Include safety filter testing