Skip to content
nanohypenanohype

Module: LLM Observability

AI-specific telemetry module layered on OpenTelemetry. Wraps any async LLM call with trace capture, records per-request cost from a shared pricing table, monitors response quality with sliding-window statistics (avg, p50, p95, trend), and exports spans to pluggable backends (console, OTLP, JSON file). Factory-based architecture with no module-level mutable state — all tracers, calculators, and monitors are instance-scoped via createLlmObserver().

Namemodule-llm-observability
Version0.1.0
Categorycomposable-modules
LicenseApache-2.0
Personaengineering
Tagsllm, observability, tracing, cost-tracking, quality, typescript
Sourcetemplates/module-llm-observability

Render it

Three front doors, one catalog. Pick whichever suits the caller.

# CLI
npx @nanohype/sdk render module-llm-observability --out ./my-app

# SDK
import { LocalSource, renderTemplate } from "@nanohype/sdk";
const result = await renderTemplate(source, "module-llm-observability", variables);

# MCP — from an agent
get_template({ name: "module-llm-observability" })

Prerequisites

ToolVersionWhy
node>=22Node.js runtime for LLM observability module

Variables

Required

ProjectName string
Kebab-case project name, used as package name and directory Must be lowercase kebab-case starting with a letter.

Optional

Description string — defaults to LLM observability module with tracing, cost tracking, and quality monitoring
Short project description for package.json and README
Exporter string — defaults to console
Default LLM exporter (console, otlp, json-file, or a custom name)
IncludeCostTracking bool — defaults to true
Include cost tracking with per-model pricing and anomaly detection

What it produces

32 files. Placeholder names such as __APP_NAME__ are what the renderer substitutes into.

.env.example
.gitignore
biome.json
package.json
README.md
src/llm-observability/__tests__/cost.test.ts
src/llm-observability/__tests__/quality.test.ts
src/llm-observability/__tests__/registry.test.ts
src/llm-observability/__tests__/tracer.test.ts
src/llm-observability/bootstrap.ts
src/llm-observability/cost/anomaly.ts
src/llm-observability/cost/calculator.ts
src/llm-observability/cost/pricing.ts
src/llm-observability/cost/types.ts
src/llm-observability/exporters/console.ts
src/llm-observability/exporters/index.ts
src/llm-observability/exporters/json-file.ts
src/llm-observability/exporters/mock.ts
src/llm-observability/exporters/otlp.ts
src/llm-observability/exporters/registry.ts
src/llm-observability/exporters/types.ts
src/llm-observability/index.ts
src/llm-observability/logger.ts
src/llm-observability/metrics.ts
src/llm-observability/quality/monitor.ts
src/llm-observability/quality/types.ts
src/llm-observability/tracer/index.ts
src/llm-observability/tracer/types.ts
src/llm-observability/types.ts
tsconfig.build.json
tsconfig.json
vitest.config.ts

Composes with

Composites that use it