Skip to content
nanohypenanohype

Module: LLM Providers

Shared LLM provider pack — the canonical interface for all AI templates. Defaults to the gateway provider, which calls the Platform's ModelGateway and holds no model credential: the gateway holds the Bedrock role on the nanohype substrate. An optional Bedrock provider calls Bedrock directly under the AWS credential chain, for an application off the substrate. Neither holds a provider key. Factory-based registry, streaming, token counting via js-tiktoken, circuit breakers, OTel metrics, and a gateway adapter.

Namemodule-llm-providers
Version0.1.0
Categorycomposable-modules
LicenseApache-2.0
Personaengineering
Tagsllm, providers, model-gateway, bedrock, streaming, typescript
Sourcetemplates/module-llm-providers

Render it

Three front doors, one catalog. Pick whichever suits the caller.

# CLI
npx @nanohype/sdk render module-llm-providers --out ./my-app

# SDK
import { LocalSource, renderTemplate } from "@nanohype/sdk";
const result = await renderTemplate(source, "module-llm-providers", variables);

# MCP — from an agent
get_template({ name: "module-llm-providers" })

Prerequisites

ToolVersionWhy
node>=26Node.js runtime for the LLM providers module

Variables

Required

ProjectName string
Kebab-case project name, used as package name and directory Must be lowercase kebab-case starting with a letter.

Optional

Description string — defaults to LLM provider pack for the Platform ModelGateway and Bedrock
Short project description for package.json and README
DefaultProvider string — defaults to gateway
Default LLM provider: gateway (the Platform ModelGateway, which holds the Bedrock role on the nanohype substrate), bedrock off the substrate under an assumed IAM role (with IncludeBedrock), mock, or a custom name
PlatformName string — defaults to ${ProjectName}
Name of the Platform CR this project deploys under. The gateway provider defaults MODEL_GATEWAY_ENDPOINT to the Service the eks-agent-platform operator creates for it, <PlatformName>-gateway in namespace tenants-<PlatformName>
IncludeBedrock bool
Include the AWS Bedrock provider, which calls Bedrock under the AWS credential chain, for an application off the substrate
ProviderCheck string — defaults to ${DefaultProvider}:${IncludeBedrock}
Derived from DefaultProvider and IncludeBedrock, and takes no value of its own. It fails the scaffold when DefaultProvider names a provider the project leaves out, which would otherwise fail at the first model call

What it produces

37 files. Placeholder names such as __APP_NAME__ are what the renderer substitutes into.

.env.example
.gitignore
biome.json
package.json
README.md
src/llm-providers/__tests__/adapters.test.ts
src/llm-providers/__tests__/bedrock-request.test.ts
src/llm-providers/__tests__/gateway-request.test.ts
src/llm-providers/__tests__/mock.test.ts
src/llm-providers/__tests__/pricing.test.ts
src/llm-providers/__tests__/provider-contract.ts
src/llm-providers/__tests__/registry.test.ts
src/llm-providers/__tests__/tokens.test.ts
src/llm-providers/adapters/gateway.ts
src/llm-providers/adapters/streaming.ts
src/llm-providers/bootstrap.ts
src/llm-providers/config.ts
src/llm-providers/index.ts
src/llm-providers/logger.ts
src/llm-providers/metrics.ts
src/llm-providers/providers/bedrock.ts
src/llm-providers/providers/gateway.ts
src/llm-providers/providers/index.ts
src/llm-providers/providers/mock.ts
src/llm-providers/providers/registry.ts
src/llm-providers/providers/types.ts
src/llm-providers/resilience/__tests__/circuit-breaker.test.ts
src/llm-providers/resilience/__tests__/request-timeout.test.ts
src/llm-providers/resilience/__tests__/stream-guard.test.ts
src/llm-providers/resilience/circuit-breaker.ts
src/llm-providers/resilience/request-timeout.ts
src/llm-providers/resilience/stream-guard.ts
src/llm-providers/tokens/counter.ts
src/llm-providers/types.ts
tsconfig.build.json
tsconfig.json
vitest.config.ts

Composes with