Module: Semantic Cache

Embedding-based LLM response caching using vector similarity search. Stores prompt embeddings alongside cached responses and retrieves them by cosine similarity instead of exact key match. Embeds through the Platform's ModelGateway by default, so the cache holds no model credential, with Bedrock Titan called directly under the AWS credential chain as the off-substrate alternate, and an in-memory vector store for development. Includes a gateway adapter for drop-in integration with LLM gateway caching strategies.

Name	`module-semantic-cache`
Version	`0.1.0`
Category	composable-modules
License	Apache-2.0
Persona	engineering
Tags	semantic-cache, embeddings, vector-search, llm, caching, typescript, model-gateway
Source	`templates/module-semantic-cache`

Render it

Three front doors, one catalog. Pick whichever suits the caller.

# CLI
npx @nanohype/sdk render module-semantic-cache --out ./my-app

# SDK
import { LocalSource, renderTemplate } from "@nanohype/sdk";
const result = await renderTemplate(source, "module-semantic-cache", variables);

# MCP — from an agent
get_template({ name: "module-semantic-cache" })

Prerequisites

Tool	Version	Why
`node`	`>=26`	Node.js runtime for the semantic cache module

Variables

Required

ProjectName string: Kebab-case project name, used as package name and directory Must be lowercase kebab-case starting with a letter.

Optional

Description string — defaults to Semantic caching for LLM responses: Short project description for package.json and README
EmbeddingProvider string — defaults to gateway: Embedding provider the README examples name: gateway (the Platform ModelGateway, which holds the Bedrock role on the nanohype substrate, and the createSemanticCache() default), bedrock (called directly under the AWS credential chain, off the substrate), mock, or a custom name
PlatformName string — defaults to ${ProjectName}: Name of the Platform CR this project deploys under. The gateway provider defaults MODEL_GATEWAY_ENDPOINT to the Service the eks-agent-platform operator creates for it, <PlatformName>-gateway in namespace tenants-<PlatformName>
VectorBackend string — defaults to memory: Default vector cache store backend (memory or a custom name)

What it produces

36 files. Placeholder names such as __APP_NAME__ are what the renderer substitutes into.

.env.example
.gitignore
biome.json
package.json
README.md
src/semantic-cache/__tests__/cache.test.ts
src/semantic-cache/__tests__/circuit-breaker.test.ts
src/semantic-cache/__tests__/embed-request-timeout.test.ts
src/semantic-cache/__tests__/embedder-registry.test.ts
src/semantic-cache/__tests__/gateway-adapter.test.ts
src/semantic-cache/__tests__/gateway-embedder.test.ts
src/semantic-cache/__tests__/similarity.test.ts
src/semantic-cache/__tests__/store-registry.test.ts
src/semantic-cache/__tests__/threshold-sweep.test.ts
src/semantic-cache/bootstrap.ts
src/semantic-cache/circuit-breaker.ts
src/semantic-cache/embedder/bedrock.ts
src/semantic-cache/embedder/gateway.ts
src/semantic-cache/embedder/index.ts
src/semantic-cache/embedder/mock.ts
src/semantic-cache/embedder/registry.ts
src/semantic-cache/embedder/request-timeout.ts
src/semantic-cache/embedder/types.ts
src/semantic-cache/gateway-adapter.ts
src/semantic-cache/index.ts
src/semantic-cache/metrics.ts
src/semantic-cache/similarity.ts
src/semantic-cache/store/index.ts
src/semantic-cache/store/memory.ts
src/semantic-cache/store/registry.ts
src/semantic-cache/store/types.ts
src/semantic-cache/threshold-sweep.ts
src/semantic-cache/types.ts
tsconfig.build.json
tsconfig.json
vitest.config.ts

Composes with

Pairs with ts-service
Pairs with agentic-loop
Pairs with rag-pipeline
Pairs with module-cache-ts
Nests inside monorepo

Composites that use it

Cost-Optimized AI Stack — Minimize LLM costs with intelligent routing, response caching, and cost monitoring.