Module: Semantic Cache

Embedding-based LLM response caching using vector similarity search. Stores prompt embeddings alongside cached responses and retrieves them by cosine similarity instead of exact key match. Ships with a Bedrock Titan embedding provider (default), an OpenAI alternate, and an in-memory vector store for development. Includes a gateway adapter for drop-in integration with LLM gateway caching strategies.

Name	`module-semantic-cache`
Version	`0.1.0`
Category	composable-modules
License	Apache-2.0
Persona	engineering
Tags	semantic-cache, embeddings, vector-search, llm, caching, typescript
Source	`templates/module-semantic-cache`

Render it

Three front doors, one catalog. Pick whichever suits the caller.

# CLI
npx @nanohype/sdk render module-semantic-cache --out ./my-app

# SDK
import { LocalSource, renderTemplate } from "@nanohype/sdk";
const result = await renderTemplate(source, "module-semantic-cache", variables);

# MCP — from an agent
get_template({ name: "module-semantic-cache" })

Prerequisites

Tool	Version	Why
`node`	`>=24`	Node.js runtime for the semantic cache module

Variables

Required

ProjectName string: Kebab-case project name, used as package name and directory Must be lowercase kebab-case starting with a letter.

Optional

Description string — defaults to Semantic caching for LLM responses: Short project description for package.json and README
EmbeddingProvider string — defaults to bedrock: Default embedding provider (bedrock, openai, mock, or a custom name)
VectorBackend string — defaults to memory: Default vector cache store backend (memory or a custom name)

What it produces

33 files. Placeholder names such as __APP_NAME__ are what the renderer substitutes into.

.env.example
.gitignore
biome.json
package.json
README.md
src/semantic-cache/__tests__/cache.test.ts
src/semantic-cache/__tests__/circuit-breaker.test.ts
src/semantic-cache/__tests__/embedder-registry.test.ts
src/semantic-cache/__tests__/gateway-adapter.test.ts
src/semantic-cache/__tests__/similarity.test.ts
src/semantic-cache/__tests__/store-registry.test.ts
src/semantic-cache/__tests__/threshold-sweep.test.ts
src/semantic-cache/bootstrap.ts
src/semantic-cache/circuit-breaker.ts
src/semantic-cache/embedder/bedrock.ts
src/semantic-cache/embedder/index.ts
src/semantic-cache/embedder/mock.ts
src/semantic-cache/embedder/openai.ts
src/semantic-cache/embedder/registry.ts
src/semantic-cache/embedder/types.ts
src/semantic-cache/gateway-adapter.ts
src/semantic-cache/index.ts
src/semantic-cache/metrics.ts
src/semantic-cache/similarity.ts
src/semantic-cache/store/index.ts
src/semantic-cache/store/memory.ts
src/semantic-cache/store/registry.ts
src/semantic-cache/store/types.ts
src/semantic-cache/threshold-sweep.ts
src/semantic-cache/types.ts
tsconfig.build.json
tsconfig.json
vitest.config.ts

Composes with

Pairs with ts-service
Pairs with agentic-loop
Pairs with rag-pipeline
Pairs with module-cache-ts
Nests inside monorepo

Composites that use it

Cost-Optimized AI Stack — Minimize LLM costs with intelligent routing, response caching, and cost monitoring.