What is ReskCaching ?
ReskCaching is a Bun-based library for secure LLM response caching, reducing API costs by storing and retrieving responses from a vector database using semantic matching.
Features
- Caches LLM responses using vector similarity matching.
- Supports multiple cache backends (in-memory, SQLite, Redis).
- Advanced response selection with customizable algorithms.
- AES-GCM encryption for secure cache-at-rest protection.
- Real-time updates via WebSockets for instant distribution.
Use Cases
- Reduce LLM API call costs with cached responses.
- Ensure consistent, high-quality LLM responses.
- Select responses based on business logic or A/B tests.
- Scale for high-throughput production environments.
