Prompt Asset & Evaluation Platform
v1.0 · ProductionPromptDeck
PromptDeck is a prompt asset and evaluation platform for AI teams. Today, most teams' prompts are scattered across code constants, note apps and chat history: a one-word change ships with zero regression testing, quality is judged by gut feeling, and every release is a gamble. PromptDeck upgrades prompts into version-controlled engineering assets — every change gets a git-style diff, every version runs against your own case sets, and score regressions block merges right in CI. Compare local Ollama models against the cloud Claude API side by side, with the whole stack available for self-hosted deployment.
Problems We Solve
- ✗Prompts scattered across code, notes and chats — no single source of truth
- ✗No regression tests for changes; 'feels better' goes straight to production
- ✗Model switches and upgrades cause unquantifiable quality regressions
- ✗Sensitive eval data cannot be uploaded to third-party SaaS
Core Features
Prompt Version Control
Git-style history and word-level diffs. Every prompt is a first-class asset with branching and rollback.
Dataset-driven Evaluation
Build your own case sets (input + expected behavior); an LLM judge scores automatically with fully traceable reports.
Multi-model Comparison
Same prompt, same dataset — local Ollama (qwen / deepseek-r1) and the cloud Claude API benchmarked side by side.
Regression Gate
CI integration: PRs trigger evaluations, and scores below baseline block the merge. Prompt changes finally get a safety net.
Evaluation Worker Queue
Evaluations run asynchronously in a queue — large case sets never block the console, with observable progress and retries.
Self-hosted Deployment
One Docker Compose command. All data lives in your own PostgreSQL; inference can run entirely on in-network GPUs.
System Architecture
Request path, top to bottom
Prompt editing, diff views, evaluation reports and comparison dashboards
Version management, evaluation orchestration, CI webhook gates
Private evaluation on local models, cloud models as reference baseline
Persistent prompt versions, case sets and eval records; async evaluation jobs
Tech Stack
Pricing Direction
Full features for invited teams, in exchange for real-world evaluation feedback
Hosted version billed per team seat, evaluation quota included
Deployed inside your network, integrated with your own GPUs and models
Your prompts deserve regression tests
Early access seats are limited — priority goes to teams with LLM features already in production.
Apply for Early Access