The Three Loops — Auto-Create, Auto-Evaluate, Auto-Maintain
Solid's core differentiator: automating the creation, testing, and ongoing maintenance of semantic models, with a human-in-the-loop review at every stage.
Unlike legacy semantic layers that only "host" models, Solid automates the hardest work: creating, evaluating, and maintaining them. Each loop has automated phases and human-in-the-loop review phases — Solid never applies a change silently.
Auto-Create
What it does: Generates a first-draft semantic model from available data assets and business context — in minutes, not weeks.
Inputs (at least one required):
- A list of tables (when the modeler knows which tables are relevant)
- Business questions (questions the model should be able to answer)
- A description of the business area or use case
What Solid generates automatically:
- Table selection and documentation
- Column selection (50–100 relevant columns)
- Relationship inference from query history
- Metric extraction from query history
- Certified SQL examples from filtered query log
- Per-entity descriptions
Human-in-the-loop: The modeler reviews and refines the first draft before advancing to evaluation.
Auto-Evaluate
What it does: Every model ships with benchmark questions and expected SQL, so accuracy is proven and regressions are caught — not assumed.
Key outputs:
- Auto-generated benchmark questions from real SQL log patterns
- Certified ground-truth SQL attached to each question
- An accuracy score calculated after each benchmark run
- Pass/fail per question, with root cause
Human-in-the-loop: The modeler reviews benchmark questions, adds new ones, removes irrelevant ones, and updates ground-truth SQL before running the test.
See Benchmarking for the full process and pass criteria.
Auto-Maintain
What it does: Detects schema drift, usage shifts, and new SQL patterns, and proposes refreshes automatically — surfaced as one-click fixes.
Three trigger types:
- Benchmark run failures — a benchmark returns failed questions; Solid identifies the root cause and proposes an exact fix.
- Data layer changes — tables, columns, metrics, or SQLs evolve in the warehouse; Solid auto-detects the drift and maps its impact.
- Production usage gaps — real MCP agent usage reveals a metric or concept with no model coverage; Solid flags the gap.
Human-in-the-loop: Every automated recommendation requires explicit modeler review before it takes effect.
See Auto-Maintain for the full detail on each trigger type.
The Human-in-the-Loop Principle
Every automated action Solid takes requires explicit modeler review before it takes effect. Solid does not silently update a model. This applies at all three phases above: model generation, benchmark questions, and automatic recommendations.
Updated 8 days ago