Add dynamic dataset api
Context
Users can only currently define standalone queries which limits the following use cases :
AI Agents: Can't accurately generate queries based on natural language questions, without a pre-defined query
Dashboard Builders: Users can't drag-and-drop dimensions/measures
BI Tool Integration: Can't accept dynamic SQL/GraphQL queries
Multi-Tenant Custom Views: Each tenant can't define their own metrics
Goal
Create dynamic API for runtime queries, with a separate entry point to queries.
I imagine the API looking something like the below:
// generated-datasets.ts (auto-generated)
export const datasets = {
trips: defineDataset({
table: 'trips',
dimensions: {
pickup_datetime: dimension.datetime('pickup_datetime'),
pickup_date: dimension.date('toDate(pickup_datetime)'),
payment_type: dimension.string('payment_type'),
// ... all columns from schema
},
measures: {
count: measure.count(),
total_revenue: measure.sum('total_amount'),
avg_distance: measure.avg('trip_distance'),
// ... common aggregations
},
}),
} as const;
// ===== DYNAMIC QUERIES =====
const dynamicResults = await db.queryDataset(datasets.trips, {
dimensions: ['pickup_date', 'payment_type'] as const,
measures: ['count', 'total_revenue'] as const,
});
// Type: Array<{ pickup_date: Date; payment_type: string; count: number; total_revenue: number }>
// ===== FULLY DYNAMIC (No Type Inference) =====
const aiResults = await db.queryDataset(datasets.trips, parseUserInput(query));
// Type: DynamicQueryResult (Record<string, any> with metadata)
// ===== AI AGENT TOOL GENERATION =====
const tools = db.generateTools(datasets.trips, {
mode: 'per-query', // One tool per predefined query
// OR
mode: 'catalog', // Single tool with dimension/measure selection
});
export const api = defineServe({
queries: {
// Auto-generated from dataset
...datasets.trips.toQueries(),
}
});
Comments3
Luke
Jul 13
Marking as done now, SQL/GraphQL compat not implemented from the original scope, will wait on pull from the community before deciding on this one
Connor Quigley
Jun 26
My use case requires dynamic queries, my team would give hypequery a try if this feature is built!
Luke
Jun 27
Hey Connor, thanks for considering hypequery! The new datasets release actually covers a lot of this, although there are some gaps, so it would be good to understand a bit more about your use case and what in particular you are looking for.
In summary, here is the support by use case, with the details feature comparison further below
AI agents: partially there, but catalog quality and tool generation need work
Dashboard builders: viable today for same-dataset drag-and-drop exploration, but metadata, direct typing and more advanced features can be added
BI integration: early; structured semantic querying works, SQL/GraphQL compatibility not yet implemented.
Multi-tenant custom views: mostly there for tenant-scoped datasets, but custom per-tenant metric/dataset registries would need more work.
Already Implemented:
Dynamic same-dataset querying exists: callers can select dimensions/measures at runtime (https://hypequery.com/docs/datasets/execution)
Dataset definitions support dimensions, measures, filters, limits, tenant keys, and time keys.
Runtime validation exists for unknown fields, invalid filters, limits, time grain, and tenant misuse.
Execution exists through the dataset client
SQL inspection exists
Serve already exposes separate dataset endpoints
React hooks already support dataset endpoints
MCP has catalog-style dataset tools
CLI can generate dataset definitions from ClickHouse schema
Main Feature Gaps
There is no general AI tool generation API beyond the MCP server.
There is no generated “predefined queries from dataset” layer. Serve exposes flexible dataset endpoints, not generated query definitions.
Measure support: common aggregations exist, but richer analytics measures are missing (percentiles etc).
Relationships between tables not yet published.
Dashboard-builder metadata is thin: labels/descriptions exist, but not rich UI/catalog metadata