Skip to main content

Add dynamic dataset api

Context

Users can only currently define standalone queries which limits the following use cases :

  • AI Agents: Can't accurately generate queries based on natural language questions, without a pre-defined query

  • Dashboard Builders: Users can't drag-and-drop dimensions/measures

  • BI Tool Integration: Can't accept dynamic SQL/GraphQL queries

  • Multi-Tenant Custom Views: Each tenant can't define their own metrics

Goal

Create dynamic API for runtime queries, with a separate entry point to queries.

I imagine the API looking something like the below:

// generated-datasets.ts (auto-generated)
export const datasets = {
  trips: defineDataset({
    table: 'trips',
    dimensions: {
      pickup_datetime: dimension.datetime('pickup_datetime'),
      pickup_date: dimension.date('toDate(pickup_datetime)'),
      payment_type: dimension.string('payment_type'),
      // ... all columns from schema
    },
    measures: {
      count: measure.count(),
      total_revenue: measure.sum('total_amount'),
      avg_distance: measure.avg('trip_distance'),
      // ... common aggregations
    },
  }),
} as const;

// ===== DYNAMIC QUERIES =====
const dynamicResults = await db.queryDataset(datasets.trips, {
  dimensions: ['pickup_date', 'payment_type'] as const,
  measures: ['count', 'total_revenue'] as const,
});
// Type: Array<{ pickup_date: Date; payment_type: string; count: number; total_revenue: number }>

// ===== FULLY DYNAMIC (No Type Inference) =====
const aiResults = await db.queryDataset(datasets.trips, parseUserInput(query));
// Type: DynamicQueryResult (Record<string, any> with metadata)

// ===== AI AGENT TOOL GENERATION =====
const tools = db.generateTools(datasets.trips, {
  mode: 'per-query',  // One tool per predefined query
  // OR
  mode: 'catalog',    // Single tool with dimension/measure selection
});

export const api = defineServe({
  queries: {
    // Auto-generated from dataset
    ...datasets.trips.toQueries(),
  }
});
Status: Completed3 comments

Comments3

  • Luke

    Team•

    Jul 13

    Marking as done now, SQL/GraphQL compat not implemented from the original scope, will wait on pull from the community before deciding on this one

  • Connor Quigley

    •

    Jun 26

    My use case requires dynamic queries, my team would give hypequery a try if this feature is built!

    • Luke

      Team•

      Jun 27

      Hey Connor, thanks for considering hypequery! The new datasets release actually covers a lot of this, although there are some gaps, so it would be good to understand a bit more about your use case and what in particular you are looking for.

      In summary, here is the support by use case, with the details feature comparison further below

      • AI agents: partially there, but catalog quality and tool generation need work

      • Dashboard builders: viable today for same-dataset drag-and-drop exploration, but metadata, direct typing and more advanced features can be added

      • BI integration: early; structured semantic querying works, SQL/GraphQL compatibility not yet implemented.

      • Multi-tenant custom views: mostly there for tenant-scoped datasets, but custom per-tenant metric/dataset registries would need more work.

      Already Implemented:

      • Dynamic same-dataset querying exists: callers can select dimensions/measures at runtime (https://hypequery.com/docs/datasets/execution)

      • Dataset definitions support dimensions, measures, filters, limits, tenant keys, and time keys.

      • Runtime validation exists for unknown fields, invalid filters, limits, time grain, and tenant misuse.

      • Execution exists through the dataset client

      • SQL inspection exists

      • Serve already exposes separate dataset endpoints

      • React hooks already support dataset endpoints

      • MCP has catalog-style dataset tools

      • CLI can generate dataset definitions from ClickHouse schema

      Main Feature Gaps

      • There is no general AI tool generation API beyond the MCP server.

      • There is no generated “predefined queries from dataset” layer. Serve exposes flexible dataset endpoints, not generated query definitions.

      • Measure support: common aggregations exist, but richer analytics measures are missing (percentiles etc).

      • Relationships between tables not yet published.

      • Dashboard-builder metadata is thin: labels/descriptions exist, but not rich UI/catalog metadata