Add dynamic dataset api

Context

Users can only currently define standalone queries which limits the following use cases :

  • AI Agents: Can't accurately generate queries based on natural language questions, without a pre-defined query

  • Dashboard Builders: Users can't drag-and-drop dimensions/measures

  • BI Tool Integration: Can't accept dynamic SQL/GraphQL queries

  • Multi-Tenant Custom Views: Each tenant can't define their own metrics

Goal

Create dynamic API for runtime queries, with a separate entry point to queries.

I imagine the API looking something like the below:

// generated-datasets.ts (auto-generated)
export const datasets = {
  trips: defineDataset({
    table: 'trips',
    dimensions: {
      pickup_datetime: dimension.datetime('pickup_datetime'),
      pickup_date: dimension.date('toDate(pickup_datetime)'),
      payment_type: dimension.string('payment_type'),
      // ... all columns from schema
    },
    measures: {
      count: measure.count(),
      total_revenue: measure.sum('total_amount'),
      avg_distance: measure.avg('trip_distance'),
      // ... common aggregations
    },
  }),
} as const;

// ===== DYNAMIC QUERIES =====
const dynamicResults = await db.queryDataset(datasets.trips, {
  dimensions: ['pickup_date', 'payment_type'] as const,
  measures: ['count', 'total_revenue'] as const,
});
// Type: Array<{ pickup_date: Date; payment_type: string; count: number; total_revenue: number }>

// ===== FULLY DYNAMIC (No Type Inference) =====
const aiResults = await db.queryDataset(datasets.trips, parseUserInput(query));
// Type: DynamicQueryResult (Record<string, any> with metadata)

// ===== AI AGENT TOOL GENERATION =====
const tools = db.generateTools(datasets.trips, {
  mode: 'per-query',  // One tool per predefined query
  // OR
  mode: 'catalog',    // Single tool with dimension/measure selection
});

export const api = defineServe({
  queries: {
    // Auto-generated from dataset
    ...datasets.trips.toQueries(),
  }
});

Upvoters
Status

In Review

Board
πŸ’‘

Feature Request

Date

7 days ago

Author

Luke

Subscribe to post

Get notified by email when there are changes.