Feedback
Roadmap

Have something to say?

Tell us how we could make the product more useful to you.

Completed

hypequery insertion support

Some thoughts on implementation and rollout: Phase 1: Define Scope - Support typed row inserts into existing ClickHouse tables. - Start with JSONEachRow only. - Exclude schema migrations, table creation, INSERT ... SELECT, and bulk file ingestion from v1. Phase 2: Add Adapter Support - Extend the internal database adapter with an insert capability. - Implement it in the ClickHouse adapter using the native ClickHouse client’s insert API. - Preserve existing query, stream, raw SQL, and cache behavior unchanged. Phase 3: Add Public API - Add a direct db.insert(table, rows, options) API. - Accept either a single row or an array of rows. - Support per-insert options like queryId and ClickHouse settings. - Keep insertion separate from the existing select query builder. Phase 4: Add Type Safety - Reuse existing schema inference so insert rows are typed from table columns. - Validate table names at compile time. - Validate row field names and value types at compile time. - Decide whether v1 rows are strict or partially optional. Phase 5: Improve Generated Types - Enrich schema generation with ClickHouse column metadata. - Detect nullable columns, defaults, materialized columns, alias columns, and generated expressions. - Use that metadata to make insert row types more accurate. - Mark server-generated columns as omitted or readonly for inserts. Phase 6: Runtime Guards - Throw a clear error if an adapter does not support inserts. - Normalize single-row inserts into arrays. - Reject empty row arrays unless there is a deliberate no-op policy. - Add useful logging around insert execution and failures. Phase 7: Tests - Unit test adapter insert forwarding. - Unit test public db.insert() behavior. - Type test valid and invalid insert rows. - Integration test insert rows into a temporary ClickHouse table and read them back with the existing query builder. Phase 8: Documentation - Document simple row insertion. - Document batching guidance. - Document supported formats and current limitations. - Add examples for generated schemas and hand-written schemas. Phase 9: Advanced Insert Support - Add optional support for custom insert formats. - Consider insertFromSelect or raw insert helpers. - Consider batch helpers for high-volume ingestion. - Consider dataset-level write APIs only after core insert support is stable.

Luke about 1 month ago

hypequery test package for query testing

A command to re-generate types, identify schema drift, smoke test the queries with performance testing and differences. Something like the below: $ npx hypequery test 🔍 Schema Check ✓ All types match database schema ⚠️ New column 'users.deleted_at' detected 🧪 Smoke Test (running queries) ✓ activeUsers (45ms, 12 rows) ✓ revenueByMonth (120ms, 24 rows) ✓ topCustomers (2100ms, 50 rows) 📊 Performance Check ✓ All queries within thresholds ⚠️ topCustomers is slow (2100ms > 1000ms threshold)

Luke 6 months ago

Completed

Add dynamic dataset api

Context Users can only currently define standalone queries which limits the following use cases : AI Agents: Can't accurately generate queries based on natural language questions, without a pre-defined query Dashboard Builders: Users can't drag-and-drop dimensions/measures BI Tool Integration: Can't accept dynamic SQL/GraphQL queries Multi-Tenant Custom Views: Each tenant can't define their own metrics Goal Create dynamic API for runtime queries, with a separate entry point to queries. I imagine the API looking something like the below: // generated-datasets.ts (auto-generated) export const datasets = { trips: defineDataset({ table: 'trips', dimensions: { pickup_datetime: dimension.datetime('pickup_datetime'), pickup_date: dimension.date('toDate(pickup_datetime)'), payment_type: dimension.string('payment_type'), // ... all columns from schema }, measures: { count: measure.count(), total_revenue: measure.sum('total_amount'), avg_distance: measure.avg('trip_distance'), // ... common aggregations }, }), } as const; // ===== DYNAMIC QUERIES ===== const dynamicResults = await db.queryDataset(datasets.trips, { dimensions: ['pickup_date', 'payment_type'] as const, measures: ['count', 'total_revenue'] as const, }); // Type: Array // ===== FULLY DYNAMIC (No Type Inference) ===== const aiResults = await db.queryDataset(datasets.trips, parseUserInput(query)); // Type: DynamicQueryResult (Record with metadata) // ===== AI AGENT TOOL GENERATION ===== const tools = db.generateTools(datasets.trips, { mode: 'per-query', // One tool per predefined query // OR mode: 'catalog', // Single tool with dimension/measure selection }); export const api = defineServe({ queries: { // Auto-generated from dataset ...datasets.trips.toQueries(), } });

Luke 7 months ago

3

Planned

Expand hypequery dev server feature set

Context Improve developer experience Goal Currently the hypequery dev server just provides documentation for the generated endpoints, it would be great to have further features for tracing, observability and debugging. Query list for inspection (raw sql & query builder syntax) and running via a UI to test the output with parameterised inputs Logs showing run query history, completion status, cache status

Luke 7 months ago

Caching support in serve package

Context Enable first class caching support in defineServe for better control and logging Goal Currently, caching is managed just at the underlying query builder level, which does not enable observability at the query definition layer. By managing this instead where users defined queries we can improve the management of cache controls, observability and reporting

Luke 7 months ago