Topics

#bigquery #browser #data-lineage #dbt #postgresql #privacy #react #rust #snowflake #sql #sql-lineage #sql-parser #typescript #wasm #webassembly

Language breakdown

Rust 68.7%

TypeScript 22.7%

HTML 4.5%

JavaScript 1.8%

CSS 1.0%

Python 1.0%

▸ Files click to expand

README

FlowScope

FlowScope includes a full web application at flowscope.pondpilot.io for interactive, multi-file SQL lineage analysis.

Under the hood, it is a privacy-first SQL lineage engine that runs entirely in the browser. Built with Rust and WebAssembly, it analyzes SQL queries to produce lineage graphs that describe how tables, CTEs, and columns flow through transformations.

The engine is designed for embedding into web apps, browser extensions, and developer tools that need instant lineage analysis without sending SQL to a server.

Getting Started

Web Application

The easiest way to use FlowScope is through the hosted web app — no installation required:

flowscope.pondpilot.io

Features:

Drag and drop SQL files or paste queries directly
Interactive lineage graph with table and column-level views
Multi-file project support with schema DDL
dbt/Jinja template preprocessing for dbt models
Export to Mermaid, JSON, CSV, Excel, or HTML reports
Librarian — AI chat panel that answers questions about your data based on lineage analysis and uploaded PDF docs
All processing happens in your browser — your SQL never leaves your machine

Command-Line Interface

For scripting and CI/CD integration, install the CLI:

# Core CLI (analysis, linting, fixing, exports)
cargo install flowscope-cli
With the bundled local web server (see "Serve Mode" below)
cargo install flowscope-cli --features serve

Basic usage:

# Analyze a SQL file
flowscope query.sql
Analyze with a specific dialect
flowscope -d snowflake etl/*.sql
Generate a Mermaid diagram
flowscope -f mermaid -v column query.sql > lineage.mmd
Export to Excel with schema awareness
flowscope -s schema.sql -f xlsx -o report.xlsx queries/*.sql
Pipe from stdin
cat query.sql | flowscope -d postgres

Output formats: table (default), json, mermaid, html, sql, csv, xlsx, duckdb

Linting

FlowScope includes a SQL linter with 72 rules covering aliasing, layout, conventions, structure, and more:

# Lint SQL files
flowscope --lint queries/*.sql
Lint and auto-fix
flowscope --lint --fix queries/*.sql
JSON output for CI integration
flowscope --lint -f json queries/*.sql

See CLI documentation for all lint options and rule configuration.

Serve Mode (Local Web UI)

Run FlowScope as a local HTTP server with the full web UI embedded in a single binary:

# Start server watching SQL directories
flowscope --serve --watch ./sql
With database schema and custom port
flowscope --serve --watch ./models -d postgres --metadata-url postgres://user@localhost/db --port 8080
Open browser automatically
flowscope --serve --watch ./sql --open

The serve mode watches directories for .sql file changes and provides the same interactive experience as the hosted web app, with all processing happening locally. Requires building with the serve feature.

See CLI documentation for all options.

Key Features

Client-side analysis with zero data egress
Multi-dialect coverage (PostgreSQL, Snowflake, BigQuery, DuckDB, Redshift, and more)
dbt and Jinja templating support with built-in macro stubs (ref(), source(), var())
Table and column lineage with schema-aware wildcard expansion
SQL linting with 72 rules across 9 categories (aliasing, layout, convention, structure, and more)
Auto-fix engine with safe and unsafe fix modes
Structured diagnostics with spans for precise highlighting
Completion API for SQL authoring workflows
TypeScript API and optional React visualization components
Librarian AI chat panel for natural-language Q&A over SQL lineage and uploaded PDF documentation (OpenAI, Anthropic, or custom endpoints)

Components

app/ — the hosted web application at flowscope.pondpilot.io
crates/ — Rust engine, WASM bindings, and CLI
packages/ — TypeScript API and React visualization components

TypeScript API

Install the core package:

npm install @pondpilot/flowscope-core

Analyze a query:

import { initWasm, analyzeSql } from '@pondpilot/flowscope-core';
await initWasm();
const result = await analyzeSql({   sql: 'SELECT * FROM analytics.orders',   dialect: 'postgres', });
console.log(result.statements[0]);

Completion API

Use the completion API to provide SQL authoring hints at a cursor position. See docs/guides/schema-metadata.md for schema setup details.

import {
  charOffsetToByteOffset,
  completionItems,
  initWasm,
} from '@pondpilot/flowscope-core';
await initWasm();
const sql = 'SELECT * FROM analytics.'; const cursorOffset = charOffsetToByteOffset(sql, sql.length);
const result = await completionItems({   sql,   dialect: 'postgres',   cursorOffset,   schema: {     defaultSchema: 'analytics',     tables: [{ name: 'orders', columns: [{ name: 'order_id' }, { name: 'total' }] }],   }, });
console.log(result.items.slice(0, 5));

Visualization

For interactive lineage graphs, add the React package and render the LineageExplorer component. See docs/guides/quickstart.md for a full walkthrough.