Siftly

Self-hosted Twitter/X bookmark manager with AI-powered organization

Import · Analyze · Categorize · Search · Explore

What is Siftly?

Siftly turns your Twitter/X bookmarks into a searchable, categorized, visual knowledge base — running entirely on your machine. No cloud, no subscriptions, no browser extensions required. Everything stays local except the AI API calls you configure.

It runs a 4-stage AI pipeline on your bookmarks:

📥 Import (built-in bookmarklet or console script — no extensions needed)
    ↓
🏷️  Entity Extraction   — mines hashtags, URLs, mentions, and 100+ known tools from raw tweet data (free, zero API calls)
    ↓
👁️  Vision Analysis      — reads text, objects, and context from every image/GIF/video thumbnail (30–40 visual tags per image)
    ↓
🧠 Semantic Tagging     — generates 25–35 searchable tags per bookmark for AI-powered search
    ↓
📂 Categorization       — assigns each bookmark to 1–3 categories with confidence scores

After the pipeline runs, you get:

• AI search — find bookmarks by meaning, not just keywords ("funny meme about crypto crashing")
• Interactive mindmap — explore your entire bookmark graph visually
• Filtered browsing — grid or list view, filter by category, media type, and date
• Export tools — download media, export as CSV / JSON / ZIP

Quick Start

Prerequisites

• Node.js 18+
• npm (comes with Node.js)

Option A — One command (recommended)

git clone https://github.com/viperrcrypto/Siftly.git
cd Siftly
./start.sh

start.sh installs dependencies, sets up the database, checks for Claude CLI auth, and opens http://localhost:3000 automatically.

Option B — Using Claude Code

If you're using Claude Code to set up the project, it will read CLAUDE.md and know exactly how to get started. Just open the project folder:

git clone https://github.com/viperrcrypto/Siftly.git
claude Siftly/

Claude Code will handle setup and start the app using your existing Claude subscription — no extra configuration needed.

Option C — Manual setup

git clone https://github.com/viperrcrypto/Siftly.git
cd Siftly
npm install
npx prisma generate
npx prisma migrate dev --name init
npx next dev

Open http://localhost:3000

AI Authentication

Siftly automatically detects the best available auth method — no configuration needed in the most common case.

Priority order

| # | Method | How | |---|--------|-----| | 1 | Claude Code CLI (zero config) | Already signed in? Siftly reads your session from the macOS keychain automatically | | 2 | API key in Settings | Open Settings in the app and paste your key | | 3 | ANTHROPICAPIKEY env var | Set in .env.local or your shell environment | | 4 | Local proxy | Set ANTHROPICBASEURL to any Anthropic-compatible endpoint |

Claude Code CLI (no API key needed)

If you use Claude Code, you're already signed in. Siftly detects your session from the macOS keychain and uses your Claude subscription (Free/Pro/Max) automatically.

The Settings page shows a green "Claude CLI detected — no API key needed" badge with your subscription tier when this is active.

Note: This works on macOS. On Linux/Windows, add an API key in Settings instead.

Getting an API key (if needed)

• Go to console.anthropic.com
• Create a new API key
• Open Siftly → Settings → paste it in

Importing Your Bookmarks

Siftly has built-in import tools — no browser extensions required. Go to the Import page and choose either method:

Method A — Bookmarklet (Recommended)

• Go to Import in the Siftly sidebar
• Drag the "Export X Bookmarks" link to your browser's bookmark bar

• Go to x.com/i/bookmarks while logged in to X
• Click "Export X Bookmarks" in your bookmark bar — a purple button appears on the page
• Click "▶ Auto-scroll" — the tool scrolls through and captures all your bookmarks automatically
• When complete, click the purple "Export N bookmarks" button — bookmarks.json downloads
• Back in Siftly → Import → drop or upload the file

Method B — Browser Console Script

• Go to x.com/i/bookmarks while logged in to X
• Open DevTools: press F12 (Windows/Linux) or ⌘⌥J (Mac), then go to the Console tab
• Copy the console script from the Siftly Import page, paste it into the console, and press Enter
• Click "▶ Auto-scroll" and wait for all bookmarks to be captured
• Click the export button — bookmarks.json downloads automatically
• Back in Siftly → Import → upload the file

Re-importing

Re-import anytime — Siftly automatically skips duplicates and only adds new bookmarks.

AI Categorization

Categorization starts automatically as soon as you import. You can also trigger it manually from:

• The Import page (after upload)
• The Mindmap page (when bookmarks are uncategorized)
• The Categorize page in the sidebar

The 4-Stage Pipeline

| Stage | What it does | |-------|-------------| | Entity Extraction | Mines hashtags, URLs, @mentions, and 100+ known tool/product names from stored tweet JSON — free, zero API calls | | Vision Analysis | Analyzes every image, GIF, and video thumbnail — OCR text, objects, scene, mood, meme templates, 30–40 visual tags per image | | Semantic Tagging | Generates 25–35 precise search tags per bookmark by combining tweet text + image context. Also extracts sentiment, people, and company names. | | Categorization | Assigns 1–3 categories per bookmark with confidence scores using all enriched data |

The pipeline is incremental — if interrupted, it picks up where it left off. Use "Re-run everything (force all)" to re-analyze bookmarks that were already processed.

Features

🔍 AI Search

Natural language queries across all bookmark data:

• "funny meme about crypto crashing"
• "react hooks tutorial"
• "bitcoin price chart"
• "best AI coding tools"

🗺️ Mindmap

Interactive force-directed graph showing all bookmarks organized by category:

• Expand/collapse any category to reveal its bookmarks
• Click a bookmark node to open the original tweet on X
• Color-coded legend by category
• If bookmarks aren't categorized yet, an inline AI Categorize button starts the pipeline without leaving the page

📚 Browse & Filter

• Grid view (masonry layout) or List view
• Filter by category, media type (photo / video), or search text
• Sort by newest or oldest
• Pagination with 24 items per page
• Active filter chips — removable individually or all at once
• Hover any card to download media or jump to the original tweet

⚙️ Categories

8 default categories pre-seeded with AI-readable descriptions:

| Category | Color | |----------|-------| | Funny Memes | Amber | | AI Resources | Violet | | Dev Tools | Cyan | | Design | Pink | | Finance & Crypto | Green | | Productivity | Orange | | News | Indigo | | General | Slate |

Create custom categories with a name, color, and optional description. The description is passed directly to the AI during categorization — the more specific, the more accurate the results.

📤 Export

• CSV — spreadsheet-compatible with all fields
• JSON — full structured data export
• ZIP — exports a category's bookmarks + all media files with a manifest.csv

⌨️ Command Palette

Press Cmd+K (Mac) or Ctrl+K (Windows/Linux) to search across all bookmarks from anywhere in the app.

Configuration

All settings are manageable in the Settings page at /settings or via environment variables:

| Setting | Env Var | Description | |---------|---------|-------------| | Anthropic API Key | ANTHROPICAPIKEY | Optional if Claude CLI is signed in — otherwise required for AI features | | API Base URL | ANTHROPICBASEURL | Custom endpoint for proxies or local Anthropic-compatible models | | AI Model | Settings page only | Haiku 4.5 (default, fastest/cheapest), Sonnet 4.6, Opus 4.6 | | OpenAI Key | OPENAIAPIKEY | Alternative provider — GPT-4.1 Mini/Nano/Full, o4-mini, o3 | | MiniMax Key | MINIMAXAPIKEY | Alternative provider — M2.7 (1M context), M2.5, M2.5-highspeed | | MiniMax Base URL | MINIMAXBASEURL | Custom MiniMax API endpoint (default: https://api.minimax.io/v1) | | Database | DATABASE_URL | SQLite file path (default: file:./prisma/dev.db) |

Custom API Endpoint

Point Siftly at any Anthropic-compatible server:

ANTHROPICBASEURL=http://localhost:8080

Architecture

siftly/
├── app/
│   ├── api/
│   │   ├── analyze/images/   # Batch image vision analysis (GET progress, POST run)
│   │   ├── bookmarks/        # List, filter, paginate, delete
│   │   │   └── [id]/categories/ # Per-bookmark category management
│   │   ├── categories/       # Category CRUD
│   │   │   └── [slug]/       # Individual category operations
│   │   ├── categorize/       # 4-stage AI pipeline (start, status, stop)
│   │   ├── export/           # CSV, JSON, ZIP export
│   │   ├── import/           # JSON file import with dedup + auto-pipeline trigger
│   │   │   ├── bookmarklet/  # Bookmarklet-specific import endpoint
│   │   │   └── twitter/      # Twitter-specific import endpoint
│   │   ├── link-preview/     # Server-side OG metadata scraper
│   │   ├── media/            # Media proxy/download endpoint
│   │   ├── mindmap/          # Graph nodes + edges for visualization
│   │   ├── search/ai/        # Natural language semantic search (FTS5 + Claude)
│   │   ├── settings/         # API key + model config
│   │   │   ├── cli-status/   # Claude CLI auth detection endpoint
│   │   │   └── test/         # API key validation endpoint
│   │   └── stats/            # Dashboard stats
│   ├── ai-search/            # AI search page
│   ├── bookmarks/            # Browse, filter, paginate
│   ├── categories/           # Category management
│   │   └── [slug]/           # Category detail page
│   ├── categorize/           # Pipeline monitor with live progress
│   ├── import/               # 3-step import flow (instructions → upload → categorize)
│   ├── mindmap/              # Interactive graph
│   ├── settings/             # Configuration
│   └── page.tsx              # Dashboard
│
├── components/
│   ├── mindmap/              # Mindmap canvas, nodes, edges
│   │   ├── mindmap-canvas.tsx
│   │   ├── category-node.tsx
│   │   ├── tweet-node.tsx
│   │   ├── root-node.tsx
│   │   ├── chain-edge.tsx
│   │   └── mindmap-context.ts
│   ├── command-palette.tsx   # Cmd+K global search
│   ├── nav.tsx               # Sidebar navigation
│   └── theme-toggle.tsx      # Light/dark mode
│
├── lib/
│   ├── categorizer.ts        # AI categorization logic + default categories
│   ├── claude-cli-auth.ts    # Claude CLI OAuth session detection (macOS keychain)
│   ├── vision-analyzer.ts    # Image analysis + batch semantic tagging
│   ├── image-context.ts      # Shared image context builder
│   ├── fts.ts                # SQLite FTS5 full-text search index
│   ├── rawjson-extractor.ts  # Entity extraction from raw tweet JSON
│   ├── parser.ts             # Multi-format JSON parser
│   ├── exporter.ts           # CSV, JSON, ZIP export
│   ├── types.ts              # Shared TypeScript types
│   └── db.ts                 # Prisma client singleton
│
├── prisma/
│   └── schema.prisma         # SQLite schema
│
├── start.sh                  # One-command launcher (install + DB setup + open browser)
└── CLAUDE.md                 # Instructions for Claude Code AI assistant

Database Schema

Bookmark          — tweet text, author, date, raw JSON, semantic tags, enrichment metadata
  ├── MediaItem   — images / videos / GIFs with AI-generated image tags
  └── BookmarkCategory — category assignments with confidence scores (0–1)
Category          — name, slug, hex color, AI-readable description Setting           — key-value store (API keys, model preferences) ImportJob         — tracks import file status and progress

Prisma + SQLite + FTS5

Siftly uses Prisma migrations for relational schema changes. In development, run npx prisma migrate dev --name <change-name> when schema changes. For runtime/prod-style startup, apply committed migrations with npx prisma migrate deploy. FTS5 (bookmarkfts) is managed at runtime in lib/fts.ts, not in schema.prisma. This is intentional for now because Prisma does not model SQLite virtual table definitions directly.

# One-command start (installs, sets up DB, opens browser)
./start.sh
Or manually:
npm install
npx prisma generate
npx prisma migrate dev --name init
npx next dev
Type check
npx tsc --noEmit
Open database GUI
npx prisma studio
Build for production
npm run build && npm start

{
  name: 'My Category',       // Display name
  slug: 'my-category',       // URL-safe identifier (must be unique)
  color: '#6366f1',          // Hex color shown in UI
  description: '...',        // Natural language description — used verbatim in AI prompts
}