GPTPortal

Prompt caching (Claude, optional)

Claude can cache a stable prefix of your prompt so you're not billed full price for the same long system prompt every turn. See Prompt caching.

| Variable | Default | Description | |---|---|---| | CLAUDECACHEENABLED | false | Turn caching on. | | CLAUDECACHESTRATEGY | conservative | How aggressively to place cache breakpoints. | | CLAUDECACHEANALYTICS | true | Log cache hit/miss analytics. | | CLAUDECACHEMAX_BREAKPOINTS | 4 | Maximum cache breakpoints (0–10). | | CLAUDECACHEDEFAULT_PREFERENCE | auto | Default caching preference. |

Custom OpenAI-compatible endpoints (optional)

Point the portal at a local or remote OpenAI-compatible server. Fully opt-in — omit these and nothing changes. See Custom and local endpoints.

| Variable | Description | |---|---| | CUSTOMOPENAIBASE_URL | Base URL of one endpoint, e.g. http://localhost:11434/v1 for Ollama. | | CUSTOMOPENAIAPI_KEY | Key for that endpoint (often a dummy value for local servers). | | CUSTOMOPENAIPREFIX | Routing prefix for its model ids, e.g. ollama. Default local. | | CUSTOMOPENAILABEL | Display label in the model picker, e.g. Ollama. | | CUSTOM_ENDPOINTS | A JSON array to configure several endpoints at once (takes precedence over the single-endpoint variables above). |

Server and security

| Variable | Default | Description | |---|---|---| | PORT_SERVER | 3000 | Port the server listens on. | | HOST | 0.0.0.0 | Bind address. 0.0.0.0 accepts connections from your local network; use 127.0.0.1 to restrict to this machine only. | | NODE_ENV | development | Set to production to enable secure session cookies. | | LOG_LEVEL | info | Logging verbosity. | | SESSION_SECRET | random | Secret for signing session cookies. Set a fixed value in production so sessions survive restarts. | | RATELIMITMAX | 100 | Max requests per IP per 15 minutes (general). | | APIRATELIMIT_MAX | 50 | Max requests per IP per 15 minutes for /api/ routes. | | ALLOWED_ORIGINS | * | Comma-separated CORS allowlist. Restrict this if you expose the portal beyond localhost. |

Providers and models

The core catalog lives in public/js/data/models.json and is organized into these categories. Model ids evolve quickly; treat the list below as a snapshot of the families present, and open the picker for the exact current ids.

| Category | What's in it | Examples | |---|---|---| | GPT Models | OpenAI's GPT line and small/fast variants | gpt-5.5, gpt-5.5-pro, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-5.3-codex, gpt-4.1, gpt-4o | | Claude Models | Anthropic Claude | claude-opus-4-8, claude-opus-4-7, claude-sonnet-5, claude-haiku-4-5, claude-fable-5 | | Gemini Models | Google Gemini 3 line | gemini-3.5-flash, gemini-3.1-pro-preview, gemini-3.1-flash-lite | | Reasoning Models | Dedicated reasoning models | o4-mini | | DeepSeek & Kimi | DeepSeek + Moonshot | deepseek-v4-pro, deepseek-v4-flash, kimi-k2.6, kimi-k2.7-code, kimi-k2.5 | | Mistral Models | Mistral family | mistral-large-latest, mistral-medium-latest, mistral-small-latest, magistral-medium-latest, codestral-latest | | Groq Models | Fast open models via Groq | llama-3.3-70b-versatile, llama-3.1-8b-instant, openai/gpt-oss-120b, openai/gpt-oss-20b | | Grok Models | xAI Grok | grok-4.3, grok-build-0.1 | | Voice Models | Speech-to-text and text-to-speech | gpt-4o-transcribe, gpt-4o-mini-transcribe, gpt-4o-mini-tts | | Image Models | Image generation | gpt-image-2, gpt-image-1.5, gpt-image-1-mini, plus Gemini image models |

Beyond the core catalog you also get:

• OpenRouter — with an OPENROUTERAPIKEY, the full OpenRouter model list is fetched live, cached for an hour, and merged into the picker. Toggle "show OpenRouter models" in the picker to include them. Capability flags (vision, tools, reasoning) and pricing come straight from OpenRouter's metadata.
• Your own endpoints — see the next two sections.

A note on choosing keys

You do not need every key. A reasonable starting point:

• OpenAI if you want the broadest single-provider coverage (chat, reasoning, vision, image, and voice all come from one key).
• Anthropic for Claude, which many people prefer for writing and code.
• Groq for genuinely fast, often free open models — a good "no-cost" option to try the portal.
• OpenRouter if you'd rather hold one key and reach everything else through it.

Using the portal

Sending messages and streaming

Type in the composer and press Enter or click Send (Ctrl/Cmd + Enter also sends). The answer streams in token by token. For models that expose their chain of thought, the reasoning streams into a separate collapsible block above the answer, and the final answer is rendered with full Markdown, code highlighting, and math once it completes.

Choosing a model

Click the model selector at the top. You can type to filter by name or family, and each model shows its provider. If you have an OpenRouter key, toggle its models on to search the full catalog. The model you pick applies to your next message; you can switch between messages. (One exception: once you've sent an image to a vision model, switching models mid-conversation can lose the image context — export and start fresh if you need to change models after an image.)

Reasoning effort, verbosity, temperature, and tokens

Open Model parameters in the composer to reveal the sliders and dropdowns:

• Temperature (0–2): randomness. Lower is steadier and more repetitive; higher is more varied.
• Max tokens: the ceiling on the response length, capped per-model.
• Reasoning effort (for reasoning-capable models): none through xhigh. Higher effort produces better reasoning but is slower and costs more.
• Verbosity (for models that support it): how long and detailed the answer is.

Presets

The Presets button (sliders icon) opens a panel where you can save the current system prompt plus temperature, max tokens, reasoning effort, and verbosity as a named bundle, then reapply it later with one click. Presets are stored in your browser's local storage. Applying a preset updates the composer controls immediately; because the system prompt is a server-side file, a preset's prompt takes effect on your next new conversation.

Compare

The Compare button (two-columns icon) lets you send one prompt to several models at once (up to six) and read the answers side by side, each with its own cost. Compare runs are stateless — they don't touch your main conversation — so it's a safe way to see how models differ on the same question before committing to one.

Vision (image input)

Click the attach button, choose an image, and send it with a message to a vision-capable model. The model receives the image and your text together.

File uploads

Attach a text-based file the same way. Its filename and contents are appended to your message, which is handy for pasting in code, logs, or documents without copying by hand. All UTF-8 text file types are supported.

Voice

Click the microphone to record; speak; click again to stop. Your speech is transcribed (and copied to your clipboard), placed in the input, and — after the model responds — the reply is read back to you. Voice needs a Chromium browser or Firefox; it does not work in Safari because of WebKit codec limitations.

Image generation

Prefix a message with Generate: followed by your prompt to create an image inline, e.g. Generate: a watercolor fox in a snowy forest. The image appears in the chat and can be downloaded. This uses the current image models (such as gpt-image-2).

Assistants mode (OpenAI)

Toggle Assistants Mode below the chat to switch into OpenAI's Assistants API, which keeps a persistent thread and can run code and read uploaded files. Your first message creates an assistant and a thread; you can reuse them across sessions by putting their ids in .env (ASSISTANTID, THREADID). This is a distinct, stateful workflow from the normal chat path and is best treated as its own mode.

Context, tokens, and cost

The composer shows a live indicator of how full the model's context window is and what the conversation has cost so far, computed from the catalog's per-model pricing. This is your early warning before a long conversation gets expensive or bumps the context limit.

Editing the system prompt and environment

Two buttons in the header open in-browser editors: one for the system instructions (public/instructions.md, the prompt prepended to your conversations) and one for your .env. Editing instructions here is the intended way to give the models a persistent persona or background about you.

Exporting

The export button downloads the current conversation as a standalone, Markdown-rendered HTML file. Exporting no longer shuts the server down — it's a pure download.

Theme

The portal auto-detects light or dark from your OS. The sun/moon button toggles it manually, and your choice is remembered.

Keyboard shortcuts

| Shortcut | Action | |---|---| | Ctrl/Cmd + Enter | Send the message | | Ctrl/Cmd + L | Clear the chat | | Ctrl/Cmd + S | Export the conversation | | Shift + Esc | Focus the input box | | Esc | Close the model dropdown | | Ctrl/Cmd + Shift + V | Toggle voice mode | | Ctrl/Cmd + Shift + F | Open the file picker | | Ctrl/Cmd + Shift + A | Toggle Assistants mode |

Custom and local endpoints (Ollama, LM Studio, vLLM)

Any server that speaks the OpenAI Chat Completions format can be plugged in without writing code. This is how you run fully local, private models — Ollama, LM Studio, vLLM, LocalAI — or point at a remote OpenAI-compatible gateway.

You declare endpoints in .env. The single-endpoint form:

CUSTOMOPENAIBASE_URL=http://localhost:11434/v1
CUSTOMOPENAIAPI_KEY=ollama
CUSTOMOPENAIPREFIX=ollama
CUSTOMOPENAILABEL=Ollama

Or several endpoints at once, as a one-line JSON array:

CUSTOM_ENDPOINTS=[{"prefix":"ollama","baseURL":"http://localhost:11434/v1","apiKey":"ollama","label":"Ollama"},{"prefix":"lmstudio","baseURL":"http://localhost:1234/v1","apiKey":"lm-studio","label":"LM Studio"}]

Once configured, the portal queries each endpoint's model list and folds those models into the picker under Local / Custom Endpoints. You address a model as <prefix>/<model> — for example ollama/llama3.2 — and it routes to the right endpoint automatically. Local servers that need no API key can leave apiKey blank. This path supports streaming just like the hosted providers.

Because these models run on your own hardware, they are free to use and your prompts never leave your machine.

Prompt caching (Claude)

Long conversations with a big, stable system prompt get expensive because you re-send that prompt every turn. Anthropic's prompt caching lets Claude cache a prefix of the request so repeated content is billed at a large discount. GPTPortal integrates this: enable it with CLAUDECACHEENABLED=true (and tune the related CLAUDECACHE* variables), or toggle it per session with the cache control in the model picker when a Claude model is selected. The promptCacheService decides where to place cache breakpoints based on your chosen strategy. If you routinely use a large custom system prompt with Claude, turning this on can noticeably reduce cost with no change to output.

Streaming, and how a message actually flows

For the curious or the contributing, here is the path a normal chat message takes:

• The browser (chatManager.js) posts to POST /message and asks for a streaming response (Accept: text/event-stream).
• routes/chat.js builds the request payload from your session, then asks providerFactory whether the chosen model can stream. If it can, the route opens a Server-Sent Events stream.
• providerFactory.handleRequestStream() dispatches to the right handler's streamCompletion(), which opens the provider's own stream and yields normalized events: thinking deltas, text deltas, a final usage event, or an error.
• The frontend renders text deltas into the answer bubble and thinking deltas into the reasoning block as they arrive, then runs the full Markdown/highlight/math pipeline once the answer completes and reconciles the final token usage and cost.
• The assistant's message is appended to the session history so the next turn has context.

HTTP API reference

All routes are behind Basic Auth. These are the ones worth knowing if you want to script against the portal or understand the frontend.

| Method | Path | Purpose | |---|---|---| | POST | /message | Main chat. JSON body: { message, modelID, temperature?, tokens?, reasoningEffort?, verbosity?, image? }. Returns JSON { text, usage }, or a Server-Sent Events stream when stream: true or Accept: text/event-stream. | | POST | /compare | Fan one prompt out to several models. Body: { message, models: [...], temperature?, tokens?, reasoningEffort?, verbosity? }. Returns { results: [{ modelID, success, content, usage, cost, error }] }. Stateless. | | POST | /reset | Clear the conversation state for this session. | | POST | /generate-image | Generate an image from a prompt. | | POST | /transcribe | Speech-to-text. | | POST | /tts | Text-to-speech. | | POST | /upload-file, /upload-image | Attach a file or image for the next message. | | GET | /api/models | The full model catalog (add ?format=frontend for the picker's shape). Also /api/models/core, /openrouter, /categories, /search, /statistics. | | GET / POST | /get-instructions / /update-instructions | Read or write the system prompt. | | GET / POST | /get-my-env / /update-my-env | Read or write .env from the browser editor. | | GET | /health, /api/system/status | Health and service status. | | POST | /shutdown-server | Intentionally stop the server (used by the setup/save flow). |

There is also a legacy POST /gemini route retained for backward compatibility; new work should use /message, which routes Gemini like every other provider.

Extending: adding models and providers

The most common task is adding a model to a provider that already exists. Because routing reads the catalog, that is usually a single edit to public/js/data/models.json: add the model entry with the correct provider and category, and it will route, price, and appear in the picker on its own. Run npm test afterward to validate the catalog.

Adding a whole new provider means writing a handler in src/server/services/providers/, registering it in providerFactory.js, and adding its key to the config. Adding a new OpenAI-compatible endpoint (as opposed to a new provider) needs no code at all — use the CUSTOM_* variables.

The full, step-by-step guide with the exact files, the handler contract, the streaming interface, and the catalog schema lives in MODEL_UPDATE.md. Start there.

Project structure

GPTPortal/
├── server.js                     # Entry point; boots src/server/core/Application
├── package.json                  # Dependencies and scripts (start, test)
├── .env / .env.example           # Configuration (yours is git-ignored)
├── Dockerfile / docker-compose.yml
├── vercel.json                   # Serverless deployment config
├── MODEL_UPDATE.md               # Guide for adding models/providers
├── scripts/
│   └── smoke-test.js             # Catalog + routing validation (npm test)
├── public/                       # Frontend (served statically, no build step)
│   ├── portal.html               # The app shell
│   ├── setup.html                # First-run key entry page
│   ├── chat.css                  # The single design-token stylesheet
│   ├── instructions.md           # System prompt (editable in-app)
│   ├── claudeInstructions.xml    # Claude-specific instructions
│   ├── js/
│   │   ├── app.js                # Frontend bootstrap
│   │   ├── data/models.json      # The model catalog (source of truth)
│   │   ├── modules/              # chatManager, messageHandler, uiManager,
│   │   │                         #   dynamicModelManager, modelUI, modelSearch,
│   │   │                         #   portalInit, presets, compare
│   │   └── services/             # contextTracker, tokenCounterClient
│   └── uploads/                  # Saved chats, prompts, and uploaded files
└── src/
    ├── server/
    │   ├── core/                 # Application, ServiceManager, RouteManager, MiddlewareManager
    │   ├── routes/               # chat, gemini, assistant, models, setup, config
    │   ├── services/             # providers/**, cost, token, context, export, modelRegistry, ...
    │   ├── middleware/           # auth, upload
    │   └── utils/                # Logger, validation, error handling
    └── shared/
        └── modelLoader.js        # Loads/caches the catalog

Deployment

Local (recommended)

The default. node server.js, reachable at http://localhost:3000/portal. If you want it available to other devices on your network, keep HOST=0.0.0.0 and connect from another machine using this computer's LAN IP; if you want it strictly private, set HOST=127.0.0.1.

Docker

A Dockerfile (multi-stage, production dependencies only, non-root user, dumb-init as PID 1) and a docker-compose.yml are included.

Build and run directly:

docker build -t gptportal .
docker run -p 3000:3000 --env-file ./.env --name gptportal gptportal

Or with Compose (which mounts your local .env into the container):

docker compose up

The container exposes port 3000. Adjust the port mapping if you change PORT_SERVER.

Vercel and other serverless hosts

vercel.json is included and the app can run as a serverless function, but understand the trade-offs before relying on it: serverless instances are ephemeral and don't hold long-lived in-memory session state, streaming behaves differently behind some serverless proxies, and you'd be exposing your keys to a hosted environment. For a personal tool, running it locally or on a small always-on server (a VPS, a home server, a Raspberry Pi) matches the app's design far better than serverless does.

Wherever you deploy beyond your own machine, set a strong USERPASSWORD, a fixed SESSIONSECRET, NODEENV=production, and a restrictive ALLOWEDORIGINS.

Security and privacy

• Your keys stay in .env, which is git-ignored. The repository cannot read or transmit them. The only places your keys go are the provider APIs you call.
• Your conversations stay local. History is held in your server-side session and, if you export or save, in files under public/uploads/. The app doesn't phone home. The only data leaving your machine is the content of the messages you send to whichever provider you selected — and, for local endpoints, not even that.
• Access is gated by HTTP Basic Auth using USERUSERNAME / USERPASSWORD, applied to every route.
• Rate limiting is on by default (general and stricter /api/ limits), configurable via RATELIMITMAX and APIRATELIMIT_MAX.
• Security headers (a Content-Security-Policy plus frame, MIME-sniffing, and referrer protections) are set on every response.
• trust proxy is set to loopback, which is correct for a local-first app. If you deploy behind a known reverse proxy, widen it deliberately.
• A caution about exposure. Basic Auth over plain HTTP sends credentials in an easily reversible encoding. If you make the portal reachable beyond your own machine, put it behind HTTPS (a reverse proxy such as Caddy or nginx with a certificate) rather than exposing it directly.

Cost management

You pay each provider directly for what you use, metered in tokens. The portal helps you stay aware of this in two ways: it shows a live cost estimate for the current conversation (from the catalog's per-model input/output pricing), and it shows how full the context window is.

The dynamic worth understanding is context accumulation. Each turn re-sends the whole conversation so far as input, so cost grows as the conversation lengthens even if your individual messages stay short:

• Turn 1 bills your system prompt + your message (input) and the reply (output).
• Turn 2 bills the system prompt + turn 1's message and reply + your new message (all input now) plus the new reply (output).
• And so on. The input side compounds; a long thread with an expensive model is where the money goes.

• Match the model to the task. Use a small, cheap, or free model (a Groq open model, a -mini/-nano variant, or a local endpoint) for routine questions, and reserve the expensive flagships for work that needs them.
• Reset when you change topics. Starting a fresh conversation drops the accumulated context you no longer need.
• Watch the live cost and context indicators. They're there precisely so a long session doesn't surprise you.
• Turn on prompt caching for Claude if you use a large, stable system prompt.
• Use Compare deliberately — it multiplies cost by the number of models you select — but it's cheap and genuinely useful for picking the right model for a task up front.

Troubleshooting and FAQ

The server won't start: EADDRINUSE. Port 3000 is already in use. Either stop whatever is using it, or set a different PORT_SERVER in .env.

The browser keeps asking me to log in, or rejects my credentials. The username and password come from USERUSERNAME and USERPASSWORD in .env. Make sure both are set and that you restarted the server after editing .env.

A model returns an error or a 400. Usually one of: the provider's key isn't set (so the model can't route), the key is out of credit, or you selected a model your key doesn't have access to. Check the server logs — they name the provider and the underlying error. Also confirm the model still exists at the provider; model ids are retired regularly.

A whole provider's models are missing or won't respond. That provider's key isn't configured. Each provider only initializes when its key is present; the startup logs list which ones came up.

Cannot find module on startup. Run npm install again. This happens after an update that added a dependency, or if the first install was interrupted.

Streaming doesn't show; the answer appears all at once. Some models and providers don't support streaming and fall back to a single response — that's expected. If nothing streams, check that a proxy in front of the app isn't buffering text/event-stream responses.

Voice doesn't work. You're probably in Safari. WebKit lacks the codecs the recorder uses; switch to a Chromium browser or Firefox.

Where do I add my key — I can't see the .env file? On macOS, dotfiles are hidden in Finder; press Cmd + Shift + . to reveal them, or use the in-app .env editor, or edit the file from your terminal. On Windows it should be visible. The most common setup mistake is leaving the file named .env.example instead of .env.

Is my data safe? Your keys and conversations stay on your machine; the only outbound traffic is the API calls you make to the providers you chose (and none, for local endpoints). See Security and privacy.

How do I update to a newer version? From the project folder, git pull origin main, then npm install if dependencies changed, then restart. Your .env is preserved because it's git-ignored.

Can I run models that never touch the internet? Yes — configure a local endpoint (Ollama, LM Studio, vLLM) as described above. Those models run on your hardware and your prompts don't leave the machine.

Testing

A smoke test validates the model catalog and routing without needing any API keys:

npm test

It checks that models.json is valid, has no duplicate ids, that every model has the required fields and non-zero pricing, that categories are consistent, that no known-retired ids linger, that every model routes to a handler, and that the streaming helpers are exported. Run it after any change to the catalog or the provider routing.

Repository documents