Over the past few months, I've been gradually reworking the core concept of the library. Quite a lot of changes piled up along the way, so 4.0 isn't just another set of new features — it's a pretty significant rethink of what Jam is supposed to be in the first place.

In short: Jam used to be more of a collection of separate authentication tools, and now it's gradually turning into a full-fledged authentication/authorization framework.

jam.Jam

The most noticeable change is the completely redesigned jam.Jam facade.

Previously, it was mostly a convenient wrapper around the individual modules:

from jam import Jam

jam = Jam(config=config)

jwt_token = jam.jwt_encode(payload={"sub": 1})
payload = jam.jwt_decode(token=jwt_token)

It worked, but with an API like this, the application still effectively knows that it's using JWT.

In 4.0, I wanted to move away from this level of abstraction. The application should work with auth (x/z) concepts, while the actual mechanism — JOSE, PASETO, sessions, etc. — should remain an implementation detail.

So now the main flow looks roughly like this:

from dataclasses import dataclass

from jam import Jam, BaseSubject


@dataclass
class User(BaseSubject):
    id: int
    email: str
    role: str = "user"


jam = Jam(config=config)

user = User(
    id=1,
    email="some@mail.com",
)

# Issue credentials for the user.
token = jam.issue(subject=user)

# Authenticate the credentials.
principal = jam.authenticate(token=token)

assert principal.subject.id == user.id

# And perform authorization.
allowed = jam.authorize(
    principal=principal,
    permission="post:delete",
)

There are two important entities here: Subject and Principal.

Subject is the entity on whose behalf authentication is performed. In the simplest case, that's a user, but it doesn't necessarily have to be one. It could, for example, be a virtual machine requesting access to certain resources.

Principal is the result of authentication. It contains the subject and the information needed for further authorization.

I like this approach much more: the specific authentication mechanism becomes a replaceable implementation detail, while application code works with a single unified model.

Framework integrations

Along with that, I've also reworked the framework integrations.

For example, with FastAPI it now looks like this:

from typing import Annotated

from fastapi import Depends, FastAPI

from jam import Jam
from jam.authz import Principal
from jam.ext.fastapi import JamAuth


jam = Jam("config.toml")
auth = JamAuth(jam)

app = FastAPI()


@app.get("/me")
def me(
    principal: Annotated[
        Principal,
        Depends(auth), # get the principal
    ],
):
    return principal.subject


@app.get("/landing")
def landing(
    principal: Annotated[
        Principal | None,
        Depends(auth.optional), # optional authentication
    ],
):
    return {
        "authenticated": principal is not None,
    }


@app.patch("/posts/{post_id}")
def edit_post(
    post_id: str,
    principal: Annotated[
        Principal,
        Depends(auth.require("post:edit")), # require a specific permission
    ],
):
    return {
        "post_id": post_id,
        "editor": principal.subject,
    }

Application code gets a ready-to-use Principal and doesn't have to deal with headers, tokens, or the specific authentication mechanism.

You can simply get the current user:

Depends(auth)

You can allow anonymous access:

Depends(auth.optional)

Or you can require a permission right away:

Depends(auth.require("post:edit"))

So authorization can be integrated pretty naturally directly into the application flow.

The individual mechanisms aren't going anywhere

With all of this, I didn't want to turn Jam into a monolith.

The individual modules can still be used independently. If you only need a specific mechanism, you can work with it directly. If you want to build a full authentication flow on top of Jam, that's what Jam is now for.

For me, this is a pretty important part of the architecture: modules should remain replaceable and shouldn't depend on each other more than necessary.

Keychain

But I still consider keychain to be the killer feature of this release.

I wrote about this idea before, but the mechanism changed a bit during implementation.

The problem I wanted to solve is pretty simple. An application shouldn't have to manage the storage and rotation of cryptographic keys itself.

For example, for JWT you can specify a keychain in the configuration:

[jam.jose.jwt]
alg = "RS256"
keychain = "jose_keys"

[jam.keychains.jose_keys]
# For now, only FileStorage and MemoryStorage are available.
# In the future, I want to add HashiCorp Vault,
# while other options can be implemented
# through the public interface.
type = "FileStorage"
path = "/opt/keys"

After that, the application works with the keychain:

from jam import Jam
from jam.keychain import FileStorage


jam = Jam(config="config.toml")

keychain: FileStorage = jam.keychains["jose_keys"]

# Add a key.
#
# If material isn't provided,
# Jam will generate the key itself
# based on the configuration.
keychain.add(
    key_id="my-key-id",
)

# Get the current key.
keychain.current(key_id="my-key-id")

# Issue credentials.
token = jam.issue(
    subject={"sub": 1},
)

# Rotate the key.
keychain.rotate()

What's important here isn't so much the ability to call rotate() itself, but the fact that the application doesn't have to know where the keys are physically stored or which one is currently in use.

For example, imagine an application has been running in production for several months, and at some point you need to replace the private key. Without a keychain, this can easily turn into a manual procedure: find the key files, replace them, figure out what to do with already issued tokens, and make sure validation of old credentials doesn't break. With a keychain, this logic becomes part of the authentication infrastructure. A new key can become the current one, while old keys remain available for validating already issued tokens.

In other words, rotation shouldn't automatically mean: old tokens stop working right now.

This is especially useful for distributed applications, where multiple service instances may temporarily be running with different versions of the configuration.

CLI management

The keychain can also be managed from the command line while the application is running.

For example:

$ pip install jamlib[cli]

$ jam keychain --config jam.toml rotate jose_keys

$ jam keychain --config jam.toml list jose_keys

This makes it possible to perform key operations without having to write a separate deployment script or access the storage directly.

For now, there are two storage backends:

• FileStorage
• MemoryStorage

I want to expand this part further. In particular, I'm planning to add an integration with HashiCorp Vault.

At the same time, I don't want to make a separate implementation for every possible secret storage. For everything else, there will be a public interface that can be used to implement a custom backend.

What changed architecturally

Probably the most important change in 4.0.0 is that Jam is gradually becoming more than just a library with a collection of auth tools.

Previously, application code looked roughly like this:

Application
    |
    +-- JWT
    +-- PASETO
    +-- Sessions
    +-- OAuth2
    +-- ...

Now I want to move toward this model:

Application
    |
    +-- Jam
          |
          +-- Authentication
          |
          +-- Authorization
          |
          +-- JWT
          +-- PASETO
          +-- Sessions
          +-- OAuth2
          +-- ...

The specific mechanisms still remain independent modules that can be used on their own. So you get a certain abstraction layer over authentication, but without trying to hide absolutely everything behind one huge class.

What's next?

In the near-term plans:

• extending keychain support to other modules
• HashiCorp storage
• SAML support in the main jam.Jam facade
• Macaroon + Biscuit tokens
• expanding authz configuration capabilities

GitHub Release | Documentation

So I decided to share my experience and show exactly what I did, so you can reproduce it yourself if needed.

Since this was still a heavily regulated enterprise environment, the question of running an LLM locally, inside the company perimeter, came up almost immediately. Claude, Codex, and other cloud-based solutions were completely off-limits.

What I Had

After several months of negotiations, I was finally given a machine with an NVIDIA H100, and I started figuring out how all of this worked. My previous experience was limited to running models on a home PC, so at first it seemed like everything would be more or less the same. It didn't take long to discover that enterprise infrastructure always has a few surprises waiting for you. My GPU only had 80 GB of VRAM. By modern model standards, that's not a lot, so model selection had to be done carefully.

I considered:

• DeepSeek
• Qwen3
• Qwen3.6
• Qwen3-Coder

After a few tests, I settled on Qwen/Qwen3-Coder-30B-A3B-Instruct because deepseek-ai/deepseek-coder-33b-base (the only reasonably usable DeepSeek model that fit into memory) turned out to be fairly weak, while Qwen3.6 wouldn't start at all because of driver issues.

More on that in a moment.

First vLLM Launch

Like any reasonable person, I started by deploying the model with vLLM and calling it directly through the API.

mkdir vllm && cd vllm

pip install --upgrade pip
pip install uv

uv venv --python 3.12 --seed --managed-python
source ~/vllm/.venv/bin/activate

uv pip install -U \
  "transformers<5" \
  "vllm==0.10.2"

uv run vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct \
  --host 0.0.0.0 \
  --port 8000

That's where the adventure began.

When Enterprise Infrastructure Meets LLMs

Some of you may have noticed that the versions of vllm and transformers are far from the latest. That wasn't accidental. The GPU was provisioned through MIG, and along with it came a whole collection of limitations. The biggest issue was the drivers. I couldn't update them, and due to a combination of architectural constraints and bureaucracy, the administrators couldn't update them either. That's exactly why I never managed to get Qwen 3.6 running.

But the fun didn't stop there. Every night at around 2:00 AM, the MIG instance would disappear from the system for a few seconds and then come back with a new UID. I had no interest in figuring out who was responsible, so I chose the most engineering-oriented solution possible: Every morning at 5:00 AM, the server simply reboots. As a result:

• a fresh UID appears;
• accumulated issues disappear;
• the system gets a preventive reboot before the workday starts.

Yes, it's a hack, but it works.

Why vLLM Alone Isn't Enough

For several days I used the model on my own without any issues. It quickly became obvious, however, that this approach only works for a single person. If other employees started using the model, I would need:

• a web interface;
• user management;
• integration with corporate authentication;
• the ability to revoke access quickly;
• an open-source solution that could be modified internally.

After a bit of research, I settled on OpenWebUI. It had everything I needed and then some.

OpenWebUI as the Entry Point

Initially, OpenWebUI lived on the same machine as vLLM. Very quickly, though, it became obvious that it was consuming resources I'd rather leave available for the model itself. So I deployed a separate virtual machine.

The resulting architecture looked like this: The user interacts with OpenWebUI, and OpenWebUI sends requests to vLLM.

This turned out to be convenient for several reasons:

• access can be restricted to specific users or groups;
• usage statistics can be collected;
• response ratings can be gathered;
• system prompts can be managed centrally.

Squeezing the Most Out of 80 GB of Memory

At this point I started experimenting with model settings. My first idea was to increase the context window. I tried this:

uv run vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct \
  --max-model-len 120000

It didn't work.

After a series of experiments, I arrived at the following configuration:

uv run vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype bfloat16 \
  --max-model-len 70000 \
  --gpu-memory-utilization 0.82 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --served-model-name Qwen-Coder \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --kv-cache-dtype fp8

This gave me roughly 70,000 tokens of context. Unfortunately, it didn't take long to discover that even that wasn't enough.

Hack #1: Rebuilding the Cache After Reboot

The most obvious solution was to use prefix caching. The problem was that every morning the server rebooted, and the entire cache disappeared along with it.

So another hack was born. After startup, the system automatically sends a predefined set of popular requests to the model.

# -*- coding: utf-8 -*-
# /opt/openwebui/scripts/token_recache_service.py

RECACHE_PROMPT: str = load_recache_data()

client.chat.completions.create(
    model="qwen",
    messages=[
        {"role": "system", "content": RECACHE_PROMPT},
        {"role": "user", "content": "warmup"}
    ]
)

In practice, I'm simply forcing the model to recompute the tokens I want cached. The list of popular requests was assembled together with an AI agent and continues to grow over time.

So far, this solution has been sufficient.

Wrapping Everything Into a Service

From there, it was mostly standard infrastructure work.

I deployed PostgreSQL instead of SQLite, configured nginx, issued certificates, and locked down access to the vLLM server as much as possible.

Only OpenWebUI is allowed to communicate with the model.

services:
  postgres:
    image: postgres:17
    container_name: open-webui-postgres
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped

This immediately solved several problems.

• First, nobody can access the model directly anymore.
• Second, if I need to perform maintenance or testing, I can simply disable the model in OpenWebUI.
• Third, the system is no longer limited to developers using VSCode, Zed, or Opencode. Any employee can use it through a browser.

Hack #2: The Model Wants Coffee Too

After some time, I noticed an interesting pattern. If nobody used the model for a long period, the first few requests in the morning performed noticeably worse. Responses took longer, hallucinations became more frequent, and overall behavior felt strange.

After several requests, everything returned to normal. At first I assumed the GPU was simply sitting idle and "cooling down." I did some searching, found that similar observations weren't unique to me, and wrote a simple script that generated a small amount of continuous GPU load.

import time
import torch

DEVICE = "cuda"

a = torch.randn((2048, 2048), device=DEVICE, dtype=torch.float16)
b = torch.randn((2048, 2048), device=DEVICE, dtype=torch.float16)

while True:
    c = torch.matmul(a, b)
    torch.cuda.synchronize()
    time.sleep(10)

It worked. But it felt like I was wasting resources. Later I replaced it with a different approach. Now, every few minutes, the system sends a meaningless request to the model:

import os
import time
import requests

URL = os.getenv("LLM_API")
TIMEOUT = 300

while True:
    try:
        requests.post(
            URL,
            json={
                "model": "qwen",
                "messages": [
                    {"role": "user", "content": "ping"}
                ],
                "max_tokens": 1,
            },
            timeout=30,
        )
    except Exception:
        pass

    time.sleep(TIMEOUT)

As a result, the model never stays idle for too long, and the first real user requests tend to perform much more consistently.

Final Architecture

As a finishing touch, our DevOps engineer and I configured authentication through FreeIPA, issued an internal certificate, and added the service to the corporate DNS.

The final architecture ended up looking like this:

What's Next?

The system is now being actively used within the company and is gradually being enhanced with additional features. As a next step, I want to move the project knowledge into a separate RAG service and focus separately on the long-term storage and restoration of caches. And very soon I’ll have a machine running on an H200, where I’ll be testing more sophisticated models.

But that’s a story for the next article.

I got into the Delta whitelist and spent some time using it for work and not-so-work-related stuff. And I really liked it — this is exactly what I was missing. It’s not some ugly terminal thing written in JS and React, nor a janky stripped-down extension for Zed or VS Code. It’s a native, fast app that does a lot right out of the box.

What does it have that Claude Code, Codex, and OpenCode don’t?

First of all, really damn good code review

Before this, I used revdiff by Umputun, but objectively speaking, it’s a pretty hacky workaround. Opening some TUI program through tmux and terminal overlays, only for it to write something like line_number: comment when you exit — eh, not exactly great.

Here, you just walk through the code, leave comments, and then send them to the agent in one action. It’s very similar to how we do reviews in GitLab or GitHub. Really damn nice.

Second, worktree-based workflow

The folks at Atom — the same people building Zed and Delta — have been pushing Git worktrees for a long time, going all the way back to the Atom days.

I like this approach too. Especially in the age of agentic coding, I don’t really want to let an agent loose directly in my working tree. It’s much more convenient to create a worktree from the current state of the project and let the agent have fun there. After the review, the changes can be moved into the main branch.

That’s exactly what Delta does: each thread gets its own worktree, and all the work happens there. It’s both safer and more convenient, especially when multiple agents are working in parallel.

Third, a rather niche use case, but...

Collaborative work.

I can share a session and work on a task together with a colleague in real time. I really like this feature in Zed itself. At work, I occasionally use it to look at code together with colleagues, and it’s much more convenient than sharing your screen during a call and trying to show someone what you’re doing.

And once coding started turning into chasing AI agents around, it became logical to share not the code itself, but the session: the plans, the reasoning, the same code, and now also the diffs produced by the agent.

Not many people need this, but personally, I consider it one of the best features in both Zed and Delta.

Fourth, NATIVENESS

YES. GOD. YES.

I am so damn tired of this entire zoo of Electron, JS, and TS. Why? What for?

(Ironically, the people who created Zed also created Electron for Atom back in the day, and now they’re trying to do everything natively. Making up for their sins 100%.)

Why, for example, would you write Bitwarden in TypeScript? Both the desktop app and the CLI. Why would you write Claude Code in TypeScript? And React on top of that?

Electron has generally become a genuine plague of modern desktop software, so the fact that Delta is fast and native is a huge plus.

What is it actually missing?

First and foremost, custom providers.

I really want to connect a work model using our own tokens (mostly so the security people don’t come after me) and work through it. Maybe this isn’t available yet simply because Delta is still in closed beta. But I didn’t see a single word about custom providers on the roadmap.

Then, of course, Git integration.

Right now, they suggest moving changes over using stash or checking out commits from the worktree in your terminal outside of Delta. I’m not really against that, but I’d like to have a slightly more convenient way of doing it. Fortunately, this is already on the roadmap, so they’ll probably improve it.

And finally, switching agents.

I’d like to first build a plan with a read-only agent and then hand it over to another agent to actually do the work. That’s not possible yet, but we’ll see how the product evolves.

At some point while working, I caught myself thinking that I don’t really perceive Delta as yet another tool for working with AI — or even as an “agentic IDE.”

With the arrival of full-fledged agents in our workflow, the development process itself has changed. Previously, a code editor was primarily where you wrote code yourself. Now, you open it mostly to read the code the agent wrote — and even that happens pretty rarely, because the diffs in Claude Code / Codex are often enough.

In Delta, all of this is there. You can read the entire project, manually edit the code if you need to, and leave comments for the robot even outside of diffs.

That’s exactly why Delta feels less like yet another little chat window bolted onto an editor and more like an attempt to build the next generation of code editor.

Here, the agent isn’t an add-on to the familiar workflow anymore. It becomes one of the core entities of the system, alongside the code (or perhaps even above it), Git, and the developer.