Jev-Gateway Guide · Organized AI

Chapter 1 of 4

The design: two layers, one job each

Why the Worker and the gateway are kept apart, who calls the Worker, and the routes and rule it runs.

[ 01 ] · Module 1 of 12

Why a Worker and a gateway

Know which layer owns the decisions and which owns the hop to each model.

The two layers do different jobs, and keeping them apart is the main design idea.

jev-gateway (the Worker) knows your business: which skills exist, how to phrase a selection question, how to score answers into a pick, which backend to try first, what to do when it fails, and what to write down afterwards.

AI Gateway knows nothing about skills. It is a proxy that sits in front of every model call. It gives each backend a stable URL, logs every request and response, counts tokens and errors, and can cache, rate-limit or require a token. You get one dashboard for every model, whether the model is hosted, in a container, or on a Mac mini.

Worker decides jev-gateway

Inbound auth for callers
Prompt building and batching
Selection rule and scoring
Backend order and fallback
D1 audit, KV cache, events

Gateway delivers AI Gateway

One URL per provider slug
Request and response logs
Latency, error and usage analytics
Gateway auth token
Optional cache, retries, rate limits

Backends answer models

Jev: hosted model
Laya: container on Cloudflare
Clef: Clef-Flash on the Mac mini
All accept the same SystemOne-style body

Public check: GET /health on jev-gateway returns the route list and each backend's status without a token. Because every backend speaks the same request shape, swapping the backend order is a config change (SELECT_BACKENDS), not a code change.

Key ideas

Check yourself · +15 XP

Which layer decides which backend to try first and what to do when it fails?

[ 02 ] · Module 2 of 12

Who calls jev-gateway

Know who sends requests and what they get back.

Callers never know which model answered unless they read the backend field in the response. They hold one token for jev-gateway and nothing else.

skill-router

Asks /v1/select-skill which skill fits a task, then runs it. It is the main consumer of the pick, shortlist and none outcomes.

Bearing

Uses the same selection and decision calls to steer work. It benefits from the KV cache when it repeats the same question.

stape-agent-toolset

Chooses a Stape template (GTM tags and server-side tracking) for a job. The stape-select bench suite measures exactly this choice.

Rubric runs

Send claims to /v1/systemone for rubric checks, answered by Jev or Laya. The daily rubric report is written from these runs.

Key ideas

Check yourself · +15 XP

How does a caller find out which model answered?

[ 03 ] · Module 3 of 12

The Worker and its routes

Know the routes, the selection rule and the config that steers routing.

The repo lives at ~/jev-gateway on the M1. The Clef work is on branch skill-selection-and-clef (commits d054385 and 72404f3), which is not yet pushed.

Route	What it does	Backends	Writes
`POST /v1/select-skill`	Turns a task and a list of candidate skills into batched questions, asks a backend, and applies the selection rule.	clef, then jev	D1 `skill_selections`, KV `sk:`, events
`POST /v1/systemone`	Checks claims against a rubric. Pins and calibration decide which backend answers and how its scores are read.	jev, laya	rubric records, events
`POST /v1/compact`	The shared asking path in `compact.ts`. `gatewayAsker()` packs short questions into one backend call.	by target	events
`GET /v1/rubrics`	Lists the rubrics that `/v1/systemone` checks claims against.	none	none
`POST /v1/skill-outcome`	The caller reports how a selected skill turned out, so selections can be compared with results.	none	outcome record
`GET /v1/skills`	Lists the skill catalog that selection chooses from.	none	none
`POST /upstream/laya`	Internal (not in the public route list). Entry point that AI Gateway's `custom-laya` provider calls back into. It forwards to the Laya Container Durable Object.	Laya container	none

The selection rule

pick≥ 0.75Top skill scores 0.75 or more and is not tied. Return one skill.

shortlist≥ 0.40No clear winner. Return the top 3 skills at 0.40 or more.

none< 0.40Nothing fits well enough. The caller decides what to do next.

Config that shapes routing

From the repo

// wrangler.jsonc  (vars, not secrets)
"CLEF_AIG_PROVIDER_SLUG": "clef",     // AI Gateway custom provider to use for Clef
"SELECT_BACKENDS": "clef,jev"          // try in order; set "jev" to bypass Clef

// src/compact.ts
type AskTarget = { slug: string; apiKey: string }
gatewayAsker(env, questions, op, target?)   // target picks the provider + key

// src/backends.ts
clefEnabled(env)   // true only when CLEF_API_KEY is set

Key ideas

Check yourself · +15 XP

The top skill scores 0.62 and nothing else is close. What does select-skill return?

Chapter 2 of 4

The hop to every model

How AI Gateway forwards each call, the three backends behind it, and the tunnel that reaches the Mac mini.

[ 04 ] · Module 4 of 12

AI Gateway, the hop to every model

Know how a custom provider turns one gateway into three backends.

The gateway named jev-gateway has three custom providers. A custom provider is a slug mapped to a base URL. The Worker calls the gateway URL, and the gateway forwards the request to the base URL, keeping the rest of the path.

The call, end to end

# what the Worker actually calls (account id elided)
POST https://gateway.ai.cloudflare.com/v1/<account>/jev-gateway/custom-clef/v1/systemone
  cf-aig-authorization: Bearer $CF_AIG_TOKEN     # proves the caller may use this gateway
  authorization:        Bearer $CLEF_API_KEY     # passed through to clef-serve

# the gateway forwards to the provider's base URL
POST https://clef.organizedai.vip/v1/systemone

Provider slug	Base URL points at	Key sent by the Worker	Used by
`jev`	Jev hosted API (TypeSafe)	`JEV_API_KEY`	select-skill fallback, systemone
`laya`	jev-gateway's own `/upstream/laya`	`LAYA_UPSTREAM_KEY`	systemone
`clef`	`https://clef.organizedai.vip`	`CLEF_API_KEY`	select-skill, first choice

Logs

Every call shows up in the dashboard under AI Gateway → jev-gateway → Logs, with the provider, status, duration and body. The response carries a cf-aig-log-id header you can match to a D1 row.

Auth

The gateway is authenticated, so a request without cf-aig-authorization is refused before it reaches any backend. That token is CF_AIG_TOKEN.

Cache headers

cf-aig-cache-ttl and cf-aig-skip-cache control the gateway's own response cache. jev-gateway relies on its KV cache instead (Module 08), because it needs the backend name in the key.

Why route Laya through the gateway when it lives inside the same Worker? So that Laya calls get the same logs and analytics as the others. The gateway calls back into /upstream/laya, and LAYA_UPSTREAM_KEY makes sure only the gateway can use that door.

Key ideas

Check yourself · +15 XP

A request reaches AI Gateway without cf-aig-authorization. What happens?

[ 05 ] · Module 5 of 12

Three backends, one request shape

Know what answers each request and why one request shape matters.

Jev hosted

The TypeSafe Jev model behind custom-jev. It is always reachable, so it is the safety net when Clef fails, and it answers rubric claims on /v1/systemone.

Laya container

A Cloudflare Container managed by a Durable Object. The image includes torch, so wrangler deploy rebuilds it (amd64) and takes a while. It is reached through custom-laya → /upstream/laya → Container DO.

Clef Mac mini

Clef-Flash (mlx-community/clef-flash-4bit) served by clef-serve on Jordan's Mac mini. It returns {model, answers, usage} for a SystemOne-style body. It is first in SELECT_BACKENDS.

Laya's loop, drawn out

FIG. 2 · Laya is called through the gateway, back into the same Worker

Key ideas

Check yourself · +15 XP

Why can the backend order change without a code change?

[ 06 ] · Module 6 of 12

The Tunnel to the Mac mini

Know how a model on a desk gets a public hostname without open ports.

Clef runs on a Mac at a desk. It has no public IP and no open ports. A Cloudflare Tunnel fixes that from the inside out: cloudflared on the mini dials out to Cloudflare and holds those connections open. When a request arrives for clef.organizedai.vip, Cloudflare sends it down one of those existing connections.

Piece	Value here	Where it is configured
Tunnel	"A1 Scout", remotely managed	Zero Trust dashboard (config lives at Cloudflare, not in a file on the mini)
Connector	`~/.nvme-stash/bin/cloudflared` as a LaunchDaemon	Jordan's Mac mini, installed with `cloudflared service install <token>`
Ingress rule 1	`clef.organizedai.vip → http://localhost:8790`	Published via the API
Ingress catch-all	`http_status:404`	Same config; anything else gets a 404
DNS	CNAME `clef` → `<tunnel-id>.cfargotunnel.com`, proxied	organizedai.vip zone (added by hand)

clef-serve, the process behind the tunnel

What it is

Python ThreadingHTTPServer bound to 127.0.0.1:8790
POST /v1/systemone needs Bearer CLEF_API_KEY, compared with hmac.compare_digest
GET /health reports the served model

How it runs

LaunchAgent com.organizedai.clef-serve
venv ~/.nvme-stash/venvs/clef (Python 3.12, MLX)
Model in ~/.nvme-stash/models/clef-flash-4bit
Config in ~/.nvme-stash/clef.env

Resilience

Loads the model once at start
If loading fails, retries every 60 s
Binding to 127.0.0.1 means only cloudflared can reach it

Measured: about 10 s per answer locally, about 30 s through the tunnel on a first call. When Clef is slow or down, the Worker falls back to Jev, so callers get an answer either way.

Key ideas

Check yourself · +15 XP

How does a request for clef.organizedai.vip reach a Mac with no open ports?

Chapter 3 of 4

One request, written down

A select-skill call step by step, the cache rule that keeps fallbacks honest, and what gets recorded.

[ 07 ] · Module 7 of 12

One select-skill request, step by step

Follow one call end to end, with and without Clef.

Step through a real /v1/select-skill call. Switch to "Clef unavailable" to see the fallback path.

FIG. 3 · SEQUENCE

Step 1 of 10

Key ideas

Check yourself · +15 XP

Clef times out on a select-skill call. What does the caller get?

[ 08 ] · Module 8 of 12

Caching: two places, one rule

Know which cache selection relies on and the one rule it never breaks.

KV cache jev-gateway

Keys start with sk: and include the preferred backend plus a hash of the task and candidates. A repeat question returns at once without touching AI Gateway. Callers can send bypass_cache to force a fresh answer.

Gateway cache AI Gateway

Available through cf-aig-cache-ttl. It caches raw model responses by request body. It does not know about fallbacks, so it is not the cache that selection relies on.

The rule important

A Jev fallback answer is never stored under Clef's key. Otherwise one bad night on the tunnel would keep serving Jev answers labelled as Clef long after Clef came back.

Key ideas

Check yourself · +15 XP

Why is a Jev fallback answer never stored under Clef's cache key?

[ 09 ] · Module 9 of 12

Audit and events

Know what gets written down for every answer and every failure.

Record	Where	When	Use it to
`skill_selections`	D1	Every select-skill answer	See what was picked, by which backend, and with what scores
`skill_selection`	Workers Observability event	Every answer	Track volume, latency and the backend mix
`skill_selection_fallback`	Workers Observability event	Each time a backend fails and the next one is tried	Spot a down tunnel or a slow mini before callers notice
Gateway log	AI Gateway	Every model call, including failures	See the raw request, response, status and duration

The response itself reports backend (who answered) and fallbacks (who failed first and why). Observe fields are flat values, not arrays, which is why fallback reasons are emitted one event per failure.

Key ideas

Check yourself · +15 XP

Where do you look to spot a down tunnel before callers notice?

Chapter 4 of 4

Run it and fix it

Where each secret travels, how the bench earns a backend its place, and what to check when something breaks.

[ 10 ] · Module 10 of 12

Secrets and where each one travels

Know each secret, where it lives and which hop it proves.

Values are never printed. They are set by piping into wrangler secret put or kept in files with owner-only permissions on the mini.

Secret	Stored in	Sent from → to	Proves
`JEV_GATEWAY_TOKEN`	Worker secret, plus each caller	caller → jev-gateway	The caller may use jev-gateway
`CF_AIG_TOKEN`	Worker secret	jev-gateway → AI Gateway	The Worker may use the gateway
`JEV_API_KEY`	Worker secret	jev-gateway → (gateway) → Jev	Access to the hosted Jev API
`LAYA_UPSTREAM_KEY`	Worker secret	gateway → jev-gateway `/upstream/laya`	Only the gateway can reach Laya
`CLEF_API_KEY`	Worker secret and `~/.nvme-stash/clef.env`	jev-gateway → (gateway → tunnel) → clef-serve	Only jev-gateway can ask Clef
Tunnel token	cloudflared LaunchDaemon on the mini	cloudflared → Cloudflare edge	This connector belongs to "A1 Scout"

Next hardening step: put a Cloudflare Access service token in front of clef.organizedai.vip, so the hostname refuses traffic that did not come from AI Gateway even before clef-serve checks its key. Also roll any Cloudflare token that has been pasted into a chat or terminal history.

Key ideas

Check yourself · +15 XP

Which secret makes sure only the gateway can reach /upstream/laya?

[ 11 ] · Module 11 of 12

Where the A1 and the bench fit

Know how bench scores decide the backend order.

The A1 drive carries the model library (stash/) between the M1, both Mac minis and the Pi. a1-scout finds candidates on Hugging Face and GitHub and queues them. agent/bench.py scores a model on decision suites using the same pick / shortlist / none rule as jev-gateway, so a bench score predicts how the model behaves as a backend.

Suite	Question it asks	Clef-Flash	Keyword baseline
`gtm-gate`	Should this GTM change ship?	0.77	n/a
`skill-select`	Which skill fits this task?	1.00	n/a
`stape-select`	Which Stape template fits this job?	0.879	0.293

That is how Clef earned first place in SELECT_BACKENDS: it scored well on the bench, so the Worker tries it first and keeps Jev as the net.

Key ideas

Check yourself · +15 XP

Why does Clef come first in SELECT_BACKENDS?

[ 12 ] · Module 12 of 12

When it breaks

Match each symptom to its likely cause and the check that confirms it.

Symptom	Likely cause	Check
Every answer says `backend: "jev"`	Clef is failing and the fallback is working	Look for `skill_selection_fallback` events and their reason
Error 1033 or 530 on `clef.organizedai.vip`	No connector online; the mini is asleep or cloudflared stopped	Tunnel status in Zero Trust; `sudo launchctl list \| grep cloudflared` on the mini
502 from the hostname	Tunnel is up but nothing is listening on 8790	`curl localhost:8790/health` on the mini; the clef-serve LaunchAgent
401 from clef-serve	Key mismatch between the Worker secret and `clef.env`	Re-pipe the same key into both places
Gateway refuses before any backend	`CF_AIG_TOKEN` missing or wrong	AI Gateway logs; the Worker's secret list
Answers take 30 s or more	First call through the tunnel, or the mini is under memory pressure	Gateway log duration versus local `/health` timing
Need Clef out of the path now	Any of the above	Set `SELECT_BACKENDS` to `"jev"` and deploy

Key ideas

Check yourself · +15 XP

clef.organizedai.vip returns 502. What is the likely cause?