Travel Approval Workstream — Close-out Record¶

Classification: CONFIDENTIAL — Internal Use Only

Document: platform/travel-approval-workstream-closeout.md · v1.0 · 2026-09-01 · GPUS-IT Owner: Rajesh Chhetry · Status: workstream closed 2026-09-01

Read this if you are picking the travel approval workflow up cold. It is not a summary of what happened; it is the state the workstream was left in. For how the thing works, read Travel Approval Workflow — this page assumes it.

Everything below is either verified through the real production path or explicitly marked as not. Nothing here is verified by a test alone, by a deploy exiting zero, or by a document saying so.

1. What shipped and is verified end to end¶

The proof: a real approver decided a real trip¶

This is what the workstream was for, and it is not a drill or a token test.


Submission	`c84bfb45` — Madison Carter, Communications, submitted 2026-08-31 15:46:12
Step 1	`smt_kfrancis` → `kfrancis@greenpeace.org`, dispatched 15:46:15
Decision	APPROVED 2026-08-31 23:02:47.98, no comment left
Non-repudiation	`decided_by_email` = `resolved_okta_email` = `kfrancis@greenpeace.org` — no divergence, which is the check migration `014` created a separate column to make possible
Cycle	7 h 16 m, dispatch to decision
Chain advance	Step 2 dispatched to `swoodley@greenpeace.org` at 23:02:49.15 — 1.2 s later, unattended

Kalina Francis had never used the system. She received a mail, followed a link, was authorised server-side against her asserted identity, decided, and the chain moved on its own and notified the next approver. Read from approval_steps and submission_approvals, not from a description of them.

Verified, with what verified it¶

What	Evidence
Submission → routing → ticket	`af7ea350` 2026-09-01 13:28:23 created → 13:28:23.87 finalized → 13:28:26 `email_sent` to queue 45 → 13:28:26.81 `submission_routed` → 13:28:28.57 approval dispatched. Five audit rows, one submission, no manual step
The queue-45 ticket body	`#USITS00368946`, read line by line. All 17 fields, form order, funding lines after `Department:`. VLN-050 CLOSED
v4 `EventRegistrationFee`	`forms.current_version = 4` live; rendered in a real ticket for the first time on `af7ea350`
Approval mail render	routing worker at `584ffdd` on MAPLE, file digests identical to repo
R2 — daily stalled-approvals	Live since 2026-08-20, delivery proven via maillog + inbox. Fires daily 12:15 UTC
R5 — daily volume	Fires 13:40 UTC. `degraded=False`
R1 — monthly summary	First scheduled send 2026-09-01 14:25. `degraded=False`, `trips=12 submissions=13 revisions=1 funding_recorded=1/12`
R3 — monthly pre-booking	First scheduled send 2026-09-01 14:50. `degraded=False`, `pre_booked=4 total_trips=12 rate=33.3`

R1 and R3 firing on schedule is the only thing that ever proved those two crontab lines exist. R5 shares the host, wrapper, venv, mailer and database with them, so it detects a shared failure and says nothing about whether those two lines are installed. Both were installed 2026-08-27 and unproven until today.

The finalize-predicate fix, shipped the morning of the first send¶

R1, R3 and R5 windowed on submitted_at with no test for whether a submission had been finalized. Three drafts created 2026-08-31 20:04–20:11 and never submitted carried a populated submitted_at — the column defaults at row creation and is only re-stamped at finalize — and were counted as August travel.

The predicate is routing_result IS NOT NULL, not a status test. finalize_submission leaves status = 'received', so received is also the status of a finalized-but-unrouted request; excluding it would drop real travel. Deployed at b95c338, digest moved c6560de7… → a1d3caa4…, both GOV-028 acceptance commands pass.

Without it, Kevin's first two reports would have read 4 of 16 on funding and 46.7% pre-booking.

Counts are per TRIP, not per submission¶

Both monthly reports express coverage against trips (12 in August), not submissions (13). The difference is the one revision, 35750caf. So funding_recorded=1/12 and rate=33.3 are correct and are not the 1/13 and 30.8% that appear in the session notes that preceded them. Anyone reconciling those numbers later should start here.

2. Open, and why — three different kinds¶

The distinction matters more than the list. Do not treat these as one backlog.

(a) Waiting on a person — no engineering work exists¶

Item	Who	What closes it
`c84bfb45`'s short queue-45 ticket	Rajesh	A note on the existing ticket carrying `FundingSource` and `CostCenter`. Not a re-route — the submission is `IN_APPROVAL` with a real approver holding step 2, there is no re-route endpoint, and inventing one to re-render a ticket would be a state change on a live travel authorisation
`EventRegistrationFee` wording	Shereyll decides; Rajesh raises	See the observation under `GOV-032`. Do not propose a wording change here
R1/R3 delivery to Kevin	Kevin, or one `sudo grep ktoruno /var/log/maillog`	`mailq` was empty and the maillog was written at 14:25 and 14:50, so Postfix relayed rather than deferred. That is the handoff, not the inbox
Destination switch — VERIFY	next real travel submission	Shipped and LIVE at `version: 5` (2026-09-01, revision `00119-wls`), still unproven on the production path. Confirmed live: `current_version = 5`, and the `actions` row in `gpus_forms` reads `destination = gpus-admin-ops@greenpeace.org` (row `27288`, written 17:15:34) — it matches the YAML. What remains is the only thing the token test could not prove: the form's own routing. The test proved the mailbox ingests into 299. It says nothing about whether a submission travels the action → worker → relay path to it. Close it by checking the next real submission's `email_sent` audit row names `gpus-admin-ops@greenpeace.org`, then reading that ticket in 299 and counting its 17 lines — the database can only tell you a message was handed over
`DEPLOY-3f63f33.sh`	Rajesh	One sudo round-trip carrying the R5 window fix and the `e4fd73c` docs commit. Matters from tomorrow's 13:40

(b) Waiting on a prerequisite — blocked, not deferred¶

Two items remain here. The destination switch left this heading on 2026-09-01 — its prerequisite (a confirmed HappyFox queue, then proven ingest into it) was met and it shipped. Both survivors have a larger blast radius than the one that left, and neither is blocked on information: each is blocked on a change someone must make somewhere else first.

Item	Blocked by	The hazard if you skip it
Category move	The target row must exist in `categories` first	`forms.category_id` is `NOT NULL REFERENCES categories(id)`. `yaml_loader.load_all()` runs one session for all 29 forms and commits once at the end. An FK violation aborts that transaction; the per-form `try/except` records it as a single form's error and keeps looping, but every later statement fails and `session.commit()` then raises into `session.rollback()`. The whole estate's form reload is discarded, on every container boot. Seed the category, then move the form
2.5(e) HappyFox API dispatch	`T3` deconfliction	Forms with both an email and an API leg against the same queue will double-ticket. The deconfliction points in opposite directions per form and there is no single rule — on `it-support-request` remove the EMAIL leg, on `employee-termination-notification` remove the five HF legs, and `contract-extension-notification` is UNREVIEWED. Travel now has a live email leg to a queue that did not exist when `T3` was written, so it joins the list of forms needing a per-form decision before 2.5(e) ships

(c) Raised and deliberately unfixed — recorded, not scheduled¶

These are correct as they stand. Reopening any of them is a decision, not a cleanup.

Row	Why it is open
`GOV-030`	The ASVS scope statement's fact D argues the frozen dispatch snapshot is acceptable because revocation exists as an administrative re-dispatch, and tells the assessor to audit that path. The path does not exist. Closes either by building it or by amending the claim — and `approval_access.py:28-30` must be amended with it
`GOV-031`	A stall at the terminal step has no chaser. R2 goes to Shereyll and excludes Kevin, both correct decisions; together they leave `e5ff2b59` — Shereyll's own trip, dispatched to Kevin — reported daily to the one person who cannot act on it. Adding Kevin to R2 is not the fix
`GOV-032`	Conditional display, declined 2026-09-01 by Rajesh and Shereyll. Do not build or scope it. Two real data points now, one each way
`GOV-033`	The September executive monthly did not send and nothing said so. Fix recommended in three parts, not implemented — part (1) changes the behaviour of a board-adjacent report and is Rajesh's call
`GOV-034`	`MAILTO=root` is inert on 7 of 7 hosts. Not a travel problem — it outlives this workstream
`PRG-029`	The `latest.pdf` freshness check. The only one of `GOV-033`'s three recommendations that would have caught the failure rather than making the next one louder
`VLN-051` / `VLN-052`	The approval audit rows reach no consumer, and a denied approval attempt pages nobody. Both raised 2026-08-31, unfixed

The three stalled requests — as-is by decision¶

1b4dfd50 (step 2, since 2026-08-19), e5ff2b59 (step 3, since 2026-08-20), a574e4cb (step 1, dispatched to sraman@ since 2026-08-26). Do not touch them. They are the live evidence GOV-030 and GOV-031 rest on, and any intervention would be a state change on a real person's travel authorisation.

a574e4cb is reviewed mid-November 2026. If it is still outstanding then, that is a decision point about the request, not about the platform.

3. What the workstream did NOT deliver¶

Stated plainly, because each of these is assumed somewhere by something that did ship.

Conditional display — declined 2026-09-01. Requested twice by Shereyll; both times a compromise shipped instead (v3 three always-visible funding fields governed by prose; v4 one required numeric with "enter 0 if none"). The platform has no conditional display and no cross-field validation, so a dependent field can be neither shown conditionally nor enforced. div_lock exists through the YAML, the fields table and the form-schema API and is inert — five live forms carry nineteen dead values, one of them fourteen. A renderer that started honouring them would change five forms nobody asked to change, silently, on the next deploy. Recorded at GOV-032, dated, with what a third request looks like.

Reassignment — never built, though fact D's argument assumes it. There is no way to move a dispatched approval step to another person. VOIDED has one write site, inside apply_decision's transaction, and it voids steps ahead of a returned or declined one — never the dispatched step itself. The whole approval API is three routes. GOV-030 is the record.

A withdraw endpoint — legal since migration 014, never written. WITHDRAWN has been in submission_approvals' CHECK constraint since 2026-08-18. Its only writer is 021_withdraw_test_chains.sql, a hand-run SQL file that retired four test chains and writes no audit_log row — so even the one exercised instance is not "a recorded event". 021 states the open design questions honestly and they are still open: who may withdraw, at which states, does it notify the holder, how does it interact with revise.

When the withdraw endpoint is built it MUST add an approval_withdrawn audit action. A user-initiated state transition with no durable trace is the gap 020 was written to close one level up. Do not copy 021's silence into it.

SubmissionStatus.tsx has no WITHDRAWN branch, so a withdrawn request shows a submitter the generic "This request is recorded with this state. Contact IT if you need more detail." Ordering matters: the branch cannot honestly be written before the endpoint that supplies an actor and a reason.

3a. DONE — the Admin Ops destination switch¶

CLOSED 2026-09-01. Gate met, switch applied at version: 5. Token test ADMINOPS-INGEST-20260901-165305 opened #U-AO00368986 in US - Admin Ops (category 299). The runbook below is kept as written because the sequence is the reusable part — the next destination change for any form should follow it — and because it records what the gate cost and why it was worth it.

Still to verify: the next real travel submission's ticket landing in 299, read in HappyFox. Until then the switch is deployed but not proven on the production path. See Step 3.

Original: PREPARED — the Admin Ops destination switch¶

Status: gated, not blocked on engineering. Everything below is written and ready; none of it may run until the queue is confirmed to exist.

Tanu has reported travel tickets landing in IT Support as a defect. It is not a defect. gpus-it-support@ is the documented interim destination (forms/travel-request.yaml:197-200) and the switch is deliberately gated.

The gate — re-verified 2026-09-01, unchanged¶

Check	Result
`email_sent` audit rows all-time	83
…of those, to `gpus-admin-ops@`	0
In the seven-address audited destination family	No — the family is `gpus-it-support@`, `gpus-people@`, `gpus-facilities@`, `gpus-finance-support@`, `gpus-it-accounts@`, `gpus-finance@`, `gpus-data-request@`
Matching queue in the pinned HappyFox map	RESOLVED 2026-09-01 — category `299`, confirmed by Tanu Garg and added to `HAPPYFOX_QUEUES` in `governance/scripts/build_audit.py`. Provenance recorded as from a person, not from `fetch_live.sh` — the seven pre-existing ids were confirmed against the categories API on 2026-08-03 and this one has not been. Re-confirm on the next API pull
Ingest into 299 proven?	NO — and this is now the only thing holding the switch. The queue existing and the mailbox delivering into it are different properties. `gpus-admin-ops@` has a row in `forms-field-audit.md` §5.12 for the first time, marked test prepared, not run

Why switching untested fails silently, which is the whole reason for the gate. email_sent records that the local relay accepted the message, not that anyone received it. happyfox_ticket_id stays NULL on this path always — that is ASVS fact F, not a symptom — so a ticket that was never created is indistinguishable from one that was. The submitter's confirmation still promises a ticket. And c84bfb45 proved the approval chain proceeds regardless of the ticket, so the one human loop that might have noticed is satisfied through a different channel. Nothing anywhere would report it.

What must move WITH the destination, and is easy to miss¶

build_audit.py holds two lists, and adding the queue id to one without adding the address to the other reproduces a defect this repo has already had.

HAPPYFOX_QUEUES — done, 299 added 2026-09-01.
EMAIL_OWNERS — the ROLE address list. It is exactly complete today: the seven addresses the forms route to are the seven listed. gpus-admin-ops@ is not among them, and the comment beneath that list records what happens when an address is missing — "falls through to the ('IND', individual) bucket, which is how a team address gets mis-reported as a personal mailbox — and, worse, how a form drops out of its owner's delivery-scoped review copy entirely. gpus-it-infrastructure@ was missing until 2026-08-04."

So the moment travel routes to gpus-admin-ops@ and that address is not in EMAIL_OWNERS, Travel Request silently drops out of the audit's review copy. Add it in the same commit as the destination change, not before — an address in the list with no form routing to it is its own noise.

Other staleness in the pinned map, now that it has been proven incomplete once. Checked rather than assumed: the six queue ids actually referenced by forms/*.yaml are 45, 81, 85, 92, 96, 100 and all six are in the map. 71 is mapped but referenced by nothing since it-support-request moved to 45 on 2026-08-06 — harmless, and worth leaving as a record of that correction. No form points at an unmapped queue. So the map was not decaying; it was missing an id for a queue that did not exist when it was pinned. §5.13 remains open on #85 resolving only by a dept-code join, which is a different weakness in the same file and is unaffected by this addition.

Step 1 — token test, FROM MAPLE¶

A laptop send does not exercise this path and must not be used as evidence. MAPLE's Postfix is inet_interfaces = localhost and relays via [smtp.gmail.com]:587 with SASL, and the relay rewrites the visible From to alerts@greenpeace.us because the header sender is not a registered send-as alias. Whether HappyFox accepts mail from that rewritten sender into that queue is precisely the unknown. Only MAPLE reproduces it.

ssh maple
/opt/gpus-forms-routing/venv/bin/python3 - <<'EOF'
import smtplib, datetime
from email.message import EmailMessage
tok = datetime.datetime.utcnow().strftime("TOKEN-%Y%m%d-%H%M%S")
m = EmailMessage()
m["From"] = "gpus-it-security@greenpeace.org"   # rewritten to alerts@greenpeace.us by the relay
m["To"]   = "gpus-admin-ops@greenpeace.org"
m["Subject"] = f"[GPUS Forms] Admin Ops queue ingest token test {tok}"   # token makes the ticket findable
m.set_content(
    f"Ingest token test for the travel-request destination switch.\n\n"
    f"Token: {tok}\n\n"
    f"If this opened a ticket, reply with the ticket NUMBER. A ticket that\n"
    f"cannot be named by number does not count as confirmation.\n")
with smtplib.SMTP("localhost", 25) as smtp:
    smtp.send_message(m)
print("handed to Postfix:", tok)
EOF

Then, and only then, in HappyFox — two things, and the second is the one people skip:

A ticket NUMBER, found by searching the token string.
That the ticket is in category 299. A ticket that opened in some other queue — a catch-all, or IT Support — is a different failure and is indistinguishable from success on the mail side: same 250 OK, same email_sent row, same empty mailq. Switching on that evidence would move travel tickets from one wrong queue to another and look like it worked.

mailq being empty proves the relay accepted the message. It proves nothing about ingest, and nothing at all about which queue.

The gate is a ticket NUMBER in 299, not a ticket. "It looks like it arrived" is the standard VLN-050 was reopened over.

Record the outcome in forms-field-audit.md §5.12 either way — that row now exists and is marked test prepared, not run. A negative result is worth as much as a positive one and is the more expensive one to discover later. §5.12 is GENERATED: edit governance/scripts/gen_audit_doc.py, then python3 governance/scripts/gen_audit_doc.py governance/scripts/audit.json mkdocs-portal/docs/governance/forms-field-audit.md. Hand-editing the page is overwritten on the next regeneration.

Step 2 — the YAML change, once a number exists¶

forms/travel-request.yaml — two edits, both required:

# line 15
version: 5                       # was 4 — a destination change is a form change

# lines 197-206: replace the INTERIM DESTINATION block and the action
# INTERIM DESTINATION REMOVED <DATE>. gpus-admin-ops@ confirmed by token test
# <TOKEN> opening HappyFox ticket #<NUMBER> on <DATE>. Queue: <QUEUE NAME/ID>.
actions:
- type: email_template
  destination: gpus-admin-ops@greenpeace.org
  template: travelrequest001

Do not delete the interim-destination note — replace it with what confirmed the change, so the next reader sees the evidence and not just the outcome.

version: 5 matters beyond bookkeeping: forms.current_version is the only deploy signal that moves on a real form change (updated_at advances on every boot regardless), so without the bump there is no way to confirm the new routing is live.

Add gpus-admin-ops@greenpeace.org to EMAIL_OWNERS in the same commit, per the section above.

The category move ships SEPARATELY, and after¶

travel-request.yaml:13 reads category: finance. Moving the form to an Admin Ops category is a different change from moving its ticket destination — the first is which section the form appears under in the portal's own listing, the second is where the email goes — and they have different prerequisites, different failure modes and different blast radii.

	destination switch	category move
Gated on	a ticket number in 299	an `admin-ops` row existing in `categories`
Fails by	tickets going nowhere, silently	`load_all()` aborting for all 29 forms
Blast radius	one form	the whole forms estate, on every container boot

forms.category_id is NOT NULL REFERENCES categories(id), and yaml_loader.load_all() runs one session for all 29 forms with a single commit. A category id with no row aborts that transaction; the per-form try/except logs it as one form's error and keeps looping, but every subsequent statement fails and the final commit() raises into rollback(). The entire estate's form reload is discarded — on every boot, until it is fixed.

Ship them apart, destination first. Bundling means one rollback cannot separate "tickets are going to the wrong place" from "no form on the portal updated at all", and those need different responses at different urgencies. The category move should be its own deploy whose only possible failure is the one you are watching for — seed the categories row, deploy, confirm 29 forms still load, then change the form's category:.

Step 3 — verification, on a real submission¶

Not on the token test. The token test proves ingest; this proves the portal's routing.

# a) the form reloaded
ssh maple 'TOK=$(gcloud auth print-access-token); PGPASSWORD="$TOK" psql \
  "host=10.34.0.3 user=maple-agent@gpus-infra.iam dbname=gpus_forms sslmode=require" \
  -tAc "SELECT current_version FROM forms WHERE id='"'"'travel-request-001'"'"';"'
#    must read 5

# b) the next real submission routed to the new destination
ssh maple 'TOK=$(gcloud auth print-access-token); PGPASSWORD="$TOK" psql \
  "host=10.34.0.3 user=maple-agent@gpus-infra.iam dbname=gpus_forms sslmode=require" \
  -tAF"|" -c "SELECT occurred_at, details->>'"'"'destination'"'"' FROM audit_log \
   WHERE action='"'"'email_sent'"'"' ORDER BY occurred_at DESC LIMIT 1;"'
#    must name gpus-admin-ops@greenpeace.org

c) — and this is the one that closes it — read the ticket in category 299 in HappyFox and confirm its 17 lines. Same standard as VLN-050: the database can only tell you a message was handed over. Confirm the queue as well as the ticket, for the reason in Step 1.

Rollback is the reverse YAML edit plus version: 6. There is no state to unwind — the destination is read per-action at send time.

The answer for Tanu¶

Thanks — 299 was exactly what we needed, and it turned out to be the first of two things. The category id closed the half that was genuinely blocked on GPI: we now know the Admin Ops queue exists. The half it did not close was whether the mailbox actually delivers into it — a mailbox can accept mail perfectly and be wired to nothing, and every signal on our side (250 OK, empty queue, a successful-looking audit row) reads identically either way, which is why we would not switch on the queue id alone. So we sent one token test from the server that actually sends this mail, and it came back clean: ticket #U-AO00368986, category US - Admin Ops, opened from alerts@greenpeace.us — which is the rewritten sender our relay uses, so it proves HappyFox accepts our real sending path into that queue and not just mail in general. That was the actual unknown. The change is now in: travel requests route to Admin Ops from the next submission onward, and IT Support stops receiving them. It was never an oversight — gpus-it-support@ was set as an explicit interim destination, written into the form's configuration as interim, because it was the only address anywhere in the portal we had ever proven could open a ticket. We will confirm on the first real request that its ticket lands in Admin Ops with all seventeen fields, and I will send you that ticket number too.

4. Where to start, if you are picking this up¶

Read Travel Approval Workflow for the mechanism.
Read GOV-030 before touching authorization, and 021_withdraw_test_chains.sql before touching withdraw. Both will save you a wrong start.
Do not run a hand-send of R1 or R3 as an introduction — report_cron.sh takes no --recipients and mails Kevin.
Anything that changes forms/*.yaml is exercised against all 29 forms on the next boot. There is no per-form deploy.