Skip to content

Attribute GPT renders to Trusted Server on observed evidence - #997

Open
prk-Jr wants to merge 32 commits into
mainfrom
feat/gpt-diagnostics-delivery-attribution
Open

Attribute GPT renders to Trusted Server on observed evidence#997
prk-Jr wants to merge 32 commits into
mainfrom
feat/gpt-diagnostics-delivery-attribution

Conversation

@prk-Jr

@prk-Jr prk-Jr commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR extends the opt-in GPT diagnostics so an operator can answer two separate questions without inferring from targeting, price, or timing:

  1. What did Ad Manager report as rendered, and did the Trusted Server creative bridge run?
  2. Which observed request path produced each GPT request, and did a later request replace an already-rendered ad?

The implementation remains observational and requires zero publisher-code changes. Diagnostics must not suppress, delay, reorder, add, or remove GPT requests; change targeting; alter auctions or timeouts; or affect creative rendering.

Evidence reported

For every retained GPT request cycle, diagnostics can now report:

  • GPT request, response, render, load, and viewability observations
  • Ad Manager line item, order, advertiser, creative, backfill, and yield-group/company identifiers when GPT exposes them
  • Ad Manager response class: empty, backfill, reservation, or unclassified_non_empty
  • Trusted Server delivery evidence from the creative bridge, without treating a filled slot as proof that Trusted Server rendered
  • Request source: trusted_server_direct, prebid_refresh, publisher_refresh, competing, or unattributed
  • An opaque Trusted Server auction ID and opportunity-to-request latency when available
  • Whether a filled render replaced the most recent earlier filled render, including elapsed time and observed creative transition
  • Normal GPT load-before-render ordering through loadObservedBeforeRender, without fabricating a negative duration or marking the cycle incomplete

Calls that bypass every installed observation boundary remain honestly unattributed. Overlapping request cycles remain ambiguous; diagnostics does not guess to improve coverage counters.

Refresh-source attribution

Request triggers are represented as short-lived, per-slot intents. Trusted Server, Prebid, and publisher evidence expire independently and the next matching slotRequested consumes the complete intent.

The diagnostics integration installs a standalone observer around the current pubads.refresh before the deferred Prebid wrapper is installed. The observer:

  • delegates exactly once
  • preserves receiver, argument count and identity, return value, and synchronous throws
  • records explicit slot lists or defensively resolves bare refreshes with getSlots()
  • remains fail-open when GPT, slot resolution, diagnostics, or shared context access throws
  • skips publisher attribution during Trusted Server's internal adInit refresh and during a Prebid-managed delegation

Prebid exposes only a synchronous diagnostic context around the delegated refresh. It is restored after normal completion and throws, including hostile setter/proxy behavior, and cannot influence targeting, auction timing, callbacks, or whether refresh runs.

Pending intent expiry is coalesced to one callback per slot/source. WeakRef avoids retaining slots where supported; older runtimes use a bounded five-second strong-reference fallback.

Auction correlation

The server copies the existing AuctionRequest.id into optional hb_auction_id winning-bid metadata for both initial-document and page-bids responses. The value is used only as opaque diagnostic correlation data; it does not affect bid selection or GAM targeting, and no auction payload is retained.

Creative and replacement attribution

The existing delivery ladder remains evidence-based:

Delivery Meaning
trusted_server The rendered creative requested its response from Trusted Server.
other_demand Trusted Server had a candidate and GAM filled the slot, but no Trusted Server creative request was observed before the attribution window closed.
no_candidate No Trusted Server candidate was observed for the cycle.
pending A candidate render remains inside its attribution window.
not_applicable The cycle is empty or has not rendered.

Replacement comparison uses GPT's primary creative ID with its source-agnostic creative ID as the fallback. Missing or one-sided evidence is omitted rather than described as a change.

Main changes

Area Change
GPT diagnostics store Per-source request intents, bounded expiry, auction correlation, replacement derivation, early-load handling, and existing creative-delivery evidence.
GPT observer Ad Manager metadata capture plus a transparent publisher-refresh observer.
Prebid integration Prebid refresh evidence and exception-safe synchronous delegation context.
GPT integration Trusted Server opportunity/auction forwarding and creative bridge evidence.
Publisher response generation Optional auction ID metadata for initial and page-bids winning bids.
Overlay/export Request source, intent, latency, auction, replacement, creative-transition, delivery, and load-order facts.
Documentation Evidence semantics, limitations, privacy boundaries, and operator verification guidance.

This branch deliberately does not import the rc/july GPT slot-handoff implementation or APS renderer behavior.

Verification

  • Full JS suite: 41 files, 672 tests
  • JS ESLint and Prettier checks
  • JS build: 13 modules
  • cargo fmt --all -- --check
  • cargo test-fastly — Fastly adapter, core, OpenRTB, and doctests
  • cargo test-axum
  • cargo test-cloudflare
  • cargo test-spin
  • cargo clippy-fastly
  • cargo clippy-axum
  • cargo clippy-cloudflare
  • Documentation lint and formatting
  • Independent specification-compliance review
  • Independent code-quality review
  • Remaining target-specific CI checks run in GitHub Actions
  • Manual verification with ?ts_console=true on representative GAM delivery paths

Safety and privacy

  • Diagnostics remains opt-in and browser-local.
  • No bid payload, targeting map, creative markup, cookie, user ID, price, or stack trace is exported.
  • Auction IDs are bounded opaque correlation values.
  • Request and creative attribution uses observed integration/GPT evidence only.
  • Callback ambiguity is surfaced rather than resolved heuristically.
  • Retention and pending expiry are bounded.

Relationship to earlier PRs

This remains the attribution follow-up to #974 and supersedes the closed pre-squash copies in #976 and #990.

ChristianPavilonis and others added 5 commits July 28, 2026 16:28
Give publisher operators an opt-in, tab-local view of GPT-observed slot lifecycles and exact DOM bindings without changing auction or publisher behavior. Keep capture bounded, export allowlisted facts only, and cover activation, correlation, presentation, and non-interference across unit and browser tests.
Scope the diagnostics navigation assertion to the fixture navigation landmark so Playwright does not match the separate site-wide Home link in CI.
The diagnostics console reported that a slot filled, but not what filled it,
so an operator could not tell whether the line item Trusted Server won had
rendered or whether a house ad, a direct-sold line item, or an Ad Manager
default had taken the slot.

Retain the Ad Manager identifiers slotRenderEnded already carries for the
delivered ad, and resolve delivery from two observed facts: the Trusted Server
GPT integration reporting which GPT slot carries a bid, and the rendered
creative requesting its markup from Trusted Server. Only that creative asks,
so a claim settles the Ad Manager decision, and its absence settles the
opposite once the attribution window closes. Nothing is inferred from
targeting, price, or timing, and an unattributable claim is preserved as an
issue rather than attached to a guess.
The overlay commits on this branch are the pre-squash copies of #974, which
landed on main as a single reviewed commit that also carries the review
changes. Adopt main's tree so the superseded copies drop out; the delivery
attribution work is re-applied on top of main's implementation.
slotRenderEnded reports the Ad Manager identifiers for the ad it delivered,
but the observer read only isEmpty, size, isBackfill, and slotContentChanged,
so a filled slot could not be told apart from a slot filled by Trusted Server.

Capture those identifiers, derive a response class from them, and settle the
delivery question with the one piece of observed evidence available: only the
creative of the line item carrying Trusted Server targeting asks Trusted
Server for its markup. The GPT integration reports the candidate slot at
adInit and the markup request from the render bridge; both calls are optional
and no-op in a tab without diagnostics activated. Nothing is inferred from
targeting, price, or timing.
@prk-Jr prk-Jr self-assigned this Aug 4, 2026
@prk-Jr
prk-Jr marked this pull request as draft August 4, 2026 15:59
prk-Jr added a commit that referenced this pull request Aug 6, 2026
Bring the GPT delivery-evidence and refresh-attribution diagnostics onto the
July release branch. rc/july already carried a later evolution of the auction-ID
plumbing (PR #922 render tracing, the delivered-winner-slot telemetry, and the
APS renderer bridge), so the overlapping Rust and JS changes keep the rc/july
implementation and layer the new diagnostics on top of it.

Conflict resolutions:
- publisher.rs: keep the rc/july `write_bids_to_state` -> delivered winner slots
  contract and prebuilt page-bids bid map; keep both new tests; update the
  incoming test provider to the current `ProviderRequestOutcome` trait.
- gpt/index.ts: keep the render-trace and APS renderer paths and add the
  creative request/response/failure diagnostics around them. The attempt is
  recorded after the APS branch, which is served by the APS universal creative
  and has no Trusted Server creative response to resolve.
- prebid/index.ts: keep the resolved bare-refresh slot list and dispatch it
  through the diagnostics-aware refresh wrapper.
- store.ts: adopt the incoming response-based `slotOnload` correlation.
- Drop the pre-squash `gpt_diagnostics_bootstrap.js` and its test, which the
  upstream #974 squash removed as superseded by server-recognized activation.

Verified: cargo fmt, clippy (fastly/axum/cloudflare/cloudflare-wasm/spin-native/
spin-wasm), test-fastly, test-axum, test-cloudflare, test-spin, parity,
vitest (812 tests), JS and docs format.
@prk-Jr
prk-Jr marked this pull request as ready for review August 6, 2026 10:09

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Reviewed PR #997 against main. The implementation has a confirmed blocker: the new browser-visible auction metadata exposes the EC-derived request identifier. I also found three medium-severity diagnostics correctness/API/resource-bounding issues. Details and concrete fixes are attached inline.

Comment thread crates/trusted-server-core/src/publisher.rs
Comment thread crates/trusted-server-js/lib/src/integrations/gpt_diagnostics/store.ts Outdated
Comment thread crates/trusted-server-js/lib/src/core/types.ts Outdated
Comment thread crates/trusted-server-js/lib/src/integrations/gpt_diagnostics/store.ts Outdated

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The diagnostics evidence model (per-slot request intents, bounded attempt lifecycle, evidence-ladder delivery) is well designed and thoroughly tested, and the presentation layer is XSS-clean (every DOM insertion goes through textContent/constant setAttribute, closed shadow root). However, the auction-correlation feature undermines the PR's own privacy contract: hb_auction_id is derived from the EC ID and ships to all users, and the type-level guards meant to protect the export surface are not enforced by any gate. Requesting changes on those grounds plus a few doc/test integrity issues; the rest is non-blocking.

Blocking

🔧 wrench

  • hb_auction_id exposes the HttpOnly EC ID to page JS and the diagnostics export (crates/trusted-server-core/src/publisher.rs:3297, inline comment)
  • Privacy type contract unenforced; one assertion already broken (crates/trusted-server-js/lib/test/integrations/gpt_diagnostics/types.test.ts:129, inline comment)
  • Replacement-after-eviction test passes vacuously (crates/trusted-server-js/lib/test/integrations/gpt_diagnostics/store.test.ts:887, inline comment)
  • Operator guide contradicts the implementation (docs/guide/integrations/gpt-diagnostics.md:399, inline comment)

Non-blocking

🤔 thinking

  • Empty render leaves its creative attempt live (store.ts:621, inline comment)
  • Pass-through publisher refreshes labeled prebid_refresh (prebid/index.ts:1274, inline comment)

♻️ refactor

  • Snapshot recomputed per subscriber: scheduleNotification calls this.snapshot() inside the listener loop (api.ts:233-235). Each subscriber gets a different capturedAt, and at the 5s TRUSTED_SERVER_ATTRIBUTION_WINDOW_MS boundary two subscribers in the same notification can disagree on pending vs candidate_unconfirmed. It is also O(subscribers × slots × cycles). Hoist one const snapshot = this.snapshot(); above the loop (in its own try so a snapshot failure still cannot escape).
  • Coverage gaps in new code paths (all verified uncovered via V8 coverage):
    • recordPublisherRefresh malformed inputs (store.ts:341, 343) - the writer fed directly from publisher-supplied refresh() arguments is the only one of the three intent writers without a malformed-input test.
    • MAX_TRUSTED_SERVER_ASSOCIATIONS LRU trim (store.ts:317-321) - the only bound in the file with no test; a regression silently drops the oldest auctionSlotId association on long-lived SPAs.
    • WeakRef-collected expiry path (store.ts:880, 884) - the WeakRef-absent fallback is tested, but not a WeakRef whose target was collected, which is the case the WeakRef exists for.
    • invalid_event_order arms for slotRenderEnded/slotOnload/impressionViewable (store.ts:642-644, 661-663, 681-683) and the non-finite-percentage arm of recordSlotVisibilityChanged (store.ts:700-710).
    • recordTrustedServerCreativeFailure with an unknown attempt ID (store.ts:502-503); safelyRecordCreativeFailure early return when the attempt was never created (gpt/index.ts:1173); badges.ts:123 incomplete-sequence line; overlay yieldGroupIds/companyIds facts (overlay.ts:206-208).
  • Test hygiene bugs:
    • badges.test.ts:276: gptDiagnosticsBadgeTextForTest.toString() returns only badgeText's own body, so the not.toMatch(/GAM winner|bidder|provenance/i) guard cannot see deliveryLabel, which is where the new delivery vocabulary lives. Assert over rendered output instead.
    • overlay.test.ts:327: the store is constructed without a defer stub, so recordPublisherRefresh schedules a real 5000ms setTimeout that outlives the test. Add defer: () => undefined matching overlay.test.ts:119.

⛏ nitpick

  • Tautological assertions: types.test.ts:124-128 and 135-137 assert hand-written literals against themselves (the identity toBe checks, the toHaveLength(8) on a literal array, and the blocklist regexes over objects the test itself wrote constrain nothing about production types); store.test.ts:622's not.toContain('other_demand') cannot fail since that string exists nowhere in the codebase; badges.test.ts:131, 144, 220-232 add not.toMatch guards on strings already pinned by exact toBe.
  • Mega-tests: overlay.test.ts:113-322 (one it, ~50 assertions across 9 scenarios), observer.test.ts:131-164 (5 behaviors with interleaved arrange steps), prebid/index.test.ts:1549-1614 (4 scenarios sharing mutable state) would be cleaner and more diagnosable as it.each, which these same files already use well elsewhere.

📝 note

  • Stale PR description: the GitHub PR body still documents the deleted 5-state delivery ladder (trusted_server / other_demand); the shipped code and guide use the 7-state ladder (trusted_server_response_sent, trusted_server_selected, candidate_unconfirmed, no_candidate, unknown, pending, not_applicable). Worth rewriting before merge.
  • Spec hygiene: the 2026-08-04 design spec declares a 4-value GptDiagnosticsRequestPath union with no forward pointer to the 2026-08-05 extension that added publisher_refresh; a one-line "Extended by ..." note would prevent a reader landing there first from getting a stale union. Both spec statuses (Proposed / Approved) should become Implemented when this merges, matching existing repo practice.

🌱 seedling

  • Open Bidding classification: a yield-group render (yieldGroupIds present, no line-item/creative ID) classifies as unclassified_non_empty despite carrying positive Ad Manager evidence; a distinct response class may be worth adding later.

CI Status

  • fmt: PASS
  • clippy (all adapters): PASS
  • rust tests (fastly/axum/cloudflare/spin/parity/CLI): PASS
  • js tests (vitest): PASS
  • browser + Fastly EC integration tests: PASS
  • CodeQL: PASS

Comment thread crates/trusted-server-core/src/publisher.rs
Comment thread crates/trusted-server-js/lib/test/integrations/gpt_diagnostics/types.test.ts Outdated
Comment thread docs/guide/integrations/gpt-diagnostics.md
Comment thread crates/trusted-server-js/lib/src/integrations/gpt_diagnostics/badges.ts Outdated
Comment thread crates/trusted-server-js/lib/src/integrations/prebid/index.ts
Comment thread crates/trusted-server-js/lib/src/core/types.ts Outdated
Comment thread crates/trusted-server-js/lib/src/integrations/gpt_diagnostics/overlay.ts Outdated
prk-Jr added 3 commits August 7, 2026 13:38
The winning-bid `hb_auction_id` carried `AuctionRequest.id`, which is
`ts-{ec_id}` whenever an Edge Cookie ID exists. That value reached
`window.tsjs.bids` and the page-bids JSON for every visitor, handing any
script on the page the identifier the `ts-ec` cookie keeps HttpOnly, and
it could not distinguish one auction from the next because it is stable
per visitor.

Mint an unrelated `ts-auc-{uuid}` per auction instead, and emit it only
when the GPT diagnostics integration is enabled, since nothing else
consumes it. `AuctionRequest.id` is unchanged for SSPs.
Keep the operator API read-only: `window.tsjs.gptDiagnostics` now exposes
only snapshot, export, subscribe, show, and hide. The evidence writers
Trusted Server's own modules use move to a separate internal channel,
`window.tsjs.gptDiagnosticsRecorder`, so the documented contract matches
what the object actually offers.

Bound the store's deferred work by retained state rather than refresh
rate. Request-intent evidence now expires lazily when the slot is next
recorded or requested instead of owning a timer per source, which also
removes the WeakRef-absent fallback that strongly retained every marked
slot until expiry. Delivery-boundary notifications share one timer that
re-arms from retained cycles.

Stop reporting a source-agnostic GPT identifier as a reservation. Those
IDs are populated for reservation and backfill alike, so they classify
as `reservation` only alongside an explicit non-backfill fact.

Evict a creative attempt whose cycle rendered empty, so a late markup
response cannot claim a Trusted Server delivery against an empty render,
and label badges from the derived delivery state instead of re-deriving
the precedence rules from raw timestamps.

Also: attribute `refresh(null)`, make the presentation switches
exhaustive, hoist the snapshot out of the subscriber loop so every
subscriber sees one capture, and make the export's attribution fields
required.

Type-check the export contract: the `expectTypeOf` assertions were never
evaluated, and one of them was a genuine error. Scope `test.typecheck`
to the type tests, since a package-wide `tsc --noEmit` still fails on
pre-existing errors elsewhere.

Fix the replacement-after-eviction test, which passed vacuously because
ten open cycles made the final render ambiguous, and cover the paths the
review identified: publisher-refresh malformed input, the association
LRU trim, the out-of-order callback arms, a declined creative attempt,
the Prebid dispatch-context restore arm, and wrapper install ordering.

Correct the operator guide where it contradicted the implementation, and
record the auction-token revision in the design spec.
Conflict in the Prebid refresh handler: main (#965) documented that the
delegated refresh preserves the publisher's original bare form, while
this branch replaced that call with the diagnostics recording plus the
scoped dispatch context. Both hold — `dispatchPrebidRefresh` passes
`slots` and `opts` through unchanged — so the resolution keeps the
diagnostics calls and main's comment, extended to say the wrapper only
scopes the shared context.

Also add `bid_id` to the auction-ID test provider's `Bid` literal, a
field main added in #996 after this branch introduced the provider.
prk-Jr added a commit that referenced this pull request Aug 7, 2026
Brings in the PR #997 review fixes: the read-only diagnostics facade
split from the internal recorder channel, lazily expiring request-intent
evidence with one shared delivery-boundary timer, source-agnostic IDs no
longer reported as reservations, the empty-render attempt eviction, and
the scoped type-check gate for the export contract.

The branch also carries a merge of main, whose #965 and #996 arrive here
as squashes of work rc/july already implements more fully. Where the two
sides describe the same feature, rc/july's implementation is kept:

- APS, adserver_mock, auction/types.rs, auction/formats.rs — rc/july's
  OpenRTB provider, renderer-aware bid_id precedence, and typed renderer
  envelope supersede main's versions, which drop fields rc/july needs.
- prebid.rs — rc/july canonicalizes the excluded-suffix list at both the
  startup and build paths already, so main's `load_config` helper adds
  nothing. Main's test is taken instead of rc/july's: it builds from raw
  settings rather than reusing the config `validate_config_for_startup`
  already canonicalized, so it actually exercises the build path.
- prebid/index.ts — a bare refresh that filtered slots must deliver the
  resolved target list, not stay bare, so rc/july's `deliveredSlots`
  behavior and its test expectation both stand.

Three fixes are ported into rc/july's shapes rather than resolved away:

- The EC-derived auction ID reached page JavaScript here too, through
  different plumbing: both collect paths inlined `request.id.as_str()`
  into `write_bids_to_state`, and page-bids passed it to
  `build_bid_map_with_auction_id`. All three now mint a per-auction
  token via `diagnostics_auction_id()`, gated on the diagnostics
  integration being enabled.
- A blank Prebid Cache UUID no longer ships cache coordinates. It loses
  the hb_adid precedence to `adid` or the bid id, so the Universal
  Creative would fetch `?uuid=<non-cache-id>` and miss instead of using
  the inline adm. The gate moves from `is_some()` to `non_empty()`, and
  main's regression test comes along with rc/july's `Bid` fields added.
- The browser-side excluded-suffix list is validated before use. The
  server only de-duplicates it, so an empty suffix matched every ad unit
  path and pulled every slot out of the refresh auction, and a non-array
  value threw inside the publisher's own `refresh()`.

Also collapses a duplicated `hb_auction_id` write in `build_bid_map`
down to one guarded insert, and points the Prebid refresh recorder at
`gptDiagnosticsRecorder` to match the new channel.

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Reviewed the current PR #997 head against main, including the final fixes for earlier feedback. The evidence model and safety hardening are strong, and CI is green. I found one source-compatibility regression and one subscriber-isolation issue; neither warrants blocking this otherwise solid change.

};
slots: GptDiagnosticsSlotExport[];
callbackIssues: GptDiagnosticsCallbackIssue[];
attributionIssues: GptDiagnosticsAttributionIssue[];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — Preserve the V1 export's source compatibility. attributionIssues did not exist on GptDiagnosticsExportV1 in main, so requiring it (and metadata.droppedAttributionIssues below) makes existing V1 object literals fail TypeScript compilation. The implementation plan explicitly calls for these additions to be optional while snapshots continue to emit them.

Suggested fix: Mark both fields optional and add a type test that constructs the legacy V1 shape without them.

return;
}

for (const listener of this.listeners) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — Isolate subscribers from one another's mutations. Every listener receives this same mutable snapshot object. A listener registered first can mutate nested evidence before a later listener runs, so the later subscriber observes data that was not captured by diagnostics.

Suggested fix: Capture once for a consistent capturedAt, then deep-clone per listener (or deep-freeze the shared value), with a two-listener regression test.

include: ['test/**/types.test.ts'],
// Errors reported from files outside `include` are pre-existing and
// unrelated; only the type assertions in the included files gate here.
ignoreSourceErrors: true,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌱 seedling - With ignoreSourceErrors: true and only types.test.ts in scope, the new unhandledCase never-guards in overlay.ts/badges.ts surface in the IDE but not in CI: a new GptDiagnosticsDelivery/GptDiagnosticsRequestPath/GptDiagnosticsResponseClass member compiles, builds, and passes this gate. The union memberships pinned in types.test.ts today are only GptDiagnosticsAttributionIssueReason and the API/recorder keys. A follow-up that pins the remaining unions the same way (expectTypeOf<GptDiagnosticsDelivery>().toEqualTypeOf<...>()) would make a member addition fail CI and point the author at the switches.


The derived `delivery` value uses these evidence-safe meanings:

| Delivery state | Panel wording |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 note - This table is the shipped 7-state ladder, but the GitHub PR description still documents the earlier 5-state ladder (trusted_server / other_demand). Worth rewriting the PR body before merge so the merge-commit reference material matches what shipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants