Machina

Chapter 6·Part III · Implementation: Correctness and Architecture

The Server-Side Measurement Protocol

6.1 Why the server has to speak

A lead-generation business measures a different physics than an e-commerce store. {COMPANY} sells pest control across Monterey, Santa Cruz, San Benito, and Santa Clara counties, and no money changes hands on the website. A homeowner who finds a wasp nest in April or a rat in a November crawlspace does one of two things: they call the phone number, or they submit a form. Revenue arrives later, offline, when a technician books and completes the job. The website's job is to produce a qualified lead and to record, with enough fidelity to be useful, where that lead came from and what it is likely worth. Everything in the measurement stack serves that single conversion, and the most valuable conversion event in the property is generate_lead.

That framing exposes why client-side collection alone is structurally insufficient. The client-side tag, gtag.js, runs in the visitor's browser. It is the correct instrument for behavioral signals that only exist in the browser: which page the visitor viewed, whether they started filling a form, which call-to-action they clicked. But the browser is a hostile execution environment for the one measurement that matters most. Ad blockers, tracking-protection browsers, and privacy extensions suppress gtag.js outright, and they do so precisely on the request that reports a conversion. When the browser tag is blocked, the conversion event is not delayed or degraded. It never happens. In the baseline the audit documented (Chapter 5, Audit Findings), the only working lead conversion was the client-side contact_form_submit, which meant the property's headline number depended entirely on the subset of visitors whose browsers permitted Google's tag to fire. There was no fallback.

The server does not share that fragility. When a homeowner submits the contact form, the request lands at api/contact/route.ts, which validates the payload, passes Turnstile, and writes a row to the contact_submissions table in PostgreSQL. That server code path executes regardless of what the visitor's browser blocks. The lead exists in the database whether or not gtag.js ever loaded. If the server, which already holds the authoritative record of the lead, also reports that lead to GA4, then the conversion count becomes resilient to client-side loss. This is the first argument for server-side measurement: the actor that already knows a conversion happened should be the actor that reports it.

The second argument is value. A browser event carries what the browser knows, and the browser does not know what a booked termite job is worth. The server does, or at least can hold a defensible internal estimate. Sending the conversion from the server lets the payload carry a monetary value and currency, which is the seed of the offline-revenue loop developed in Chapter 11, The Offline-Revenue Loop, and the value-based optimization discussed in Chapter 9, Identity, Attribution, and Value.

The third argument is durability of context. The contact_submissions row already stores ga_client_id, utm_source, utm_medium, utm_campaign, gclid, landing_page, and referrer. The raw material for correct attribution sits in the database at the moment the lead is created. A server-side send can attach that context deterministically rather than reconstructing it from a browser that may have navigated away.

Google exposes exactly one supported mechanism for a server to write events into a GA4 property without a browser: the Measurement Protocol. This chapter dissects the property's Measurement Protocol implementation as the audit found it, why it silently produced zero data across the entire property lifetime, and the field-by-field reconstruction that made it work, including the deliberate refusal to fabricate data for the fraction of leads that cannot be honestly attributed.

6.2 The endpoint and its contract

The GA4 Measurement Protocol is an HTTP POST to a single collection host. The documented endpoint is:

https://www.google-analytics.com/mp/collect?measurement_id=G-WH410Z73V1&api_secret=<SECRET>

The request body is JSON. The measurement_id and api_secret travel as query parameters; the api_secret is minted per data stream in the GA4 admin UI and authorizes writes into stream 14548640408. The endpoint is intentionally terse in its responses. A syntactically acceptable request returns HTTP 204 No Content. The Measurement Protocol does not validate semantics on the production endpoint and does not return an error body for a malformed-but-parseable event; it returns 204 and silently discards what it cannot use. That design choice is the trap at the center of this chapter, because a 204 means "I received your bytes," not "I recorded your event," and an implementer who never inspects the response, or who inspects it and sees anything other than 204, is flying blind.

The minimal body carries a client_id and an events array:

{
  "client_id": "1847293016.1712000000",
  "events": [
    { "name": "generate_lead", "params": { } }
  ]
}

The client_id is the linchpin. It is GA4's pseudonymous identifier for a browser, the value GA4 itself writes into the first-party _ga cookie, and it is the join key that binds a server-sent event to the same user, session, and traffic source that the browser's own events established. Send a generate_lead with the correct client_id and GA4 stitches it onto that user's timeline: same acquisition source, same session, same geography. Send it with a client_id GA4 has never seen, and GA4 does the only thing it can. It treats the event as a brand-new user arriving from nowhere, which in channel terms means (direct)/(none).

Beyond client_id and events, four fields separate a Measurement Protocol payload that merely arrives from one that attributes correctly:

  • session_id, sent inside the event's params, ties the server event to the browser's current session. Without it, GA4 cannot associate the event with the session the visitor is actually in, and session-scoped attribution collapses to direct.
  • engagement_time_msec, also in params, is the field that makes GA4 count the event as coming from an engaged session. GA4's session and user models were built around the assumption that events carry engagement time; a server event that omits it can be dropped from session counts. The convention for a server event that has no meaningful dwell time is to send 1.
  • ip_override, at the top level of the body, tells GA4 to geolocate the event using the visitor's IP rather than the datacenter's. Omit it and a lead from Salinas geolocates to wherever the VPS lives.
  • timestamp_micros, optional, lets the server backdate an event; the property does not need it for real-time lead sends and leaves it off.

Google also publishes a debug endpoint, and using it is the difference between an engineer who hopes and an engineer who knows:

https://www.google-analytics.com/debug/mp/collect

The debug endpoint accepts the identical payload and, instead of a bare 204, returns a JSON body containing a validationMessages array. An empty array means the payload passed every validation rule Google applies. A populated array names the offending field and the reason. The production endpoint will never tell you that engagement_time_msec is malformed or that an event name is reserved. The debug endpoint will. Every payload change in this chapter was validated against debug/mp/collect before it was allowed near production.

6.3 The primary defect: a 404 nobody read

The audit's most consequential finding was not a subtle attribution error. It was total data loss hiding behind a plausible-looking block of code. Both api/contact/route.ts at line 384 and api/careers/route.ts at line 288 posted their Measurement Protocol event to the wrong host:

https://www.google.com/mp/collect        ← what the code sent to
https://www.google-analytics.com/mp/collect   ← the documented endpoint

www.google.com is not a Measurement Protocol collector. It does not know what /mp/collect means. The reconstruction confirmed the failure live rather than by inspection, in keeping with the adversarial method of Chapter 4, Auditing Methodology. The same payload was curl-ed against both hosts:

POST https://www.google.com/mp/collect            → HTTP 404
POST https://www.google-analytics.com/mp/collect  → HTTP 204

The wrong host returned 404 Not Found. Every server-side generate_lead the property ever attempted hit a 404 and was thrown away by Google's front-end before any analytics system saw it. This had run across the entire lifetime of the property since its creation on 2026-04-24. The server event engineered specifically to survive ad blockers, the one designed to be the durable backbone of lead measurement, produced exactly zero events. Not a degraded signal. Zero.

The consequence rippled into the property's configuration and explains a finding that would otherwise look like an oversight. generate_lead had never been promoted to a key event. That was not negligence. Nobody marks an event a key event when the event has never once appeared in the reports, and it had never appeared because every send 404'd. One defect (the wrong host) produced a second symptom (no key event) that looked independent and was not. The lesson generalizes: a server-side send that is never verified against its HTTP response can fail completely and leave behind a configuration that looks merely incomplete rather than broken.

The reason the defect survived is the 404's silence in the calling code. The original implementation issued the POST and did not inspect the status. A 404 and a 204 were indistinguishable to a function that never read the response. This is why one of the smallest changes in the fix, logging non-200 responses, matters out of proportion to its size. Swallowed errors are how a stack loses data for months without a single alarm.

Two secondary defects sat behind the host bug, and both would have kept the event from attributing correctly even after the host was fixed. First, when the _ga cookie was absent, the code fell back to a synthetic client_id of the form server.<timestamp>, minting a fresh phantom user for every such lead. Second, the payload omitted session_id and engagement_time_msec entirely, so even a correctly hosted event would have landed as an unsessioned, unengaged hit and attributed to (direct)/(none). Fixing the host was necessary and not sufficient.

6.4 The fix, field by field

The reconstruction rewrote the Measurement Protocol block in both routes. Each change addresses a specific, named failure mode.

Host. www.google.com/mp/collect became www.google-analytics.com/mp/collect. This alone moved the response from 404 to 204 and is the change without which nothing else matters.

client_id, gated not faked. The synthetic server.<timestamp> fallback was removed. The event now sends only when a real ga_client_id extracted from the _ga cookie is present. When it is absent, the code skips the Measurement Protocol send. The reasoning is developed in full in section 6.5; in short, a fabricated identifier does not recover a lost user, it manufactures a phantom one, and a phantom user pollutes exactly the reports the property exists to produce.

session_id, parsed from the container cookie. GA4 does not store the session id in the _ga cookie. It stores it in a per-stream cookie named _ga_<container>, where the container suffix derives from the measurement ID. The value of that cookie encodes the session. Two formats exist in the wild, and a correct parser handles both. The legacy format begins GS1 and is dot-delimited; the session id is the third dot-separated field. The newer format begins GS2 and is dollar-delimited within its third field; the session id is the s-prefixed segment. The parser reads the leading token, branches on GS1 versus GS2, and extracts the session id accordingly:

// Extract session_id from the _ga_<container> cookie.
// GS1 (legacy): GS1.1.<session_id>.<n>.<engaged>.<ts>.0.0.0   → dot-delimited
// GS2 (current): GS2.1.s<session_id>$o1$g1$t<ts>$j0$l0$h0     → dollar-delimited
function parseSessionId(gaContainerCookie: string): string | null {
  if (!gaContainerCookie) return null;
  const parts = gaContainerCookie.split(".");
  const format = parts[0]; // "GS1" or "GS2"

  if (format === "GS1") {
    // session_id is the third dot-separated field
    return parts[2] ?? null;
  }

  if (format === "GS2") {
    // parts[2] holds the $-delimited payload; the s<...> segment is the session id
    const payload = parts[2] ?? "";
    const seg = payload.split("$").find((s) => s.startsWith("s"));
    return seg ? seg.slice(1) : null;
  }

  return null;
}

Handling both formats is not defensive over-engineering. A returning visitor may carry a cookie written by an older library version, and a parser that assumes one format silently returns null for the other, which drops the event back to unsessioned direct traffic. The reconstruction handles the union.

engagement_time_msec. Set to 1. This is the minimum honest value for a server event with no measurable dwell, and it is what makes GA4 treat the event as belonging to an engaged session rather than discarding it from session-scoped metrics.

ip_override. Set to the request's client IP. A pest-control lead is worthless to the business if GA4 places it in the datacenter's city instead of the homeowner's. For a business whose entire model is geographic (four named counties, dozens of city and location pages), correct geolocation is not a nicety. It is the axis along which the seasonal, by-county analysis is done.

value, per service. The original flat value: 150 was replaced with a per-service map, provisional and internal:

const LEAD_VALUE_USD: Record<string, number> = {
  termite: 400, fumigation: 500, commercial: 400,
  "bed-bug": 250, rodent: 180, general: 150, pest: 150,
  ant: 100, "wasp-bee": 100, other: 120,
};

A booked fumigation is worth several times an ant call, and a single flat number throws that structure away. The values are labeled provisional and internal for a reason argued in Chapter 9: GA4 lead value is a bidding and analysis signal, never shown to a customer, so a defensible internal estimate is both sufficient and appropriate. Refining these numbers against real job_value data closed offline is the work of Chapter 11.

form_name renamed to form_source. The server payload aligned its parameter name to the taxonomy the rest of the property uses, so that the server event and the client events populate the same custom dimension. Taxonomy governance is the subject of Chapter 8, Event Taxonomy and Custom Dimensions; the point here is that a server event that invents its own parameter names is an orphan in the reports even when it arrives correctly.

Bare source/medium/campaign dropped. The original payload included top-level source, medium, and campaign fields copied from the lead record. GA4 reads event-scoped source, medium, and campaign as manual traffic signals, and injecting them on a conversion event can overwrite the session's real acquisition source with whatever string the code happened to send. The correct join to the original traffic source is client_id plus session_id, not a re-declared source. The bare fields were removed. This is the same class of attribution corruption analyzed in Chapter 9.

Non-200 responses logged. The POST now inspects its HTTP status and logs anything other than a success. Had this line existed originally, the 404 would have surfaced within a day of the first lead. Silence in the failure path is what let the defect live for months.

The careers orphan, removed not renamed. The careers route had sent a Measurement Protocol event named career_application, a different name than the client-side career_application_submit key event, so it was an orphan even if it had reached GA4. Rather than rename it to match and thereby double-count a low-volume form, the reconstruction removed the server send entirely. The client career_application_submit key event already covers job applications, and the volume does not justify the resilience machinery a lead conversion needs. Deleting code is a legitimate fix.

6.5 The 69 percent, and the discipline of skipping the rest

The hardest decision in the chapter is not a parsing detail. It is what to do when the join key is missing.

The client_id lives in the _ga cookie, which gtag.js sets. A visitor whose browser blocks or never loads gtag.js submits the form (the form is server-validated and does not depend on Google's tag) but arrives at api/contact/route.ts with no _ga cookie and therefore no ga_client_id. The reconstruction measured the size of this gap directly from the source of truth, the database, rather than estimating it. Of 29 leads in contact_submissions, 20 carried a ga_client_id and 9 did not. That is a 69 percent capture rate.

The 31 percent that is missing is not a timing artifact. Once gtag.js sets the _ga cookie, it persists for two years, so a returning visitor who was ever tracked carries the cookie on later visits. The missing third is ad-block and privacy-browser loss: browsers that never let gtag.js run, so the cookie was never written. No amount of code on the server recovers a client_id that the browser was never allowed to create.

Faced with that gap, the original code chose to fake it, substituting server.<timestamp>. The reconstruction chose to skip it. The reasoning is that a synthetic client_id does not repair attribution; it fabricates a user. Each server.<timestamp> value is unique, so every untrackable lead becomes a distinct new user in GA4, arriving from (direct)/(none), inflating the user count and smearing the channel mix with ghosts that correspond to no real acquisition. A property that exists to answer "which service and which county produce leads" is actively degraded by injecting users whose source is a lie. Skipping the send costs one conversion in GA4. Faking it costs the integrity of every aggregate the phantom touches. The reconstruction judged the honest gap cheaper than the dishonest fill, and the row remains in PostgreSQL either way, so the business never loses the lead itself. It loses only the GA4 conversion credit for a lead it cannot honestly attribute.

A live test proved the skip logic behaves as designed. Row id 32 was a real submission through the header quote panel, made on a clean Windows Chrome user-agent that had never visited the site. It landed in the database with an empty ga_client_id, which confirmed the tester's browser sent no _ga cookie, and the code correctly skipped the Measurement Protocol send rather than minting a server.<timestamp> ghost. The id-32 diagnosis is the empirical proof that the gate works: a real lead with no client id produced a database row and no phantom GA4 user.

Two environmental constraints, noted here because they shaped what verification was possible, made id-32 necessary rather than optional. Headless Chrome in the test environment could not reliably load gtag.js from Google's CDN, and Cloudflare Turnstile on the public forms blocked automated submissions. A real human submission was the only way to exercise the full form-to-server path end to end, which is why the definitive evidence for the skip is a hand-made lead rather than a scripted one.

6.6 Verification

The reconstruction treated a change as unverified until three independent signals agreed.

First, the payload validated clean. Sent to https://www.google-analytics.com/debug/mp/collect, the corrected generate_lead returned validationMessages: []. Every field name, type, and reserved-name rule that Google enforces passed. The debug endpoint is the only place Google will tell you a payload is well-formed before it silently swallows a malformed one in production, and an empty validationMessages array is the strongest structural guarantee available.

Second, production accepted it. A real generate_lead sent to the live www.google-analytics.com/mp/collect returned HTTP 204, the same 204 that the wrong host could never produce.

Third, it appeared where a human could see it. The event surfaced in GA4 Realtime within about 20 seconds of the send, registered as the generate_lead key event. Realtime is the fastest confirmation that GA4 not only received the bytes but recorded them as an event and recognized it as a conversion. Debug-clean, a live 204, and a Realtime appearance as a key event together close the loop that the original 404 left permanently open.

The promotion of generate_lead to a key event, deferred so long because the event had never once appeared, followed naturally once the event started arriving. The relationship between this server conversion and the client-side contact_form_submit and quote_form_submit, and the deliberate decision to keep all three as key events until a Google Ads link forces a single canonical choice, belongs to Chapter 7, Conversion Architecture, and is not repeated here.

6.7 The corrected payload, assembled

The complete corrected send, anonymized, reads roughly as follows:

const clientId = parseClientId(cookies["_ga"]);          // GA1.1.X.Y → "X.Y"
if (!clientId) return;                                     // skip, do not fake

const sessionId = parseSessionId(cookies[`_ga_${CONTAINER}`]);
const value = LEAD_VALUE_USD[serviceKey] ?? LEAD_VALUE_USD.other;

const body = {
  client_id: clientId,
  ip_override: requestIp,
  events: [{
    name: "generate_lead",
    params: {
      currency: "USD",
      value,
      form_source: formSource,
      service_type: serviceKey,
      session_id: sessionId ?? undefined,
      engagement_time_msec: 1,
    },
  }],
};

const res = await fetch(
  `https://www.google-analytics.com/mp/collect?measurement_id=G-WH410Z73V1&api_secret=${API_SECRET}`,
  { method: "POST", body: JSON.stringify(body) },
);
if (res.status !== 204) {
  console.error("MP send failed", res.status, await res.text());
}

Every element of that block traces to a named defect: the host that returned 404, the gate that replaced the phantom fallback, the two cookie parsers that supply client_id and session_id, the engagement_time_msec that keeps the event in session counts, the ip_override that geolocates a Salinas homeowner to Salinas, the per-service value that distinguishes a fumigation from an ant call, the aligned form_source parameter, the absent bare source fields, and the status check that will never let a silent 404 hide again.

6.8 What this chapter establishes and what it defers

The server now speaks, correctly, to the endpoint that records events, for the 69 percent of leads it can honestly attribute, and it stays silent for the 31 percent it cannot rather than fabricating them. That is the durable backbone the property lacked from creation through the audit. The custom dimensions this event populates (service_type, form_source, and the bounded error_message) are registered and governed in Chapter 8. The identity, session, and value semantics touched here are analyzed as a system in Chapter 9. The monetary value becomes real revenue, and the client_id becomes an offline conversion upload, in Chapter 11. The consent posture that governs whether the browser ever sets the _ga cookie in the first place is the subject of Chapter 10, Privacy and Consent by Design. What this chapter fixed is the mechanism: a server that already writes every lead to PostgreSQL now also, and reliably, writes the ones it can attribute to GA4.