Data Quality and Governance

13.1 The Premise: Correctness Decays

A GA4 property is not a building you finish and photograph. It is a garden that reverts. The property described across this dissertation was, at the moment of the July 2026 audit, a working example of that reversion. Its server-side generate_lead event had never once reached Google because both api/contact/route.ts line 384 and api/careers/route.ts line 288 posted to https://www.google.com/mp/collect, a host that returns HTTP 404, rather than the documented https://www.google-analytics.com/mp/collect, which returns HTTP 204. Chapter 6, The Server-Side Measurement Protocol, dissects that defect and the correctness rewrite that resolved it. This chapter answers a harder question: the bug lived across the entire lifetime of the property, and nobody noticed. The measurement was wrong from creation on 2026-04-24, and the wrongness was silent. No dashboard turned red. No report showed a gap. The event simply did not exist, and its absence looked identical to a business that had not yet generated a lead.

Governance is the discipline that would have caught that. It is the set of standing rules, filters, contracts, and reconciliation checks that keep a correct property correct after the engineer who corrected it moves on. A property can be built right and still rot, because traffic patterns shift, developers ship new forms, a marketer renames a UTM campaign, and an ad-block update changes the client-side capture rate. Without governance, each of those events degrades the data by a few percent, and the degradations compound until a quarter of analysis rests on numbers no one can defend. This chapter treats data quality not as a one-time cleanup but as a running control loop, and it is deliberately concrete about the specific failure modes this pest-control property exhibited or remains exposed to.

13.2 Internal Traffic Filters and the Server-Side Blind Spot

The first and most banal source of contamination is the business measuring itself. The owner loads the homepage to check a price. A developer submits a test lead through the header quote panel. A contractor QA-clicks every call button on a Tuesday. Every one of those actions is a real GA4 event, indistinguishable in the raw stream from a genuine prospect in Salinas, and at 527 sessions over thirty days a handful of internal sessions is not rounding error. It is a measurable fraction of the sample.

GA4's native answer is the internal traffic filter. You define an "internal traffic" rule in the data stream settings that matches a source IP address or CIDR range, which stamps matching hits with a traffic_type value of internal, and then you activate a data filter at the property level that excludes those hits from processing. The filter is retentionless in the sense that excluded events never enter the reports at all. This is standard hygiene and worth doing for the office IP and the owner's home IP.

The subtlety this property exposes, and the reason internal-traffic governance cannot stop at the IP filter, is that the most important internal event does not originate from a browser at all. The generate_lead conversion fires server-side, from the Next.js API route running in a Docker container on the briccs Coolify/Traefik VPS. When a developer submits a test lead, the client-side contact_form_submit may carry the office IP and get caught by the filter, but the server-side Measurement Protocol call is a POST from the datacenter. Worse, after the Chapter 6 rewrite, that call sets ip_override to the request IP so the conversion geolocates to the prospect rather than to the VPS. An IP-based internal-traffic filter keyed on the office network therefore has nothing reliable to match: the server event either presents the datacenter's IP or presents whatever IP the request carried, and in neither case does it inherit the browser-side internal classification. The filter that protects the client stream is structurally blind to the server stream.

Keeping test leads out of production data consequently requires a guard at the code level, not only in the GA4 console. Two mechanisms combine. First, set the GA4 internal-traffic rule to match the traffic_type event parameter rather than IP, and have the server code send traffic_type: "internal" on the Measurement Protocol payload whenever the request originates from a known internal context, a staging environment, or a test flag. GA4 will then classify and exclude that server event on the same footing as browser hits. Second, gate the send itself: the rewritten routes already skip the Measurement Protocol call when no real ga_client_id is present, which, as Section 13.6 shows, is exactly what happened when a tester submitted row id 32 from a clean Windows Chrome browser that carried no _ga cookie. That skip is a correctness feature (it refuses to mint a phantom user) and doubles as a governance feature (a cookie-less QA submission produces no server conversion to contaminate). The general principle: a filter you configure in a UI can only see what reaches it, and a server-side event routes around browser-scoped controls, so every server sender needs its own test-suppression path.

13.3 Bots and Unwanted Referrals

Two contamination sources sit below the internal-traffic layer. GA4 excludes traffic from known bots and spiders automatically, matching the IAB International Spiders and Bots list, and this exclusion is neither configurable nor visible; you cannot turn it off and you cannot audit what it removed. For a Cloudflare-fronted site the practical effect is that ordinary crawler noise is handled upstream by Cloudflare and downstream by GA4, and the middle is thin. The residue that survives is sophisticated automation that mimics a browser, and the honest governance posture is that GA4 will not catch all of it. The reconciliation check in Section 13.7 is the backstop, because a bot that submits the contact form shows up as a database row, and a divergence between form-fill volume and human-plausible session volume is visible there.

Unwanted referrals are the second source. The thirty-day baseline showed referral traffic from facebook.com, blue.bbb.org, chatgpt.com (tagged ai-assistant), and yelp.com. None of those is a problem to suppress; they are legitimate discovery paths and belong in the channel mix. The referral-exclusion setting earns its keep in a different scenario: when a payment processor, an authentication redirect, or a subdomain of your own site bounces the user back and GA4 reads the return hop as a new referral session, severing attribution from the original source. This property has no online transaction and no payment redirect, so the acute version of the problem does not apply. The governance rule is still worth stating: review the referral list quarterly, and the moment a Brevo tracking domain, a Cloudflare challenge host, or a www-to-apex hop starts appearing as a referrer, add it to the unwanted-referrals list before it fragments sessions. Absence of the problem today is not a reason to stop watching for it.

13.4 Data Retention as a Governance Decision

Event data retention is a one-field setting with outsized consequences, and the audit found it at the floor. The property sat at TWO_MONTHS. Phase 0 raised it to FOURTEEN_MONTHS with a single dataRetentionSettings PATCH through the Admin API. The change is not retroactive, so data already aged out stays gone, and it takes roughly 24 hours to apply.

The reason this is a governance decision and not a checkbox is that the retention window does not mean what most operators assume. GA4's standard aggregate reports retain their data effectively indefinitely; the retention setting governs only the data available to Explorations and funnel analyses that pivot on event-scoped or user-scoped custom dimensions. The eight custom dimensions this property carries (cta_location, session_source, page_type, job_slug, location_slug, service_slug, form_source, pest_slug, detailed in Chapter 8, Event Taxonomy and Custom Dimensions) are precisely the fields that a two-month cap would have amputated from any year-over-year exploration. For a business whose demand is seasonal, with termite swarms in spring and rodents in cooler months, the analytically valuable question is how this March's by-service and by-county lead mix compares to a window the business has not yet lived through. A two-month retention window makes that comparison impossible by construction, because the comparison period expires before the comparison can be run. Raising the ceiling to fourteen months buys exactly one year-over-year lookback plus a two-month margin, which is the minimum viable window for a seasonal operator and the maximum a GA4 standard property allows.

The governance framing is that retention is a claim about what analysis the business intends to be able to perform later, made at a moment when that analysis is not yet needed and therefore easy to under-provision. The default floor optimizes for storage minimalism the business does not care about. The correct setting optimizes for the seasonal comparison the business will care about in eleven months. Because the setting is not retroactive, the cost of getting it wrong is unrecoverable, which is the defining property of a decision that belongs under governance rather than under configuration.

13.5 The Measurement Plan as a Living Contract

The artifact that holds all of this together is MEASUREMENT_PLAN.md, committed to the repository during Phase 2. A measurement plan that lives in a slide deck or a shared doc is a description; a measurement plan that lives in the codebase next to the code that implements it is a contract, because it sits in the same review flow as the code and a divergence between the two is visible in a diff. When a developer adds a form, the plan is the file that says which event that form must fire, with which parameters, and whether that event is a key event. When a reviewer approves the pull request, they are approving the plan and the implementation together.

The plan encodes the register-before-send rule, which Chapter 8 establishes as the taxonomy's first law and which this chapter enforces as governance. A parameter that the code sends but that no custom dimension has registered is collected by GA4 and then discarded from every report, invisibly. The event carries the value; the reports never show it. This is the quietest failure in the entire GA4 surface because nothing errors: the payload is valid, the event lands, and the parameter evaporates at the reporting boundary. Phase 1 registered service_type, cta_label, and error_message specifically so that the values the code already sent would stop evaporating. The governance rule that prevents recurrence is sequential and non-negotiable: register the custom dimension in GA4 before, or in the same change as, the code that first sends the parameter. The plan is where that ordering is written down and where a reviewer checks it.

The registration of error_message also demonstrates the safety half of the rule. That dimension is safe to register only because the code sends bounded codes, server_error and network_error, never raw user input. Registering a dimension that could carry free text would turn the taxonomy into a PII exfiltration path, since custom-dimension values flow into reports and, eventually, into BigQuery. The plan records not just what is registered but why it is safe to register, which is the kind of institutional memory that prevents a future developer from widening error_message to include a user's typed message without understanding what they are doing.

13.6 The QA Workflow: DebugView and the Debug Endpoint

Governance needs a place to catch defects before they ship, and GA4 provides two instruments the audit used as a standing QA loop. For client-side events, DebugView renders a live per-user stream of events and parameters when the browser is in debug mode, which turns the abstract question "is my page_view firing with the right page_type" into a visible answer. The page_view centralization was verified this way: headless Chrome read the dataLayer and confirmed the homepage pushed {page_type: home}, /services/termite-control pushed {page_type: service, service_slug: termite-control}, and /locations/salinas pushed {page_type: location, location_slug: salinas}, one push per route, no duplicates. The duplication that DebugView would have exposed had it been running earlier is exactly the double-fire that Enhanced Measurement's pageChangesEnabled produced on client navigation, which Phase 1 disabled.

For server-side events, the analogous instrument is the debug Measurement Protocol endpoint, https://www.google-analytics.com/debug/mp/collect. It accepts the same payload as the production endpoint and returns a validationMessages array describing anything malformed, without recording the event. The rewritten generate_lead payload validated against it and returned validationMessages: [], which is the empty-array result that means the structure, the parameter names, and the event shape all conform. Only after that clean validation was a production generate_lead sent to the live endpoint, where it returned HTTP 204 and appeared in GA4 Realtime within about 20 seconds as a registered key event.

# QA gate, in order:
# 1. Validate structure, record nothing:
POST https://www.google-analytics.com/debug/mp/collect   -> expect validationMessages: []
# 2. Send for real, confirm transport:
POST https://www.google-analytics.com/mp/collect          -> expect HTTP 204
# 3. Confirm arrival:
#    GA4 Realtime shows generate_lead within ~20s

The workflow is a gate, not a suggestion. The debug endpoint answers "is the payload well-formed," the live 204 answers "did GA4 accept the transport," and Realtime answers "did the event actually arrive and register." The original 404 bug would have been caught at step two by anyone who looked, because a 404 is not a 204, and the entire post-mortem reduces to the observation that no one ran step two. Two environmental constraints shaped how this gate ran in practice and are worth recording for anyone reproducing it: headless Chrome in the test environment could not reliably load gtag.js from Google's CDN, and Cloudflare Turnstile on the public forms blocked automated submissions, so exercising the full form-to-server path required a real human submission rather than a scripted one.

13.7 Reconciliation as a Canary

Every control described so far is preventive. Reconciliation is detective, and it is the single most important governance mechanism for this property because it is the check that would have caught the 404 bug on day one. The mechanism is a comparison between two independent counts of the same real-world event. GA4 reports a generate_lead key-event count. PostgreSQL holds the contact_submissions table, one row per lead, written by the same API route that fires the conversion. Those two numbers measure the same thing through two different pipes, and a healthy property keeps them in a stable, known ratio.

The ratio is not 1:1, and understanding why is what makes the canary trustworthy rather than a source of false alarms. The server generate_lead fires only when a real ga_client_id is present, because the rewrite deliberately skips the send rather than fabricate a synthetic server.<timestamp> client id. The database captures every lead regardless. The measured capture rate is the governing constant: of 29 leads in the database, 20 carried a ga_client_id and 9 did not, a 69 percent capture rate, and the missing 31 percent is ad-block and privacy-browser loss rather than timing, because the _ga cookie persists two years once gtag sets it. So the expected relationship is that GA4's generate_lead count runs at roughly 69 percent of the database row count over any comparable window, drifting slowly as the ad-block population shifts.

That expected ratio is the canary's calibration. The alert does not check for equality; it checks that GA4's count stays inside a band around 69 percent of the database count. The original bug drives that ratio to zero: the database keeps accumulating rows while GA4's generate_lead count stays flat at nothing, which is the exact signature of a silent collection failure and the exact state the property sat in for its whole life. A ratio that collapses toward zero means the pipe is severed. A ratio that climbs toward or past 1.0 means something is double-firing or the client id skip logic regressed. A ratio that drifts down gradually means the ad-block share is rising, which is intelligence, not an alarm. The reconciliation query runs on a schedule, compares the two counts over a trailing window, and pages a human when the ratio leaves its band.

-- Trailing-7-day lead count from the source of truth:
SELECT count(*) AS db_leads
FROM contact_submissions
WHERE created_at >= now() - interval '7 days';
-- Compare against GA4 generate_lead key-event count (Data API) for the same window.
-- Healthy: ga_leads / db_leads ~ 0.69.  Alarm: ratio -> 0 (collection severed).

The database is the source of truth here for a specific reason: it is written by application code inside the same transaction path as the business action, it is not subject to ad-block, and it does not depend on a third-party endpoint being reachable. GA4 is the instrument being audited; PostgreSQL is the reference standard the instrument is checked against. That asymmetry is what makes the check meaningful. Checking GA4 against itself would have revealed nothing, because a property that never received an event reports zero with total internal consistency.

13.8 Signals to Watch

Beyond the reconciliation canary, two continuous signals report on the health of the fixes Phase 1 shipped. The first is the Direct/(none) share. The baseline showed 234 of 527 sessions as (direct)/(none), and part of that mass was artifact rather than truth: the pre-fix Measurement Protocol payload omitted session_id and engagement_time_msec, so even a correctly transported event would have attributed to (direct)/(none) for lack of session stitching. Chapter 6 added session_id, parsed from the _ga_<container> cookie across both the legacy GS1 dot-delimited and newer GS2 dollar-delimited formats, plus engagement_time_msec. The governance expectation is that the direct share falls after this change as previously-unattributable events acquire their real source. Watching that number trend down is the confirmation that session stitching works; a direct share that stays pinned near its old level, or climbs, means the session id parse is failing silently and events are still landing unattributed.

The second signal is the form_error rate. The client fires form_error with form_source and a bounded error_message whenever a submission fails, and the newly registered error_message dimension makes the failure reason legible in reports. A low, stable form_error rate is the resting state. A sudden climb is the earliest available warning that a form is broken, an API route is throwing, or a validation regression is rejecting good submissions, and because the parameter distinguishes server_error from network_error the report points at whether the failure is on the server or the transport. This signal matters disproportionately for a lead-driven business with no online transaction, because a broken form is not a degraded metric; it is lost revenue arriving as silence. The form_error rate converts that silence into a number that can trip an alert.

13.9 What Breaks Silently, and Who Holds the Keys

The connecting theme of this chapter is that the dangerous failures in GA4 are the ones that do not error. A 404 endpoint returns cleanly to code that ignores the status. An unregistered parameter is accepted and discarded. A retention floor expires data on a schedule no one is watching. A missing session_id attributes conversions to nobody in particular. None of these throws an exception, fails a build, or reddens a dashboard, and each degrades the data by an amount too small to notice in a week and too large to ignore over a quarter. The property in this study lost the entirety of its server-side conversion data for its whole lifetime, and the loss was invisible precisely because nothing complained.

Governance is the practice of installing complaints where the system provides none. The internal-traffic filter plus the server-side traffic_type guard complains about self-measurement. The register-before-send rule, enforced through the committed MEASUREMENT_PLAN.md, complains in code review about parameters that would evaporate. The DebugView and debug-endpoint gate complains before a change ships. The PostgreSQL-to-GA4 reconciliation ratio complains when collection goes silent, which is the one complaint that would have caught the founding defect. The Direct/(none) and form_error signals complain when a shipped fix regresses.

A final governance concern is that all of these controls are themselves configuration, and configuration has an owner. Chapter 15, Access and Permissions Model, details the boundary: a GA4 Editor can change custom dimensions, key events, retention settings, and Enhanced Measurement, which covers retention and taxonomy governance, while managing user access requires Administrator and creating the BigQuery export link additionally requires Owner on the destination Cloud project that no GA4 role grants, and reading accessBindings requires the distinct analytics.manage.users scope that a token minted only with analytics.edit cannot satisfy. Governance that anyone can silently alter is not governance. The permissions model is what makes the retention setting, the internal-traffic filter, and the key-event definitions hold their shape between the moment an engineer sets them correctly and the next audit that confirms they are still correct. The keys and the rules are the same discipline viewed from two sides, and a property keeps its correctness only as long as both are maintained together.