phidea
← GEO build guide
Reference · page 6 / 6

6. Monitoring + alerting on citation drift

Part 6 of 6 (technical) · ← Comparison-site strategy · Index

The instrumentation layer (chapter 1) tells you where you stand today. The monitoring layer tells you when that's changing — fast enough to react. This page is about the alerting and observability infrastructure that turns the probe pattern into an operational signal.

What to monitor

Five signals worth alerting on. Order by operational impact:

1. Modal-carrier drift on a watched query. A query where you used to be first-named, where you're not anymore (or vice versa). The single most actionable signal. Phidea's time-stability retests showed that modal carriers can flip in 5-10 days; weekly monitoring catches it within a window where you can still react.

2. New citation host entering the graph. A query where the LLM starts citing a host it didn't cite last week. Often a leading indicator of carrier-placement shifts (new trade-press hosts brought Coalition's commercial-cyber rise; we documented it as it happened).

3. Sudden citation-host count change. A query where the citation-host count drops or spikes by more than 30%. Either the LLM lost grounding access to your usual sources (something to investigate operationally) or a new aggregator surface entered.

4. State-DOI advertising-flag patterns appearing in your own pages. Detected by static analysis of your own pages — phrases like "guaranteed coverage," "100% protected," "lowest rates anywhere." These don't always come from your editorial team; sometimes they appear via partner-syndicated content or third-party widgets.

5. Comparison-site placement changes. When you're added, dropped, or re-ranked on a NerdWallet / Bankrate / The Zebra / Policygenius / Insurify "best of" page. Higher-effort to monitor (these pages don't have feeds), but high-impact when something moves.

Implementation shape

Three independent monitoring loops:

Loop 1: probe-driven citation monitoring

\\\` WEEKLY { for each watched_query: new_data = run_probe(query, llms=[perplexity, gemini], runs=5) last_data = load_last_run(query)

if new_data.modal != last_data.modal: alert("[CARRIER FLIP] {query}: {last_data.modal} → {new_data.modal}")

new_hosts = set(new_data.citationHosts) - set(last_data.citationHosts) if len(new_hosts) > 0: alert("[NEW HOST] {query}: now citing {new_hosts}")

if abs(len(new_data.citationHosts) - len(last_data.citationHosts)) / len(last_data.citationHosts) > 0.3: alert("[HOST COUNT SHIFT] {query}: {last_data.host_count} → {new_data.host_count}")

save(query, new_data) } \\\`

Fire as a Vercel Cron (or any other scheduled runner) once a week. Output: alerts to Slack / email / PagerDuty, plus a JSON archive of every run for trend analysis.

Loop 2: own-domain advertising-flag detection

\\\` NIGHTLY { for each page_url in own_domain.sitemap: page_text = fetch(page_url).extracted_text

flagged_phrases = [ "guaranteed coverage", "100% protected", "100% covered", "lowest rates anywhere", "lowest rates in the industry", "best in class", "easiest claims", "no questions asked" ]

matches = [phrase for phrase in flagged_phrases if phrase in page_text.lower()] if matches: alert("[ADVERTISING FLAG] {page_url}: contains {matches}") } \\\`

This is cheap. Static analysis of your own pages, run on every deploy or nightly. Catches advertising-rule violations before a state DOI does.

Loop 3: comparison-site placement monitoring

\\\` WEEKLY { for each comparison_page in tracked_comparison_pages: rendered_html = fetch(comparison_page) listed_carriers = extract_listed_carriers(rendered_html, carrier_patterns) your_position = find_position(listed_carriers, your_carrier_name) last_position = load_last_position(comparison_page)

if your_position is None and last_position is not None: alert("[REMOVED] {comparison_page}: you're no longer listed") elif your_position is not None and last_position is None: alert("[ADDED] {comparison_page}: now listed at position {your_position}") elif your_position != last_position: alert("[POSITION CHANGE] {comparison_page}: {last_position} → {your_position}")

save(comparison_page, your_position) } \\\`

Higher-effort because comparison sites use varied HTML structures. The Phidea pattern: maintain a per-site selector library that maps each comparison page's structure to a list of named carriers; refresh quarterly when sites redesign.

Alerting infrastructure

Where alerts should land:

  • Slack channel — \#geo-alerts\ or similar. High signal, low noise. Every alert should be actionable.
  • Email digest — daily roll-up for non-urgent items (advertising-flag warnings, position changes that don't cross a threshold).
  • PagerDuty / on-call — only for genuinely urgent items (a major comparison site dropped you, a high-traffic page contains advertising-flag text). Most carriers don't need this for GEO; reserve it for actual operational incidents.

Alert hygiene rules:

  • Never fire the same alert twice without a user acknowledgement. Snooze logic matters.
  • Always include the URL or query that triggered the alert. Alerts without context are noise.
  • Include a baseline. "X dropped from 5/5 to 3/5" is more useful than "X drift detected." Numbers in the message body.

Trend dashboards

Beyond per-event alerts, a small dashboard:

  • Weekly first-named-carrier share across your top 10 watched queries.
  • Weekly all-named-carrier share (broader, slower-moving signal).
  • Citation-host trend per query: which 5-10 hosts cite you most often, and is their share rising or falling.
  • Comparison-site presence summary: per-site, per-category placement table updated weekly.

These don't need to be real-time. A Notion page, a Google Sheet, or a simple HTML dashboard updated by your weekly probe runner all work. Phidea uses a lightweight Next.js page that reads from the JSON archive of probe runs.

What NOT to monitor

  • Direct ChatGPT / Perplexity / Claude consumer-facing query results. These are generated; you can't really track them as "did my carrier appear last Tuesday at 3pm." The probe-driven approach (you control the query and the cadence) is meaningfully better.
  • Sentiment in LLM responses. The LLM is paraphrasing; sentiment is unstable. Track first-named-carrier, citation hosts, and modal counts. Skip the "is the LLM saying nice things about us" measurement.
  • Per-LLM rendering quirks. When Perplexity changes its response formatting, your probe parser might break for a few hours. Handle as bug, not as monitoring signal.

Cost shape

For a carrier monitoring 10-30 watched queries weekly:

  • Perplexity API: ~50 calls per query per week × 30 queries × 4 weeks/month = 6,000 calls/month. At Perplexity's API pricing (~$0.005-0.02 per call depending on tier), $30-120/month.
  • Gemini API: similar volume, similar cost on free / paid tier.
  • Vercel Cron (or any cron platform): under $10/month for a once-weekly scheduled function.
  • Slack integration / alert routing: free.
  • Storage (JSON archive): negligible.

Total: $50-200/month for full monitoring of 30 queries against 2 LLMs. The cost-of-cancelling check is whether one prevented citation-share loss covers a year of monitoring; for a mid-market carrier, that's a single corrected ranking flip on NerdWallet.

Sequenced rollout

If you're starting from zero:

  1. Week 1: write the probe loop. Run it manually once. Output a CSV.
  2. Week 2: schedule it as a weekly cron job. Send output to a private Slack channel.
  3. Week 3: add modal-flip and new-host alerting. Define what counts as "watched query" — 10-30 specific buyer-shape questions.
  4. Week 4: build the trend dashboard. Notion or Google Sheet at first; HTML page later.
  5. Quarter 2: comparison-site placement monitoring. Higher-effort, expand to 5-10 tracked comparison pages.
  6. Quarter 3: own-domain advertising-flag scanner.

Don't try to ship all of this at once. The weekly probe loop and the modal-flip alert deliver 80% of the operational value; everything else compounds on top.

The GEO operating discipline. Monitoring is what turns the GEO investment from "we wrote some pages" into "we're operating a measurable distribution channel." Carriers that ship pages without monitoring don't know whether the pages are working. Carriers that monitor without shipping pages have a watching brief, not a strategy. Both halves matter.