Case study · SEO analytics

Measuring the SEO cost of a pop-up ad

Search Console · BigQuery · Python + Cloud Scheduler ETL · Tableau

Question: a pop-up ad went live on the site. Did it hurt organic search performance — and by how much? Answer: yes, measurably. Eight page categories dropped more than one SERP position in the launch week (average −1.81), and CTR fell across most landing pages. The evidence supported withdrawing the ad.

Building the data pipeline

Search Console has no built-in transfer to BigQuery, so I built a lightweight ETL: a Python script on Cloud Scheduler, costing effectively nothing, landing daily SERP data in BigQuery. Daily granularity is what makes event analysis like this possible — the standard Search Console UI can't isolate a two-week window cleanly.

Impact on rank

Scatter plot of week-on-week SERP position change — Week-on-week position change by page category. The pop-up launched late W35 and was removed late W37.

Week-to-week position changes normally hug zero. In W36 — immediately after launch — nearly all page categories lost rank, with the biggest movers dropping several positions, and a partial recovery only after the ad was withdrawn.

Table of rank changes by page category — Eight page categories dropped more than one position in W36, averaging −1.81. Most had not recovered within the observation window.

Impact on click-through

Scatter plot with movement arrows showing position and CTR change — Movement arrows: average position vs. SERP CTR, pre-launch vs. pre-withdrawal. The majority of pages lost both rank and CTR.

Using CTR neutralises the traffic weight of individual landing pages and isolates the change itself. The pattern is consistent: the pop-up depressed both position and click-through.

Does a stronger page resist the shock?

Trend of rank versus resistance — Preliminary pattern: weaker landing pages were more prone to rank loss during the shock weeks.

A secondary question: do high-ranking pages absorb a shock better? The relationship isn't conclusive, but the tendency was there — weaker pages fell harder in W36–W37.

Why this matters

This is the kind of analysis that settles an internal debate. "The pop-up converts" and "the pop-up hurts SEO" were both opinions until the pipeline existed; after it, the trade-off was quantified and the decision was quick. The same setup generalises to any before/after feature evaluation, query-level monitoring, or long-term SEO tracking.

← Back to all work