A few weeks back, Google revealed that it would be altering how Quality Scores are reported. They clarified that it’s not about how these scores are calculated, but rather how they are presented. This appears to be a clear indication from Google that the 1-10 Quality Score shown in AdWords is merely a “front” for your actual “real” score.
If nothing changed about your “real” Quality Score, and this is truly just a glossy reporting change, then there should be no effect on your other metrics. BUT, there’s another way to interpret Google’s announcement. Google might be saying that they haven’t changed the factors that go into Quality Score, but they have changed how they normalize the results (which is how they can compare disparate advertisers). If that’s true, then there could be substantial impacts to your account. Quality Score is half the equation to determine your Ad Rank and an important part of determining the price you pay.
Either way, if you carefully track your Quality Scores (or ask your agency to do it for you), you probably noticed a sudden change with no logical explanation. Since the announcement by Google, we’ve been watching our accounts carefully, trying to figure out just exactly what Google was planning to do.
The first challenge was looking for the date of the change. Google hadn’t told us exactly when it was coming, so we had to stay on top of our tracking and keep “slicing and dicing” the data. After all, we didn’t even know exactly what we were looking for. Was it going to affect all keywords? Just branded terms? Just low-quality score keywords? Generally raise scores? Generally lower them?
All in all, it actually took us until a few days after the fact to be sure of what we’d seen. And what we found was …
Google Took Away Your 10’s
Quality Scores (the ones we can see) have generally fallen into a bell curve. For a long time, when we asked prospective WordStream clients “What do your Quality Scores look like today?” they all said “Mostly 7s.” More recently, they said “Mostly 4s.” But even in that period, most people had a healthy number of 10s that drove a good amount of their traffic – branded terms, by and large. Which makes sense. As Elisa pointed out last week, you are obviously the most relevant advertiser on your brand. Now, that’s simply not true. Some of our brand terms still have Quality Scores of 10. But a lot of them now have 7s and 8s.
Without going into specifics, our clients’ accounts in the past have generally gotten about 20% of their impressions from keywords with a Quality Score of 4 and another 20% from their 10’s, with a relatively smooth curve between. It ends up looking like a roller coaster. Since the change, however, we see something that looks more like a traditional bell curve, with about 20% of the impressions coming from 5’s and a smooth curving going down in each direction. (Except for that funny little blip tail at Quality Score 1. Isn’t that weird?)
What does all this mean?
A few things. First, please don’t punish your agency for your recent drop in Quality Score 10’s. It’s all Google’s fault. (And Google’s not punishing you, either.)
Second, mark this date in your calendar. If you’ve been measuring the impact of your optimizations on your Quality Scores, August 2 will be a “break” in your data. Similarly, if you are a huge quantitative analysis nut and are making optimizations based on a correlation between CTR, Average Position, and Quality Score, you’ll need to redo all that math.
Only time will tell whether this change in Quality Score is just a reporting change or something more significant. I would expect, over time, for average cost-per-click to go up if Google is no longer giving out the Quality-Score-10 discount as freely.
Caveats
I should close by saying that (hopefully obviously) I’m working with a small sample size. I looked at data from WordStream’s client base, which tends to consist of small and medium-sized businesses rather than huge enterprise accounts. And I tried to control for other changes that might have impacted Quality Score (basically looking for campaigns where “all else is equal”), but our clients are constantly optimizing their campaigns. All this is to say “YMMV” – your mileage may vary. In fact, one other marketer did find something very different (a shift to higher scores).
How about your account? Have you been tracking your Quality Score changes? Did you notice a loss of 10’s? Sound off in the comments.