Evaluating All The Campaigns In The World

The ad industry has never been short on hyperbole and unreliability. It has been a problem for decades.

The era of online advertising has only deepened this problem.

One of the troubling new phenomena is the way third parties -- the trade press, the business press, academia, and trade organizations -- have become complicit in the reckless boosterism of online advertising at the expense of the critical thinking and skepticism that used to be considered their responsibility.

This was reinforced last week when I read an article on MediaPost entitled:  

"WARC: Over Half Of The World's Most Impactful Campaigns Are Digital-Led."

WARC is an acronym for an organization that modestly calls itself the "World Advertising Research Center."

The thrust of the article was this:

"The World Advertising Research Center (Warc) has issued its annual rankings of the most effective marketing campaigns...The Warc analysis found that 55% of the world’s most impactful campaigns are digital-led."

Upon reading this, my first thought was this: How do you evaluate all the marketing campaigns in the world? There must be tens of thousands of marketing campaigns launched every year. You'd have to have thousands of people on staff, speaking dozens of languages, monitoring all media, spending months on analyzing sales data and consumer sentiment. What a remarkable undertaking!

My next thought was, hold on, this must be a click-bait headline from a publisher intentionally misrepresenting the WARC story to make a few extra bucks. But then I went to the WARC website and there it was plain as day...

 "The Warc 100 is a ranking of the world’s best marketing campaigns..."

I was hooked. What an amazing project. I couldn't wait to discover how they determined this. Until I read the rest of the sentence...

"...based on performance in effectiveness and strategy competitions."

Wait a minute... "based on performance in effectiveness and strategy competitions?" What the hell does that mean? So I dug further.

This led me to a page that described what MediaPost called WARC's "rigorous methodology developed in consultation with King's College London." Now I'm really impressed. So I read on.

What I found was a truly majestic exercise in making the obvious incomprehensible. I invite you to read the methodology here, but I warn you -- don't do it near sharp objects.

Once you fight your way through the hogwash and the labyrinthine description, you suddenly realize that the WARC 100 is essentially an award for winning awards.

WARC, of course, wants to blind us with science, so the way they put it is...

"Warc tracks different advertising competitions around the world – all of them require entrants to show the impact of a campaign. Campaigns (and the brands and agencies behind them) are awarded points based on the prizes they win in those competitions."

Which, of course, is thinly disguised double-talk for it's an award for winning awards.

In order to swallow WARC's assertion that these are the world's best campaigns, you have to accept all of the following:

• Advertising awards are a reliable proxy for effectiveness
• WARC's choice of which award shows to consider (which they refuse to divulge) are a valid sample
• WARC's arbitrary weighting scheme is scientifically defensible.

If any one of these assumptions is erroneous, the whole formulation falls apart. It is my opinion that every one of the premises is erroneous.

Furthermore, a campaign cannot even be considered unless someone entered it in one of the awards shows WARC draws on. That alone is enough to make its claim of identifying the world's most effective campaigns spurious.

Anyone who has spent 15 minutes in an agency knows the following:

• Some agencies spend hundreds of thousand of dollars entering awards shows every year (sometimes called "award-whores".) Some spend nothing.
• Some agencies have people on staff who are dedicated specialists in generating "case histories" that are effective at winning awards.
• Some agencies have people who specialize in knowing the most favorable categories to enter to produce an award.

Further, anyone who has ever judged an award show knows the following:

• On every judging panel in every awards show there are brilliant people and complete fools. It is not unheard of for the fools to prevail.
• The judging itself is like every other aspect of life -- frequently ruled by emotion, not rationality.

Unlike many in advertising, I am not an awards hater. I think great stuff deserves to be recognized. Advertising and marketing people do a lot of suffering and are entitled to some satisfaction once in a while. If an award does that, I'm happy for them.

When I was a copywriter and a creative director I was always proud and gratified when we won a major award. But I never deluded myself into the silly belief that an award was in any way an accurate reflection of effectiveness. I knew how the game was played.

The WARC 100 winners can arguably be billed as "the world's most awarded marketing campaigns." But to bill them as "the world's most effective campaigns" is simply brazen puffery. And to draw a conclusion from this that the best campaigns are "digital-led" is fatuous and irresponsible.

The effectiveness of advertising is a serious business that deserves serious scrutiny. If you are going to make an extraordinary claim -- that you have uncovered the world's most impactful campaigns -- you have an obligation to demonstrate that you have devised a judicious, reliable methodology for determining this..

WARC's methodology is superficial and unconvincing.