Well, well, well. Remember that article I wrote back in July about how Pokémon Go wasn’t going to be magic fairy dust that makes billions of dollars for local businesses?
Forbes now has an article from Alex Konrad affirming this. There’s no denying that Pokémon Go has been an overwhelming success, even as the typical user dropoff has commenced. It’s likely not helping that the weather is getting colder too. That said, all of the inspirational stories about that pizzeria that made a ton of money as a result of a few lures, that ice cream shop the game ostensibly “saved” seem to have done nothing but act as demonstrations of the cherry picking fallacy.
I lampooned the cottage industry of Pokémon Go “business guides” and “marketing toolkits” that promptly flooded the Internet like memes after a 2016 Presidential debate. All of these “guides” were lacking in anything other than wishful thinking to be riding a wave and a couple of unsubtle suggestions that you should call the respective company and work with them. I’ve stated again and again that just because a particular business finds success with a particular channel doesn’t mean that all businesses will. Correlation, as always, does not imply causation. Jason Shu of small business database startup Womply was able to confirm this.
Jason Shu read [the case studies] excitedly. After more than 13 years at Boeing, Shu leads data science and analytics at Womply, a five-year-old startup in San Francisco with access to a database of more than 2 million merchants, many of them small businesses, through partnerships with the big credit card processors. Shu, an analyst and a data engineer dove in to corroborate the exciting trend. But they couldn’t find one.
“We were disappointed to find that despite all the stories we were told, the boost wasn’t necessarily true from a statistical sense,” Shu says. “We want people to know that when the next big thing hits the markets with a lot of following.”
I’ll go even further and say that you shouldn’t even care about “the next big thing.” You should be pursuing what works for your business. Be it newsletter marketing, Facebook advertising, or freaking Google+ if it’s demonstrating results. There are a myriad of factors that come into play when looking at a potential channel to reach customers: Business size, industry, product type, product or service, product tangibility, the list goes on. It’s impossible for any one channel, even one as widespread as Pokémon Go, to account for the flavors and subflavors of a business in a way that makes it the next “big thing” everybody can profit from.
Or, as Shu puts it:
Shu doesn’t doubt that individual businesses like those who spoke to press saw a major jolt in activity because of the game. But for many others that saw better business in July, a host of other factors could have been involved. According to Womply’s data, the difference in sales between merchants embracing Pokémon GO and those not was statistically insignificant.
That’s really what irked me when the press, marketing companies and bloggers started fawning over Pokémon Go back in July. It can’t just be a new, exciting opportunity for the right businesses with a measured response to wait and see if it’s sustainable. It has to be the next hot new thing that you need to sign up for or you’ll go bankrupt.
Of course, it didn’t matter that, as I’ve also previously stated, encouraging people to stand outside your business catching an augmented reality Electabuzz is not going to make them think “Gee, I sure could use a beer right now. I think I’ll go into this place that has a Pokémon Go sign!” All that the app did was facilitate the type of hangout behavior you would see at bars, fast food places, or pizzerias with another similar type of promotion.
If anything, market saturation when businesses jump on a good idea may be the best time to avoid it. We’ve gone down this road with Facebook, where Facebook ad fatigue has become an issue due to too many businesses geotargeting local areas. We’ve certainly been there with newsletter marketing, where Google now generally sends the glut of newsletters people receive in Gmail to a separate promotions tab. If everyone is trying to get in through a door, you’re going to have a hard time muscling past the crowds.
Shu basically found, again, that anecdotal evidence did not translate into hard data. Especially when a lot of the case studies were being presented by marketing companies who had a financial stake in encouraging more small merchants to sign up with them.
The big problem with all those Pokémon GO success stories, Shu says, is that they’re using anecdotes and data analysis that was “cherry-picked” to confirm a happy outcome. Shu points to Revel Systems’ infographic on the point-of-sales system’s customers that found the $2,000 per week sales boost and increased foot traffic at 82% of merchants with PokéStops. Revel’s analysis only looked at three cities—New York, Chicago and San Francisco—and studied a calendar period, July, that Shu argues is peak tourist time. In Revel’s data, merchants who were near a PokéStop saw a 63% boost in sales. But in Womply’s data, merchants who were not saw a similar 61% increase.
So to summarize, marketers and analysts vastly overestimated the impact of Pokémon Go because they wanted to believe it was a success and probably wanted local businesses to find a new outlet as a way to attract customers. There was a romantic notion to Pokémon Go when it first blew up about how it would bring communities together and encourage merchants to thrive. Hopeful dreams can only carry you so far. Let’s get back to doing what works for businesses, at least until the next big thing takes the world by storm for three months.