Friday, August 3, 2012

A bit more on persuadables

Following up on the last post, persuadables are one of those ideas with a high utility to difficulty ratio. They aren't hard to understand but they come in handy. So I thought I'd tack on a simple but fairly realistic example.

You own a casual dining place, think Red Lobster. You're sending out good coupons -- a popular twenty dollar entree for  fifteen  (still leaving you a profit of about a few dollars) -- to a random selection of people in the area. You've seen an uptick in business associated with the offer and you're seeing new faces (always a good thing), but the mailings cost you a quarter a piece and with a response rate of about two percent, the campaign is costing you a lot of money.

So you hire a statistician who builds a logistic regression model that lets you rank recipients and you only mail people who are highly likely to respond. Your response rate is now ten percent. A while later, though, you notice that your number of customers since you started using the model is back down to pre-coupon levels and your profits are way down. What happened?

The explanation lies in the three primary kinds of people who got your coupon in the mail. The first group is non-responders. They cost you a quarter a piece. Next are people who normally wouldn't have gone to your restaurant but decided to because of the coupon. These are the ones you like; they bring in money and may go on to become regulars. Finally, there are people who used the coupon but would have come by even without it. Giving them a coupon represents five and a quarter in lost revenue.

Remember, the model was built to rank likelihood to respond and as a general rule, the people most likely to come by after receiving a coupon are the people who would have come by anyway. By mailing only to the top deciles, you effectively selected the worst possible customers to market to.

This may seem like an obvious mistake, but it's not that unusual. Most people  who've worked with marketing analytics can come up with a few examples, some of which came with hefty price tags.


2 comments:

  1. Similarly with product recommendations-- you want to recommend products people like but that they wouldn't know about otherwise. Products they're sure to like but also sure to already know about aren't so helpful.

    From http://radar.oreilly.com/2012/03/drivetrain-approach-data-products.html:

    "Amazon’s recommendation engine is probably the best one out there, but it’s easy to get it to show its warts. Here is a screenshot of the “Customers Who Bought This Item Also Bought” feed on Amazon from a search for the latest book in Terry Pratchett’s Discworld series. All of the recommendations are for other books in the same series, but it’s a good assumption that a customer who searched for “Terry Pratchett” is already aware of these books"

    ReplyDelete
    Replies
    1. There's a related concept involved with reducing path length in a graph. If you're connected to a subgraph where everyone in the subgraph is connected to everyone else in the subgraph, additional connections won't do you much good. This one can have important implications for improving communication in organizations.

      Delete