TripAdvisor’s FlyScore: Not yet harnessing the power of crowds

By improving their FlyScore tool, TripAdvisor could harness the power of the crowd to help buyers make better decisions about which airlines to fly with.

Have you ever had a really bad experience with an airline?An overbooked flight. Hours waiting on the runway. A last minute cancellation for entirely predictable weather. Wifi promised, but hardly delivered. Some airlines have a special talent for ripping us off.

As you walk away from your airline disaster, you often ask yourself: what now? Where do you channel your rage? How do you help fellow travelers from falling victim to the same airline shenanigans? How can you get a little vengeance? And last, how do you avoid these pitfalls in the future?

You can complain to customer service, but that won’t do much good. Chances are, you and everyone else will just keep flying the same airlines anyway. This is just the cost of traveling, right?

I wager it doesn’t have to be this way. If everything from Amazon products to restaurants to Uber drivers can have customer reviews, why not airlines? In fact, recently TripAdvisor tested this idea. Nine months ago they introduced FlyScore, an airline scoring system based on customer reviews meant to “demystify the flight shopping experience” [TripAdvisor].

FlyScore is an innovation unique to TripAdvisor. None of the major flight purchasing sites offer airline reviews (as best as I’ve seen), or an airline ranking. If fact, flyers who want to know how airlines stack up have pretty limited options. The US Department of Transportation produces reports each month thick with numbers, but light on insights. Flyers can go to forums like to complain to fellow passengers, but its unclear how many folks are actually reading.

Unlike any of these other options, FlyScore embeds passengers reviews directly in each flight a buyer considers purchasing. See the example below of a return trip from Boston to Bogota.












Not convinced that travelers are going to care about this new tool? Unfortunately, neither am I. FlyScore demonstrates that crowdsourcing information alone is insufficient. What matters is whether or not you are collecting useful information.

The first problem is that FlyScore itself is pretty opaque. What is FlyScore ranking, exactly? The reader can’t double-click to find out. If you happen to be interested, TripAdvisor says the score is calculated based on “qualitative traveler reviews, the quality of the aircraft, in-flight amenities, and duration of the itinerary”. This neglects things like the likelihood of delays, costs of changing your flight, and how airlines deal with bad weather.

Next, what the accompanying reviews tell us is largely what we already know. In sum, it looks like American Airlines is pretty mediocre. Big surprise. Slightly superior ratings on things like legroom and seat comfort might induce a buyer to choose American over Spirit (right), but only for reasons most already understand.

Unfortunately, if buyers don’t find the FlyScore system useful, posters aren’t likely to provide insightful reviews either. TripAdvisor’s fifteen thousand reviews may seem like a lot, but its hardly a drop in the bucket considering American flys 51.1 million passengers a year [Wiki]. By comparison, Amazon’s Fire Stick has over fifty thousand reviews [Amazon].

What FlyScore needs to do is tap into the anger of the crowd to highlight the ways in which airlines can be different from each other in ways that buyers might not have anticipated. Rather than asking reviewers simply to rank their experience on a few metrics, TripAdvisor could focus on generating specific incident reports for things like long runway relays, late weather cancellations, overbookings, and indiscriminate enforcement of baggage dimension requirements. It is easier to engage customers on specific, negative incidents because you offer them a conduit for their anger.

Next, TripAdvisor should channel this anger into data that is useful for buyers. This means metrics on which airlines are most likely to leave you stranded, stuck, running late or paying gross fees.  In less polite terms: the likelihood of getting screwed. Stronger engagement would allow TripAdvisor to collect information on specific routes, which have as much to do with the variation in service quality as the airline itself.

Last, the platform could eventually help travelers connect with each other in real-time when disaster strikes. Ever been in an airport during a major storm? Everyone is trying to get information to make important decisions on the fly, but only a few travelers are able to get real-time information from customer service reps. FlyScore could help fellow travelers share information and collaborate, putting their power back in their hands.

One way or another, in the future our frustration with airlines will not go unrecorded. Companies like TripAdvisor only have to unlock the power of the crowd to help flyers better avoid airlines that are likely to screw them.









DeNA Case: Failure of HBS Alumna


ArcBazar: Crowdsourcing Design

Student comments on TripAdvisor’s FlyScore: Not yet harnessing the power of crowds

  1. Nice post James! I haven’t noticed this new feature on TripAdvisor so it’s interesting to read your summary + feedback on it. The problem with providing an “overall” score is each person weighs different factors contributing to “the one” score differently. For me, value and cleanliness is are most important whereas legroom is not too important (I’m 5 6 so it’s never an issue). For a tall person, it’d be a different story. Hence, I find the breakdown under “Rating Summary” useful but overall score, not so much.

    I wonder whether TripAdvisor could let users reveal preferences by letting them stack rank or weight rank the different factors. TripAdvisor could then calculate “Personalized Overall Score,” enabling a user to quickly compare across airlines in a relevant manner. That would be amazing!

  2. I love the idea of helping travelers connect with each other in real-time when disaster strikes. There are so many times when a flight has been canceled and everyone lines up to ask the service reps the exact same question, except the line is 20 people long and so inefficient. There must be a better way to disseminate this information and share knowledge about the flight. You can also include features that can help people swap seats or get access to something while in air (e.g., Tylenol). Someone should build this!

  3. I think this is a novel idea, but I think the decision tree for customers on flights is largely still centered around the cost of the flight. I think the natural process is for customers to consider general buckets of airlines, discount and mainstream, and then compare price across each of those categories. I think most of the frustration around flights happens around delays, which is partially out of the control of the airline; if you happen to have a flight while Boston is hit by a snowstorm, you’re probably out of luck. I think for the rest of the things like legroom, service on the plane, comfort on the seats, I largely associate these with the airline itself, but at the end of the day, is the variation between American legroom and United legroom that big of a deal to make you switch to a more expensive ticket? I think this is a nice function, but unlikely to drive significant switching from platforms like Expedia or Kayak to TripAdvisor for flights.

  4. James, great post. What are your thoughts on whether there is a conflict of interest between the airlines (who ultimately pay booking commissions) and TripAdvisor (if they are explicitly asking customers to share negative reviews, as you propose)?

  5. Nice post, James. I too never noticed flyscore on Tripadvisor. As I read your post, one question comes to my mind: How can Tripadvisor better leverage this data in routine flight booking? Can it have the user pre-submit the preferences based on what matters most to her – is it legroom or is it timely departure and arrival, or response in emergencies/ cancellations? Then Tripadvisor can rank the flight options based on the score of the options on the features that matter the most to the user. What say?

    Further, to the point of collecting the data, I am not sure whose customer I am if I book flight through Tripadvisor. Tripadvisor acts as an affiliate and sends me to the airline website where I complete the transaction (post which Tripadvisor hopefully gets the commission). But I have already left TripAdvisor and am solely a customer of the airline now. How is Tripadvisor tracking which flight I booked using which email id (I can easily use a different id while booking the flight than the one I use on my Tripadvisor account) so it can send me email later to give feedback? Next, airlines already ask to submit detailed feedback (which is often ignored by people). Why would a user submit another set of feedback on Tripadvisor?

  6. James, thanks for your post! I didn’t know this feature existed, and like others I am wondering how (and if) it should be better utilized. More transparency into the ranking is a good start – but it seems like TripAdvisor needs to figure out exactly why this is a useful tool. For hotels, it’s obvious; factors like location, price, amenities, etc. are illuminated in reviews and help users to make a purchase decision. But for flights, I agree with the comment above that for most, the price is the deciding factor. I wonder if it would be better to integrate user reviews into SeatGurus (also owned by TripAdvisor) to build out a more comprehensive, all-in-one airline review platform that leverages TripAdvisor’s enormous power of the crowd. (p.s. I’m going to work at TripAdvisor next year, so I’ll bring this one up 🙂

Leave a comment