Re: Half and Half - Oct 12, 2022
Posted: Tue Oct 18, 2022 2:23 am
@HoldThatThought:
I've honestly tried to stop relying on star ratings for any commodity -- movies, restaurants, books, you name it. I'd rather not let my judgment be influenced by ratings before I experience the product myself. I'm not always perfect about that, especially with more expensive purchases like a hotel room, but even then I take a closer look at actual customer comments rather than star ratings in case there are major red flags to watch out for.
And at least with those other sites like Yelp or IMDB or Amazon or Google, you will usually have thousands of reviews at your disposal that can give you information on the product you're getting. That's not the case with crosswords. There aren't many sites that review non-NYT puzzles, and with Fiend, a handful of cranks who drop 1-star ratings for no actual reason have an outsized effect on other solvers, the ones who use them to determine which puzzles they solve.
In some cases I'm also opposed to star ratings on principle since some businesses like Uber actively use them to evaluate their employees. The thought of a company punishing its workers because some customers didn't give them 5-star ratings is something I find really disturbing. The stakes with the star ratings at Crossword Fiend aren't as high as that, obviously, but it would still be nice if something as complex as a crossword -- its theme, its answers, its clues, how it all ties together, and ultimately something that a constructor pours a huge amount of time into creating for others' enjoyment and often not much money -- weren't cheapened with the same kind of algorithm that we use to evaluate every other product in our lives. It's an artform, not something that should be reduced to a star rating.
You said earlier that no experienced constructor gets heartbroken over a handful of 1-star ratings. Maybe not, but I know several veteran constructors (Matt Gaffney, Matt Jones, Pete Muller, among them) who have publicly expressed their own frustration with people dropping 1-star ratings without bothering to provide any feedback. For my part, I don't even look at the star ratings. Literally. I scroll past them immediately on opening up the page. Better to ignore them if they're not going away.
And yes, I agree that it'd be nice if everyone who visits Fiend could take the ratings with a grain of salt, but unfortunately that's not what happens in practice. Again, I've already heard from many commenters who have said they use them to decide which puzzles they solve. You can say "well it's those people's fault for relying on an algorithm," but I blame the system for encouraging that thinking rather than individual solvers. These are most likely people who aren't super-fast solvers, don't have as much time in the day to solve every puzzle out there, probably don't spend any time thinking about how the star ratings are flawed and unreliable. They're still potential folks who would become regular solvers of any puzzle reviewed on Fiend, but aren't because the star ratings have convinced them otherwise. The star rating system is the cause of that.
I guess my question is this: What meaningful benefit is there to the star ratings, and what would be the downside to abandoning the star rating system altogether? It a) isn't a reliable measure of a puzzle's quality, b) causes unnecessary stress for at least some constructors, and most importantly, c) prevents anyone who relies on it from becoming regular solvers of puzzles that don't meet some arbitrary numerical threshold. All of these are problems that could be solved if the ratings just weren't there. I have a very hard time imagining how there's some actual benefit to them that outweighs those issues.
I've honestly tried to stop relying on star ratings for any commodity -- movies, restaurants, books, you name it. I'd rather not let my judgment be influenced by ratings before I experience the product myself. I'm not always perfect about that, especially with more expensive purchases like a hotel room, but even then I take a closer look at actual customer comments rather than star ratings in case there are major red flags to watch out for.
And at least with those other sites like Yelp or IMDB or Amazon or Google, you will usually have thousands of reviews at your disposal that can give you information on the product you're getting. That's not the case with crosswords. There aren't many sites that review non-NYT puzzles, and with Fiend, a handful of cranks who drop 1-star ratings for no actual reason have an outsized effect on other solvers, the ones who use them to determine which puzzles they solve.
In some cases I'm also opposed to star ratings on principle since some businesses like Uber actively use them to evaluate their employees. The thought of a company punishing its workers because some customers didn't give them 5-star ratings is something I find really disturbing. The stakes with the star ratings at Crossword Fiend aren't as high as that, obviously, but it would still be nice if something as complex as a crossword -- its theme, its answers, its clues, how it all ties together, and ultimately something that a constructor pours a huge amount of time into creating for others' enjoyment and often not much money -- weren't cheapened with the same kind of algorithm that we use to evaluate every other product in our lives. It's an artform, not something that should be reduced to a star rating.
You said earlier that no experienced constructor gets heartbroken over a handful of 1-star ratings. Maybe not, but I know several veteran constructors (Matt Gaffney, Matt Jones, Pete Muller, among them) who have publicly expressed their own frustration with people dropping 1-star ratings without bothering to provide any feedback. For my part, I don't even look at the star ratings. Literally. I scroll past them immediately on opening up the page. Better to ignore them if they're not going away.
And yes, I agree that it'd be nice if everyone who visits Fiend could take the ratings with a grain of salt, but unfortunately that's not what happens in practice. Again, I've already heard from many commenters who have said they use them to decide which puzzles they solve. You can say "well it's those people's fault for relying on an algorithm," but I blame the system for encouraging that thinking rather than individual solvers. These are most likely people who aren't super-fast solvers, don't have as much time in the day to solve every puzzle out there, probably don't spend any time thinking about how the star ratings are flawed and unreliable. They're still potential folks who would become regular solvers of any puzzle reviewed on Fiend, but aren't because the star ratings have convinced them otherwise. The star rating system is the cause of that.
I guess my question is this: What meaningful benefit is there to the star ratings, and what would be the downside to abandoning the star rating system altogether? It a) isn't a reliable measure of a puzzle's quality, b) causes unnecessary stress for at least some constructors, and most importantly, c) prevents anyone who relies on it from becoming regular solvers of puzzles that don't meet some arbitrary numerical threshold. All of these are problems that could be solved if the ratings just weren't there. I have a very hard time imagining how there's some actual benefit to them that outweighs those issues.