The Disrespectfulness of reporting play counts

Misleading with Play Counts

One of the topics that often bubbles up in the rotating ‘issue of the day’ conveyor belt of online discussion is that of reviewers and play counts – specifically, the question of whether reviewers should include these as part of their review content so as to let people decide how seriously they should take their views. I refer you to this recent Reddit thread as an example of the discussions around the issue. We don’t include play counts on Meeple Like Us, and I’m going to use today’s editorial to talk about why.

First of all, let’s outline the case for play counts so that we can get a reasonable idea of why people might like to see them. For this I’ll directly quote rather than paraphrase to let the advocates for the position make their points without my potentially misinterpreting their opinions. Let’s go with the original poster of the thread linked above as a start:

I know there are some reviewers who do this, but I wish everyone would. I really think it provides important context. And on that subject, I do honestly believe you should play more than once to review a game, especially since so many game have variable player powers, starting postions, set ups, etc.

Reddit user /u/NKGra says:

There’s a world of difference between three identical steam reviews, one from someone with 1 hour played, one with 20 hours played, and one with 200 hours played. That’s important context. If 1 hour played guy is complaining about balance issues I’m likely to mostly disregard those balance complaints, and rightfully so.

/u/Pixelstitchedunicorn says:

So many people in this thread saying it’s a bad thing because people would complain about it (“you haven’t played enough” or “you played too much”), but people are already complaining anyways when a review goes against their opinions. I prefer more data, even if I don’t take all of it into account when reading a review.

/u/KamahlFox:

I actually do find this valuable for a variety of reasons. First is how deep did they go on the game – if it’s something with a ton of variety, it could help to know how truly varied that content is (both in terms of balance and gameplay mechanism). If it’s something without a lot, it’s just as important to see if what’s available will actually hold up to repeated play sessions and not just become stale.

/u/VidimusVici:

Someone who has played 5,000 games but only played them twice is going to have a completely different review than someone who has played 50 games but has played the current game in question twenty times. Its up to the customer to research both types of opinions, imo, and I don’t really see an issue with that.

These aren’t all the comments in favour in that thread, but they’re the ones that are a) about this specific topic and not tangents, and b) are reasonable and not overly drenched in Reddit snark. Together they form a perfectly fair proposal with a lot of merit. There absolutely is a difference between a first impression review and a review after several plays. Structural problems in a game with regards to replayability and balance are sometimes only possible to determine after multiple sessions. Adding a note with regards to play count wouldn’t take long to do and would add a genuinely quantifiable value that would leave readers with more data than the alternative. All of these are statements with which I can agree.

I just don’t think they argue for the conclusion that ‘therefore everyone should add their play count to their reviews’.

A few months ago I posted a defense of review scores – I believe they are important as a filter for readers, and as a way for reviewers to be accountable for their own opinions. No equivocating is possible when you have to nail your colours to the mast. Sure, lots of people stop engaging with a review once they’ve seen the score but they were never your readers in the first place. Your readers aren’t drive-by traffic – they want to know what you have to say.

Picture of a big metal horse for no reason

There are similar arguments to be made for including a play-count, and it seems on the surface like it’s the same issue and should resolve down to the same conclusion. It’s not though, and it’s because of where one finds the locus of its meaning. A review score is my summation – it reflects the final numerical scoring that I present to you as a formal part of a review. It represents a value that is the encapsulation of thought, consideration and occasionally a fair bit of adjustment over the course of writing. It represents me providing you with a judgement. I don’t just slap a number on a review and call it a day. I think about that number more than you might believe.

Play counts on the other hand represent me giving you what is essentially an audit. It requires no thought and no consideration. Many have argued that it provides context, and that’s true. However if you’re going to honestly provide a correct figure, it also provides no context about the context. Play means different things to different people and critical evaluation is a skill that gets better with use. A play count is a useless value for what people are seeking because it represents play sessions whereas what a review needs for credibility is invested analysis. I’d be more inclined to record ‘thinking time’ than ‘play counts’ if there was any way to accurately record it. Analysis means far more than the mechanical act of playing unless a review is focusing purely on surface level details. Some of that thinking time is likely to happen concurrently with playing time, but make no mistake – thinking time extracts value from playing time far more effectively than playing time compensates for a lack of thinking.

But even in that – some people just think better than others, y’know? Maybe I spend three hours thinking about a game and come up with conclusions to which someone else arrives after one. Balance issues are a particular example of that – there are people that see the exploitable strategies in front of them as soon as they learn the rules. Others might play for years before stumbling into them. I will identify many accessibility issues in a game the minute I set up a board. I can write a teardown on many games after the first time I play it. That’s just not going to be possible for many others. What does play count even mean in that context? It’s a number, but not a meaningful one.

The simple fact is that someone that doesn’t critically engage with a game won’t have a meaningfully different view after one hundred plays than they will have after one. Someone that plays once and then thinks hard about that game for six hours afterwards will have an awful lot more of value they can say than someone simply reporting on the play sessions.

That’ not to say that multiple plays of a game aren’t vital, because they are. Multiple plays are necessary to ensure that the raw material that goes into consideration of a game is drawn from suitably varied sample points. Repeated play helps ensure early freak occurrences and novelty don’t result critical over-steering. It helps guard against a victory or a loss unduly tainting appreciation. Some people simply enjoy games more when they’re winning, and it’s important to know if a game is still fun when you’re not. Reporting on a play count though doesn’t capture any of that. Despite carrying no meaning people will draw their own inevitably erroneous conclusions by virtue of its presence.

This is the crux of the issue from my perspective A review score conveys a meaning chosen by me. A play count conveys a meaning chosen by you, and you weren’t there when I played the game. Probably. Hi Pauline!

Others in the linked thread have pointed out it would be more valuable to see how often the game was played at different player counts, or for a reviewer to document their methodology for play. Again, these are undoubtedly quantifiable metrics but I think it also mistakes a provision of data for a provision of information. I don’t believe anyone is informed by any of these things. Instead, I think people are likely to be misinformed because fundamentally it all misses the truth of the matter – all genuinely critical reviews are inherently subjective, context-dependent, and driven by biases and personal preferences. You can’t turn a review into something more objective by filtering it through a flawed personal algorithm fuelled by unsophisticated data-points. It doesn’t matter how many data points you get, or how you weight them. Garbage in, and garbage out… and I think play count is, in this context, a garbage measure.

The sun shining through a tree

A better measure is play count multiplied by play variety multiplied by thinking time multiplied by thinking quality. Even in that the numbers you could assign to any part of that would be garbage measures and what would come out would be a garbage calculation that told you nothing. I’m willing to bet I tell a lot more from a single play of a game than someone that hasn’t been reviewing games on the internet for the past few years. Game literacy is like literacy in anything – you learn the patterns, tropes and clichés. Some people might be able to tell you about the rules. Some, through virtue of experience, can tell you why those rules exist. Others, and this is more important for a review, call tell you how those rules interact to create or facilitate a feeling or an experience.

So, how do you arrive at a level of reliance on a review if reporting on these metrics has no value? How do you know a reviewer has a played a game enough that you can trust their opinion? How do you know they have enough experience of play to give their review the weight you want?

You don’t, but at least you’re not left in the position of thinking you do.

The Internet has created a culture where quantification is considered a goal in and of itself, and an almost fetishistic belief that numbers are the same thing as meaning. We value what we can measure, but we can’t always measure that which should be valued. This leads to a shallowfication of knowledge. The cursory nature of online engagement means that these numbers are often taken in isolation, ripped out of their context, and lauded or loathed as a product in and of themselves. When they are brought together in a new context it’s often without respect for the original nuance.

That’s a big problem because nowadays people don’t build the same kind of personal relationship to independent critics as they once did. Larger sites and review aggregator outlets are the clearing houses of criticism and this means that things often feel disjointed and impersonal. Relatively few of the larger review houses have individual critics sufficiently differentiable or prolific that they can have a predictable voice. Their coverage can seem almost willfully contradictory as a result.

This disconnect between reviewer and reader is critical in understanding the call for this kind of data. People are asking for the important meaning that alienation has washed away. What they want though can only really emerge with engagement. Without the host environment of a larger body of work, secondary values such as trust or even respect are effectively amputated. The raw numbers – designed to live in a particular soil – are given life and importance of their own when they don’t warrant it. I may have disagreed with Roger Ebert on a large number of films, but I had a personal relationship with his work that ensured it retained meaning even when his opinion diverged from my own. That doesn’t exist so much now, and even the genuinely high profile reviewers are still rarely household names because their outlet is so much bigger than them. You don’t get Rodney Reviewer or Carla Critic’s view of a game. You get a number that IGN spat out that was then filtered through Metacritic into something so distant from its meaning that it may as well be randomly generated. In my defense of review scores I talked about that and noted the dangers of quantification and attempting to compare the incomparable. I live with that in review scores because at least it’s being done with a number to which the author assigned meaning. It’s preposterous to hope anything of worth can emerge from a number which derives from the soulless and perfunctory audit implied by play counts.

A secondary consequence of the reviewer-audience disconnect is that much of the culture on the Internet loathes criticism. They have lost, or abandoned, the sense of their own important role in proceedings as well as the connection with the individual voice of an individual critic. You often see people on Reddit or BGG pompously dismissing the work of reviewers by saying ‘Just read the rule book’. It’s definitely true that for some reviews that’s probably just as good in real terms. Bad reviews are bad reviews and they probably merit this response. However, there is rarely any sophistication in this pronouncement – rarely any target-finding. It’s often just a blanket dismissal of the entirety of all critical output. What you’ve found in those circumstances is someone that is admitting to their own failure or inability to engage with the criticism they have read or watched. They see no credibility in reviews because they have mistaken what the best reviews are supposed to achieve. More critically, they’ve also fallen foul of believing that critical work is for idle digestion.

A review should be more than something you passively consume. It should be something with which you critically engage. If you want to get the real meaning of a review, you need to engage in a creative act of interpretation of your own. If you pay attention to a review, at least the ones that aren’t terrible, you’ll find that there’s more going on that the word-count might imply. People ask for play counts for many reasons, but for some it’s because they’re not willing to actually evaluate the meaning of what a review is saying.

Some kind of flower, again for no reason

Reviews report on their play count all the time if you just pay attention. Paying attention means giving a review more than a cursory once over, and also considering how it fits in with the larger body of a reviewer’s work. I know that sounds a lot more intensive that me telling you how many times I’ve played a game, but it’s the only way to get the actual information that is being sought. Believe it or not, people do trust me to guide them to the best game purchases for them – and that’s because we have a connection. It may be parasocial and largely one-way (save for comments and our Patreon Discord), but make no mistake – that connection is vitally important for a reviewer and few that take the job of criticism seriously would throw it away.

One the remarks I made in the linked Reddit thread is that if the text of a review doesn’t convince someone the reviewer played it enough, a play count won’t add anything new. It supplies no new critical content, and importantly – as discussed above – adds no actual context. It might create in someone a sense of confidence but that’s entirely an illusion constructed in the reader’s head.

It’s going to be obvious in a review that is actually worth that description .whether someone is making an off-the-cuff judgement or a critique that is informed by deeper experience. If a reviewer discusses replayability in something other than the most surface way, there’s your evidence of multiple play-counts. If they discuss how the game works at different play counts, there’s your evidence of the same. If that isn’t sufficiently meaningful or informative, a play-count doesn’t alchemically transmute it into something better. If a single number in a review is expected to do critical heavy lifting, it better be the result of some serious thought and consideration of its own. A play count just isn’t up to that task.

So, how many times should a reviewer play a game before they review it? It turns out the answer was there all along – they should play it enough times. For some reviewers and some games, that will be once. For others it might be dozens of times. Reviewers should play games until they feel their experience has equipped them with the perspective to do the game justice in an analysis. The only person placed to make that decision is the reviewer, and if that reviewer is any good they’ll take the job seriously enough to reach that point of enough.

In the end, reviews have to be a collaboration of trust. I trust that you’ll read what I have written and engage with it. You trust that I will write something worth your time and make the effort to ensure I’m able to do that. Know my methodology and my diligence through my writing, because you can’t get an appreciable sense of it any other way. Don’t ask me to insult you with something as meaningless as my play counts. You’re too important to me to try to mislead you in that way.

  3 comments for “Misleading with Play Counts

  1. Darryl
    18/09/2019 at 5:32 am

    I agree with all of this, and have an additional reason that I don’t get that worried about play counts: Why should I expect a reviewer to play a game more than I will?

    Look, if the “enjoyability” of a game only surfaces after a few plays, it’s unlikely to ever get those plays. If I don’t enjoy my first play of a game, and I don’t immediately see why I might enjoy it more next time, that’s a death knell. There are lots of great games in the world. Why should I spend my time on a game that I might like if I put enough time into it, when there are lots of games that are great starting on the first play?

    The converse — games that are fun at first but don’t hold up to repeat plays — also seem pretty easy to pick out after an initial play. “That was fun, but I don’t need to play it again any time soon” is definitely something I’ve said/thought after playing some shallow games. I’ve also thought “Oh, this seems like it would be better/worse with more/fewer players.” And I’m not even a games reviewer; I’m not thinking about anything deeper than “Do I want to play this again?”. So if I trust a games reviewer to give me a meaningful opinion at all, I trust them to know how much they have to play a game to form that opinion.

  2. Jessica Eccles
    18/09/2019 at 5:32 am

    I’ve had similar thoughts, but you’ve managed to articulate them so well.

    “We value what we can measure, but we can’t always measure that which should be valued.”
    If you want a tldr this really gets to the heart of the matter.

  3. mangozine
    18/09/2019 at 5:32 am

    Great article, as usual, Michael. Thank you.

    I have different ways of approaching reviews depending on the target audience/market, but generally prefer not to include this sort of information (unless I’ve genuinely only played a game once — which is a rare happenstance) because it should hopefully be readily apparent that (a) I have usually played the game several times before I try and review it, and (b) I have given due consideration to asymmetric set-ups, balance issues, etc. as I perceive them…

    My reviews (esp. for The SPIRIT magazine) often break down a lot of the rules and mechanics of games, because this can often be a deciding factor in whether a game is likely to be suitable for a prospective player or purchaser (as well as how long it takes to play) – I will always add ‘fluffy stuff’ about whether I enjoyed playing it, how it felt at some points, what incident(s) may have occurred, etc. as well as offering an opinion as to whether or not this particular game is likely to suit a particular type of gamer.

    Like you say, bottom-line is that a review is one person’s opinion of a game based on their experience with it, and I would freely admit if I had an awful experience with a game, but also say something if other players warmed to some aspect of the game that I didn’t, which is fine imho: horse for courses, etc.

    I would probably take issue with the idea that reviewing a game after only a few plays somehow makes your opinion less important than someone who has played the game 50+ times — I haven’t yet met anyone in the gaming universe that has played a game they don’t like 50+ times! I’d go so far as to suggest that with the increasing transience of gamer attention-spans nowadays, it’s rare for any reviewer to get the same game to the table more than 10+ times (probably less than that), and if you don’t like it after 2 or 3 games, your opinion could be very important to someone who is about to drop £50+ on a new game that they might not actually enjoy.

    Clearly, if you have played Root a gazillion times with all different set-ups and player counts, you know the game very well, but just as easily if you have played it only once and it really wasn’t to your taste, I’d suggest that you perhaps also consider (a) not reviewing it at all, or (b) trying it again at least a few times.

    Just an opinion, but hopefully clarifies where this particular reviewer (me) is coming from when I review a game, whether on my blog or for a magazine.

    Alex Bardy

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.