Being Mindful About Feedback

The final round of pools just ended; you’re chatting with your friends while trying to work out a place that will seat a party of 25 at 10:00 pm. “I went 3–2 this time” says one of your friends. Another joins in, “I went 4–2”. Finally, a third puffs out their chest: “I went 0–2 but both people I lost to made it out of pools”. If this resonates, give me another few minutes of your time. I want to talk about feedback and improvement.

First, some terminology; reliability and validity. For a measure to be reliable, it has to be consistent. If you step on a scale and see a different number every time, then you probably don’t have a very good scale. Validity means that whatever you’re using is actually measuring the thing you’re trying to measure. If you want to measure temperature, get a thermometer. Ideally you want a measure to be both reliable (gives you the same answer) and valid (is actually measuring what you intend to measure), but ideal isn’t easy to achieve.

An example I’d use when teaching was predicting performance in class as a function of shoe size. It’s a ‘reliable’ measure because people’s feet don’t tend to grow or shrink over the course of a semester. It even seems ‘valid’ because someone wearing a size 12 shoe is probably going to do better than someone wearing a size 3. The problem lies in the confounding factor of age. A confounding factor is something that can also explain the connection you‘re observing. It can even make it seem like a connection exists when it does not. Of course someone with bigger feet, usually known as an adult, would do better in a college class than a literal child (…usually).

Let’s apply these concepts to the different kinds of feedback players get on the regular. Because I favor Arc System Works games, I’ll talk about squares. Most “anime” fighting games follow this system. After a number of matches, you are given a colored square that falls somewhere along a spectrum. The color of the square will get warmer or cooler depending on whether you are winning or losing (some fine details omitted). So, is this a good measure of skill?

First, the square can change dramatically. If someone at the highest end of the spectrum loses even a single match to someone toward the middle, they will almost certainly lose a rank. If two players at the highest rank play a one-sided set of matches, the loser might find themselves ranked down to the middle. An hour or two ago they were at the height of the scale yet now share rank with an intermediate player — despite being the same (albeit potentially saltier) person as before. So I ask again, is this a good measure of skill? H*ck no!

Now, I know what you’re thinking. “Ricki, you’re a smart and beautiful vtuber, but your brain is poisoned by horse girls and anime. Other games use a league system. It’s much better.”

Yes, some games use different systems where it might take longer to shift in rank. However, the fundamental problems are still there, only now drawn out over time. These systems are less measures of your progress as a player so much as they are measures of how long you’ve spent climbing the ranked ladder. In this case, whether or not you’ve improved as a player is confounded heavily with time in a way that makes it impossible to say this is a good measure of progress.

I started with these examples because playing online is common experience;I went through a phase where I thought reaching a high enough rank would mean I was finally a good player. However, the spectre of bad feedback still exists in the real world. Let’s recall that opening anecdote about placings at a tournament. Maybe you won three matches in the last tournament and four this time. That might seem impressive, but doesn’t speak at all to the strength of the players you beat. Conversely, one could go 0–2 and lose immediately to strong players. In either case, a person’s placing doesn’t speak to how well they played but more so to who they played against.

I’ve now talked about three different forms of common feedback around fighting games and why they’re ultimately not good measures of personal progress. For those who figured this out before reading, congratulations: you’re well on your way to becoming a doctor (I’m sorry. I’m so sorry). However, even knowing this may not stop you from caring about rank. Our brains tend to gravitate toward feedback; especially in situations where our performance or our roles are ambiguous.

Knowing all this, it still releases good brain chemicals to see the numbers go up. Even worse, there is a whole different dynamic going on in there called the self-serving bias. Most people tend to internalize good outcomes (winning) and explain away bad outcomes (losses). Our brains want to latch onto feedback even when we know it isn’t useful, even more so when that feedback is good.

So what is a brain-haver to do? Mercifully, we have the ability to be aware of our own thoughts; meta-cognition. It’s not bad in itself to feel good about seeing your rank rise. However, it’s important to be critical about the feedback you’re getting. Ultimately, rank isn’t a good measure of your progress as a player. Staying aware of that and being realistic with yourself is an essential part of making progress.

I want to close this one with an anecdote. Like I said before, I mostly play Guilty Gear. Guilty Gear, like most anime games, uses the colored square system — I spent years striving for the highest level. I had convinced myself that once I had the pink square, I would be able to see myself as a strong player. This was rooted in misinterpreting what that square represented. Playing the game became less about learning and more about racking up wins to see if the color of my square would change. I wasn’t playing to win, I was playing not to lose. It wasn’t until I started being more realistic about the game’s feedback that I was able to actually achieve my goal.

Next week: formulating good questions and why it can be so dang hard!