Does randomness in games limit skill?

random-games-skill

Today I’m following-up on a twitter discussion I recently had with Kory Heath, inventor of the miraculous game-cum-teaching-tool Zendo, among others.

The question under debate: does randomness in games limit skill?

Kory, who voted nay, cited a talk by Magic: The Gathering designer Richard Garfield in which he uses a toy game called Rando-Chess to argue randomness doesn’t limit skill (relevant part starts at 2:39):

Rando-Chess is the same as normal Chess, except after the game is over, the players roll a die and if it comes up “1”, the winner becomes the loser and vice-versa. Though Rando-Chess has more randomness than Regular Chess, it has equal skill, since you can apply all your Chess knowledge to improve your chances of winning Rando-Chess. This seems to demonstrate randomness doesn’t limit skill.

I told Kory wasn’t so sure. He asked where my doubts lay but I couldn’t explain them without writing scores of tweets, so I told him I’d reply via blog. And Lo! I’m actually following through.

Before I get to my argument, two caveats:

1. I’m not sure what camp I’m in and my argument below could be bunk. I just want to explore the possibility that Garfield’s argument papers over some complexity.

2. For soon-to-be apparent reasons, my argument only applies to strategy games, by which I mean games whose first aim is to get players grappling with interesting strategy/tactics stuff in an attempt to win. It’s not a universal argument about the role of randomness in games, or even in orthogames (the games about which Garfield made his claims).

Randomness Limits Skill…Sometimes

Let’s start with the following claim: Rando-Chess is bad. Including randomness in the way Rando-Chess does makes the game worse than normal Chess (a claim Garfield himself makes in his talk).

If I played Rando-Chess, and won the Chess game but lost on the die-roll, I’d be frustrated I outplayed my opponent but lost anyway on a single random event which negated in one instant all my prior decisions.

If that’s not frustrating enough for you, let’s replace Rando-Chess with Super-Rando-Chess. In Super-Rando-Chess, we use a 100-sided die. If the die lands on any number from 1 to 49, the winner becomes the loser and vice-versa. All arguments that apply to Rando-Chess also apply to Super-Rando-Chess, and the two games are bad in the same way. One is just more bad in that way than the other.

But why should it matter if these games are bad? The Rando-Chess’ quality should have no bearing on the logic of Garfield’s argument regarding skill, right? Alas, I think it might!

What if, to make a good strategy game that includes randomness, it must be incorporated in such a way that it does limit skill? Below I explain why I think it could be true. My case consists of two contentions:

Contention #1: for randomness to make a strategy game better rather than worse, it must be difficult for a player to determine the respective extents to which random events and her own choices determine the outcome of each game.

Why? Imagine losing a game which conforms to this requirement. When you lose, since you’re unsure what led your loss, instead of getting frustrated, you think about what you could have done differently, and to tease apart the factors involved. The game gets you thinking about strategy, which is the the point of a strategy game.

On the other hand, if you know why you lost, you have the same problem as in Rando-Chess: when you lose due to some random event(s), you’ll know it, and it’ll be frustrating for the same reason Rando-Chess is frustrating: it negated your choices and you know it.

This is why my argument only applies to strategy games: if strategy isn’t the main focus of a game, the quality isn’t necessarily hurt if randomness diminishes the importance of strategy. Many games focus more on the “thrill of finding out what happens”, to create a vivid story or a gambling atmosphere, etc, where that’s the case. Contention #1 therefore doesn’t apply there. 

Contention #2: the harder it is to distinguish the effects of your own choices from those of random events on a game’s outcome, the harder it is to accrue skill. 

The harder it is to know how your choices affected a game’s outcome, the harder it is to know how to change them in the future, and thus the harder it is to improve. If I’m examining the choices I made in a game I lost, how do I know if I made bad choices or if I got unlucky? If I can’t distinguish the effects of randomness from my own choices on the outcome, I’m stuck.

As a result, the rate at which I can accrue skill is limited and that limits the ceiling on the skill I can reach for a game with the time I have to play and study it. The difference in skill between the best players and average players will not be as large as it is for games where it’s easier to distinguish the effects of random events and choice, or for games without randomness.

Put contentions #1 and #2 together, and you get this:

Good strategy games incorporate randomness such that it’s hard to distinguish the effects of player choice vs. randomness, which limits the rate at which players can accrue skill, which limits skill. Ergo, randomness limits skill in good strategy games.

I wonder if Garfield would agree. There’s another essay on the same topic where he says “The reward for skill depends on how much luck there is in a game…“, which suggests he might be open to my feedback-centric argument.

Anyway I sense I haven’t fully thought this through and may be problematic assumptions lurking. For example:

Maybe knowing you got screwed by randomness isn’t the problematic as I’ve claimed. Maybe I’m overgeneralizing from my own preferences. In fact, in Garfield says randomness should make you feel that, when you lose, you’re unlucky, but when you win, it’s due to skill. On the other hand, while that may be true for many sorts of players, I doubt it’s true for strategy lovers. After all, it’s this “knowing you got screwed” quality that makes Rando-Chess a bad game.

Another assumption is, when I say skill, I’m referring to skill real humans can acquire in practice, not skill “in the game” but out of human reach. For example the best Chess engines now play more than 500 ELO points better than the best humans, and some of that extra skill may be impossible for humans to acquire. My contention #2 above doesn’t apply to this sort of theoretical skill. Which kind of skill should we be talking about?

If you can bring other such assumptions to light in the comments I’d be grateful.

Side note: Everyone knows Reiner Knizia is one of the great game designers. What makes his games so good? One feature of many is they make it very hard to know where skill ends and luck begins. In fact Knizia’s games inspired my idea that games should exhibit this quality. Tip o’ the old hat to the master.

Nick Bentley

Advertisements

19 thoughts on “Does randomness in games limit skill?

  1. Thanks for the thought-provoking article. Understanding the effects of randomness in strategy games is a difficult trudge uphill at night. Sometimes I think insights people express are more due to randomness than their skill at thinking and expressing thoughts.

    I think a serious flaw in your argumentation is that you do not sufficiently differentiate skill expression allowed by a game and how well a game measures skill. For instance, Rando-chess has the same depth of skill expression as chess, but is worse as measuring skill because the added die roll fuzzes the win rates of players.

    A strategy gamer would at least conceptually favor a game that does a better job of measuring skill, but I think much of the pleasure in playing strategy games is that of expressing skill–doing the mental gymnastics required to optimize your move given the rules of the game and the game’s state and then manifesting it in the system and watching it play out. Tic-tac-toe fails to engage adult strategy gamers because the range of skill expression allowed is tiny–not because it’s necessarily bad at measuring the relative skill of players.

    Furthermore, I contend that randomness does not have any necessary impact on skill expression that isn’t greatly overshadowed by the effects of the rest of the game rules in concert. If the rest of a game’s rules work to create difficult optimization problems that the player enjoys solving, some randomness in the result does not diminish the joy of trying to solve those problems–some randomness may deepen that joy because randomness may shift the planning burden in some degree from long-and-deep to short-and-broad, which many players, including myself, like a bit more.

    1. Great comments! I agree I should’ve articulated the expression/measurement distinction more clearly in the writing (it was very clear in my head while writing).

      It seems that your counter argument is a refutation of my contention #1. You may be right. As I say, I’m undecided about the virtues of my own argument.

  2. I strongly disagree with contention #1. Your only justification for it is that it’s frustrating to know you lose due to luck. I disagree with that: I think it’s frustrating to lose in general, but personally I find it less frustrating if I know it was due to luck, than if I don’t know why I lost. I am satisfied with my choices because I know what they did for me and I know they gave me the best odds of winning. Most people I game with would agree with this attitude. From this I think your universal statement that one way is inherently more frustrating is false.

    I also strongly disagree with your conclusion – even if I were to accept contention #1, which I don’t, making it harder to acquire skill does not limit skill. It’s easy to acquire skill in tic-tac-toe, and harder to acquire it in chess. If it’s harder to acquire skill, then acquiring skill… takes more skill! In other words, in addition to the skills that are already part of the game (being able to choose the right moves, something you have not shown to be diminished), there will also be increased skill required to analyze might-have-beens, and determine what the EV of your choices was.

    If I can speculate, I think that your mind is emotionally attached to the idea that a “skillful” game has high correlation between outcome (win or loss) and skill. Your contention #1 takes it as a given that visible disconnects between skill and outcome are frustrating. Your contention #2 notes that players in a game with mixed-together skill and chance may increase their skill more slowly, leading to likely lower disparities in player skill levels. This sounds to me as though you are thinking again about the strength of the correlation between winning and skill, and how for a given amount of marginal experience, a player will gain less skill and therefore show lower gains in win rate. In your conclusion you take this as a sign that the game has less skill available overall.

    This contradicts the premise of the rando-chess hypothetical example. The idea there is that you only need the slightest correlation between choices and outcome for the choices to matter in the game. And as long as the choices matter in the game at all, the skill it takes to make those choices correctly is a real, meaningful thing. By this definition, the one Richard Garfield is talking about, a “skillful” game is one where there is a lot of stuff to think about and get right which matters. By comparison, I believe that you are still thinking of a skillful game as one where the outcome can be most easily predicted by skill. In that context, your argument makes sense, and we are just debating what we mean by how much skill is in the game, which is a bit silly.

    1. Wow. Thanks for the long and thoughtful reply. It’s one of the few criticisms I’ve seen responding to what I’ve really said, instead of what the respondent imagined I said. Congrats for understanding my flailing attempt to communicate.

      I’ll respond to your two points separately.

      First, regarding contention #1, I also agree I may be overgeneralizing, and point out as much near the bottom of my essay. This argument may only apply to apply to a subset of strategy lovers, and I don’t know how big that subset is.

      One thing you say in this regard struck me. You say:

      “but personally I find it less frustrating if I know it was due to luck, than if I don’t know why I lost.”

      For me “not knowing why I lost” is an opportunity to learn something new about the game and its strategy. So it might be said that my argument applies to the extent one cares about learning, as opposed to say, just experiencing the game as a standalone thing. Maybe. I don’t know. I hope it was clear from my essay that this topic confuses me.

      Regarding your second argument, I’ll push back a little. Regarding contention #2, you say:

      “This sounds to me as though you are thinking again about the strength of the correlation between winning and skill, and how for a given amount of marginal experience, a player will gain less skill and therefore show lower gains in win rate. In your conclusion you take this as a sign that the game has less skill available overall.

      This contradicts the premise of the rando-chess hypothetical example.”

      If I understand what you’re saying, you may have misunderstood me: my example *does* contradict rando-chess, because rando-chess doesn’t incorporate luck according to contention #1, and is therefore a bad game. Contention #2 doesn’t apply to games that incorporate randomness like Rando-Chess does, so there’s no conflict.

      With Contention #2, I am indeed “thinking again about the strength of the correlation between winning and skill”, because the stronger that correlation is, the more information the player has about how to improve. So the stronger the correlation, the faster one can learn. If you can disentangle the effects of luck from your own choices, then the correlation doesn’t matter because you can figure out whether you made the right moves just as well whether you won or lost. But I’m specifically talking about games for which you can’t do that, as per contention #1.

      1. I’m glad you found my comment interesting and respectful!

        Here’s what I meant by “This contradicts the premise of the rando-chess hypothetical example.” My understanding is that the point that rando-chess is trying to demonstrate is that you can lessen the correlation between winning and skill, while leaving it essentially the same game with the exact same amount of “skill”. I don’t see this as dependent on specifics like whether it follows your rules of contention 1. Rather, I see it as illuminating that there is a real quality games can have, which is independent of correlation between player skill and game outcome, and which is representative of how much opportunity there is in the game to make better choices.

        I’m taking your point #2 to be this (and you can correct me if I’m wrong): suppose a game has harder-to-see connections between your choices and the outcome of the game. Then it becomes harder to acquire skill, and therefore someone who has played more than someone else will have acquired less marginal skill than if they had been playing, say, chess. And because they have acquired less marginal skill in the same amount of play, their marginal experience level will have less of an impact on the outcome of the game than, say, chess. Therefore, because we are measuring the skill in a game by how much your experience level and talent affect the outcome, this game has less skill than, say, chess.

        This chain of reasoning contains the premise that skill in a game can be measured by the correlation between player skill/talent/experience and game outcome. And what I meant was that the rando-chess example says that skill in a game is something different.

        I’m finding this interesting to think about – specifically about what exactly do I think is a good measure of the amount of skill in a game. To some extent I want it to be positively correlated with how long it takes you to learn all there is to learn about it. (I find that kind of amusing given that you’re taking speed of learning as skill indicator in the exact opposite way!) Here’s another toy game: You and I take turns guessing the next digit of pi. After our guess the correct digit is revealed and if you guessed wrong, you lose. So, properties of this game:

        1) Experience is a HUGE advantage. If neither of us know anything about pi beforehand and have good memory, then roughly speaking, whoever has played more previous games will have seen more digits of pi and therefore will win. So it’s almost the case that having just two more games of experience puts you from losing almost 100% of the time vs a player to winning 100% of the time against them. That kind of craps on the idea that measuring the correlation between experience and win rate (and the steepness of the learning curve) is meaningful for skill, if you agree that this game is not particularly skillful.

        2) The learning curve goes on forever! There’s no end to how much skill you can have at this game. No matter how good someone is at it, you only need to be one digit better and you will beat them over 90% of the time! So OK, maybe the length of the learning curve isn’t that indicative either.

        Hmmmm…. I’m not thinking that straight today and I don’t know where I’m going with this, so I’ll leave it there.

    1. It’s an interesting question. I’d argue that there’s not as much of what we would normally call “strategy” in Poker as in many other games, and yet, according to good players (i.e. not me) there’s an apparently important and subtle skill in there: reading other players. I have no idea how to understand or think about this skill in relation to my argument. Must think more.

  3. I’m not Richard Garfield, but I’ve bought enough magic cards to feel entitled responding 🙂
    Jokes apart, I think your assumption that good games make it hard to distinguish the effect of randomness and skill is wrong.
    Take backgammon, the quintessential game of “skill AND luck”. The draw of the game is that you can attribute your win to skill, but your losses to luck and thus not be discouraged.
    Anyway, in all games with skill there is an easy way to assess your skill vs luck : results of repeated play. This is why backgammon is played in matches of 7 points and poker in tournaments of hundred of hands. In the long term the luck averages out and you can assess your skill.
    I think your assessment is influenced more by “casual games” like Knizia euros; but in competitive tournament games (poker, backgammon, magic) it is easy to separate luck and skill.
    An example game: “make-your-luck-chess”. You play six games of chess, and for each win you get a number of your choice between 1 and 6. At the end of the game you roll a die and the player who owns the result wins 🙂

    Having said all that, it’s true that in some way luck “limits” skill. Actually it only makes skill more difficult to assess in the short term and thus to improve quickly; but this compromise is balanced by the thrill of playing with much better opponents without a foregone conclusion.

  4. I’m not Richard Garfield, but I’ve bought enough magic cards to feel entitled responding 🙂
    Jokes apart, I think your assumption that good games make it hard to distinguish the effect of randomness and skill is wrong.
    Take backgammon, the quintessential game of “skill AND luck”. The draw of the game is that you can attribute your win to skill, but your losses to luck and thus not be discouraged.
    Anyway, in all games with skill there is an easy way to assess your skill vs luck : results of repeated play. This is why backgammon is played in matches of 7 points and poker in tournaments of hundred of hands. In the long term the luck averages out and you can assess your skill.
    I think your assessment is influenced more by “casual games” like Knizia euros; but in competitive tournament games (poker, backgammon, magic) it is easy to separate luck and skill.
    An example game: “make-your-luck-chess”. You play six games of chess, and for each win you get a number of your choice between 1 and 6. At the end of the game you roll a die and the player who owns the result wins 🙂

    Having said all that, it’s true that in some way luck “limits” skill. Actually it only makes skill more difficult to assess in the short term and thus to improve quickly; but this compromise is balanced by the thrill of playing with much better opponents without a foregone conclusion.

  5. Hi Nick!

    I think there are a number of things going on in Contention #2 that could use some unpacking. I’m not going to claim that I’ve unpacked them all, but here are my current thoughts. (Some of this will echo Sean’s comments above.)

    When I talk about how much “skill” a game requires, or how “deep” it is, I essentially mean “the minimum size of the computer program that would play that game well”. More colloquially, the harder it is to find the best moves in a game, the more skill it requires.

    After I play any game that contains “good” elements of chance (in your sense), part of my challenge is to figure out what portion of the outcome was due to luck. I argue that this challenge should unproblematically be counted as part of the “skill” of the game, and the harder it is to determine what moves are best (in this case because of the randomness), the more skill the game contains. (Accruing or applying that skill may be too difficult, or tedious rather than fun, but that’s a different question.)

    Consider Chess vs. Go. It’s pretty certain that it’s harder to find the best moves (even in hindsight) in Go than in Chess. So we could argue the analogue of Contention #2: “The harder it is to distinguish between a good move and a bad move in a game, the harder it is to accrue skill.” And that probably is true in this case! It probably is harder to accrue skill in Go than it is in Chess. I guess you could therefore argue that, given the amount of time I have left in my lifetime, there’s a lower ceiling on the amount of skill I can accrue at the game of Go than of Chess, but that’s starting to get pretty fuzzy. And I would definitely resist the conclusion (or the way of speaking) that states that skill is therefore more limited in Go than in Chess.

    I realize you didn’t make any of these arguments about Go and Chess. My point is that I think that the arguments regarding randomness are analogous, and fail for similar reasons. The luck element (among other things) makes it very difficult to determine the best moves in No Limit Texas Hold’em, and that is a real and central aspect of the skill of the game. Even if it’s just semantics, I would find it very confusing to say that that “limits the rate at which players can accrue skill, which limits skill” in Texas Hold’em. FWIW, I think it’s pretty certain that No Limit Texas Hold’em is deeper than Chess (given my definitions), and I suspect it’s even deeper than Go.

    Maybe we should be talking about games that require *too much skill*. There’s definitely a sweet-spot for humans. I can imagine a game which is an order of magnitude deeper than Go or Poker, in which there are definitely moves that are much better than others, but they are so hard for humans to find that we may as well be choosing our moves randomly. Indeed, the difference between average human players and the best human players will not be large in this case. But I still prefer to draw the lines cleanly: this hypothetical game objectively “contains more skill” than Go or Poker. We just aren’t capable of acquiring much or any of that skill.

    I mention this, because you say, “If I’m examining the choices I made in a game I lost, how do I know if I made bad choices or if I got unlucky? If I can’t distinguish the effects of randomness from my own choices on the outcome, I’m stuck.” This implies that you think determining good play in the face of randomness can be prohibitively difficult. I think it depends on the game — it’s on a spectrum, like everything else. It *could* be prohibitively difficult, but so could finding good moves in some super-deep game of perfect information. In any case, I think it’s confusing to say that “skill is limited” in such games. I’d rather say something like “our skill is limited” when it comes to those games — that they’re “too deep” or require “too much skill”.

    — Kory

  6. Without discounting any of the pushback on the individual points, let’s take your conclusion (“Ergo, randomness limits skill in good strategy games”) as fact for a minute. As presented, it seems that this is a negative to you. The more interesting meta-question is, “is it a negative, or is it a tool?” As a single facet to consider, does a game with randomness create additional marketability to beginning players? By creating a mechanism when an inexperienced player *can* do well, do you create a small bit of experienced player frustration, in order to “welcome” the universe of new players (something publishers will often spend more time obsessing about that designers). If a top 10-ranked M:tG player loses to a 12-year old due to a lucky deck meeting a bad shuffle (even for a single round), does the fleeting frustration the experienced player experiences compare to the euphoria that the 12-year-old experiences (and transmits through his network of friends)? It’s the “Any Given Sunday” theorem: if there’s a chance, however slim, that either team will win, both team’s fan bases are energized. I’d view the concept presented neutrally, as something that can bring added value to a game if used appropriately.

    1. I agree with everything you’ve said here – i.e. it’s not a negative for me. It’s a tool. I like randomness in many games.

    2. Great point. For me, it actually is possible for a game to be “too deep” or require “too much skill”. So if you were to convince me that randomness limits skill in good strategy games (or that this is the best way of speaking about it), I wouldn’t necessarily view this as a bad thing.

      My game-designer friend James Kyle talks about how he likes games that allow players to express their own personalities and styles through play. I think randomness in general creates a lot of breathing room for this. In effect, it allows players to put a lower value on long-term success, and play more hopefully, or more flamboyantly, etc., while still having fun and sometimes winning.

  7. But remember … It’s a mathematical fact that all games without a random element are either unfair (best play always results in the same winner) or futile (tic-tac-toe). I personally want a balance of the two.

    1. As a theoretical matter, yes, but as a practical matter, from a game design point of view, not really. The point is relevant for perfect players, i.e. not humans.

  8. Roy’s comment has some thoughtful irony– so let’s see, pure strategy games are “by mathematical fact” unfair… because it’s difficult to find someone who matches you equally? I’ve usually looked at luck in a game (whether poker, backgammon, or a Knizia-design) as compelling because of the bet, statistically sizing up my chances. Then there is that *ka-ching* moment when a player pulls just the right card or die-roll, and for some that’s a w-a-y bigger rush than rook-taking-knight.
    And of course luck can temper the uneven-ness. So, if I’m good for 60% of strategic victories and my opponent is good for 40%, then if we play a game that is augmented to deliver 20% variability by luck, does that mean we’ve made the game even… or less so, for every additional percentage of luck? Anyway, just made we think of luck as more of a “great equalizer” in games vs. “the thrill” of hitting on a well-calculated bet.

  9. Also, for me the perfect strategy/bet/luck game from a modern designer isn’t from Reiner, it’s the game Boomtown (Bruno Cathala/Bruno Faidutti). God, that moment when I think I’ve snookered my buddy into overbidding for a mine-town is as good as anything I’ve ever felt at a poker-table. I wish I knew how to card-count for Boomtown!! For me, it’s the perfect mix of strategy/luck.

  10. A random thought:
    Rando-chess may be the perfect example of a unlimited skill + luck game in theory, but i’m not sure it’s a fair example. The game takes place while the players are involved in it, while they are making moves and decisions. The game ends, there is a winner that could be CHANGED by the die roll. The luck appears after the end of the game, that’s why it doesn’t limit skill expression.
    At best you can define it a game followed by a bet.
    Imagine the popularity-chess, where people play a game of chess then a jury decides who wins. You can write this in the “game manual” but it doesn’t make it part of the game, rather than a following event.
    Also, I think Contention #1 is, like another user said, not universally correct the way it’s stated.
    But if you let go the concept of better or worse and focus on the experience it could led to something like this “For the game experience to be different including or removing luck in a game, luck has to limit skill”.
    I don’t want to get too deep in the philosophy about “What is a game, when does it start and when does it ends?” but I strongly believe that what I stated earlier about players decision matters more than “when the last event described on the game manual happens”.
    Since I like extreme example, let me give you Age-Chess, where you play a game of chess, then the first player that dies loses the game. It’s not a game that lasts a life. It’s a game that lasts like a normal chess game, but takes more time to define the winner.
    Hope my flawed english didn’t hurt my argument too much!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s