In the last post I explored some possible reasons why Flashpoint's turn order is just so forgettable. In this post, I present the results of my little experiment and explore what it all means.
A brief recap
Flashpoint is a co-operative game about firefighting. Its turn order requires each player to undertake three steps: take their actions to fight fire / save people, advance the fire itself, and replenish potential victims. After seeing people frequently forget to advance the fire, I theorised that something about this turn order makes it forgettable and, further, that an order in which players advance fire first would be more memorable.
Part I covers my three main working theories: first that the current turn order requires the players to 'break character'; second that the game subverts some hidden expectations about dice; and third that the sequencing of challenges and resolutions is counter-intuitive.
I have now played the game with a range of different people changing only one rule. Instead of having players advance fire and add POIs at the end of a turn, I’ve added them at the beginning. So where the game turn order is written is Actions > Advance fire > POIs, I have changed this to Advance fire > POIs > Actions.
Crucial caveats
It's worth repeating that my methods are nowhere near as scientific as they could be. I’ve just tried to do what is realistically within my time constraints, getting as much bang for my buck as possible (and, of course, have fun in the process!):
- I focused on noting the advance fire roll, not the POI rolls.
- I played with people who had never played before or people who had not played for a long time to limit prior exposure to the existing turn order
- I played without expansions
- I played in both basic and 'experienced' modes
- When I started noting the first session I wasn't as diligent as I was later on, so I don't think that I recorded every occasion correctly. I have opted to only include those I noted, and record these numbers as "at least".
The results
Game session #1 – No change
Setup:
- 3 players – me and two colleagues, who are both newbie players
- 2 games played – the first in basic mode, the second in experienced
- Original turn order maintained
Results:
The dice roll was forgotten on at least 5 occasions on the first game and at least 4 on the second. I did not observe whether or not we rolled for POIs.
This effectively served as my control.
Game session #2 – New turn order
Setup:
- 2 players – me and my dad, who has not played for a long time
- 2 games played – both in basic mode
- New turn order put into place
Results:
The advance fire roll was forgotten 2 times in the first game and then not at all on the second game. A definite improvement. However, we also still forgot to roll for POIs on at least 3 occasions.
Game session #3 – New turn order
Set-up:
- 3 players – me and two friends who were newbie players
- 2 games played – one in basic, then one in experienced
- New turn order tested again
Results
The advance fire roll was forgotten once in the basic game and not at all in the second. Another definite improvement. Again, POIs were forgotten at least 3 times.
The flipped turn order is more memorable – but why?
Though limited, my results strongly suggest that flipping the turn order by doing the game's own admin first leads to a definite improvement in memorability. In the time since I've discussed it with other people. Anecdotally, their experience was similar when they have tried it. That's a successful experiment and a potential design win already.
However, proving (or disproving) the specific hypotheses is a much harder problem that I have been wrestling with for a while for this follow-up.
1) It works against the game’s intrinsic role play
After the experiment was over I asked the people playing if they thought this was plausible based on their experience. When I also posted about it on BoardGameGeek, some posters there agreed that it made sense to them.
But to actually prove that this is a cause specifically you would need another experiment. One possible option that I've considered is a re-theming of the game. If you could construct a a mechanically similar game, in which it wasn't out of character to advance fire, you could compare the results and see if there was any difference in propensity to forget: either by re-themeing or making it more abstract entirely. That's still a challenging task however, given the way that fire destroys the victory points that the players are trying to "win" (POIs) and eventually ends the game if it advances too far (the house collapses). Whatever you call the fire, it's going to be hard for players to feel "on its side" unless you change other rules.
2) Rolling dice sets expectations that Flashpoint subverts
Trying to isolate the effects of expectation created by dice in an experiment seems even harder. The only way that I can think of to tackle this definitively is find a group of people not at all familiar with dice as a control group. To anyone attempting this: good luck with that.
But it gets somewhat more complex from there. Since those people have no expectations about dice at all, you'd expect them not to be as familiar what to do with dice anyway. This means you'd also need to prime their expectations in another direction first. Perhaps if they were trained to play another game beforehand – one which sets an expectation that you roll last – you could see if they now find rolling at the end of the turn easier to remember. That would at least prove that expectations are important. But this is all desperately hypothetical and a bit of a reach: I don't think that we should hold our breath for anyone testing this kind of thing soon.
That being said, an in-experiment observation seems relevant to this one – some in-game behaviour of my own, in fact. Across all of the games (with turn order as original and flipped), when I finished my turn I very deliberately placed the dice in front of the next player, a habit that I picked up quite a while ago. On a couple of occasions in the first session (original turn order) the player to my left tried to roll them right away. They had to be corrected, of course, because that's not what the rules say. But then on one of these occasions they then also forgot to roll at the end of the turn. In the subsequent games (new turn order) players who were passed the dice rolled them right away almost every time. When they did it, it was so fast that it was like a reflex action: my visual prompt seemed to instantly prompt behaviour without much thought.
That's only a small observation and if you pass a player anything they're more likely to remember that they need to do something with it. But, in the very least, such a strategy can only work if the player is supposed to do something with the dice right away, and the speed of the reflex suggested something very ingrained about dice to me.
3) We expect challenge to be followed by resolution, not the other way around
As I mentioned in part 1, I thought that this one is on the least sure ground before I began the experiment.
To test for this specifically, I think that the most practical approach would be to compare against many different games with a variety of different sequences of challenges and resolution, flipping the order in each case and then measuring the results. My hunch is that within what we see as the container or delineator – i.e. a single turn – having the challenge first within the container will prove easier to remember. But there's nothing in my experiment that can validate or disprove this specifically.
Other observations – the problem with POIs
Notably placing new POIs was still often forgotten even in my new turn order. I don’t have a proper baseline for how often they are forgotten in the control game so I don’t know if this error rate was higher. But either way, the fact that they were still forgotten more than the fire advancement suggests that there is something else going on. My speculation is that the POI placing mechanics still suffer from two other issues. Firstly, it’s hard to just remember an administrative sequence. If you’re given the dice it’s clear that you need to do something with them and you know that there's at least one thing to remember at the start of a turn. If players need to just remember to do another thing with them immediately after however, then the human mind’s enormous potential to simply forget stuff will lead to more errors; we are often lost without a prompt. Secondly, the game asks the player to constantly look out for not having enough of something. But a negative is nearly always harder to spot than a positive. It’s much easier to look for a token than be aware that there is no token of a certain type, so unlike the advance fire roll (which always has to happen), this one seems to start at a disadvantage.
What would I do differently?
If I ran the experiment again, I would definitely look further into the memorability of POI placement as well. There is something interesting there in the necessity of prompts to remember all but the most basic sequences.
I'd also aim to play more games and with a larger variety and number of players. From the first game I would have my noting practice down more clearly. I used Google Keep this time, but I think that in a fast paced-game a good old fashioned notepad with simple strike marks each time someone forgot to do something would do the trick.
Concluding thoughts
I have to conclude that the turn order in which the fire is advanced first is more memorable.
I suspect that all of the factors that I've suggested contribute in some way given the above, but it's hard to pinpoint the precise causes, or the contribution of each, without further analysis.
But there are a few things that I am going to take away from this and incorporate into my own design thinking. There's some evidence for aspects of my theories and most of this is rooted in common sense and basic usability logic. If we waited around for a research level of certainty, we wouldn't get much designed at all. I think that I can reasonably say that 1) it can’t be better for cognitive load to be knocking a person constantly out of the theme than keep them in it, 2) it’s got to be easier to work with people’s expectations of dice than against them, and 3) even if it doesn't actually make a difference either way, I cannot believe that a turn order in which challenge follows resolution can be more intuitive than to have your challenge first.
With thanks to the good folk of BoardGameGeek (esp. Laura Creighton and Marc Lecours) for their feedback and opinions, Jordi Garcia, who gave me so many good ideas about this, Tim Richardson for early inspiration and encouragement, and for any readers of this blog who have so patiently waited for part II.
Leave a comment