The basic idea is this: the conception of narrative that you probably got back in school was one of escalating conflict and tension, followed by a climax and resolution, then denouement. This is also how I tended to think good games should feel. It has a lot of intuitive appeal, especially in the light of various practical problems boardgames usually have. Given that hobbyist games don't tend to get a ton of replay and mixed experience levels are very common, you'd like to give your players a chance to warm up with some lower-stakes conflicts early before proceeding to the high-stakes endgame. It also serves as a built-in catch-up mechanism since players who make poor choices or have bad luck early can still get back into the game with good moves later.
But as Hamlet's Hit Points makes clear, while this may be true on the macro level, this misses out on a very important key to how narratives keep and hold your attention during the moment-to-moment action. The book takes three classic plots – Hamlet, Dr. No, and Casablanca – and plots the action on a hope/fear axis. In each moment (or narrative "beat") your empathy for or feelings about the protagonist or other characters in the narrative are moving somehow: towards hope that things are going to work out well, or towards fear that they are not. Narrative tension relies on skillfully moving back and forth between these drives, not giving you too much hope without an injection of fear, or vice versa. In his analysis of Hamlet – an analysis I agree with – he finds that "down" (towards fear) or "up" (towards hope) beats in the story never cluster together in groups of more than about 3 in a row.
Bearing in mind that everything is obvious once it's been properly explained, this seems so clearly true, and so useful to GMs, designers, and people just wanting to understand a bit more about games, it's surprising nobody's said it before. Maybe they did, they just didn't have as clever a title or explain it as clearly.
Anyway, this simple concept has a great deal of explanatory power as to why some games work from a narrative perspective and why other, quite similar games don't.
Before starting, I'll stipulate a lot of boardgames don't necessarily succeed or fail based on emotional engagement or narrative. There is a branch of boardgames (let's call it the Caylus/Age of Steam branch) that fans like because of the pure intellectual challenge, and as such perhaps has more in common with a puzzle than a play or movie. Some players enjoy the lengthy period of frustration followed by the exhilaration of finding a solution. Having said that, let's also not make the mistake of associating "narrative" strictly with "thematic", or not looking at how nominally abstract games can engage us emotionally. Many successful abstract games, like the GIPF-series games or the classics like Chess and Go, do work with this pattern of balance between hope and fear.
With these caveats though, looking closely at the hope/fear beats of boardgames shows pretty quickly why some games are so engaging and some are not. I'll look at a pair of games, one successful, the other not so much: Reiner Knizia's Lord of the Rings and Arkham Horror.
The turn structure of these cooperative games, which is "do something good"/"do something evil", is clearly aimed at this modulation. In Lord of the Rings, the fear of what the tile draws from the bag are going to be is quite visceral. Looking at the structure of the events on the boards, which is what drives the fear of those tile draws, usually the events that occur early in each narrative are of the structure "meet some condition to receive a significant reward, otherwise suffer a significant penalty", which give the players hope for success but fear of failure. Later events tend then to get very bad, but at this point they are balanced against the hope of actually finishing the episode and moving on to the next, when the game reset involved in the episode transition gives the players a large jolt of hope as they move on to face the next challenge. Then when you get a chance to take your turn, you almost always receive clear, immediate, useful rewards that feed your hope of getting out of this mess alive.
So why does Arkham Horror not work as well as Lord of the Rings? If you think about it as an exercise in trying to move between hope and fear in reasonably tight circles, it's fairly obvious: Arkham Horror neither reliably rewards the characters to give them hope nor does it reliably put them in enough danger to be really fearful. Often you will visit a building with some hope of receiving a useful item or piece of information, but too often the rewards are minor, nonexistent, tangential to what you are trying to achieve, and generally not enough to inspire hope. The Mythos cards rarely have the dimension of meting out rewards or punishment that could inspire hope or fear, they are simply one-off events that the characters too infrequently can't do anything at all to anticipate, they simply respond. They are also too unreliable in their effects to get into any kind of cycle between hope and fear. An event that is not foreseen with at least some clarity can't inspire fear. The same thing can be said for character actions: too often there isn't enough you can do to give you hope, because clues are unavailable, you have to waste time in the hospital to recover health or sanity, and a route towards positive progress is not reliably open. Without some way to reliably make significant forward progress, we are denied the jolt of hope we need to keep interested.
This is not to say Arkham Horror can't get onto this virtuous cycle; sometimes the cards flow well and the situation develops in an interesting manner. But compared to the well-plotted structure of Lord of the Rings, Arkham Horror is relying on the luck of the draw to get into a good narrative zone. This is obviously not a great way to do this. As board gamers we tend not to like "scripting" in games, but scripting is obviously a mixed blessing. To the degree that it constrains player choice, it's not great. But narrative needs structure to work. As a recent convert to the GUMSHOE roleplaying system (designed by Robin Laws), I appreciate the book sections in the Esoterrorists book (also in Trail of Cthulhu) where he talks about railroading and the importance of giving the players the illusion of player control while keeping them on the narrative straight and narrow. These things are not contradictory.
While the comparison between Arkham Horror and Lord of the Rings is clear, you can see the logic here in tons of boardgames. For me, the difference between Dominion and Thunderstone in that Dominion is a fairly linear procession, while Thunderstone has some of this modulation. The flow of monsters up from the depths of the dungeon obviously helps. If you think of breach effect, traps, and treasures as hope/fear modulators and amplifiers, they make a lot of sense. Crucially, by giving you a set of characters with at least some personality that you can hope will advance in level and get more powerful while being afraid that they will die, Thunderstone helps you get invested in the game and actually feel something. Dominion gives you nothing.
There are plenty of other good examples in thematic games. Small World, where hope spring eternal when you draft a new race – and the personalization of the races and powers make a huge difference in our being able to identify with them – but gives way to fear as the race reaches the end of its rope and it becomes incredibly fragile in decline. Agricola is another classic manipulator, catching you between fear of starvation and ruin and the hopes that you have to build your farm, and thinking in these terms its tremendous popularity is easily explained. Classic games like Dune, Civilization, Titan, or Republic of Rome operate on longer time scales, but have amplified peaks and troughs of hope and fear, as anyone who has stared at their opponent across a combat wheel in a high-stakes battle in Dune can attest. Traditional games like Risk give you a lot of hope on your turn as your armies rampage across the board but then leave you to be very afraid of what your opponents are going to do to you once you pass the dice.
When you think about it terms of hope and fear, the visceral appeal of card driven wargames, especially the good ones like Hannibal, Successors, and Paths of Glory are likewise easily explained. Even titles which may not be as solid on game system merits (like Labyrinth or Twilight Struggle) can nonetheless be compelling because of the way they are always jerking you between hope for the cards you are holding and fear of what your opponent is going to do to you. Similarly, block games like Rommel in the Desert and EastFront manage this hope/fear balance, as they have the players playing in an environment of scarce information which is revealed in fits and starts, sometimes answering questions, sometimes creating new problems for you to grapple with, and giving you plenty of room to create your own hopes and fears.
Being about RPGs, one challenge that boardgames face that Hamlet's Hit Points doesn't talk about is how a narrative can keep this structure of hope and fear going when you play the game 5, 10, 15 times and know the general contours of the experience. This is not as much of an obstacle as you might think. The source narratives Laws analyzes are plays and movies which have survived a fair amount of repeat viewing. These emotional experiences the narrative aims to evoke are fundamentally manipulative. If you succeed the first time you can probably do it again.
This sort of modulation is obviously not the only way that narratives can be compelling. As mentioned in the book, rules are made to be broken, and some of our most compelling art comes from rules-breaking. But the lessons of Hamlet's Hit Points are extremely powerful as a tool to understanding what makes games tick.