So there's a fellow going by the nom de guerre of Nikephoros, writing on the blog Bringer of Victory. He and his Warhammer "metric" are gaining popularity. I believe this article to be his original post on the subject. Lately I've come to realize that I was unfairly dismissive of the execution of his attempt to design a Warhammer 40,000 evaluation metric for list-building. In part this is because I did not address it fairly and objectively the first time I reviewed it, and in part this is because the apparent popularity of this metric requires that it be seriously and objectively refuted.
If it sounds like I'll be bending some of my arguments to fit some pre-conceived notions, in suggesting the Nikephoros' metric (hereafter NM) need refutation. I'm going to be using the traditional essay format of stating my thesis, explaining my reasoning, and then reiterating my thesis as the conclusion to an argument formed by my reasoning. And as any good scholastic philosopher knows, if you cannot escape the conclusions of your argument, go back and edit the introduction so that you're arguing for those conclusions!
In this case I've come to the conclusion that NM is flawed in both general principles and detail, even if it is both well-meaning and heroic in the attempt.
To summarize NM as honestly and charitably as I can, NM is about correlating expected performance with an empirical record of wins on the tabletop. The expected performance is measured alone four axes: Dead Marines from optimal shooting ('DMS'), Dead Marines on an optimal charge ('DMCC'), Dead Rhinos per game ('DRPG'), Dead Land Raiders per game ('DLRPG'). These numbers are interpreted on a point-by-point basis. So while one weapon may have raw power on its side, it may cause more proportionally. These number are aggregated on a unit by unit basis for armies at a particular point level. Nikephoros uses the top four lists from the 2010 Nova Open to generate scores.
So let's go back to how one wins games in Warhammer 40,000. One way of winning any game is to eliminate your opponent entirely. Another way is to win more kill-points. A third way is to capture more objectives. The first way definitely depends on your army's ability to kill the other player's army, and the other player's ability to both kill your units and avoid getting their own killed. Likewise you don't have to reach a certain number of kill points so much as get more than your opponent. Tournaments like the NOVA, for instance, even defer winning by two kill points or less to secondary victory conditions. Finally, winning objectives is a matter of having more objectives held on the turn the game ends, which again is also accomplished by contesting objectives as well as killing units.
Don't get me wrong, killing stuff, especially Space Marines is important in Warhammer 40,000. But if you only consider killing Space Marines, and then only certain Space Marines (and vehicles), and don't consider the utility of glancing hits given the new rules for cumulative damage effects, and make unsupportable assumptions about optimal positioning in a zero-sum game, then you're no longer doing useful math.
So besides generating these metrics from highly artificial and unusual (aka optimal) conditions, against a disproportionately represented material asset, Nikephoros doesn't even generate accurate numbers. Units armed with Missile Launchers, for example, apparently have no effect on AV14 Land Raiders, which is (strictly speaking), false. In game theory terms it's like arguing for rock in Rock-Scissors-Paper because lots of people like rock, and rock does well when people take scissors. In other words this isn't game theory, it's the same old bad statistics that have mislead people about the optimal tactics and strategies in Warhammer 40,000 time and again, not because the numbers themselves are inaccurate, but because they have not been properly contextualized according to the game theoretic concepts that the rules of the game actually employ.
So while Nikephoros is on the right track in advocating the application of game theory to the game of Warhammer, he is not actually applying the right parts of game theory, and thus failing to provide adequate context to test the value of truth-bearing statements about the game. I've heard mention that people admire Nikephoros for being a good scientist, and certainly his approach is actually a very good one if you're planning on testing some natural phenomenon. However, he and his followers should not fool themselves into believing that attempting to treat Warhammer units like major league managers treat baseball players is in any way realistic or useful. Warhammer 40,000 is much more like one of the toy-games of mathematicians than a complex phenomenon like major league baseball.
I mean, obviously NM is useful in the sense that it gets people thinking, which is better than not-thinking and gives me an opportunity to harp on about the right way to do these things. But NM is not useful in the sense that it gets people thinking the right way, which is about using the material defined by an army list on the space and time of the board to achieve the game's Nash Equilibrium.
So to review my thesis. I said that I would show how NM is wrong in both detail and general principles. I have shown how NM ignores the general principles of the game, concentrating on killing models at the expense of winning the game, since killing models is neither necessary nor sufficient for winning the game. I have shown how NM is wrong in detail, citing both the un-justifiable assumption of optimal conditions, and lack of computation of marginal results such as glancing hits. These results are no mere mistakes of calculation, they are indicative of severe theoretical problems underlying the NM, in both detail and principle.
I would like to add that NM also fails to show how time, material, and position act to synergize and 'de-synergize' (can't think of a better negation for synergize...) the kill-numbers that it generates. Finally, and this isn't Nikephoros' fault considering how difficult it is to collect and appraise play-testing feedback, NM isn't applied in a rigorous empirical fashion to a sufficiently large sample, as if empirical testing could tell us anything about winning strategies for a game like Warhammer 40,000.
Put another way, if you're going to tell me that a unit of six Long Fangs with five Missile Launchers will be expected to kill exactly 0 Land Raiders per game while killing more than 8 Rhinos per game, I'm going to dismiss you out of hand for never actually reading the fucking rules.
Sorry Nikephoros, but you're doing it wrong, and misleading people.