A re-post from the Preserving Games blog, October 19, 2009.
Fábio Petrillo et al., “What went wrong? A survey of problems in game development,” Computers in Entertainment 7, no. 1 (2, 2009): 1-22.
This February 2009 article from the Computer in Entertainment magazine of ACM takes a look at the game industry and compares its difficulties to the larger software industry. Specifically the authors analyze twenty postmortems from the archives of Gamasutra.com to characterize the problems that plague game development. I believe Gamasutra has discontinued this series but postmortems are still published by sister publication Game Developer.
A postmortem “designates a document that summarizes the project development experience, with a strong emphasis on the positive and negative aspects of the development cycle.” After reviewing the literature discussing problems present in the software industry, the authors begin to analyze the problems described in the postmortems. The games covered and the problems identified and quantified are in a table that describes the number of occurrences and overall frequency (click for larger image). Note: sometime in the future (the Web 3.0 future?) I would provide a link to the actual dataset rather than a .PNG showing you a picture of the dataset.
The authors’ categories provide a helpful navigation to the issues that arise in a game development project. As they note, this study sees the most cited problems as unreal or ambitious scope and features creep, both constituting 75% of all problems described. Notable for game archivists is a 40% frequency for the lack of documentation problem as well. The authors note low occurrences for crunch time and over budget (25%), both “said to be ‘universal.'” It’s difficult however to draw expansive conclusions from a small dataset. Moreover postmortems were not team projects or collaboratively written, rather a single participant is responsible for the postmortem. The authors usefully provide other limitations to put the data in context.
The authors conclude that the electronic games industry does indeed suffer from problems in the larger software industry (overly ambitious plans and poor requirements analysis) as well as woes peculiar to itself: the first to experiment with new technologies, tool problems, and collaboration between disparate professionals, among others.
On a final note, the postmortems are still available at Gamasutra, and they are really fascinating reads. It becomes clear just how young an engineering and creative discipline digital game-making is, and how much fluctuation there is in how a game turns out. There are some great examples and stories there; the authors of this article cite quite a few of them.
Walker Sampson 