Nov 15, 2018

I have just returned from Blizzcon2018, where I had the opportunity to ask a few questions to several Blizzard employees on how their teams work and what project management frameworks they use to develop their content. This article is my humble attempt to stitch their comments, publicly available information (other articles, job postings), my knowledge of modern development and a healthy dose of speculation to give you all a window into how Blizzard creates – in their own words – the most epic entertainment experiences ever.

Blizzard employee’s comments can be grouped as follows:

Engineering teams use agile, while the design and art teams use waterfall
Design teams are around three releases ahead of the engineering team
Art teams are around one to two releases ahead of the engineering team
Producers and team leaders use a mixture of in-house project management tools and vendor tools
Support teams have developed tools to capture detailed logs of crashes and bugs as soon as they happen. The information is then stored in a database that is optimized to aggregate issues and find error patterns
There are many cross-group events, including playing production demos, at Blizzard that help employees learn what other teams are working on, and/or connect at a personal level

The fun factory

I was initially surprised by the waterfall comments and the clear indications that game development teams were structured as functional teams. My expectation was that game development teams would be fully agile cross-functional teams. However, after thinking it through, Blizzard approach makes sense.

Blizzard has had small, agile, cross-functional teams in the past – and very likely there are a few agile teams right now figuring out the next hit in some dark dungeon. Team 7, the team responsible for Hearthstone, started so. Indeed, agile practices are excellent for pre-production, iterating to “find the fun,” but they are less effective in production, where the effort is centered on steady content creation. As most Blizzard teams are devoted to creating new content to existing games the need to use agile to de-risk the subjective process of creating a fun game decreases and the need for an efficient assembly line of new content (expansions, DLC, events) increases. Furthermore, cohesive storytelling and long-term character arcs benefit from the careful planning and design that is typical of the traditional waterfall.

Thus, I came to the conclusion that Blizzard content producing teams are essentially a “fun factory”. Functional teams are not individually working in waterfall or agile; instead, they are left to self-organize in the best way possible according to their discipline (art/design/ engineering) and collectively managed using a combination of lean and Kanban. Blizzard employees’ mention of waterfall or agile as their team’s methodology was likely more of an indication on whether they were documenting upfront or using a living and prioritized backlog to drive the work.

From Toyota to your keyboard

In automobile factories, many steps must be completed in order before the final product is ready to hit the market, this sequential process created natural bottlenecks in the most labor intensive or complex steps. Modern lean production and Kanban has its roots in Toyota factories in the 1940s. The idea is to streamline production lines and to deliver quality products on time. The lean part is about continuous improvement in the process to eliminate unnecessary (waste in lean parlance) steps and balance the workload in every stage to avoid bottlenecks. The Kanban part is about bringing transparency to the process, to give ample time to the management and employees to respond to any issue that might arise.

Game development can be understood as a sequential process, in the sense that the design team can work in new storylines and mechanics, then the art team picks up the themes and set by the design team (or technology limitations) to create assets. The engineering team could theoretically work in parallel with the art team; however, the need to service the current version of the game (patches and bug fixes) and any infrastructure or content creation tools means that the bulk of the engineering team lives in the present while the other teams create the future.

However, the problem with sequential processes that require specialized talent is that there is little to do while you wait for the others to finish their work – in other words, employee allocation is inefficient.

Representation of waterfall game development

This worker allocation inefficiency is a significant issue for developers concentrating in releasing completely new games every 2 years – as I cover in my article don’t skip pre-production – but less so for Blizzard, as we already established that most of their teams are creating content for existing games according to a long-term vision. In their case the design team can move immediately to work on the next expansion as soon as they pass their vision to the art and engineering teams, so too the art team can start working on the next expansion as soon as they finish. Lean management and Kanban would then be used to organize the tasks in a way that any waiting time is minimized.

Development teams are at different game expanion cycles

The resulting organization should resemble the graph above, a few observations though:

Forward-looking teams still dedicate some effort to instruct balance patches and learn from player interaction with the current version. Player feedback will be incorporated in future releases
Audio and cinematics teams are not included in the graph for simplicity
The distinction between gameplay engineers and other engineers is necessary to highlight that gameplay engineers’ work is to create new features for upcoming expansions while the other teams mostly support the current version (and the PTR when close to a new release). The weeks leading to a new version release should have some of the engineering teams coordinating patch distribution and promotion with Blizzard’s IT teams.

The pros and cons of Blizzard structure

Category	Pros	Cons & Mitigation strategy
Vision	Tight story: As mentioned before, cohesive storytelling and long-term character arcs are easier to do in waterfall	Harder to make corrections: As design and art are planned far ahead, any change of course could potentially invalidate tons of assets and storylines, thus possibly losing thousands of work hours. A common theme in Reddit posts about Blizzard is that the developers don’t list to the fans or that they are too slow to act. Blizzard does listen, but they will prefer to include player feedback in a later expansion – likely two or three release cycles ahead, the space the design team is at
Quality	Blizzard polish: The cohesive theme described by the design team is taken by the other teams to create a game filled with the trademark Blizzard qualities of vibrant visuals, light-hearted fun and attention to detail	Delayed releases: In project management, the triad of cost, time and quality are all interdependent; Blizzard quest’s for quality means that time is sacrificed. Indeed, Blizzard is famous for releasing games “when they are done.” The way to mitigate this is to streamline project pipelines (using lean analysis) and not overcommit (using previous experience to better estimate project scope). The Hearthstone team track record of delivering expansions on time is evidence that Blizzard has been getting much better at timeboxing their efforts
Employee experience	Smooths employee mentoring: Less experienced employees get to interact more, and thus learn, from experienced colleagues on a day to day basis. This is a core component of Blizzard’s core pillar of “Learn and Grow” Foster specialization: Functional teams are better at utilizing resources with limited availability than cross-functional scrum teams, this in turn, means that system and assets can be polished to a high level	Could lead to tribalism: a known defect of organizations that are structured around functional areas is the tendency to form tribes. I was not able to see any of this from the outside, and I guess that Blizzard’s management is careful to sponsor as many special cross groups as possible (boardgame enthusiasts, runners, foodies, etc.) to have people from different teams create personal connections. Information silos: another well-known defect of all functional team centered organizations, is that information is contained within teams and is not flowing freely through the greater development group (i.e., the design team makes a change and is not communicated to the art team in time). From what I could gather Blizzard tries to solve the information problem with a combination of good collaboration software and frequent team demonstrations, where teams show what they are working on to other teams for feedback, seems to be encouraged

Epic projects require good collaboration and project management software

Blizzard uses a mix of Atlassian products and custom-built project management software to maintain as much transparency as possible – They have been an Atlassian shop for a long time now, and probably use at least Jira (for backlog and bug management), Confluence (for documentation) and Trello (for Kanban tracking). I heard they had automated bug reports and I am pretty sure they must have some auto reporting mechanism in their level design tools, to ensure senior producers have visibility on the status of the assets in the pipeline.

If you are interested in applying to Blizzard it wouldn’t be a bad idea to pick up some Atlassian product experience. If you are reading this to understand how can you improve your production kung-fu, keep in mind that these tools augment the work of leads and producers, but don’t replace sound production management – build your software platform around your ideal pipeline and organizational model, not the other way around.