To Drill Or Not To Drill?! is an educational board game designed to teach common citizens who might be uninformed about the natural gas fracturing process to approach the issue from a more informed and neutral perspective.
The board of To Drill Or Not To Drill?!
The players of To Drill Or Not To Drill?! compete for six months to see who can collect the most money from building wells on different plots of land. But in order to do so, they must correctly identify biased arguments that both support and oppose drilling for natural gas.
Course Project for Design of Educational Games during the spring semester of 2015 – Carnegie Mellon University, Pittsburgh, PA
Instructional / Game Designer
- Conducting cognitive task analysis to define and refine learning goals of the game
- Designing the game experience using the EDGE framework
- Conducting play tests and incorporating feedback over eight iterations
To Drill Or Not To Drill?! is a board game for 2 to 3 players ages 13 and up, and is an educational game aimed to influence the attitudes of players towards the subject of hydraulic fracturing for natural gas, or as many know it, “fracking”. The goal is for players to adopt an informed and neutral perspective in regards to fracking.
The game is set in a narrative in which you and the other players represent your own natural gas drilling company, and you are competing over the next six months to see who can make the most money. The person who makes the most money will earn a contract to continue drilling from all of the wells, so the stakes are high! Each round of gameplay represents one month of time, and the player must make a series of decisions. The players each start with $150.00.
The first move a player can make is to buy wells or land, which are priced differently according to their color. When a person buys a well they can use one of their markers to indicate it is owned by them. At the end of a player’s turn, they can collect money from all of their active wells. But in order to make a newly bought well active, another player submits a pro-fracking argument that describes why it is a good idea to build this well. The player whose turn it is must identify whether this argument is biased or not. If they identify the argument correctly, the newly bought well stays active for the next phase of the turn. If they incorrectly identify the argument, the well becomes inactive for the remainder of their turn.
Sample of cards used in gameplay.
Next, a player must identify another argument in order to collect money from their remaining active wells. This time, another player submits an anti-fracking argument, in which the consequence is to restrict some of the player’s highest yielding wells. If the player identifies the argument incorrectly this time, all of the wells become inactive; so in other words, they cannot collect money this round. If they identify a biased argument correctly, there is no immediate consequence and they can collect money from all of their active wells. If they identify an unbiased argument, they must restrict the wells that the card specifies, and then they are able to collect money from their remaining active wells.
Dynamics and Aesthetics
The goal of each round is to keep as many wells active as possible, because the more active wells means the more money collected. It is important to note, though, that a player is not required to buy a well each turn, and may need to assess if it is worth it to try to buy a new well, or to just try their chances at the ones they already own. Beyond that, a player may be wondering whether it is worth is to buy a more expensive well, because more expensive wells yield more money. If the player feels like they have low knowledge on recognizing biased arguments in the context of fracking, though, this could be a major risk. The advantage to taking these risks is getting the upper hand in your money count, since you only have 6 rounds to collect the most.
The resulting aesthetics from these dynamics are competition and challenge. The name of the game is making more money than your opponents, and you must know your stuff to do it.
The process of designing To Drill Or Not To Drill?! involved conducting a cognitive task analysis to define and refine learning goals, designing the rules and experience of playing the game while considering the EDGE framework, and conducting play testing sessions with target users to refine the design of the game.
Cognitive Task Analysis
In order to measure the explicit and implicit attitudes of citizens uninformed about the hydraulic fracturing process, a measure was created which elicited participants’ opinions on a specific subject category, had the participants read three facts about that subject category, then elicited their newly informed attitude on the same subject category.
Since hydraulic fracturing is a complicated process and much of the debate involves vastly different concepts, it made sense to break the task down into subject categories. While some people have strong opinions in regards to drinking water and hydraulic fracturing, others are more concerned about air emissions and their relationship with hydraulic fracturing. As a result, the set of tasks included five major subject categories:
General: This was participants’ general attitude about the hydraulic fracturing process. This also gave the researchers an idea of whether or not the participant had initial biases.
Drinking Water: This was participants’ attitudes about how hydraulic fracturing and drinking water are related.
Air Emissions: This was participants’ attitudes about how hydraulic fracturing and air emissions are related.
Water Availability: This was participants’ attitudes about how hydraulic fracturing and water supply are related.
Social Distractions: This was participants’ attitudes about how hydraulic fracturing and social distractions like traffic and noise are related.
Each category had three corresponding statements: one which supports hydraulic fracturing (bias pro), one which opposes hydraulic fracturing (bias anti), and one which takes a neutral standpoint.
Each category also had three corresponding facts. One fact was from a biased perspective which supported hydraulic fracturing (bias pro), one was from a biased perspective which opposed hydraulic fracturing (bias anti), and one was from a neutral perspective. The bias pro facts were retrieved from a pro-fracking interest group’s website, the bias anti facts were retrieved from an anti-fracking interest group’s website, and most of the neutral facts were chosen from either one of those sites, or a collection of research papers provided from the subject expert.
The facts and statements were checked by an expert who validated their biases and their correctness. All facts were true, and the ones that could have been argued to be false were simply over exaggerations or assumed causal connections that were not rigorously tested. A sample of a subject category with corresponding facts and statements is included in Figure 1 below, the full set is shown in A.1 in the Appendices. The facts were in a random order within each category when the participants viewed them.
A sample subject category. This subject category is about water availability and its relationship with hydraulic fracturing. Bias anti are highlighted in yellow, Bias pro are highlighted in green, and neutral statements are not highlighted.
- The researcher asks the participants to give their general attitudes about that subject category and its relationship with hydraulic fracturing.
- Based on the participant’s response, the researcher uses the three statements as a field guide. For example, if the participant had expressed a lot of concern with hurting our water supply, the researcher would ask, “Would you say you agree with [bias statement anti]? Would you disagree with [neutral statement]?”. This would allow the researcher to quantify the attitudes of participants on a rough 1-5 scale, with 1 being very “anti” and 5 being very “pro”. If the participant agreed with the bias statement but also agreed with the neutral statement, then the score would be 2 or 4.*
- The participant read (or in the case of Google Hangout interviews, listened to) the three facts associated with that subject category.
- The researcher asked the participant to reflect on their attitude about that subject category, again using the statements as a guide to measure.
- Each measure from step 2 and 4 were recorded in order to see which subject categories the participants had the most difficult time adopting a neutral attitude towards, or if any pre-existing biases guided their decisions.
*It is important to note that the statements were read to the participants so that they could not see all three at the same time. It was the researcher’s thought that if the participants had to choose between three statements that best reflect their attitude, uninformed individuals may have the tendency to choose the most neutral statement.
Although many of the participants asked about the reliability of the sources of facts, it was decided that this information would not be disclosed. This was due to the assumption that participants’ implicit bias would guide how much they believe (or do not believe) a specific fact, and that they would be more likely to perceive facts biased toward their implicit attitudes as being more reliable than those that challenge their current attitudes.
- Assistant curator from Pittsburgh, PA
- Retails pharmacist from Pittsburgh, PA
- Quality control clerk for developing computer chips from Reading, PA
In order to analyze the data that was collected, the researchers visualized each of the participant’s measured attitudes on the same scale. This was done in order to visualize if there were any trends that may have indicated it was hard to adopt a neutral attitude in that subject category, or if one side of the argument had seemingly stronger evidence that influenced uninformed citizens. Figure 2 shows the results for each subject category. The participants are coded by color.
The results of initial and newly informed attitudes on each subject category for all participants in the cognitive task analysis.
Through analyzing the data was collected, some difficulties in adopting neutral attitudes about a specific subject category began to reveal themselves. For example; in regards to air emissions, the first participant ignored the fact that was comparing the CO2 emissions from burning natural gas to those of burning other fossil fuels like coal. Since the participant had an initial bias against hydraulic fracturing, the facts about other air emissions like VOC’s seemed more valuable to her argument and thus informed her attitudes after reading the facts. On the other hand, the other two participants who had less of an initial bias hung on to the fact about how clean natural gas is in comparison to burning other fossil fuels, which informed their “pro” attitude after reading the facts.
Another area of difficulty was the subject of water availability. While all of the participants noted that they did not know how much water the process used before reading the facts, all of the participants were surprised to learn that “hydraulic fracturing takes 1-8 million gallons of water per job.” Although this is much less than other fossil fuel extraction methods like digging mines, evidence including numbers and figures seemed to resonate with the participants, suggesting that the “anti” arguments in regards to water availability and hydraulic fracturing may be perceived as stronger to uninformed individuals.
This information will be useful in addressing specific difficulty factors in the game; that is, in regards to air emissions, the game will have to make clear that natural gas does produce less CO2 emissions than other fossil fuels, however, the process of hydraulic fracturing may release other harmful chemicals into the air. In addition, water that is used in hydraulic fracturing is less than other natural gas extraction processes and can sometimes be recycled, but it is important that local officials work with companies to ensure that enough water is available to the community.
Another area of interest which was brought up in each of the participants’ transcripts was in regards to the job market and hydraulic fracturing. Although this was not included as an original task category, the fact that it was brought up by each participant points to the fact that this may be important information to include in the game, since that is information that is more salient to uninformed individuals.
Surprisingly, most of the participants were able to recognize bias facts for reasons that did make them bias. For example, the first participant pointed out, “Sounds like bullshit” when she read a fact that was bias pro in regards to water availability. That means the learning goal of “recognizing bias facts” is of less importance than originally anticipated before starting the analysis, and so less emphasis will be put on this goal in the development of the game’s instruction.
Overall, the game had 4 major playtests. The majority of players were college graduates who live and work around Pittsburgh, and happened to be at my apartment while I was working on the game in the past 4 weeks.
The original game had much more complex mechanics and was much less fun for players. In addition to collecting money, the players would also collect respect points for limiting wells, and the higher yielding wells yielded more respect for restricting. This means the players were competing to spend more money to get higher yielding plots of land, but were actually hoping to draw a card which shuts those wells down in order to win the game. Since this did not fit with the goals of adopting a neutral attitude towards fracking, it was decided to add some cards that include arguments which support fracking, and which the players could also earn respect for identifying accurately.
With this revised version of the game, each color plot had it’s own deck of cards, and the more expensive plot’s deck included larger rewards for identifying both pro and anti fracking arguments. If the player owned more than one color plot, then they would roll a pair of dice to decide which deck they would draw from. The highest yielding decks had a lower probability of being drawn. The major problem here was that the reward for identifying a pro argument was additional money and respect, and the reward for identifying an anti argument (if it is unbiased) limited the amount of money you can collect more and gave you an equivalent amount of respect. The amount of rewards a player would receive due to chance was not equal to the amount of punishments a player would receive due to chance, so this was evidently a major problem. In order to address this issue, it was decided that you must identify a pro argument in order to build a well, since it makes more sense to submit a pro argument in that context. In order to collect money from the wells, you had to correctly identify an anti argument.
Finally, it was decided that the result of identifying an argument incorrectly is that you simply cannot collect money, and that identifying an argument correctly simply gave you the opportunity to make money. This was an attempt to simplify the game and make it more directly linked to the learning goals. By eliminating respect points, the players have one goal in mind: making money. But in order to make money, they are not allowed to side with biased arguments. This also allowed us to reduce the number of card decks to two; one for pro arguments and one for anti arguments, since the reward was not linked to a specific well anymore.
The final playtest showed better results in regards to the players having fun and being less confused about the rules. One last change that was made was cutting the number of round from 12 to 6. By shortening the game, the players are encouraged to take more risks and made the game much more exciting.