Player-reported Impediments to Game-based Learning

J. Tuomas Harviainen

School of Information Sciences
University of Tampere

Timo Lainema

Turku School of Economics
University of Turku

Eeli Saarinen

Turku School of Economics
University of Turku

Abstract

This article addresses the question of how games function as learning tools, from the perspective of player-stated problems. It is based on interviews and essays, collected from university students who reported problems dealing with unrealistic levels of trust and competitive play leading to searching for the logic in the game as an artifact instead of considering decision alternatives which would work in real-world situations. According to the respondents, the main cause of problems appears to be that for many participants, games are framed as an activity that is to be done competitively. Along with reporting the impediments, the article discusses potential solutions.

Keywords

briefing, learning impediments, simulation/games

Introduction

The history of training games and simulations goes back at least 3,000 years (Keys and Wolfe 1990). Journals such as Simulation & Gaming have documented training games and their results for four decades, and while their significance is under frequent debate, games and simulations are accepted as a valuable part of the educational toolkit - at least in fields where they are in constant use. Their educational significance, especially in areas considered physically or fiscally risky (e.g. crisis management, aviation, business), remains strong. In this article, we present critiques expressed by participants of business simulation/games, outline it in context, and suggest options for removing or at least weakening those learning impediments.
The process of adding game-like elements to activities, “gamification,” is offered as a solution to various problems in both learning and general society (McGonigal 2011), and as certain organizations successfully adopt play as part of their culture (Warmelink 2011), it is necessary to also see the problems of the way simulation/games function as learning tools. It appears that at least some players - in their own view, at least - do not reach the full learning potential from such artifacts. This, according to them, is because they concentrate on game-play in a manner that prohibits some types of learning incentives from functioning. This may not be the full truth, but since in all game-based learning we are currently just extrapolating from case examples and do not have a commonly shared theory for understanding it (Klabbers 2003), such reports have to be taken seriously until more credible learning assessment methods have been devised.
In this article, we answer the question of what business simulation/game players themselves see as potential impediments to their game-based learning, and how to possibly solve the reported problems. It draws on both simulation/game research and the study of educational and recreational games and simulations in other areas. Combining these and player interviews, we display how the perceived “game” context and the resulting competitiveness affects learning. While games may very effectively negate existing mental models (Tsuchiya and Tsuchiya 1999) and teach systemic thinking (Senge 2006), it appears that players are often still fixed into the mental model (a deeply ingrained assumption, generalization, or even picture or image that influences how we understand the world and how we take action) of a game being solely a competition (we will later discuss the negative consequences of business games considered solely as competitions). As a result, they default to a game-based logic, instead of using the simulation/game as an opportunity to learn real-life practices and skills. We believe this to be a major cause of problems with game-based learning.

Methodology

Participant interviews were conducted by Harviainen, following the discourse-as-interview practice (unstructured, often informal discourse that includes the interview questions; see Kvale and Brinkmann 2009). The respondents were selected randomly. Five interviewees were students (three majoring in business studies, two from others faculties), two (Alex and Robert) were top-tier executives in their mid-50s, who had both played business games as a part of their studies. Three of the students (Jack, Jim and Iris) were interviewed multiple times. The interviews were translated from Finnish by Harviainen.
The interviewees were initially asked general questions about their experiences with business simulations. As all spontaneously expressed experiences of problems with learning from simulation/games, follow-up questions were directed to ascertaining more data about those issues, in a probing interview (Stewart and Cash 2006). The answers were then compared to 102 player essays, collected by Saarinen and Lainema from simulation/game-based courses they taught, in order to make sure that the reported problems were not singular incidents, tied to any single game, or to deployment by a particular teacher. (The student interviewees had written similar essays after the simulation/games they played, but their essays were not included in this particular data set.) The player-reported problems we discuss in this article were all mentioned by several respondents, from multiple games. We believe that this process was sufficient to remove the risk of the participants reporting only game-like qualities of the simulation-games, as responses from both data sets pointed to the same conclusions, of their own initiative. As Jørgensen (2012) has shown, game players sometimes report widely dissimilar experiences and motives than outside observers believe them to have. Regardless of whose assessment of the learning situation – observer or player – is more accurate, this obvious dissonance requires more research. Thus the necessity of this article.
All respondents had experiences with commercial digital games as well as digital business simulation/games. The business games from which the interviewees responses came (most prominently LEKA, RealGame and Intopia) were too numerous and complex to describe here, but they all deal with high-realism simulation of the management of fictional companies that purchase materials, manufacture products from those materials and then sell the products. Some of the games had companies of only one type, whereas in others, participants steered companies which also collaborated with other student-led companies along a supply chain between suppliers and manufacturers. The students had played both turn-based (“batch-processing”) and clock-driven (“real-time processing”) business games, the corporate-professional respondents solely turn-based ones. Our data showed, however, that very similar answers arose regardless of which exact combinations each respondent had played. Gender furthermore had no significant influence on the students' responses, nor did their study majors. The sole significant difference we found was that according to two interviewees, business students in their deployment groups had been much more likely to take risky, big loans than had the non-business students who played in the same deployments.
We have chosen to exemplify common answers by suitable interviewee quotes in each chapter, but all of the findings were reported by multiple respondents in both interviews and essays.

Simulations and games as learning tools

In business simulation games – unlike in the actual world – participants are free to experiment with policies and strategies without fear of jeopardizing a real company (Senge and Lannon 1997). Such freedom is nevertheless relative to the gaming situation and can be for example constrained by team play or time zone differences. Even in such cases, where group pressure may limit experimentation, it includes the kind of reflection and inquiry for which there is no time in the hectic everyday world. Thus, Senge and Lannon argue, participants learn about the long-term, systemic consequences of their actions. Simulations designed for general purpose management learning are useful for helping individuals to see relationships between various business decisions and potential outcomes (Senge and Fulmer 1993). The insights gained from experience with these activities may be transferred to an organization when participants face situations that remind them of similar challenges faced in the simulation.
Lately simulation gaming has been seen as the kind of learning activity that realizes the ideas of constructivist learning (Lainema 2009). Lave and Wenger (1991) emphasize that the traditional view of learning as internalization is too easily construed as an unproblematic process of absorbing the given, as a matter of transmission and assimilation. Instead, the focus should be on the skills of reflectivity of the learner, not on remembering. Constructivism focuses on the process of knowledge construction and the development of reflexive awareness of that process (Bednar et al. 1992). Learning is a process of enculturation that is supported through social interaction and the circulation of narrative (Brown et al. 1989). Instruction should not focus on transmitting plans to the learner but rather on developing the skills of the learner to construct plans in response to situational demands and opportunities. Instruction should provide contexts and assistance that will aid the individual in making sense of the environment as it is encountered (Duffy and Jonassen 1992). The learner is building an internal representation of knowledge, a personal interpretation of experience (Bednar et al. 1992). Learning is an active process in which meaning is developed on the basis of experience. Learning must be situated in a rich context, reflective of real-world contexts for this constructive process to occur. The goal is to portray tasks, not to define the structure of learning required to achieve that task.
Duffy and Cunningham (1996) exemplify constructivism in the form of problem-based learning. The focus should be on developing the skills related to solving the problem as well as other problems like it. Skills are developed through working on the problem, i.e., through authentic activity. It is impossible to describe what is learned in terms of the activity alone or in terms of the content alone (p. 190): “Rather, it is the activity in relation to the content that defines learning: the ability to think critically in that content domain, to collaborate with peers and use them to test ideas about issues, and the ability to locate information related to the issues and bring it to bear on the diagnosis”. The teacher does not teach students what they should do/know and when they should do/know it. Rather, the teacher supports the students in developing their critical thinking skills, self-directed learning skills and content knowledge in relation to the problem.
Simulation gaming exercises seem to support and fulfill most of the constructivist requirements for effective learning environments (Lainema 2009). All the aspects above – developing the reflective and interpretative skills of the learners, the social interaction of the learning environment, the narrative nature of a game, self-directed nature of the learning process, the richness of the game learning environment, the potential authenticity of the experience – support the use of simulation games in education. Authenticity and realism has a role both in simulation games and constructivism (Lainema 2009). The learning artefacts themselves are called simulation games. What is simulated is some of the critical features of the reality (Saunders 1995). Keys and Wolfe (1990) define a management simulation game as a simplified simulated experiential environment that contains enough verisimilitude, or illusion of reality, to include real world–like responses by those participating in the exercise. The very essential feature of supposed authenticity of the simulation games, that is the basis for potential learning, is not without problems, as we will discuss later in this paper (see Schaffer and Resnick 1999, on how “authentic” can be varyingly interpreted in educational contexts).
According to Elgood (1996) games in general have some advantages compared to lectures:
  • Games can have considerable subject knowledge built into them. They can pose problems, demand answers and respond to the answers with a judgment that is knowledge-based and right. Students learn by experience rather than by hearsay.
  • In a lecture, it is not guaranteed that the information transmitted is also listened to and understood. Although this cannot be guaranteed in a game environment either, the nature of business games usually creates interest on the subject matter, because the participants are themselves operating on the matter. This would suggest that in games even if less is being officially transmitted, much more is being received.
  • In games motivation is further enhanced by the expectation of enjoyment and freedom of action that is associated with the word ‘game’. Human individuals are often competitive by nature.
  • In games there is usually discussion between the participants. Thus, the views of many people are being considered. Elgood mentions that the process of understanding may receive more help from discussions with somebody operating at one’s own level rather than with an expert.
Our respondents, in both interviews and essays, pointed to these advantages, yet at the same time cast a shadow on uncritically accepting them as reliable and sufficient, as will be discussed below. They also mentioned shifts in perspective, in accordance with the idea of changing mental models. The means of simulation gaming to be used to change governing mental models and to create a shared mental model among participants (Tsuchiya and Tsuchiya 1999) are:
  • Voluntary learning: the fun element of gaming encourages participation.
  • Creation of turmoil: the conflict and turmoil created by a game raises a doubt in mental models and lessens resistance to change.
  • Big picture: when the participants grasp the whole, the individual mental models become larger. As a consequence, compatibility of participants’ mental models increases and the creation of a shared mental model becomes possible.
  • Compression: compression of time and space makes experiential learning possible and accelerates the learning process: the outcomes of actions are experienced in a short period of time.
  • Risk-free environment.
  • Shared experience: essential for the creation of a shared mental model.
  • Rich interpretation of history: the expansion of our comprehension of past experience through gaming enables us to learn from small episodes within the real world.
  • Cause maps: the cause maps developed through participation in playing help participants to interpret and make sense of their lives.
As a summary from a literature survey, Keys and Wolfe (1990) note that many of the claims and counterclaims for the teaching power of business games rest on anecdotal material or inadequate or poorly implemented research designs. These research defects have clouded the business gaming literature and have hampered the creation of a cumulative stream of research. Dickinson and Faria (1995) sum up the findings of five major review articles (viewing all together 160 studies) on the effectiveness of business game training compared against other instructional methods. Simulation gaming was found to be superior in 46.9 %, not as effective in 16.9 % and no learning differences were reported in 36.2 % of the cases. They conclude that simulation games are at least as good as other instructional methods and possibly superior.
That conclusion seems a bit naïve, as drawing this kind of assumption of the applicability of simulation gaming in general in different educational situations is quite questionable. As Villegas (1997) notes, no concrete evidence exists that simulation games are superior or more effective than other techniques used in training. Gosenpud (1990), in turn, states that researchers should stay away from the kind of study where the experiential method is compared with others (usually in terms of some very general measure of cognitive learning). This kind of study is value laden, stimulates unnecessary controversy and the knowledge gained from it is in terms of winners and losers, nothing else.
Keys and Wolfe (1990) note that most of the research has been focused on team performance in games, not learning, with the assumption that high performance teams learn the most from a game experience. Thus, they note, research is needed to evaluate the relationship between learning in a business game and performance in a game. We think that we need research on the conditions which facilitate learning through gaming. Performance is interesting mostly from the point of view of student assessment.
Business games are often, quite inconsistently, criticized because of the lack of well-designed evaluative studies to establish learning validity. Although this is certainly partly true, it should be recognized that it applies even more so to learning from the commonly accepted teaching methodologies such as cases, lectures, or simpler exercises (Keys and Wolfe 1990). One should furthermore note that research on training simulations and games does not necessarily apply beyond immediate boundaries, because fundamental differences may exist between educational uses of commercial games, designed-for-education games, and the drill-simulations of, say, soldiers and firemen (Whitton 2009).
Motivating games are exceptionally good at prompting out-of-game information seeking and learning. Even if nothing formal is learned, players may pick up things such as etiquette, group management and social skills (Duchenaut and Moore 2005). Different games foster different kinds and facets of learning. Whitton (2009), following Gagné, et al. (1992), lists five main categories:
  • Intellectual skill (Concepts, rules and relationships, and making discriminations such as using algebra to solve a mathematical puzzle).
  • Cognitive strategy (Personal techniques for thought and action, such as developing a mental model of a problem).
  • Verbal information (Relating facts, such as recalling the names of the bones in the hand).
  • Motor skill (Actions that use muscles, such as dancing).
  • Attitude (Beliefs and feelings, such as choosing to read detective fiction).
Digital games can be used to support all five types, even as one single game will not support all of them. For university level training, the intellectual, cognitive strategy and attitude facets are the most important, as they directly tie into what is taught at that level. While games can assist in memorization and comprehension, their true value lies in the higher domains (Whitton 2009), offering spaces for experimentation and cognitive development through the freedom to try and fail (Tsuchiya and Tsuchiya 1999). They also support affective learning (as per Bloom 1956), even when players go against teacher or designer intent, because such “functional bad play”, i.e. exploration of the game-system’s limits, too, is often beneficial (see Myers 2010). So where is the problem?

The map is not the territory

Simulations and games may mimic real-life situations as closely as possible, but they are not real life. This is especially true of simulations that already deal with abstractions like virtual money. Lloyd (2007), in her study of the learning practices of Australian firemen, found that even physical simulations are considered “theory learning” by participants. Until the lessons are turned into embodied knowledge remembered by their bodies, it is not seen as real knowledge. This is despite the fact that games and simulations have been documented as producing psychophysiological reactions in line with real-world experiences (Kivikangas et al. 2010).
Educational games depend on immersion for their learning potential. It is not just a lure for getting people to learn, but a fundamental requisite of their proper function as educational tools (Balzer 2011). On the other hand, too much immersion is again harmful to the learning (Henriksen 2008), and needs to be addressed by proper briefing before and after, as immersion and reflection may contradict each other (Henriksen 2010). Furthermore, games designed to be educational need to be directed, in order to produce the desired results (Hsu 1989). Sandbox-type games (open-form games which offer lots of freedom of action) are not as suitable.

Accentuated trust

Jim: “Before the start of the game [given that we ended up on different levels of the supply chain], Jack and I decided to form a permanent chain. We would buy solely from his team, and he sell exclusively to us, no matter what happened. So we were able to act as [if we were] one larger company, which gave us an edge over the others in the game. The downside was that if Jack would have decided to screw us over at any point, we would have been doomed.”
Q: “Would you have trusted someone you did not know beforehand that much in the game?”
Jim: “No.”
Q: “Would you trust someone that far in real-life business deals?”
Jim: “No.”
Q: “Not even a friend like Jack?”
Jim: “Hell, no. The stakes are so different there.”(Jim, business student)
One of the key problems mentioned by the interviewees was a perception of unrealistic trust amongst players. In Roberts’ (1969), terms, this is like learning to fire blanks while real corporations use “live ammo”. When playing a game with people they know, participants know what sort of behavior they can expect from their fellows. At the same time, playing with people one personally knows heightens arousal and positive valence created by the game, even if the players are not present in the same room (Kivikangas et al. 2010). As this is combined with on one hand the lack of real monetary risks, on the other the social risk of betraying one’s friends, a situation of potentially accentuated trust is born. Jack and Jim both stressed that because the setting was artificial, they felt no need to even calculate the risks involved in trusting each other, and instead just went “all in.”
Therefore, at least in the case of some players, levels of cultural trust and mistrust related to the consequentiality of exchanges of resources (see Misztal 1996) are altered. In other words, certain players develop temporary systems of trust that may have no correlation to what they would do in normal life. Whether this is towards increased or lessened trust depends on the persons in question and their past history. As far as ethics are concerned, it can be considered good that players do honor their deals, but real-world economics cannot be trusted to always function that way.
It would thus appear of interest to conduct experiments in forcing players to re-evaluate such trust. The simplest, but rarely feasible, way would be to have only strangers play together. One possible option for addressing accentuated trust would be the assignation of additional goals, such as working as a traitor against one’s own team on behalf of a competitor or sub-contractor, looking for such a traitor, or seizing managerial control of one's team. This can be done by administering pre-written characters (i.e. fictional personas players have to portray), thereby allowing participants to distance their game-internal actions from their real-world personalities and thus lessening associated stigma. Smaller alterations, such as the definition of a mandatory managerial or teamwork-participation style, are also possible.

Excessive competitiveness

Educational games are very much games of skill. In many cases, no chance component is present. This seems to cause some players to treat them as competitive, even when directly instructed to do otherwise:
Q: “Were you instructed to play individually and to treat it as a learning exercise?”
Iris: “Yes, very clearly.”
Q: ”Did you nevertheless play it as a competition against each other?”
Iris: “Yes, totally. Of course we did.”(Iris, natural sciences student)
Corporations compete with each other in real life, and managers sometimes make irrational choices, so why is there harm in competitive play? This is because one of the key advantages of simulations is that they make it possible for players to try out various strategies without tangible risk to themselves or a real corporation (Crookall, Oxford and Saunders 1987). If the competition is too fierce, there is no chance for significant exploration, or correction of errors later on, as the players’ minds are set on performance, not learning. Making errors becomes just something fatal to winning, not a means of learning. Due to excessive competition, chances are not taken, as participants stick to what is perceived as winning strategies, even if those strategies would be completely absurd in real life. This phenomenon was noted by all of our interviewees, pointed out in essays, and mentioned as the most irritating facet of business games by the two corporate professionals.
Several solutions to this problem exist, ones that do not remove a teacher’s ability to assess performance or learning. They do, however, introduce other complications. For example, constant teacher observation is highly inefficient and time-consuming. One option is to simply make the players work without knowledge of each others’ progress. This, however, requires single-session runs (to prevent corridor-talk) as well as the removal of any possibilities of the players comparing “scores.” It is also a move away from realism - real corporations devote significant resources to environmental scanning, in order to ensure their position on the market (Choo 2002), so to use it would be again a step towards game-based logic.
One possibility is increasing transparency, which can be done in several ways. Lainema (2008) has recommended opening up the “black box” of game mechanics for the players. This would permit more exploration of techniques instead of vague ideas and adaptation to new tactics when others appear inefficient. Likewise, managing tools should be visible, so as to differentiate between management skill and technical knowledge (Lisk, Kaplancali and Riggio 2012). Another option would be the introduction of environmental scanning into the simulation itself: by devoting game-internal resources to corporate benchmarking, players would be able to analyse the performance of the other companies and apply the successful methods in their own teams. This would lead to both increased realism and collective learning, as players could directly benefit from each others’ experiments.
One more option would be the development of more complex simulation-games played as competition against the game system itself, but with the presence of multiple players in them, so that the human aspect is not lost. Research on massively multi-player online games (Ibid.) has shown that this will foster shared learning, effectively breaking down what Huysman and de Wit (2002) call the “individual learning trap,” where people hoard what they learn. Competing against each other, in turn, accentuates the trap, as players are not willing to risk losing their edge. In such a case, the social nature of learning is turned against itself, because a significant part of the social setting (the players) promotes not sharing what is learned.

Unnatural risks

“When I studied, I hated the business games. Our team, made of people with years of actual business experience, was always losing, because we tried to run the fictional company like a real one, while the younger students just played it as a competition. Therefore as the end approached, we too would start making [real-world-wise] insane choices, like selling all the workers to slavery, as we were graded by profit alone. [...] [I’d] cut it in mid run, to prevent last-minute changes. And judge it with criteria not tied too much to just profit.” (Alex, top-tier executive in a multinational company)
The game situation is not seen as matching a real one. The taking of unnatural risks, “as if there were no tomorrow, because we knew there would not be one," was reported by all interviewees and in several essays. As noted above, one of the key points of using simulation/games is that they are without real risk (Tsuchiya and Tsuchiya 1999), and well-calculated risk-taking is what allows businesses to progress and succeed, yet as reported by the interviewees, the perceived lack of game-internal risks causes (or at least allows) participants to treat the game less seriously.
A serious problem is embedded in that assumption: people will not necessarily explore and experiment, but rather just seek the “correct answer” defined by the game designers for gaining the optimal result. Instead of using the possibilities of the game being not real to see how failure may happen (Myers 2010; Crookall, Oxford and Saunders 1987), they start stabbing in the proverbial dark to find the magic bullet. In such situations the damage goes beyond just stupid risk-taking: despite of what game-education idealists claim, the players are actually learning the wrong thing - believing that instead of calculated risk-taking, one should utilize unsuitably high risks so as to potentially get a similarly high return - which brings us back once more to the risk of game-internal learning. There is not much use to a fireman who is unable to stay safe because he has been dealing with just fake fire.
Ways out of this trap exist, the first and foremost being active facilitation where absurd decisions are immediately put to question, a process that, however, can usually be implemented only in small-scale games. It is highly unlikely that any real-world wagers would actually remove the problems. Neither are they solvable through improved interface design: as shown by Faaborg (2005), even highly advanced live-action role-playing games, easily able to meet and even transcend all criteria set for “mode III” training simulations by Klabbers (2000), i.e. being “learning environments in which the learners are given the opportunity to interactively build their own system of resources and rules[, and which provide] conditions for the interactive self-reproduction of social systems,” suffer from severe cases of inflation.
Several potential solutions exist. It is possible to implement a “public trust“ measurement/grading system, which would mimic stock market reactions to outrageous risks. That option, however, carries along with it a designer’s or teacher’s own set of value judgments, potentially leading to misrepresentations of a free market. Another solution is much more insidious, and possibly much more engaging: virtual property and possessions have real value to people who actively participate in continuous online environments (Castronova 2005). Similarly, many players appreciate public achievements in other forms of networked play. Therefore one significant alternative would be the linking of learning games/simulations to an achievement system of some kind, one which other players would be able to voluntarily peruse or ignore, as they see fit.
A teacher may also extrapolate from the end-state of the game, testing the viability of the choices. Selling off one's means of productions and realising stocks of half-produce would merely mean that the company would crash, whereas the carefully planned production and supply chain would make it through the extrapolation. Several interviewees recommended this approach.

Outright cheating and game-based logic

“Our team was doing bad, until we, between play, heard from older students that there was a bug: we could charge as much as we wanted for [a product], as its sales numbers were fixed, regardless of price. That way we acquired money really fast, and were able to invest it into other projects, enhancing our general efficiency.”(Jim)
Simulation players do cheat. When asked for more details, Jim said that “it felt right, given the game-nature of the activity,” yet said he would not cheat in “normal games.” The nature of this seemingly strange response lies in participant expectations: in a training situation, everyone wants to shine, and the activity itself is outlined as a competitive game, i.e. something many people take as “not so serious” by default (see Salen and Zimmerman 2004). There therefore exists the idea that everyone is willing to do what it takes to win, even if the simulation-game is not framed as a competition. This makes dishonesty seem normal.
Dishonesty, in this case, means not traditional “cheating,” i.e. the breaking of game rules or social contracts relating to the game for the purpose of winning it (Consalvo 2007). Rather, it is the following of game-based logic (Suits 1978) instead of real-world logic, as mentioned above. Under normal gaming circumstances this would not be a problem. In a learning game, however, it distracts from learning (Henriksen 2008). Instead of reflection, players may either skip the lesson completely, as part of “just a game,” or adopt game-logic based ideas to real life: for example, in 2008 Nascar driver Carl Edwards tried to bounce his car off a wall, an idea he claimed to have picked up from video games (Harris 2008).
The ability to read a game as a system is essential to both enjoyable game-play and to learning from that play (see Wardrip-Fruin 2009). If the players however utilize their abilities of game-system analysis ("procedural literacy"; Bogost 2007) solely to exploit the game system, and fail to compare it to real life, their learning remains tied to game-based logic instead of reality. Game-based logic may not in all cases be harmful to learning processes, but the risk exists at all times. If players try and understand the logic of the game, in order to either exploit it or to score at least some points to save face, when they are far behind others (Henriksen 2008), they may learn to look for similar mechanisms in real life. If, however, they skip the lessons as simply situation-related and insignificant, all that which is learned during play may be discarded after it, despite even a good debriefing (Kim 1993). That which is viable in the game may not be viable when confronted with the materialistic reality. This is why it is imperative to recognize that performance is not the same as learning, and to apply that idea to grading the students’ play (Knotts and Keys 1997).
Lainema (2003) lists six potential problems between a simulation game profit, student learning and students’ performance assessment (p. 90). First, profit making in a game does not correspond to learning. Secondly, those participants who make the most mistakes may very well also learn the most (being in line with the educational views on problem-based learning). Thirdly, is success in real-world equal to a better understanding of business logic? It is probable that factors like individual character and gender play an important role in the real-world, but not necessarily in games. Fourth, making the biggest profit sounds like quite a one-sided goal and research variable. Business organizations can succeed well in the light of some other values (like human well-being and committed employees) and short-term profit making may be secondary but the implicit assumption underlying many business games is that good financial performance is a good measure of the participants’ overall achievement, encompassing strategy formulation and implementation. Burgess (1999) refers to the management literature about the difficulty of ascertaining in practice a direct relationship between the two variables. This highlights the danger of assuming that good performance implies good strategy. Fifth, a formally excellent strategy may not necessarily lead to success in a game. In a truly interactive game, how a certain strategy works depends on the opponents' strategies. For example, if all the participants choose the same formally excellent strategy, probably none of the participants succeeds very well. Sixth, as Burns et al. (1990) note, a false assumption exists, which connects measures of performance and measures of learning in many studies. Performance indicants can imply learning when learning has not occurred.
It is therefore absolutely necessary that game-based teaching not tie itself to just playing. The students have to reflect on the play afterwards, both through formal debriefing as well by themselves. Focus should be placed especially on dynamics between the decisions they made and the results of those decisions, so as to not reward cheating, or, more precisely, cheating that does not contribute to learning. This is in line with rewarding overall performance and learning, not just elements such as direct profit (Whitton 2009).

Discussion

Players bring much more than just their school training into a simulation. Gosenpud (1990) describes the problematic area illustratively: the learner often learns things not intended by the designer, and often this unintended learning is more valuable because it is relevant to the learner. Designers cannot furthermore rely on players “stealth learning,” i.e. learning without realizing it, as stealth learning, in many cases, prohibits reflection, because the learner is not sufficiently aware of it (Whitton 2009; Henriksen 2008). Evaluation, defined by the designer, may miss the real worth of the experiential because what is valuable for the learner is defined by the learner and may have nothing to do with the designer’s intention. Players can have highly significant pre-expectations that affect learning, but not everyone does. Not all people who have grown up with digital culture are interested in gaming, nor do all players consider them worthy as learning tools (Whitton, 2009). Most of what players do during a game is based on rules and behavior patterns not specified in the game material, but on what they bring along. Without that, the play would not be possible (Crookall, Oxford and Saunders 1987). Of special interest in this article’s case is that they bring their concepts of “game” into the game, playing it accordingly. The upside of this is the potential for extended learning - players who like the simulation/game will want to continue playing, and voluntarily plan and study for the game (Lainema and Saarinen 2009). This was reported by all five of the currently-studying interviewees, as well as in many essays. The downside, as discussed, is that a game is easily seen as being a competition, where learning is incidental and ancillary to the task of being the best. That, too, can be used for great advantage with a properly done debriefing, as much of the real learning takes during post-game reflection. Yet we believe this to still be insufficient.
Many games create significant cognitive changes during play, but the changes, in most cases, soon dissipate. Practical skills can be retained longer, but without use those too will wither. Therefore it is necessary to anchor the learning into existing modes of action or a cultural context that keeps it active (Harviainen and Lieberoth 2012), even when the game itself is used as an anchoring tool for earlier learning. Pre-game workshops are also highly useful (Bruun 2011). They explicate goals, both in playing style and results, and enable the players to find more opportunities for personally suitable anchoring.
We are aware that all of the recommendations we have suggested in this article are known to be in current use somewhere. The issue here, however, is rather different: The underlying problems - regardless of whether they were experienced by just these interviewees, or are more ubiquitous - must be understood, and changes implemented in order to address the real impediments. This brings us back to Senge (2006): if we are to teach game-players systemic thinking, we ourselves have to see the simulation-games as systems, not simply directed tools. To do that, it is necessary to look beyond the confines of simulation-games, into recreational games, and the way they engage players. If the game-as-system has been suitably framed, participants will contribute to it. Changing the frame changes their perspectives on the system, including what within it is considered good and bad (Van der Heijden 2004).
It is highly unlikely that the reported problems would result from failures in deploying the games, given that our respondents reported learning impediments with several simulations, all of which were run by different supervisors. The issue is more complex, and not something that should be left unanalyzed because it provides weapons for those who oppose game-based learning. It appears that just like theorists, players too may view simulations and educational games differently, based on their personalities, the way they relate to gaming in general, and each game’s traits (type of play, winning conditions, etc.). For some, educational games are representations of real worlds and real-world phenomena, even when they contain fantasy elements. For others, like our interviewees and essay writers, they are “operating realities” in their own right, meaning they are seen as not necessarily having direct or explicit representational power, ability or value. They are competitions, with their own rules which take precedence over learning purposes. As in recreational games, this does not negate educational value, yet works against immediate reflection, and should be addressed during debriefings. Asking the participants, for instance, how they think the game session differed from reality - instead of just assuming they know - may be sufficient as a tool for starting an anchoring process (Harviainen and Lieberoth 2012). For this purpose, we especially recommend the use of reflective essays as a part of the debriefing process, preferably also between game sessions. Reflective essay, in this context, is a written report of a participant’s views and even feelings about game session activities. It is a personal reflection based on personal experience, in which a set of guiding questions or topics help participants to focus on relevant issues and thus gain the most benefit from the time invested in simulation game sessions.
It is quite possible that the players who suffer from the problems listed in this article do gain educational advantages from the games, they just do so differently. It is our hypothesis that for those players, the anchoring aspect of the learning games is heightened: By playing, they do not learn as much as some others do during play, but their style of play allows them to use the game to apply their earlier learning for a superior performance which, in turn, leads them to retain that earlier knowledge better. It is obvious that further studies are required, so as to both ascertain this hypothesis as well as to see whether the players who report learning impediments actually, incidentally learn as much as the others do, and are simply either less adept at recognizing learning, more adept at recognizing potential problems, or both.

Conclusion

Certain players appear to seek performance at the expense of learning, as they consider games to be a competitive activity. This guides them to unrealistic trust and a game-based logic of business phenomena (e.g. searching for the logic in the game as an artifact instead of considering decision alternatives which would work in real-world situations), as well as outright cheating. Such players may nevertheless learn by playing, even if they do not recognize that fact themselves. In order to combat these perceived learning impediments, experiments should be conducted with new types of games, while adding more thorough briefings and especially reflective debriefings and anchoring to currently used simulation/games. By guiding student-players to see the wider array of options offered by game-based learning, their enjoyment of the activity can be preserved, while simultaneously increasing the likelihood that the educational goals of the activity will also be reached. In order to do this, the idea of game performance - profit - as the main criterion for grading students has to be abandoned.

Bibliography

Bednar, A. K., D. Cunningham, T. M. Duffy and J. D. Perry “Theory into practice: How do we link?” in T. M. Duffy and D. H. Jonassen (eds.), Constructivism and the technology of instruction: A conversation, p. 17-34. Lawrence Erlbaum, Hillsdale, NJ, 1992.
Bloom, B. S. Taxonomy of educational objectives, handbook 1: The cognitive domain. David McKay Co. Inc, New York, 1956.
Bogost, I. Persuasive games: The expressive power of video games. The MIT Press, Cambridge, MA, 2007.
Brown, J. S., A. Collins, and P. Duguid. “Situated cognition and the culture of learning.” In Educational Researcher vol. 18 (1989): 32–42.
Bruun, J. “Pre-larp workshops as learning situations”, in T. D. Henriksen, C. Bierlich, K. Friis Hansen and V. Kølle (eds), Think larp: Academic writings from KP2011, pp. 172-193. Rollespilsakademiet, Copenhagen, DK, 2011.
Burgess, T. F. “Strategy support during a business game using an expert system,” in D. Saunders, and J. Severn, (eds.), The international simulation & gaming research yearbook: Simulations and games for strategy and policy planning, pp. 87-101. Kogan Page, London, U.K., 1999.
Burns, A. C., J. W. Gentry and J. Wolfe. “A cornucopia of considerations in evaluating the effectiveness of experiential pedagogies,” in J.W. Gentry (ed), Guide to business gaming and experiential learning, pp. 253-278. Nichols/GP, London, U.K.,1990.
Castronova, E. Synthetic worlds: The business and culture of online games. University of Chicago Press, Chicago, IL, 2005.
Choo, C. W. Information management for the intelligent organization: The art of scanning the environment, 3rd Ed. Information Today, Medford, 2002.
Consalvo, M. Cheating: Gaining advantage in videogames. MIT Press, Cambridge, MA, 2007.
Crookall. D., R. Oxford, and D. Saunders. “Towards a reconceptualization of simulation: From representation to reality,” in Simulation/Games for Learning vol. 17 no. 4 (1987): 147-171.
Dickinson, J. R. and A. J. Faria. “Simulation gaming for sales management training and a demonstration,” in D. Saunders, The simulation and gaming workbook volume 3: Games and simulations for business, pp. 99-109. Kogan Page, London, U.K., 1995.
Duchenaut, N. and R. J. Moore. “More than just ‘XP’: Learning social skills in massively multiplayer online games,” in Interactive Technology & Smart Education vol. 2 (2005): 89-100.
Duffy, T. M., and D. J. Cunningham. “Constructivism: Implications for the design and delivery of instruction,” In D. H. Jonassen (ed.), Handbook of research for educational communications and technology, pp. 170-198. Macmillan Library Reference, New York, NY, 1996.
Duffy, T. M. and D. H. Jonassen. “Preface,” in T. M. Duffy, T. M. and D. H. Jonassen (eds.), Constructivism and the technology of instruction: A conversation. Lawrence Erlbaum, Hillsdale, NJ, 1992.
Elgood, C. Using management games, 2nd Edition. Gower Press, Aldershot, U.K., 1996.
Faaborg, M. L. “Quantifying in-game economy: A contribution to the analysis of the in-game economy of larps,” In P. Bøckmann and R. Hutchison (eds.), Dissecting larp: Collected papers for Knutepunkt 2005, pp. 189-203. Knutepunkt, Oslo, Norway, 2005.
Gagné, R. M., L. J. Briggs and W. W. Wager. Principles of instructional design. Harcourt Brace & Co, Fort Worth, TX, 1992.
Gosenpud, J. “Evaluation of experiential learning.” In J. W. Gentry (ed.), Guide to business gaming and experiential learning, pp. 301-329. Nichols/GP, London, U.K., 1990.
Harris, M. “Johnson wins, Edwards hits wall on purpose.” Hampton Roads, 2008. http://hamptonroads.com/2008/09/johnson-wins-edwards-hits-wall-purpose (accessed Nov. 2012)
Harviainen, J. T. and A. Lieberoth. “Similarity of information processes in games and rituals: Magical interfaces.” In Simulation & Gaming vol. 43 no. 4 (2012): 528-549.
Henriksen, T. D. “Extending experiences of learning games - or why learning games should be neither fun, educational or realistic.” In O. Leino, H. Wirman and A. Fernandez (eds.), Extending experiences: Structure, analysis and design of computer game player experience, pp. 140-162. University of Lapland, Rovaniemi, 2008.
Henriksen, T. D. A little more conversation, a little less action: Rethinking learning games for the organisation. Doctoral dissertation, Aarhus Universitetet, Aarhus, DK, 2009.
Henriksen, T. D. “Moving educational role-play beyond entertainment.” In Teoría de la educación - Educación y Cultura en la Sociedad de la Información vol. 11 no. 3 (2010): 226-262.
Hsu, E. “Role-event gaming simulation in management education: A conceptual framework and review.” In Simulation & Gaming vol. 20 no. 4 (1989): 409-438.
Huysman, M. and D. de Wit.. Knowledge sharing in practice. Kluwer Academic, Dordrecht, the Netherlands, 2002.
Intopia inc. Intopia. [PC] 1963. http://www.intopiainc.com/
Jørgensen, K. “Players as coresearchers: Expert player perspective as an aid to understanding games.” In Simulation & Gaming. vol. 43 no. 3 (2012): 374-390.
Keys, B. and J. Wolfe. “The role of management games and simulations in education and research.” In Journal of Management vol. 16 no. 2 (1990): 307-336.
Kim, D. H. “The link between individual and organizational learning,” in Sloan Management Review vol. 35 no. 1 (1993): 37-50.
Kivikangas, J. M., I. Ekman, G. Chanel, S. Järvelä, B. Cowley, M. Salminen, P. Henttonen, and N. Ravaja. ”Review on psychophysiological methods in game research.” In Proceedings of DiGRA Nordic: Experiencing games: Games, play, and players, 2010. http://www.digra.org/dl/db/10343.06308.pdf (accessed Nov. 2012)
Klabbers, J. H. G. “Learning as acquisition and learning as interaction.” In Simulation & Gaming vol. 31 no. 3 (2000): 380-406.
Klabbers, J. H. G. “Interactive learning of what?” In Percival, F., Godfrey, H., Laybourn, P. and Murray, S. (eds.), The international simulation & gaming yearbook vol. 11 (2003): 257-266. Napier University, Edinburgh UK.
Knotts, U. S. Jr. and J. B. Keys. “Teaching strategic management with a business game,” in Simulation & Gaming vol. 28 no. 4 (1997), pp. 377-394.
Kvale, S. and S. Brinkmann. Interviews: Learning the craft of qualitative research interviewing. Sage, Los Angeles, CA, 2009.
Lainema, T. Enhancing organizational business process perception – experiences from constructing and applying a dynamic business simulation game. Doctoral dissertation, Turku School of Economics, Turku, Finland, 2003. http://info.tse.fi/julkaisut/vk/Ae5_2003.pdf
Lainema, T. RealGame. [PC] 1997. http://www.realgame.fi/index2_eng.php
Lainema, T. ”Perspective making: Constructivism as a meaning-making structure for simulation gaming.” In Simulation & Gaming vol. 40 no. 1 (2009): 48-67.
Lainema, T. and Saarinen, E. ”Learning about virtual work and communication - the distributed case,” in Molka-Danielsen, J. (ed.), Proceedings of the 32nd Information Systems Research Seminar in Scandinavia, IRIS 32, Inclusive Design. Molde University College, Molde, Norway, August 9-12, 2009.
Lisk, T. C., U. T. Kaplancali and R. E. Riggio. “Leadership in multiplayer online gaming environments.” In Simulation & Gaming vol. 43 no. 1 (2012): 133-149.
Lloyd, A. “Learning to put out the red stuff: Becoming information literate through discursive practice.” In Library Quarterly vol. 77 no. 2 (2007): 181-198.
McGonigal, J. Reality is broken: Why games make us better and how they can change the world. Penguin Press, New York, New York, 2011.
Misztal, B. A. Trust in modern societies: The search for the bases of social order. Polity Press, Cambridge, U.K., 1996.
Myers, D. Play redux: The form of computer games. The University of Michigan Press and the University of Michigan Library, Ann Arbor, MI, 2010.
Roberts, A. L. “What's wrong with business games?” In R. G. Graham and C. F. Gray, (eds.), Business games handbook, pp. 47-51. American Management Association, New York, NY, 1969.
Saarinen, E., T. Lainema & S. Lähteenmäki. “Experiencing virtual team membership. Decentralized decision-making processes leading to meaningful learning.” In Proceedings of the Seventh IASTED International Conference on Web-based Education WBE '08, pp. 327-332. Anaheim, CA, ACTA Press 2008.
Salen, K. and Zimmerman, E. Rules of play: Game design fundamentals. MIT Press, Cambridge, MA, 2004.
Saunders, D. “Introducing simulations and games for business.” In D. Saunders (ed.), The simulation and gaming workbook: Vol. 3. Games and simulations for business, pp. 13-20. Kogan Page, London, U.K., 1995.
Senge, P. M. The fifth discipline: The art & practice of the learning organization. Revised and updated with 100 new pages. Currency Doubleday, New York, NY, 2006.
Senge, P. M. and R. M. Fulmer. “Simulations, systems thinking and anticipatory learning.” In Journal of Management Development vol. 12, no. 6 (1993):.21-33.
Senge, P. M. and C. Lannon. “Managerial microworlds.” In Technology Review Vol. 93, No. 5 (1997): 62-68.
Shaffer, D. W. and M. Resnick. “’Thick’ authenticity: New media and authentic learning.”, In Journal of Interactive Learning Research, vol. 10, no. 2 (1999): 195-215.
Stewart, C. J. and W.B. Cash. Interviewing: Principles and practices. McGraw-Hill, Boston, MA, 2008.
Suits, B. The grasshopper: Games, life and utopia. University of Toronto Press, Toronto, CA, 1978.
Tsuchiya, T. and S. Tsuchiya. “The unique contribution of gaming/simulation: Towards establishment of the discipline,” in D. Saunders and J. Severn (eds.), The international simulation & gaming research yearbook: Simulations & games for strategy and policy planning, pp. 46-57. Kogan Page, London, U.K., 1999.
Van der Heijden, K. Scenarios: The art of strategic conversation. Second edition. Wiley, Chichester, U.K., 2005.
Villegas, J. “Simulation supported industrial training: A method for increasing the competence of people in companies”, in P. Saunders and B. Cox (eds.), The international simulation and gaming yearbook, volume 5: Research into simulations in education, pp. 144-164. Kogan Page, London, U.K., 1997.
Wardrip-Fruin, N. Expressive processing: Digital fictions, computer games and software studies. The MIT Press, Cambridge, MA, 2009.
Warmelink, H. “Towards a playful organization ideal-type: Values of a playful organizational culture.” In Proceedings of DiGRA 2011 conference: Think Design Play, Hilversum, The Netherlands, September 14-17, 2011.
Whitton, N. Learning with digital games: A practical guide to engaging students in higher education. New York, NY: Routledge, New York, 2009.
Transactions of the Digital Games Research Association Vol 1 No 2, ISSN 2328-9422, March 2014
© The text of this work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 Generic License (http://creativecommons.org/licenses/by-nc-nd/2.5/)
IMAGES: All images appearing in this work are propertyof the respective copyright owners,and are not released into the Creative Commons. The respective owners reserve all rights.
 
 
Published by the Digital Games Research Association.