Game Design in an Internet of Things
Abstract
Whilst no consensus yet exists on how the Internet of Things will be realised, a global infrastructure of networked physical objects that are readable, recognizable, locatable, addressable and controllable is undoubtedly a compelling vision. Although many implementations of the Internet of Things have presented these objects in a largely ambient sensing role, or providing some form of remote access/control, in this paper we consider the emerging convergence between games and the Internet of Things. This can be seen in a growing number of games that use objects as physical game pieces to enhance the players’ interaction with virtual games. These hybrid physical/digital objects present game designers with number of interesting challenges as they i) blur the boundaries between toys and games; ii) provide opportunities for freeform physical play outside the virtual game; and iii) create new requirements for interaction design, in that they utilise design techniques from both product design and computer interface design. Whilst in the past the manufacturing costs of such game objects would preclude their use within games from small independent games developers, the advent of low cost 3D printing and open software and hardware platforms, which are the enablers of the Internet of Things, means this is no longer the case. However, in order to maximise this opportunity game designers will need to develop new approaches to the design of their games and in this paper we highlight the design sensibilities required if they are to combine the digital and physical affordances within the design of such objects to produce good player experiences.
Keywords
Games, Internet of Things, Interaction Design, Phygital, Affordances
Introduction
There is little doubt that the video game industry is in a period of massive upheaval and despite the recent revisions of all the major consoles there have been a great many media reports predicting the ‘death of console gaming’ (Cousins 2013). This pessimistic view is primarily fuelled by significant reduction in console games sales over recent years and comes at a time when the popularity of social and mobile games has risen exponentially (Cousins 2013). Console games developers are arguably becoming risk averse and primarily concentrating on sequels rather than new titles (Czarnota 2013). This relates to the fact that their development costs are so much higher compared to developers of mobile, PC and social games who are able to be much more creative and speculative in their game designs as their financial risks are much lower. The tools associated with the development of mobile, PC and social games are also much more accessible to those who may be interested in a broader range of interactive entertainment such as experience designers and digital artists; some of whom are beginning to explore much more challenging themes within games such as sexual orientation (Anthropy 2012). These factors mean that there is a very real possibility that disruptive innovation (Christensen 1997) could occur within the console games industry with the emergence of cheaper console alternatives that double up as set-top boxes for streaming television and film, or ones based on advances in 3D printing, open-source operating systems, open hardware, and improved connectivity which could radically change the market.
Many of the enabling technologies that are fuelling this potential disruption are also at the heart of the so-called Internet of Things (IoT) which also has the potential to create a platform to explore innovative interactive entertainment. Although a clear consensus has yet to be established on how to realise the IoT, a global infrastructure of networked physical objects (Kortuem, Kawsar, Fitton, and Sundramoorthy 2010) or things that are readable, recognizable, locatable, addressable, and/or controllable via the Internet (NIC 2008) is undoubtedly compelling for a range of industry sectors including games and toys. While many implementations of IoT have presented the objects in a largely ambient sensing role or as some form of remote access or remote control, arguably the most successful implementation yet seen is the game Skylanders: Spyro’s Adventure from Activision which places the object (thing) at the heart of the activity by using it to control the characters within the game (Coulton 2012). Skylanders is a Role Playing Game (RPG) in which the characters are actually Radio Frequency Identification (RFID)/Near Field Communications (NFC) enabled physical game pieces that are used to swap characters in and out of the virtual game using the ‘portal’ (RFID/NFC reader which connects to the console via USB or Bluetooth). One of the other unique aspects of the game pieces is that a character’s type, name and abilities are modified through game play and stored on the physical game piece rather than on the console. This focus on physical/digital (phygital1) things produces a number of very interesting effects such as: blurring the boundary between toys and games, expanding existing modes of game play to the physical world, providing the opportunity for physical play outside the game, creation of innovative phygital interfaces, and a novel business model around the figures (Coulton 2012). Disney Infinity (infinity.disney.com) now offers a similar concept to Skylanders with NFC collectable figurines synchronised to game play using an NFC enabled base, but they have added a free form, ‘sandbox’ mode of gameplay within the video game. While Skylanders and Disney Infinity are arguably the most successful examples of IoT in terms of games there are also a number of ‘app toys’ starting to appear such as LEGOs, ‘Life of George’ (george.lego.com), Disney’s App Mates (www.appmatestoys.com) and the YetYet from Totoya Creatures (www.totoyacreatures.com). Except for these few examples, toys and games are two developer communities that largely operate independently, yet this novel interplay between toys and games facilitated by IoT could bring them together to produce new and mutually beneficial opportunities in terms of both games, and new economic models (Ng 2012). Whilst research is starting to emerge to describe the hybrid games/play that these objects facilitate (Kultima, Tyni, and Mäyrä 2013) in this paper we are concerned with the factors that game designers must consider in relation to the interaction modalities available in the design of these objects for their inclusion within virtual games.
Physical Game Objects
The game objects we are considering here can also be viewed as a form of Tangible User Interface (TUI), where TUIs can be defined as providing a physical form of digital information and facilitates the direct manipulation of the associated bits (Ishii 2008). However, as these game objects are generally located within the game space on a screen, they could also be considered a form of Augmented Virtuality (AV) in that they conform to its general definition of physical objects that are dynamically integrated into, and can interact with, the virtual world in real-time (Drascic and Milgram 1996). Although it is not wholly compliant with Drascic and Milgram’s original definition which related to the mixing of visual images from the real object within the virtual world on a display (Drascic and Milgram 1996), as demonstrated by projects such as Augurscope (Schnädelbach, Koleva, Flintham, Fraser, Izadi,S., Chandler, Foster, Benford, Greenhalgh, and Rodden 2002), the physical location of the object on the screen presents a comparable effect. In terms of AV games there are few specific examples relating to games as the majority of research relates to Augmented Reality (AR) (Bernardes Jr, Tori, Nakamura, Calife, and Tomoyose 2008) and Table Top AR in particular (Kato, Billinghurst, Poupyrev, Imamoto, Tachibana 2000). Indeed many of the desired features for TUI design in AR (Kato et al 2000) are applicable to AV. In particular, the features of:
- The form of objects should encourage and support spatial manipulation (Kato et al 2000);
- Object affordances should match the physical constraints of the object to the requirements of the task (Kato et al 2000).
An additional consideration for game designers creating objects is that some of the interactions made possible with such objects may be representative of what Jesper Juul (Juul 2009) describes as mimetic interfaces in relation to casual games like Wii Sport and Guitar Hero. These games require players to perform actions that closely resemble the physical activity required by the avatar on the screen. These interfaces make games easier for players less familiar with the more ‘traditional’ configuration of game controllers utilizing buttons and joysticks to pick up and play casual games. As Juul describes, the requirement for players to strum the guitar controller in Guitar Hero requires no explanation as most would be familiar with the image of a musician playing a guitar. Whilst this prior knowledge may make learning the game easier, it also means that if the game challenges these expectations it is likely to put players off as there is a ‘metaphor mismatch’ (Hinske and Langheinrich 2004) between the game and controller.
Whilst descriptions of mimetic interfaces might suggest they are the same as Natural User Interfaces (NUI) they do not hold to the definition offered by Daniel Wigdor and Dennis Wixon (Wigdor and Wixon 2011) “natural refers to the user’s behaviour and feeling during the experience rather than being the product of some organic process” and, indeed, suggest a natural experience “is NOT best achieved through mimicry”. This description leads us to suggest that mimetic interfaces should be considered as a subset of the broader area of NUI.
Another interesting component within this classification discussion is the emergence of game interfaces based on real world artefacts such as in the game Rocksmith from Ubisoft. This game comes with a cable that allows players to plug their own guitar into an Xbox 360, PlayStation3 or PC. In addition to the expected song tutor activity the system also provides a ‘Guitarcade’ of mini games that are designed to practice specific techniques. For example there is a ‘Space Invaders’ themed activity to help improve fret placement. In some ways such games invert the NUI paradigm as, arguably, the natural interface for many of the mini games would rightly be a traditional game controller.
Whilst the classification of phygital game objects is complex what is clear from the previous discussion is that game designers must draw from a wide range of disciplines that consider interaction modalities if they are to ensure that their design of such objects enables players to effectively operate in real and virtual space. In the following section we present a discussion around the design of such objects that draws understandings from both product design and computer based interaction design through which game designers might address this challenge.
Interacting with Objects
In terms of general interaction design there are many ways that it has been presented but in this research we will utilize the sketch by Bill Verplank (Verplank 2009) recreated in Figure 1 as the basis for discussion.
Figure 1: Interaction Design as defined by Verplank.
Verplank frames interaction design as a designer answering the three questions: How do you do? How do you feel? How do you know? (Verplank 2009).
If we start with ‘doing’ then Verplank distils this to a consideration of whether the interaction is continuous or discrete (represented by the handle and button respectively within the sketch) and while both of these are applicable, and available, in the design of phygital game objects we must extend this also to consider that they are likely to be implemented as a combination of real or virtual interactions. Game designers must, therefore, consider carefully how such interactions are used and combined so that it ‘makes sense’ to the player. For example, in Skylanders adding a physical game character to the portal causes both a color change on the physical portal and the character to materialize on the screen which is important when considering the focus of the player as we shall discuss later.
In terms of how the users ‘feels’, while he considers this in relation to the users physical senses Verplank presents it in relation to the medium through which the interaction is presented to the user. This draws upon the work of Marshal McLuhan and his definition of a media as “any technology that ... creates extensions of the human body and senses” (McLuhan, M., McLuhan, E., and Zingrone 1996). McLuhan categorized media in terms of ‘hot’ and ‘cool’, whereby a hot media, such as print, is one that dominates one particular sense absorbing our attention and leaving little room for participation, while a cool media (sometimes described as fuzzy) that engages across our senses leaves space for participation (McLuhan, Fiore, and Agel 1967). When McLuhan wrote this original definition he considered television as a cool media although nowadays it would more likely be considered towards the hot end of the spectrum when compared to games and this illustrates that these are, as McLuhan intended, not static definitions but dynamic concepts. Arguably the Internet doesn't really fit into McLuhan's definition as it both encourages participation but it also commands our attention and often dominates our senses. As the phygital game objects under consideration would be expected to expand the interaction across many the player’s senses they are likely to appear at the cool end of the spectrum. In relation to the overall design of phygital objects, cool features are likely to attract and engage whereas hot features can be used to provide very specific activities, such as help, and designers would normally be expected to consider a combination of these within the design. For example, an interactive character toy could be used to voice the hints and tips of a game, specific combinations of interactions with the toy could unlock special powers, or using the toy in a certain physical location might unlock special locations within the game.
The final question of ‘how we know’ is illustrated by Verplank as paths and maps and he argues that often the best overall interaction design utilizes a combination of both (Verplank 2009). Note this categorization is derived from Kevin Lynch’s work in his 1960 book, ‘The Image of the City’, which studied how people developed an understanding of the layout of a city. Paths are primarily step–by-step instructions that guide the user through an interaction and it is generally regarded as the easiest form of interaction as the user only needs to know one step at a time (path knowledge). Paths are commonly used in situations that require immediate action by users who are likely to be experiencing the required interaction for the first time (i.e. emergency door release) or in game tutorials. Maps essentially represent knowledge obtained through the interface affordances that help users construct coherent mental models from which new tasks and uses can be inferred. The knowledge maps build on affordances through the interactions performed by the users in multiple scenarios using objects and systems that provide similar interactions. Thus the interaction design of phygital objects for games requires games designers to not only fully understand the virtual aspects the affordances they are perhaps used to, but also to extend these to include the affordances we associate with physical objects to ensure their overall game design does not cause confusion for the player. It is therefore important to consider the concept of affordance in more detail and in the following section we present such a discussion.
Affordances of Phygital Objects
The original concept of affordance was conceived by Ecological Psychologist James J Gibson (Gibson 1977) to define the actionable properties between the world and a person. He uses the example of a flat surface that affords ‘sitting on’ whereas a ‘pointy’ one would not. The important aspect of this is that an affordance of an object exists whether it is acted upon or not. The concept of affordance was most notably developed for design by Donald Norman extended from what he regarded as Gibson’s ‘real affordances’ to include ‘perceived’ affordances (Norman 2002). He argued that affordances “play very different roles in physical products than they do in the world of screen-based products” (Norman 1999) and unlike Gibson he also believed that affordances could be dependent on the experience, knowledge, or culture of the users (Soegaard 2003), for example, in Japan you would expect to read comics right to left and front to back. Further, Norman uses this as a means of distinguishing between the properties of an object that are controllable by a designer and those that are fixed. In the case of the design of real objects, both the actual and perceived affordances are controllable, whereas for screen-based interaction generally only the perceived affordances are under the control of the designer, “as the computer system comes with built-in physical affordances” (Norman 1999). For example, all computer screens support the affordance of touch whether they are touch sensitive (i.e. respond to the touch) or not. If we add a graphical target on a touch sensitive screen we are providing visual feedback that advertises the affordance that touch interaction is supported, and this creates the perceived affordance of the user. This is an important point as it emphasises that while game designers might be used to incorporating the established perceived affordances within the built-in affordances of gaming systems and their controller, computers, or phones, they are unlikely to have contributed to the built-in affordances of the physical systems as these have traditionally been the preserve of product designers.
Bill Gaver stressed further the importance of affordances for design when stating “affordances exist whether or not they are perceived, but it is because they are inherently about important properties that they ‘need’ to be perceived” (Gaver 1991). Gaver also introduced with the concept of ‘sequential affordances’ which describes how in many situations a users action on a perceptible affordance then leads to new information relating to the next affordance in the sequence. In relation to games design this concept of sequential affordance can be considered alongside procedural rhetoric, for as the meaning of the game is communicated through participation (Bogost 2008), so are the affordances of its interaction design. To help designers consider the ease of use of the object/system they are designing, Gaver developed a figure (Gaver 1991), which considers whether perceptual information is available to user which we have recreated as Figure 2.
Figure 2: Affordance relative to Perceptual Information adapted from Gaver (1991).
In this diagram a false affordance exists when there is no action possibility although the perceptual information implies that there is, however, others would argue that in such a case it is not that the affordance is incorrect but rather it is that the perceptual information is incorrect (McGrenere and Hon 2000). This would occur if a game designer created a physical object with a feature that looks like a button that could nott actually be physically pressed. A correct rejection occurs when there is no affordance and no perceptual information to specify it whereas a hidden affordance exists when the affordance is present but the specifying perceptual information is not. Dan Saffer suggests hidden affordances may actually be regarded as ‘discoverable’ (Saffer 2013) in recognition that designers may deliberately allow them to be revealed through accidental use or deliberate exploration. This is similar to the practice of game designers leaving hidden elements, or ‘easter eggs’, within their games that are discovered by accident, this practice hints at a possible interesting opportunity yet to be applied to game objects. For example a poseable toy relating to an in-game avatar could unlock unexpected abilities when physically manipulated into a certain position by the player.
Whilst recognising Gaver’s contribution, McGrenere and Hon state that in order to “use affordances to evaluate and improve design, it is useful to think of the degree of an affordance” and to “regard affordances as binary is to oversimplify them” (McGrenere and Hon 2000). To illustrate this they created Figure 3, which presents a two-dimensional space where one dimension describes the ease of which an affordance can be undertaken and the other dimension describes the clarity of the information that describes the affordance. Each dimension is a continuum and the goal for the designer is to first determine the necessary affordances and then to maximize each of these dimensions (McGrenere and Hon 2000). We suggest that phygital objects should be considered as dynamically traversing this space and designers should endeavour to unite approaches to affordances from both the physical and digital domains dynamically throughout the game. This means that game designers cannot consider real or perceived affordances separately or that these affordances are fixed and must address both of these throughout the design and testing of a game.
Figure 3: Improving Affordance Design adapted from McGrenere and Hon (2000).
Alongside affordances, Norman also defined ‘cultural conventions’ (Norman 2002) which also serve to constrain user interaction and derive from users’ conventional interpretations of how they should interact with a particular artefact (in our case the game object). Norman further subdivides these cultural conventions into physical, logical, and cultural constraints. Physical constraints are related to the artefact, for example, in the case of Skylanders and Disney Infinity the figures can only interact with the virtual games when in contact with the portal or base and the NFC reader has a detection range of approximately 2 centimetres. Logical constraints are when users make judgements to deduce the nature of the interaction. For instance in Call of Duty as our avatar represents a human soldier we assume it can perform the physical actions we ourselves might perform. Cultural constraints are conventions shared by a cultural group. There are many shared cultural conventions in games that are reflected in things such as house rules2 but more noticeably in relation to game genre. For example we expect casual games to be short with very simple interactions. The challenge for game designers is in understanding which conventions they might inherit when designing a game object of a particular form.
The final section of our discussion considers the game space within which the interaction takes place. In his book The Casual Revolution Jesper Juul (Juul 2009) divided game space into: player space, screen space, and 3D space in order to highlight that in many casual games, such as those using Wii Sports, the player space plays a much more significant role than many more traditional console games (Juul 2009) especially to the audience. This division of game space provides a useful way for designers to consider where the focus of attention is for the player might be when interacting game objects in screen-based scenarios. This consideration will allow designers to clarify: in which space and how the interaction takes place; and in which space and how feedback on that interaction is presented to the player. Figure 4 provides two such interaction scenarios and unlike the casual games explored by Juul the question whether the games are either single or multiplayer does not dominate the discussion as it is anticipated both scenarios would support either type of game are more concerned which space is the focus of attention for the player/s currently playing the game rather than others who may be watching the action.
- In this scenario the screen of the tablet provides a surface through which the physical game objects interact with a virtual game that could be represented in either 2D or 3D space.
- In this scenario movements of the physical object are transferred to the screen via a wired/wireless link and as with the previous case the virtual game can be represented as either 2D or 3D space.
Figure 4: Example Game/Interaction Spaces for Game Objects used with Screens.
It is important to note that although these scenarios are indicative of many of the current implementations of games incorporating phygital objects they are not the only possibilities and designers are free to explore the configuration of such interaction spaces. For example, scenario (a) allows relatively limited interaction by the player as the physical object needs to be in contact with the screen throughout play, although this could easily be combined with a wireless link from scenario (b) to allow much more information to be transmitted. Note that in this paper we have not considered either the communication technology the game objects might utilise (i.e. Bluetooth LE, WiFi, ZigBee, etc) or which architecture they might utilise to support the development of services (i.e. web service devices, virtual cloud devices, and peer to peer devices) which would undoubtedly affect the operation of the object.
Putting these technical considerations to one side if we return to our scenarios and consider the likely focus of attention of the player in these situations we can see that whilst scenario (a) provides a space where the player can switch focus between the physical object and the virtual game relatively easily, in scenario (b) their focus may change considerably and it is much more likely that feedback from the game on the screen could be easily missed. Whatever the nature of the overall game space created, designers need to consider carefully how the player will interact within each of the sub-spaces during the course of the game and in the case of toys the virtual game may not be present at all within some play activities. Therefore all possibilities need to be given serious consideration when creating the object.
Having provided some general guidance of factors that must be considered for the design of phygital game objects in the subsequent section we will provide practical examples of how these may be put into practice.
Interacting with Phygital Objects
In this section we explore the creation of game objects with examples that incorporate specially designed 3D game objects and the customisation of a Makie doll (makie.me). Makies are 3D printed poseable action dolls whose faces and features are designed by the customer. A web interface and iPhone app allows users to choose what the Makie looks like by adjusting: the eyes, nose, ears, jaw, smile, hands, feet, hair, and clothes. The other valuable feature in relation to this project is that the head and torso of the Makie have been deliberately left hollow to allow owners to ‘hack’ their doll with electronics such as Lilypad Arduino (a specially designed slot is provided in the Makie head for this device).
In the following sections we consider the two scenarios previously defined in relation to the design of the objects and their interaction.
Direct Interaction of Phygital Objects with Touch Sensitive Screens
In this first section we consider scenario (a) by evaluating how phygital game objects can be created that exploit the underlying technology of multi-touch input devices, such as the iPad, rather than by the creation of of new gestural interactions that are then associated with the objects (Buxton, Hill, and Rowley 1985). Where early touch sensitive phones and tablets employed resistive touch screens, which required physical pressure to be applied to create a touch event, capacitive touch screens, seen on the vast majority of current phones and tablets, exploit the electrical properties of the human body through mutual or self capacitance. Therefore any phygital game objects produced must allow the conductance of the player fingers to pass through them and onto the surface of the device in order that they register an interaction. An illustration of how simply this concept can be achieved is shown in Figure 5a whereby a Makie was fitted with clothes made from conductive cloth allowing the charge from the players fingers to be transferred to the surface of the iPad running an app that presented the surface as if it was water.
Figure 5: a) Makie fitted with clothes made from conductive cloth interacting with iPad; b) Physical game object produced from conductive material that acts as an object within a virtual game.
In Figure 5b we present another example that extends this approach by creating the game objects using conductive material. In particular, we consider the objects for use within a iPad game we have dubbed Pong+ (Burnett, Coulton, and Lewis 2012). The game is played with the iPad horizontal and using physical game objects as ‘mallets’, as you would in the physical game of Air Hockey, but with a virtual game ‘puck’ as might be seen in the Atari classic Pong. The game mallets were 3D printed, painted with conductive paint (//www.bareconductive.com), and have conductive cloth fitted to their base to mimic the touch points that might otherwise be produced by multiple fingers. Placing this particular implementation into context it can be regarded as a fully embodied dynamic spatial tangible interface (Ullmer and Ishii 2000). The research associated with this game not only highlighted the benefits of the physical affordances of the mallets over an alternate purely virtual implementation it also highlighted that the number, size and spacing of the touch points that can be tracked simultaneously within such a game is highly device dependent (Burnett et al 2012). This is because no standard exists that specifies these parameters for either phones or tablets which means that designers may have to invest considerable time evaluating the capabilities of many devices if they are aiming for cross platform compatibility of their game objects.
This approach creates passive objects and thus differs from the work presented by Yu et al who proposed active objects (i.e. ones containing there own power supply and electronics) for tangible interfaces (Yu, Chan, Lau, Tsai, S.S., Hsiao, I.C., Tsai, D.J., Hsiao, F.I., Cheng, Chen, Huang, and Hung 2011). The scheme of Yu et al is to electrically modulate touch points at frequencies they suggest are beyond what is likely to be produced by human fingers but is still detectable by the device (Yu et al 2011). They offer this as a solution that overcomes the limitation of passive solutions that can only support a limited range of information within a single object. However, while they present no users studies they do state that because of computational speed required for the device to detect this frequency modulation “limits the use to static objects” (Yu et al 2011) and as such would drastically limit its applicability for many games genres and suggests it would be better suited to simple app toys (Hinske et al 2008, Mandryk et al 2000).
Interaction with Phygital Objects
In this section we explore the scenario (b) in which players interact directly with the object but not physically with the screen. To illustrate this concept we have taken advantage of the Makie as a digitally playful object in that it supports hacking to allow it to detect other objects. In Figure 6 we present a Makie that has a Near Field Communication (NFC) reader fitted within its interior and which is based on a Lilypad Arduino coupled to NFC module that supported the development of a bespoke antenna which could be fitted into the torso of the Makie. Turning the Makie into an NFC reader means that any associated Makie virtual world can be extended with any number of additional items, similar to the way that the readers of Skylanders and Disney Infinity allow the characters to be swapped within the games. The interaction of these new objects with the Makie can then be transferred to a game running on a phone, tablet or console wirelessly using either WiFi or in this case low power Bluetooth. It is worth noting that as an alternative for, or in addition to, the NFC reader, other sensors such as accelerometers, magnetometers or a gyroscope could be added to detect specific movements of the Makie.
Figure 6: Makie augmented with NFC reader.
Figure 7: Virtual Space Invaders controlled by physical base and fire button.
To illustrate how other alternative scenarios might emerge we have combined scenarios (a) and (b) to create a novel implementation of the old arcade classic Space Invaders as shown in Figure 7. In this game a 3D printed object is used to control the position of the players ‘base’ and then a micro switch provides the ‘fire’ control via a low power Bluetooth link. At present the switch is implemented using a separate RFduino (www.rfduino.com) board and switch shield although these could be redesigned to fit within a single game object.
Phygital Object Design Spiral
While we envisage the phygital game objects would follow the design spiral shown in Figure 8, in which an, idea is explored through different alternative solutions which are prototyped and tested to obtain a suitable final solution we recognise that a key aspect that will faced by many small independent game designers creating phygital objects is how to extend the spiral beyond the maker culture, from which many such objects emerge, into a fully fledged production cycle. Whilst open source hardware and low cost 3D printers facilitate the easy prototyping of game objects they would generally be considered as too expensive for large scale production. In order to reduce costs designers are generally required to shift production towards injection moulding and bespoke electronics. This is an important consideration for the designers of such objects as it will likely require a large injection of capital to facilitate this type of production in specialised factories. However, as we are seeing many examples of prototype devices sucessfully gaining such capital through crowdfunding sites such as Kickstarter (//www.kickstarter.com) this does not represent an insermountable barrier.
Figure 8: IoT Object Design Cycle.
Conclusion
We are undoubtedly undergoing a period of change within the video games industry and seeing a shift from the dominance of the major consoles towards devices and systems that arguably support more experimental game design. Whilst the growing dominance of phones and tablets in the computing world is a significant factor, it is further fuelled by advances in 3D printing, open-source operating systems, open hardware, and improved connectivity.
Although the first examples of IoT game objects are principally emerging from the major game publishers and often relates to an established brand, the low barriers to entry of these technologies means they are well within the reach of small independent game developers. However, creating physical objects requires new considerations to be adopted within the overall interaction design, as they need to combine both digital and physical interactions within a unified player experience.
Designers therefore, are not only confronted with simply discrete and continuous interactions they must consider whether they are implemented either in a real or virtual manner. How they are combined is likely to require a number of prototypes if the final outcome is to ‘make sense’ to the player.
By considering the interaction through the terms of hot and cold media which relates to the medium through which they are represented aids designers in considering interactions that go beyond the current dominance of vision and touch. The concepts of hot and cold are also useful to consider how information or feedback is presented to players.
The notion maps and paths helps us consider the knowledge obtained by players through the interface affordances, which helps them construct coherent mental models from which new tasks and uses can be inferred. However, as was discussed, although the affordances of physical objects may be readily perceived through their design, in the case of virtual systems the perceptual information which reveals the affordance needs to be dynamically attributed. Such information is not simply either perceived or not perceived it exists on a continuum which relates to a player’s ability to undertake a particular affordance which in turn is affected by dynamically changing cultural conventions associated with certain affordances.
The subdivision of the game space into player space, game space, and 3D space is a useful way for designers to consider where the focus of the player’s attention may be and where, and how feedback for a particular interaction is presented to the player. We would acknowledge that this subdivision reflects the current dominance of screen-based interactions and therefore new models may need to be developed that reflect alternative game spaces and ones in which the object may be the dominant entity.
In our examples we have considered the two scenarios that represent the main approaches to facilitating the interaction between a physical object and screen based virtual games. The first scenario utilises the screen of a tablet as surface through which the physical objects interact with a virtual game in either 2D or 3D space. The primary approaches for creating such objects are either passive or active with a greater range of information being offered by active objects. However, passive objects are better suited for dynamic and continuous interaction with the surface of a phone or tablet. The second scenario is whereby movements of the physical object, or interactions with the physical object, are transferred to the screen via a wired/wireless link. Whilst this approach enables the inclusion of a greater number of objects, and potentially the detection of more complex interactions, it comes at the cost of greater complexity of the phygital object. The final example shows that these two scenarios are merely two of many possibilities available that game designers are free to develop these hybrid interactions in ways they feel are most appropriate to the game.
Within this research we have presented a design spiral for phygital objects that illustrates that while the design process may incorporate aspects common to many forms of design the movement from techniques associated with the maker community to ones more suited to large scale production is a particular challenge for manufacturers.
Finally we reiterate that this paper is speculating on a future in which creating game objects that link the physical and the digital presents an exciting and practical opportunity for game designers. However, such objects require interaction design approaches that not only utilise understandings from product design and graphical user interface but also how they might effectively be combined dynamically. In this research we have highlighted a range of design sensibilities that game designers will need to adopt if they are to create such games.
Acknowledgments
The research presented in this paper has been made possible primarily through the Arts and Humanities Research Council (AHRC) project The Creative Exchange to whom we express our gratitude. We would also like to thank Natasha Carolan, ex Head of Atoms, at Makielab for providing us with our Makie ‘Toad’ to support this research.
Endnotes
1. We are using the term phygital as a way of emphasizing that these are a class of objects that have not simply had some digital functionality embedded within then but are connected devices whose functionality and operation is designed to exist simultaneously in both virtual and physical space.
2. House rules are rules not formally part of the official rules of the game but are devised by the players themselves and can range from small additions to whole scale deviations in game play.
Bibliography
Anthropy, A. Rise of the Videogame Zinesters: How Freaks, Normals, Amateurs, Artists, Dreamers, Drop-outs, Queers, Housewives, and People Like You Are Taking Back an Art Form, Seven Stories Press, 2012.
Bernardes Jr, J. L., Tori, R., Nakamura, R., Calife, D., and Tomoyose, A. “Augmented Reality Games”. In: Extending Experiences: Structure, analysis and design of computer game player experience. Lapland University Press, 2008.
Bogost, I. Unit Operations: an approach to videogame criticism, 49-73, The MIT Press, 2008.
Burnett, D., Coulton, P., and Lewis, A. “Providing both physical and perceived affordances using physical games pieces on touch based tablets”. In Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System (IE '12). ACM, New York, NY, USA, , Article 8 , 7 pages, 2012.
Buxton, W., Hill, R., and Rowley, P. “Issues and Techniques in Touch-Sensitive Tablet Input”. In Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques SIGGRAPH ’85. ACM Press, New York, NY, 215-224, 1985.
Coulton, P. “SKYLANDERS: Near Field in Your Living Room Now”, In: Ubiquity: The Journal of Pervasive Media. pp. 136-138, 2012.
Cousins, B., “Weapons of Mass Disruption #3: How and Why Consoles Will Die”, Kotaku 1/7/13, http://kotaku.com/5973498/weapons-of-mass-disruption-3-how-and-why-consoles-will-die, 2013, Accessed 14 October 2013.
Christensen, Clayton M. The innovator's dilemma: when new technologies cause great firms to fail, Boston, Massachusetts, USA: Harvard Business School Press, 1997.
Czarnota, J. “Developers – your role in the coming war, and the ways you can benefit from new modes of game production”. Gamasutra 10/4/13, http://gamasutra.com/blogs/JedrzejCzarnota/20131004/201690/Developers__your_role_in_the_coming_war_and_the_ways_you_can_benefit_from_new_modes_of_game_production.php?print=1, 2013, Accessed 14 October 2013.
Drascic, D., and Milgram, P. “Perceptual Issues in Augmented Reality”, in: S.S. Fisher, J.O. Merritt and M.T. Bolas (Eds.), Stereoscopic Displays and Virtual Reality Systems III, 123-134, SPIE Press, 1996.
Gaver, W.W. “ Technology affordances”. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91), Scott P. Robertson, Gary M. Olson, and Judith S. Olson (Eds.). ACM, New York, NY, USA, 79-84, 1991.
Gibson, J. J. “The Theory of Affordances”. In R. E. Shaw & J. Bransford (eds.), Perceiving, Acting, and Knowing. Lawrence Erlbaum Associates, Hillsdale, NJ, 1977.
Hinske, S., and Langheinrich, M. “RFIDice – Augmenting Tabletop Dice with RFID”. In: Journal of Virtual Reality and Broadcasting, vol. 5, no. 4, 2008.
Ishii, H. “Tangible bits: beyond pixels”. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (TEI '08). ACM, New York, NY, USA, 2008.
Juul, J. (2009). A Casual Revolution: Reinventing Video Games and Their Players. Cambridge: MIT Press, 2009.
Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K., and Tachibana, K. “Virtual object manipulation on a table-top AR environment”. In Proceedings of Int. Symp. on Augmented Reality, 2000.
Kortuem, G., Kawsar, F., Fitton, D., and Sundramoorthy, V. “Smart Objects as Building Blocks for the Internet of Things”, IEEE Internet Computing, vol. 14, no. 1: 44-51, 2010.
Kultima, A., Tyni, H., Mäyrä, F. “Dimensions of Hybrid in Playful Products”, Proceedings of Academic MindTrek Conference 2013 "Making Sense of Converging Media", Tampere, Finland, 2013.
Norman, D. A. Affordance, conventions, and design. Interactions 6, 3, 38-43, 1999.
Norman, D. A. The Design of Everyday Things. Basic Books, 2002.
Mandryk, R.L., and Maranan, D.S. “False prophets: exploring hybrid board/video games”. In CHI '02 extended abstracts on Human factors in computing systems. ACM, New York, NY, USA, 640-641, 2002.
McGrenere, J. and Ho, W. “Affordances: clarifying and evolving a concept”, Proceedings of Graphic Interface 2000 Conference, Montreal, Canada , 1–8, 2000.
McLuhan, M., Fiore, Q., & Agel, J. The medium is the massage. Bantam Books, 1967.
McLuhan, M., Eric McLuhan, E., and Frank Zingrone. Essential McLuhan. Basic Books; 1st edition, 1996.
National Intelligence Council. “Disruptive Civil Technologies: Six Technologies With Potential Impacts on US Interests out to 2025”. http://www.fas.org/irp/nic/disruptive.pdf, 2008, Accessed 9/10/2013.
Ng, I. “Value & Worth: Creating New Markets in the Digital Economy”. Innovorsa Press. Report of the TSB-funded Preparatory Studies on IoT Convergence, October 2012: tiny.cc/iotprepstudies. Accessed 9/10/2013.
Saffer, D. Microinteractions: Designing with Details, O'Reilly Media, 2013.
Schnädelbach, H., Koleva, B., Flintham, M., Fraser, M., Izadi,S., Chandler, P., Foster, M., Benford, S., Greenhalgh, C., and Rodden, T. “The augurscope: a mixed reality interface for outdoors”. In Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves (CHI '02). ACM, New York, NY, USA, 9-16, 2002.
Soegaard, Mads “Affordances”. http://www.interaction-design.org/encyclopedia/affordances.html, Accessed 9/10/2013.
Ullmer, B. and Ishii, H. “Emerging frameworks for tangible user interfaces”, IBM Systems Journal, v.39 n.3-4, 915-931, July 2000.
Verplank, W. “Interaction Design Sketchbook by Frameworks for designing interactive products and systems”. http://ccrma.stanford.edu/courses/250a/lectures/IDSketchbok.pdf Accessed 9/10/2013.
Wigdor, D. and Wixon, D. Brave NUI World: Designing Natural User Interfaces for Touch and Gesture, Morgan Kaufmann; 1st edition, 2011.
Yu, N.H., Chan, L.W., Lau, S.Y. , Tsai, S.S., Hsiao, I.C., Tsai, D.J., Hsiao, F.I., Cheng, L.P., Chen, M., Huang, P., and Hung, Y.P. “TUIC: enabling tangible interaction on capacitive multi-touch displays”. In Proceedings of the annual conference on Human factors in computing systems 2011. ACM, New York, NY, USA, 2995-3004, 2001.
Transactions of the Digital Games Research Association Vol 1 No 3, ISSN 2328-9422, October 2014
© The text of this work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 Generic License (http://creativecommons.org/licenses/by-nc-nd/2.5/)
IMAGES: All images appearing in this work are propertyof the respective copyright owners, and are not released into the Creative Commons. The respective owners reserve all rights.