Gamification as an assessment tool: why play the game?

by Ella Hafermalz and Kristine Dery

We asked those attending DISRUPT.SYDNEY on Friday (26 Sep 2014) to pose questions relating to gamification ahead of our gamification debate. In a nutshell our attendees wanted to know, what’s beyond the hype? i.e. Is gamification just a new buzzword around digital? Is there anything different here beyond the delivery? Does gamification really add value?

In this post we move beyond all the bells and whistles and try to open up the discussion about gamification warts and all. To do that we have focussed our attention on gamification as a useful assessment tool in the context of e-talent management. We discuss here how gamification might be a useful tool for engaging talent in recruitment, training, and skills assessment.
 
Gamification refers to applying game elements and game thinking to traditionally non-game processes. These “games” are a growing addition to the e-talent management tool kit particularly now that more assessment tools are delivered on-line. Attracting the right candidates, assessing the best talent, and then recruiting them has traditionally proved challenging and expensive. Increasingly organisations are turning to gamification to deliver the elements of fun and competition that attract and retain the attention of their future talent.

L’Oreal, for example, have created the serious game Reveal to help attract and select top talent. After signing up to the game online the candidate navigates through various simulated departments of the cosmetics giant performing tasks and overcoming challenges. If the candidate performs well in a particular department they are advised to consider a career in that area. If performance is exceptional (placing you on the game’s leaderboard relative to other players) L’Oreal’s talent management team deliver further rewards with offers to engage in their recruitment process. This and other serious games and gamified processes that are designed to attract, assess, train, and retain talent are often developed on the basis of psychometric testing principles. Testing talent can be useful, but it’s important to first understand what the data generated from testing will be used for.

As we enter the Talent Decade, organisations will rely on their HR function to build ‘effective talent systems and measurement tools’ that will ‘support strategic business decisions and strengthen workforce capacity’ (Canadian Conference Board, 2014). This is where gamification has a potentially important role to play – in the intersection between talent systems that determine the desired characteristics and capabilities to meet strategic goals and the measurement tools that assess current and potential talent.

Reveal and other talent games like it are essentially tools for assessing talent more effectively. Gamification, therefore, is essentially a way of telling a story around data. When a candidate plays Reveal they are generating data. They receive data in the form of feedback, and data is collected to assess their performance. The game is the story that helps to order that data in a way that enables the employer (or potential employer) to understand more about the “player” and make more informed decisions based on information that they would otherwise find very difficult and expensive to attain. The “player” is experiencing the story of the game, which while engaging them is also providing them with insights into the organsiation enabling them to make decisions around whether they want to continue or opt out of the “game”.

As industry professionals start to experiment with gamification, it’s worth understanding more about assessment. Education specialists have a long tradition in examining assessment so perhaps it is worthwhile taking a look at some of their frameworks to get an idea of what lies beyond the “game”.

Education specialists focus on three main types of assessment  diagnostic, formative, and summative. Each type of assessment has a different purpose and is carried out at a different stage in a learning process or, in the case of talent management, a selection or development process.

Type of Assessment

Stage of Talent Management process

What is it used for?

Diagnostic

Attraction and enrolment in the selection process

To assess a candidate’s current capabilities and identify their potential for the role or the organization

Formative

Activities and tasks during the recruitment  process to enable the candidate to display their capabilities and learn

To give real-time feedback on performance to enable both the candidate and the organisation to understand more about each other and assess whether to continue

Summative

Sorting and ranking at the end of the recruitment process

To measure achievement and inform ranking decisions

Games such as Reveal are good examples of how to incorporate more than one type of assessment and to integrate these across the entire gamified recruitment and selection process. Reveal has a diagnostic element because it tells candidates where their aptitude lies. It is formative because candidates learn about the skills needed to succeed in the L’Oreal environment. It is also a summative assessment tool, because performance data is used to select and contact high achievers, who then go to the next ‘level’ of the recruitment process.

Not all gamified talent management applications need to be this sophisticated or comprehensive. Some gamified processes focus on one assessment type. For example, KPMG use their game as a formative process to provide training on the capabilities located within the organisation throughout the world, Deloitte are building leadership training apps using gamified strategies. These examples of gamification are designed to engage staff in the mechanics of games to stimulate learning processes through fun interactions and competitive missions. They provide a degree of feedback to the “players” to indicate the retention of new skills or knowledge, and they might also generate data to further manage the game itself. However, there are no major implications for the player at a summative level and results are typically not used to rank or rate the player in a way that has consequences for their job.  

Gamification can also be designed purely for summative assessment where there are winners and losers. This is the area that has occupied the recruitment space for some time. Typically on-line application systems where CV’s are uploaded and sorted according to key words fall into this category. Candidates search for unwritten rules to play the game to make it through the diagnostics where they are quickly assessed and ranked. There may be a few iterative loops along the way but essentially candidates are moving through a series of diagnostic/summative loops as the organisation attempts to sift talent. This process, while taking on gamified practices, is arguably problematic and is simply playing old games using new media. This is, we would argue, not creating new value. If, however, the possibilities of assessment processes are clearly understood then we begin to see new opportunities to generate better results can be harnessed through gamification.

Candidates in gamified processes (such as those implemented by L’Oreal, Google and others) are not simply sorted in a summative assessment process but are engaged through a more formative series of interactions that generate data to allow:

  1. the skills of the player (applicant) to be assessed
  2. the player to get an idea of the organisation and assess their own fit, and 
  3. feedback to be built in to make the assessment a learning process for both parties.

In this way we see gamification move beyond the hype to adding real value both during and after the assessment process in recruitment.

Ella Hafermalz is a PhD candidate in Business Information Systems at the University of Sydney and is a qualified teacher. Dr Kristine Dery is a Research Scientist with the MIT Sloan School of Management and has expertise in HR and digital technologies.    

Advertisements
Posted in Gamefication | Leave a comment

Waiting on the Riverbank: Intent Casting in Practice

by Kristine Dery, informed by interview with Ruben Martin    

Today I decided it was time to buy a new printer. This is not an activity that I want to spend too much time doing and yet it is an important purchase. I largely work from home, or cafes, or bookstores. It is an enviable way to work for the large part but it does require good support technology to maximise the upside without getting the downsides that come with minimal office support. The right printer, I have discovered, is critical. I know exactly what I want: a smallish unit, colour printing, scanning, copying, and wireless to print from any of my mobile devices. All I need is to be matched with the printer that meets my needs. Simple? Not so. Our Business-to-Consumer (B2C) retail model requires me, the customer, to do all the work. You see, the brand decides what to make, the retail network decides how it will be distributed, and the marketers decide what they will tell me about in the way they think that I want to hear it. This leaves me, the customer, to sift through all the messages of all the brands to find a printer that meets the needs that I have already determined. And apparently this is “customer focussed” retailing! Not so.

Doc Searls, alumnus fellow of the Berkman Centre for Internet & Society at Harvard, suggests that the capabilities of the new digital landscape will shift the power of the buying process to the consumer…a process he called Intent Casting. Rather than me having to search through the plethora of product to short-list my printer, I would simply lodge my intent to purchase on-line and the appropriate printer would find me. Suppliers of printers that met my set of criteria would compete for my business either by being first or the best matched respondent. The intent casting model suggests a dramatic shift not only for traditional business models in the distribution of products and services, but also for how companies are organised and communicate internally in order to make this shift. The B2C model is flipped on its head to become C2B and with it comes a new way of thinking about customer relationships.

We have already seen shifts in this direction as users engage with digital platforms and new business practices emerge. Two industries that have seen intent casting disruption are recruitment (e.g. Linkedin) and taxis (e.g. Uber). Traditional recruitment processes started with a company posting a job description and waiting to see who applied. Applicants would either be those in the market already for a new role or someone who found the job ad by chance and was persuaded to think about making a change. Linkedin provided a marketplace where those seeking employment could cast their intent by positioning themselves in the market using key words, blog posts, and network connections thereby creating connections that build relationships and job opportunities. Equally those seeking to employ have access to search mechanisms that enable them to cast very targeted propositions. Not only has this disrupted the recruitment industry by significantly reducing the need for many of the traditional recruitment agent’s activities but it has also created a significant shift in the way recruitment practices are constructed and organised. Uber is based on a very similar model that places the customer at the forefront of the process with capabilities to select the type of taxi service based on their pre-constructed cost or service quality parameters and then cast their intent to a community of service providers who then connect with the customer to deliver the product. Delivery is then monitored by the customer and product satisfaction is co-created with both the customer and the driver having the capability to post descriptions of their experience. Uber drivers have a relationship with the customer from the moment they respond to the cast intent until completion of the service.

In addition to changes at the C2B level, we are starting to see organisations such as Quivers disrupting the B2B market by introducing intent casting models to change the relationship between the owners of the brands and the retail network. Traditionally the brand (e.g. Nike, Prada) distributed their product through retailers who then in turn developed the relationship with the customer leaving the brand at arms length. Quivers is a channel disrupter focused on “Conscious Commerce” that enables the customer to determine their product choice from the brand website and cast their intent to buy via Quivers to a predetermined group of retailers giving them all the opportunity to respond and fulfil the order. This preserves the brand’s relationship with their retailers but gives the brand control over the customer experience by collecting purchase data that can then be used to deliver better outcomes for the brand. Rather than retailers being the centre of the buying process and able to offer a range of brands to the customer, the brand takes control of the decision process but then casts the customer’s intent to the retailer who in-turn fulfils the order direct to the customer. Retailers in this B2B intent casting model have to be constantly connected and capable of instant responses to claim the order ahead of competitors, which requires new skills, mobile technologies and new priorities.

So back to my printer. Had I been able to engage with an intent casting platform I would have framed the perfect cast, waited quietly on the riverbank and waited for the fish to bite. Now that sounds like the way to shop.

Posted in Digital Commerce | Leave a comment

You’re It! Play the #twitag game at Disrupt.Sydney 2014

by Ella Hafermalz

Are you on twitter? Coming to Disrupt.Sydney? Join in the #twitag game. Enlist someone at Disrupt.Sydney and take a selfie with them – post the photo on twitter with the hashtags #disruptsyd and #twitag and tag your selfie buddy to pass the challenge on to them. Our organisers Kai and Kristine have already started the game, see their example here – it’s Kristine’s turn now. Advanced players should include an interesting fact about their selfie buddy, so we can all get to know each other a little better!

Posted in Uncategorized | Leave a comment

Disrupt the disruption – and take a breath!

bScott Ward 

I recently read an article called “The last true hermit”. It was about a guy who had spent 27 years living in pure solitude
in the forests of Central Maine USA.

For 27 straight years he didn’t speak to a soul
except for a single “hello” when bumping into some bushwalkers one time.

The article related that for all of those years, he’d
spend his beautiful days simply experiencing and contemplating the world around…

As someone who lives and breathes the dynamic connectivity
that comes with working in digital transformation, the story of this man’s
solitary existence stood in stark contrast to my own daily experience.

It made me think of all the ways we now connect
with each other and how subtlety pervasive and convenient technology has become;
I was amused to discover, after reviewing the apps on my mobile phone, that I
have over 38 different channels to connect with people on a single device!

Don’t get me wrong, I love the access our new
world offers. I love that old monopolies and long held bastions of power are
been turned upside down and redistributed along lines more reflective of merit.
I love the potential we now have to tap into, harness and direct the brilliance
that sits across all humanity… we truly are in an age of miracles.


Be it environmental, political, medical or whatever your passion, now is the
time to get amongst it and lead the charge in directions that are meaningful to
us.

However… amongst all this change…. amongst all the
hype that comes with breakthrough discoveries… the experience of the hermit
reminds us that there is an elegant beauty that has been with us throughout all
the ages that sits beyond the complexity of technology.

Posted in Uncategorized | Leave a comment

We need social networks that disrupt!

by Cai Kjaer

Social networks have always existed in one form or another, but the rise of social media platforms has been able to enable the creation and sustainability of these on a massive global scale.

Initially, companies – driven by ROI considerations – seemed reluctant to embrace these tools inside the firewall, but times have changed. This is clearly evidenced by a rapid growth in the adoption of enterprise social network tools as reported by IBM’s Center for Applied Insights in their report Raising the Game that was published in August 2014. Based on a survey with nearly 1,200 decision makers the researchers found that enterprise social platforms in the period from 2012 to 2014 had moved from piloting to adoption.

It appears that executives are realising that ‘social’ is not about goofing around the water cooler, but is ultimately all about organisational performance. The growth of the enterprise social software market is another sign. From about USD$1b in 2010 it is expected to grow to USD$8b in 2019. If it didn’t deliver tangible benefits companies wouldn’t invest.

The increasing importance of networks to deliver business outcomes is now also being addressed by well-known management experts. Harvard Professor John Kotter who made Organisational Change a must-do, outlines in his most recent book XLR8 (pronounced ‘Accelerate’) how organisations need to adopt a ‘Dual Operating System’. According to Kotter, hierarchies are extremely good at doing known things over and over again, but are very poor at implementing new practices. Networks on the other hand play a critical role in getting new practices implemented.

We need these social networks to disrupt hierarchies and structures that are so dominant in our organisations. Whether or not networks can completely eliminate hierarchies is yet to be seen, but Zappos is certainly giving it a go. In the meantime we need to create organisations where hierarchies and networks will co-exist.

Posted in ESN, social networks | Leave a comment

Corporate Rationing and the Challenge of Disruption

by Simon Terry

The Russians have absolute proof that the Bible is Wrong. According to the Holy Book originally there was chaos and then there was order. The Russians know from experience that this is not so. First there was planning then there was chaos. – Soviet era joke from Russia. 

In the Soviet era in Eastern Europe jokes about queues and rationing were plentiful. People laughed in an effort to mitigate the real economic damage of central planning. In dealing with disruption, the consequences of rigid corporate planning can have real damage on the ability of the organization to meet the market.

Corporate Rationing

We do not apply the same standards to our companies as we do to our political systems. To achieve their quarterly earnings per share (‘EPS’) expectations, many large corporations resort to central planning processes with a goal of excluding any uncertainty from performance. Budgets are set and allocated in advance to deliver a number. As a result, there is limited flexibility for the organization to respond to changing circumstances or new opportunities.

Budgets are often based on hierarchical silos, further limiting the ability of the systems and processes within the organization to respond. Reallocation of budgets across silos is a major challenge within a plan year, as it has consequences for power and status of leaders, the achievement of silo-based performance and incentive plans and potentially the fate of entire functions.

The consequence of this central planning is that the modern corporation is full of rationing processes and queues. The rationing goes under many names, such as strategic priorities, investment management committees, project pipelines, resource availability, etc. The business impact is that a corporation has many barriers to responding to the market around it. To deliver certainty of the EPS outcome, the organization deliberately constrains its response to improvement opportunities and declines or delays profitable customer opportunities.

Rationing in organisations can be as absurd as that under central planned economies. Resource plans across silos or measures are often mismatched so that a budget may exist for an activity but the ability to hire additional full time employee equivalents or spend the money on vendors does not exist. Growth in sales may not be matched with additional resources to manage onboarding of customers or ongoing support. Frustration of employees inevitably rises.

Rationing also has a consequence for the scale of economic activity that organisations will consider. Given the effort required to manage the way through the rationing queues small projects are often simply abandoned. Many organisations even limit entry to the rationing process to investments of a significant scale. Each of these further constraints creates more areas in which an organistion will not respond and constrains smaller or marginal lines of business.

Consequences for Disruption

If all your competitors ration to achieve comparable metrics of performance this system of corporate rationing is sub-optimal, but rarely fatal. Everyone playing the game plays with an assumption of limited resources. The big players resources will still be bigger than smaller players and dramatic change is less likely.

However, the introduction of disruptive competitors changes the game. Disruptive competitors often deliberately choose different performance metrics to incumbent players as they see the customer opportunity and the business model differently. They are not constrained by the rationing systems that restrict decision-making and action in the larger incumbent players.

Accentuating the difference in approach, many digital disruptors enter markets with an abundance mindset and not a constrained mindset. They embrace risk and experimentation. Many a traditional organization has bemoaned their ability to compete as a large corporate when your new small agile competitor has more fundraising capacity than you, is more willing to deploy capital quickly and disregards the measures of performance that you traditionally demand?

Before an organization can respond to the market in this situation, it needs to tackle the challenge of undoing its traditional model of hierarchical rationing. The organization needs to become more responsive to its market and pursue the economic opportunities in front of it, instead of an arbitrary central plan. Power to act needs to be devolved from central functions and all areas of the organisation need the capacity to respond to demand, threats and opportunities.

These change are easy to say but incredibly difficult. Change on this scale involves the creation of an entirely new system of economic activity. It changes the roles of participants in the system of the corporation from employees, to managers to the shareholders of the organization. That kind of change is complex, hard and takes time that an organization under threat lacks.

Every company that relies on central planning and rationing as part of its management process should consider the implications of this approach and go looking for the hidden consequences. The time to build a more Responsive Organisation is before it is needed to fend off a digital competitor.

At the May Day Parade in Moscow, Leonid Brezhnev and other Russian officials watched as usual as the long parade of Soviet Military power – missiles, tanks, armoured cars and the like. At the end of the parade came a little truck with three middle aged men sitting in it. Comrade Brezhnev turned to the Defence Minister and asked: “Who are they?” The Defence Minister replied “Those are three economists. You would not believe the destructive power that they possess.” – Soviet era joke from Russia 

Posted in Disruption | Leave a comment

Management for control or responsiveness – the fault line in gamification

by Simon Terry

Digital disruption is surfacing yet again a long running a battle between two schools of thought. One school of thought is that human beings need extrinsic motivations, centrally planned processes and close management and control to improve efficiency and achieve valuable outcomes. Another school seeks to leverage the inherent motivations, human creativity and maximise the potential of people to create their own future. Any debate on gamification triggers this debate anew.

Gamification involves the use of extrinsic processes to shape human behaviour and motivation. Through use of the processes of games, gamification leverages our natural motivations to compete, to earn status or to solve challenges in the pursuit of goals set by the rules of a game. In its application of a series of external rules to optimise human behaviour, gamification is a descendent of Frederick Winslow Taylor’s scientific management with its goal of measuring and optimising human behaviour. Implicit in Taylor’s approach is the belief that smart people with the right data can better design the outcomes of others.

Under the assault of digital disruption, organisations are questioning their ability to centrally plan responses to a changing world. Gamification appears to offer a way of controlling for a better outcome without the burden of planning, detailed process management and constraining policies. If the rules of the game allow for flexibility to use better ways achieve the outcomes desired, then where is the harm? Winning additional effort in pursuit of prescribed goals is a valuable outcome of gamification and it can foster new conversations about performance.

The challenge that gamification must address is that human creativity includes the ability to understand and to change the system in which human activity occurs. Competitors can and will change the game in a situation of digital disruption. Customers can change the game. Even your employees can understand and change the game. What happens if the rules of the game need to change or even the outcomes need to change in a rapidly changing market?

Human beings have a remarkable ability to work out the path of least effort to achieve an outcome in a game. It has been said that if you want to find the flaw in a sales performance system give it to a sales team. We have used the dynamics of games to manage sales teams for years. In too many situations, sales team have ‘gamed the sales performance system’ to achieve outcomes that maximise their personal returns without necessarily meeting the purposes of the organisation or its customers. People can collaborate and change the dynamics of competitive games or can choose to compete in collaborative games. If we rely on extrinsic rules and motivation, there is a danger that the rules that shape behaviour will not adapt quickly enough to these kinds of change. The smart people managing the game aren’t always aware of the behaviours occurring however good the game metrics.

Fostering an alignment of intrinsic motivation with the purposes of an organisation is by no means a clear or easy path. However, it clarifies the role of leadership and offers the advantage of leaving the smart people to work on their own work, not that of others. Mastering the complexity of leveraging intrinsic motivation is the work of Responsive Organisations.

Posted in Uncategorized | Leave a comment