Measuring Avatar Engagement: the $100 million question

The Playboy Sim

Everywhere I go in virtual world circles, I seem to encounter folks wrestling with the question: how do you measure avatar engagement? (Check the new Meta Metaverse blog and the eSheep blog for interesting commentary on this.)  For companies looking to use virtual worlds for marketing, avatar engagement is the Holy Grail. Non-profits are also struggling with how to track if avatars coming to their virtual headquarters are getting the message and incorporating it into their real lives.  Researchers are keen to develop measurements to unpack quantitatively how avatars experience different virtual environments and consume media they are presented with.

Avatar engagement is a thorny question, and one that has dogged other new media and web-based spaces for a long time now. As Steve Bridger says, "it’s like eating and elephant, you don’t know where to begin."  I’m not a metrics geek, so I can’t speak to the technology needed to create better engagement measurement tools.  (I’m always impressed with Tateru Nino’s Mixed Reality Headcount she uses on New World Notes.)

What I am more interested in is one step up — figuring out what are the relevant engagement questions the metrics are designed to answer.

Regardless of the kind of enterprise you are creating in virtual worlds, it seems to me that there is a discrete set of questions you should be trying to answer.  These progress from simpler to more in-depth questions that your engagement metrics might help you unpack:

First Order Questions: basic metrics

1. How many are coming?
2. How long are they staying?

Second Order Questions: Delving deeper into avatar activity

1. Do the same avatars return? How many times, how often?
2. Do they bring / refer friends?
3. What are they doing?

Third Order Questions: Expected / unexpected activity

1. Are they following a discrete path in your sim?  Is this a path you intentionally created?
2. Are they interacting with objects?  Are these the objects you intended for them to interact with?
3. Are they interacting with other avatars?  Are these avatars your staff, your associates, your talent, their friends, or strangers?

Fourth Order Questions: Deeper knowledge about the audience

1. What sorts of avatars come to your sim?  i.e. furries, goreans, robots, male/female/other
2. What people come to your sim? RW geographic location, age, gender, income, etc.
3. How do they find out about your sim? Web, viral video, referral, etc.

Fifth Order Questions: Matching engagement to behavior (i.e. "the money questions")

1. Did engaging with your VW presence make them more favorable your product / brand / message?
2. Which kind of engagement made them most favorable? (i.e. events, contests, freebies, sandboxes)
3. Were avatars engaged with your VW presence in ways that you did not anticipate? (i.e. user-generated role-playing, unplanned social events, unintended applications of your VW products)

Many of these higher-order questions don’t seem easily answerable in most virtual worlds without a high level of access to user data.  For the kinds of custom-made, research-focused worlds and gaming environments that Dr. Ted Castranova and the Department of Defense are planning, this might be possible. But for worlds like There and Second Life, there are significant privacy and technical surveillance issues that would have to be addressed. 

The Second and Third Order questions could be partially addressed with custom HUDs, attached devices and other technical add-ons that could track and report on avatar behavior with a finer grain of specificity.  The Fourth and Fifth are probably most easily answerable using surveys and ideally in-depth focus groups. 

I think one of the most promising ways forward was noted by renowned educational theorist Dr. John Bransford, who talked last year about how to create testing environments for virtual world education. In his view, what is needed is a more scientific approach to testing claims made about the viability of virtual environments for enhancing education, but creating different kinds of educational experiences and seeing which ones are the most effective. 

From a marketing perspective, this is the kind of rigorous, quantitative approach that would help get beyond the very fuzzy science of measuring avatar engagement.  If we are serious about wanting to understand why engagement matters, that’s really the sort of approach that gathers real data, builds knowledge and assembles best practices.