news from the intersection of branding & technology

Posts Tagged ‘Affectiva’

Emo-Metrics and the New “Q”

In neuromarketing, Shopper Marketing on May 20, 2011 at 3:04 pm

by Jeff Sandgren

Never play poker with David Berman!  CEO of Affectiva, a rising star company in the world of neuromarketing, his team is developing innovations in affective computing to help understand how people feel in order to improve products and experiences.

Be especially careful if he offers to let you wear one of his ‘special’ wristbands. Just give him your money and get it over with, because your ability to bluff will be gone.

The wristband in question is the new Q™ Sensor, a product that was officially launched this month after having been tested with over a hundred beta clients over the past year.  The Q detects and records physiological signs of stress and excitement by measuring slight changes in the skin’s electrical conductivity. This measurement, called electrodermal activity or galvanic skin response[1], is a human response component of emotional reactions.  The Q Sensor senses this measure go up when you are feeling excitement, stress, fear, or engagement, and drop when you are bored or relaxing – and might therefore know whether you really are holding an inside straight, or just faking it.

As the insightful reader has already deduced, this technology wasn’t invented for anything like gambling.  The original prototypes were developed by the M.I.T. Media Lab’s Affective Computing group run by Professor Rosalind W. Picard.  Dr. Picard and Dr. Rana el Kaliouby originally began work together with National Science Foundation (NSF) funding to develop technologies to help people on the autism spectrum, as part of a commendable effort to improve communications for people in the autism spectrum who have trouble reading and processing emotions.  Beyond those benevolent beginnings, though, it is another opportunity, like EEG and eye-tracking (see “Neurodesign” article) for neuromarketers to explore another, more common human foible – our susceptibility to the stimulus of shopping.

The inventors specialize in a science known as ‘affective computing’, which means computing technologies that “sense and respond respectfully to human emotions”, according to Dr. Picard.  Additional tools in their bag of technology tricks include the Affdex™ facial recognition and chromatic cardio monitoring –more on these in a moment.

The core technology for the Q Sensor has been validated in peer-reviewed journals demonstrating the accuracy of the electrodermal activity sensors used on the wrist compared to FDA-approved electrodermal activity sensors wired to fingertips. The validation is especially important for scientific researchers who have been limited in the past to using wired sensors in lab settings.

While basic skin sensing techniques have been used for a century (i.e. in lie detectors); previous methods typically require electrodes, wires and a lab setting. Affectiva has expanded the range of possibilities by making a comfortable sensor the size of a wristwatch, freeing people to wear it during everyday activities outside a lab. The key experiential difference is that the wearer quickly and easily moves beyond any physical distraction of wearing a monitoring device, and the device  enables essentially unlimited mobility.  Researchers can measure real people doing real everyday activities.

Eschewing games of chance (with the certainty of losing), and hungry for more insights on how the science of marketing can penetrate the art of human decision-making, we spoke to a proud and excited David Berman on the day of Affectiva’s first commercial launch, version 1.0 of the Q, called the Q Curve because it is contoured to fit on your wrist with the comfort of a bracelet or watchband “… in a way that is highly resistant to motion artifacts.”  The next version, the Q Pod, will release in a different form later this year, allowing easier wearing by children or adults in alternate locations.

We started by wondering how the company came about.  Not surprisingly, given the altruistic inclinations of the scientists, these inventions almost didn’t see the light of commercialization.  According to Berman, the science spent almost ten years in the MIT Media Lab, funded by a National Science Foundation grant, and supported by a number of corporate sponsors.  As the solutions began to show more and more promise, the pressure from the sponsors to incorporate the  technology into commercial products grew also, until Dr. Picard and Dr. el Kaliouby finally turned to Media Lab Director Frank Moss for resolution.  His solution, akin to the one that created E-Ink Corporation (and subsequently, years later, the Kindle and Nook), was to create spin-off so that the scientists could realize their nobler endeavors. By scaling the technology in the commercial world, they can help make it accessible and affordable to non-profits and fund philanthropy programs. Affectiva was born.

Berman came on board about nine months later, a move to leverage his experience gearing up WebEx, where he was instrumental in implementing the software-as-a-service sales and marketing model that grew revenue by a factor of 100.  David is no stranger to the world of start-ups … or success.  He was President of Worldwide Sales and Services for the company that was recently acquired by Cisco.  “It turned out to be a technology that made peoples’ lives better, and allowed them to have a more effective meeting experience without having to hit the road,” said Berman of WebEx.  “In the same way, Affectiva is providing game-changing technologies to improve other human experiences.”

And there are other correlations of note.

“There’s a sort of correlation between monitoring autistic children and multi-tasking consumers,” explained Berman, “because in one way you’re using the same core technology to let people share emotion to communicate more effectively.”

There’s good reason for Affectiva’s choice of the first two solutions.  The Q is designed to measure arousal, but by itself it doesn’t report whether that arousal is negative or positive.  This coordinate on the emotional map, which Affectiva refers to as ‘valence’, is captured instead by their other upcoming solution, the Affdex facial expression monitoring system.  By combining the inputs of how strongly I am responding to my environment and stimuli, together with the sentiment analysis of whether I do or don’t like a particular arousal, researchers can get a clearer picture of my location on my internal emotional map.

Affdex builds on the work of Paul Ekman, a psychologist who developed the Facial Action Coding System (FACS) to taxonomize every human facial expression.  The doctors automated many of the key elements of this code into a video analysis platform, thus enabling the software to report on the likely emotions of people whose facial expressions are being captured by a video recorder or webcam.

Avoiding Groupon’s Super Bowl Ad Flop

“The neat thing about the Affdex platform,” said Berman “is that you can monitor real time emotion at scale.  Let’s say Groupon had tried this by testing their intended Super Bowl ad (in case you missed the Antibuzz, it was a colossal failure by almost any account).  If they had sent out a trial version of the ad to a few thousand internet viewers who opted in with their webcams on, they could have known up front that people wouldn’t approve of an ad poking fun at culturally endangered Tibetans.”

Readers with webcams are encouraged follow up this newsletter by trying the technology themselves.  Simply click on, enable your webcam, and watch a few commercials.  As you’ll see, the software will detect whether or not you smiled – and this is just one simple example of a much broader range of emotional expression interpretation in the platforms capabilities.  It got me right on all three attempts.

As David pointed out, the potential is more than just ‘flop avoidance’.  Marketers could use this to monitor on an ongoing basis whether web content was “getting stale”, or even as a trigger for interaction, by using the emotional read to choose the timing and content for a particular call to action.

“We’re doing a lot of integration work with some of the larger ad measurement companies,” he revealed, “and really building out the platform ahead of the product launch.  Plus, we’re really having a lot of fun!”

“We’re starting with measuring media, but there’s a whole next wave of interaction that this technology enables.”

Regarding privacy concerns, Affectiva “takes the highest road” and is 100% opt-in.   All research participants are volunteers.  “If you make people’s life better, we believe they’ll opt-in.  They key is making sure they get enough value out the experience. At the end of the day, measuring customer experience means consumers will get a better product or more tailored solution.”

Speaking for the editors, we’d LOVE to know what people are thinking when the read our blog at .

Another example for our readers in the retail space is the MIT Mood Meter, where a deployment of cameras around the MIT campus counts smiles to determine the general sentiment of students, faculty and visitors.  See , and imagine having that insight, that easily about your shoppers.

What about the mysterious third solution?  A little further out, Affectiva is developing a solution that monitors blood circulation by detecting tiny variances in skin tone, as another way of measuring arousal without any worn hardware.  We’ll let you know when news breaks on that.

Early Learning at IPG Shopper Sciences

We were able to connect with one of the earliest users in the consumer goods space, John Ross, CEO of Mediabrands Shopper Sciences, who is using Affectiva and other solutions to gather data as people shop, improve the shopping experience, and create more value for customers.

Interpublic Group, as readers probably know, is one of the giant media companies with agencies around the world.  To serve the broad array of customer needs regarding shopper marketing, they decided a few years ago to build dedicated lab space, the IPG Media Lab, starting with one in Los Angeles.  The New York Lab is scheduled to open in the fall.  Charter clients included Microsoft, Sony, Johnson & Johnson, Bayer, Kohl’s, and Home Depot.

“One important consideration,” Ross told us, “was to facilitate a place where learning and pure research could take place with teams that were NOT necessarily attached to a client, allowing us to advise all our clients more objectively.”

“When the lab identifies trends, products spin out of that,” said Ross “and Shopper Sciences is an example.  In this world where the shopper is so informed, so powerful, we need a new toolset to understand shoppers and the decisions they make.

“Most of the marketing world focuses on consumption, on consumers.  They look a brand awareness, brand consumption, brand tracking.  Retailers focus on transactions like register logs and ecommerce reports.  What’s left in the middle is
WHY – why shoppers behave as they do.”

Shopper Sciences’ tool set include such lofty elements as ‘purchase barrier matrices’, ‘neural shopping matrices’, ‘physical response diagnostics’, and proprietary tools to score point of purchase materials, all to get a better handle on how  shoppers make decisions.

Clearly, it’s a perfect environment for Affectiva solutions.  Ross saw them as a tool set that would allow them to bring a quantitative measurement to emotional decision-making.  A big plus, according to Ross, was that the form factor enabled a methodology that could be readily scaled for retail.

“The form factor was so elegant that it fades into the background,” complimented Ross.  “This is important because shoppers tend to answer surveys smarter than they behave; they under-report the emotional side of their decision-making.  These tools give us better insights to what they’re really feeling.”

Shopper insights versus Buyer insights

Ross cited, as an example, a recent study of young women cosmetics shoppers.  A major brand client was concerned about representation in stores, and wanted to better understand shopper behavior.  Using the Q Sensor, they were able to determine that women were engaged and confident when picking the first product, but then became more stressed when they tried to find additional cosmetics in suitable colors. As a result, very few kept the same brand across the various cosmetics.

With these insights, the brand and retailer are developing improvements to reengineer the shopper experience by making it easier for shoppers to match colors in-category.

“We’re bringing the emotional state of the shopper into the ‘fix the sales’ process,” says Ross.  “Presupposing that the shopper isn’t emotional does them a disservice. What looks like rote behavior by shoppers often isn’t.  Shoppers bring in a burden of worry, concerns, and need for education.  These new techniques allow us the scale to reveal the ‘micro-moments’ when the shopper becomes distressed or disengaged.”

As brands and retailers seek to improve  consumer experience (and their own performance), there seems to be a clear need to go beyond ‘Buyer Insights’, which only assess what people actually bought, into the more transient realm of ‘Shopper Insights’, looking at what’s going on inside the minds of those who are on the verge of making a purchase decision.

“We have all this data and agency reporting on buyers,” agrees Ross, “but the data on the poor shopper is so thin by comparison, that it gets lost in the strategic decision-making process.  We’re on the cusp of bringing the voice of the shopper into the process, and we believe this will transform retail. ”

The voice of the shopper, the internal voice that captures both rational and emotional elements, is perhaps the compass by which our consumer-driven economy can recover its direction.  Ground breaking research at the intersection of branding and technology is pointing the way.   JTS

The Q Sensor also measures temperature and 3-axis motion for additional
contextual interpretation of user state

%d bloggers like this: