Human-Centered Technology: Implications of Advances in Facial Recognition and Human Mobility

Writers of speculative fiction are having a hard time coming up with far-fetched ideas faster than entrepreneurs can bring them to market. Consider how many common science fiction tropes have become goods for consumption by the mass market in just the past few years:

  • Human matter compilers (3D printed organs)
  • Clones of mammals (including primates this year)
  • Anti-gravity rooms (Indoor skydiving)
  • Private rockets (courtesy SpaceX)
  • Robots on Mars (Opportunity is still sending back pictures)

Now a new futuristic vision from sci-fi films like "Minority Report" is coming true. We’re not talking about the psychics that can predict the future (although data and analytics are getting very close), but facial recognition software that can identify individuals and potentially send them private holographic ads at the right time and place. However, these advances in Facial Recognition and its use for “identifying” an individual have deep implications at many levels of everyday human existence - both individually and collectively - a sense of being, privacy, loss of anonymity, a sense of belonging to a group, coping with a sense of being surveilled and an onerous responsibility to preserve all these human values based on how one navigates the new world.

In this brief essay, we explore some of the key aspects as really high quality facial recognition gets adopted in a widespread manner across the technology landscape.

Facial recognition technology has advanced considerably over the past decade. The most advanced latest facial recognition software, combined with machine learning algorithms, has gone mainstream, with applications from “Mission Impossible” style security clearance to real-time personalized emojis that mirror the sender’s facial features. The more images the software has to work with, the better it can identify faces from every angle and even in partial shadow.

Shared facial data at scale and advances in camera technology, storage and compute hardware has made deployment of these systems at different form factors economically viable. Here are a few ways that facial recognition is likely to become an essential part of daily life in the years ahead:

  • The End of Debit Cards – Bank of America is leading the investment in facial recognition tech and other biometric sensors, along with partners like Samsung, Microsoft and PayPal to get ahead of the market.
  • Passport Control – Australia’s Vision-Box is already experimenting with facial recognition at airports to replace passports for frequent travelers.
  • The Price of a Smile – For consumers still getting used to paying with their phones, it might come as shock to learn that Chinese consumers have made purchases with their faces alone. Facial recognition initiatives in China allow consumers to authorize payments or gain access to buildings.
  • Future Crime Watch – For the past three years, the Arizona Dept. of Public Safety has been coordinating their criminal investigations with the Dept. of Transportation, both of which use facial recognition databases.
  • You've Got the Look – Personalization equals relevance, and that's becoming more important every day in the war for consumer attention. Marketers in retail are using facial recognition for sentiment analysis in store. This is one way that brick and mortar locations can prove superior to online competitors, offering dynamic assistance on the spot.

Potentially every human interaction could be first gated via a facial recognition system intervention. Facial Recognition technology is the first major technology wherein an individual can be identified “passively” in real-time at any point of interaction without the individual actively volunteering “identity” information. People are “authenticated” on the fly. Once authenticated in such a manner, all the data about the individual that has been stored thus far can be brought to bear on “modulating” that interaction - be it with another person, system, organization etc. Thus far all “sensors” have been anonymous at scale. For example, street cameras in a city such as London monitor the street - not an individual.

Once we integrate a facial recognition system with the video feed from the camera, the “data” possibilities are endless. Some key aspects of deploying facial recognition are -

  • Individual context can be added to every human event either in public or private. Every sensor interaction with an individual (or other entities) can be tracked - Did I open the apartment door or did my dog do it?
  • At each interaction, we will know specifically how many “kinds” of people are there on any “attribute” - For example, in a bus - one may know exactly - how many men, women and children are there - Not just a count. We will know how many people representing different ideologues are there in the bus. As the data gets richer and linked to other currently de-identified data sets, the potential implications are enormous. As tech firms innovate, there will be an even greater potential for new business models and innovative concepts in the near future.
  • Imagine if these data were shared - between organizations or centrally held and also made as a public utility - government, industrial and consumer applications are boundless.

For example, imagine not recalling an individuals name at a cafe when they walk up to your table - an ongoing facial recognition system - could prompt you with the name of that individual.

Facial recognition augmented systems can amplify social interactions both positively and negatively (imagine big brother). One can choose apriori - what kind of crowd you want to interact with. Imagine - before hitting a bar - knowing what kind of people are in there now or the bar deciding what kind of folks they want to let in. Instead of a public spot, one can also decide what kind of “employees” you want working at different times of the day - depending on the organization. Services can be highly hyper-personalized - on the fly. Even before you place an order by entering an restaurant - they can prep for you by seeing you walk up the street.

  • One of the major “moats” preventing exploitation of people data currently is the high-cost of “identifying” an individual - linking him/her or physical world activities to online activities. The whole tech industry is investing at scale to bridge this barrier. Potential applications are enormous in numerous contexts. Current uses of facial recognition technology have been to identify the few individuals who do not meet the norms of society, primarily from a security perspective. Using the same technology to identify the larger crowd has numerous pros and cons.
  • Human mobility data that is currently harnessed via smartphone data at an individual level will be complemented with actual video-tagged data. Note that the visual feeds can come from cameras mounted statically or in motion - imagine drones and tethered balloons and other aerial monitoring systems. The constant potential of being identified may lead to new kinds of “vacation spots” - islands in the cyber grid - places where there is no possibility of being identified.

Technologically, as many business processes, systems and consumer goods get augmented with Facial Recognition systems, it will be interesting to see how consumers adapt to this technology. Are consumers willing to give up the anonymity provided by a password with facial recognition. Are the benefits worth the risk? Recent studies also have shown how Facial Recognition systems are highly biased.

This is a big barrier - would a business want to risk this? When a consumer talks to a customer service agent, the consumer would also like to know everything about the agent. Would this current information asymmetry between consumers and business be resolved. Imagine a new age Glassdoor for customer service agents? Consumers and businesses need to be educated on various natural barriers of communication that will be bridged - extra information may not be that helpful in providing better outcomes.

One of the interesting questions to answer would be - what does advanced facial recognition mean with respect to GDPR - the EU driven global movement to enhance/protect and secure an individual’s privacy. Maybe one cannot deliver the holographic advertisement that was mentioned earlier. The economic costs for a business may be onerous to deploy a facial recognition system.

In the real world, friends and family immediately recognize each other by their faces. However, along with recognizing the individual - you get the “whole” set of attributes about that individual. Identifying an individual means much more that just getting “access” to a resource as current systems are being positioned.

It is going to be an interesting road that our society as a whole has to traverse collectively, as these facial recognition systems get deployed at scale over the next few decades. The everyday consumer will hopefully ultimately decide where do facial recognition systems fit into our everyday life - ranging from the dystopian to utopian.

Also Read "Place Analytics with Human Mobility Data - An Overview"