Artificial Intelligence research is changing Facebook as we know it now, and how we even conceive of its capabilities going forward. Consider how AI is expanding the capabilities of visual recognition.
Expanding on its photo-tagging prowess, Facebook Messenger is now testing a new feature called Photo Magic that scans photos and reminds you to send them to the friends who are in them. The test, which kicks off this week in Australia on Android followed by iOS mobile devices, is just the tip of the AIceberg.
Facebook Artificial Intelligence Research (FAIR) team in Paris, led by Yann LeCun, also contributed to the launch of Moments in June, a standalone app that helps you privately share photos with friends. As Facebook comments about Moments in its video below, “The app uses facial recognition technology to organize the photos in your phone according to which friends are in them,” yet another way that FB is visually ID-ing and staying on top of users’ content and photos in a helpful way.
Those AI-based enhancements are happening to Messenger, which is getting ready to roll out its biggest virtual helper of all: M, a digital assistant that aims to give Siri and Cortana a run for affection (and ad dollars). As The Wall Street Journal notes, “If M works, it could mean Facebook will open up a major new front in its battle with Google for your attention, and in the process give the social networking site access to a gigantic pool of direct-response advertising dollars that has so far eluded the company.”
And if M doesn’t succeed? It’s still “in the vanguard of services that will bring about a sea change in how we interact with computers, a massive transition from using our devices as finicky tools to asking our devices to simply take care of things for us,” WSJ comments.
Exploring the boundaries of AI makes sense for Facebook in terms of white space to explore. More than 2.5 billion people worldwide use messaging services, a figure that’s projected to grow to pass 3.6 billion by 2018, according to Activate.
The market for smarter messaging is a massive global opportunity for Facebook , which already has more than 700 million Messenger users, to differentiate itself from Google, Microsoft and Apple—and, on a bigger scale, to differentiate its brand from its tech and social peers.
Artificial intelligence, which powers M, Apple’s Siri and Microsoft’s Cortana, is a huge are of research and investment for Facebook. As WSJ notes, Alexandre Lebrun, who heads the team behind M, is having people quiz M with requests from restaurant reservations “to sending your friends a parrot.” The human testers are, in effect, training M to understand the complexity of human queries and improving Facebook’s AI algorithm.
AI is also powering Facebook’s Visual Q&A, a mobile-based system that uses multiple approaches to “deep learning” including convolutional neural networks and end-to-end memory networks that transform data into accessible consumption for audiences not normally able to do so, such as people who are visually impaired.
“Think of what this might mean to the 285 million people globally who have low vision capabilities or the 40 million who are blind,” Facebook CTO Mike Schroepfer wrote in a blog post on the company’s milestones in artificial intelligence. “Instead of being left out of the experience when friends share photo content, they’ll be able to participate.”
At its annual F8 developer’s conference in March, Facebook demonstrated its AI capabilities by demonstrating the results of a query to a computer about lengthy Lord of the Rings passages. As Schroepfer commented at the time, “You can really get these systems to understand deep, minute differences.”
Schroepfer says the goal is to enhance computers to “understand language more like a human would—with context, instead of rote ones and zeroes memorization like a machine.”
“You can imagine tons of useful applications for Facebook,” said Schroepfer, such as responding to a query “if there’s a baby in a photo, where the baby is standing, and what it’s doing? My New Assistant was able to correctly respond that yes, a photo did include a baby, and that it was in a bathroom, and having its teeth brushed.”
“While a sighted person would have no trouble understanding the impact of a photo of a stunning sunset over San Francisco Bay, a blind person would get nothing out of it without additional context,” Fast Company comments. “With Facebook’s AI technology, however, that user’s screen reader—a tool that converts text to speech—could tell them this: “This image may contain ‘nature, outdoor, cloud, grass, horizon, plant, [or a] tree.’”
Facebook is fine-tuning such AI advances so that computer vision will not only see but understand. “Our team has created not only a system that has taught machines this skill, but also a state-of-the-art research system that can segment images 30 percent faster than most other systems, using 10x less training data across industry benchmarks,” Schroepfer wrote.
Already, Facebook’s computers can make inferences about whether virtual blocks stacked unevenly will topple over with a 90 percent accuracy rate, and researchers are now trying to teach a computer how to play the Chinese board game Go.
“Our AI research efforts — along with our work to develop radical new approaches to connectivity and our work to develop immersive new VR technologies — are a long-term endeavor,” as the company states. “But if we can get them right we will be able to build systems that are smarter and more useful, enable developers to create immersive new experiences, and make it possible to connect everyone in the world.”
“The reason this is exciting is because it’s scalable,” Schroepfer told Fast Company about the possibilities of the AI-savvy M digital assistant. “We could deploy (the assistant) for the entire world. The promise I made to all the AI folks that joined us is that this is the best place to get your work to a billion people.”
Below, find out about Facebook’s AI work from Yann LeCun in an interview with Bloomberg: