Deep learning at the center
The past decade has seen the democratization of photography through the smartphone revolution, with more pictures taken every two minutes than were taken throughout the entirety of the 1800s, according to some estimates. In fact, InfoTrends predicts more than a trillion images will be taken this year alone.
“Cameras generate so much data, and a lot of times you need immediate action and analysis from this data,” she says. “This research area is one that I’m very passionate about and its increasingly becoming more important to HP.”
Lin and her team were the creators of Pixel Intelligence, a powerful portfolio of computer vision algorithms that helps print service providers make sense of visual data. The algorithms can find faces within an image or find the same face in multiple images with great accuracy. They can also recognize specific objects (such as a specific logo), detect whether people are smiling or their eyes are shut, and sift through hundreds of images to make a collage of the best photos. Moreover, they work in real time, opening up new avenues for improvement in devices like smart home assistants that constantly monitor their surroundings.
The common language of Pixel Intelligence and Lin’s current research is an area known as computer vision, a branch of AI that helps computers to understand visual information. In recent years, computer vision tasks have become more accurate with machine learning models. “At HP, we’ve started using it to enhance our ability to understand images,” she says.
This creates potential commercial applications and new capabilities such as in social sharing. When you upload a photo on a social media site, the application downsizes the photo so it's faster to transmit, which reduces the quality of photos. Lin and her team have looked at using deep learning to scale up these photos for large-format printing. They’re also researching how to “recompose” photos, to create images that are more professional or creatively composed, even when shot on a smartphone.
The organization and retrieval of images, Lin says, also has huge opportunities. How do you make sure the photo and memories captured on smartphones are not lost when people replace them throughout their lives? Or how do you sort thousands of photos of your kids? Lin is curious about what kind of software could enable you to tell a photo story from babyhood to college. “We are thinking about how to use AI to tell stories with your photos,” she says. “It can mine through collections, events, and digital archives, too.”
