By Joe Lewis

Have you ever wondered how Google has been able to return accurate images whenever you enter a query? Or maybe you wondered how best to improve “spellcheck” applications in order to allow for the recognition of words outside of the dictionary. These are just some of the ideas Google has recently been working on. During the recent IST Graduate Symposium, Peter Norvig, Director of Research at Google, presented some of the advancements that his company has made in areas of text and image modeling. Text and image modeling involves identifying the needs of a specific set of users.  Norvig described the history of these concepts in relation to how artwork, movies, dictionaries, and books originally set out to address people’s needs.  The advancement of these concepts was the focus of Norvig’s presentation.

In the realm of image modeling, Norvig first detailed some of the advancements shown at SigGraph 2007, a digital media conference held every summer. These advancements include Automatic Photo Resizing and Scene Completion. Automatic Photo Resizing is the concept of being able to adjust the size of an image without distorting its original content. This is done by taking an isolated area of pixels in an image and using a mathematical algorithm to accurately expand or condense that area in real time. Scene Completion looks at the process of editing incomplete photos by comparing the unfinished area to millions of similar photos in a database.  A correct match allows for the completion of the image. Both of these design concepts were not originally developed at Google.

Specifically, Google’s Image Modeling research focuses on People Annotation and Image search Queries.  People Annotation is the concept of making sure that the correct individuals are returned whenever an image query is entered. For example, how do we know that the images returned when searching for Steve Jobs, the CEO of Apple, are really pictures of him? While many people know what he looks like, there are some who do not.  Google’s People Annotation algorithm compares results of such queries to the millions of real images of famous individuals that are stored in Google’s databases. In essence, this ensures that most of the images that are returned correspond to the real Steve Jobs, and not some other individual. Image search Queries use image models to find images that meet users’ needs without the use of keywords. This involves analysing the colors and distinct features that could appear similar images.

Google’s advancements in text modeling include Segmentation and a more advanced version of Spellcheck. Segmentation is the concept of dissecting a cluster of text in order to figure out what is actually being stated. For example, given the string “smallandinsignificant,” how would individuals or computers figure out the actual individual words within this cluster.  Segmentation looks at the first “s” and measure the probability of that being a word.  In this case, it is not likely that “s” is a word.  Therefore, it will move on to “sm” and use the same idea until it comes up with the answer, “small and in significant.”  This allows the computer to get close to what the user might be actually be looking for, which is “small and insignificant.”

Google’s newly developed Spellcheck uses more than the dictionary in order to check spelling. Instead, it uses databases of words obtained from multiple sources in different environments. Using this technology, words like Google or Facebook will no longer have the red lines beneath them.  These are just a few examples of how Google plans to continue to use text models to assess users’ needs.

Overall, these advancements in Image and Text Models emphasize the importance of trying to identify the exact needs of users’. With concepts such as People Annotation and Segmentation, users will be able to get more out of the services that Google offers.  Norvig ended his presentation with the quote, “There is nothing nowadays that our children fail to know.”  This serves to illustrate how, as technology improves, information will continue to become more accurate and beneficial.