Visual Processing For Social Media
Andrew Gallagher, Tsuhan Chen
Abstract
For the first time in human history, literally billions of images are available to anyone with access to the Internet. The sheer number of images provides challenging in indexing and organization. Furthermore, this social media presents an unprecedented opportunity to understand how people interact socially by providing a window into the lives of millions of people. This tutorial will present the fundamentals of visual processing of social media and outline recent developments in the field.
When we see other humans, we can quickly make judgments such as their demographic description and identity if they are familiar to us. We can answer questions related to the activities of, emotional states of, and relationships between people in an image. We draw conclusions based not just on what we see, but also from a lifetime of experience of living and interacting with other people. Even simple, common sense knowledge such as the fact that children are smaller than adults allows us to better understand the roles of the people we see. We describe contextual features, called social context, drawn from a variety of sources, and models for understanding images of people with the objective of providing computers with access to the same contextual information that humans use.
Further, we show that computer vision can play a role in helping us learn about people. We now are able to see millions of candid images of people on the Internet. Using visual processing techniques, we are able to gain insight into the lives of people, and their behavior in social situations, that simply was not possible before. In other applications, each image that is shared implicitly represents a vote of interest. By mining this rich data source, applications can perform high-level tasks such as prediction election results or planning vacation tours.
This tutorial will emphasize that people act in predictable ways, for example that human patterns of association contain regular structure that can be effectively modeled and learned. From a broad perspective, this work presents a loop in that our knowledge about people can help computer vision algorithms, and computer vision can help us learn more about people.
Speakers
Andrew C. Gallagher
Andrew C. Gallagher joined Kodak Research Labs in 1996 after earning a B.S. from Geneva College, and initially developed image enhancement algorithms embedded in many of Kodak's consumer imaging products. From this effort, Gallagher was awarded more than 90 U.S. Patents and Kodak's prestigious Eastman Innovation Award in 2005. Gallagher received the M.S. degree from Rochester Institute of Technology in 2000 and the Ph.D. degree from Carnegie Mellon University in 2009, both in electrical and computer engineering. Recently, his interests are in the arena of improving computer vision by incorporating context, human interactions, and image data. Further, Gallagher enjoys working in the areas of graphical models and image forensics. He is currently a senior principal scientist with Eastman Kodak and is the Awards Chair for the Rochester Section of the IEEE, and the past chair (2009-2010) of the Rochester Section IEEE Signal Processing Society.
Tsuhan Chen
Professor Tsuhan Chen has been with the School of Electrical and Computer Engineering, Cornell University, Ithaca, New York, since January 2009, where he is the Director of the School, and the David E. Burr Professor of Engineering. From October 1997 to December 2008, he was with the Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania, as Professor and Associate Department Head. From August 1993 to October 1997, he worked at AT&T Bell Laboratories, Holmdel, New Jersey. He received the M.S. and Ph.D. degrees in electrical engineering from the California Institute of Technology, Pasadena, California, in 1990 and 1993, respectively, and the B.S. degree in electrical engineering from the National Taiwan University in 1987. Tsuhan served as the Editor-in-Chief for IEEE Transactions on Multimedia in 2002-2004. He co-edited a book titled Multimedia Systems, Standards, and Networks. He received the Benjamin Richard Teare Teaching Award in 2006, and the Eta Kappa Nu Award for Outstanding Faculty Teaching in 2007. He was elected to the Board of Governors, IEEE Signal Processing Society, 2007-2009, and a Distinguished Lecturer, IEEE Signal Processing Society, 2007-2008. He is a Fellow of IEEE.