Computer Science
2013 Xerox Research Opportunities
Prof. Jeffrey P. Bigham
Department of Computer Science
jbigham@cs.rochester.edu
Research Project #1: Real-time Human Computation
Over the past few years, human computation -- integrating the intelligence and decision-making skills of people in computational processes -- has been shown a practical means to add true intelligence to computer programs today. As an example, computer vision is difficult, and so it can make sense to have a computer program query humans out on the web when it needs information about an image, instead of trying to do this automatically. Research goals include (i) developing methods for quickly integrating the input of dynamic pools of workers into actionable decisions, (ii) designing and implementing toolkits that enable developers to easily include real-time human computation as part of their own programs, and (iii) devising methods for estimating the expected latency for answers from different sources of human computation from past experience. Students working on this project will participate in the design of methods for achieving effective real-time computation and contribute to an open source toolkit allowing others to use real-time human computation in their own projects.
Research Project #2: Real-Time Captioning by the Crowd
We are developing a system called Scribe enables the crowd (people recruited from the web) to caption speech in real-time. Each crowd worker is unable to type at natural speaking rates (200-250 words per minute), and so Scribe systematically directs workers to type part of what they hear and then merges the partial captions back together using a new online version of Multiple Sequence Alignment (an algorithmic approach for reassembling DNA during shotgun sequencing. Collectively, crowd captions have less latency and are often more readable than those created by either expert stenographers or Automatic Speech Recognition (ASR). Crowd captions train the ASR, allowing it to adapt to the speaker, domain, and environment. Students working on this project will do work toward a deployable prototype of the system, develop techniques for integrating automatic speech recognition into the system, or design approaches to make the crowd more successful at entering captions quickly.
Research Project #3: Chorus: Crowd-Powered Conversational Agents
Autonomous systems cannot yet reliably engage in an open-ended dialogue with users due the complexity of natural language processing, but online crowds present new opportunities to do so. Our Chorus system seeks to enable real-time two-way natural language conversation between an end user and a single virtual agent powered by a distributed crowd of online humans. Chorus maintains consistent, on-topic conversations with end users across multiple sessions even as individual members of the crowd come and go by storing a shared, curated dialogue history. While users see only a steady stream of dialogue with a single conversational partner, multiple crowd workers collaborate to select responses via an interface that allows them to rapidly scan conversational history, identify relevant parts of a conversation, and select between responses. Chorus was recently covered in the MIT Technology Review magazine:
http://www.technologyreview.com/news/429118/artificial-intelligence-powered-by-many-humans/.
Students working on this project will contribute to mechanisms designed to incentivize crowd workers toward quality responses, and adaptation of Chorus to specific applications (for instance a conversational assistant for mobile phone users or blind users).
Prof. Jiebo Luo
Department of Computer Science
jluo@cs.rochester.edu
Research Project:
Professor Jiebo Luo's research spans image processing, computer vision, machine learning, data mining, medical imaging, and ubiquitous computing. He has been an advocate for contextual inference in semantic understanding of visual data, and continues to push the frontiers in this area by incorporating geo-location context and social context. A recent research thrust focuses on exploiting social media for machine learning, data mining, and human-computer interaction, for example, mining the wisdom of crowds for social, political, and economic prediction and forecasting. He has published extensively with over 200 papers and 70 US patents (check http://www.cs.rochester.edu/u/jluo/).
The following areas are under current research:
- Computer vision: recognition of objects, scenes, people, locations, actions and events from images and video
- Machine learning: learning with large-scale web data, cross-domain learning, active learning
- Social media data mining: prediction, forecasting, suggestion, and recommendation using open source data
- Human computer interaction: interactive visual recognition; integration of crowd and machine intelligence
- Biomedical informatics: structural and topological analysis from 3D imaging data; surgical video analysis
- Mobile and pervasive computing: context-aware applications; multimodal inference from multiple sensors
Prerequisite: programming experience in C++, Matlab, Java, or Python
