Click on image to enlarge.

The UCSD Division of Calit2 has awarded support for the Provocateur Project and its principal investigator, CREL Director Shlomo Dubnov, among the inaugural recipients of Calit2 Strategic Research Opportunities (CSRO) grants. The project will develop a system and environment for participatory events that will support audience back-channeling, crowdsourcing and analytics with moderator functions for novel models of brainstorming, learning and entertainment. The popularity of web conferencing and content sharing has been driven by a desire to increase productivity and operate cost-effectively on digital media and electronic text from remote locations. The system to be developed under the CSRO grant will also be useful in new entertainment forms, such as telematic performance, multiuser computer games and participatory film and music performance events. "The Provocateur project points to the importance of creating a system that enables and supports an intensive social interaction that is distributed among the attendees in the room as well as among participants across the Web," said Ramesh Rao, Calit2's division director at UC San Diego. The prototype system will be deployed for use during seminars, conferences and performances at various locations in Calit2's Atkinson Hall headquarters as well as other venues on and off campus.

CREL is participating in a project with the California Institute for Telecommunications and Information Technology (Calit2), San Diego Supercomputer Center (SDSC) and UCSD's Cognitive Science department to study artificial agents, such as humanoid robots and animated characters, which are actively being developed for use in various domains such as entertainment, retail, education, and healthcare. But how should artificial agents be designed? It may seem like a good idea to make artificial agents look as human-like as possible, especially if they will be used in social settings. However, we soon encounter a phenomenon called the "uncanny valley": As an agent’s appearance is made more human-like, people’s reactions to it becomes more positive and empathetic, until a point at which the increasing human-likeness leads to the agent being considered repulsive, disturbing, or "zombielike". "We are very excited to embark upon this interdisciplinary research program. Scientific research on how people perceive, respond to, and interact with artificial agents is needed as these technologies can impact almost every aspect of our lives from entertainment to education and healthcare," says Ayse Saygin, project lead and assistant professor of cognitive science at UC San Diego. For example, many viewers found the film Polar Express creepy, whereas Avatar received more positive evaluations. Despite significant anecdotal evidence, there is little scientific data to characterize the uncanny valley; read more about the uncanny valley here: http://en.wikipedia.org/wiki/Uncanny_valley. "In our lab we are exploring the uncanny valley using various different methods such as perceptual experiments, physiological recording (GSR, heart rate), and neuroimaging," says Saygin. Activities of the project for a group of motivated and technically skilled high school students include:

• Finding, digitizing and editing images and movie clips of normal and "uncanny" characters.
• Creating animations of normal and "uncanny" characters.
• Setting up and analyzing online questionnaires and experiments (e.g., present pictures or movies on the web and collect responses).
• Recording and analyzing heart rate and galvanic skin response.

"Our goal is to improve our understanding of how the human brain enables social cognition, as well as to inform engineers and designers in developing future agents that are well-suited to their application domains, as well as to our brains,” says Saygin. "We will be able to test how humans interact with a broad spectrum of avatars and to learn how the nervous system supports social cognition – an undeniably important skill and part of what it means to be human." Support for the project includes a one-year Calit2 Strategic Research Opportunities (CSRO) grant awarded in May 2010.

CREL is participating in a project with the San Diego Supercomputer Center (SDSC) and UCSD's Music Department to develop new methods for transforming real-time data from power meters on the UC San Diego campus into advanced visualizations that can bring the data to life. It's part of externally funded research to demonstrate integration of on-site renewable energy production with the UCSD campus-wide microgrid. Within the context of SDSC's data storage and analytical tools for use in visualizing data, CREL researcher and Computer Music visiting scholar Joachim Gossmann is focusing on the audio component of visualization, in a project on "Sonification of UCSD Campus Energy Consumption." Sonification will provide a powerful aural input to the visual experience of trying to understand energy consumption -- creating a spatialized soundscape assembled from "tipping-container" synthesis metaphors. Virtual buckets are filled by current energy flow, emitting an audible “click” impulse once a certain level is reached. Psycho-acoustic differentiation by spatialization and timbre will allow the transparent superposition of simultaneous streams – resulting in a soundscape in which listeners can re-orient and focus their attention. "The goal is to get the sensibility of our hearing and our abilities to listen and be involved in what we do on a whole new level," says Gossmann. According to UCSD Director of Strategic Energy Initiatives Byron Washom, "I see this project as pioneering in its field, someday enabling plant managers to be able to detect energy anomalies both aurally and visually." UC San Diego's microgrid serves a 1,200-acre, 450-building campus with a daily population of 45,000 running two 13.5 megawatt gas turbines, one 3 megawatt steam turbine and a 1.2 megawatt solar-cell installation that together supply 82 percent of the campus’s annual power."

San Diego, Feb. 24, 2010 -- Researchers at the University of California, San Diego are reaching across disciplines to push the envelope of cultural and learning opportunities in the age of social networking and digital media. They have established a new Center for Research in Entertainment and Learning (CREL). It will focus on grassroots creativity and learning while promoting research that combines artificial intelligence with interactive and participatory media to allow people to be more creative, informed and make better decisions in educational as well as entertainment environments.

CREL is housed within the UCSD Division of the California Institute for Telecommunications and Information Technology (Calit2).

"The rise of the Internet, the Web, digital media and social networking environments introduces new methods for the creation of educational and cultural content," said CREL founding director Shlomo Dubnov, a professor in the UCSD Department of Music. "We are witnessing the emergence of new amateur practices and innovative designs in which multiple users collaborate, produce, annotate and re-use shared multimedia content."

Research and development in the new center will use advanced IT technologies to help people deal with meaning in complex situations, such as questions that arise during story narratives, from films to business situations. And of course understanding how music sets the mood and frames expression is an indispensable part of this research. So by combining Semantic Web platforms with analytics and data mining, annotation, visualization, sentiment and emotion analysis, and making it all ubiquitous via mobile technologies, digital media and design, CREL intends to reach out to industry, including the many companies in California that are transforming digital media.

Audience members use their laptop to participate in an online chat that is integrated into Shomo Dubnov's Kamza and Bar Kamza, performed here in UCSD's Loft.

"Calit2 has a mandate to help keep California on the bleeding edge of innovation in order to ensure that the state is able to create the next Silicon Valley or the next wireless hub," said Ramesh Rao, director of the UCSD division of Calit2. "This is critical to future economic growth – and jobs – in California, and we are hopeful that CREL will be able to engage with industry partners to ensure that our research breakthroughs translate into technologies for tomorrow’s marketplace."

The new center will support pre-doctoral and post-doctoral fellows, enable undergraduate and graduate students to connect research with their campus courses, and provide infrastructure for collaborative grant writing and research. "The rising interest in service learning, or creating environments for discovery-based curricula, might provide a timely response also to the crisis in funding for education," observed Dubnov, pointing to CREL’s broader potential impact.

Membership in CREL will be open to UCSD faculty and affiliated members interested in building a digitally mediated society. In addition to CREL director Dubnov and Calit2’s Rao, founding UCSD faculty participants in the new center include communication professor Morana Alač, psychology professor Diana Deutsch, as well as cognitive scientists Richard Belew and David Kirsh, all from the Division of Social Sciences; Natasha Balac and Chaitanya Baru from the San Diego Supercomputer Center; music professor David Borgo from the Division of Arts & Humanities; Philip Bourne, School of Pharmacy/Pharmacological Science; computer scientists Charles Elkan and Thomas Powell from the Jacobs School of Engineering; and affiliated member computer scientist Fionn Murtagh from Royal Holloway, University of London and ICT Science Foundation Ireland.

UCSD music professors -- percussionist Steven Schick and vocalist Phil Larson -- are surrounded by the audience while interactive chat and commentary play out on four large screens surrounding the audience.

The unique interdisciplinary nature of the center will permit a systemic look into novel methods of participatory media (i.e., involving group interaction). These methods will include ways to leverage audience back-channeling, crowd-sourcing and distributed moderation techniques for engaging the public in meaningful interaction with artists and educators.

Dubnov also notes that CREL will reach out to companies and communities that use the Internet and digital media, because they can benefit from the research to get their message across or to get feedback from their users or customers.

The new Center for Research in Entertainment and Learning will be housed in Calit2, which will provide the infrastructure and administrative capabilities to manage the establishment of CREL. "We also plan to work with different UCSD divisions and colleges in developing participatory and collaborative learning programs for students," said Dubnov. "We will also establish research collaborations with other research centers on campus which engage in social, educational and creative use of digital media technologies."



According to Dubnov, "CREL will also promote the integration of the center’s interdisciplinary research into the arts, entertainment and classroom programs."

External collaboration and support are expected from Centre for Computer-Supported Narrative and Semantics, IEEE Computer Science Technical Committee on Computer Generated Music (IEEE CS TC on CGM), which provides know-how of Standard IEEE 1599 (for multi-layered representation of information in music, and multi-media in general); Multi-Database and Multimedia Database Group of Keio University (Shonnan Fujisawa Campus, Japan); the Institute for Research and Innovation (IRI) of the Centre Pompidou (Paris, France); and the Institute for Research and Coordination in Acoustics and Music (IRCAM) (Paris, France); and Web development agency, PINT.com.

Related Links

Center for Research in Education and Learning http://crel.calit2.net
Calit2 http://www.calit2.net
IEEE Technical Committee on Computer Generated Music http://www.computer.org/portal/web/tandc/tccgm;jsessionid=4adb77c17f544daf93555230283c
MARCEL Network http://www.mmmarcel.org
IRI Centre Pompidou http://www.iri.centrepompidou.fr/
IRCAM http://www.ircam.fr/
Keio Multi Database and MultiMedia Database Group http://www.mdbl.sfc.keio.ac.jp