Cultural Criticality

As a broad interdisciplinary project, Cultural Criticality explores Complex Systems theory as the basis for broad public understanding of culture, technology and its political-economic implications. Using theories of information theory, growth of networks, coalition game theory, and cultural aspects such as the role of noise in political-economy and scapegoat theory, the project aims to provide an alternative foundations to understanding of the role of creativity, and ethics and aesthetics in AI and other emerging technologies.
The project revives and reinvigorates discussions of Hayek, Marx, Freud, and other philosophical approaches to understanding culture and society.

On relations between Political Economy and Music (from "Sound in Time" class):

AI-Assisted Nonverbal Communication

During the 2020-2021 academic year, students and faculty were displaced from their in-person learning environments, depriving many of the face-to-face interactions that facilitate strong communication, sense of community, and collaborative learning. Though the most contagious period of the pandemic may be settling, remote course offerings will continue to be offered, and students-- especially those involved in distance education- will still seek to build connections with their classmates and teachers.

Nonverbal communication plays a critical role in such connection, with eye contact being a primary modality. Beyond the social and conversational importance of gaze, understanding a person’s fixation is necessary to make possible any form of synchronous performance (particularly in the fields of music, dance, theatre, public speaking, and other performance arts). The problem easily extends to non-musical classrooms, too. Though students may elect to join lectures without offering their video, it is essential that professors provide their video to communicate through facial expression and gesture. Of course, a common occurrence in any classroom (virtual or not) is lack of participation when a professor asks a question to prompt student engagement. This is only worsened when the professor loses their modality of nonverbally requesting student engagement: eye contact.

Recent advances sponsored by the UCOP Innovative Learning Technology Initiative have allowed for development of course tools to aid in textual, captioned communication of gaze. This gaze communication prototype was used to augment student experience in Music classrooms, where nonverbal communication allowed for synchronous live performance with musician interaction and improvisation. In the existing prototype, a user’s gaze target can be broadcast to all participants as an overlay to the user’s video feed. For example, if three users (Anna, Bob, and Cecil) are in a meeting, and Anna uses the gaze augmentation system, then her video feed will display Bob’s name when she is looking at Bob, and Cecil’s name when she is looking at Cecil.

The video below includes project motivation, overview, demos, and future research goals:

Telematic Soundpainting: Restoring Nonverbal Communication to the Virtual Classroom