The Arts and Sciences of Collection Representation

home define research people courses workshops 2008 collection representation

at the ACM Computer-Human Interaction (CHI) '08 Conference, 5-10 April 2008, Florence, Italy

aim and topics

This one-day CHI08 workshop seeks to specify and investigate the contextualized operation of significant processes of collecting, such as selecting, combining, organizing, and distributing, within dynamic digital mixed media environments. Given that the delivery of information resources is ultimately experiential, what are the requirements for these operations with respect to representing collections for meaningful human and machine interaction and manipulation?

This workshop brings together artists, designers, curators, librarians, DJs, VJs, ethnographers, psychologists, semanticists, and system builders to share and develop new integrative methods for representing multimedia collections. The outcome of the workshop is a report containing the results of the workshop as well as the next steps in development. A follow up publication in form of a special issue in a journal such as TOCHI or Journal of Human Computer Interaction is planned.

This workshop builds on the results of the ACM Multimedia Workshop 'Multimedia for Human Communication - From Capture to Convey (MHC 05), which developed a general model of commonly carried-out tasks in information manipulation and consumption [9].

participants

This workshop invites the participation of artists, interface designers, graphic designers, art collectors, and developers working with samples, surrogates, and recombinant information. This includes:

collage artists working with physical or digital media
musicians and VJs who remix audio - video samples;
members of cultural institutions, such as museums and libraries, who are involved in selecting, preserving, annotating, organizing, and presenting media collections;
installation artists, tangible media makers, and developers of responsive environments and context-aware applications or applications that assemble elements from diverse sources;
ontologists, cultural theorists and ethnographers involved in collecting;
cognitive scientists who consider the cognition of associations;
system builders: designers, hackers, scientists, and engineers who make tools and applications to support processes of collecting and combining. This includes tools for sample-based composition, and for developing collections in any digital medium. It also includes applications that dynamically generate recombinant information from collections of information resources.

submissions | important dates

Please send a position paper of 2-4 pages in ACM conference format to andruid

cs.tamu.edu

Position Statement	Oct 3, 2007
Announcement of Acceptance	Nov 3, 2007
Workshop Date	April 6, 2008

background

Art and media provide a perspective on found objects as representations. Sampling of found objects has been used as a source of materials for composition for almost a century [7]. Samples can function simultaneously as sensory representations, and semantic and navigational surrogates. In that way a surrogate functions as "a replacement for an original item, … which gives some description of the item, and how it can be obtained" [2]. Typical examples of surrogates include entries in catalogues, bibliographic citations, Google result set elements, bookmarks or playlists, and video clips. Each surrogate does not only exist as replacement, but is also a significant object itself. People often form ideas and make decisions based on surrogates. They are visual, auditory, and kinesthetic. Surrogates operate semiotically, as signs, by affecting meaning through signifier - signified relationships [1]. As a unit of meaning, a surrogate can be recontextualized. An example is the work of hip-hop artists such as Public Enemy [10] who use samples of James Brown as surrogates in their work.

Rather than seeing collections as atomized lists of separate elements, recombinant information puts surrogates together to create connected wholes [6]. Units of meaning are assembled. The result then is a connected knowledge space. Its structure may differ depending on the aim of the user. A digital librarian most likely addresses surrogates as descriptive elements. A VJ or a curator might understand surrogates rather as conceptual and creative material, which enable organizing the material as an experience. End users need both of these perspectives.

Understanding the dynamics of recombinant information with respect to its establishing processes, such as collage [7, 13], montage, detournement [3], hypertext [10], and remix [8, 14], is essential to navigation in common knowledge spaces. It form the basis for retaining some identification of citations and finding the way back to source material, and also for representing new forms of knowledge. The outcome might be that the reader of the space is provoked into new interpretations of meaning. Recombination involves processes and techniques of selection, positioning, fastening, and transformative treatments

The goal of this workshop is to investigate commonly carried-out tasks and activities in practices of information authoring, distribution, and consumption. Essential questions that need to be answered include:

What are people doing with collections? What are their goals, needs, and desires?
How can representations support creative processes such as exploratory search [15], collection sensemaking [12] [4] and information discovery [5]?
What types of surrogates are in use? What types are possible? What range of syntagmatic (personal speech) and paradigmatic (institutional speech) expressions do they facilitate?
What metadata needs to be gathered to support information visualization, personalized context-oriented adaptation, and creative recombination, in addition to retrieval and navigation?
What issues of privacy are relevant and how can we address them?

workshop organizers

andruid kerne

andruid kerne is a research artist scientist focused on expressive interfaces, people collecting information, time-based media, visualization, real time distributed architectures, wearable physiological computing, and responsive environments. He is an assistant professor of Computer Science at Texas A&M University, where he directs the Interface Ecology Lab. Kerne’s output has been presented by the Guggenheim Museum (New York), ACM SIGGCHI, SIGGRAPH, Multimedia, Digital Libraries, and Hypertext, New York Digital Salon, ISEA (Paris), COSIGN, the Milia New Talent Competition (Cannes), Ars Electronica Center (Linz), the Boston Cyber Arts Festival, the Pan-African Theater Festival (Ghana), and the town square of the village of Anyako (Ghana). His work has been supported by the National Science Foundation, the Rockefeller Foundation, Dance Theater Workshop, the Spaulding-Potter Fund for Innovative Education, and the Texas A&M Humanities Informatics Initiative. He holds a Ph.D. in Computer Science from NYU, and an M.A. in Music Composition from Wesleyan.

frank nack

frank nack is a senior researcher at LIRIS, University Lyon 1, France. He is also an associated senior researcher at the Semantic Media Interfaces group at CWI, Amsterdam. He obtained his Ph.D. with a thesis on “The Application of Video Semantics and Theme Representation for Automated Film Editing,” at Lancaster University, UK. The main thrust of his research is on the representation, retrieval and reuse of media in distributed hypermedia systems, educational hypermedia systems that enhance human communication and creativity, computational assistance for the development, maintenance and usage of hypermedia systems and distributed hypermedia systems, computational applications of media theory & semiotics, automated video editing, interactive storytelling, and computational humor theory. He was member of the MPEG-7 standardization group where he served as the editor of the Context and Objectives Document and the Requirements Document, and chaired the MPEG-7 DDL development group. He organized various tutorials with a focus on media semantics and workshops at ACM MM, CHI and ICME. Frank is on the editorial board of IEEE Multimedia, where he serves as associate editor in chief and edits the Media Impact column.

alia amin

alia amin is a PhD student at the Semantic Media Interfaces, CWI, The Netherlands. She holds a post-graduate degree in User System Interaction from Technische Universiteit Eindhoven, The Netherlands, as well as a master in Communication and Media Engineering, Fachhochschule Offenburg, Germany. She has been involved with information presentation research for industries and universities projects for some years now. Her last industrial experience was working for the Customer Insights department at Vodafone, The Netherlands. Her field of interest includes mobile technology, semantic design and automatic presentation.

eunyee koh

eunyee koh is a PhD candidate at Interface Ecology Lab, Computer Science Department in Texas A&M University. Her research focuses on digital information collecting, multimedia information retrieval, and semantic clustering and visualization. She has various publications in her research area (ACM Document Engineering, JCDL, ED-Media, and ACM Multimedia), and she was a speaker at the 2006 World Futures Conference. Her last industrial experience was working for Motorola, which gives her experience in mobile technology. She graduates B.S. in computer science and engineering from Seoul National University in Korea.

references

Barthes, R. Mythologies, New York : Farrar Straus and Giroux, 1972
Burke, M., Organization of Multimedia Resources, Hampshire, UK: Gower, 1999.
Debord, G. Situationist Anthology, Bureau Public Secrets: Berkeley, 1981.
Graeber, R., Kerne, A., Henderson, K. ZooMICSS: A Zoomable Map Image Collection Sensemaking System (The Katrina Rita Context), Proc ACM Multimedia 2006, in press.
Kerne, A., Koh, E., Dworaczyk, J., Mistrot, M., Choi, H., Smith, S.M., Graeber, R, Caruso, D., Webb, A., Hill, R., Albea, J., combinFormation: A Mixed-Initiative System for Representing Collections as Compositions of Image and Text Surrogates, Proc JCDL 2006, 11-20.
Kerne, A., Sundaram, V. A Recombinant Information Space, Proc COSIGN 2003, 48-57.
Lippard, L., Dadas on Art, Englewood Cliffs, Prentice-Hall, 1971.
Manovich, L., The Langauge of New Media
Nack, F., ACM Workshop on Multimedia for Human Communication - From Capture to Convey (MHC 05): http://homepages.cwi.nl/~media/conferences/mhc05/mhc05.html
Nelson, T. Literary Machines, Watertown MA: Mindful Press | Eastgate Systems, 1982.
Public Enemy, It Takes a Nation of Millions to Hold Us Back, Def Jam, 1988
Russel, D., Stefik, M., Pirolli, P., Card, S., The Cost Structure of Sensemaking, Proc CHI 1993.
Spies, W. Max Ernst Collages: The Invention of the Surrealist Universe. NY, Harry Abrams. 1988.
Stockhausen, K., Elektronische Musik, 1952-1960, Kurten
White, R.W., Kules, B., Drucker, S.M., schraefel, m.c., Supporting exploratory search: Introduction, CACM 49(4), 2006, 36-39.