The Movie Tagger project (1) was initially inspired by media artist and researcher Michael Naimark’s grand vision of a future in which every movie ever made could be richly tagged and parsed with time-based metadata (Naimark et al. 2010). To help realize this vision, in the alpha stage of the project on which I have been lead researcher and which I will discuss below, Naimark and our team collaborated with Zane Vella and others at Related Content Database (RCDb – now Watchwith), a company focused on commercial applications of time-based metadata.

Previous academic research had used such tagging, or annotation, to help identify formal patterns in film and other linear media (Cutting, Brunick, and Delong 2011; Tsivian 2009; Butler 2009; Manovich and Douglass). By focusing on an entire film, or corpus of films, rather than a sequence, these projects represented a departure from the traditional methodological emphasis on close readings in film and media scholarship. But, even as they expanded the scope of analysis, these earlier metadata projects also typically constrained the focus of inquiry to very specific formal parameters like shot length.

More subjective interpretive strategies of close reading (like those we are most familiar with in film and media studies) are traditionally applied to a single sequence, and thus can be difficult to translate into formal tagging schemas applicable across multiple films or even across multiple sequences within a single film. Questions of form, context, and filmic interpretation cut to the heart of familiar debates in film studies, often pitting formalism against more theoretically driven frameworks of analysis—psychoanalytic, post-colonial, feminist, queer, or Marxist, to name just a few. (2) The Movie Tagger project engages these issues insofar as time-based metadata tags introduce implicit arguments about the relationship between form and meaning, and raise questions about the degree of abstraction versus context specificity required for analytical interpretation.

Critical interpretive frameworks, such as those that examine the relationship between ideology and form, do indeed present significant challenges for hybrid human-machine collaborations due to inherent tensions between computational modes of abstraction and human analysts’ ability to recognize context specificity. But even neoformalist approaches to understanding meaning are not intended to be context independent. Specific formal devices elude universal interpretation, and as Kristin Thompson writes:

It is risky to assume that a given device has a fixed function from film to film…. Any given device serves different functions according to the context of the work, and one of the analyst’s main jobs is to find the device’s functions in this or that context. (Thompson 1988, 15)

This recognition of context specificity points to the role of the human analyst in synthesizing meaning. Such an acknowledgement stands in contrast to much of the existing innovation in the area of film analytics which relies on computer vision to do the formal analysis of the film frame, detecting elements like shot changes, luminosity, chroma, composition and dynamism within a frame. Rather than focus on these sorts of computationally abstractable features, we were instead interested in combining human annotation with computational visualisation strategies in order to explore opportunities for alternative “modes of seeing” in media scholarship. In an attempt to defamiliarise the process of film scholarship itself, we wanted to put more subjective interpretive strategies (strategies that push beyond formalism) in dialogue with macro-scale approaches to tagging.

Folksonomic tagging platforms, such as the online image sharing community flickr, offer opportunities to discover unexpected connections among user-contributed media. By opening up the process of tagging media to a large audience of users, folksonomies enable serendipitous categorisation strategies to emerge that no one contributor could have predicted on their own. In this sense, they represent a collaboration of what might be termed ‘ontological effort’: the hashing out in public of what it means for something to stand in as a category for a larger set of things. For static images on flickr, for example, questions of ontology are implicit in an, often silent, conversation that plays out among large groups of people who, through aggregation, gravitate to particular semantic conventions. Furthermore, in folksonomies, tagging schemas ultimately take on a bi-directional relationship to the contents of the database. As Movie Tagger Alpha Principal Investigator, and Associate Professor at the University of Southern California’s School of Cinematic Arts, Steve Anderson has pointed out, rather than understanding metadata strictly as a supplementary and ex post facto practice oriented toward the organization, storage and retrieval of database contents, instead we can also think of metadata as having a performative dimension. (3)

But this kind of collaboration becomes more complicated when applied to temporal media. Time-based metadata not only involves an interpretive framework applied to a piece of media, but also implies a selection of temporal boundaries (“in” and “out” points). While in film studies, the ‘shot’ has often been defined as a primary unit of analysis, time-based metadata does not privilege particular kinds of temporal boundaries, and instead, can apply just as easily to an entire film or to a single frame. This openness means that the ontological work of time-based metadata often involves miniature theories about change over time. For example, when ‘in’ and ‘out’ points are coupled with interpretive metadata, the calibration of these points involves tacit assumptions about the beginning and completion of pro-filmic actions. And in this sense, time-based metadata makes itself available to verb-like sequential or causal frameworks of meaning, in contrast to the more nominative or adjectival tendencies of metadata associated with static images. Or rather, it is perhaps more correct to say that time-based metadata addresses modalities of both change and stasis while the tags of static images primarily accommodate the latter (and can merely suggest the former).

Responding to these unique challenges, a variety of projects (academic, artistic, and commercial) have been carrying out work that uses time-based metadata as on organizing principle. This eclectic mix of projects includes work by film scholars, information scientists, interactive cinema artists, and emerging web start-ups. In each case, the projects’ designers face questions about what and how to annotate in films.

A number of researchers have been developing analytical tools to enable large-scale research on formal features of time-based media. These projects include: Cinemetrics (Tsivian and Civjans), a platform for film scholars to record precise shot lengths while viewing in real-time; Shot Logger (Butler), a platform for annotating visual style in film and television; and the suite of projects produced by the Software Studies Initiative under the title Cultural Analytics (Manovich et al.). These works visualize large datasets of media (films, animations, video games, comics, artwork, web sites, etc.), and they have collaborated with Cinemetrics to visualize how shot length has evolved in different periods and geographical centres over the history of film.

Another set of projects utilises time-based metadata primarily as a way of organizing database narrative engines. In such projects, metadata categorisation supports the recombinant rules that enable smaller clips to be reorganised with each viewing. Database narrative research includes work by the Garage Cinema lab (Davis 2003), the Soft Cinema project (Manovich and Kratky 2005), and works by the Labyrinth Project research initiative (Kinder et al.). More recently, a set of projects under the title Enactive Cinema uses the spectator’s emotional participation to drive live editing decisions, relying on viewer bio-feedback to alter the course of the project’s narrative trajectory (Tikka 2008).

A further set of projects focuses on the clip as a unit of analysis. In these, metadata is used in a more curatorial role (as tags that enable users to discover connections within a film or across multiple films). In Critical Commons, tags serve as scaffolding for the opening up of critical commentary in the academy. A project by Steve Anderson, the website serves as a repository for connecting film clips and critical analysis. Utilised by film scholars, educators, and media makers alike, the project represents an unprecedented assertion of fair use for the public redistribution of copyrighted media in critical contexts (Anderson). (4)

Alongside these academic projects, commercial websites are also creating clip-based repositories in an effort to capitalise on social interactions among fans. For example, Anyclip enables users to find clips of interest by searching for keywords in a script and emphasises sharing through other social media sites. Movieclips seems to have a somewhat larger content base and focuses on curated categories oriented to film buffs. It should be noted that both Anyclip and Movieclips currently constrain the metadata categories available to those imposed by the sites’ creators—in other words, the antithesis of a folksonomic tagging structure.

In addition to these existing projects, this alpha phase of the Movie Tagger project sought to draw, also, from the rich community of filmmakers and film scholars at the University of Southern California (USC). Using this academic community as a test bed, we shifted away from a traditional notion of crowd sourcing to explore instead models that might more aptly be described as “expert sourcing” or “partner sourcing.”  It should also be noted, that this phase of research was not specifically focused on the mechanics or user interfaces that might support a sustained collaborative platform, but rather we were interested in exploring more fundamental questions about what would happen if we remapped existing interests of filmmakers and scholars to the research opportunities afforded by time-based metadata. What questions does this approach help answer? What sorts of research questions might it open up that we could not have anticipated?

Movie Tagger Process / Methodology

In the early stages of the project I interviewed a broad range of faculty to explore a variety of approaches to metadata annotation. Interviewees ranged from prominent industry figures and innovators to academic film scholars, and accordingly their interests and research foci ranged across a variety of topics. A total of twelve interviews were conducted including eight with USC faculty (Steve Anderson, Tara McPherson, Ellen Seiter, Henry Jenkins, Jeremy Kagan, Norm Hollyn, Tom Holman, Midge Costin), two with either current or former USC graduate students (Chris Hanson and Jesse MacKinnon), and finally two with external faculty whose research intersects directly with metadata analysis of film (Jeremy Butler from the University of Alabama and Eric Faden from Bucknell University).

My role as interviewer was, in some ways, also that of a cultural ambassador or translator. The process involved working with faculty to map their insights and research questions onto possibilities of metadata analysis and data visualisation. For example, we might start from a scholar’s interest in a subject like the evolving representations of ‘work’ and ‘race’ in a contemporary television series set in the American South (as was the case for Tara McPherson, who was preparing to write about the television series True Blood). Within the interview we would try to unpack how this interest might be adapted to a set of time-based tags that could be applied over an entire episode or corpus of episodes. In this example, tags might include specific workplace locations, racial identification of characters, and additional information about whether a character depicted a service worker, physical labourer, or other kind of worker. This faculty model would later be tested and further refined by our team once we had begun the process of annotation.

While some of our faculty informants, like McPherson, were quick to understand the research potential of aggregating data, for others this approach needed further explanation, given that most of the interviewees were accustomed to researching and thinking about films in terms of a methodology of close reading. Accordingly, this stage often involved a process of disciplinary translation in order to expand the perspective of existing research interests from a micro- to macro-analytical lens. As a testament to the open mindedness of our faculty collaborators, though, each of the interviews resulted in a variety of potential metadata models.

A small but dedicated team of undergraduate researchers from USC’s Institute for Multimedia Literacy (including Jason Lipshin, Corianda Dimes, and Kera Kadir) were paid hourly to take on the Herculean task of watching and exhaustively annotating select movies with time-based metadata tags. This process involved the typing of tags into a spreadsheet while watching a film or television series. During early stages of the project, we chose to use ordinary spreadsheet formatting as an intentionally basic annotation platform. Later, as we iterated with our commercial partners at RCDb, (5) we used spreadsheet templates and, in later stages, adopted customizable notation to help facilitate data visualisations. Our batch of test films included 5000 Fingers of Dr. T, Harry Potter and the Sorcerer’s Stone, Minority Report, The Nightmare Before Christmas, Singin’ in the Rain, Strange Days, True Blood (episode 1), Vertigo, West Side Story, and Written on the Wind.

In the early stages we deliberately held off on imposing rigid tagging protocols in order not to foreclose unexpected discoveries. Accordingly, metadata coders were encouraged to experiment with more open tagging strategies as a way of exploring the conceptual terrain of each media work under analysis. So, for example, the concept of ‘race’ might more loosely be connected to the ‘race’ of vampires in True Blood or to racial tension between vampires and humans, and taggers organized their metadata schemes organically to accommodate such ontological anomalies. As we started to hone metadata models derived from faculty research, and as we developed more specific data visualisation strategies, the coding format gradually became more standardised. For example, at a certain point, we constrained the annotation to specific tags and particular contexts in which they should be used. Nevertheless, throughout the process, insights and observations of the coders themselves served an important role in helping us to refine, challenge, and iterate on the faculty-inspired metadata models. As our team worked, they became increasingly adept at identifying and addressing anomalous instances that challenged our existing tagging ontology. In this sense, it should be pointed out that they progressively became a group of expert taggers, an ideal complement to our faculty informants. Moreover, as they conducted tagging, they would frequently meet with other members of the Movie Tagger team to iterate on tagging schemes, and revise or revisit the conceptual assumptions behind our faculty models.

Finally, in collaboration with Eddie Elliott at RCDb, we developed a series of data visualisations in order to explore our annotated films. Some of these visualisations also served as interactive portals to the films themselves, so that as one scrubbed through an annotated timeline, corresponding moments from the film would play. Other visualisations deemphasised the temporal dimension and instead focused on mapping aggregate data about relationships between characters in a film.

Examples of Deeper Metadata Exploration

Many of the faculty interviews provided inspiration for metadata tagging, but the research of two film scholars in particular, Steve Anderson and Henry Jenkins, went on to serve as instigation for deeper exploration. Both of these scholars have used close reading of individual clips as a launch pad for critical analysis. While each began his academic career under the tutelage of mentors steeped in the traditions of formal analysis, they subsequently developed research trajectories that took them beyond the boundaries of traditional film scholarship into areas of emerging media practice. In this sense, they were ideally positioned both as ethnographic informants and experimental collaborators, able to operate in a hybrid mode of inquiry.

In his research Jenkins has investigated, inter alia, representations of adult-child relationships in media (Jenkins 2003), so, based on his work, tagging schemas were developed to map, for example, the occurrence of intergenerational affectionate and aggressive touch and language in the films 5000 Fingers of Dr. T and Harry Potter and the Sorcerer’s Stone. Meanwhile, Anderson’s research has explored the representation of virtual reality in Hollywood films, including the way these technologies are used to remediate historical events such as the Rodney King beating. This interest led to an exploration of how race, gender, crowds, and police violence are depicted in the film Strange Days. With his expert input, tagging schemas were developed to understand the representation of crowds and police in Strange Days, and later, in West Side Story.

To begin with Anderson’s input, he had been interested in how Strange Days, like other movies, enacts repeated associations of new media technologies (in this case the Virtual Reality [VR] “wire”) with violence— in this case against women and African Americans. Through the interviews with Anderson, we also became interested in studying the way that the film depicts crowds and bodies in motion, in the hope of understanding how that dimension evolves over the course of the film. Researcher Jason Lipshin adapted these foci to a metadata schema for annotating bodies in movement that distinguished between dancing bodies and mobbing bodies.

During this early stage of the project, we also encouraged experimentation with a wide range of annotation topics. The tags for Strange Days included ‘police,’ ‘virtual reality (for references to the VR “wire” in the film), ‘brutality towards women,’ ‘violent,’ ‘non-violent,’ ‘dancing bodies,’ ‘unrest,’ ‘sexualized,’ and so on. These tags were then used by as indices for an interactive iPad visualisation. [Figure 1] shows a screen shot of the interface dial using the touchscreen of the iPad to scroll in realtime through the movie, allowing us to access any point in the movie that had been annotated by a particular tag. Using this iPad interface we can verify that the presence of the VR wire on screen clearly and consistently correlates with violence and sex, which is a longstanding trope of virtual reality in the cultural imaginary.

[Figure 1] iPad app utilizing circular timeline that visualizes the tags collected for Strange Days and plays clips when the playhead overlaps a tag (for example the two tags in red above).

In addition, the iPad interface helped us to identify instances throughout the film of dancing, celebrating bodies that later shift to mobbing bodies (often indicated through tags of ‘Racial Protest’). In between these two phases we often saw tags for ‘LAPD’ or ‘Police Brutality,’ suggesting that presence of police consistently preceded or exacerbated violent unrest, often transforming dancing bodies into violent mobbing bodies. To illustrate a single example, [Figure 2] depicts a crowd of dancing bodies at a Y2K new years celebration.

[Figure 2] ‘Dancing Bodies’ in Strange Days

The image below depicts a crowd from the same scene but at this stage they have become violent. Notice the police in the mid-ground clubbing a person in the mob.

[Figure 3] ‘Police Brutality’ in Strange Days

In the case above, we see a direct inciting of violence by the police—a scenario that was highlighted by metadata tags but would have been accessible through close analysis. In other cases, however, the relationship appeared less obvious (police just “happened” to be cut in sequence, sometimes at an entirely different location from those dancing). Regardless, the police seem to be serving the thematic role of precipitating violence, a reading that is supported by the historical context of amplified racial tensions between LAPD and African-Americans in post-Rodney King Los Angeles. By analysing the tags in this way we deliberately avoided distinguishing between causal and non-causal forms of sequentiality, but in doing so reveal larger patterns and open up new questions for close-analysis. Such broader structural patterns suggest the potential of data-aggregation across a much more extensive data set—for example, mapping the impact of police presence across time in dozens or hundreds of films and then looking for correlation with historical statistics such as the incidence of police brutality in specific contexts. Along these lines, we began to experiment with using similar tagging schemas across multiple films.

Building on what we had learned from our experiments with tagging Strange Days, we wanted to explore the themes of dancing and violence in a radically different genre of film. We chose West Side Story in part because of similar themes of dancing bodies and bodies in violence as well as the presence of police.

[Figure 4] Linear timeline visualisation of tags in West Side Story

This time we also decided to look at intensities of dancing to see if that would reveal any consistent patterns. We also chose to experiment with linear timelines, this time as a way of exploring more detailed relationship between tags in sequence. In contrast to Strange Days, in West Side Story the police often precipitated the breaking up of the rival gangs. When the LAPD is tagged in Strange Days, subsequent sequences demonstrate greater chaos, violence and dynamism in the frame, whereas in West Side Story, the police have a consistently calming or mollifying presence on the movement of bodies on screen. In [Figure 4] above, we see that the presence of police (in yellow) corresponds, in many cases, either to a subsequent drop in high-intensity interracial dancing (purple), to a cessation of interracial conflict (black), to an interruption of interracial love (pink), or to some combination of these factors.

[Figure 5] Cops breaking up dancing/violence in West Side Story

In thinking through the implications of this comparison between Strange Days and West Side Story, Anderson reflected upon the importance of reflexivity in our tagging scheme, which he suggests enabled the discovery of emergent analytical vectors and serendipitous sites of comparison. Strange Days and West Side Story would surely have otherwise remained segregated by genre and chronology, but by juxtaposing them we can start to see the potential of extending this analytical vector to include a great many more films dealing with related themes. Through these unlikely connections, time-based metadata not only encourages the unmooring of context for particular formal features or pro-filmic actions but also productively defamiliarises the demarcations of genre, subject matter, and historical period.

Connecting these observations about our reflexive process to the productive pairing of micro- and macro-analytic frameworks, Anderson also argues that the coexistence of close reading and aggregated analytics across two or more films could represent a “cinematic corollary of the type of ‘distant reading’ advocated by literary historian Franco Moretti (2007). Like Moretti, this mode of analysis seeks to derive insights that are both highly detailed about the contents of a single work (the literal remediation of George Holliday’s video of the King beating in Strange Days, for example) and concurrently embedded in a context so broad that its contours may only be legible with the aid of computational processes.” (6)

To turn next to Movie Tagger’s collaboration with Henry Jenkins, this internationally renowned media scholar has written about the advent of the permissive parenting movement in the 1950s in relation to pop culture, in particular examining the interactions between adults and children in the film 5000 Fingers of Dr. T, an adaptation of one of Dr. Seuss’s stories (Jenkins 2003). Jenkins points out how the film illustrates a tension between opposing philosophies of parenting, one permissive—­epitomized by Mr. Zabladowski—and the other domineering—embodied by Dr. Terwilliker. Taking a cue from the themes of Jenkins’s research, we were interested in aggregating metadata about communication between adults and children. In particular, we looked at how authoritative and permissive language towards children played out in the film and at whether aggressive or affectionate touch between adults and children might also serve as a key signifier. (7)As we refined our metadata model, we found we wanted to specify the actor and receiver of the action. So we included information about which party was initiating the touch and which party was the receiver. Likewise, we distinguished between speakers and addressees in examples of authoritative and permissive speech. The image in [Figure 6], below, depicts an example of Dr. T. waking up Bart out of a nightmare. This was tagged as an example of ‘aggressive touch’ (from Dr. T to Bart).

[Figure 6] Aggressive touching in 5000 Fingers of Dr. T.

[Figure 7] Authoritative language in 5000 Fingers of Dr. T.

In Bart’s dream world Dr. Terwilliker becomes an autocratic figure who commands an army of piano playing pupils. The image in [Figure 7] depicts a moment that was tagged as authoritative language.

[Figure 8] Permissive Language and Affectionate Touching.

Mr. Zabladowski also figures in Bart’s dream world. In [Figure 8] he is shown in a sequence that was tagged as both affectionate touch and permissive language.

[Figure 9] Network graph illustrating authoritative and permissive language over the entirety of 5000 Fingers of Dr. T.

In [Figure 9] we visualised authoritative and permissive language using a graph that illustrates network relationships. Red signifies authoritative language and green signifies permissive. We also indicated speaker and addressee by varying opacity. Closer to the addressee the opacity is highest while as the joining line approaches the speaker the opacity decreases.

Using the above graph we can make observations about which adults are speaking most authoritatively to Bart (with Dr. T, and to a lesser degree, Bart’s mother both speaking more authoritatively than Mr. Zabladowski). We can also observe that Bart himself is speaking permissively to Mr. Zabladowski, perhaps revealing an implicit claim the film makes about the effectiveness of permissive parenting strategies.

[Figure 10] Network graph illustrating affectionate and aggressive touch in 5000 Fingers of Dr. T.

[Figure 10] depicts the distribution of touch (coded as affectionate, aggressive, or neutral). Additional information is noted if the touch is mutual, attempted (but failed), or magic. As in the authoritative and permissive language network graph, this visualization distinguishes an actor from receiver of touch by varying opacity. Closer to the touchee the opacity of the joining line is highest while, as the line approaches the toucher, the opacity decreases. From this data visualisation we can observe that both Mr. Zabladowski (Mr. Z.) and Bart’s mother exhibit a combination of affectionate and aggressive touch towards Bart while Mr. T. exhibits instances aggressive touch only.

We were interested in understanding how a metadata tagging scheme might be ported to another film. Given Henry Jenkins’s research on Harry Potter fandom, and given the very different take on parental authority that the Harry Potter stories seem to offer, we decided to use the film Harry Potter and the Sorcerer’s Stone as a test-subject for the same metadata framework that we had employed for 5000 Fingers of Dr. T.

In contrast to 5000 Fingers of Dr. T. the Harry Potter story world accommodates authoritative language from adults who are depicted warmly. For example, in [Figure 11] below, Professor McGonigal, a stalwart ally of the children throughout the story, can nevertheless be seen sternly lecturing Harry.

[Figure 11] Professor McGonigal using authoritative language

At the same time, the figure of Hagrid, relates more permissively to Harry and also frequently initiates affectionate touch, as in [Figure 12], below.

[Figure 12] Hagrid demonstrating affectionate touch towards Harry

The contrast between McGonigal and Hagrid, both of whom represent beneficent forces in Harry’s life, suggests that Harry Potter and the Sorcerer’s Stone would have a more complicated relationship to the adults in his life (compared to Bart in 5000 Fingers of Dr. T.).

The data visualization in [Figure 13], below, demonstrates that Hagrid and, to a lesser degree, McGonigal and Dumbledore express permissive language towards Harry. Authoritative language towards Harry comes from his adopted parents (Uncle Vernon and Aunt Petunia), McGonigal, Quirrell, and Filch.

[Figure 13] Network graph illustrating authoritative and permissive language in Harry Potter and the Sorcerer’s Stone.

Instances of permissive language towards Harry from Snape and Voldemort point to anomalous cases in which sarcasm (in the case of Snape) and bargaining (in the case of Voldemort) created tensions for our existing tagging schema. Rather than see these anomalous examples as failures, however, we found them to be important opportunities for revisiting close analysis.

[Figure 14] Network graph illustrating aggressive and affectionate touching in Harry Potter and the Sorcerer’s Stone.

Similarly [Figure 14], a network graph illustrating instances of touch contains examples in which pro-filmic actions created ontological tensions for our tagging schemas. In particular, we found anomalous cases of touching that did not easily conform to our metadata categories.

[Figure 15] Harry Potter touching (and being touched) by Quirrell/Voldemort.

For example, in [Figure 15] Harry attempts to remove Voldemort’s hand (which is actually the hand of Quirrell, whose body has been possessed by Voldemort). Harry does not realise that his power over Voldemort will result in Quirrell’s hand getting burned. In such a case, a number of questions emerge: Which character, Quirrell or Voldemort, is the receiver of the touch? According to metadata tags, both Harry and Quirell exhibited instances of aggressive touching towards one another, as illustrated in [Figure 14]. But is this example of touch aggressive, since Voldemort is being harmed, or neutral, since Harry is not aware that he has this power yet? Similarly, in other moments in the film, characters frequently “touch” one another remotely through their wands. Should such instances count as touching?

Such interpretational questions point to issues of abstraction and context specificity that I raised in my introduction. The lens of close analysis can help us to probe anomalies in the data and productively complicate our tagging framework by pointing out gaps and ontological tensions. Casting these anomalies as “merely” anomalous, then, would miss the point, for they speak to the important role that close analysis can play in critically re-examining the assumptions behind macro-analytic frameworks of data visualisation. Rather than seeing close-analysis and metadata as incompatible, we should emphasize instead the valuable dialogue that can occur at the intersection of micro- and macro-analytical lenses. The metadata collection that followed on from our collaborations with Anderson and Jenkins, then, led to revealing data visualisations and also provided a rich resource for revisiting these scholars’ topics of close analysis. Since computationally assisted “seeing” can risk obfuscating the very processes of meaning making that close analysis is adept at identifying, our experience is that pairing the two methodologies helped to challenge our data aggregation models to attend more closely to the hidden assumptions behind particular schemas of categorisation.

Conclusions and Future Directions

This phase of the Movie Tagger project can be understood as a design intervention in two very different domains. First, we took an existing approach to visualizing metadata that our partner company RCDb had developed with commercial contexts in mind—for example, using metadata to track product placement—and we adapted their architecture to serve as a tool of ideological analysis. Second, the project represents an intervention into the existing methodologies of film and media research. Along these lines, Anderson has flagged up the following two points as indicative of the transformational impact that the process of thinking through metadata has had on his own scholarship:

(1) Once you start thinking in terms of tagging movies in real time, you don’t ever really go back. My mental model for movie watching for the past year has been the construction of metadata schemes and cognitive databases for all media that I take in.

(2) I’ve begun to take seriously the merging of data with metadata when it comes to the massive database that is the history of film and television. It’s only a matter of time before we have the realistic ability to have metadata function as the primary viewing interface for what may be considered One Big Movie – that movie being everything ever committed to a format that can be accessed by the metadata. (8)

Anderson’s point about “One Big Movie” suggests a world in which metadata authoring, manipulation, and visualisation becomes a taken-for-granted, always available, norm of viewership. Such an emerging media landscape would open up as yet vastly unexplored areas of research. But it would also mean that questions about how to strike the right balance between context specificity and abstraction would no longer be confined to academic debates about meaning and criticality. Likewise, the commercial embrace of Big Data (9) also represents an unmooring of interpretive context; the attendant ontological tensions created by porting metadata frameworks from one domain into another may become a feature of our everyday media consumption. As humanists we have an important opportunity to put our own analytical toolsets in dialogue with the rhetoric and machinery of data visualisation, to exhume ontological tensions from metadata frameworks, and to demand that critical frameworks such as gender, race, power, ideology, among others, be addressed by emerging tools of media research.

Endnotes:

(1) Michael Naimark originated the Movie Tagger concept and his leadership guided the project through each stage of development. Additional founding members included: Steve Anderson, Maya Churi, Perry Hoberman, Andres Kratky, Erik Loyer. Earlier phases of the project have involved collaborations with Scott Fisher and the Mobile and Environmental Media Lab.During this research phase, Naimark acted as the primary liaison with Zane Vella, of RCDb, a company that is pioneering commercial applications of time-based metadata. Steve Anderson also guided and facilitated the research in his role as Principal Investigator for this phase of the project.

…[A] fascination with archives and their aura of absent presence is unsurprising, their parallel uncertainties offering a perfect exchange of allegories. […]

[A]rchives remain sites for inviting theft, revision or the discovery of unexpected truths; respected arenas for retrieval of what was hidden and the always-engaging investigation of who has been left out and why. [Frenkel, 2011: 163, 166]

Film archiving is a rapidly changing field as a result of the accelerating development of online digital technologies. Taking as its case study the example of SP-ARK (1), the Sally Potter online film archive, this article proposes a notable shift from the traditional single-user archive model to emerging multi-user, collaborative forms of archival scholarship. The digital preservation and presentation of archival materials dramatically impact upon the nature of the types and levels of access to primary film materials and their associated ephemera that are afforded. Moreover, the nature of the discoveries, insights and findings that can be made through online digital interfaces are radically altered. In what follows, I will explore the SP-ARK model, both through close analysis of the archive itself, and also through an empirical user study of students who access the archive for their studies. I will argue for a re-thinking of archival process design and will contend that, in an ever-shifting digital landscape, the archival planning for future feature-film projects might be usefully considered at the earliest stages of the production.

Traditionally, the primary sources generated through the creative process of feature filmmaking are housed within specific physical locations that are not easily accessible and not always open to the public. In the UK, the national film repository is the British Film Institute (BFI), some of the materials are available to view online, but only those that are not restricted by copyright issues. The BFI are currently working on their latest BETA version of a searchable archive. (2) There are also a growing number of both subscription-based and open-access national broadcast archives online. In the UK, these include Box Of Broadcasts (BOB) (3) a subscription-based off-air recording and video archive service delivered by the British Universities Film and Video Council (BUFVC) (4) and EU Screen (5) and Europeana (6) in Europe. There are also the twelve public sector regional film archives across the UK. These house audiovisual assets such as newsreels, reports, documentary footage from television broadcasts alongside amateur footage of historical value. The equivalent online digital archives within the fictional and dramatic realm of cinema are extremely limited, yet could be invaluable tools in the maintenance and preservation of cinematic and cultural heritage. As Gerhardt and Kaufman (2011) have noted:

This disconnect – perhaps we call it an ‘A/V gap’ – is largely a function of attitudes and behaviours within teaching, production, and publishing. It is also an outcome of the paucity of high quality audiovisual work now available for educators. As we note in our 2010 Film & Sound Think Tank film, Knowledge Is…, (7) despite the leading investments of JISC and others worldwide, only 5% of our audiovisual history is digitized and available to educators and the public online (2011: 3).

 The recent film review undertaken by the Department of Media, Culture and Sport (DCMS) in the UK also supports this view. It concludes: ‘The percentage of material readily accessible by the public in National and Regional Archives remains very small in proportion to the size of the collections. Most of the film material held in archives around the UK is owned by rights-holders who understandably will only make their material available in a way that is consistent with their commercial interests’ (DCMS, 2012).

In addition to those archives listed above (BFI, BOB, EU Screen, Europeana, Regional Film Archives), there are a very limited number of repositories that house the culturally rich resources of fictional filmmaking across the globe. In the UK, the only known openly accessible online version of such an archive is SP-ARK. Internationally, in Japan the entire archive of prolific Director Akira Kurosawa has been uploaded and is freely available to view, access and download online, but is only currently available in the Japanese language. (8) These exemplars are invaluable and enriching resources for film education, since in their exposure of all the materials related to a film production (photographs, video assist footage, casting recordings, scripts, storyboards and developmental paperwork) they reveal the often hidden creative practices of fictional filmmaking. As Mayer contends in relation to SP-ARK, ‘It reflects the dailiness of labour involved in filmmaking as opposed to the heroic narrative portrayed in mainstream films’ (2008: 201). Within educational contexts, access to such primary resources facilitates close textual analysis and in-depth examination of films – learning practices that could not be achieved without such access. Such resources also have the potential to provide unique and unprecedented sites for communication, collaboration and the establishment of both online and physical networks. As the DCMS report highlighted, it is the key issue of licensing that appears to be foreclosing the development of open access to film and cinema resources. Although this continues to be a problem, fortunately it has not been so for the case of SP-ARK, since the copyright to all of the materials in this archive belongs to Sally Potter and her production company Adventure Pictures (AP). They have chosen to allow access and use of the materials via a Creative Commons licensing model (more specifically the materials are made available through a BY-NC-ND (9) license).

SP-ARK is an interactive online project based on the multi-media archive of world-renowned filmmaker Sally Potter. Potter’s work features across the curriculum and syllabus of many university courses, both in the UK and abroad, in relation to the subject areas of film, media and literature. Potter herself is a key figure of study within these disciplines (see Appendix 1 for a list of her films, see also Mayer [2008, 2009] and Fowler [2009].).

With her 2004 project Yes, Sally Potter was one of the first film directors to utilise the opportunities offered by blogging and online communication through message boards throughout the making of a film. Potter posted video poems and responses to events throughout the film’s production. The message boards enabled fans to communicate with and respond to Potter online. This approach exemplifies Potter’s interest in nascent technologies to inform her filmmaking practices. (10) This interest, coupled with the nature of AP (a small, independent production company) that could not respond to the many requests that it received for behind-the-scenes access to materials, provided the necessary impetus and motivation for the establishment of the SP-ARK archive. At this time Potter’s work was also increasingly being featured and referenced within higher education curriculum, and so students were also joining the call to access Potter’s extensive archive of storyboards, photos, casting videos, contracts, to name but a few of the materials. This was evidenced in the high volume of letter and email requests that AP received on a daily basis. AP perceived the traditional archive model as rather old fashioned in regards to accessibility. It was clear from the online response to Yes that fans of Potter’s work were also enthusiastic about the open and public exchange of ideas. The concept of SP-ARK was publicly launched in July 2007 at the Society for Cinema and Media Studies (SCMS) conference in Chicago.  (11)

Figure 1: The initial SP-ARK Prototype (2007)

The prototype pictured in Figure 1 included access to 600 archival items from the film Orlando (1992). The SP-ARK website was built and hosted using the WordPress content management software system. It facilitated the navigation of the archive via its pioneering ‘pathways’ feature, which meshed the principles and practices of blogging with archival browsing. This built on the idea of adding value and provenance to the materials by using social networking to enable users to talk to one another about what they discovered. Users were able to build their own unique ‘pathway’ through the archive’s content as they explored a particular theme or process, and they were able to save items that they had viewed.

This type of archival browsing, which is embedded into the infrastructure of the site, is obviously not achievable through the boxed (and un-sign posted) presentation of materials within a traditional physical archive. Intuitive browsing is extended and supported digitally by the affordance that users are able to annotate each individual item in their pathway, with their own comments, observations and streams of thought, as well at to describe and save the pathway itself. Other users are then able to access each other’s pathways (when they click on an item all associated pathways are displayed – see Figure 2) and to link to them, which offers a further level of user-led archival exploration. Users are also able to directly communicate with one another using the messaging tool. In previous research, Dr Charles Drazin from Queen Mary, University of London (QMUL) noted that ‘from a teacher’s perspective what was great about the site was to be able easily to visit students’ pathways and to see their thoughts take shape. It facilitated the provision of ongoing feedback as students worked on their assignment in a way that is not feasible in traditional coursework’. (12)

In February 2012, after an extended period of development and the archiving of further materials from the film Orlando, the latest version of SP-ARK was released to include a revolutionary ‘Visual Browser’ (which, for the first time in a film archive, allows users to visually analyse the film’s original rushes, from single frames to complete shots, and incorporate these in their pathways – see Figure 4; see here for a short video that sets out the new features), (13) social network integration (Facebook, Twitter etc), the pathways tool, and access to over 4000 assets. Users are able to view clips from the film as well as a myriad of associated materials including the scripts, storyboards, still images, location and developmental paperwork.

Figure 2: Multiple levels of browsing the archive: From left to right – The Production Process Taxonomy, ‘Used in pathways’ links, Search Field, Tags and Related Items/Videos.

The current version of SP-ARK has deployed standard web development and is built in the WordPress programming language PHP. It is a bespoke development (as is it is not based on any other technology) and uses the open source MySQL database managing system. The archive’s descriptive information (texts, tags, asset descriptions) is all held in this database and the assets are handled as ‘flat files’. (14) User information (name, institution, email address – which are generated when a user requests a login to the site), and user pathways are also stored in the database. The site supports multiple levels of browsing and searching which are all accessible without a login to the site. (The login enables users to create pathways and to message other users). Users are able to access materials through a taxonomy-led approach (15) through advanced text search fields, the assignment to each asset of a number of descriptive one-word tags, (16) related material links (see Figure 3), the pathways and the Visual Browser (see Figures 4 and 5). Once an archive asset is opened, users are then able to continue browsing the archive in a non-linear and exploratory fashion by clicking on the links to the asset’s ‘related items’ (every SP-ARK item is linked to other associated items). For example, a page of film script is linked to a corresponding film clip, which could then be linked to a call sheet, a continuity report, production design images, location notes etc.

Figure 3: Block scheme of the interactive image browsing system

Figure 4: The Visual Browser (2009) Visual browser representing semantically coherent content from the SP-ARK archive

Figure 5: The Visual Browser embedded into the SP-ARK interface

In order to explore user interactions with the archive within an educational context, I conducted a focus group with a class of 20 year-2 undergraduate US-based Film Studies students who were undertaking an assignment that involved exploring the nature of the archive within the context of the Film and Digital Age. The focus group transcript was coded into five key themes that emerged throughout the open discussion. Students were asked to highlight and discuss the positive and negative aspects of SP-ARK that they encountered whilst undertaking their assignments. The key themes identified were: the digital archive model; interpretation; openness; accessibility; and social learning. Findings are summarized below.

The Digital archive model

Students immediately began to discuss issues around the navigation of the archive as they had the task of finding one particular asset that inspired them.

[Student] We started going through personal notes and going through the written part of it, and that got really interesting because you started seeing it in a different way, rather than just watching it visually. I don’t know, it just brought it to a different level.

The students clearly expressed a passion for being able to explore the film of Orlando in this way.

[Student] In comparison to like a physical archive, was the amount of ways that everything was organized […] I just like that you found overlap a lot easier, whereas if you’re looking at things physically, it would take a lot of effort to actually find how everything overlapped.

This observation regarding the organizational structure of the archive illustrates SP-ARK’s intentional interface design as I have outlined it.

[Student] I felt if I was in a physical archive, I wouldn’t be able to keep referring back to what the final product looked like.

Students had previously attended a seminar led by the British Film Institute, and so had a clear understanding of the nature and limitations of access within physical film archives.

[Student] Now becoming involved with the visual information, it’s no longer just going onto JSTOR [academic platform] and reading an article, but we’re actually really seeing the clip from the film, from an out-take.

Students also clearly appreciated the ability to refer back to the source material as they correlated and contrasted the different sources.

Interpretation

The students reflected on their very first encounter with the archive:

[Student] I think we just got really excited actually, […], because we were like, look at this, did you know that this was supposed to be in there, and all of a sudden we had all these new understandings of the film and we were kind of interpreting the film on our own.

The students conveyed a clear appreciation for the empowering nature of the archive whereby they were enabled and encouraged to make their own discoveries.

Openness

The open nature of the archive was also a key point of discussion as students had not experienced anything similar before:

[Student] To actually see these primary materials, it’s such a unique experience, because I’ve never had a chance to actually see a director’s notes on their first ideas of a film.

Accessibility

The negative points raised in the discussion were mostly based on difficulties in accessing and searching for items:

[Student] I’m not a film major so going into it I didn’t really know what to look for, and I think what might have been helpful to me would have just been a list of all the tags for me to look at, and figure out what actually is up here.

[Student] I kept trying to search within the script or within notes that you can only search like title there aren’t many tags there, so I had to go through and like read through the pages, it just got really tedious, so I don’t know if its possible for you to like search for words within the script.

This comment highlights the known issue of the limitations of the searchability of the analogue source materials. Orlando was produced in 1992, in a period where documents had been type written. Where documents had been digitally produced, only the paper-based versions remained, and so were scanned in as image (and therefore not text searchable) files.

[Student] What would have been really helpful for me if you had like the scene you’re looking at, that maybe surrounded by all the papers, the script for that scene, the call sheet for that scene, so like you could put right next to each other the clip and then all the physical papers.

This comment reinforces the original intention of SP-ARK to have this capability built within the interface, as demonstrated in the prototype that was shown at the Media Festival Arts in 2010 (Sheppard, 2010).

[Student] If the visuals could be bigger and more of a focal point, I think it would allow them to be more accessible to a student […] I think of when I go into iTunes, and I can see a huge album cover, or the cover of a DVD and the pictures the first thing I see and if I want more information on it the text comes afterwards.

[Student] I didn’t know the specific tag to begin with, so when I was typing in dress or just things like, I was thinking about like Google, or costume or wardrobe, I think the proper tag is costumes, but because I left the ‘s’ off, it didn’t come up for me.

These insights demonstrate the high expectations that the students had of the search functionality in its ability to present information visually as a result of their prior experiences with commercial examples such as Google and iTunes. When I asked the class to indicate whether the search box was the first method of searching that they applied, they all raised their hands. No one had used the taxonomy function to look for an item.

Social learning

The students explored issues around being able to tag the items themselves as a result of the difficulties that they faced in finding materials within the archive:

[Student] I kind of think that a lot of the powers are the crowd sourcing power, I think if pathways really demonstrated its strongest point, this idea of letting anyone go in, pick out the smallest details and connect the dots, and I kind of think the idea of allowing user tags would almost open up like Wikipedia in a sense […] to allow users to go in there and play with it and allow users to jump off each other that’s when your going to get the multiplication factor where you have this rapidly expanding base of knowledge.

This highlights the level of digital literacy at which the students were working, as does the following comment that expresses a desire for further malleability of the archive:

[Student] In terms of the pathway itself, it was really great, that you can move it around, so its not just a linear pathway, because there were times when I had multiple examples, and it would have been nice to do a centre bubble, branching off with the multiple examples, and then something branching off there, and it would have been interesting to experiment with different structures of the pathway.

The key benefits of the SP-ARK archival model as gleaned from this focus group were as follows:

·        The openness of the resource allowed a valued access to previously unseen and inaccessible materials;

·        Students embraced the opportunity to work visually and intuitively with these materials;

·        SP-ARK facilitated a sense of empowerment, a deeper engagement and understanding of the archival materials and the surrounding subject area through being able to cross-reference and directly compare source materials with archival materials;

·        A social learning model was facilitated through the Pathway’s infrastructure, which stimulated the open sharing of ideas and practices.

The key areas that have been highlighted for development are:

·        More content to be made available to be able to access more of Sally Potter’s work and to expand beyond a single author platform;

·        The searching and tagging needs to be made more efficient and effective, the ability for users to tag assets needs to be considered;

·        Further links to other assets needs to be provided (which could be through user generated methods);

·        The facility to search and ‘mine’ the video content more efficiently and effectively;

·        To be able to work with the video content more creatively and fluidly, to be able to remix and adapt content.

The SP-ARK archive provides a unique example of the successful marriage between the principles of open educational resources and open archives. Our findings as part of this archive project above could help to inform others working with primary resources, clearly highlighting the key benefits of digital preservation, presentation and access. Students emphasised the value of a digital archival model to facilitate the ability to simultaneously view source materials alongside more conventional archival materials. This proved to be extremely beneficial and empowering to the learning process. The only other known example of this model within the field is EVE (Everyone’s Virtual Exhibition) hosted by The Bill Douglas Centre Online Catalogue for the History of Cinema and Popular Culture at the University of Exeter (17) from which similar beneficial conclusions have been drawn. Through EVE’s online interface, users are able to convene their own online exhibition through the curation, annotation and presentation of artefacts, which can be published in the same way as a SP-ARK Pathway, to be made publicly visible to other users (see Prosser, 2005).

SP-ARK raises important questions for both film studies and film production studies as disciplines. For example, how might the close analysis of film production processes that digital archives like it enable extend our understanding of cinema in ways that usefully move beyond the close textual analysis of the finished film? (18) Further extension of SP-ARK’s content and functionality are indeed currently being explored through a collaborative project entitled ‘The anatomy of a film production: Understanding the people, the roles, the processes and the careers on set of Sally Potter’s new film’ between myself at the University of Brighton and Potter’s production company AP. This project seeks to extend the reach of the archive into vocational and professionally based education. Film sets and feature film production processes tend to be closed and hidden from view, but are areas of extreme intrigue and interest, from the viewpoint of audiences, fans, would-be filmmakers, academics and teachers and those wanting to forge a career in the film industry.

Potter’s latest film (yet to be titled) has just completed production, and during the process, key members of the production crew were furnished with portable Flip cameras to record their daily activities on set. The materials thus produced will create an interactive audiovisual ecology of the film production process, capturing all of the individual crewmembers’ contributions to the creation of the film through personal testimony, revealing working dynamics and practices within a contemporary location-based film production. The project registers all the roles on set from runners and caterers to camera, sound, costume, make-up, continuity, set builders, sparks, extras to heads of all the departments. It will extend the reach of the archive’s resources beyond the production phase into the postproduction, marketing and distribution of the film. This, in turn, will expand the role and functionality of SP-ARK as an invaluable and innovative open resource for young people and students seeking careers in the film industry, and to educators and academics teaching contemporary film production and processes.

Will emerging technologies that facilitate the intuitive retrieval and mining of audiovisual archival materials, such as HTML5 video architecture, (19) provide digital archives like SP-ARK with new ways of presenting and navigating materials that move beyond the traditional database and archival rubrics? Further advice and support on these questions could be gleaned from user group communities around technologies such as Reel Surfer (20)and Popcorn, (21) an open source video platform developed by Mozilla. The BBC is currently developing an Application Programming Interface (API) within their Redux (22)initiative in order to facilitate the accessibility and search ability of their extensive archives, as are the British Film Institute. (23) The development of digital technologies and open source software, in particular in relation to the advancement of online video, provides a unique opportunity to semantically gather and present materials to audiences in new and open ways, which move beyond the traditional (closed) archival model. For example, Popcorn facilitates the real-time linking of data from video streams. These initiatives seek to draw out the benefits and efficiencies of collaborative resource generation, exploring the challenges of sustainability and expansion of both the resources and the encompassing user-group community.

The findings of the SP-ARK case study, many of which are set out above, will not only inform the future direction of this pioneering archive itself, an endeavor which has always placed the educational community at the core of its development (initially at the Screen School at Goldsmith’s College, see Mayer: 2008). But they also have the potential to support and inform the approaches of other, emerging online film-based repositories as they grapple with the issues of openness, reuse and licensing.

Whatever happens with future developments, the benefits that such resources are already bringing to higher-education academics and students are invaluable. The pathways tool lends itself to the critical and analytical study of primary materials for both undergraduate and postgraduate study within numerous disciplines. It also fosters a model of collaborative scholarship whereby ideas are shared and developed, which moves far beyond the traditional single-user archival model. The resource as a whole also has the potential to enhance and enrich teaching and learning practices within film and media disciplines, and beyond, as well as to encourage other high-profile filmmakers and organisations to allow online access to their work in the future. SP-ARK exemplifies the successful combination of an archive and an educational resource, providing an invaluable and compelling models for social and participatory learning, as well as for the development of open academic practice more generally.

Endnotes:

References:

British Universities Film & Video Council [Online] http://beta.bufvc.ac.uk/ Accessed 02/02/12

Ćalić, J., Gibson, D., Campbell, N., Efficient Layout of Comic-Like Video Summaries. IEEE Transactions on Circuits and Systems for Video Technology, 17(7). ISSN 1051-8215, pp. 931–936. 2007.

DCMS SMITH, R. H. L. 2012. A Future For British Film, It begins with the audience… A UK film policy review. London: Department for Culture, Media and Sport. Print.

EU Screen: Europe’s Television Heritage [online] http://www.euscreen.eu/ Accessed 02/02/12

Europeana: Europe’s Cultural Collections http://www.europeana.eu/portal/ Accessed 02/02/12

Fowler, C. 2009. Sally Potter, Urbana, University of Illinois Press.

Frenkel, V. 2011. Capture and Loss: Memory, Media, Archive. Public, 163-169.

Gagon, J. 2011. The time of the audiovisual and multimedia archive. Public, 59-63.

Genette, G. 1997. Paratexts: Thresholds of Interpretation, Cambridge, Cambridge University Press.

Gerhardt, P., and P.B. Kaufman. Film and Sound in Higher and Further Education: A Progress Report with Ten Strategic Recommendations: HEFCE/JISC, 2011. Print.

Gray, J. 2010. Show Sold Separately: Promos, Spoilers and Other Media Paratexts, New York, New York University Press.

Kurosawa, A. Online Film Archive, [online] http://www.afc.ryukoku.ac.jp/Komon/kurosawa/index.html. Accessed 15/12/11.

Mayer, S. Expanding the frame: Sally Potter’s digital histories and archival futures in Screen 49:2 Summer 2008. Print.

Mayer, S. 2009. The Cinema of Sally Potter: A Politics of Love, London: Wallflower.

Prosser, D. (2005) Reflections on the use of EVE in Higher Education, April 2005, [online] at: http://billdouglas.ex.ac.uk/eve/eve_ped_use.pdf, last accessed 18/06/12.

Ren, K., Sarvas, R. and Ćalić, J. Interactive search and browsing interface for large-scale visual repositories in Multimedia Tools and Applications, Volume 49, Number 3, September 2010, pp.513-528.

SP-ARK video documentation:

An Introduction to SP-ARK (2010)

Features an overview of the original SP-ARK WordPress Site, in addition to clips from her films and documentary footage of working online with Yes.

Produced by: Adventure Pictures

Duration: 6’53”

Online at: http://www.mefeedia.com/watch/47266737, last accessed June 18, 2011.

SP-ARK: A Workshop at Goldsmith’s College (2007)

The SP-ARK team worked with students at Goldsmiths College to show how the pathway concept of SP-ARK will work.

Produced by: Adventure Pictures

Duration: 2’04”

Online at: http://blip.tv/sallypottercom/sp-ark-a-workshop-at-goldsmith-s-college-351099, last accessed June 18, 2011.

SP-ARK, Building the world’s first interactive multimedia archive (2010)

Presentation by Christopher Sheppard at the Media Festival Arts, London, 10^th September, 2010.

Produced by: Adventure Pictures

Duration: 13’38

Online at: http://www.mefeedia.com/watch/47266733, last accessed June 18, 2011.

Appendix 1

Sally Potter’s Films­­­­­­­­ (further information can be found at: http://sallypotter.com/)

­­­Latest film – yet to be titled (post-production) 2013

Rage 2009

Yes 2004

The Man Who Cried 2000

The Tango Lesson 1997

Orlando 1992

I Am an Ox, I Am a Horse, I Am a Man, I Am a Woman (documentary) 1988

Tears, Laughter, Fear and Rage: Rage (TV documentary) 1987

Tears, Laughter, Fear and Rage: Tears (TV documentary) 1987

The London Story (short) 1986

The Gold Diggers 1983

Thriller (short) 1979

Hors d’oeuvres (short) 1971

Play (short) 1970

Jerk (short) 1969

Copyright:

Frames # 1 Film and Moving Image Studies Re-Born Digital? 2012-07-02, this article © Sarah Atkinson. This article has been blind peer-reviewed.

In contemporary digital culture, we are perpetually surrounded by a plethora of images whose barrage is in great and constant need of theorization if we are not simply to be swept away by it. To address this need, it is crucial that scholars be able to use and reuse the images that circulate around us in order to actively theorize them, their functions, and their effects. Moreover, in an era of online publishing in which images and video clips may be incorporated alongside text, the reuse of images and clips becomes an integral part of academic argumentation, enriching our understanding of visual culture by reframing and thereby re-comprehending fragments of its actual products. However, contemporary copyright laws and conventions frequently impede this vital practice by encouraging copyright holders to restrict the use of images and/or to seek licensing fees.

The U.S. Society for Cinema and Media Studies has been at the forefront of developing Best Practices in Fair Use for Media Studies scholarship in order to clarify how copyrighted material may be used in academic work. In addition to addressing questions about using films, clips, and film stills in teaching, SCMS has also attempted to articulate a clear set of guidelines for the use of copyrighted film images in academic publishing. Kristin Thompson has written a Report of the Ad Hoc Committee of the Society For Cinema Studies on the “Fair Usage Publication of Film Stills” in which she discusses at length the necessity for scholars to be able to include film stills as illustrations related to their scholarly arguments. She notes that, rather than using publicity or production stills, “For purposes of analyzing finished films…many scholars believe that photographs made from frames of the actual film strip are preferable, since they reproduce an actual composition that appears in a shot.” However, she notes, “The legal status of such reproductions of frames has remained problematic.” (1)

Thompson – writing for the Ad Hoc Committee – provides a very useful explanation of the cases in which the use of an image derived from a single film frame to illustrate a scholarly argument may most likely be considered an instance of fair use. Thompson’s analysis provides an excellent basis on which scholars and academic journals may justify their publication of film stills. (2) Indeed, Thompson makes a persuasive case for the film still as “transformative,” which is one basis upon which a claim of “fair use” may be made. Nevertheless, the threat that copyright holders may sue because they claim that their “property” has been “stolen” continues to inhibit the use of imagery in scholarship. In response, I would suggest that Thompson’s argument might be further strengthened – and expanded – by a phenomenological examination of the “reproduction” of an image and its recontextualization within a written text. Through an exploration of the “reuse” of an image as a form of “direct quotation” rather than simply a “reproduction” or “illustration” and as constituting a particular kind of viewer/reader experience, we may come to a more nuanced understanding of what may or may not constitute a “transformative” use of an image. Indeed, I argue that the reuse of an image – including but not limited to a film still – may, in fact, serve as a quotation and therefore may qualify as an instance of fair use.

Identity

The direct quotation as a form involves a tension maintained between identity and transformation. According to the 1871 book by Hiram Hadley entitled Lessons in Language: An Introduction to the Study of English Grammar, a direct quotation is “a repetition of the exact words of another.” (3) Given that Hadley’s book was published before many of the technologies of image production and reproduction were invented, it is no surprise that this definition refers only and specifically to “words” rather than to other kinds of signs. Even now, however, the “repetition” of an image (filmic or otherwise) is rarely, if ever, referred to in terms of quotation. Such repetition is much more frequently referred to as the “reproduction” of the image. However, I would argue there are reasons to consider certain repetitions of an image in general and a film image in particular within other texts as a form of quotation.

To begin with, in the case of both written quotation and image “reproduction,” in order to maintain the identity of the object as such, some things must be repeated “exactly” while others may be altered to some degree. When we quote a written source in a written text, what must be “exact” in its repetition is the identity and order of the words (or letters and spaces). In the case of written text quoting another written text, the “exact” form of the text does not extend to the font in which the original was printed, the size of the lettering, or even the layout of the letters on the page. What matters is that the letters and spaces retain the same relation to one another in terms of their order. The repetition of a written quotation in spoken form involves the transformation of one type of symbolic sign into another type of symbolic sign and from one medium to another; however, the spoken articulation is required to maintain only the order of words – emphasis (except where italics indicate emphasis in a written text), intonation, pitch, rhythm, and accent are not considered relevant to the identity of the quote. When written quotations are derived from spoken language, the quotation involves the opposite transformation of one type of symbolic sign into another, and from one medium to another. In this case, the question of whether to include “ums” and other sounds that may have interrupted the flow of words points to the difficulties that arise in quoting spoken language “exactly” in written language. However, in either case, a quotation of a previously written or spoken phrase is required to replicate only the precise identity and order of the words. It is important to note, however, that one may add, remove, or replace words from within the quotation through the use of ellipses and brackets. Although this convention is intended primarily to shorten a quote by omitting “irrelevant” words, removing and adding words nevertheless has the potential to disrupt and reconstitute meaning in a significant way. Thus, it is clear that the notion of the direct quotation as a “repetition of the exact words of another” is “exact” only in very particular ways. Phenomenologically – in other words, in terms of the experience of the reader – the quotation may have shifted in its design and layout (when writing is quoted in writing), its medium (when spoken language is quoted in writing or vice versa), and its continuity. Yet only the identity and order of the words matter to the identity of the direct quotation as such.

Like the direct quotation of a written passage, the “reproduction” of an image and the maintenance of its identity requires that some things be repeated “exactly.” Indeed, instead of the preservation of word identity and order, in the case of the image, what is crucial is that all visual elements retain the exact same spatial relation to one another. However, in order for these spatial relations to be exactly consistent, the image must necessarily be a mechanically copied representation with an indexical relation to the “original” (although the copy may sometimes be several times removed from the original). If the original image is not photographed – whether through analog or digital processes – but rather approximated by the human hand, the spatial relations may be altered (however slightly) which means that the image would cease to be an “exact” copy and would no longer function as “evidence” in the same way that an indexical copy produced by mechanical means would. Indeed, as numerous scholars have shown, the mechanically copied sign has historically been endowed with a privileged relation to the real and hence is more valued as “evidence.” (4) Thus, it appears that the image must be indexically, mechanically copied in order to function as evidence. (5)

However, just as the written quotation may be altered in some ways and still remain the “same” phrase, there are certain ways in which an image may be altered and still be considered the “same” image. For instance, the image may be altered in terms of its scale. This has generally been the norm in still photography. Analog photographs are generally derived from small negatives – the size of the negatives varying by format – from which a positive of (potentially) any size could be created. No matter the scale of the print, so long as it is based on the same negative, it is considered the “same” image. Digital photographs may also be resized both on the screen and when they are printed. 

This is clearly the convention in relation to the use of film stills in academic publishing as well. Indeed, the use of a film still within a written text is often referred to as a “frame enlargement.” According to the Society for American Archivists, a “frame enlargement” is “a photographic print made from a single frame of a motion picture.” (6) An example of a frame enlargement of Child of the Big City (Yevgeni Bauer, 1914) produced by the Harvard Film Archive and posted on the HFA Collections Blog reveals how a single image from a filmstrip is singled out for enlargement. (7)

The term itself suggests an element of alteration involved in producing a “frame enlargement,” but above the definition does not register this as an alteration, simply stating that the frame enlargement is “made” from a single frame. Moreover, the term also points to a phenomenological paradox. A frame “enlargement” is both larger and smaller than the image in the “film” itself. Assuming that it is mean to fit into a book or journal of some kind, it is necessarily larger than the tiny frame that makes up 1/24^th of a second in a standard film reel. Were it not, it would be difficult for the viewer to see the image clearly. However, the “frame enlargement” is also much smaller than the projected cinematic image of which it is a constituent element – which, of course, may itself be projected at different scales. In terms of the viewer’s experience, however, the scale of the film image is, in fact, significantly reduced (unless, of course, the film was viewed on a very small screen or the film still is very large). However, as in the case of the still photograph, it is apparent that changes in scale are not generally regarded as constituting a significant transformation of the film image.

There appears to be an exception to this convention, however, when the change in scale is extreme. If the image is “reproduced” at such a minute scale that it ceases to be recognizable, it may not be regarded as the “same” image anymore. This seems to hold true in the case of small images collaged in order to create a larger image, such as the collage of thousands of tiny images from gay pornography arranged to generate a recognizable image of allegedly homophobic US Senator Rick Santorum’s face.

From the perspective of the viewer’s experience, the pornographic images are barely recognizable as such and appear to be “superseded” by the larger image of Santorum’s face. Other examples of photo mosaics are less ironic, for instance, an image of Barack Obama made up of newspaper covers from the day after his election, which reads more as an homage than a critique. (8) 

However, so far as I can tell, no one has sued over the use of tiny versions of their imagery in such “photo mosaics,” suggesting that no one considers these images as the “same” images now that they are miniscule parts of a larger whole. (9) Recognizability – a category which depends entirely on the viewer’s perception – seems to be the major factor in this convention. If the image is too small for the human eye to see clearly and it is part of a different image that the human eye can perceive clearly, it ceases to be the “same” image. (10)

Yet these conventions surrounding changes in the scale of the image are complicated by the fact that changes in scale are almost inevitably accompanied by an alteration in “quality” or “resolution.” As images are enlarged, they begin to lose their “definition” (a term whose multiple meanings are telling). This is certainly true in the case of the “frame enlargement.” Indeed, the Society for American Archivists definition above notes that “frame enlargements are generally of poorer quality than movie stills.” (11) In the case of an analog image, the larger the image, the “fuzzier” it will become as edges become less clearly defined. In the case of a digital image, the larger the image, the more pixelated the image becomes so that eventually it appears “blocky.” The question of whether a change in resolution can at some point produce a “different” image, however, has been a source of contention. This debate was crystalized in the lawsuit filed by photographer Jay Maisel, who shot the photograph of Miles Davis that was used on the cover of Davis’ album Kind of Blue, against Andy Baio, who produced a chiptune tribute to Kind of Blue entitled Kind of Bloop, generating synthesized music through the sound system of a vintage video game using Davis’ melodies. Baio licensed all the music; however, he also commissioned a pixel art recreation of the original cover – a photograph – but did not license the image since he believed that the pixel art recreation was sufficiently different from the photograph to be considered fair use. Even though, from the perspective of the viewer’s experience, the difference between the covers of Kind of Blue and Kind of Bloop constitutes a recognizable split between the experience of an indexical photographic image and that of a non-indexical (but, rather, loosely iconic) pixel image, Maisel sued and Baio ended up settling out of court. The cover is no longer available with the album although it can still be seen online. Baio’s blog, which relates this story in detail, also raises important questions about the relationship between resolution and transformation. He created a series of images, all derived from the Kind of Blue photograph with increasing degrees of pixelation, until the original image becomes unrecognizable. Next to this series of images appears the caption “Where would you draw the line?” (12)

Thus, the question remains whether there is some point at which a change in resolution produces a “different” image. If the degree of detail of the image is reduced so much that the elements of the image cease to “resolve” into a recognizable object, the viewer’s experience of the original image and of its unresolved “derivative” cease to have any coincidence. Nevertheless, it is clear that, according to current conventions (both academic and legal), relatively limited changes in the scale and resolution of an image (filmic or otherwise) are not usually regarded as constituting a “new” or “different” image. (13)

Another way in which an image may be changed yet continue to be regarded as the “same” image is that a color image may be reproduced in black and white. The “subtraction” of color, which makes “reproduction” less expensive, suggests that color is – at least in certain circumstances – not considered essential to the identity of the image. However, the reverse does not seem to be true. In the case of still photograph, Swedish artist Sanna Dullaway has been celebrated for the “artistry” involved in her colorization of iconic photographs of people such as Abraham Lincoln, Charles Darwin, and Anne Frank and of other historically significant photographs. (14)

John Hutchinson writes in the UK Daily Mail, that:

While purists may disagree with the modernisation of such historic pictures such as the celebrations that marked VJ (Victory in Japan) day, images of devastation from Pearl Harbour and the shocking shot of a Viet Cong prisoner being shot in the head by an army general (demonstrating the brutality of the Vietnam War), the colour pictures do appear to re-emphasise the scene they are portraying. (15)

This reference to “modernisation” and “reemphasis” suggests that Hutchison perceives Dullaway’s additions of color as producing a “different” image. To an even greater degree, it seems, color may not be added to a black and white film image and continue to be considered the “same” image, a fact to which the controversy of the late 1980s and early 1990s surrounding the colorization of black and white films attests. Scholars, filmmakers, and industry players argued for and against colorization on various grounds. (16) Interestingly, however, as Yuriko Saito has shown, one of the major objections to colorization was the feared alteration of the audience’s experience of the film. (17) Given the uproar surrounding colorization, it seems somewhat odd that the removal of color from a film image is considered unproblematic. From the perspective of the viewer’s experience, the transformation is just as significant. Nonetheless, “reproduction” of a color image in black and white is generally not considered an alteration of the image’s identity.

Thus, it appears that changes in scale and resolution (to some degree) and the subtraction (but not the addition) of color are not regarded as changing the identity of an image. In the case of the film still, however, there is another transformation involved: the transformation of a moving image into a still image. (18) Although film frames make up the underlying substrate of the film medium, they are never experienced by the film viewer as still images but rather as moving images, an experience that takes place over time. (This is slightly different in the case of the video frame, which we may experience as such when we pause the video. However, pausing the video is generally regarded as an interruption of the text rather than an integral part of the text.) If we redefine what is “the same” and what is “transformative” in terms of viewer experience, the film still and the film are identical for only 1/24^th of a second (or a similar fraction for other formats with different frame rates), which is a period too brief for the human sensorium to identify as a distinct experience. Thompson notes that:

The commercial exploitation of films typically involves their being projected in such a way as to create the illusion of movement…Frame enlargements, however, do not duplicate the film in this way. A film frame, when printed on a page, cannot be projected as a portion of the original. It cannot create the illusion of movement, nor does it reproduce the sound that most films still in copyright involve. Even if we were to print every single frame of a film in a book, the result would in no way replicate the viewing experience. It is hard to imagine a person who has seen even thousands of frames reproduced deciding that he/she had “seen” the film and as a result had no need or desire to see it projected. (19)

Few would argue that still images and moving images belong to the same medium since, as Thompson clearly indicates, the phenomenological disparity between the viewer’s experience of the film still and of the moving image is so great. To argue that the film still is a “reproduction” of the film is to assume that reproduction refers only to a replicated object (the frame of film) rather than to a reproduced experience. And yet, when we remember a film we have seen, we remember the experience of seeing moving images, not the series of still images on a strip of wound celluloid sitting in a can. Hence, the film still cannot be reduced to the “same” experience as watching a film even if it is produced from the film object. Nonetheless, the film still is generally regarded as a “reproduction” of a film at least insofar as copyright holders continue to seek licensing fees.

Thus, we can see that in the case of both written quotation and image “reproduction,” certain elements (though not precisely the same ones) may be altered while the identity of the phrase or image is considered to remain fundamentally “the same.” However, because of the requirement that an image be an indexical, mechanical reproduction of the original in order to remain the “same” image, there is no middle ground upon which an image may be directly “quoted” without also being “reproduced.” This creates a double bind for those who wish to “reuse” an image. An approximation of the image is not “direct” enough to act as significant evidence while an indexical, mechanical reproduction of the image is too “direct” and raises the specter of copyright infringement. The “permissible” changes in scale, resolution, color, and motion do not alter the identity of the image precisely because they do not “count” as transformative. (This despite the fact that, from the perspective of the viewer’s experience, changes in scale, resolution, color and motion – at least changes of a certain degree – are potentially very transformative.) Indexical, mechanical replication is the only thing that can both insure that the image is the “same” while simultaneously opening anyone who uses the “reproduced” image up to charges of copyright violation. I would argue that we should reframe indexical, mechanical reproduction – in other words, the fact that all of the elements of the image retain their exact spatial relation – as parallel to the requirement that the words in the direct written quotation retain their “exact” order. Spatial relations in the image “reproduction,” like word identity and order in the direct written quotation, becomes the ground upon which the identity of the image is established and against which the new writer may construct new ideas.

Transformation

Indeed, the value of a direct written quotation does not lie simply in its identity as such but also in its “productivity” within its new context, this productivity made no less important by its dependence on a subjective evaluation on the part of the reader. While the identity of the direct written quotation must be maintained, its significance must be transformed through its recontextualization, producing something “new.” Indeed, although it is crucial that certain things remain the same within the quotation marks in a written quotation, it is also imperative that something “different” be produced in the relation between what is inside and what is outside the quotation marks. Of course, any repetition in a new context alters the quotation and produces something “new” and “different.” As Meir Sternberg notes of written quotations:

The framing of an element within a text entails a communicative subordination of the part to the whole that encloses it. However accurate the wording of the quotation and however pure the quoter’s motives, tearing a piece of discourse from its original habitat and recontextualizing it with a new network of relations cannot but interfere with its effect…To quote is to mediate and to mediate is to interfere. (20)

Indeed, the act of recontextualizations always “mediates” or “interferes” with the “effect” of the quoted material. And yet, if “too much” of the original is quoted, it may seem that the new text is simply showcasing the original text rather than effectively reframing and thereby transforming its meaning in some way. Moreover, if the written quotation is not fully “unpacked” in relation to the new text in which it appears, the “interference” may not be considered a “productive” use of the quotation. This notion of a “productive” use seems to occur in the interaction between that which is inside the quotation marks and that which is outside – as if there were a chemical reaction that might or might not take place. If it does not occur, the quote is “wasted.” In addition, when a quotation is imported into a new text, to be productive, it seems that the quote must be fully “integrated” into the new text. To insert the quote without integrating it into a new text, would simply be to put it in a new context – to just repeat it – rather than to make it part of a new text – to repeat and, in doing so, transform it. Indeed, it is the tension between something “exactly” the same within the quotation marks and something different produced between what is inside and outside the quotation marks that defines the ontology and productivity of the written quotation. 

The problem of taking “too much” differs in the case of the still image and the film still. In the case of the still image, it is impossible not to take “too much” while maintaining the identity of the image. Taking a part of a still photograph, for instance, (except as a detail of a larger image, also presented in full) would alter the spatial relations of the image. Even cropping the image is generally considered an alteration of the identity of the image. As a result, as I have established above, if only part of the image is taken, it is no longer the same image. In the case of the film still, the tiny fraction of the film (and filmic experience) it embodies suggests that it cannot be regarded as taking “too much” of the film itself. However, a film still reads as a “complete” image, which may be why it continues to raise copyright issues. Despite these different relations between still images and film stills and their source texts, I would nevertheless argue that the still image can be transformed by its recontextualization in the same way as the film still can.

Indeed, it follows from the discussion above that the productivity of the image in a written essay hinges on a transformation in relation to what is immediately external to the “frame” that marks the boundary between the image and what is outside the image. The image must interact with its new environment to produce something “new.” However, while the written quotation can be “integrated” into the new written text by virtue of being of the same medium, this is not true for the image “quoted” within the written text. Indeed, because still images are not of the same medium as writing, they are usually regarded as “illustrating” the written text rather than being fully “integrated” into it. 

This distinction between “integration” and “illustration” is crucial in that it is, in part, the written quotation’s successful integration into the new written text that makes it seem “productive.” The written quotation is marked as “other” but it becomes a piece of the written text into which it is inserted. It does not illustrate but rather contributes – almost but not quite seamlessly – to the text at hand. Experientially, although the reader’s eye will note the presence of quotation marks, the eye may continue reading the quotation without a pronounced spatial or temporal interruption. A written quotation of a written or spoken text remains similar to its source in that the source, like the quote, is experienced over a period of time. Thus, the quoted text and the new text can be experienced as an uninterrupted flow.

An image, however, cannot be “integrated” into a written text in the same way. When we are reading a written text, if an image accompanies the written text, our eyes must stop reading the words in order to take in the nearby image. Although we may look at the image for a long or short time, this act of looking is experienced (if unconsciously so) as an interruption. We must stop reading, look at the image, then look back at the written text. As a result of this disruptive experience, the image remains more noticeably “other” to the written text than does the written quotation inside quotation marks. I would argue that it is, at least in part, this shift in the mode of “reading” that leads us to refer to an image within a written essay as an “illustration” rather than a “quotation.” (21) Yet, in an academic essay about an image, if the written text is analyzing the image, it seems disingenuous to argue that image simply “illustrates” the analysis. If we regard the frame of the image (the boundary between inside and outside the image) as similar to a set of quotation marks, it is the interaction between inside and outside that has the potential to produce a transformative effect and a new significance. Yet we have a very limited vocabulary with which to discuss this interaction between image and written word, so the interaction is reduced to “illustration,” a term that implies the image to be subservient to the written word, an auxiliary rather than integral and productive part of the new text. (22) It is worth noting that a similar problem occurs in the case of online journals that include film clips along with the written text. Although the film clip, like the written text, is experienced in time, the switch from reading to viewing a clip is also likely to be experienced as an interruption. The film clip cannot be seamlessly integrated into the written text any more than the film still can.

Despite the fact that an image cannot be fully “integrated” into a written text, I would nevertheless argue that we need to reconceive of the interaction the words of a written essay and the accompanying image as not an illustration but rather as a quotation because it produces – or has the potential to produce – something “new.” While the experience of a written text and an image cannot be seamlessly united, the inclusion of an image may allow for a productive interaction between written text and image. Indeed, when an image appears with a written text about that image, my experience of seeing the image under discussion is often one of epiphany. Something that was abstract and unspecific when it was described in only written words becomes concrete and particular as I gaze at the image. I am able to reexamine the image within the context of the argument being made and, as a result, new readings of the image become possible. Thus, despite the lack of seamless integration involved in using an image in a written text, a “something new” is generated between the image and its new context. By the same token, however, if the image is not fully “unpacked” in the text, it may not constitute a “productive” reuse of the image. In the case of a film still, if the still is included simply as a visual metonym for the film and is not directly discussed, then it may retain its status as an “illustration” rather than as a productive quotation. If they are to be considered quotations, images must not simply serve as “decoration” for the written text. Words and image must collide and in their collision transform their respective significances and, in doing so, also transform the viewer’s experience of each one. This is, of course, the basis of a claim of fair use.

Implications for Remix Film and Video

While the argument above is focused on the use of images in academic publishing, the discussion of the “quotation” of an image also has important implications for thinking about the use of film clips in “remix,” a broad term for films and videos that appropriate preexisting audiovisual materials. While, as established above, an image cannot be fully integrated into a written text because it exists in another medium which our sensoria experience differently, using a film still or a film clip in a film or video would seem to achieve a greater parallelism between the quotation of an image and a quotation of written phrase. When filmmakers appropriate stills or clips – audio, visual, or both – into a new text, this audiovisual quotation may be integrated into the new audiovisual text just as a written phrase may be integrated into another written text with the minor intervention of quotation marks. Of course, just as a written quotation must be marked as such by quotation marks, it is arguable that filmmakers should also mark their appropriations by citing their sources in some form or another. Although a standard format for citing visual sources in a film has yet to be developed, as theorist Patrik Sjöberg has noted, there is no reason that this cannot be done. (23)

Those who pursue charges of copyright infringement, however, do not profess to care about citation. Rather, they simply assume that visual media cannot be “quoted” since the term “quotation” is associated only with words. By this logic, the mechanical reproduction of any portion of an image, still or moving, is an act of theft. However, if we reimagine the mechanical reproduction of audiovisual clips as a form of “direct quotation,” whose identity must be maintained as such in the same manner as the identity of the direct written quotation, and if we require that the incorporation of this audiovisual “quotation” into a new audiovisual text transform the significance of both the clip and the surrounding text, then such audiovisual appropriation is not an act of theft so much as one, as Sternberg suggests, of productive “interference.” Indeed, academic scholarship, artistic creation, and the production of new knowledge itself are based on the act of “interference.” The role of the intellectual or artist is precisely to “interfere” with sedimented ideology that masquerades as and is blindly accepted as “universal truth” or “common sense,” preventing us from imagining other, potentially better ways of thinking and being. If we cannot “interfere” with audiovisual texts through “quotation” in the same way that we regularly “interfere” with written texts, then the academic and artistic projects may come to a halt. I would argue that in a society in which communication increasingly takes place through audiovisual rather than written forms, we must begin to rethink quotation as a multimedia tool. 

Moreover, the above comparison between the direct written quotation and the film still as a form of direct quotation also suggests that we must rethink fair use less in terms of a transformed object than a transformed viewer experience. Of course, as I have demonstrated, certain slight transformations of viewer experience of the object such as relatively minor transformations of scale and resolution and the removal (but apparently not the addition) of color, are generally regarded as, by and large, insignificant to its identity (although many film purists would argue that changes in quality and resolution make a major difference in terms of viewer experience). Major transformations of scale and resolution, however, do change viewer experience to a degree that the object (as co-constituted by the text and the viewer) can no longer be regarded as the “same.” Moreover, the shift from a time-based to a non-time based form in the case of the film still or the removal of a short temporal segment from a longer piece in the case of the film clip, also significantly alters viewer experience. A film still and a relatively brief film clip do not provide the same experience as the complete film text. Where we draw the line between a significant and insignificant change in viewer experience will continue to be a point of contention. However, I believe that rethinking the reuse of still images, film stills, and film clips within academic or artistic texts as producing an experience parallel to that of reading a direct written quotation within another written text may begin to clarify this distinction. And where the repetition of an image or clip functions as or like a direct quotation, it must be considered an instance of fair use.

Endnotes

Works Cited

Baio, Andy, “Kind of Screwed.” Waxy, 19 July 2011. http://waxy.org/2011/06/kind_of_screwed. Accessed 12 April 2012.

Dempsey, Michael, “Colorization.” Film Quarterly 40, no. 2. (Winter, 1986-1987): 2-3.

Harvard Film Archive Collections Blog, http://blogs.law.harvard.edu/hfacollections/2009/02/. 9 February 2009. Accessed 14 May 2012.

Hoffman, Kevin, “Five best Obama Photomosaics on Flickr.” 22 December 2008. http://blogs.citypages.com/blotter/2008/12/5_best_obama_mo.php. Accessed 14 May 2012.

Holland, H. Brian, “Social Semiotics in the Fair Use Analysis.” Harvard Journal of Law & Technology 24, no. 2. (Spring 2011): 335-391.

Huffington Post, “Paul Cadden’s Unbelievably Photorealistic Drawings.” 26 April 2012. http://www.huffingtonpost.com/2012/04/26/paul-cadden_n_1453584.html Accessed 14 May 2012.

Hutchison, John, “The past just got a whole lot more colourful: Painstaking touch-up of world’s most famous black and white pictures.” Mail Online, 19 January 2012. http://www.dailymail.co.uk/news/article-2088611/Swedish-artist-Sanna-Dullaway-injected-colour-host-historic-photographs.html. Accessed 8 May 2012.

Leibowitz, Flo, “Movie Colorization and the Expression of Mood.”
 The Journal of Aesthetics and Art Criticism 49, no. 4 .(Autumn, 1991): 363- 365.

McCloud, Scott, Understanding Comics, (2d ed. New York, Harper Collins, 1993).

Naik, Pratik, “Unbelievable colorization of black and white images,” http://fstoppers.com/pics-unbelievable-colorization-of-black-and-white-iconic-images. Accessed 8 May 2012.

Nichols, Bill, Representing Reality: Issues and Concepts in Documentary. (Bloomington: Indiana University Press, 1991).

Saito, Yuriko, “Contemporary Aesthetic Issue: The Colorization Controversy.” Journal of Aesthetic Education 23, no. 2 (Summer, 1989): 23-24.

Silvers, Robert, http://www.photomosaic.com/. Accessed 8 May 2012.

Sjöberg, Patrik, The World in Pieces: A Study of Compilation Film. (Stockholm: Patrik Sjöberg, 2001).

Society of American Archivists, “Frame Enlargement.” http://www2.archivists.org/glossary/terms/f/frame-enlargement. Accessed 12 April 2012.

Sternberg, Meir, “Proteus in Quotation-Land: Mimesis and the Forms of Reported Discourse,” Poetics Today 3, no. 2. (Spring 1982): 107-156.

Thompson, Kristin, “Report of the Ad Hoc Committee of the Society For Cinema Studies, ‘Fair Usage Publication of Film Stills.’” http://www.cmstudies.org/?page=positions_policies. Accessed 9 April 2012.

Unicorn Booty. http://unicornbooty.com/blog/2012/02/21/omg-a-rick-santorum-portrait-made-entirely-of-gay-porn-nsfw-ish/. Accessed 5 May 2012.

Young, James O. “Still More in Defense of Colorization.”
 The Journal of Aesthetics and Art Criticism 50, no. 3 .(Summer, 1992), 245- 248.

Copyright:

Frames # 1 Film and Moving Image Studies Re-Born Digital? 2012-07-02, this article © Jaimie Baron. This article has been blind peer-reviewed.

I hope history can realize that its significance is not in universal ideas, like some sort of blossom or fruit, but that its value comes directly from reworking a well-known, perhaps habitual theme, a daily melody, in a stimulating way, elevating it, intensifying it to an inclusive symbol, and thus allowing one to make out in the original theme an entire world of profundity, power, and beauty. [Friedrich Nietzsche, “On the Use and Abuse of History for Life”]

In 2009, Amsterdam’s EYE Film Institute invited the public to remix twenty-one film fragments from its collection of early Dutch films.  The remixes were shared using Creative Commons licenses, inviting future users to remix the remixes ad infinitum.  In 2012, EYE Film launched CelluloidRemix.nl, a website devoted to expanding the collection of EYE film fragments, as well as the participatory practice of remixing.  The site allows users to download and upload films, remix this content using EYE’s own online software, and share remixed works through Open Images, a platform developed by the Dutch Institute for Sound and Vision.  At the end of November 2009, Paul D. Miller (aka DJ Spooky, That Subliminal Kid) debuted a remix of Dziga Vertov’s Kino-Glaz (1924) and Kino Pravda series at the State Hermitage Museum in St. Petersburg.  That same year, Chandler McWilliams, an independent artist and software designer created Silent, a flickering combination of frames from the canon of silent cinema.  The work was automatically generated by custom software that matched the length of each frame to the audio data from the soundtrack.  More recently, Scottish electro-acoustic musician Ross Whyte has joined early archival images with the sounds of audio accidents and mechanical malfunctions.  Whyte refers to these works as “glitches” and describes them as “rhythmic events” that reveal the impermanence of both audio and visual artefacts. (1)

This is only the beginning of new beginnings for the origins of the moving image.  Hundreds (maybe thousands) of amateur remixes have multiplied alongside these institutional and professional adventures in remix culture.  Youtube and Vimeo are bursting with audiovisual experiments in early and orphan film, as well as reassemblages of the silent canon.  These remixes mark the contemporary proliferation of digital film archives and video-sharing platforms as numerous film institutes have joined the Netherlands and made significant portions of their collections digital, streamable, and downloadable.  But the remix also raises crucial questions for film historians.  What do these revisions do to and for the film object?  What kind of histories do they tell (or repeat)?  And: where does the remix belong in the archive?  For its part, the EYE Film Institute uses its remix platform as a form of community outreach and a promotional tool for its “real” archival content, an approach that both confirms a hierarchy of historical value and inadvertently generates new forms of digital detritus: the orphan offspring of orphaned originals.

In this essay, I take the remix as a starting point for engaging the intersection between film objects, early and silent film historiographies, and contemporary visual culture.  I rethink the historical continuities and affinities that have been drawn between old and new media, film history and digital technology.  That is, I am not interested in what remixing might share with historical film practices like montage, collage, and bricolage, with modernist modes of fragmentation and détournement.  I do not read the remix as the genealogical descendent of early twentieth-century film forms or the re-emergence of a cinematic lineage that has been hibernating underground.  Rather, I explore what the remix tells us about film objects and film historiography.  I argue that the remix is a metahistorical work, a mode of historical expression that is fundamentally about film artefacts and historical telling.  In taking early and silent film as its raw material, the remix reveals, albeit imperfectly and indirectly, the false analogies and figures that have inhabited these histories for more than three decades.  Like the “flâneuse” in Catherine Russell’s parallax historiography, the remix is an “impossible concept” and a hyperbolic counter-model. (2)  It enables us to think beyond recuperation and preservation, beyond text and context, beyond physical artefacts and archives.  As I will argue, the remix also opens onto the possibility of new film histories and historiographic futures: not the digital annihilation of the celluloid archive, but a reinvigorated theorizing of film history that owes and offers something to the living present.

New Media, Old Theory: The Post-Structural Remix

I imagine that few film historians will be anxious to bring the contemporary digital remix within their disciplinary purview.  And with good reason.  Remixes are difficult to take seriously.  They tell jokes, play tricks, and run in referential circles.  They are ugly, stuttering forms that bear the traces of digital reproduction and compression.  Their images are pixelated, interlaced, and made by users of all kinds.  But perhaps most damning (and worrying) for the film historian and the film-historical project: remixes conceal the contours of their sources—the beginnings, middles, and ends of original objects—as they manipulate these materials into contemporary visual moulds.  Indeed, remixes tear historical artefacts apart and sew them back together in motley, unfamiliar, and seemingly anti-historical ways.  By nearly every measure, remixes are bad film objects: copies of copies, deviant simulacra, the kind of derivative visual forms that Plato warns against: “Imitation really consorts with a part of us that is far from reason […].  Imitation is an inferior thing that consults with another inferior thing to produce inferior offspring.” (3)

Remixes also lack historicity.  They are the ephemeral expressions of contemporary popular culture and the dialogical babble of multiple and contradictory historical indexes.  At once present tense and not-real time.  In his essay “What Comes After Remix?” new media scholar Lev Manovich leaves the question unanswered.  He proclaims the twenty-first century “the era of remix” and then stumbles: “I don’t know what comes after remix. But if we now try to develop a better historical and theoretical understanding of the remix era, we will be in a better position to recognize and understand whatever new era will replace it.” (4)  For those who are worried about the imprecise reach of the remix, Manovich offers little comfort.  The difficulty of envisioning a post-remix era perhaps emerges out of the remix itself, out of its expansive and indefinite boundaries, its voracious appetite for any object or artefact whatever.  As a practice, the remix generates (and regenerates), producing a seemingly endless becoming of the new and of the now that extends as far as the eye can see into the future.  Kirby Ferguson’s four-part viral video series, Everything Is a Remix, offers a more radical take on the ahistoricity of the remix: it has no beginning and no end; it is the always and already of being, thought, and creativity.

And yet, for all that seems anti-historical or a-temporal in this popular practice, the remix and the discourses that swell to defend what it means and why it matters are haunted by theoretical ghosts.  The remix returns us to Roland Barthes, Jacques Derrida, and Michel Foucault, to the challenges that they posed against the authority of authorship and the stability of origins.  The remix fulfils the promise of the post-modern text, what Barthes describes as “a multi-dimensional space in which a variety of writings, none of them original, blend and clash […], a tissue of quotations drawn from the innumerable centres of culture.” (5) With the post-structuralists as our guides, the remix no longer shimmers of the new, but as Ferguson suggests, appears familiar and worn, an assemblage of the always and already theorized.  Read alongside the claims of Derrida’s Of Grammatology, the remix is just another form of writing, no more unstable or destabilizing than the practice of inscription ever was.  For, as the French philosopher explains, “there has never been anything but writing; there has never been anything but supplements, substitutional significations which could only come forth in a chain of differential references.” (6) And, again, in the closing paragraphs of Foucault’s “What is an Author?”, the contours of remix culture come into view.  In this work, however, the effects of remixing are still to come:

I think that, as our society changes, at the very moment when it is in the process of changing, the author function will disappear […].  All discourses, whatever their status, form, value, and whatever the treatment to which they will be subjected, would then develop in the anonymity of a murmur. […] We would hear hardly anything but the stirring of an indifference: What difference does it make who is speaking? (7)

For remix theorist Eduardo Navas, it no longer makes a difference.  The remix murmurs with the anonymous collective.  It is the direct descendent of the polysemous future Foucault imagined. (8) But even those remix advocates who think and write beyond the boundaries of this theoretical lineage do not manage to escape the arguments that the post-structuralists made in defence of the remix before there ever was such a popular practice.  In his writing on the remix and copyright reform, Stanford Law Professor and founder of Creative Commons Lawrence Lessig argues that the remix shifts the balance of power from a “RO” (Read Only) culture of professional makers and mass consumers to a hybrid “RW” (Read Write) culture of user-creators.  He also argues that this practice produces anonymized, sometimes global communities, bound only by their shared remix practices. (9) In the remix collective, no one asks after the image or image-maker, but only the possibilities and processes of discursive appropriation.

One could say much more about the relationship between post-structuralism, authorship, and contemporary remix communities.  More interesting, however, is the way in which the return to these strands in post-structural thought suggests that there are significant historiographic stakes in and for the remix.  As Thomas Elsaesser explores in a recent essay, the route from a digital imaginary through post-structuralism encounters crucial metahistorical questions. (10) I would therefore like to retrace this path through the counter-historical threads of post-structural thought before returning to the particularities of film historiography and the productive challenge of the remix.  The post-structural rethinking of authorship and writing was, after all, a rethinking of the ontology of origins, artefacts, and historical objects.  In the hands of post-structuralists, the foundations of historical practice rupture and give way.  The essential tool of historical expression—writing—loses its privileges, its claims to objectivity and neutrality, as it collapses in a series of endless substitutions and equivalences.  For post-structuralism, there is no historical writing, just writing, discourse, supplements.  Nor is there any such thing as beginnings and ends, tidy lines or continuities.  Rather, as Edward Said describes in his meditation on beginnings, “a better image is that of the wanderer, going from place to place for his material, but remaining a man essentially between homes.” (11)

The Voice That Speaks Itself: A (Brief) History of Historiography

In The Archaeology of Knowledge, Foucault interrogates what he calls “discursive formations” and “rules of formation” or, to recall Les Mots et les Choses, groups of things and the word-systems that produce those groups of things.  At the centre of the Archaeology is a very sceptical claim about language and knowledge in the human sciences (including history, natural history, and ethnography).  Foucault argues that language produces the very objects it seems only to represent.  Language is not a value-neutral mode of transmission, but always-and-already entangled in a regulating system that it cannot escape, describe, critique, or unravel.  Foucault thus collapses the boundaries between language and objects, words and things.  Both, he claims, are part of and produced out of discursive systems that bind them together and define their encounter.  Foucault encourages us to dispense with “the enigmatic treasure of objects anterior to discourse” and redirect our attention to the body of rules that enable [objects] to form […] and thus constitute the conditions of their historical appearance.” (12)

In the “Historical a priori and the Archive,” Foucault articulates precisely what this means for history.  As Elsaesser rightly points out: no origins and no beginnings.  But we also lose artefacts, events, and historical knowledge.  We can only write histories of discursive formations, of the systems that produce historical events: “Instead of seeing, on the great mythical book of history, lines of words that translate in visible characters thoughts that were formed in some other time and place, we have in the density of discursive practices, systems that establish statements as events […] and things […]. They are all these systems of statements (whether events or things) that I propose to call archive.” (13) Foucault invites us to forget History.  Or, at least, to reframe what we mean when we use the term.  The historical event joins words and things in the vast repository of discursive formations.  History becomes a “complex volume”: heterogeneous, discontinuous, fragmented by changes and transformations in discourse, in the body of rules that regulate its appearance and representation.  It follows, then, that the writing of History—a combination of words, objects, and events—is a poetic, creative act like any other discursive practice.  For Foucault, the historian’s task unfolds within and upon the archive, at once a system of statements and a heterotopian counter-site.  It is the fantastic figure of a boat at sea—not the integrity and fixity of a book or monument—that stands in metaphoric, proximate relation to historical practice. (14)

Foucault is not the origin of history’s epistemological crisis.  He belongs to a much broader set of twentieth-century discursive formations, which includes the historiography of Michel de Certeau, Hayden White, and Dominick LaCapra, among others.  Together, they extricate historical studies from the sciences (both natural and social); bring the discipline into contact with developing concerns in the literary humanities; and reconsider both history’s formation in the present tense and its obligations to present time.  De Certeau is perhaps Foucault’s closest ally in historical thinking, insofar as he understands history as a discursive construction and the practice of history as a process of making, forming, and fabulation: “From wastes, papers, vegetables, indeed from glaciers and eternal snows, historians make something different: they make history.” (15) For his part, LaCapra claims that no one did more “to wake historians from their dogmatic slumber than Hayden White.” (16) In “The Burden of History,” (1966) a foundational contribution to what would come to be known as the “linguistic” or “discursive” turn in historical studies, White puts pressure on the contingency of historical methodologies, on the one hand, and the fierce insularity of the discipline, on the other.  Historical practice, he argues, cobbles together a toolbox from late-nineteenth century positivism and mid-nineteenth-century art and literature, “modes of analysis and expression that have their antiquity alone to commend them.” (17) The burden of the contemporary historian is to open up the borders of the field and recognize the methodological impurities that have been there all along.  If historical studies acknowledged the experimentations already at the centre of its disciplinary practice, it could begin to experiment more explicitly and radically, while likewise interrogating the experiment as such and making the diverse, multiple forms of historical knowledge visible, legible, and open to critique.  No longer beholden to an impossible and objective History, historical studies could embrace manifold and imaginative histories as well as a rigorous analysis of its multiple historiographic methods.  This approach would “permit the plunder of psychoanalysis, cybernetics, game theory, and the rest […].  And it would permit historians to conceive of the possibility of using impressionistic, expressionistic, surrealistic, and (perhaps) even actionist modes of representation.” (18) As a discipline in the mid-twentieth century, history concealed the present tense of historical thought, as well as the dynamic practices that constituted contemporary life.  For White, history could no longer remain hermetically sealed against the complex present of its own production and still defend itself as a disciplined endeavour.

If, as LaCapra suggests, White woke the discipline, others would more thoroughly deconstruct it.  LaCapra’s own work “plunders” literary theory and psychoanalysis in order to interrogate the aleatory, performative, and ambivalent aspects of rhetoric, as well as the extremist positions of historical practice: documentary objectivism and self-reflexive relativism.  While he criticizes the rhetorical tropes—metaphor, metonymy, synecdoche, and irony—that guide White’s intervention and re-structure the discipline, he nevertheless extends White’s concern for the rhetoric that shapes historical expression and the temporality that binds historical thought.  LaCapra takes the concept of “transference” (from Freud) as a way of framing (and, in many ways, simply naming) the encounter between past and present, artefact and historian as a necessarily dialogical one.  LaCapra understands the historian’s craft as an imprecise and imperfect exchange of voices, “a certain excess in relations between self and other that calls for understanding and representation yet is not fully open to mastery or knowledge.” (19)

In the practice of early and silent film history, we have largely elided the crises and questions that redefine historical studies in the late twentieth century.  There is, perhaps, an historical explanation for the blind spot.  The study of early film history was developing as historical studies writ large entered this era of upheaval. The origin of early film studies coincides with the first wave of responses to Of Grammatology (published in English translation in 1976), as well as the publication of several key post-structural texts that engage explicitly with historiographic concerns. (20) The 1978 International Federation of Film Archives (FIAF) conference in Brighton, U.K., brought early film scholars and archivists together for the first time and screened an unprecedented number of films from archives around the world.  This (now, mythic) meeting nevertheless countered the post-structural discussions of ontology, absence, and difference with visibility and visual plenty.  For early film scholars, 1978 marks a year of material abundance, a moment in which the startling void between the Lumière factory and the Griffith studio was filled with nitrate.  It is little wonder that the problems of history did not present themselves in the very moment at which so many problems seem to have been solved.  In the decades that follow this first encounter, the archival impulse to collect, preserve, and restore early film material defines the methodology of the early film historian.  Even as scholars like Richard Abel, Tom Gunning, Charles Musser, Ben Singer (among so many others) explore the fluid movements of film practice from the fairground to the music hall, as well as the complex interactions between film and modernity, both the ontology of film objects and the assumptions underlying film-historical methodology go uninterrogated (when they are not wholly invisible).

In his comprehensive handbook for film historians and archivists-in-training, Silent Film: An Introduction (now in its third edition), archivist-historian Paolo Cherchi Usai seemingly disrupts these foundations and takes a radical metaphysical view of the archival object.  In his tenth and final “rule” for film historians, he defines the individual film as inherently unstable: “The original version of a film is a multiple object fragmented into a number of different entities equal to the number of surviving copies.” (21) For Usai, every original film is always and already fractured into multiple copies that have been circulated and seen and come to rest (or deteriorate) in an archive.  Usai distinguishes between external and internal histories of film, or the histories of circulation and spectatorship, versus the history of a particular celluloid copy.  In several passages, he describes film history as an impossible task and an act of “imagination,” echoing both Foucault and de Certeau.  He likewise recalls LaCapra’s concern for the uncertain and excessive exchange between films and film historians.  This, too, requires imaginative leaps.  However, Usai does not develop the concept of imagination, nor does he propose a new historical method out of his redefinition of the film artefact (as multiple, fractured, imprecise). Rather, “imagination” names a privileged kind of work that can be done, but not explained, taught, defended, or analysed.

For all of film’s scratches, flaws, and multiple, irreconcilable histories, Usai manages to recuperate film and the practice of history beneath a set of incoherent signs: the self-evident and the sacred. He argues, “Whatever archive we have decided to visit or conceptual tool we have adopted, the one thing which should never be questioned is that films are given a chance to speak for themselves.” (22) Usai nevertheless pivots from the film that speaks for itself to an enigmatic form that has nothing to say.  Like gods, myths, and foreign bodies, film is a mystery; only the most faithful visionaries can understand it:

There is a gap between the producer of silent motion pictures, the contemporary viewers of these images, and today’s audiences.  We may well attempt to fill this gap, but the absences are very deep in the case of silent cinema: too many material and historical variables separate us from it, and our patterns of perception of moving images have remarkably changed in the meantime.  Herein lies the challenge of studying silent cinema: both the greatest discipline and a visionary mind are needed in order to bring back to life something which is relatively close to us in time.  It is closer than prehistoric art or the music of ancient Egypt, but it can be no less mysterious and elusive. (23)

In the end, Usai’s historiography echoes nineteenth-century travel writing more than twentieth-century historical thought.  But it is precisely this anachronistic and theoretically ambiguous approach to film artefacts and historical practice that, more broadly, informs the methodologies and epistemology of film history.

Usai’s handbook poses a set of important historiographic questions that cannot be solved by the autonomous voices of film, nor the visionary mind of the historian-seer.  If we are separated from film objects and audiences, what is the dialogical, imaginative work that bridges the gap?  If, as Usai suggests, we recognize the inherent “multiplicity of ‘original’ prints,” as well as the internal and external histories of every copy, how, then, do we ever come to know film and what kind of knowledge do we have when we do (or when we do not)?  And, finally, if we understand that a film is comprised of multiple copies, each of which is in a state of continual becoming (decayed or restored), what are the burdens of the film historian to trace these changes across the twentieth and twenty-first centuries?  Which changes matter for history and which do not?

A final historical note.  In 2004, Cinema Journal gathered a series of essays on the state of historical thinking from a diverse range of film scholars. Taken together, the contributions reconfirm the stability and hermeticism of film history, even as they call for intertextuality and comparative methodologies.  With few significant exceptions, the essays hum in a kind of collective agreement about best practices. They portray a discipline without any difficult questions to answer and no significant crises to theorize, analyse, or debate.  Among the contributors, only Jane Gaines and Robert Sklar sketch possible lines of flight out of a field that cannot seem to engage the contingency of its own becoming, nor think the dialogism of its historical methodologies.  Gaines proposes a feminist historiography that would not only recuperate the histories of women, but also challenge the “classical,” linear narratives that dominate film history and conceal their absence. (24) In his contribution, Sklar describes the experience of attending a conference with film historians in the 1980s; at the time, he noted that these newcomers to historical studies were “whiggish” in their disciplinary confidence and unprepared for the upheavals to come.  The film historians “had not yet fully recognized that the practice of historiography is fundamentally dialogical.” (25) Sklar recalls thinking that film history, like any other historical field, would eventually reflect on its own methodologies and encounter a series of metahistorical crises.  He predicted that “in several decades’ time emerging film historians would ask new questions about the past and debate new perspectives that were likely to be substantially different from those that scholars of the 1980s had valorized.  Film historiography almost certainly would have moved on to territories as yet uncharted.” (26) Those crises and questions never came.  While film history has expanded to include the previously excluded (women, racial and ethnic minorities, queer communities), the territory of the discipline has never been deterritorialized (and reformed).  He concludes, “What remains lacking is a discourse on metahistoriographic perspectives that might pull together multiple strands and reorient the field.” (27) In what follows, I would like to return to the remix in order to take up the task that Sklar sets for us.

Sunbeam: Signs of Life and Lines of Flight

Variation on the sunbeam from Aitor Gametxo on Vimeo.

On April 21, 2011, a twenty-two-year-old Basque film student named Aitor Gametxo uploaded a remix of D.W. Griffith’s one-reel Sunbeam (1912) to Vimeo.  By his own account, Gametxo had watched Griffith’s film only once before beginning his “deconstruction of the original work.” (28) Gametxo’s Variation on the Sunbeam divides the screen into six distinct frames (three on the top and three on the bottom) and redistributes Griffith’s film across this new cartography.  The remix spatializes narrative time as it challenges the frame-by-frame linearity of celluloid, one of the very structures of film that Griffith’s narrative techniques made more flexible (not “this” and then “that”, but “meanwhile”).  In Variation, events unfold simultaneously, side-by-side in visible space.  Gametxo reveals the internal coherence of Griffith’s divided mise-en-scène, as characters travel seamlessly up and down stairs, through doorways, across the cut.

More importantly for our purposes here, the film offers a model for thinking metahistorically.  I would like to pose this small visual expression—one example, among many, circulating online—against the origins of film history and the methods of film historiography.  I also would like to take this work seriously as an act of “deconstruction” and a metahistorical form of expression. Gametxo’s Variation offers one possible model of post-structural historiography.  It acknowledges the gaps and absences in historical understanding (along with what we already know); foregrounds the contingent and dialogical encounter between historian and artefact; explores the complex materiality of analogue and digital copies; and, most startling, replaces the act of writing with the process of digital remixing. Variation moves in two simultaneous and contradictory directions at once: back to the origins of film and the foundations of film historiography and into the digital present where both those origins and foundations get remixed.

By the end of August 2011, Variation on the Sunbeam began to attract the attention of cinéphiles, professional scholars, and film historians.  It was featured on the Audiovisualcy group forum at Vimeo, where it attracted the attention of film critic and video essayist Kevin B. Lee.  On August 31, Lee posted the video with a brief description at his Fandor Keyframe website.  On September 5, Kristin Thompson shared Variation on the blog she writes with David Bordwell, Observations on Film Art. (29)  On September 6, Roger Ebert tweeted about the video with a hat-tip to Thompson and Bordwell.  On September 8, Luke McKernan shared the “singularly inventive film” on The Bioscope, a blog dedicated to silent cinema.  McKernan wondered why more silent film “deconstructions” could not be found online: “Do the films not interest students and lecturers as much as they might?  Do they lack the sense of cool that may come from deconstructing the cinema of today?” (30) To date, Variation on the Sunbeam has been viewed just over 2,000 times.  It’s not exactly a viral sensation, but neither is Griffith’s digitized original with just over 6,000 views on Youtube.  When one searches for Sunbeam online, both versions appear: Griffith’s first; Gametxo’s a close second.  Had the video gone viral, this visual hierarchy could have easily shifted (and, of course, there’s still time).

Among those who posted the video, Thompson offers the most substantive analysis of the relationship between Griffith’s Sunbeam and Gametxo’s Variation.  Thompson distributes praise where the remix adheres to the original film and the “already-said” of film history.  She commends Gametxo’s attention to the temporal expressions of intercutting, as well as the way in which the video’s six frames faithfully represent the domestic space of the original film.  The child’s room is always positioned in the upper left corner, the top of the stairs appears in the top centre frame, and the two downstairs rooms are positioned on the left and right bottom frames with the hallway in-between them.

While Thompson applauds what the amateur Gametxo repeats and gets “right,” she nevertheless finds “technical disadvantages” in precisely what he distorts, excises, or contributes in excess of the original film.  Put another way: Thompson criticizes Gametxo’s remix for being a remix.  Thompson seemingly mistakes the remix for an act of repetition or substitution and reminds us (somewhat redundantly) that the remixed copy fails to replace the original work:

It would be impossible, I think, to entirely follow the story just from seeing Variation. The shots are so reduced in size to fit into the grid that small but important gestures and details get lost. […] The titles are small and difficult to read, and since they pop up simultaneously with the action, it’s almost impossible to read them anyway. One cannot tell where the titles originally came in the flow of shots, though one can always check the original film. Another problem is the cropping of the images on all four sides. (31)

There is perhaps little risk of viewers confusing Gametxo with Griffith.  The real threat emerges elsewhere and is one that Thompson indirectly announces in her own reading: “What is remarkable is that a 22-year-old film student […] found a simple, elegant method to demonstrate what we already knew, but with greater precision and vividness than could be done with prose analysis.” (32) In this brief passage, Thompson not only frames writing as a form of historiography among other possibilities, but she also nominates Gametxo’s video (and perhaps, the “deconstructive” remix writ large) as one of those possible alternatives.  Here, Thompson suggests that film historiography could be otherwise (and, it seems, better).  Even more interesting, this small sliver of a statement encourages us to compare the written to the remixed, text to image, and to notice that both approaches stand in a distant, disconnected relationship to historical events and objects.  In other words: the remix seems to communicate something about film history and film historiography.

I would argue, with Thompson, that Gametxo’s remix deviates considerably from Griffith’s original Sunbeam.  But I would also argue, against Thompson, that it is precisely these deviations and departures—the video’s imprecise and dialogical play with the original—that make it a valuable work for metahistorical thinking.  While Variation on the Sunbeam reconstitutes Griffith’s narrative forms and visual architectures across its six individual frames, it also rips Griffith’s film apart and reshapes the fragments into a series of open and seemingly endless permutations.  In the variation, narrative events expand across the frame, unfold simultaneously, and collide with the film’s original Biograph titles, always too soon or too late to offer any kind of explanation.  Not only does the remix exclude portions of Griffith’s Sunbeam, but its multiple frames also contain multiple, simultaneous narrative events.  With each viewing, new points of contact and comparison emerge.  These visual structures offer a variation on Griffith, as well as a metahistorical study of variation itself.  The film makes the instability and flexibility of film artefacts visible and integrates the multiple, wandering circuits of reading and interpretation into the experience of spectatorship.

Like all remixes (and, one should note, all histories), Variation on the Sunbeam forcefully inscribes the present tense upon the film artefact.  The remix demands that we see and read its contemporary time.  It is these traces of the new and of the now that perhaps account for what Luke McKernan perceives as “that sense of cool.”  Less noted, however, is the complex and inextricable way in which the remix joins past and present.  In Variation, the materiality of Griffith’s familiar mise-en-scène meets the ephemeral nothingness of digital divisions; the rips in a particular celluloid print are joined by digital noise; and the American auteur collides with a young student living in Barcelona.  Both Lee and Thompson invited Gametxo to share his motivations and methodology.  His response emphasizes a dialogical encounter between past and present, between the contemporary “I” and an altogether different time and place:

I love watching old (and odd) films and thinking about things that are different from the purpose they were created for.  We are able to take some footage, which is temporally and geographically unconnected to us and remodel, or refix, or remix it, giving birth to another work. This is the way I see the found-footage praxis. About this particular film, The Sunbeam, […] this [film] was unknown for me, so that the first watching was crucial. While I was enjoying it, I was wondering what the place where it was shot looked like. I suddenly imagined it as a two-floor house, where the characters cross in some moments. Also the doors were essential to fix one part with another. This was the main idea where I worked on. (33)

Gametxo divides his process into two distinct impulses: thinking against the original object and imagining the past.  Both impulses designate an act of creative fashioning, born out of the distance between Gametxo and Griffith, the digital present and film-historical origins.  And both impulses underpin all forms of historical expression, whether written or remixed. Gametxo’s methodology resonates, however accidentally, with the post-structural historiography that White, De Certeau, and LaCapra envision.  What is more, Variation on the Sunbeam includes a distinct figure of the gaps that give way to acts of imagination and the boundaries in our own historical understanding.  Variation inscribes Griffith’s Sunbeam within a visual tableau of absence and emptiness or, perhaps, digital presence.  Gametxo never fills all six frames with visual and/or textual content.  Rather, the original is reduced in size and recombined with itself, but always tied to at least one empty frame.  In this way, the “already-said” of film history encounters an unmistakable sign of the unknown, unknowable, and the remixed (reimagined) histories to come.

But let us set the “I” and Gametxo himself aside.  Or think differently about the production of historical knowledge.  I have already placed too much pressure on one remix and perhaps inadvertently produced one new, digital auteur.  In Variation on the Sunbeam, what so strongly counters our historiographic intuitions is, on the one hand, that the film was made by a student who had only recently begun his study of film and, on the other, that it could have been made without a student, without a human, without a mind for history, with little more than a software program and the internet’s digital scraps.  One could easily mistake the rhythmic patterns of Variation for a variation on Lev Manovich’s Soft(ware) Cinema and one would be excused for suspecting that Gametxo might just be one of the many uncertain formations of identity (virtual, collective, imagined) that posture and play tricks online.  There is nothing about this video that requires extensive training in film history, nor even the intervention of the human hand.  And yet, as Thompson claims, it repeats—unknowingly—what we already know.  How do we explain this coincidence of knowledge?  How do we understand the historical gifts of the amateur or the automatic?  How does the digital remix manage to threaten professional prose with a more “precise” or more “vivid” history?  In the end, Variation does not improve upon film history.  This is the wrong (evaluative) framework.  Rather, it returns us to the post-structural shift in historical thought and reminds us of the accidents, imprecisions, and contingencies implicit in every historiography, perched as historical practice always is between different discursive formations (historian and archive, present and past, expression and events).  If Variation manages to approximate what we already know, it does so using a toolbox of found footage and found methods that it shares with an equally contingent discipline.  Variation nevertheless communicates more than what we already know (and less of what we do).  It repeats some things and not others.  It remixes with a difference.  This does not exclude the video from film history or historiography, or somehow define it as a technically deficient practice.  Variation instead productively counters the stability of film historiography with the possibility of manifold and imaginative alternatives, each of which produces new forms of historical knowledge.

The digital remix is a form of film history, but it is also a challenge to the hermeticism of film historiography and a radical point of comparison for metahistorical thought.  As a practice, film history remains disconnected from historical studies elsewhere, as well as from the present of its own production.  Claims to methodological precision, rigour, and neutrality, as well as a rhetoric of care for film artefacts and film history, have preceded debates about methodology and rhetoric, and elided theories of history and film materiality altogether.  In this essay, I have argued that we open the boundaries of the field to both analyse the methodologies we insist could not be otherwise and consider whether we might be missing the meaning of (historical and methodological) differences.  If the remix is not an historiographic approach we are willing to take seriously, we need better arguments in defence of the serious methods we have settled upon.  In any event, the remix is the future to come, the object of our future histories and historiographic concerns.  The debate promises to be worth the effort.

Endnotes:

Frames # 1 Film and Moving Image Studies Re-Born Digital? 2012-07-02, this article © Katherine Groo. This article has been blind peer-reviewed.