CyberTouch: An interdisciplinary research on Dance, Music and Live Coding

Name: Orpheus Instituut
Price range: $

Artist statement by Vasileios Agiomyrgianakis and Stella (Styliani) Dimitrakopoulou

Share this article

"CyberTouch" is an experimental audiovisual performance which combines dance and music with live coding, and a prototyped wearable interactive technology to create audiovisual experiences. It is the outcome of an interdisciplinary research informed by Cybernetics, a structured improvisation between two individuals, a composer and a performer. Our goal is to explore and propose new relationships and strategies in performing practices and interactive audiovisual arts, while expanding the potential of the human body.
... +

Introduction

“CyberTouch” is an experimental audiovisual performance which combines dance and music with live coding, and a prototyped wearable interactive technology to create audiovisual experiences. It is the outcome of an interdisciplinary research informed by Cybernetics, a structured improvisation between two individuals, a composer and a performer. Our goal is to explore and propose new relationships and strategies in performing practices and interactive audiovisual arts, while expanding the potential of the human body.

Interdisciplinary collaborations do more than construct new ways of making art because they are positioned at the intersection of different domains of practice and research. This performance shows the immense potential for inventive art creation, through a combination of profound knowledge of the arts and artistic sensibility with a solid awareness of technology, languages and capabilities.

Using structured improvisation we experiment with movement, text, sound and imagery. Additionally, movement data are processed and explored through the senses using a wide range of modalities (multimodal data) such as sonification, visualisation, and movement (Agiomyrgianakis 2022). Embodiment through body movement translation is also explored using multimodal interaction.

The conceptual framework is based on low-cost multi-sensorial networked technology and multimodal sensing such as measurement (quantity through the numbers), processing (through sensing), and finally meaning. The acting architecture contains sensors, data channels, numeric data with digital signal processing, and rendering with a multimodal output including sound, movement, graphics and text (Agiomyrgianakis 2021). So, our goal is to both theoretically investigate the necessary design concepts and practically construct a modular and portable performance environment that is a) Embodied by wearable sensors, b) Embedded in an audiovisual live coding framework, c) Enacted through dance.

Preparation

The design of the wearable (glove) emphasizes a manner that is both ergonomic and straightforward to utilize. The glove is made of a thin elastic fabric and fits a medium-sized right hand. It is stitched with layers of self-gripping strap to allow for easy insertion and removal of components such as the Raspberry Pi Zero W and sensor, as well as the battery and buttons. Except for the thumb, the goal was to add two buttons to each of the glove's fingers.

Influences

Ascott describes in his book “Telematic Embrace” (2007) that it is crucial to create joints between scientific discipline and contemporary aesthetic discourses. In CyberTouch we decided to draw metaphorical analogies in order to build a bridge between art and cybernetics.

Being interested in the body's relationship to interactive audiovisual technology within performance, our choreographic and audiovisual aesthetic is informed and inspired by artists such, Merce Cunningham and John Cage, the New York group Troika Ranch and Loie Fuller, among others. Furthermore, our influence comes from recent studies such as examples of a full body suit as part of an open-source initiative by chordata.cc which have been used in game design, animation, digital arts, electronic music, or physical therapy. In the sphere of musical performance, substantial study has been done on gestures and motion tracking and interactive technologies are used to link stage movement to music and video during the performance (Wanderley and Batier 2000, Schneider 2010, Alaoui 2019).

In particular, CyberTouch and Loie Fuller's Serpentine dance are two works that bring on stage the latest technology of their era. Loïe Fuller in her Serpentine dance (1920) used electric bulbs to colorfully light the stage and the costumes. CyberTouch makes a clear reference to Fuller's Serpentine dance, considering it an epitome in the field of dance and new technology. As the art historian Tom Gunning writes, her Serpentine dance “provides the most complex example of a technological art of motion” (p. 85). There are certainly many similarities between the two performances to the extent that CyberTouch is a tribute to the Serpentine dances from the 1920s to the 2020s with the technology of our era. It is not meant to be a reconstruction or a reenactment of Fuller’s performance, it is rather to propose a revisiting of this work as “an act of love” (Dimitrakopoulou 2016). It is also important to note a great difference between the two. The electric bulbs used by Fuller were an invention for general use, whereas in CyberTouch a technological device was created for the creation of an artistic work specifically made from the perspective of the performer's body. It is exactly this relationship to the performing-dancing body that we further examine, looking for the ways that this technology alters, extends or constrains the performing body of a dancer.

Furthermore, quite a few dance productions from the 1990s tested and utilised a variety of interactive technology. Another illustration of how a "dialogue" is performed between the technology and the body staged as expanded by the technical components is the dance solo In Plane (1994) by the New York group Troika Ranch (Leslie 1997). As in the work In Plane, CyberTouch uses a wearable technology with wireless motion sensing system, which measures the degree of movement and sends the movement data via OSC communication through Wifi transmission to the computer so as to control parameters of computer graphics, and sound. Technology is shown as an extension of the body that is controlled remotely via the Glove wearable system. The piece also highlights the constrained choreographic possibilities of this context, as the body must service the glove’s sensors, even while technology as a body integrated extension appears to allow the body to manage the surroundings remotely.

A multiple transformative body

In CyberTouch, the dancer (Stella Dimitrakopoulou) dons a long white dress reminiscent of the one used by Fuller in her Serpentine dance. She holds a stick in her left hand to elongate the dress, just as in the original performance. Additionally, she wears a handmade glove on her right hand that features a gyroscope and 8 pushbutton switches. This glove continuously transmits data to the computer via WiFi, which in turn controls the SuperCollider and openFrameworks programming environments, generating sounds and graphics. During the performance, the dancer employs her movements to modify the audiovisual environment, which can also be adjusted in real-time by the live coder.

In Serpentine dance all materials were equally important and nicely balanced, as Vaso Barbousi writes, “for Fuller, the fabric with the folds was tied to her dance, her artwork” (p. 37, translated by S.Dimitrakopoulou). Respectively, in CyberTouch, movement, sound, text and graphics, costume and new technology are brought together in order to create a whole. The costumes in the Serpentine and in CyberTouch are not only garments nor ornaments, but instead they are functional parts of the artwork providing multiple layers for screening in motion and extending the dancer's body. In this context, the body functions as a surface or a series of surfaces upon which the code and visuals are projected. While this may suggest that the body becomes an object, it is important to note that the body is far from static in this performance. Instead, it is in a constant state of motion and transformation, consistently altering its appearance and relationship to the technology being employed.

With the support of the fabric in motion and the colorful lights of new technology, the body is transformed, creating various images. Critics of the time such as Octave Mirbeau (1893), Jules Huret (1900) and Arthur Symons (1899) wrote about Fuller that she looks like a butterfly, a flower, a phantom, a fairy, a flame, a cloud, a rainbow and a meteor. This transformation made the Serpentine the epitome of Symbolist ideals. According to Tom Gunning “the symbolists found in Fuller's dance an image of their own conception of creativity: the symbol as the Radiating centre of unending transformation” (2000, p. 42). In CyberTouch, the intention is not to create particular images to be deciphered and analyzed as symbols. Instead, the performance aligns with Henry Bergson's concept that “the body is changing form at every moment; or rather there is no form, since form is immobile and the reality is movement. What is real is the change of form: form is only a snapshot view of a transition” (p. 302). This suggests that the performance is not concerned with the creation of fixed images or symbols, but rather with the fluidity and constant change inherent in movement and the interaction between the performer's body and the technology.

Returning to the Serpentine, critics wrote of Fuller that she is a “cloud constantly changing with the rhythm of an invisible body” which makes her be seen as “an apparition, not a woman of flesh and blood”, “the body appears only through its rhythm, which simultaneously emerges from the body and conceals it” (quoted in Gunning, 2006, p. 32). Therefore, by not adhering to any specific form, the dancer embodies an apparition, an Art-Nouveau sculpture in motion, and a futuristic android. What is described is multiple bodies that can be seen as an object (the projection surface), a series of images in motion and an apparition without flesh, a body that is physically concealed in order to re-appear in different constantly changing forms through various mediums such as light, rhythm and movement. In CyberTouch, these multiple manifestations of the dancer’s body are extended further to include also the body as a machine, a musical instrument and an apparatus.

Observing the body in this specific action and in relation to Merleau-Ponty's phenomenological approach that defines the body not as an object but as a subject through which we are placed in the world, this work comes to redefine the body and propose a body that exists simultaneously as a subject and as an object. The body is not only a projection surface, but also a human organism and a machine that can select and activate the visual and auditory environment through its movements. From a phenomenological perspective, "the clarity of the boundaries of the body, the distinct components of identity, the borders between human organisms and machines" are destabilized once again (Makrinioti, 2017, p. 18). This hybrid form becomes the central element of the live audiovisual performance that explores the body in cyberspace. A multimodal body that does not differentiate the human from the cyborg but rather explores further the increasingly blurred boundaries between human and machine, following Donna Haraway’s (1985) argument that we must embrace this hybrid identity in order to create a more equitable future.

CyberTouch performance involves various ways in which the transformed body is featured. The body is presented as an apparition, a hybrid multi-instrument that creates both sound and moving images, and as an author-creator who participates in a real-time collaborative process. Furthermore, the idea of the body's multiplicity that is explored in the Serpentine performance is expanded upon in CyberTouch to encompass the multiple roles of the performers on stage. This allows for a dynamic and multifaceted presentation that engages the audience in a unique and captivating way.

Collaboration: Improvisation as a means for a multimodal dialogue that blends the distinctive artistic roles

Who is the choreographer, who is the composer, who is the performer? We mix and blend the distinctions of these different roles since we both improvise in the moment changing all modes (sound, graphics, movement, text). It is a conscious decision to leave this open and fluid in the form of a conversation instead of constraining it in order to create a scripted performance work with fully prescribed choreography, composition and visual imagery. Instead this experiment is a duet between artists who through movement can manipulate sounds and graphics on the fly.

In our effort to bring together two distinct artistic and scientific fields of research, we find that improvisation provides a fruitful ground for the development of a common language. Drawing on tools from improvisational methods such as those developed and used in dance, music and live coding, we create a channel for interaction and communication between us. We discovered that free and structured improvisation as the first attempt is a suitable and secured method to practice and explore a multimodal dialogue between performers.

The improvisation included four different modes, text, sound, graphics and dance. Below we describe these main four elements of improvisation and the ways that these were developed.

TEXT: Two windows that contained coding evaluation and data representation. Coding evaluation included live coding for the creation of sound and visuals, comments related to computer music and computer graphics techniques. In order to give the coding environment artistic value and to make the live coding notation easier for the audience to read and understand, comments are written in accordance with their relationship to the code.

SOUND: The primary live coding framework for sound includes sounds from both live coding and dancer's wearable system. In that instance, the live coder can perform and switch between a variety of sounds that are evaluated by live coding in addition to switching between the wearable system's noises and their behavior while being performed on the fly. A variety of sound synthesis techniques, including additive, FM, granular, sampling, and wavetable, were used to create each sound, as well as, generative processes and algorithmic composition paradigms like Chaos and self-similarity. All the sound and data management of CyberTouch are based on BA classes (Agiomyrgianakis, 2021) for live coding and performance practices in SuperCollider.

GRAPHICS: Four different visualization environments controlled by the dancer through the glove. Their creation was based on the particles and flow dynamics framework which were developed in four different environments. In this framework the live coder and the dancer can control the parameters of the particle system and fluid dynamics which is based on a flow dynamics C++ library in openFrameworks by Mehmet Selim Akten. This is a real-time 2D cell-based fluid and particle simulation in which parameters can be manipulated by the wearable and the live coding. The buttons of the wearable can switch between 4 fluid - particle simulations. Additionally, the data from the accelerometer sensor are used to manipulate parameters such as size, density, and colour of particles amongst others. The aforementioned four graphic and sound environments that the dancer controls were interconnected and controlled through the glove and the movement. Thus, the first sound mode was always running together with the first graphic mode, the second sound mode with the second graphic mode and so on.

DANCE: Early in the rehearsals it was made clear that the main aspects that influenced the sound and graphics through the movement of the right hand, were the speed and the position of the right hand in space (x, y, z axis). It is important to note that the choreography was not following a specific melody or rhythm of the music since the music was not performed based on rhythmic events, instead it was based on chaotic patterns. Thus, it was appropriate to develop and structure the choreography as a structured improvisation. During rehearsals the movement material was created in response to the four main sound and graphic environments. The different qualities of the sound and the graphics would lead to different atmospheres through repetition. The performance would last for about 10 minutes and was structured in 5 parts based firstly, on the creation of different images (influenced by Fuller’s images) and secondly, on the speed of the movement that was altering the sound. These five stages were the following: 1. Introduction - Slow pace, 2. Butterfly - Acceleration of speed, 3. Cocoon - Slow pace 4. Gun Like Arm ( Slow and steady pace), 5. Closing (Medium pace and slowing down to stillness).

The relationship between movement and sound was therefore based on the creation of four different atmospheres rather than on a detailed composition or choreography that would work as a basis determining the direction of the whole performance. Structured improvisation was a great tool for us to allow enough space for the chaotic principles to be progressed both in choreography and in music. In the future we can foresee trying out other chaotic structures focusing more on the use of chance methods and random patterns, such as those used by John Cage and Merce Cunningham.

Although the sound and the movement were interconnected, this interconnection was not necessarily made visible to the audience exactly because of the chaotic behavior of the sound. The repetitive development of the movement was creating a recognisable rhythmic choreographic pattern but this was not obviously translated to counter rhythmic events in sound. A combination between chaotic and rhythmic patterns in the future would be interesting to explore further as it would reveal more clearly the interconnection between sound and movement to the audience.

The blending between movement, sound, text and graphics through structured improvisation allows us (a choreographer-dancer and a composer-live coder) to expand our roles even further. The dancer-choreographer on stage is given the tools to become also a composer and a visual artist that creates their surrounding environment through her movement. The composer-live coder is given the opportunity to extend his set up to dialogue between art-forms (dance and audiovisual livecoding).

Video Documentation

CyberTouch was presented at Chimeres Space in Athens (Greece) on January the 10th, 2020 :

Conclusions

This system, including the wearable controller and the live coding framework for computer music and graphics, aids in the creation of collaborative projects and performances including choreographic and live coding practices. It extends both the dancer's body and the live coding setup. The dancer can simultaneously think and move as a composer, a musical instrument, a graphic designer and a choreographer. In order to successfully develop a choreography for both movement, sound and visual composition, the dancer is required to become a proficient user of the wearable technology and to be familiar with its functioning

In the past ten years, similar acts employing related technology have been presented, however so far these actions are primarily displayed in constrained artistic venues and Universities and discussed mostly among academics. In order to disseminate this technology to the broader public, one of our goals is to display it in public and underground creative places like the Chimeres Space in Athens. Students and other interested parties who come into contact with this technology can use it thanks to its development and low cost nowadays. The bulk of the audience members who attend our performances are the first to experience the fusion of technology and the arts; they are also the potential users of this technology in the future for both research and artistic purposes.

In order to investigate more connections and links, we intend to develop the primary materials (audio, visual and movement) as well as the technology, including, among other things, skeletal tracking techniques and machine learning. Also, we want to improve the dramaturgy of the performance so that the audience can understand the key components more clearly. These components include live coding, technology, the relationship between art forms, and interactivity amongst the performers.

References

Agiomyrgianakis, Vasilis., Zannos, Ioannis. 2022. Exploring Scientific Data Through Dance: An Alternative Approach to Sonification as Form of Artistic Expression. International Conference DCAC. Corfu.

Agiomyrgianakis, Vasilis. 2021. Live Gloving: Experimental schemes for live coding performances. International Conference DCAC. Corfu. 2021.

Agiomyrgianakis, Vasilis. 2021. ISW: Experimental schemes for live coding performances. An experimental videogame with a wearable controller for creating immersive soundwalks. International Conference DCAC. Corfu.

Ascott, Roy. 2007. Telematic Embrace: Visionary Theories of Art, Technology, and Consciousness (E. A. Shanken, Ed.; First edition). University of California Press.

Barbousi, Vaso. 2004. Dance in the 20th Century. Athens: Kastaniotis.

Bergson, Henri. 1911. The Cinematographical Mechanism of Thought and the Mechanistic Illusion -- A Glance at the History of Systems -- Real Becoming and False Evolutionism, Chapter 4 in Creative Evolution, translated by Arthur Mitchell, Ph.D. New York: Henry Holt and Company. (pp. 272 – 370).

Dimitrakopoulou, Stella. 2016. (Il)legitimate performance: copying, authorship, and the canon. Doctoral thesis, Trinity Laban Conservatoire of Music and Dance.

Fdili Alaoui, Sarah. 2019. Making an interactive dance piece: Tensions in integrating technology in art. In Proceedings of the 2019 on designing interactive systems conference. (pp. 1195-1208).

Gunning, Tom. 2006. Loie Fuller and the Art of Motion. In Masterpieces of Modernist Cinema, edited by David Bordwell and Kristin Thompson, 23-41. New York: Columbia University Press, 32.

Gunning, Tom. 2000. The Films of Fritz Lang: Allegories of Vision and Modernity. London: British Film Institute.

Haraway, Donna. 2013. A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century. In Simians, Cyborgs, and Women: The Reinvention of Nature, 149-181. New York: Routledge.

Huret, Jules. 1900. Enquête sur l'évolution littéraire. Paris: Bibliothèque-Charpentier.

Makrynioti, Dimitra. 2017. The Limits of the Body: The Phenomenology of Somatic Experience. Patakis Publishers.

Miller, Leta, E. 2001. Cage, Cunningham, and Collaborators: The Odyssey of ‘Variations V.’ The Musical Quarterly, vol. 85, no. 3,, pp. 545–67. JSTOR, http://www.jstor.org/stable/3600996.

Mirbeau, Octave. 1893. La Danseuse. Le Figaro.

Sawin, Kepner, Leslie. 1997. Dance and digital media: Troika Ranch and the art of technology. Digital Creativity, 8:1, 11-19, DOI: 10.1080/09579139708567069

Symons, Arthur. 1899. The Symbolist Movement in Literature. London: Archibald Constable and Company.

Wanderley, M. Marcelo., and Batier, Marc (eds.). 2000. Trends in Gestural Control in Music, Ircam- Centre Pompidou.

Imprint

Authors

Vasileios Agiomyrgianakis and Stella (Styliani) Dimitrakopoulou

Issue

Date

30 June 2023

Footnotes

1 This contribution has been madepossible through the financial support of the project «ΔΗΜΙΟΥΡΓΙΚΟΣ ΚΟΜΒΟΣ ΤΕΧΝΩΝ *MIS:5047267*»code 80504, ΕΣΠΑ 2014-2020, ΕΠΑνΕΚ;HAL (Hub of Art Laboratories), co-financed by Greece and the EuropeanUnion and implemented at the IonianUniversity, Corfu.

Back to issue overview

Share this article

Leave reply

Your browser does not meet the minimum requirements to view this website. The browsers listed below are compatible. If you do not have any of these browsers, click on the icon to download the desired browser.

Google Chrome

Mozilla Firefox

Microsoft Edge