On Timbre Networks

Name: Orpheus Instituut
Price range: $

Between Metaphor, Simulation, and Model (and Metaphor)

Article by Juan Parra Cancino and Fernando Rosas

Share this article

go to intro

The pressure imposed by the fast-paced succession of technological advances favours the adoption of new tools as emulation of pre-existent ones, while hindering more innovative and transformative approaches. In this context, one of the persistent challenges is to develop an aesthetic language that can resonate with the transdisciplinary world in which digital music creators operate. This proposal presents the notion of Timbre Networks to address the following questions:

What musical structures can be created in resonance with our transdisciplinary and interconnected world?
Which would be a suitable musical instrument to convey such structures?

... +

tags: timbre, interaction, live electronics, telematics, improvisation, mapping

intro network particle

Background

What musical structures can be created in resonance with our transdisciplinary and interconnected world?
Which would be a suitable musical instrument to convey such structures?

Introduction

Timbre Networks is a creative method developed by Juan Parra Cancino as part of his Doctoral work “Towards a performance practice in Computer Music”, which was completed in 2014. It aims to integrate a tele-communicative, algorithmic, and poetic understanding of the concept of “networks”. This is done by applying dynamic mappings of control streams, which are synthesized from live and telematic musical sources, to network-based synthetic sound agents, such as stochastic synthesis and Boolean network pattern generators (Ref. [1]). From an artistic and performative perspective, this setup aims to expose the multithreaded role of the computer music performer, blurring the boundaries between composition, digital lutherie (Ref. [2]) and performance into an integrated entity. Linearly-structured composition procedures are replaced by networks of interdependent sound engines and manipulators, and rules that determine the initial states of each element and the thresholds where those states are transformed. Performances correspond to the unfolding of this network over time and space, according to the notion of “self-organisation” recently developed in (Ref. [3]). The spatialisation features of the piece are controlled in real performance using Egg, a hardware controller designed in collaboration with Lex van den Broek, head of the Elektronica Werkplaats at the Royal Conservatoire in The Hague, Netherlands.

This particular presentation focuses on a subset of Timbre Networks labelled TN_chain. This is a series of works (or rather, a single chain-shaped work) designed within the research cluster “Music, thought and technology” of the Orpheus Institute, which aims to employ each iteration/performance to inform, define and enrich the "following" iteration/performance. Our choice is to do this by preserving aspects of the performance that might be described as "highly volatile" (for example, a dancer reacting to specific elements of an improvisation music setting). These preserved aspects are converted into a stream of data, which is used to drive (and therefore, automatise) musical parameters of a future performance.

Project

Unlike traditional (and also some artistic) sonification strategies, that seek for a degree of clarity between the data used to generate sound structures and the sonic output, the sensorial results of the Timbre Networks are complex (and confusing) by design. Therefore, a purely perceptual evaluation of how the metaphoric elements negotiate the decision-making process would not suffice, and additional methods are necessary. We propose to perform comparative data analysis over digital traces left by the succession of performances that serves as “digital archaeology”, giving account of evolving traits throughout the various renditions. In particular, we apply various metrics of complexity to signals and digital structures from different past performances, in order to confirm if complexity grows in time following patterns seen in e.g. biological evolution (See Ref. [1]). Through this method, we seek to test a key hypothesis inherent on the creative premise of TN_chain: it corresponds to a single piece, which has a consistent structure that travels through successive renditions. Following this rationale, we explore to what extent the unfolding over time realised by the performers involved deals with two parallel fluxes of time: the “now” of a single performance, and the consequential relationship between the present and the residual ghosts of past performances.

Having focused so far primarily on the setting up of the artistic component of the Timbre Networks project, we are now on the forking of trying to create stronger ties with the scientific community, by means of seeking ways of comparing, analysing, understanding what these networks are and, hopefully, move from a state of metaphor that has been used to create musical outputs towards the creation of a model that can be scientifically proven (and disproven) and then move back into the metaphor realm.

The roadmap: Context and motivation - Overview of proposal - Main challenges - A tentative solution

First motivation: the idea of non-linear time in music performance.

Music is thought of as something that exists in a finite period of time. We have been thinking how we can challenge this structure, by ways of analysing the technical possibilities that we have, how we can think and change that sense of time.

Understanding music composition as algorithmic composition.

Rather than considering musical structure as the development of musical events over time, Timbre Networks focuses on creating structures where different sounding actors will have (variable) degrees of interdependence with other sounding actors by means of creating “threads” whereby the actions done by one performer could be transformed into control information for the manipulation of the sound sources of a different performer. That is where the notion of networks emerged as a strong, relatable metaphor: thinking of each sounding actor as a “node”, and each processing engine, each way of mapping the effects of one system into the other, as “threads”.

The idea of using improvisers, not only musical improvisers, but dancers, and other artists, had to do with the expectation that this kind of artists have developed strong skills to malleable connect to their own instrument. Not only to be able to adapt to different stimuli from other performers, but also to be able to react, to transform their own relationship to their instrument in real time. This was fundamental in the creation of processes that would be challenging, interesting and dynamic, musically speaking.

The second motivation behind the project was to explore and “explode”, accelerate, force the way the role of the electronic performer in front of an audience; move from a state of emulation (having the electronic performer act as an electronic version of a traditional instrumentalist, what I sometimes describe as the 'one finger percussionist'), where all you have to do is be 'on time' to trigger the 'next' instruction, towards role where the uniqueness of the setup can be explored, and presented. Seeking an analogy in the traditional musical roles, perhaps the conductor is the one that shares similar responsibilities: be able to control, transform, balance and regulate the sounding contributions of others.

The idea of creating a system that is not fully automatised comes from the need to preserve the human performative aspect as a central component of the network. This has the secondary feature: to be able to interact with other improvisers, other traditional musicians, and find a way to give them an environment, where, although called to interact from their strength and expertise as musicians, they would find novel challenges, challenges that they wouldn't find in more traditional improvisatory setups. To make this possible, the incorporation of a telematic component on every Timbre Networks performance is essential: it allows us to test the displacement between physical actions and sonic manifestation, a natural feature of the musical system for electronic performers, but not a common place for traditional instrumentalists.

Our goal was to create an environment for improvisers, something driven by the gathered information of previous performances. In the world of big, messy data, the goal is how to find the right funnel to identify salients, and commonalities. From a sociological perspective, we can consider myths as some sort of data science of experiences: A tribe accumulates generations of survival which accumulates into 'big data' and these survival experiences need to be stored and disseminated in a way that can help the survival of future generations. Myths therefore become the first data science.

So how can we build a summarised version, a “myth” of our Timbre Network for current and future improvisers to interact with? We put improvisers to play with the system, and we seek ways for the system to learn and preserve the good things for the future, so the system can guide future improvisers.

The way the system accumulates the previous experiences and performances

How to implement this “collective memory”, not as much to create a collective guide, but rather, to allow the system to create its own layer of positive 'confusion' towards the performer as a way to increase the challenge for the traditional improvisers that 'enter' this musical space: They would hear instrumental sounds, electronic sounds behaving as controlled by traditional instruments, but actually, what is happening is that each telematic component of every previous performance is stored in the system and used as control information for future performances. Therefore, there are a few 'ghosts of performances past' serving two functions on each iteration: Over time, it has served as a gradual automatisation of the system, taking away some control from the hands of the electronic performer of the system, coming from the “emulation” metaphor (triggering events, activating cues. Controlling the spatialisation of the sources) opening the challenge of redefining the role of the performer of such system into more exploratory, improvisatory, creative tasks.

Architecture of the system

At the centre, there is the “hub” an analysis and scaling engine. It takes both sounding and control signals coming from each source ('node') and scales them into control rate and/or MIDI range and sends them back into the system. These are the fixed connections. There are also dynamic connections: a set of counters and scalers that allow the electronic operator to control in real time how fast or slow all the semi-automatic processes run, and trigger new mappings during the performance: in this way, a particular gesture of the external controller might change mid-gesture both in scale or in actual parameter being acted upon, and the role of the electronic operator is to identify the nature of the change and react musically to the new conditions of the system. There is a collection of physical movements in the performance that don't make much linear sense, but are essential to keep the engines active, and arguably, contribute another layer of visual/auditory confusion, challenging and interesting, to the performative event.

Performance: AoH_T&K, by Juan Parra Cancino. Toma Gouband, percussion, Kana Nakamura, Networked dance, Juan Parra Cancino, programming and live electronics. Performed during the "Open Circuit" event organised by the "Music, thought and technology" research group at the Orpheus Institute, Ghent, Belgium, on September 3, 2017.

Example: Tokyo Dancer

In this example, here is a telematic dancer from Tokyo (Kana Nakamura). The circle on the right is tracking the movement of the negative between the hands of the dancer (video 1). The tracker reacts when the white point on the background is covered by the hand. This is transformed into three different control signals: one is the circle/position, the second is an x/y coordinate graph connected to 2 sliders controlling dynamics in the patch and the third is a threshold detector: whenever she covers exactly the white point that the image is calibrated on it will trigger a single event (video 2). What is interesting to me mapping-wise is that the single trigger is what has the most dramatic effect in the whole system, since that trigger will change a whole preset sequence for a boolean pattern generator, which is in turn controlling how many synthesis engines are functioning.

The system is waiting for one of those triggers to select randomly between three stored pattern maps (video 3).

00:00 / 00:00

Challenges

Building on the ideas, implementations and iterations that we have been realising, we can now identify the following challenges:

We want to continue developing this system for improvisers to play with, keeping a veil of 'confusion' in the mapping between incoming material and what the system gives back. So how will this be perceived from the audience perspective? For the performers, the challenge might be interesting: something responsive, yet hard to 'control', but for the listeners/audience chances are that they will only perceive chaos. So how to deal with this? How to balance the (traditional) expectations of an audience towards the 'understanding' of a performance event?

Questions for the future

What would be an “interesting” version?

One where every actor is able to react to the sounding elements, to slowly recognise which elements of the network react to his/her playing and in turn explore different approaches to contribute/interact with the network, seeking different outcomes. (playing as in game, exploring). Playing “with”, rather than playing “it”.

What would it be that “it works properly”?

Where the “static elements” of the network (the data streams brought from previous performances, become sufficient to generate an entire performance that sustains in time without “entropy”... Or at least with a balance point that changes dynamically from iteration to iteration.

How can we evaluate that we are creating an environment where improvisers challenge the way they interact with each others (and themselves)? How can we evaluate that we are creating an environment where each iteration preserves a level of consistency that allows for different performances to be perceived as related to each other?

The telematic nature of (parts of) the network provide not only a way of emphasising the dissociation between action and sonic consequence of (elements of) the network, but It provides the opportunity to focus on certain aspects of the relationship between the network and the telematic node. This could be a way to test the global behaviour of the network, in different iterations, in different ways.

What can we learn from science? In particular, from Network Science and Complexity theory, in order to guide us in the construction of these systems?

Our system is composed of feedback loops and pseudo-evolutionary processes. Therefore, there is a lot of knowledge that can be applied from Complexity and Network theory to test and help develop current and future iterations. In turn these disciplines are young, and not 'finished', so the challenge presents itself: How can we provide insights from the artistic practice back to science to help develop future knowledge?

The last challenge is how can we evaluate the evolution of the system?

We have this system, that with each iteration 'gains' experience, it merges and mutates, therefore, something is moving. But how can we make sure that is moving consistently towards some direction?

And is it the direction that we want?

We want the notion of 'non-linear' time in musical structure to be preserved. We conceptualise the performances then as actuations, 'moments' of an ongoing piece, that emerges periodically in performances. So how can we understand the notion of time inside each performance in connection with the times in each previous performance and with the notion of time of the concept of the system/work as a continuous musical entity? Is there a way to address that interesting friction in a quantitative way?

A possible answer to this last challenge: to test the system without any new input: using the iterations we have so far and focusing on the dynamic mapping of these pre-recorded iterations, in order to learn more about what are the musical salients of each one of the separated recorded 'ghosts', see whether we can compare them, classify them and determine for example, that continuous pitch material is a better control signal for reverb density in the network. What we are working on now is to create a number of these 'test performances', having the electronic operator perform the dynamic mapping of these no-input systems. Since this is still a musical event, something needs to produce sound. Therefore, the operator brings from time to time some of the pre-recorded 'ghosts' back to the audible rate, enough to feed the system and have it running.

How to interact with the data produced:

One way would be to use solo (no-input) iterations, where the function of the electronic performer is to shift the mapping of the (pre-stored) iterations from sounding to controlling. This will serve the purpose of testing in performance how different parts of the network react to the same data stream in different ways, helping clarify, for example, the way certain sounding streams (percussive, continuous, pitch oriented, noisy) affect different parts of the network over time, and use that information to calibrate, re-map dynamically the network in iterations where live performers focus their contribution in one or another kind of sound source.

Solo Performance at Be-Part art Gallery, Waregem, BE. Performed on April 27, 2019

References

[1] S.A. Kauffman. The origins of order: Self-organization and selection in evolution. OUP USA, 1993.

[2] S. Jordà. Digital Lutherie: Crafting musical computers for new musics' performance and improvisation. PhD. Thesis. Pompeu Fabra University, 2005.

[3] F. Rosas,, P. Mediano, M. Ugarte, and H. Jensen. "An information-theoretic approach to self-organisation: Emergence of complex interdependencies in coupled dynamical systems." Entropy 20, no. 10 (2018): 793.