Muta: material-driven sound synthesis in a mutable feedback instrument

Name: Orpheus Instituut
Price range: $

Article by Mia Windsor

Share this article

This article discusses my current practice of making and improvising with Muta, my mutable feedback instrument that resonates materials, spectrally processes their signals, and feeds these signals back into the materials. The first section will contextualise Muta’s design in relation to my approach towards improvising in quiet/no-PA environments and how this constraint led to speculative approaches to sound making and synthesis in my practice. The second section will discuss the mutability of Muta and how this playful process of changing materials is akin to Karin Knorr Cetina’s notion of ‘tinkering’, that it gives Muta agency, and that emergence can occur through this. The final section will discuss the configuration of the signal routing and processing, the complex behaviours that emerge from this and how I attune to these sounds and forces as an improviser.
... +

tags: Speculative Synthesis, materiality, Feedback, improvisation

This clip demonstrates a very cohesive set with Muta and encompasses most of the approaches mentioned in this article:

1 Defining Materials

When using the term ‘materials’ in this article, I am referring to physical things like glass, water, popping candy, etc., and not the digital system that routes and processes the signals. Although all these things could be argued to be ‘material’, this distinction is helpful for describing the process of causing these materials to interact, and the emergent properties of the instrument as a whole. This distinction follows Luc Döbereiner’s (2020, 612) approach of recognising entities (bodies, media) as different but non-hierarchical. He argues that this distinction is useful because emergence is caused through the interaction between and coupling of different entities. Döbereiner describes emergence in reference to John Holland (1998, 225) as when behaviours emerge that “cannot be obtained by summing the behaviours of the isolated components” which suggest the importance of the configuration of the components and the differences between them, as this article will further suggest. In my experience of playing Muta, these components, in the way they have been configured, act as a singular entity, or rather, instrument. However, the distinction between the materials and the digital system is helpful in discussion.

Muta is illustrated in the diagram below. The materials are changeable, the digital system remains mostly the same. The lines in the diagram illustrate all the possible signal paths and signal alterations.

2 Aesthetic motivation

The music I listen to and the scene that I am a part of is aligned with quiet, abstract, improvised performances, often without a PA system. Prior to this, I spent a lot of time playing gigs with a setup that involved an input processed by live electronics (spectral resynthesis, granular synthesis etc.). I never felt especially connected to my setup and would always spend more time tweaking and automating my patches than practicing and playing with them. Part of the issue with these setups was that I felt like I was controlling an assemblage of independent processes rather than a cohesive instrument. Removing the PA (a given for most electronic musicians) destabilised my practice allowing me to rethink my approaches towards synthesis and signal processing. The music referred to below comes from various scenes. There is a common thread between these scenes, but it is hard to pinpoint. Michael Pisaro-Liu (2009) echoes this when discussing onkyō and AMM in relation to Wandelweiser:

“While I think it’s fair to say that something is being shared by these various musical streams, I would prefer at the moment not to name what that is (in part because I have no idea what to call it).”

The artists I am especially interested in from these scenes are those who make sound through interacting with objects, instrument preparations, feedback, ‘hacked' instruments, radio interference and various other assemblages. Below is a list of examples:

The Japanese onkyō scene: emerged in the early 2000s in connection with a small core group of improvisers including Sachiko M, Toshimaru Nakamura and Otomo Yoshihide. This scene emerged from the now defunct venue Off Site in Tokyo, a small, detached house in a residential area where performers had to play at an extraordinarily low volume so as to not disturb neighbours, so much so that outside sounds would enter the performance. This caused the performers to develop an aesthetic of playing with the noise of the space. Ito Atsuhiro—who ran Off Site along with his wife Ito Yukari—largely credits the neighbour’s noise complaints for the emergence of this new form of music (Plourde, 2008, 280). Each of these artists also works with experimental approaches to electronics. For example: Sachiko M performs by manipulating the test tones from a sampler to make extremely high-pitched noise, and Toshimaru Nakamura pioneered the ‘no input’ mixer technique. These experimental approaches towards electronics, combined with background noise, create a distinct quiet aesthetic, with this noise flowing equally between the ‘background’ and the performers.
The ‘Non-Music Beijing scene’: affiliated with artists such as Yan Jun, Li Song, Zhu Wenbo, Zhao Cong and Sun Yizhou formed more recently due to a similar situation as in the onkyō scene. In Beijing, artists have turned to unconventional spaces such as people's apartments or public underpasses without electricity—where Sun Yizhou hosts concerts—because access to venues with sound systems is limited. These performances often end up involving everyday objects and activities (Default Den Haag, 2025).
‘No PA’ gigs in the UK: many of these artists from the Beijing scene have ties with the UK DIY experimental scene and have inspired these artists and promoters to work within similar constraints and find new means of making and amplifying sound without a PA. Artists in the UK who are currently doing this include Mengting Zhou, Li Song, Ryoko Akama, Thomas Carroll, John Macedo, Linda Jankowska, Anthony J. Stillabower, and Rory Salter.
Keith Rowe: Rowe’s approach to prepared and modified guitar and live electronics in the free improvisation group AMM is an inspiring example of improvising expressively with noisy electronics alongside acoustic players. His solo work also aligns with the aforementioned minimal aesthetic and feels adjacent to onkyō, especially as he has played a lot with onkyō musicians.
Performances of Wandelweiser scores: many of which involve quiet improvised interactions with objects in unconventional spaces.

Although my own constraints as a maker and improviser are not forced upon me like in the onkyō and ‘non-music Beijing’ scenes, I have found that listening to this music and regularly performing in no-PA/quieter settings has reshaped my relationship with technology. Choosing to fit into the spaces I am in at the volume and spatial positioning of an acoustic player, as these artists do, has led me to rethink my approach towards amplification asking questions such as:

How can I amplify signals from my computer? Now my default approach is removed.

This led me to then ask:

Could I amplify signals through the instruments/materials I am playing?

This led me to speculate about resonating the materials with transducers as Andie Brown (2023) and Scott McLaughlin (2015) have done. I then began to cause feedback through materials which has been done before with drums by Scott McLaughlin and Anthony J. Stillabower (2024), Regan Bowering and a trio of Conal Blake, Regan Bowering and Li Song. The trio also takes a relational approach with multiple drums and microphones/speakers being moved around the room to change the feedback: a similar relationality is found in Muta.

The novel part of Muta then came about through three realisations. Firstly, that I can cause complex relations between multiple materials through precisely routing their signals into themselves, each other, and themselves via each other. Secondly, that I can go further than Brown (2023) and McLaughlin (2015) and approach resonance more dynamically by spectrally resynthesising signals while playing. And finally, that it is possible to combine the timbral characteristics of signals, hence the introduction of convolution and frequency modulation, resulting in more complex relations in the system. While other DSP complexity could be added here, DSP in this project is simply a tool to explore the resonance of and interaction with and between materials in my performance practice. The possibilities of these three novel approaches in this complex system have been vast as this article will illustrate, resulting in a continually developing performance practice.

3 Instrument Agency

When improvising, I like the instrument I play to have a significant amount of agency so that I can have a dialogue with it. I have two approaches towards achieving this: the first is to play with unfamiliar or broken instruments to discover the sounds that they make and the forces they exert while I play. This is a frequent approach in the Beijing scene with artists often playing with unamplified household objects when concerts take place in people’s living rooms; there have also been some creative approaches to this at events such as Random Gear Festival at Cafe OTO where players are assigned random objects/instruments to improvise with. The second approach is to play with instruments that have complex, unpredictable behaviours such as feedback instruments like Toshimaru Nakamura's no-input mixing board or the various instruments created by artists and instrument designers associated with the Feedback Musicianship Network. Both approaches are present in the aforementioned quiet noisemaking scenes. Reflecting on this, I designed Muta to have both qualities. Muta is, as its name suggests, mutable in that I can change the materials connected to it, which allows for a constant process of exploration and discovery as I continually connect new materials. Muta also has agency through the complex behaviour that occurs due to the configuration of the instrument (by feeding signals between the materials, the computer and back into themselves), resulting in further possibilities for emergence.

3.1 Instrument Agency: Mutability

First, focusing on the idea of playing unfamiliar instruments, I wanted to create a mutable instrument that could hypothetically be connected to any materials, so that the materials could shape both the timbre and the behaviour of the system itself. This framework allows the instrument to grow: continually attaching materials to generate new emergent behaviour that I can then respond to in improvisation. The primary way in which the materials are activated is by sending sine tones to transducers placed on the materials (transducers are speakers without cones, which use the materials they are placed on as resonators). To connect a new material, I have to find out its resonant frequencies. I do this by sending in an ascending sine tone, listening for the frequencies it resonates loudest at, and then list these frequencies in a text file for the patch to read from.

My choice of materials is informed by the aesthetic I am aiming for in performance. The timbral palettes of the videos embedded in this article differ greatly with some being more ambient and pitch centric and others being more noisy and abrasive. The different materials relate to one another differently too. I continually try new materials with Muta to explore its possibilities. Carin Knorr Cetina (1979) argues that scientific knowledge is formed through interacting “successfully” with material reality and describes scientific laboratory work as a ‘tinkering’ process. This idea of tinkering could be applied to any practice, including making and performing with Muta. Ben Spatz (2017) suggests that tinkering is a “process of combining and recombining bits and pieces almost at random in order to see what works,” they also allude to the more destructive nature of tinkering stating that “it is often not clear when tinkering whether the situation is getting better or worse. And even when there is clear improvement, one may wonder if the situation could be made better still.” The idea of tinkering has a playfulness and an air of transgression to it, like throwing ideas at the wall to see what sticks but being able to try literally anything—the process of trying new materials with Muta is a speculative tinkering process. In addition to discovering the behaviours of the two main materials, I place various preparations on them such as tin foil, chopsticks, skewers, springs etc. and if the material is also a vessel (such as a glass or a bowl) I fill it with water or jelly and place a hydrophone in the liquid and pick up the signal via the liquid. When picking up the signal via the liquid I can also affect the signal by putting popping candy or fizzing mixture (citric acid mixed with bicarbonate of soda) in the liquid, or by blowing bubbles into it. The materials also differ in how they can be activated; when playing with a spring I like to hit it with a spoon which causes a resonance with a long decay, or wobble it around, which creates a continually oscillating signal.

The viscosity of jelly allows for more tactile control of feedback through the bowl/jelly: pushing my finger down on, or into the jelly in certain places causes the pitch of the feedback to change. I can alternate pitches by alternating between these pressure points.

In this video, a few pitches emerge from this alternation of finger position and pressure on the jelly. I resynthesise the lowest feedback pitch that comes through and continue to develop from there while still playing with the jelly.

The process of trying different materials in performance and seeing whether they work is a tinkering process; I try new materials and preparations without knowing how they will sound and behave in relation to one another and the system as a whole. Throughout the videos, one can see lots of ‘aha’ moments where Muta does something interesting that I am not expecting. There are also moments where I try something that doesn’t work, and I have to find a way out of, which provide an interesting improvisational challenge. This constant changing of materials gives Muta a kind of agency that keeps me on my toes while improvising.

3.2 Instrument Agency: System Agency

In addition to the agency provided by changing materials, I also wanted there to be possibilities for emergence within Muta’s set-up and routing. I designed Muta as a complex network, where signals from contact microphones placed on the materials (microphones that sense audio vibrations through contact with solid objects) could be sent back into themselves and to each other via the transducers, with the option to spectrally resynthesise, and combine/modulate signals in between. The priority was to avoid overcomplicating the DSP techniques: first, because this project is intentionally a performance practice rather than a coding project, second, because a balance needs to be struck so that the signals can be altered while not making them so complex that they no longer resonate the materials, thus letting the resonant characteristics of the materials shine. I control the system with a MIDI controller and use one slider for each of the signals (direct, resynthesised, and combined) which can be sent into either of the materials, affecting relations in the system. The system is self-contained, so the user has precise control over what is affecting what. The system can sustain itself and complex patterns can emerge simply through turning up a slider or two, yet there is still scope for direct expressive control with most gestures having an immediate effect. This strikes the perfect balance for me as a performer as it allows me to play with an instrument that is complex and evolving and goes beyond the capabilities of an acoustic instrument, while still remaining tangible and connected as a single cohesive instrument. When playing Muta, I attune to it by listening for new sounds, feeling for new forces and interacting with them, having a continuous dialogue with the instrument. I am using the word attuning here in reference to the philosopher of science Andrew Pickering’s (1995) use of the term, which suggests a dialogue between human agency and the materials they are working with. Though the idea of ‘tuning’ is common in music, Pickering’s idea is not constrained to musical concepts of ‘pitch’, ‘melody’ etc., but rather responding to any behaviour, in this case, sonic and vibrational behaviour from the materials. He likens the idea to tuning a radio or a car engine in such a way that one doesn’t know what the initial state will be. From that initial state you tweak to find a desired state, which is a speculative process. Once I have tinkered with the materials to discover their capabilities, I explore these capabilities further by attuning to them.

3.2.1 Feedback

The basis of Muta’s complexity lies in its nature as a feedback instrument. Muta is a positive feedback system: if the output increases, the input increases exponentially and vice versa, thus causing unstable behaviours. Muta is built to harness this instability as a tool for emergence. Simply sending a signal from one material into itself can have complex results:

The start of this clip (before the cymbal comes in) illustrates how direct feedback through the spring can change drastically depending on how it is triggered. A clean, high pitch comes through in the spring feedback initially. However, briefly exciting it with a low resonant frequency causes a multiphonic to come through when it feeds back. This happens because the harmonics from the spring feedback are competing with the residual resonance from the low frequency, resulting in movement between them that is so fast, it is perceived as multiple pitches sounding at once. I attune to this sound by bringing in convolution to intensify the noisiness, turning that down and then repeating the action a few moments later to try and hear the multiphonic again. However, this time the action is repeated while the spring is feeding back at a high pitch, rather than silent. The result is different: there is no multiphonic and instead a slower pulsing alternation between the same pitches that made up the multiphonic.

This clip shows the fizzing mixture placed in the glass causing low pitched feedback that steps up through various harmonics. Using the fizzing mixture is the only way I have found for lower pitches to be activated in the glass, thus demonstrating that the fizzing mixture fundamentally alters the resonance of the system. I attune to this resonance by quickly removing the preparation placed on the glass so the harmonics resonate clearly. I then let the feedback run until it stops.

Another example of emergent alteration of the system can be heard here, when extremely high feedback pitches come through only when metallic objects (a spoon and tin foil) come in contact with the glass. Referring back to Holland (1998, 225), these are clear examples of behaviour that ‘cannot be obtained by summing the behaviours of the isolated components’. This demonstrates that speculatively tinkering with materials in combination with feedback can reveal new sonic possibilities to attune to.

3.2.2 Spectral Resynthesis

In addition to direct feedback, I have also implemented spectral resynthesis of the signals. When a button is pressed, the signal is analysed in relation to how it is distributed within the frequency bands at a given moment. It is then resynthesised as a bank of sine tones that correlate to the seven partials with the highest amplitudes (in order of amplitude) with the corresponding frequencies. I then can adjust these sine tones accordingly. Although this process is reductive in that it only captures a snippet of the signal; the frequencies captured often re-resonate the materials or interrupt the resonance in emergent ways. These resynthesised signals act as a resonant addition to the instrument, rather than an alteration or obscuration of the existing signal. This approach is much more dynamic than just exciting the materials with sine tones because it depends on the present behaviour of the materials, which is variable.

Resynthesising the signal often results in the frequencies that the materials are resonating at and their harmonics coming through, which can be amplified or attenuated. This is demonstrated clearly in this video:

Often, the resynthesised pitches are a few hertz off the initial resonating frequency thus causing interesting phase patterns and beatings which have ended up forming a very significant part of Muta’s sound (as can be heard throughout the recordings). Beating also happens between the direct feedback and sine tones.

This video demonstrates an interesting exercise of iteratively resynthesising the resynthesised tones. The tones get higher throughout until they’re almost out of the range of human hearing. This demonstrates another potential beyond direct feedback in Muta’s recursive possibilities; this dynamic resynthesis approach can fundamentally shift the behaviour of the system as a whole.

Another unexpected phenomenon emerging in resynthesis is the resonant frequency of the hydrophone being picked up. This suggests that the materiality of the technology itself has a defining influence on the system. My hydrophone happens to resonate at a frequency of 50hz; this frequency can be heard being used to rattle the metallic preparations coupled with the glass in this clip:

When I initially designed Muta, I had not anticipated the phase patterns occurring or the hum of the hydrophone being picked up in resynthesis, yet both these behaviours have become fundamental parts of the system that I now attune to in performance. Although I have not been referring to this system as material, there is a clear materiality to these behaviours: they have agency and define the character of Muta.

3.2.3 Routing and Combining Signals

In addition to signals being sent into the materials that emit them (feedback), when playing Muta, it is also possible to send signals from one material to another. Here is an example of triggering feedback through the bowl by hitting the spring:

The relations can become more complex than this when signals end up feeding back via the other material. In addition to this, I wanted to explore ways in which the signals from the materials could be combined, to increase the complexity of possible relations within the system. I have done this in two ways. The first method is through convolution: multiplying the frequencies of a signal with every frame of a captured recording (the impulse response) such that the frequencies shared are accentuated and those that are not, attenuated. This is typically used in convolution reverbs where a signal is multiplied with impulse response of a space. However, upon discovering Oliver Thurley’s Percussion Studies (2021) album, I learned that convolution may combine any two signals, not only using impulse responses of spaces. This results in drawn out spacey sounding textures with the accentuated frequencies clearly dominating. With Muta, transients such as striking the materials or adding popping candy act as impulses triggering the response. Øyvind Brandtsegg’s Liveconvolver allows constant recording and updating of the impulse response so that the user can continuously feed two signals into it (with some latency in the output).

This clip shows feedback and convolution happening between both glasses. The feedback triggers the slightly delayed convolved signal which in turn triggers the feedback, resulting in a self-regulating pattern. I listen and tweak the feedback to bring out different pitches while being careful controlling its loudness.

The other method I have utilised for digitally combining signals is frequency modulation: modulating the frequencies of the sine tones being sent into one material (the carrier) by the frequencies of the sine tones being sent into the other material (the modulator) creating more complex, yet still related waveforms.

At 2:46 onwards, the sound of the spring is strongly modulated thus creating an incredibly noisy timbre. I like to describe this performance as my ‘harsh noise’ set. It is so intense that the transducer starts cutting out.

This clip demonstrates the relation between modulation and convolution in Muta. Causing one material to be modulated by another means that more frequencies are shared by the materials. This then causes the convolved signal to intensify to a point where it heavily distorts.

4 Conclusion

One can see and hear throughout this clip from the start of this article (which encompasses most of the aforementioned approaches) how playful and intuitive performances with Muta are. In this performance, I lean into certain emerging timbres and patterns and gradually change them. Textures range from slow build ups to very intense moments, to residual convolved signals accompanying gentle bubbles, to pure sparse feedback.

Muta developed through speculation beyond conventional means of amplifying synthesised signals. Through this, I have created an instrument that uses its fluid pathways between material and machine to develop new forms of emergence. Muta has become a vehicle for a rich improvisation practice that continues to grow as I connect new materials to the instrument; tinkering with materials, being surprised, and then carefully listening and attuning to the sounds and behaviour of the materials and the system that connects them. There is a further wealth of possibilities within Muta and I intend to keep connecting materials in order to see how they drive the system.

Below is a Bandcamp Archive of all of the recordings I have done so far with Muta. There is also a YouTube playlist available of these recordings.

Bibliography

Brown, Andie. 2023. Exploring approaches to the development of a contemporary artistic practice for glass and electronics. Ph.D. thesis, University of Huddersfield. Available from: https://pure.hud.ac.uk/en/studentTheses/exploring-approaches-to-the-development-of-a-contemporary-artisti

Cetina, Karin Knorr [Karin D. Knorr], 1979. Tinkering Toward Success: Prelude to a Theory of Scientific Practice. Theory and Society. 8.3. 347–76.

Default Den Haag. 2025. focus: default#20 - 23.01.2025 + default#21 - 24.01.2025, with Sun Yizhou, Zhu Wenbo, Zhao Cong, Li Song, Able Noise. [Online]. [Accessed 8 February 2025] Available from: https://defaultdenhaag.substack.com/p/focus-default20and21

Döbereiner, Luc. 2020. Materiality. Contingency and Emergence of Compositional Material, Contemporary Music Review. 39. 5. 602-617. [Accessed 30 January 2025]. Available from: https://doi.org/10.1080/07494467.2020.1852803

Holland, John H. 1998. Emergence: From Chaos to Order. Cambridge, MA: Perseus Books.

McLaughlin, Scott. 2015. Resonant Systems: multiphonic resonance complexities in sine-wave excited cymbal clusters. [Accessed 11^th August 2024] Available from: https://lutins.co.uk/ResonantS...

Pickering, Andrew. 1995. The Mangle of Practice: Time, Agency, and Science. Chicago: University of Chicago Press.

Plourde, Lorraine. 2008. Disciplined Listening in Tokyo: Onkyō and Non-Intentional Sounds. Ethnomusicology. 52.2. 270-295.

Spatz, Ben. 2017. Embodiment as First Affordance: Tinkering, Tuning, Tracking. Performance Philosophy. 2.2. 257-271. [Accessed 8 February 2025] Available from: https://eprints.hud.ac.uk/id/eprint/31101/1/Spatz_FirstAffordance.pdf

Discography

AMM. 1983. Generative Themes. https://www.youtube.com/watch?v=fdQ1hb8uSO4&t=2069s

Anthony J. Stillabower and Scott McLaughlin. 2024. temporary resistance, reflected by itself in the turbulence. leaf///wave sound. https://leafwavesound.bandcamp.com/album/temporary-resistance-reflected-by-itself-in-the-turbulence

Conal Blake. Regan Bowering and Li Song. 2023. Two Movements. Feedback Moves. https://feedbackmoves.bandcamp.com/album/two-movements

John Macedo. 2014. John Macedo – The Intimate Sound System. YouTube. https://www.youtube.com/watch?v=pZTg3b-owyc&t=92s

Keith Rowe. 2016. The Room Extended. Erstwhile Records. https://erstwhilerecords.bandcamp.com/album/the-room-extended

Li Song. 2024. Two Mobile Phones and Snare Drum. https://lisong.bandcamp.com/album/two-mobile-phones-and-snare-drum

Linda Jankowska, Anthony J. Stillabower and Jo Christman. 2023. Jitterbug – Annea Lockwood. YouTube. https://www.youtube.com/watch?v=RnfIAEtws_g&t=1005s

Oliver Thurley. 2021. Percussion Studies. LINE. https://lineimprint.bandcamp.com/album/percussion-studies

Pressure Cooker Relief Valve. 2023. Music from New Moon. https://pressurecookerreliefvalve.bandcamp.com/album/music-from-new-moon

Regan Bowering. 2023. Solos for_ _ _ _ spaces. Bezirk Tapes. https://bezirk.bandcamp.com/album/solos-for-spaces

Rory Salter. 2024. On the Floor, by the Door. https://rorysalter.bandcamp.com/album/on-the-floor-by-the-door

Ryoko Akama. 2018. dear martin. verz. https://verzimprint.bandcamp.com/album/dear-martin

Sachiko M, Toshimaru Nakamura and Otomo Yoshihide. 2004. Good Morning Good Night. Erstwhile Records. https://erstwhilerecords.bandcamp.com/album/good-morning-good-night

Sun Yizhou. 2024. typewriter in the rain. Aloe Records. https://aloerecords.bandcamp.com/album/typewriter-in-the-rain

Mengting Zhou and Li Song. 2023. two rooms. presses précaires. https://pressesprecaires.bandcamp.com/album/two-rooms

Yan Jun. 2018. yan jun – island show pt.1. YouTube. https://www.youtube.com/watch?v=-WvNm1TrSuI&t=1s

Yan Jun. 2021. yan jun – living room tour 2021 beijing. YouTube. https://www.youtube.com/watch?v=hHexwrE_CoE

Zhao Cong. 2025. Zhao Cong - 18/01/2025 - Wharf Chambers.YouTube. https://www.youtube.com/watch?v=B4gLv-aRbLI&t=1309s

Zhu Wenbo. 2022. Four Lines and Improvisation. Aloe Records. https://aloerecords.bandcamp.com/album/four-lines-and-improvisation