Your browser does not meet the minimum requirements to view this website. The browsers listed below are compatible. If you do not have any of these browsers, click on the icon to download the desired browser.
We have all means to convey sounds in trunks and pipes in strange lines and distances.
There's this COVID thing. Perhaps you've heard of it? Wouldn't it be great if there was a dead-simple Jacktrip installer for Mac and Windows that would let all us suddenly-shut-in musicians gather their friends together for a friendly jam-session? Perhaps some developers are also suddenly-shut-in and would like to take a run at this?
This will be an account of JackTrip during a particularly meaningful episode in its evolution, namely the first year of the COVID-19 pandemic and the months leading up to it. Music making and music changed during this period, at least mine did, and I would vouch that the lessons learned (and the music lessons missed) reach deeply into musicking across the globe. The pandemic and the many social, environmental and political changes and challenges disrupted any sense of business-as-usual for almost all of 2020. I was abroad at the start of the new year in Europe for JackTrip-related concerts, teaching and research. From it's beginning in the early 2000s, JackTrip has been a technology that provides high-quality, low-latency audio connections between multiple sites over the Internet. I was off to work with groups in Germany and Italy improving it and teaching about it.
The Orpheus Institute (Ghent, Belgium) had invited me for a weekend visit in which we'd engage the topic of musical improvisation. The Institute has a long history of concerts, seminars and publications in the field. To cut right to it, my visit resulted in a CD recording that begins and ends this account. The Time Crystal album is being released today (5-Feb-2021) and this memoir I'm sharing accompanies the occasion. Coincidentally, this date is also the anniversary of my concert at Silent Green in Berlin (5-Feb-2020), an event that followed the visit to Ghent by a few weeks. The Ghent visit wasn't JackTrip related but that concert was. Neither was Ghent intended as a recording date. And neither was the music from the Ghent session foreseen as a start of a year-long JackTrip-based panglobal ensemble with weekly concerts, but it was. These weekly sessions have changed both JackTrip and my own music. At the time of the visit one year ago, that which was to transpire, a global pandemic and a year-long shutdown of everything, was still a couple months away.
As is already evident, precise dates are a feature of this chronicle. I began it exactly one year from the visit to Orpheus (which after the fact became our Time Crystal recording session). I have finished it on this anniversary date of the Silent Green concert. And doubly-ironically, I began this writing on battery power, at home without power and still locked down. Much of Northern California was in the same way and days like this are tokens of all the broader changes. Such times have never been a part of my life or my parents' lives. My Mother (91 years old) only recalls measures that were being taken "that time back with the influenza" in 1957-58.
At present, the continuing dismal situation makes for a growing practice of getting together online to make music through microphones and headphones. We gather sonically from our isolated remote places and enter a closeness which defies the mandated physical separation. Video conferencing technology has been getting an incredible workout. Better choices for music are JackTrip and other music-specific applications which improve the lag situation and sound quality. Because of the difficulty with video lag, ensembles using a hybrid video / JackTrip approach can see each other but gestures are delayed relative to the sound arrival. Despite this, will the convenience of playing together in this way becomes a substitute for always gathering physically once we can as existed previously? I'm so looking forward to when it is possible to get back together and satisfy that strong urge.
Back to my time in Europe. In January, 2020 I was working on several JackTrip projects. At the Technical University of Berlin, I was leading a class and co-leading an ensemble. At the Politecnico di Torino, I was embedded in the EE Department and teaching a PhD course. At both schools, technical enhancements to JackTrip were underway with teams engaged in solutions to well-known problems. By mid-February these were all slated to wrap up and I would shift to Canada to complete and premiere the "Earth Symphony" for the 50th anniversary of Earth Day. My wife June and I were making our last bi-weekly trip between Germany and Italy, and when we passed through Milan airport we underwent a temperature screening. It wasn't something we knew much about but then there it was again as we departed from Turin on our way back to the States. Voom. Epidemic and no further trips. ("Voom" = sonification of a viral explosion)
My Winter teaching and concerts in Europe wrapped up nicely and our teams agreed to continue working on several projects together. Links to those wonderful outcomes follow. The day before returning home to the States, I was invited along with my colleagues and hosts at the Politecnico to the Rector's Office for what I thought would be a quick introduction and goodbye since I'd only been indirectly in touch with him for administrative formalities. But it turned out he had a vision of inviting our participation in the Fall Biennale di Technologia because the theme that was to be remote collaboration. Prescient? No kidding.
Canada didn't happen. Neither did the Earth Day 2020 celebrations in Vancouver nor the symphony. That Earth Day will be remembered as the one that was ruptured by planetary forces. The pandemic was raging by March. Quarantines were established inside and between countries across the world. One of the most brilliant immediate adaptations to lockdown was musical when Milanese started singing together from their balconies. Prof. Cristina Rottondi, a singer and expert in network music performance (and my colleague at the Politecnico di Torino), suggested recording Amazing Grace together between her place in Milan and my home in California. Global music making with JackTrip was well-established and we knew what we were doing. But with the pandemic doing it from home was a new experiment.
Amazing Grace 24-Mar-2020
Cristina and I recorded Amazing Grace and she incorporated it into a press release "Suonare insieme, dalle due sponde dell’Oceano." We used the same hub server code in JackTrip as for earlier concerts over the Winter. She was on win10 and I was on linux. We connected to a server at Stanford.
Quarantine Session #1 28-Mar-2020
Locked at home in Europe at our homes in the Bay Area, more experiments ensued. The setup was again a mix of laptops and the server was at Stanford. Juan Parra of Orpheus Institute along with my recent host at the Techincal University, Berlin, Henrik von Coler, plus another member of the Elektronisches Orchester Charlottenburg, Klaus Scheurmann, made a concert together with three of us who work at CCRMA: Constantin Basica, Fernando Lopez-Lezcano and myself.
Quarantine Session #2 4-Apr-2020... #38 7-Feb-2021 (and more to come)
The previous week's concert came out well. Repeat! And repeat some more. Following sessions have included many guests.
Internet Ensemble Tech Force 6-Apr-2020
Classes everywhere were disrupted before completion of the Winter term and students had largely returned home. The coming Spring term at Stanford was going to be completely online so I made a pitch to start a JackTrip class devoted to helping musicians connect from home. The wild card would be internet service but we knew from the recent sessions it might be possible. Having moved around a lot in Europe recently I'd had a chance to compare networks and I'd been astonished by what was possible when I tried JackTrip from in different cities. Back home, I found I could now connect with several channels on my moderate tier, nothing special, service plan that I hadn't changed in 7 years, remarkably better than years past. This was getting good.
The semi-weekly class, 153b IETF, became a crucible for testing and improvements. Initially conceived in order to muster and train technical help for Stanford's ensembles, it morphed into something that changed JackTrip itself, and grew a core set of enthusiasts including those ensemble members who I thought we would simply support but who gained their own understanding quickly and started to experiment. Some participants joined from across the country and abroad but most were spread across the Bay Area.
The technical ingredients were initially:
homes with commodity internet service of all kinds
a mix of laptops with various audio interfaces and sound gear
servers at CCRMA
JackTrip v1.1 and testing of a v1.2 release candidate
Zoom and Jitsi video conferencing
Open Broadcast Studio (OBS) for broadcast via web streaming services
During April, the jacktrip_users mailing list was starting to awaken. It was a list that had been extremely quiet for years. JackTrip was a "mature" codebase and mostly serving university-based music projects and the list saw very few inquiries or comments. Also, the JackTrip Github project was showing signs of heating up.
In January, Max Wagenbach, for his project in my Berlin seminar had painstakingly re-integrated my CCRMA-hosted JackTrip version back into the Github project and made tweaks which improved the project's accessibility. The reason for my separate repository had been experimentation with a radically diverging feature called Wide Area Internet Reverb. It involved a lot of code changes from v1.1 and I didn't want to disturb the "reference version." It was working well, serving its community, and maybe I'd break something. Git is complex and I simply didn't understand it well enough to know how to work side-by-side in the same repo on Github. The decision to work elsewhere was partly nervousness and partly the fact that Github had recently been acquired by Microsoft. Who could say if its open source principles would be preserved. The separate repo that was hosted on a community edition of GitLab at CCRMA also supported the ensemble work we were doing at TU. My blunder was having copied the project rather than forking it.
Max's work was timely in hindsight because it barely preceded the uptick in interest coming with the pandemic and Max deserves huge thanks for his dedication and insights. David Runge, also in Berlin and the JackTrip package maintainer for Arch Linux, pointed out updates needed for modern hardware and together they educated me on how a bonafide community project should be structured. Their cautionary tale was that isolated versions would probably lead to a crazy-making split (like thejack1 and jack2 versions we still live with).
The isolated copy had gained one particularly important feature which then became key to ensemble work worldwide during the pandemic. I consider it as an example of when musical experimentation creates a feature that then takes on a life of its own. In this case, it was largely a product of the Raspberry Pi music ensemble work which Henrik von Coler, Nils Tonnätt and others were doing with me at TU. Henrik's vision when we discussed it prior to my arrival was a group of 10 participants performing together in an ambisonic space that enclosed the audience. No need for a mixer and a big tangle of wires like the Elektronisches Orchester Charlottenburg (EOC) had been touring with. Our seminar used Raspberry Pi's (built by the students) and ethernet audio connected to a central sound server computer. Each musician directed sound source location in the ambisonic space renderer (via Pd-based OSC controls). Each musician could fly around the 3-D space. Fittingly, Henrik named the concert "Hub Mode" and our ensemble performed with it at Silent Green on 5-Feb-2020. The concert included my piece "Remote Control" performed by the EOC using the same system. The new JackTrip feature automatically maintains the mix of clients on the hub server dynamically, repatching it as they come and go. JackTrip's "hubPatch" option offers a number of configurations, loopback, mix, etc. More are being added continuously.
By May time frame, IETF meetings were the forum for sharing discoveries, testing, ease-of-use improvements, and attracting a few entrepreneurs. A key component in JackTrip's success with home connections is hub server mode. Most home setups involve a wifi router which allow multiple computers to connect and block unwanted traffic into the home's local area subnets. A wired ethernet cable connection to the router is essential and most have physical ports available that support this. A JackTrip computer running on the home's "wired" subnet initiates a connection to an ensemble's hub server and it works without needing a public address for the JackTrip device or manually- opened UDP network ports. Such is not the case with JackTrip's peer-to-peer mode which is why it remains a specialist's option. Ensemble ease-of-operation also extends to the audio connections which, as mentioned above, are automatically patched by the hub server as clients come and go. A practice which has evolved for pandemically-distanced groups derives from the panglobal ensembles and the TU Raspberry Pi ensemble. A hub server "room" is provided for the ensemble either physically or in the cloud and that's where the group gathers.
Group playing in IETF quickly revealed a hub server misfeature in which I'd hardwired the FPP setting to a very conservative value. Up until then, and for our concerts in Berlin, FPP was stuck at 1024. Local IETF jam sessions and a Kronos Quartet concert needed lower values and happily proved that really good rhythmic sync was possible from home, right down to ca. 20 ms one-way across the Bay Area (with an FPP setting of 32).
The Kronos concert in July was a milestone. An audio engineer in Berkeley ran a hub server at Stanford and the quartet connected from their locations across San Francisco. Mixing, reverb and effects were added and the result was live-streamed to the audience. It was a dusk concert for N. American viewers and a dawn concert in Slovakia where the concert was hosted as part of the Pohoda in the Air Festival.
Another Summer milestone was the Bay Area Ragazzi Boys Choir coming together online for rehearsal. The entire choir of 80 began singing together. A hub server in the cloud and Raspberry Pi devices at each of their homes has become their platform as the result of Mike Dickey's engineering contributions.
I remember 5 people independently mentioning the idea of forming a foundation. By September we did that. The JackTrip Foundation came into being as a way to coalesce broadly expanding capabilities and power up efforts to help musicians still under the shadow of the pandemic. As Summer moved into Fall, pandemic numbers were being influenced by the change of seasons and late Fall became increasingly difficult. I initiated a new Stanford course, "Network Performance Practice" and students attended by from California, Michigan and Texas.
The Winter holiday season was particularly dire. Stanford's hopes to combine some in-person teaching with remote instruction fell apart. As I write this in February, we don't know our plans for the next term. And as virus mutations proliferate we hear talk of a second epidemic and forecasts of a much longer horizon of return to normalcy. The Foundation feels like absolutely the right thing at this point. Through it, there's significant potential to accelerate availability of the technology. The open-source nature and Foundation's volunteerism will remain central and its commercial side will provide scalable service availability and sources for inexpensive devices.
There's magic on the horizon. We're creatures who in our physical realms listen and function with inherent delays. We are adapted to live in lag, so to speak. Our sense of place and presence make profound use of time delays as sounds bounce around and sounds allow us to synchronize with each other. The instruments we make for music derive their pitch from recirculating waves. We know dolphins and bats are exquisitely capable of using echoes for orientation and we are the same. I like to think Internet acoustics is the new realm into which we're expanding. Pre-COVID, that was more on the level of a thought experiment and now we're accelerating towards it.
Better understanding of micro-scale temporal phenomena is developing through JackTrip practices and interesting developments are expected. For example, packet loss concealment research to cover gap sfrom transmission dropouts is proving that we already have technologies which can do decent "ahead of real-time" prediction. "Pretention" (in phenomenlogy-speak) could be almost science fiction-like. But it's actually not, our brains do it, and it's a part of what philosophers and psychologists would call time consciousness. Add in that simultaneity across larger network distances can be achieved through prediction -- a possibility which is already under investigation -- and here we go... I dream of Time Crystals of this kind. Musicians enclosed in soundscapes that can be transmitted. Sound objects that are made of network delays. Hold that thought in your mind, like a crystal!
Your browser does not meet the minimum requirements to view this website. The browsers listed below are compatible. If you do not have any of these browsers, click on the icon to download the desired browser.
