Q3 2021New Home, New Music and Collection Browsing

Hi everyone, what a quarter this was!

The big "real-life" project which I mentioned briefly in the last newsletter kinda concluded. We're now, finally, living in our new apartment, after 18 months of renovation and construction work, and three years after buying it (in a pretty desolated state). There's still so much to do, but that's how renovations go. At least so I've been told.

It was a long, tiring and frustrating process, but so totally worth it.

Sleep

On the work side, we finished a programmable ink project at Ink&Switch. We're going to present it during the LIVE 2021 workshop at SPLASH, and we're still working on the final essay which should be out later this year.

We also started ramping up to another project in the End-User Programming track; there even was a job posting, which you could see, if you follow me on Twitter.

The rest of this newsletter is about my new album, and an experimental interface for collection browsing.

A bit of backstory first.

Making music was what originally led me to learn programming. When I got interested in making experimental music, Ableton Live was the only thing I had — but everything I was making with it, sounded like, well, Ableton Live. Luckily just around that time Max for Live was added — a way to embed Max/MSP code inside Ableton Live devices. I started re-building various audio effects with my own quirks — a delay with miscalculated time (so it would "drift" a bit), a looper with tons of weird options (and a lot of audio cracks when the loop was readjusted), etc.

Building my own tools was a big part of recording my first album: Out Of Forgetting, released on the Audiobulb label in 2010. Since then, I released an EP, and another full length album, and then nothing happened for a long while.

I played a couple of shows every year, and released single tracks and remixes here and there over the last ten years, but I couldn't finish (or even get to the middle really) any new bigger audio work. I've been collecting material for what feels like forever, and when the pandemic hit last year, and everything slowed down, I also had more brain-space to focus on music.

Gaze Into The Sun: Physical Copy Photo

I spent almost every weekend working on new, and old, tracks. I think the process was also kinda therapeutic for me, or maybe self-pacifying in a sense, I'm not sure.

Once things felt complete, I decided on a whim to share the material with a label that I've been a fan of for a while: Fluid Audio. Dan replied just a day later, and a couple of months after that — here we are — my new full-length album, first one in almost ten years, is available for listening!

I don't think there are any physical copies available anymore, but you can always get the digital version.

Reference material and prior art is an important part of research and design work — and since research and design is what I now spend most of my time on, it's important to me to deal with all the different snippets I keep on collecting. I use two online platforms for storing things for later: Pinterest where my main visual references end up, and Pinboard for links to articles and interesting websites. I also take a lot of screenshots and store them all, permanently, in a folder on my hard drive.

These materials are important for my work, and I was worried that since online services tend to change and disappear over time, I might will lose them sooner or later, so a while ago I built a tool for backing all of this up called Archivist.

One problem with the original UI to Archivist is that there's no way to see everything, all at once. I can either scroll and see a couple of images at the same time, or remember the right phrase to search for — this is the main difference of Browsing vs Searching.

You search for a specific thing, and you browse in an open-ended fashion.

I wanted to browse my Archivist collections.

In general, I'm pretty sceptical about UMAP and other ML-y things. I played around with a couple of online "browsers", but they are often used for artificial data (usually the infamous Iris flower data set), or for collections that I didn't make myself, so I have little emotional attachment to them. But, the ability to zoom out to see the whole collection was very compelling, so I decided to try this on my own data.

Archivist Browser (obviously I'm terrible at naming) is a way to browse Archivist collection using the UMAP embedding combined with a high-performance UI.

You can fluidly zoom through the whole collection:

Since images often occlude each other, there's a fisheye distortion around a mouse cursor, which has been useful enough not to need any more sophisticated ways of dealing with this issue yet:

Finally, you can get to the source of the image you're hovering over:

So far I've been using it to rediscover old materials that I forgot about — UMAP seems to be very good at clustering visually similar items together.

archivist organic forms
archivist weird electronics

The data-processing pipeline is implemented in node.js using TensorFlow.js and umap-js libraries, and the frontend is written in Rust using Nannou framework.

Books I enjoyed recently:

On the web:

I still can't believe I'm writing this from a couch of an apartment that we actually own.

Always happy to hear your thoughts, see you around in December!

Szymon Kaliski © 2021
mailtwitterwebring