Q2 2021Executing Incomplete Programs and Magic Glass

Hi everyone!

Most of my time over the last three months went into Ink&Switch, where we were ramping up on a new project, culminating some of the research I've been involved in for the past year. We're also finishing (hopefully!) a big "real-life" project, so there wasn't that much time for side-explorations, but I managed to squeeze in two smaller things that might be interesting to some of you.

At Freezeframe we sent out first alpha release to friends & family, it's exciting to get feedback from other people using our small workspace refrigerator. More info coming soon!

The consensus in the "serious programming" crowd seems to be that working with strict types and strict compilers is a good thing, and as a serious developer, you should learn to read the error messages, and work with the compiler to write your program; and then, once it compiles successfully, it works correctly.

I never learned to be patient, and I much prefer a program that does something, even if it does that thing incorrectly. I'm not alone with this, as a friend pointed out:

A running program, even if not correct, feels closer to working than a program that doesn't run at all

Running incomplete programs with type checking is still an open research topic, one of the most famous examples being Hazel, which not only allows you to run unfinished programs, but can also (partially) type check them! It's definitely worth looking into, but: one — it's a research project, and two — a totally new universe to adopt (which you probably shouldn't do right now, because of the first point).

I was wondering if there's a small step I could take in that direction, with an already existing language. JavaScript maybe isn't an obvious answer here, but it has two rarely used features (some would argue, anti-patterns), that allowed me to hack a quick experiment together:

  1. with — which extends the scope with the provided object, for example:
    with (Math) {
      // notice that we don't have to use Math.sin or Math.PI
      const x = sin(2 * PI);
    }
  2. Proxy — which allows you to intercept and overwrite fundamental operations on a given object, for example:
    const obj = new Proxy({}, {
      get(_, property) {
        return `you requested: ${property}`;
      }
    });
    
    console.log(obj.hello);
    // => you requested: hello
    console.log(obj.world);
    // => you requested: world

You can combine these two, in a way where every get or apply returns the same Proxy object, creating infinite chains that never fail. Then, you can put that object in with (...), so any time you call for something that doesn't exist, you will get that Proxy object, which will evaluate to itself. This solution obviously doesn't type check in any way, but it's a fun experience to work on a program that (almost) always runs, even if you didn't finish writing it, and it's interesting to see how powerful vanilla JavaScript can be.

Below is a quick demo of building a small React component with immediate feedback and no _ is not defined errors popping up randomly:

The second experiment was inspired by research on Toolglass and Magic Lenses by Bill Buxton et al.

Toolglass widgets are new user interface tools that can appear, as though on a transparent sheet of glass, between an application and a traditional cursor.

I've been interested in taking a stab at implementing a system like this ever since seeing various takes on it from Alex Warth and Patrick Dubroy. Here's how far I've gotten in a couple of afternoons:

I found out a couple of interesting things. First of all, magic lenses are fun to use, and feel distinctively different to what most of the computing is like right now. Because the transformations are pure functions (they don't destroy the underlying material, just like a physical lens wouldn't), you can experiment without any fear of losing the original data. You also often get unexpected results, which is great for Novelty Search.

Not everything is great though. It's often hard to say what's going on with multiple overlapping toolglasses, and if the interaction is best understood with no more than one-two lenses, then it might imply, that some other UI primitive could work just as well (or even better). Additionally, z-order matters for the output (stroke blue → mirror results in different image than mirror → stroke blue), and I couldn't find a nice way to visualize that ordering. Making a side-pane hierarchy UI didn't feel right, and automatically reordering z axis on interaction was even more confusing.

Books I enjoyed recently:

  • The New Media Reader — it seems as if I was subconsciously collecting the works combined in this book for past couple of years; also, any book on technology that starts with a short story from Borges is worth checking out; I'm not alone in praising this book, and if you were to pick just one of the things I'm recommending, this should be it
  • HCI Remixed — collection of famous HCI practitioner's essays on papers that were most influential for them; the book was a bit uneven for my taste, some writings felt like a summary of the underlying material, where some of them gave me new, interesting points of view
  • The Act of Creation — cult classic on where breakthroughs come from; quite long but inviting read, going into a ton of historical tangents all circling around the topic of Where Do Ideas Come From?
    • I also read Art & Fear which is almost an antithesis to "The Act of Creation", stating that art comes from repeatedly "walking the walk", which I think Koestler would agree, is just one of the "magic ingredients"
  • three nicely complementary readings around "rationality":
    • The Philosopher's Toolkit — as the title suggests, it's a set of tools for making, thinking about, and evaluating (philosophical) arguments; probably not very useful on its own, but often gives an interesting framing of more and less famous philosophical arguments
    • The Scout Mindset — a bit of a lighter read on why "if only they were rational they would understand" is often the wrong framing of a problem
    • In the Cells of the Eggplant — it's the first time I read work-in-progress book (well, what is available right now); even if only partially written, it presents an interesting take on how to be rational about rationality itself
  • in my philosophy thread I made a detour into logic:
    • Logicomix — a very light introduction to the topic through the lens of Bertrand Russel's life, in a form of a comic book
    • A Concise Introduction to Logic — a bit more formal introduction to most topics around formal logic, a good jumping-off point into further reading
    • I skimmed An Introduction to Mathematical Logic which I'll for sure revisit once I want to explore a bit more of the mathematical side of logic
    • "Sets, Logic, Computation", "Incompleteness and Computability", and "Boxes and Diamonds" from Open Logic Project — again partially skimmed those, but I really enjoyed the journey from formal logic, through lambda calculus, to modern computation

On the web:

What do you think about this shorter style compared to the previous one?

Always curious to hear your thoughts and comments on any of the topics I mention. And in the meantime — see you in a couple of months!

Szymon Kaliski © 2021
mailtwittergithubwebring