I've been also continuing research at Ink&Switch around the topic of "programming by drawing" for a tablet form-factor (trying to bring some of the Dynabook ideas to life). Latest spin-off work includes in-house Lisp with a projectional editor. This work is not public yet, but if you're interested in these topics, I'm happy to give a demo on a call.
At flow/control, we finished a bit more conceptual consulting project with Clay. We worked on exploratory research on integrations and enrichments — basically how to make Clay a central point for connecting various web applications through a end-user programmable spreadsheet.
Most of my time over the last three months went into Ink&Switch, where we were ramping up on a new project, culminating some of the research I've been involved in for the past year. We're also finishing (hopefully!) a big "real-life" project, so there wasn't that much time for side-explorations, but I managed to squeeze in two smaller things that might be interesting to some of you.
Running incomplete programs with type checking is still an open research topic, one of the most famous examples being Hazel, which not only allows you to run unfinished programs, but can also (partially) type check them! It's definitely worth looking into, but: one — it's a research project, and two — a totally new universe to adopt (which you probably shouldn't do right now, because of the first point).
Designing and implementing tools with a good cognitive fit for user and task is an exercise in removing gratuitous difficulties and finding familiar, expressive interface metaphors. Because there is an existing way the task is being performed by existing users (...), it is possible to invent a new tool idea and quickly test the idea with users to see if it in fact makes a task easier. Often users are able to state with confidence whether or not they believe a new way of doing a task will be easier and more efficient based solely on hearing the idea, without having the new tool implemented. However, designing and implementing a dissemination strategy with a good social fit for the user community is far more difficult. A good social fit means users eagerly give up an old tool and adopt a new tool. The time constant in social fit (new product adoption cycle) is typically much longer than the time constant for cognitive fit (duration of an authoring task). New tools that are free to try, easy to learn, similar to existing tools, demonstrate rapid payback, small and fast are likely to spread rapidly in a community of users.
— ATG Education Research - The Authoring Tools Thread - Jim Spohrer
Non of our recent work would happen without Figma. From boards with brief breakdown, desk research, data discussions, concept drawings to WIP screenshots, branding guide and UX user journeys. We bring visual thinking to the way we both design and code.
— Marcin Ignac - https://twitter.com/marcinignac/status/1200022203728838656
I understand, then, why researchers flock to the safety of institutions. Imagine studying something that nobody else is studying, for reasons you can't really articulate, without knowing what the outcome of your work will be. For the truly obsessed person, the need for validation isn't about ego; it's about sanity. You want to know there's some meaning behind the dizzying mental labyrinth that you simultaneously can't escape and also never want to leave.
— Nadia Eghbal - https://nadiaeghbal.com/independent-research
Perhaps a more subtle practice I've found helpful is to find situations where the desired outcome is considered unremarkable.
I like being in a PhD program because "produce novel, high-quality research" is just what everyone is doing, not some cool special thing.
— Geoffrey Litt - https://twitter.com/geoffreylitt/status/1375116106734702594
The development of a program is not linear. Programmers neither write down a program in text order from start to finish, nor work top-down from the highest mental construct to the smallest. They sometimes jump from a high level to a low level or vice versa, and they frequently revise what they have written so far (...). For the purposes of programming support, that is all we need to know. Although the causes and nature of deviations from top-down development have inspired much research, the implication for a programming environment is quite straightforward; the order of working should be left to the programmer, not prescribed by the environment.
— Usability Analysis of Visual Programming Environments - A "Cognitive Dimensions" Framework
As you're typing them they're in a live editing preview mode, but when you press enter they get applied "permanently".
(...) you don't have to hit enter or a button to make it execute, and you don't have to undo if it didn't do what you wanted it to. The code is continuously parsed and interpreted as it is edited (as long as it is valid). The function-invocations register themselves with the editor UI, which draws these visualisations and keeps state for them as long as the editing session exists and the function invocation is not removed. This lets the user tweak the code and make sure it works, as well as tweak the selections etc. Once the user is happy with the result, they can commit the changes.
The traditional cognition approach assumes that perception and motor systems are merely peripheral input and output devices. However, embodied cognition posits that the mind and body interact "on the fly" as a single entity. An example of embodied cognition is seen in the area of robotics, where movements are not based on internal representations, rather, they are based on the robot's direct and immediate interaction with its environment. Additionally, research has shown that embodied facial expressions influence judgments, and arm movements are related to a person's evaluation of a word or concept. In the latter example, the individual would pull or push a lever towards his name at a faster rate for positive words, than for negative words. These results appeal to the embodied nature of situated cognition, where knowledge is the achievement of the whole body in its interaction with the world.
Decades of research have established that wealth above a certain level does not add to an individual's satisfaction with life, and older people who have achieved considerable worldly success often report that raising their children and enjoying their adult company has given them more satisfaction than any career recognition they obtained
Some of my favorite work in tools for thought comes from idiosyncratic Twitter tinkerers. This group often produces fascinating work, but it's usually missing one or more of these steps. The most common pattern seems to be: a bricoleur identifies some powerful idea about a representation and design a prototype, but then fails to engage seriously with observing and deriving insight from the systems they've built. Sometimes this comes from technical barriers—the prototype is too quick-and-dirty to be used in a serious context, so their insight is limited. But I think there's also a cultural gap here, a missing research practice of careful, diligent observation and synthesis. Too often these projects have the flavor of "Look, I made a thing! Isn't it cool? How many people can I get to use it?". But the question they need to be answering is: "What powerful, generalizable ideas can we learn from this project? How should the next wave of systems build on this?"
— Andy Matuschak - Ratcheting progress in tools for thought
- designing the questions to ask after building a prototype (or even building the prototype to ask specific questions) is important part of the puzzle - building just for building sake lacks the feedback loop
These representations weren't mere scientific "discoveries". Each of them essentially enabled all subsequent scientific breakthroughs thereafter. A powerful new form of representation affects everything, forever.
— Bret Victor - CDG Research Agenda
- new powerful notations "stick around" and infect human thinking for generations
On various timescales needed for thinking and research
I've advocated "learn everything and then forget it except for the perfume". This can create a mental space for thinking which will inescapably be helped by what we know - it's really hard to completely forget! - but in which what we know (mostly meaning what we believe!) is far enough away to allow us to feel things, listen to our subconscious whispers, and generally barge around.
— Alan Kay - https://www.quora.com/What-advice-would-Alan-Kay-give-a-curious-individual-to-improve-their-ability-to-think-and-learn-Is-there-a-place-to-see-his-library-%E2%80%94-every-book-person-and-research-he-has-studied/answer/Alan-Kay-11
One research study asked students to think about an important exam. Half of the students were asked to put in writing specific plans of what/where/when they would study. Later, all students were asked to do a word association test. The group of students that did not write any study plans produced more word associations related to studying because studying was still on their mind; the group who did write down their study plans did not exhibit a comparable bias during the word association test.
From technical side, FabFungus was a culmination of three personal research projects:
We started by exploring the dataset, using variable in-house pex library, writing a lot of small exploratory apps to test the ideas and see what we are dealing with. We were also researching ways of rendering on such a huge display, in order to keep artifcats away.
Because my first time tracking system has been built before I've started tracking time, I have no idea how long it took to build it. The idea was pretty simple though, I wanted to store all the logs in a database, and be able to query and analyse them. The log consisted of starting and ending times, project's name that it belonged to and type — either personal or work, later another type, research was added, but I didn't use it much. I also tracked payments with date, amount and project name. Everything was held together by project name, so my tracking was only concerned about how I work on very specific things.
I enjoyed this month's exploration and learning a new language as nice break from tool-making research.
I was quite tired after last year, where I worked on monthly projects and pushed hard to publish something every month. I also knew that this year would be a bit different due to my personal life, so I had to plan accordingly. I decided to aim for working on one personal project every day, but without any tight deadlines; this could be reading a research paper, exploring some technology if I had time, or playing a bit of music.
Next time, I would probably aim for either fully research/learning related residency, or one where I know almost everything, and produce artwork. I've tried to combine both, and ended up only building the tools, without ever actually making anything bigger with them.
First week was full of learning in the most frustrating way — trying and failing over and over again. I still didn't know what exactly I wanted to do, and was exploring different techniques. I finally settled on using SDFs (signed distance fields) after short research on meshing isosurfaces.