Szymon Kaliski

Feedback Loops in Research are Long

• it's easy to feel unproductive on a given day, since research work can't be easily distilled into a set of todo-items

  • goal setting and reviews become very important, since it's easy to be lost in the day-to-day

• but, it's extremely important not to forget about the feedback loop, to not ignore it! - it's not only building that matters, it's the review, insights, and improvement over time

  make tools (for learning, not for thought exactly), publish them, observe their use, distill insights, share

  — Robert Cobb

  • Understanding Through Building makes the most sense long term, if it impacts the next thing that will be built

Some of my favorite work in tools for thought comes from idiosyncratic Twitter tinkerers. This group often produces fascinating work, but it's usually missing one or more of these steps. The most common pattern seems to be: a bricoleur identifies some powerful idea about a representation and design a prototype, but then fails to engage seriously with observing and deriving insight from the systems they've built. Sometimes this comes from technical barriers—the prototype is too quick-and-dirty to be used in a serious context, so their insight is limited. But I think there's also a cultural gap here, a missing research practice of careful, diligent observation and synthesis. Too often these projects have the flavor of "Look, I made a thing! Isn't it cool? How many people can I get to use it?". But the question they need to be answering is: "What powerful, generalizable ideas can we learn from this project? How should the next wave of systems build on this?"

— Ratcheting progress in tools for thought ↗ - Andy Matuschak

• designing the questions to ask after building a prototype (or even building the prototype to ask specific questions) is important part of the puzzle - building just for building sake lacks the feedback loop

Backlinks

1. 2020-12-13Research and Day-to-Day¹