make tools (for learning, not for thought exactly), publish them, observe their use, distill insights, share
— Robert Cobb
Some of my favorite work in tools for thought comes from idiosyncratic Twitter tinkerers. This group often produces fascinating work, but it's usually missing one or more of these steps. The most common pattern seems to be: a bricoleur identifies some powerful idea about a representation and design a prototype, but then fails to engage seriously with observing and deriving insight from the systems they've built. Sometimes this comes from technical barriers—the prototype is too quick-and-dirty to be used in a serious context, so their insight is limited. But I think there's also a cultural gap here, a missing research practice of careful, diligent observation and synthesis. Too often these projects have the flavor of "Look, I made a thing! Isn't it cool? How many people can I get to use it?". But the question they need to be answering is: "What powerful, generalizable ideas can we learn from this project? How should the next wave of systems build on this?"
— Ratcheting progress in tools for thought - Andy Matuschak