Solving Things Visually

  • two main things here:
    1. visualization (charting) as a way to see and understand the data
    2. solving things by means of drawing/sketching - diagrammatic/geometric thinking
  • for the first one:
    • visualization is a macroscope - allows us to work with/look at complex ideas:
      (source: Model Driven Management of Complex Systems)
    • we have a lot of tools for creating them in software, but no "silver bullet"
    • I have a feeling that there are important primitive interactions/UIs missing for creating a generic data-visualization-building software
  • for the second one:
    • Bret Victor makes a point that programming is currently blindly manipulating symbols - visualization is a solution for this blindness

      the author sees and manipulates indirect symbolic representations, and must imagine how they give rise to dynamic behavior

    • diagrammatic/geometric thinking seems to be an important skill in problem-solving, a cognitive style (after Seymour Papert) which allows us to work with the ideas visually, usually by sketching
      • Dynamic Medium could be great for thinking this way - if only we had proper tools!
  • 3Blue1Brown's YouTube math explainers are very visual, and seem to give great intuitions into mathematics because of that
    • can I imagine a tool for directly manipulating them into existence?
      • Chalktalk by Ken Perlin seems to be one such imagining (with actual implementation) visually building a pendulum in Chalktalk
      • Apparatus by Toby Schachman is another one (later productised into Cuttle) Bézier curve construction in Apparatus
    • would such imaginary tools be good only for transferring knowledge, or also for gaining new knowledge? - in other words would I invent calculus just by having access to a Dynamic Medium that works this way?

Point of View Is Worth 80 IQ Points

— Alan Kay

  • we can provide new point-of-views in software through (data) visualization

So far, I haven't actually made much use of intersections, but it's likely that I'm not yet "thinking geometrically" enough.

— Bret Victor - Drawing Dynamic Visualizations Addendum

  • Bret mentions "thinking geometrically" a bunch of times in this article, I think it's an important insight of feedback loops between tools we use, and styles of thinking that they promote (Tools Building Us)

Sanjay had a neat trick of using GraphViz to debug his compiler by outputting and laying out various compiler graphs (e.g. CFGs). It is obvious when you think about, but a normal compiler dev wouldn't think to do that.

Visibility is everything. In many cases the most effective person isn't the smartest or even the most qualified, it's the person who has visibility into the problem. Learning to make things visible gives you a superpower.

  • this could be helpful when Solving Things Visually - data is a something that user has tangible access to, it occupies space instead of being ephemeral
  • Solving things Visually - and Alan Kay's anecdote about drawing a circle - when thinking "visually" one wants a compass, and not a sin(x), cos(x) equation
  • interaction is important part of abstraction - being able to interact with the visualization (and see other visualizations change) helps us build mental model of the solution space (Solving Things Visually)
    • human brains are good at matching visual patterns, but slow (and bad) at logical inferences
    • it makes sense to turn these inferences into visual representations, so we can reason about them with our visual side of brain
    • it makes sense to be able to play with them to gain even better understanding of them - static visualization is not enough, interaction with living system combined with visualization is important
  • I'm also not sure if any tool will be able to give me more IQ points (although a point-of-view can - solving things visually)
  • it's interesting to look at "simulator" problems from the point of view of VPLs - visualizing program state/execution is more useful than visualizing the AST - this, again, ties into Solving Things Visually
  • making Solving Things Visually possible (Cartesian Coordinates)
    • Data Visualization can be thought of as an alternative, sometimes more powerful, interface to data
Szymon Kaliski © 2021