Solving Things Visually

  • two main things here:
    1. visualization (charting) as a way to see and understand the data
    2. solving things by means of drawing/sketching - diagrammatic/geometric thinking
  • for the first one:
    • visualization is a macroscope - allows us to work with/look at complex ideas:
      macroscope
      (source: Model Driven Management of Complex Systems)
    • we have a lot of tools for creating them in software, but no "silver bullet"
    • I have a feeling that there are important primitive interactions/UIs missing for creating a generic data-visualization-building software
  • for the second one:
    • Bret Victor makes a point that programming is currently blindly manipulating symbols - visualization is a solution for this blindness

      the author sees and manipulates indirect symbolic representations, and must imagine how they give rise to dynamic behavior

    • diagrammatic/geometric thinking seems to be an important skill in problem-solving, a cognitive style (after Seymour Papert) which allows us to work with the ideas visually, usually by sketching
      • Dynamic Medium could be great for thinking this way - if only we had proper tools!
  • 3Blue1Brown's YouTube math explainers are very visual, and seem to give great intuitions into mathematics because of that
    • can I imagine a tool for directly manipulating them into existence?
      • Chalktalk by Ken Perlin seems to be one such imagining (with actual implementation)
      • Apparatus by Toby Schachman is another one (later productised into Cuttle)
    • would such imaginary tools be good only for transferring knowledge, or also for gaining new knowledge? - in other words would I invent calculus just by having access to a Dynamic Medium that works this way?

Point of View Is Worth 80 IQ Points

— Alan Kay

  • we can provide new point-of-views in software through (data) visualization

So far, I haven't actually made much use of intersections, but it's likely that I'm not yet "thinking geometrically" enough.

— Drawing Dynamic Visualizations Addendum - Bret Victor

  • Bret mentions "thinking geometrically" a bunch of times in this article, I think it's an important insight of feedback loops between tools we use, and styles of thinking that they promote (Tools Building Us)

Sanjay had a neat trick of using GraphViz to debug his compiler by outputting and laying out various compiler graphs (e.g. CFGs). It is obvious when you think about, but a normal compiler dev wouldn't think to do that.

— https://twitter.com/seanmcdirmid18/status/1225852873096876032

Visibility is everything. In many cases the most effective person isn't the smartest or even the most qualified, it's the person who has visibility into the problem. Learning to make things visible gives you a superpower.

— https://twitter.com/ibdknox/status/1225860283744174080

Debugging may simply involve carefully watching for an unexpected change

— Aesthetics of Computation - Unveiling the Visual Machine

  • if we have perfectly visual systems, we can "See Everything" and just watch for what's going on, and notice when things don't work
Bret Victor's Thesis
Cognitive Medium
Data Layer
  • this could be helpful when Solving Things Visually - data is a something that user has tangible access to, it occupies space instead of being ephemeral
End-User Programming
  • Solving things Visually - and Alan Kay's anecdote about drawing a circle - when thinking "visually" one wants a compass, and not a sin(x), cos(x) equation
Future Of Coding
Ladder Of Abstraction
  • interaction is important part of abstraction - being able to interact with the visualization (and see other visualizations change) helps us build mental model of the solution space (Solving Things Visually)
    • human brains are good at matching visual patterns, but slow (and bad) at logical inferences
    • it makes sense to turn these inferences into visual representations, so we can reason about them with our visual side of brain
    • it makes sense to be able to play with them to gain even better understanding of them - static visualization is not enough, interaction with living system combined with visualization is important
Memex and My Limits
  • I'm also not sure if any tool will be able to give me more IQ points (although a point-of-view can - solving things visually)
Simulator
  • it's interesting to look at "simulator" problems from the point of view of VPLs - visualizing program state/execution is more useful than visualizing the AST - this, again, ties into Solving Things Visually

    What kind of a thinker would you become if you grew up with an active simulator connected, not just to one point of view, but to all the points of view (...) so they could be dynamically tried out and compared?

    — User Interface, a Personal View - Alan Kay

The Incredible Power of The Right Interface
  • making Solving Things Visually possible (Cartesian Coordinates)
    • Data Visualization can be thought of as an alternative, sometimes more powerful, interface to data
Visualizing the Problem Domain is More Useful than Visualizing the Program Shape
  • in an ideal world, we want to be able to Solve Things Visually which requires seeing and acting on the model of the problem, not on how the problem is represented in the computer
VPL
Working in the Dynamic Medium