Szymon Kaliski

Visualizing the Problem Domain is More Useful than Visualizing the Program Shape

• VPL is usually understood as nodes-and-wires (or colorful boxes) - but these approaches are not really "visual" in a sense of visualizing the problem domain - they visualize the program shape - the one-dimensional text buffer turns into two-dimensional canvas, and the naming of intermediate values into wires between them
  • but, this is rarely useful! - the shape of the program doesn't tell us much, the problem space of the domain does, but it rarely maps 1:1 to the shape of the program
  • additionally, in most VPLs, the output "happens somewhere else" - the domain of your problem is at best a second-class citizen (dropping a "canvas" component somewhere in the graph), at worst there is a single "output" in the end (seeing the output in a render box)
• one great counterexample is PANE ↗ from Joshua Horowitz:
  • PANE makes the data the first-class citizen, and the wires are transformations
  • this way, you always work with, and see, the representation of the problem, not of the machine that solves it
• in an ideal world, we want to be able to Solve Things Visually which requires seeing and acting on the model of the problem, not on how the problem is represented in the computer
• Naked Objects ↗ are interesting through this lens - they visualize, and make directly manipulatable, (some of) the "business logic" objects in a way that no additional UI is necessary ("interface" as a way to mediate between internal logic, and user-facing logic)

  The basic idea is that most user interfaces get in the way by intervening between the users and the actual system objects, enforcing process oriented interactions and introducing UI - object synchronization problems. Giving the user the ability to interact directly with the objects and enforcing a 1:1 relationship between what the user sees on the screen and the objects used to represent them allows the users to become problem solvers rather than process followers.

  — The Naked Objects Framework ↗

---

We see code on the left and a result on the right, but it's the steps in between which matter most. The computer traces a path through the code, looping around loops and calling into functions, updating variables and incrementally building up the output. We see none of this.

(...)

Traditional visual environments visualize the code. They visualize static structure. But that's not what we need to understand. We need to understand what the code is doing.

Visualize data, not code. Dynamic behavior, not static structure.

— Learnable Programming ↗ - Bret Victor

"Visual Programming" failed (and continues to fail) simply because it is a lie (...)

When I write code, for example a layout algorithm for a set of GUI elements, I visually see the data in my head (the GUI elements), then I run the algorithm and see the elements "move" into position dependent upon their dock/anchor/margin properties (also taking into account previously docked elements positions, parent element resize delta, etc). This is the visual I need to see on screen! I need to see my real data being manipulated by my algorithms and moving from A to B. I expect with this kind of animation I could easily see when things go wrong naturally, seeing as visual processing happens with no conscious effort.

Instead visual programming thinks I want to see the textual properties of my objects in memory in fancy coloured boxes, which is not the case at all.

— https://news.ycombinator.com/item?id=11245187 ↗

• again, the "shape" of the code doesn't matter - the "shape" of the problem domain does

[Dynamic Authoring] The author creates the material by directly manipulating representations of behavior and data, instead of manipulating a structure. Manipulation takes place in the data domain.

— CDG Research Agenda

Backlinks

1. 2022-04-04Sketching, Modeling and Programming¹
2. 2022-02-01VPL¹