Szymon Kaliski

Building Parametrium

Interactive parameter space explorer for P5.js

Parametrium is a parameter space explorer for P5.js sketches. It analyses abstract syntax tree to find constants in uploaded code, and modifies them with interactive evolutionary algorithm. It was built during my 1-project-a-month meta-project in March 2017.

Intro

When I work with creative generative systems, it feels like I spend as much time fiddling with different knobs as I spend writing code. By knobs I mean not only GUI sliders, but also code constants, array sizes, etc. The bigger the system is, the more knobs it has, and it becomes harder and harder to explore. It also usually feels like the nicest results come out of extremely specific set of values, and once I find them, it's scary to start fiddling with them again.

I always wanted to able to move through this parameter space fluidly and see the results in real time. To collaborate with the machine, instead of setting everything by hand.

In February I explored evolutionary algorithms for generating simple GLSL code in WallGen, and I started wondering how much of that evolutionary process could be abstracted and re-used for more general purpose.

AST

First step in making this a reality was to detect all the numbers in code. I could have used regular expressions, but it's usually a bad idea when it comes to very complex structures (like code). A proper way of analysing most of programming languages, is parsing them into abstract syntax tree - data structure describing the source code. This tree can be then traversed and transformed, and as long as it's kept in proper shape, it can be pretty-printed back into text form. So a simple identity function on code would turn it into AST and back into code: code -> AST -> code.

I've spent a few days trying out different libraries for building and transforming ASTs, and settled on recast. Parsing with recast is pretty simple in it's most basic form:

const ast = recast.parse(sourceCode);

recast.visit(ast, {
  // visit Literal - numbers, strings, etc...
  visitLiteral: function(node) {
    // node.value has raw and parsed value,
    // node.value.value is parsed: 17 instead of "17"
    if (isNumber(node.value.value)) {
      console.log(node, 'is a number!');

      // I now know that node is a number, so I can modify it
      node.value.value = Math.random() * node.value.value;
    }

    // tells recast to keep on going
    this.traverse(node);
  }
});

I tested this on few simple P5.js sketches, it worked well until I hit loops, and colors.

As for the loops, the problem is that numbers can be used to iterate over array, and array[1] exists, but array[0.3551355] does not:

for (var i = 0; i < 10; i++) {
  points[i].draw();
}

If starting or ending conditions get changed to floating point numbers (or numbers outside of points.length) this would break horribly, with the infamous undefined is not a function. I had to find a way to detect if number is part of for/while loop, there are two easy ways in which this could happen: for (var i = 0 ..., and for (i = 0 .... It's enough to walk few times up in the tree, and find out whether parent node is for or while.

Unfortunately there's one much more problematic case:

var n = 10;

...

for (var i = 0; i < n; i++) { ... }

I've spent a good few days on this one, and I'm still not sure whether my approach is the best one, but it seems to work. Whenever AST finds a number, which is a part of variable declaration (var i = 10), I ask AST for a whole scope in that place, and traverse it, looking for anything that involves this variable's name (i in this example). When I find something, I test again if it's a part of for/while, and act on that number differently.

Relevant piece of code can be found on github: code-transform/index.js. If you know a better way, please submit an issue or pull request.

Colors are a simpler version of this problem, for example turning background(100) into background(300) (or worse background(-100)) doesn't make any sense. Numbers found inside of few specific functions (fill, stroke, background, etc...) should be limited between 0 and 255.

So, another check that I run finds out if the callee (function that calls some piece of code) matches one of pre-defined functions. We have two basic cases here, simple one: fill(10, 20, 100), and complex one, same as with for/while loops:

var r = 120;

...

fill(r, 10, 20);

I tackled it exactly the same as with loops, but instead of looking for for/while I look for callee names.

A relevant piece of code can be found on github: code-transform/index.js.

Both solutions are not ideal, but cover most of the cases I could find.

Final code analysis and transformation has to be split into two parts to fit the evolutionary algorithm approach. I find numbers and their types first and run evolution on this list (with mutation taking in account the type of a number). When new set of numbers is evolved, I generate a new code, using the modified array and traversing through the input again, this time replacing numbers at matching indexes.

Displaying the results

Another thing to tackle was how to show more than one sketch at a time. The simplest solution would be to force users to use instance mode and provide different containers for each of the sketches. My aim was to make the tool as frictionless as possible and to not force users to change anything about their code, so I thought of using <iframes> to contain the sketches. I wanted to have this running without a backend, so responding with different HTML documents containing different modified sketches for different URLs, would be tricky to do. Luckily there's srcdoc attribute on <iframe> in html5, and the final rendering code became very simple:

const generateHTML = code => `
  <style>
    * { margin: 0; padding: 0; }
  </style>

  <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/p5.js/0.5.7/p5.js"></script>

  <script type="text/javascript">
    ${code}
  </script>
`;

const Renderer = ({ width, height, code }) => (
  <div>
    <iframe
      scrolling="no"
      width={width}
      height={height}
      srcDoc={generateHTML(code)}
    />
  </div>
);

Attempt at automated testing

While I was working with AST part of the project, I wanted to test on as many different P5.js sketches as possible. Someone on twitter suggested using openprocessing as a source of sketches, and I quickly wrote this simple scraper: openprocessing-scraper. There's over 500 megabytes of sketches there, with over 2700 P5.js ones, it didn't make sense to test them all by hand.

I experimented with simple electron application that would load all the sketches in directory one-by-one and report on errors. Unfortunately, it turned out that most of the sketches reported as broken had already been broken before my AST transformer did anything. On the other hand, errors that I was looking to find wouldn't necessary show up as JavaScript errors, so I gave up on this for now.

In case anyone would like to take this idea further, I left it in the repo: code-transform-electron-tests.

Fin.

This project was the first one this year that really pushed me to learn something new — traversing and transforming JavaScript ASTs. I've spent more time exploring and failing, since I knew all other parts were either already there (evolution), or easy to make (UI).

As usual, there's a lot that could be improved, and I think with relatively little work, this could support different creative-coding libraries as well.

You can play with live version here: http://szymonkaliski.github.io/parametrium.

Code is open sourced and available with MIT license here: https://github.com/szymonkaliski/parametrium.