As I was working on Programmable Ink projects at Ink&Switch, in hopes to record better-looking demos (which never really panned out), I cobbled together some OpenCV to detect four markers glued to a cutting board.
You can read more about the project in the section below, which originally appeared in the Q2 2022 newsletter.
As you probably know, most of the work I've been doing for a while, has been tablet-oriented. The thing that we're constantly experimenting with are the touch-and-stylus interactions. To visualize what's going on, we usually add an overlay showing fingers and stylus positions, but that is just a proxy for actually seeing how the application is used.
To mitigate this, I've built a very simple DIY document camera using:
ArUco markers glued to the cornersI use the ArUco markers for perspective transformation to fake the camera floating directly above the table.
The code itself is just about 300 lines of Python and OpenCV.
At first, I tried to prototype this in a browser using OpenCV.js ↗ but the performance was pretty bad — which is unsurprising, as the official build is emscripten, not wasm.
I've never really tried the unofficial wasm build, and I just defaulted to what everyone recommends: "just use Python".
Here's a short demo (and a sneak peek at the "sketchy logic solver" project I mentioned in the intro):
And since I know the exact ArUco marker sizes, one additional fun experiment was adding measuring capabilities to the tool:
Of course, this is something that Kevin already explored in detail ↗, concluding that the accuracy is not good enough for use in a CAD context — which I agree with (forget being 1MM accurate with 1080P webcam). I'd be curious to try this with a better camera (anyone knows of any inexpensive 4K ones that are not terrible?), but ultimately good calipers probably win here anyway.
While we're on the topic of webcams, I was so happy to stumble upon uvcc ↗ which allows you to configure the Logitech-family webcams without using their clunky app.
Finally, I spent an afternoon trying to package this as an .app and failing with both py2app ↗ and pyinstaller ↗.
In the end, I made an Automator wrapper around a CLI one-liner which works great for my usecase — starting the app directly from Spotlight search.
420 words published on 2022-06-13 — let me know what you think