Selected work and projects
Sino is a single-bell belltower where a bronze bell is engaged via an electronic transducer / resonator. Sino is a collaboration with Mike Rijnierse. I have worked mainly on the electronics for driving the transducer, software, sound-design and composition.
In western culture, the bell initially had an crucial function as a call for alarm or a signal indicating the closing of city gates. In Sino the bell is placed upside down resembling a singing bowl. Often used in Eastern rituals, singing bowls are also known as ‘standing bells’ or ‘resting bells’. In Sino the bell does not signal a call for alarm or action, but as a ‘resting bell’ it becomes an instrument for inner reflection, a call for contemplation.
Using a resonator in stead of a clapper to engage the bell is a very different technique. A strike of the clapper brings out the entire spectrum of harmonics as a result of the bell's shape (the tuning), giving us a the rich bell sound with a typical short attack / long delay envelope. With the resonator, although the frequency palette is still limited by the tuning and the harmonics present in the bell, we can pinpoint and bring out a single resonant frequency and engage it for as long as we like. This way we can deconstruct the harmonic qualities of the bell into signle frequencies and from that build a new soundscape with multiple layers of different timbres.
Sino was first presented as part of the exhibition: A State of Amplitude. As this exhibition featured multiple sound works, the soundtrack was shaped as a slow build-up to an apex followed by a period of silence, to give room and connect with the other works in the exhibition.
The development of this (work in progress) music instrument started as an experiment to exploit ‘variable samplerate’ techniques with modern hardware, utilizing the ‘new’ raspberry pico microcontroller.
The base of the sampler voice consists of the 4$ Raspberry Pico microcontroller in combination with 12bit AD-converters. The Pico microcontroller is a good fit, because of its unique FPGA like Programmable IO. With DMA (Direct Memory Access) and this Programmable IO technique we can stream data fast enough on this microcontroller to achieve variable playback speeds ranging from 0Hz to 1.5Mhz. In musical terms this allows an audio recording in CD quality to be pitched or down by 5 octaves, without the need to do any rounding, averaging or interpolation of the sample data in software. This way we avoid distortion or artifacts (aliasing) caused by software pitching / resampling. We keep audio data clean and pure and eliminate the need for anti-aliasing filters in the output stage.
The oscilloscope shows a triangle wavetable consisting of 600 datasamples at a frequency of 2000 Hz, demonstrating an effective variable samplerate of 1.2Mh. Inspired by Ensoniq’s first sample synthesizer from 1984 : The Mirage, each sample voice in this instrument will be equipped with a dedicated analog filter. In this instrument the hardware filter will be configurable via software for different configurations and use cases. Below the triangle wave on the oscilloscope you can see that same signal being fed to a AS3320 analog filter in a 24db lowpass configuration with resonance. The cutoff frequency is swept by another AD-converter that is being controlled by the microcontroller.
Geotool is an instrument by Louis Braddock Clarke. Louis asked me to be his mentor as part of his grant from Stimuleringsfonds Talent Development. Geotool is a portable magnetic field recorder, it features three fluxgate sensors, GPS, SD-card logging and provides complex auditory feedback on real-time measurements. Together we worked on the first two prototypes of this instrument, experimented with different sensors and solved many bugs. The design of Geotool is the first phase of Louis' new research/art project.
The Time Bandit is a Midi-Clock device featuring an internal clock with a smooth range from 0.1 to 255.0 beats per minute. Next to a Midi-Master Clock for 5 instruments, the Time Bandit can function as a general purpose two in / five out Midi Merger and Midi Splitter. The Time Bandit is custom built instrument for: René de Haan.
I've worked on a couple of projects for DPI Animation House, commissioned by the Rotterdam Harbour - Futureland visitor center. This time the project entailed the digitization of the harbour. The story is told via an Augmented Reality application using object tracking. I've build the project with the Unity3D game engine, I've written all the software for this project, including custom shaders (for the hologram effect) and some procedural animations.
made in 72hours for the LudumDare46 gamejam
- You Play Red, White is your opponent, capture all white cells to win the game
- Avoid white’s bullets, or use them to sacrifice one of your Red cells
- You can switch between your cells with TAB or select them with left-click
- Capturing a white cell performs a divide and gives you two red cells
- Explosions can cause chain reactions, if other players or opponens are caught in the blast.
- White cells hit by white team crossfire can divide
- It’s hard to write proper A.I in a gamejam weekend
- Keep Red Alive!
I was a guest teacher for Graphic Design Techweek 2020. As a point of departure for this workshop I wrote 2 lectures: The first about generating chaos, ranging from dice rolls to noise to entropy for software random generators. The second was applying chaos to your own project by means of rule-sets, mapping, filtering and transforming data. Students worked on their own original ideas, and used their preferred type of randomness to influence some part of their projects. The end result was a collection of unique and diverse experiments; a multiplayer painting game, a musical pendulum noise instrument, some procedural poster designs, a city mapping system for determining where to place a grafitti and a satellite/weather music generator.
Earlier in 2018 I wrote software for Underware's Grammatography project. For the exhibition 'From Typography to Grammatography' at the Print Gallery in Tokyo, Underware also asked me to design an accompanying soundtrack. I build a system to generate / synthesize the soundtrack in real-time and match the live rendering of the animated glyphs.
The album 'Pythagoras - Live at Pulchri' is released! Recorded at Ilja Walraven's DNA expo in 2015. René and I also worked on an accompanying video clip with Ilja Walraven's paintings
René de Haan: Synthesizers, Mellotron
Rob Bothof: custom instrument / 4track looper
Peer Wassenaar: Bass Guitar
Joshua Samson: Drums, Percussion
Global Progress Platform is a prototype online data visualizer for global data analysis and presentation, commissioned by : Purpose Plus. Data is retrieved real-time from the 'World Bank Open Data' database and visualized in a 3D web environment. I produced both design and code.
see also: https://www.purposeplus.com/world/
This is a light installation in the Auditorium of former embassy building for the United Stated of America in The Hague. The embassy is curated by West Den Haag as an exhibition/event space. After months of work the auditorium has been restored in its original state, but with a few twists. For the auditorium I developed a light installation encompassing 20 individually wireless controllable lights and 25 meters of controllable led-strips.
I designed and produced all the custom hardware and software this project and wrote a custom communication protocol in stead of using DMX. As it was not possible to hard wire the lighting in the auditorium, all lights are controlled via WiFi. Each light has it own wireless micro controller and power led driver build into the housing. While most of the the time the animation of the lights is used in a subtle way, the installation is capable and tuned for some fast and synchronous sequencing for live music events.
I worked with the type foundry Underware on their new concept 'Grammatography', a form of digital typography that combines the expressive possibilities of handwritten text with the accessibility of digital texts. For Grammatography I developed software tools to produce and aid in the design of higher order font interpolations. The concept was introduced in the HOI-tour and has won several type design awards: TDC (New York), TDC (Tokyo) and German Design Award (gold).
Below you see: 1) Biang is the most complex character in contemporary Chinese, consisting of 58 strokes. This animated biang is technically just like any other letter in a font, made inside one glyph slot. And then animated with our own code and higher order interpolation, HOI. 2) Bello write, a font with real animated calligraphic brush strokes, infinite high resolution, no stop motion frames, just perfectly controlled curved High Order Interpolation.
The RMC is a complete music improvisation setup build entirely from scratch. It is my personal live improvisation rig and the result of years of experimentation, improvisation with all kinds of d.i.y. instruments and live setups. Inputs feature Japanese arcade controls and high quality pots and faders via a custom low latency input scheme. Processing is done on the UP2 development board with Intel Celeron N3350 CPU, 2 Gb ram, running debian linux in combination with a teensy 3.6 board (for input scanning and led control). Software is developed in C++ in combination with puredata (in the form of libpd) as an audio engine. Dual Midi-Clock output is provided to jam with friends in sync.
From a musical stand point this setup features a 4 channel audio-looper, a unique arpeggiator / sequencer combi, per channel pitch control, filters, effects and various input sources like synths, drums and samples. The system is road-ready and toured across Europe within the first weeks after its initial realization.
I've been working on a real-time terrain generator / word-builder as part of the future game project. This started out as a couple of experiments to try out Unity3D's new job system. This Job System allows high performance code in c# by using efficient multi-threading and CPU cache optimization techniques. I think the job-system is extremely powerful. With this i'm able to generate high quality landshapes in real-time with millions of polygons in the CPU, with room to spare. Generating this in realtime saves a lot of memory and the terrain can be extremely large, as we only need to generate landscape around the camera. My terrain generator is build on multi layered perlin noise, but with a couple of twists. Height is not calculated linearly, but is mapped via a couple of curves that can be tuned manually. Via these curves we can shape ridges and create some erosion like features, without expensive simulation computation. I've build a cool LOD (Level of Detail) system for this terrain too. Close to the camera, terrain slabs use a lot more polygons for up close details than in the distance. Different levels of detail are crossfaded via fast noise clipping in the terrain shader (see one of the video's below). A little bit of temporal aliasing in post will smooth the result.
Piano / Forte is a collaboration with Mike Rijnierse. This work involves one or more piano harps swinging in the air, like pendula. The piano's sound is invoked via multiple resonators. A control signal is send to these resonators, making the piano strings vibrate when their tunings corresponds with the control signal's frequencies. One could think of it as an analog vocoder, where the piano harp is the carrier, and the modulator consists of software controlled sound transducers (resonators). This modulation signal is generated in real-time by a generative algorithm with input sensors placed on the piano harp. These sensors measure speed, orientation and acceleration. As a result, high speed swings introduce more harmonics in the control signal and increase the amplitude, accentuating the movement of the object. In the case where the installation consists of multiple piano harps, like in the Sound of Science exhibition at DordtYart, both instruments communicate with each other via ethernet. They play together and enhance the performance, as they are can create a generative chord composition together. Even though the harps swing out of phase at different heights, they are able to plan their performance ahead of time, make sure they play in key on overlapping swings and can pause at corresponding intervals. For this project I produced the software, electronics, composition and sound design.
see also: https://mikerijnierse.nl/piano-forte/
Tennis vs SHMUP. This is a prototype game, made in 72 hours for the LudumDare 41 gamejam. Written in c# and using the Unity3D game engine. The theme for this competition was: Combine 2 Incompatible Genres.
play here: Court Invaders
For KIMO.ai, an online learning platform, I have worked on multiple data visualizations. This prototype for the knowledge cloud aims to visualize a collection of learning content. The graph does not organised the content in a classically hierarchy, but based on similarities between subjects, similarities in content and the ease for a student to transition from one piece of content to another. Another process in the back-end would be able to calculate an ideal learning path and display it in this graph. The positions for the different nodes are not fixed and are not precalculated. I used a process of simulation where each node attracts / repels the others based on their similarities. This way the cloud gradually shapes itself into an optimal configuration.
Polytope is a chroreographic object by Ludmila Rodrigues. Polytope engages an open-ended dialogue with visitors and transforms into a physical drawing meandering in the air. For the exhibition of Polytope at FILE, São Paulo I designed a generative soundtrack. A raspberry pi, combined with a PIR sensor, tracks movements. influences the timbre via filter modulation and changes the generative melody.
see also: Ludmila Rodrigues - Polytope
I made the digital 3D model for the ' Snip' by BROOS (artist duo: Pietertje van Splunter & Zeger Reyers). The real model is milled by Blowups. Snip has landed on top of a parking lot in Leidse Rijn / Utrecht, where it is now looking over Berlijnplein square.
Entropy #1 is a generative light sculpture. It generates mondrianesque compositions in a 12 x 12 grid of multicolored light cells. The generation of composition is governed by various rule sets based on the Fibonacci sequence.
Moonolith is an interactive installation by Martin Bricelj / MOTA. I've written the software for both interaction and graphics for the second iteration of this artwork. The software displays an accurate representation of the nights sky. Through motion tracking the sculpture becomes a playful light and sound instrument where visitors can interact with the stars. Graphics are rendered using the Unity3D gameengine and the Kinect is used for depth tracking of movement. To improve performance i've also written a custom computeshader which does a fast depth sort on the gpu.
see also: baraga.net
I built the electronics prototype for SoundShades. A concept and design by Mike Rijnierse. Soundshades consists of a pair of single cell Liquid Crystal Displays that can be controlled via bluetooth audio. Commissioned by Mike Rijnierse.
Relief is an installation in collaboration with Mike Rijnierse. An ultrasonic Speaker sprays high pitched sounds over a relief of panels. playing with the principle of echolocation. A technique that is used by bats, dolphins and other animals to determine the location of objects using reflected sound. Unlike some of the sounds used by animals, the Sound of Relief is in the human audible range. The Speaker is controlled by an algorithm that creates a list of random waypoints and follows this path smoothed by a Catmull-Rom interpolation. The speaker paints a audio-picture of the 3D shape, reflecting it into the surrounding space. Light responds accordingly. For this installation I've worked on hardware interfacing, software and sound design. Relief has been exhibited in at Novas Frequêntias Festival in Rio de Janeiro and Locating ArtScience exhibition in GreySpace / TodaysArt The Hague.
See also: Mike Rijnierse
Turbo Transform is a high speed shape-fitting / shape shifting game. I made it for the Ludum Dare #35 game competition in 48hours. It was build in Unity3D using C#. It features 5 levels of increasing difficulty. As it was quite a complete little mini game I published it on itch.io
download / play : Turbo Transform
Panorama Continuüm was a site-specific installation by BROOS (artist duo: Pietertje van Splunter and Zeger Reyers) for Panorama Mesdag. It features a real time sky simulation of birds and clouds. I produced the animation, graphics, simulation code and sound.
The simulation is projected through a set of lenses that are generally used for dome projections. To achieve a undistorted dome projecten. The image needs to be shaped and counter distorted in a special way to combat lens deformation and fit the shape. In order to design the simulation, while the rest of the installation was still being build I had to create a virtual (yet another simulated) version of the installation. The simulation of the birds uses a classic boids / swarming behaving (see Boids), but combined with a choreography to alternate slowly between chaotic and dense bird flock visualization and peaceful sky. Clouds are generated via multi layered perlin-noise.
The installation CUBE uses mirrors at perpendicular angles to create a kaleidoscopic light projection of multiplied reflections. In contrast to the classical kaleidoscope, where light enters the object, CUBE embodies the light inside and projects its inner multiplication outwards into space. CUBE is choreographed by an autonomic algorithm that directs the synergy between light and sound. Cube is a collaboration with Mike Rijnierse. For this artwork I have produced hardware interfacing, sequencing, software, sound-design, choreography and various software alterations to use CUBE(s) as a live instrument in different contexts.
CUBE has acquired international acclaim, it has been exhibited in various places in the Netherlands, Kikk Festival / Belgium, Manifesta / Palermo, Islamic Art Festival / Emirates and Schema Art Museum / Cheongju, South-Korea. Next to an unconstrained artwork, CUBE has been part of theater production AKASHA, a music liveset by MURW and part of dance choreography Feyerabend.
CUBE & MURW at Grasnapolsky 06 Feb 2016
This was experimental visual/text generator for the purpose of bombing drive-by digital graffiti from the window of an automobile. Part the "BARACCA inside job" 2015 tour. Software was written entirely on the road in c++.
I worked with DPI on their Oculus Ride for Port of Rotterdam. The Oculus Ride was a flyover virtual reality experience, demonstrating what the area of 2e Maasvlakte would be like in 2060. I've written all the code for this project, including a custom Level of Detail System, smooth camera path interpolation and a lot of the programmed animation for moving cars, monorails, ships, etc. I also produced the sound design for the vr-ride. Build with Unity3D and Oculus Rift. I do not have any visual documentation of the project myself, but the linked promo video will give an impression of the whole project including the VR Ride
An Animation based on a poem by Leo Vroman. Leo vroman was a poet, biologist (famous for Vroman effect) and an illustrator. He also made digital illustrations entirely by writing code, a unique style which served as an inspiration for this animation.
Storyboard and Design by René de Haan, Animated by Rob Bothof.
Music: by 15 Apen featuring Frans van Deursen and produced by Nico Brandsen.
Gatekeeper was a light installation in the former VROM building (ministry of traffic, urban planning and environment). The installation emphasizes the hidden quality of the building as a city gate. The installation consisted of 2 sequences of 20 lights controlled via WIFI.
"This must be 2019" was a an expo of futuristic sculptures in the bunker in Westbroekpark The Hague by artist duo Topp & Dubio. For this expo i made a soundtrack generator named: Timaliser. This generator was a piece of software that would slowly analyse images of the works in the expo and generate realtime music compositions. Written in python and puredata.
PrePostByPass was a video installation presented at the State-X festival in HOOP-builind and at Zeebelt Theater. Produced together with Judith van der Made and with music by Andrie van de Kuit. For the installation I build a custom real time VJ engine for 3 projectors. It featured projection mapping, midi controlled grid projections and video overlays.