Real-time / TouchDesigner — Audioreactive & Installation

Code that runs while you watch it, image that answers sound and sensor in the same instant.

What it is

A craft of building moving image as a live system rather than a finished file: node-based and code-driven engines render every frame on the fly, so visuals respond to audio, data feeds, sensors and performers in real time. The medium is the installation, the concert and the architectural surface — works that exist only while the machine is running, with no camera and no pre-baked footage. Output ranges from flickering data screens to multi-storey LED façades and laser-drawn rooms.

How it is made

Node networks, not timelines

Tools like Derivative's TouchDesigner and Notch are node-based visual programming environments: operators are wired into a signal graph that runs continuously, so the result regenerates itself as you edit. There is no render-and-wait step — the full, final image is alive in the viewport.

Signal in, image out

The defining move is feeding live input into the graph: audio amplitude and FFT bands, MIDI/OSC, camera and depth sensors (Kinect/RealSense), or external data streams. Parameters are mapped to these signals so geometry, particles and shaders deform in lockstep with sound and motion.

GPU shaders, particles, and the screen as canvas

Heavy looks come from GPU compute: particle systems, feedback loops, instancing and custom GLSL/shader passes. The same graph is then aimed at the venue — a dome, a façade, a stack of LED panels or scanning lasers — with projection mapping or codec pipelines (e.g. NotchLC into media servers) carrying it to the surface.

Where it comes from

Procedural roots

TouchDesigner descends from PRISMS and Houdini lineage: Derivative was founded in 2000 by Greg Hermanovic, Rob Bairos and Jarrett Smith, spun off from Side Effects' procedural toolset to make a real-time interactive animation engine. The 'always-alive, no-compile' philosophy comes straight from that procedural heritage.

Data as material, sublime as goal

A parallel strand treats pure data and the binary itself as the image. Ryoji Ikeda (b. 1965), via raster-noton and projects like 'datamatics' (2006–) and 'test pattern' (2008–), pushed flicker, barcode and 0/1 fields to the threshold of perception — the 'sublime' overload that much audioreactive work still chases.

Live tools meet the gallery

The discipline matured where software authorship and performance overlap: Robert Henke (co-founder of Ableton) built laser performance 'Lumière' (from 2013) by treating each note as a simultaneous sound and image; Notch, founded by Matt Swoboda, brought real-time motion graphics to arena stages, making 'render live, every night' the live-events default.

Artists

  • Ryoji Ikeda

    data.tron (2007); test pattern (2008–); datamatics (2006–); the transfinite (2011); data.scan (2009–)

    Converts raw data and binary into ultra-fast black-and-white flicker synced pixel-for-pixel to sound — the canonical data-as-image installation.

  • Robert Henke

    Lumière I–III (2013–2017)

    Ableton co-founder who draws with scanning lasers, composing each note as a single audiovisual event in real time.

  • onformative

    Meandering River (2018); Collide (2016); IBM FLUX (2017)

    Berlin generative-design studio (Cedric Kiefer & Julia Laub) building real-time, data-driven installations where code is the primary material.

  • Vincent Houzé

    Fluid Structure (2017); Fluid Structure 360 (2018)

    NYC-based artist running state-of-the-art GPU fluid and particle solvers in real time, so an amorphous simulated mass deforms live under audience presence.

  • Nohlab

    IN-LINE (2018–); ARIUM; ARKHE (w/ Decol)

    Istanbul studio (Candaş Şişman & Deniz Kader) generating real-time light, fluid and projection fields tuned to architecture — LED corridors, hazer holograms and GPU fluid solvers.

  • Bileam Tschepe

    River (2019); wonderful (2020)

    Berlin TouchDesigner artist/educator (‘elekktronaut’) building organic, nature-inspired audioreactive particle and feedback systems rendered live.

  • Ouchhh

    DATAGATE (2018); DATAMONOLITH_AI

    Istanbul studio turning scientific datasets into AI-generated 'data sculptures' on monumental LED forms — 'data as paint, algorithm as brush.'

  • Refik Anadol

    Machine Hallucinations (2016–); Unsupervised (MoMA, 2022)

    Coined 'AI data sculpture'; trains GANs on vast image archives to render machine 'dreams' onto walls and façades.

  • United Visual Artists (UVA)

    Volume (2006); Momentum (2014); Our Time (2016); Massive Attack live tours (2008-)

    London studio building large-scale light, sound and code installations and the live visual systems behind Massive Attack's tours.

  • Ryoichi Kurokawa

    rheo: 5 horizons (2009); syn_ (2011); octfalls (2011); ad/ab Atom (2017)

    Japanese artist composing tightly synchronised audiovisual works where granular sound and fractured imagery move as one signal.

  • Daito Manabe / Rhizomatiks

    Perfume live visuals; Pulse (electric-muscle face works); Phase Forms (2025); Fencing Visualized

    Tokyo studio pairing dancers, sensors and code to drive bodies, drones and projection in tightly synced real-time performance.

  • Tarik Barri

    Versum (2008-); live visuals for Thom Yorke, Atoms for Peace, Nicolas Jaar

    Dutch audiovisual composer who built Versum, a 3D world flown like an instrument so every object both sounds and renders live.

  • Joanie Lemercier

    Nimbes; Brume (2018); Constellations; All the Trees

    French artist drawing with projected light and fog, mapping precise geometric line-work onto surfaces, vapour and landscape.

  • 1024 Architecture

    Etienne de Crecy Cube; Euphorie-Crise-Recession trilogy; Walking Cube; Core

    Paris collective (Pier Schneider & Francois Wunschel) staging audioreactive structures, projection-mapped cubes and kinetic light installations.

  • Quayola

    Strata series; Pleasant Places; Jardins d'Été; Captives; Laocoön

    Italian artist who runs computer-vision and simulation over old-master paintings and landscapes in real time, letting the algorithm dissolve them into shifting strata and brush-vectors.

  • Memo Akten

    Simple Harmonic Motion (2011); Learning to See (2017); Forms (with Quayola, 2012)

    Turkish artist driving real-time particle and neural-network systems from live motion and sound, so the screen reorganises itself in lockstep with a performer or audience.

Frames

Ryoji Ikeda

  • test pattern [n°15] by Ryoji Ikeda
    test pattern [n°15] · Ryoji Ikeda · Source
  • test pattern [100m version] by Ryoji Ikeda
    test pattern [100m version] · Ryoji Ikeda · Source
  • test pattern [n°5] by Ryoji Ikeda
    test pattern [n°5] · Ryoji Ikeda · Source
  • data.tron (2007) by Ryoji Ikeda
    data.tron (2007) · Ryoji Ikeda · Source
  • data.scan [n°1-9] (2009–) by Ryoji Ikeda
    data.scan [n°1-9] (2009–) · Ryoji Ikeda · Source
  • the transfinite (2011) by Ryoji Ikeda
    the transfinite (2011) · Ryoji Ikeda · Source
  • test pattern [n°4] by Ryoji Ikeda
    test pattern [n°4] · Ryoji Ikeda · Source
  • datamatics [ver. 2.0] / 4'33" by Ryoji Ikeda
    datamatics [ver. 2.0] / 4'33" · Ryoji Ikeda · Source

Robert Henke

  • Lumière II by Robert Henke
    Lumière II · Robert Henke · Source
  • Lumière III by Robert Henke
    Lumière III · Robert Henke · Source
  • Lumière III (circles) by Robert Henke
    Lumière III (circles) · Robert Henke · Source
  • Lumière I by Robert Henke
    Lumière I · Robert Henke · Source

onformative

  • Meandering River (2018) by onformative
    Meandering River (2018) · onformative · Source
  • Meandering River (2018) by onformative
    Meandering River (2018) · onformative · Source
  • Meandering River (2018) by onformative
    Meandering River (2018) · onformative · Source
  • Meandering River (2018) by onformative
    Meandering River (2018) · onformative · Source
  • Meandering River (2018) by onformative
    Meandering River (2018) · onformative · Source
  • Collide (2016) by onformative
    Collide (2016) · onformative · Source
  • Collide (2016) by onformative
    Collide (2016) · onformative · Source
  • Collide (2016) by onformative
    Collide (2016) · onformative · Source
  • IBM FLUX (2017) by onformative
    IBM FLUX (2017) · onformative · Source

Vincent Houzé

  • Fluid Structure 360 (2018) by Vincent Houzé
    Fluid Structure 360 (2018) · Vincent Houzé · Source
  • Fluid Structure 360 (2018) by Vincent Houzé
    Fluid Structure 360 (2018) · Vincent Houzé · Source
  • Fluid Structure 360 (2018) by Vincent Houzé
    Fluid Structure 360 (2018) · Vincent Houzé · Source
  • Fluid Structure 360 (2018) by Vincent Houzé
    Fluid Structure 360 (2018) · Vincent Houzé · Source
  • Fluid Structure (2017) by Vincent Houzé
    Fluid Structure (2017) · Vincent Houzé · Source
  • Fluid Structure (2017) by Vincent Houzé
    Fluid Structure (2017) · Vincent Houzé · Source
  • Fluid Structure (2017) by Vincent Houzé
    Fluid Structure (2017) · Vincent Houzé · Source
  • Fluid Structure (2017) by Vincent Houzé
    Fluid Structure (2017) · Vincent Houzé · Source

Nohlab

Bileam Tschepe

  • River (2019) by Bileam Tschepe
    River (2019) · Bileam Tschepe · Source
  • wonderful (2020) by Bileam Tschepe
    wonderful (2020) · Bileam Tschepe · Source

Techniques

  • audioreactive sound-to-visual mapping
  • real-time per-frame GPU render
  • projection mapping
  • FFT / audio-feature driven parameters
  • node-based signal patching
  • generative rule systems
  • data-driven visualisation
  • multi-channel spatial AV
  • LiDAR / point-cloud capture
  • sensor-responsive interaction
  • kinetic light / DMX
  • GPU compute particle systems
  • optical-flow / computer-vision input
  • feedback-loop accumulation buffers
  • instancing & point-cloud rendering
  • NDI / Spout texture sharing

Tools

  • TouchDesigner
  • Notch
  • Max/MSP
  • vvvv
  • Unreal Engine
  • GLSL
  • Ableton Live
  • Resolume
  • Kinect / depth sensors
  • MadMapper
  • Spout / NDI
  • Houdini (engine / asset prep)
  • Processing / openFrameworks
  • ONNX / StreamDiffusion (real-time AI)

What to take from it

Map one strong signal, not ten

The most legible audioreactive work usually drives motion from a single clear control — overall loudness or one frequency band — rather than wiring everything to everything. Pick the signal that matters and let it dominate.

Constraint breeds the look

Henke gets complexity from 'very basic geometric shapes' because lasers force it; Ikeda lives in black/white and 0/1. Choosing a deliberately narrow vocabulary (one color space, one primitive) reads as a signature, not a limitation.

Design for the surface, not the rectangle

These works are built knowing the final canvas is a dome, façade or LED maze. Compose for the real geometry and viewing distance from the start — the same graph fails on a flat 16:9 and sings wrapped around a building.

Further reading