AI / Latent-Diffusion as a Medium (Latent Walks)

Treating the trained model as an instrument — motion drawn by moving through its latent space, not by typing prompts.

Ruled out

Included to keep the atlas complete. This is the one school ruled out as a method for our own film — generative/latent output reads as the "AI-slop" look we are deliberately working against; we study it to know what to avoid, not to use.

What it is

A code-medium practice where image is generated by a neural network (GAN, StyleGAN, diffusion) and motion comes from traversing the model's high-dimensional latent space frame by frame. The artist builds or fine-tunes the model on a chosen dataset, then choreographs a path — a 'latent walk' — through that space; each sampled point is decoded into a frame, so continuous interpolation reads as continuous movement. The aesthetic is determined by the training corpus and the trajectory, making the model an authored instrument rather than a vending machine for clips.

How it is made

Build the latent space (train / fine-tune)

Curate a dataset and train a generator — GAN, StyleGAN2, or fine-tuned diffusion — so the model's latent space encodes the look you want. The corpus is the palette: ocean and flame footage, 17th-century portraits, 10,000 hand-labelled tulip photos. Whatever you feed it is all it can ever see.

Walk the space frame by frame

Pick start and end latent vectors and interpolate between them; decode each intermediate point to a frame. Smooth, slow paths give coherent morphs; jumps give convulsive cuts. The choreography of the trajectory — speed, looping, which axes you move along — is the directing.

Drive or feed the walk

Make the path responsive: live camera into image-to-image inference (real-time reaction), a self-referential feedback loop (the model re-reading its own output, never repeating), or an external data stream (a market price, sensor data) steering position in latent space.

Where it comes from

From GAN portraits to autonomous machines

Early GAN/StyleGAN work (2017-2019) established the morphing-portrait idiom; Klingemann's self-feeding loop pushed it to a machine that generates an endless, non-repeating stream with no pre-set choreography — the latent walk as a living system.

The model as a mirror, not a camera

Akten's 'Learning to See' reframed inference itself as the subject: a network can only reconstruct the world through the dataset it was trained on. This turned latent generation into a statement about bias and perception, not just spectacle.

Foregrounding the human and the data

Ridler and Eaton insisted the dataset is an authored object — hand-labelled tulips, an artist's own figure-photography archive — pushing back on AI 'objectivity' and grounding latent motion in deliberate, hand-built corpora and custom loss functions.

Artists

  • Mike Tyka

    Portraits of Imaginary People (2017-); Us and Them (2018)

    Co-founded Google's Artists and Machine Intelligence program and, before the diffusion craze, trained stacked GANs on ~20,000 faces to generate portraits of people who never existed.

  • Alexander Mordvintsev

    DeepDream (2015)

    Google engineer who in 2015 froze a trained network's weights and let the image change toward whatever the neurons recognised, amplifying learned features into DeepDream's hallucinatory pareidolia.

  • Robbie Barrat

    AI Generated Nude Portrait series (2018); art-DCGAN (open-source)

    Trained a GAN on thousands of WikiArt nudes and released the art-DCGAN code openly, embracing the network's failures — figures dissolving into fleshy blobs — as the work itself.

  • Helena Sarin

    Neural Bricolage (2018-); #bashoGAN

    Software engineer and artist who trains GANs on her own small, hand-built datasets of everyday objects, arguing against the compute arms race of giant pre-trained models.

  • Gene Kogan

    ml4a / Machine Learning for Artists (2015-2021)

    Artist and programmer who built ml4a, a free book, tutorials and tools, and gave over 200 workshops teaching neural networks and generative art to non-coders.

  • Holly Herndon & Mat Dryhurst

    Spawn (Proto, 2019); Holly+ (2021)

    Work mainly in latent audio rather than image: trained a neural model on Herndon's own voice so anyone can sing through her vocal likeness, governed by a DAO.

  • Mario Klingemann

    Memories of Passersby I (2018); Neural Glitch (2018)

    Pioneer of the autonomous latent walk: GAN feedback loops that drift endlessly through portrait space, and Neural Glitch works where he permutes a trained model's own weights to make it misread its inputs.

  • Memo Akten

    Learning to See (2017–); Deep Meditations (2018)

    Real-time image-to-image inference where a network reconstructs a live camera feed only through the dataset it was trained on — waves, fire, clouds — making the model a mirror that can see nothing it has not already learned.

  • Refik Anadol

    Machine Hallucinations (2019–); Nature Dreams (2020)

    Trains StyleGAN on millions of images, then steers a custom latent-space browser through the model in real time to produce fluid, ever-morphing data-paintings.

  • Anna Ridler

    Mosaic Virus / Myriad (Tulips) (2018–2019)

    GAN trained on 10,000 of her own hand-labelled tulip photographs; the bloom's stripes — its position in latent space — are driven by the live price of bitcoin, insisting the dataset is an authored object.

  • Scott Eaton

    Entangled II (2019)

    Feeds high-speed footage of paint dropped in water through his custom figure-trained 'BODIES' network so human anatomy continually rebirths out of fluid via learned form.

  • Sofia Crespo

    Neural Zoo (2018–2022); Artificial Natural History (2020)

    Trains GANs on large image sets of organisms, then interpolates through the latent space to generate impossible 'specimens' — recombinations of learned texture and form that no real creature ever wore.

  • Jake Elwes

    Latent Space (2017); CUSP (2019); The Zizi Show (2020)

    Treats the trained generator as an instrument — walking ImageNet latent space, morphing wading birds at the marsh edge, and 'queering the dataset' so a StyleGAN's drag deepfakes visibly break down where the model's bias shows.

  • Paul Trillo

    Washed Out — 'The Hardest Part' (2024), first commissioned Sora music video

    Directed the first officially commissioned music video made with OpenAI Sora — ~700 generated clips cut to 55, built as one continuous forward zoom through a couple's memory, text-to-video only with no still-image conditioning.

  • Cristóbal Valenzuela (Runway)

    Runway Gen-1 (Feb 2023), Gen-2 (Mar/Jun 2023), Gen-3 Alpha (Jun 2024), Gen-4 (Mar 2025)

    Co-founded Runway, which co-released Stable Diffusion (2022) and then shipped the Gen-series video models that moved the field from latent walks to prompt-and-reference text/image-to-video.

  • Robin Rombach (CompVis)

    High-Resolution Image Synthesis with Latent Diffusion Models (CVPR 2022); Stable Diffusion (2022)

    Lead author of the latent-diffusion paper that moved diffusion into a compressed latent space — the technique this whole school is named after — and a core author of Stable Diffusion.

Frames

Mario Klingemann

  • Neural Glitch (2018) by Mario Klingemann
    Neural Glitch (2018) · Mario Klingemann · Source
  • Neural Glitch (2018) by Mario Klingemann
    Neural Glitch (2018) · Mario Klingemann · Source
  • Untitled (GAN figure / Microscopy) by Mario Klingemann
    Untitled (GAN figure / Microscopy) · Mario Klingemann · Source
  • Memories of Passersby I (2018) · Mario Klingemann · Source

Memo Akten

  • Learning to See: Gloomy Sunday (2017) by Memo Akten
    Learning to See: Gloomy Sunday (2017) · Memo Akten · Source
  • Learning to See: Gloomy Sunday (2017) by Memo Akten
    Learning to See: Gloomy Sunday (2017) · Memo Akten · Source
  • Learning to See: waves (2017–) by Memo Akten
    Learning to See: waves (2017–) · Memo Akten · Source
  • Learning to See: Gloomy Sunday (2017) by Memo Akten
    Learning to See: Gloomy Sunday (2017) · Memo Akten · Source
  • Learning to See: Gloomy Sunday (2017) by Memo Akten
    Learning to See: Gloomy Sunday (2017) · Memo Akten · Source
  • Learning to See: Gloomy Sunday (2017) by Memo Akten
    Learning to See: Gloomy Sunday (2017) · Memo Akten · Source
  • Learning to See: Gloomy Sunday (2017) by Memo Akten
    Learning to See: Gloomy Sunday (2017) · Memo Akten · Source
  • Learning to See: Waves (2017–) by Memo Akten
    Learning to See: Waves (2017–) · Memo Akten · Source

Refik Anadol

  • Machine Hallucinations — Nature Dreams (2021) by Refik Anadol
    Machine Hallucinations — Nature Dreams (2021) · Refik Anadol · Source
  • Machine Hallucinations — Nature Dreams (2020) by Refik Anadol
    Machine Hallucinations — Nature Dreams (2020) · Refik Anadol · Source
  • Machine Hallucinations — Nature Dreams (2020) by Refik Anadol
    Machine Hallucinations — Nature Dreams (2020) · Refik Anadol · Source
  • Machine Hallucinations — Nature Dreams (2020) by Refik Anadol
    Machine Hallucinations — Nature Dreams (2020) · Refik Anadol · Source
  • Machine Hallucinations — Nature Dreams (2020) by Refik Anadol
    Machine Hallucinations — Nature Dreams (2020) · Refik Anadol · Source

Anna Ridler

  • Mosaic Virus (2018) by Anna Ridler
    Mosaic Virus (2018) · Anna Ridler · Source
  • Mosaic Virus (2019) by Anna Ridler
    Mosaic Virus (2019) · Anna Ridler · Source
  • Mosaic Virus (2019) by Anna Ridler
    Mosaic Virus (2019) · Anna Ridler · Source

Scott Eaton

  • Entangled II (2019) by Scott Eaton
    Entangled II (2019) · Scott Eaton · Source
  • Entangled II (2019) by Scott Eaton
    Entangled II (2019) · Scott Eaton · Source
  • Entangled II (2019) by Scott Eaton
    Entangled II (2019) · Scott Eaton · Source
  • Entangled II (2019) by Scott Eaton
    Entangled II (2019) · Scott Eaton · Source

Sofia Crespo

  • Neural Zoo — trinity (2018–2022) by Sofia Crespo
    Neural Zoo — trinity (2018–2022) · Sofia Crespo · Source
  • Neural Zoo — chimera (2018–2022) by Sofia Crespo
    Neural Zoo — chimera (2018–2022) · Sofia Crespo · Source
  • Neural Zoo — micro beauty (2018–2022) by Sofia Crespo
    Neural Zoo — micro beauty (2018–2022) · Sofia Crespo · Source
  • Neural Zoo — the lovers (2018–2022) by Sofia Crespo
    Neural Zoo — the lovers (2018–2022) · Sofia Crespo · Source
  • Artificial Natural History — coral (2020) by Sofia Crespo
    Artificial Natural History — coral (2020) · Sofia Crespo · Source
  • Artificial Natural History — butterflies (2020) by Sofia Crespo
    Artificial Natural History — butterflies (2020) · Sofia Crespo · Source
  • Artificial Natural History — fish (2020) by Sofia Crespo
    Artificial Natural History — fish (2020) · Sofia Crespo · Source
  • Artificial Natural History — bird (2020) by Sofia Crespo
    Artificial Natural History — bird (2020) · Sofia Crespo · Source

Jake Elwes

  • Latent Space (2017) by Jake Elwes
    Latent Space (2017) · Jake Elwes · Source
  • CUSP (2019) by Jake Elwes
    CUSP (2019) · Jake Elwes · Source
  • CUSP (2019) by Jake Elwes
    CUSP (2019) · Jake Elwes · Source
  • CUSP (2019) by Jake Elwes
    CUSP (2019) · Jake Elwes · Source

Techniques

  • GAN training on custom datasets
  • latent-space interpolation & latent walks
  • diffusion text-to-image generation
  • text-to-video & image-to-video diffusion
  • DeepDream feature amplification
  • neural style transfer
  • prompt-conditioned generation
  • ControlNet & conditioning
  • temporal-coherence tricks for video
  • model fine-tuning & LoRA
  • img2img loopback warp (Deforum 2D/3D keyframed camera)
  • AnimateDiff motion modules
  • SLERP latent interpolation for smooth walks
  • outpainting / infinite-zoom

Tools

  • StyleGAN
  • RunwayML
  • Stable Diffusion
  • ComfyUI
  • Midjourney
  • Sora / Veo / Kling
  • Disco Diffusion
  • TouchDesigner + StreamDiffusion
  • custom PyTorch
  • Deforum
  • AnimateDiff
  • Pika
  • Luma Dream Machine
  • Kaiber
  • Runway Gen-1/2/3/4

What to take from it

The dataset is the look

Before any motion, decide the corpus. A coherent, narrow, hand-curated dataset (one subject, consistent framing) yields legible morphs; a noisy one yields mush. Treat collecting and labelling as the primary creative act.

Choreograph the path, don't just sample

Random latent jumps look like a screensaver. Author the trajectory: slow easing, looped circuits, holding on stable attractors, deliberate collapse into abstraction. Motion quality lives in the path, not the model.

Bind the walk to a signal

The most alive pieces tie latent position to something external — a live camera, a feedback loop, a data feed. Mapping a real-world variable onto a latent axis turns a pretty morph into a system that means something.

Further reading

Read the AI Cinematography guides