WEEK 14
Craft and Code x Gesture
Linking it back to craft and code
This week I circled back to the craft and code intersection that first drew me in, but with a clearer question. I was clear on what gesture is in my research, but I had to make sure that I was confident in defining where craft and code could come in.
Look back at readings
Going deeper into craft and gesture readings could help me define where craft could come in now that I am looking at embodied movement as well.
Craft in my research
How can I define craft, could it be materiality of it?
Code
Can cold hold traces of embodied movement?
[ Craft and gesture readings ]
Re-reading craft through behaviour
What I learned from craft and gesture texts
I realised I had been borrowing values of craft without fully unpacking them. When I stripped it back to behaviours, a deeper definition appeared. For me, craft involves:
Material intimacy where I feel and respond to resistance.
Process over outcome where meaning sits inside the making, not just the final
image.
Slowness and attention where micro adjustments matter.
Risk because I cannot fully control the result.
Embodied knowledge where decisions often appear in the hand before the mind explains
them.
Sennett, Pye, Ingold, Pallasmaa and Manning each added weight to this picture. Sennett writes about repetition and tacit knowledge as ways of thinking through the hand. Pye grounds variation and hesitation as signs of judgement rather than flaws. Ingold reframes making as a temporal correspondence between maker and material. Pallasmaa positions the hand as a cognitive instrument, and Manning’s minor gesture helped me see micro variation as generative rather than noise.
Together, they made craft feel less like an aesthetic and more like a way of thinking through materials and movement.
[ Digital interaction readings ]
Embodied interaction and loss through digitisation
What I learned from HCI and interaction texts
Embodiment is already present in HCI, but these readings showed me how gesture is compressed into data and how much expression gets flattened when it becomes computational.
Paul Dourish — Embodied interaction
Gave me language for how digital systems already rely on skilled physical practice. Gesture, posture
and
rhythm are acknowledged, but mostly treated as input. Little attention is paid to how those embodied
qualities travel through to the output.
Jacob et al. — Reality based interaction
Showed how interfaces lean on existing human skills like grasping, gesturing and navigating. To make
that
legible to a system, gesture must be compressed into measurable values. That reduction sits at the
heart of
my project.
Wensveen & Djajadiningrat — Interaction Frogger
Brought in rhythm, flow and emotion as part of gesture. They also showed how expression is simplified
again
when it becomes computational. This helped me see more clearly what tends to get flattened when
gesture
moves into code.
[ Computational design readings ]
Computation as material and trace
What I learned from computational design texts
These readings helped me see computation as part of the same chain as gesture and material, and clarified how digital systems leave their own kind of trace once expressed in physical form.
Matt Ratto — Critical making
Frames computation as part of a hybrid workflow where thinking happens through both materials and
technical
systems. This helped me place gesture and code along the same chain instead of treating them as
separate
worlds.
Cecilia Vallgårda — Computational composites
Argues that computation behaves like a material when it is coupled with physical substrates, and only
becomes
visible when expressed through matter. This aligns closely with my plan to express gesture patterns
through
foil traces, where the foil surface shows computational decisions.
Reas & McWilliams / Rosa Menkman
Form and Code breaks pattern generation into repetition, transformation and rule-based
behaviour,
confirming that gesture data can act as parameters inside generative systems. Menkman’s writing on
glitch
treats error as digital trace, revealing internal processes the way tool marks reveal manual labour.
Together,
they helped me avoid a simplistic claim that digital systems leave no trace at all.
[ Research gap and trace ]
Code is craft, craft is code
Shared foundations, different traces
My lecturer’s phrase “code is craft, craft is code” was not meant literally. It pointed to the overlap. Both involve skill, iteration and decision making. They differ in how they reveal process.
In craft, the process tends to stay visible on the surface. You can see hesitation, corrections and pressure as marks. In code, process often disappears. The debugging, rewriting and small decisions are hidden once the final output is rendered.
This led me back to the gap from Week 13, but through a slightly different lens. I started to phrase it as:
How can the hidden, temporal qualities of embodied action be made visible across digital and material systems?
The question is less about forcing “craftiness” into code, and more about following gesture, rhythm, hesitation and pacing as they move across translations, noticing what stays visible and what drops out.
[ Testing directions ]
Two paths, one chosen
Code to craft instructions vs gesture pipeline
I explored two possible directions. In Direction A, code generates craft instructions. Patterns are produced first, and craft follows the system. It was structurally neat, but it placed the body after the system. Embodied decision making became secondary. It felt like a path I could justify on paper, but not one that matched my instincts.
Direction B followed the sequence I had been circling around for weeks: gesture to code to craft. Here, the body initiates, code interprets and craft materialises. This direction holds the qualities I care about most: slowness, rhythm, risk, translation and trace. It also matches the pipeline defined in Week 13.
[ Framework ]
Formalising the pipeline
Gesture to data to pattern to craft
From there I formalised the framework into three translation stages. Each one handles the same movement in a different way, and adds its own kind of interpretation.
Stage 1: Gesture capture
What qualities of movement can I record in the first place. Here I focus on speed, hesitation, pauses
and
proximity. This stage treats gesture as embodied data, before any computation starts reshaping it.
Stage 2: Computational patterning
How code reinterprets those qualities. Movement becomes parameters such as line density, jitter,
thickness
and distortion. This is where the system simplifies, amplifies or reorganises parts of the original
gesture.
Stage 3: Material craft
How a physical material reintroduces resistance and trace. Embossing depth, tension, puncture marks
and
surface irregularities carry the final record of the gesture as it passes through the whole pipeline.
The goal is not perfect preservation. The point is to study how gestures transform at each stage and why: what survives, what gets flattened, and what new characteristics appear once the gesture becomes pattern and then material.
[ Experiment 8: Studying ways to capture gesture ]
Understanding what the system responds
How the gesture sketches evolved
Before building the full gesture studio, I made two smaller p5.js sketches to understand what kind of movement the system could meaningfully read. These early sketches helped me see what was useful, what was noise, and where the direction should go.
Sketch 01
Finger tracking test
The first sketch simply tracked my finger through the webcam. It showed me how the system followed motion,
where it lagged, and how the line shifted when my hand rhythm changed. This made it clear that the system
was already reading timing, drift and small corrections in the gesture.
Sketch 02
Three instructed gestures
The second sketch introduced three instructed gestures. I wanted to see how different forms behaved when
they were performed by the same person. Even with fixed instructions, each gesture had its own rhythm and
tension. The differences made me realise that too many gesture types would make comparison impossible.
Final direction
Continuous loop, 10 seconds
This pushed me toward the final version of the gesture studio: one continuous loop performed within a ten
second countdown. Simple, repeatable and structured enough to reveal variation between people. This became
the foundation for the Loop Trace system in Week 15.
[ Planning next experiments ]
Prototype 2 for the catalogue
Grouping work into two experiment families
By the end of Week 14, I grouped the next phase of work into two families that will form Prototype 2 of the catalogue of making.
Experiment family 1: Gesture capture and patterning (p5.js)
Gesture studio tests, loop based capture and behaviour mapping for speed, pauses and proximity.
Experiment family 2: Material translation
Foil embossing and tracing with a stylus or improvised tools, so gesture becomes both a digital pattern
and a physical trace.
These two families give me a clean path forward: capture movement, reinterpret it computationally, then translate it into material trace.
[ What shifted ]
A specific research focus
I’m no longer juggling loose ideas about craft and code. The project now centres on following how a movement changes as it passes through different stages. The readings helped me see an actual gap, instead of me trying to force one.
[ What I am following ]
Gesture as embodied movement
I am treating embodied movement as the link that holds the whole system together. The way someone moves, the rhythm, the pauses, the pressure and the small shifts, becomes the anchor for every decision. This keeps the project grounded and stops me from drifting back into vague themes.
[ How I will move ]
What the movement becomes
I need to track what happens as a movement gets recorded, processed and worked back into a physical form. If I can follow those changes honestly, the story of the project will reveal itself.