Designing things in times of immersive AI
A write-up of my talk at the Design Thinkers Academy, 28 May 2026
I was invited to speak at the Design Thinkers Academy about where Cities of Things sits today, and I used the occasion to do something I’d been circling for a while: take the temperature of the whole project. What does it mean now that generative AI is everywhere — and what happens when that intelligence stops living on our screens and moves into the things and spaces around us? That shift is what I’ve come to call the agentic turn, and it’s the thread running through everything below.
Where this comes from
Cities of Things began in 2017–2018 as a research initiative and design lab at TU Delft, set up by Professor Elisa Giaccardi and myself as visiting professor. We weren’t interested in the smart city as a story about data collection, sensors, and dashboards. We were interested in a different question: what does it mean to live together with intelligent things, objects, and systems? In 2021 the work became an independent foundation — the Cities of Things expertise center for physical and embodied AI — which is now my main focus.
The premise has stayed remarkably stable. We treat things as a kind of citizen. Not just as devices that sense and collect data in real time, but as data-enabled artefacts that can act proactively and, crucially, begin to behave socially. As designers we found that an irresistible question: if a thing starts to behave like a social entity, how do we relate to it?
Five dilemmas
Before any of the prototyping, we mapped the tensions that this raises. Five dilemmas have guided us ever since, each a spectrum rather than a yes/no:
- Responsibility — private versus public
- Priority — does the human or the system come first?
- Relationship — is the thing a tool or a social partner?
- Adaptation — does the thing adapt to the human, or sometimes the other way around?
- Delegation — do we hand over a single task, or everything?
You can see these dilemmas play out in the wild already. We don’t really have delivery bots on Dutch streets yet — only a few experiments — but in the US and China they’re common, and the TikTok and Reels footage of them is telling. People form an emotional connection, or they get angry, or — most interesting to me — they help. When one of these things gets stuck on the pavement and a passer-by frees it, and the thing responds well to that help, there’s a small moment of joy there. That tells you something about the relationship we’re building.
Civic prototyping: the Wijkbot
The way we study this is not by sketching it out but by building it. Credit here goes to Tomasz Jaśkiewicz, lector of civic prototyping at Rotterdam University of Applied Sciences, who came up with a beautifully simple hack: a hoverboard platform you can stand on, turn into a remote-controlled base, and quickly build a robot on top of. That became the Wijkbot (Hoodbot).
We took it everywhere — hackathons, half-day workshops, and out onto the street, including the big market in the Afrikaanderwijk, where we ran a project for a year and a half. The method is wonderfully direct: you stand at a distance, you mimic the robot’s behaviour, and you watch how people relate to it in real time. We did this with students, with professionals, in Rotterdam and as far as Hamburg, often together with the systemic co-design network, and we developed it into a workshop format that digs into the deeper layers of people’s motivations.
The driver underneath all of it: not technology built for us by big tech, but technology we can build ourselves, as citizens.
From things to systems
In the last couple of years the questions have grown outward — from the single thing to the system. If a community builds a commons-based, bottom-up form of governance around both human and non-human actors, what economic models start to appear, and how do we design for them? This is the strand that increasingly defines the work.
The ChatGPT moment
When we started, we understood machine learning and could reason about these things, but the technology wasn’t there yet. The transformer arrived around 2018, but it became legible to everyone only with the release of ChatGPT in November 2022. After that, everyone understood what generative AI could mean — for conversation, text, images, search.
If you sketch the arc, you move from textual and conversational, through images, search and reasoning, into multimodal combinations — and then into the two stages that interest me most: agentic AI, and physical AI.
Agentic is the threshold. An agent is something you delegate a task to. Something becomes agentic when it starts to act proactively — taking actions you didn’t explicitly ask for, because its rules and knowledge lead it to. That’s the moment we start building genuinely new relationships with things, especially once those things are embodied in our environment.
Co-performance, and the sweet spot
I keep coming back to the concept of co-performance, introduced by Kuijer and Giaccardi in 2018. When we live alongside AI on these surfaces, we have a choice of strategies: delegate everything; treat the AI as a tool that must follow us; let it take over the human role; and so on. None of the extremes is interesting. The interesting work is finding the sweet spot in between — what does AI do best, what do humans do best, and how do we choreograph a co-performance that may itself shift over time?
Immersive AI — and its shadow
That’s the optimistic reading. There’s a shadow too. I describe the emerging condition as immersive AI: intelligence that is everywhere and connected to the things in our environment. But we already live in a state of mind addicted to small chunks of media, needing a continuous drip of the new. So I sketched a small model — partly prompted by a group of Unilever marketers asking about exactly this.
Put three forces together — immediacy culture, agentic AI, and the physical environment — and they form a super-stimulus effect: a flywheel of dopamine dependency and environmental triggers, personalised, which is a potentially dangerous mix. We should be aware that we risk building a world where we live inside immediacy and AI black boxes, using intelligence built into everything without understanding how it came about.
The way I frame the whole landscape: physical AI is the fixed environment turning computational — a lamp post, a bin, anything with an algorithm in it becomes part of a “mirror world” of opportunities. Embodied AI is the moving things, the city bots, now carrying that same generative capability. Where they meet — when we interact with embodied things inside a physical-AI context — that’s immersive AI. New affordances, new services, and new consequences to think through.
Four conditions of interplay
How we experience this depends on the kind of interaction we have. There are four conditions, along two axes — one-time versus ongoing, and a single thing versus a system of things:
A brief brush with one object is an encounter; a brush with a coordinated system is a passage. An ongoing bond with one thing is companionship; living inside the system is inhabitation.
The manifesto: a seven-year reflection
Around last November, seven years in, I decided to reflect on Cities of Things itself — what is its state in 2026? So I interviewed around thirty people: academics, designers, practitioners, former students, anyone with an opinion and a role. The conclusions are gathered in a new manifesto, and a few threads from those conversations shaped the thinking that follows.
One interviewee talked about emerging milieus of intelligence — systems built from the bottom up — and argued that design should stop focusing only on the object and attend to the supporting environment. Don’t start by designing a nice thing and its insides; start with what it touches and what surrounds it.
This sent me back to my own unfinished 2018 research on predictive relations. Our interplay with a thing is shaped by the mental model we hold of it — we expect a certain response. That response is shaped by the thing’s intelligence, and by our own history and profile, which we can understand. But there’s an extra layer: it’s also shaped by every interaction every other person has had with that thing, anywhere in the world, at any time, because these systems are continuously connected. What does that do to the relationship? That’s the open question I’d parked, and immersive AI reopens it.
And it deepens further with agency. Is an object a single agent? Or, as Simone Rebaudengo put it in our conversation, is an object an assembly of different agents working at different tasks?
A story from the (near) future: The Grid
To make this tangible, I told a speculative story set in 2033.
Two pressures shape its world. An energy crisis has made us far more deliberate about how we use and organise energy. And a loss of trust in the centralised internet has pushed people toward bottom-up mesh networks, node to node. Imagine a phone company — say, Fairphone — that doesn’t just make a phone but a module you can drop into anything to spin up a mesh easily. Call it Fairmesh, a service.
Meet Lena. She enters the building where she lives and immediately feels its state — the building gives her feedback through the device she wears. Inside, all the things are connected in a mesh; and the buildings themselves mesh together across the city, a hypermesh layered on top.
What I find compelling here is the counter-movement to centralised intelligence. Alongside the big models — the OpenAIs and Geminis — there’s a push toward small, dedicated language models. The heat pump has its own tiny LLM, tuned only for its task; so does the solar panel. They’re conversational and intelligent within their narrow remit, and through the mesh they begin to talk to each other, generating new knowledge and behaviour in the space between them.
The residents organise too. In the story they’re a cooperative — De Warren — that holds a quarterly assembly to decide their shared goals: use less energy this quarter, or give away the surplus they’ve saved; who governs whom; what the rules are. And into that assembly the things contribute as well, each offering its own overview of what it has done.
None of this is wholly invented. In Amsterdam, the Schoonschip project — a community of floating houses connected to the grid through a single plug — already runs its own internal energy economy: batteries, solar panels, residents buying and selling surplus among themselves and with the outside. The Grid simply asks what happens when those balancing systems stop being merely algorithmic and start becoming intelligent. (I even fed the scenario text into NotebookLM and got back a two-minute film — and it captured nicely how, in this world, the things become genuine players in the equation. Who rules whom?)
Ten design considerations — the Manifesto 2026
This brings me to the heart of it: ten design considerations for working in immersive AI. I prefer “considerations” to “commandments.” They fall into four rough groups.
Understand and be honest — the first five.
- Design for Legibility of Agency (LA) — make the intelligence readable.
- Provide Predictive Symmetry (PS) — arrange a relationship between human and thing that is genuinely trustworthy.
- Emerge Relational Integrity (RI) — let the relationship hold together honestly.
- Keep Peripheral Presence (PP) — use the intelligence only when it’s actually needed.
- Reveil Assemblage Transparency (AT) — when multiple agents or stakeholders coordinate around a person, that coordination should be perceptible and its boundaries knowable. Be honest about it to the one human in the loop.
Friction leads to more engagement — the next two.
- Appreciate Friction in Interactions (FI) — build friction in, deliberately. Don’t make things so smooth that people stop thinking.
- Incorporate Temporal Honesty (TH) — a thing with vast knowledge can leap to conclusions you don’t follow and didn’t want. Respect the difference between a new encounter and an established relation; don’t simulate relational depth that hasn’t been earned; let people walk the path of a decision themselves.
Embed in new forms of governance — the commons layer.
- Leverage Democratic Precedence (DP) — communities should define the kind of environment they want before agentic systems arrive. Governance infrastructure must precede technological infrastructure.
- Ensure Distributed Sovereignty (DS) — keep control distributed rather than concentrated.
Be an open platform — the last one, which arches over the rest.
- Open up to Build Back (BB) — people must be able to introduce their own agentic things, modify existing assemblages, and keep the possibility space open against closure. Don’t hand people a finished system to use; give them things they can build and rebuild. “Vibe coding” is a hot term, but the underlying principle — we can make this ourselves, fairly easily — is a powerful one.
These are still in development; there’s a card set to think with, and an open discussion about whether to fold in the resource and societal impact of these systems too. The current set is mostly about how we design once we’re in this situation — how to make things that are genuine, good and honest.
What remains human
My personal conviction is that a real shift is underway in how AI is embedded in the things we use. Since ChatGPT we’ve treated it as a tool we reach for. But what happens when AI is embedded in everything, always there, and able to become agentic — to initiate interactions with us? That always-on intelligence is a new form of relationship with our environment, and it isn’t only technical or functional. It’s social. It’s about what just happened, socially, between us and the thing.
We have always designed for humans — to serve us, to be usable. The extra quest now is to design not only what works best for us, but what keeps us human — what prevents us from becoming less so. With all the convenience these systems offer, if we’re not aware of how much of the human part of ourselves we’re delegating away, we lose some humanity in the bargain.
I’ll close where one interviewee pointed me, to Donna Haraway. I’m critical precisely because I’m optimistic. I can only be optimistic about the usefulness of all this if I’m also willing to be critical and look hard at where the dangers are.
Cties of Things is an independent expertise center for physical and embodied AI. More on the manifesto and the research behind this talk at citiesofthings.nl. — Iskander Smit, iskander@citiesofthings.nl
This write-up is written by Claude Opus 4.8, based on the transcription of me presenting the deck, combined with the PDF of the slidedeck. Let me know if you like the whole story 🙂
