If Memory Serves
The Promise and Perils of AI “Time Travel”
I’m standing in my childhood bedroom. My fingers trace the edges of pristine Lego sets, their plastic sheen untouched by the decades that have passed. A plush Ewok, long lost to the cruel culling of my move to college, now sits defiantly on my bed like something that’s come back from the dead. Everything is just as it was on a June day in 1986.
But this isn’t 1986, and I’m not really here—at least, not physically. I’m immersed in a VR reconstruction of my childhood bedroom conjured from a few old photographs and the near-miraculous capabilities of generative AI.
The experience is profoundly weird. It’s more than just seeing or remembering—there’s something visceral about stepping into the past like this.
I’m writing this post to explore why I think “virtual memories” like the one I created could be a big deal soon and why their potential feels, at times, both thrilling and dangerous. Let’s dive in.
What are Virtual Memories?
I’m using the term “virtual memories” to describe three-dimensional digital models assembled from photos with the help of generative AI. These models can be viewed on a 2D screen like your smartphone’s or “visited” in a VR headset like the Meta Quest.
Today, virtual memories can be created with a bit of work and a few different apps. Anyone can experiment with it. If you’re working with old photos, the process is imperfect and requires some tinkering. But I think those wrinkles will be ironed-out before long.
Here’s an overview of recent developments in the field:
Apple’s Spatial Photos: Announced at WWDC 2024, this upcoming feature for the Vision Pro headset will convert 2D photos into 3D scenes. While not fully navigable virtual environments, spatial photos will offer an immersive viewing experience like 3D movies in theaters.
Neural Radiance Fields (NeRF): Over the past two years, NVIDIA, a company increasingly recognized beyond gaming circles, has released software capable of constructing navigable 3D environments and objects from a small set of photos. (The underlying technology was developed by Google and academimc researchers.) This technology needs a few reference images to work from, I used a version of it to create my virtual trip into the 1980s.
Meta’s Generative 3D Models: In the summer of 2024, Meta showcased technology to create 3D models based on text descriptions. This advancement could potentially fill in details missing from the original source material, such as photographs, when reconstructing virtual environments.
These technologies are arrows all pointed in the same direction: a future where creating immersive, explorable versions of past experiences from limited photos and videos will be easy.
Why Virtual Memories Are Hopeful
Experts have found that visiting the past in VR can help people struggling with dementia. Although it might sound futuristic, this kind of treatment fits into a broader and older intervention called “reminiscence therapy.” Before VR headsets existed, this usually involved having patients (especially older people or those with conditions like dementia) talk about their past experiences, and therapists would use photos or familiar objects to help trigger those memories.
A more expensive option under development across Europe and the US are “Dementia Villages” designed to allow residents to live in recreations of previous time periods. The Conversation recently published a fascinating writeup on this. (For an example in the US, see Glenner Town Square in San Diego.)
Recently though, researchers have been exploring the use of VR in this context. For instance, a UK startup, Virtue, created an app called LookBack VR that lets people experience 360-degree scenes from the past. It achieves this using virtual 3D models of past locations combined with “filmic recreations” recorded with a special camera. According to Wired, LookBack is really helping some dementia patients connect with positive memories.
A VR film series called “Wayback” – again based in the UK - recreates historical moments in VR to help people suffering from dementia. Their first film depicted a street party for the Queen’s coronation in 1953. (I can only imagine how much fun it was for the actors who got to participate in recreating this scene.) Again, reports show patients laughing more, being more engaged, and seeming happier during and after these sessions. There’s a touching film about this on their website.
These examples make me feel hopeful. Technology can sometimes isolate us, but here it is, potentially bringing people together and helping them reconnect with their memories.
And, at a glance—admittedly a non-expert one—it seems like AI-generated “virtual memories” based on personal photographs might offer a useful extension of these approaches. In fact, VR memories could be even better because unlike current VR films and dementia villages, this experience is autobiographical - it’s based on your photos and your life. As Kellie Morrissey and James Hodge of Newcastle University wrote in The Conversation:
We know from our research that looking to the past can be effective for some symptoms of dementia, but usually only when it’s specific to the person’s own life history.
How Could Virtual Memories Harm Us?
Remember how I mentioned that generative AI helped me create my virtual memory from a few photos? Well, sometimes it gets stuff wrong - like really wrong. Many of my early attempts at rebuilding my childhood bedroom produced impossible doors, and bookshelves that doubled in number as I walked past them. My favorite fluke was when I accidentally wrote the word “zoo” instead of “zoom.” Instead of slowing zooming in on a window, the AI manifested a surrealistic winged otter-monster that emerged from the shadow beneath a window sill.
This weirdness stems from the way AI tries to help reconstruct scenes: It's making a best-guess about objects (or entire parts of rooms) that weren't included in the original photos.
For me, this was interesting. But for someone already struggling with their memory, I think it could be confusing and even upsetting.
And then there’s the issue of grief-processing. I imagine that many people might want to revisit memories of, say, a home they shared with loved ones who have passed away. But I worry that virtual memories might make it harder for some of us to move on. Remember the old man in Pixar’s “Up” who didn’t want to leave the house he had shared with his late wife?
Science fiction, our culture's early warning system, has touched on some of my concerns. Decades ago, a Twilight Zone episode called “Walking Distance” told the story of a man who had the chance to revisit the summer when he was eleven... he refused to leave. More recently, a Black Mirror episode showed elderly people falling in love in a VR reconstruction of the 1990s.
With all of the change and uncertainty of life in the 2020s, it might be tempting to take shelter in an earlier decade. The past just makes more sense to us. But living there virtually could make us all lonelier and alienated from the present.
Lessons from Law
In some ways, society has been here before. When TV first came out, people were excited about its potential for education. Shows like “Sesame Street” were game-changers, especially for kids who didn’t have access to the best schools. But then, of course, people started worrying. There were concerns about kids watching too much TV and not using their imagination. They even had Senate hearings about TV violence and juvenile delinquency in the 50s and 60s.
We saw similar things happen with video games. At first, everyone talked about how great they could be for problem-solving skills and strategic thinking. But then came the worries about addiction and kids becoming antisocial and violent.
Most recently, we've seen the same pattern play out with social media. It was supposed to connect us all and give everyone a voice. Now we’re all worried about whether it’s making us depressed, anxious, and lonely.
With each of these technologies, it was often left up to lawmakers and policymakers to balance the tech pros and cons. Take TV: Starting in the 1930s, laws were enacted to ensure TV stations acted in the “public interest.” Over time, this led to mandates that TV channels had to air educational programming for children, limit ad content during cartoons, and require accessibility features like closed captioning. Concerns about video games (and a healthy dose of government pressure) led to rating systems to help parents figure out what games were okay for their kids. Something similar is happening with social media, although the laws are still being written. It’s been messy, but in each case, society has muddled by.
I expect we’ll see a similar pattern unfold with virtual memories. We may see the development of guidelines for ethical AI-generated content in virtual memories or perhaps rating systems to help users understand the potential emotional impact of certain experiences. Ultimately, as a society, we’ll need to balance the positive potential of virtual memories - such as their use in reminiscence therapy - against the risks, like the potential for memory distortion or unhealthy escapism.
Thoughts to Leave You With
"No man ever steps in the same river twice, for it's not the same river, and he's not the same man." - Heraclitus
Even if we could recreate our past perfectly in VR, we wouldn't be the same people we were then. Our memories are integral to who we are, but they don't define us completely. It's crucial to keep this in mind as we explore this new technology.
The possibilities are exciting to me. Revisiting important moments in our lives could be powerful for reflection and healing. But, I think we should be cautious not to become too entangled in the past. After all, the present is truly all we ever have.
What do you think?
Does the idea of stepping into your memories in VR sound exciting to you? Or do you think it would be upsetting? I’d love to hear your thoughts on this.
Until next time, thanks for being here.
-Mike