Medicine and the Metaverse: New Technology Allows Doctors to Travel Inside Your Body

Interested in knowing what’s next for the gaming industry? Join gaming executives to discuss emerging parts of the industry in October at GamesBeat Summit Next. Register today.


The world of technology is rapidly moving from flat media seen in third person to immersive media experienced in first person. Recently dubbed “the metaverse,” this major shift in consumer computing has sparked a new wave of enthusiasm for the core technologies of virtual and augmented reality. But there is a third technology area called telepresence which is often overlooked but will become an important part of the metaverse.

While virtual reality takes users into simulated worlds, telepresence (also called telerobotics) uses remote robots to bring users to remote locations, giving them the ability to look around and perform complex tasks. This concept dates back to 1940s science fiction and a seminal short story by Robert A. Heinlein titled Waldo. If we combine this concept with another classic sci-fi tale, Fantastic trip (1966), one can imagine tiny robotic vessels penetrating inside the body and swimming under the control of doctors who diagnose patients from the inside, and even perform surgical tasks.

I know this sounds like pure fiction, but a start-up in Hayward, California recently “flew” a tiny robot inside the digestive tracts of human subjects. The company is Endiatex, and had the opportunity to discuss their technology and vision with CEO Torrey Smith. As a technologist involved in telepresence research From the first days, I was impressed by the progress made by Endiatx. But before we get into the details, let’s step back a few decades and explain why their breakthrough seems like such an unexpected step to me.

The first telerobotic prototypes

My first experience with an immersive telerobotics system goes back more than thirty years. I still remember the first time I looked into a helmet, grabbed a set of periscope-like handles, and looked around in a room that wasn’t the one I was sitting in. It was 1991 and I was working in a lab at NASA that had some of the first prototypes of a mobile telerobotics system. It was developed by Fake space labs and Telepresence Research and it allowed to control a mobile robot with a camera system that returned stereoscopic images in real time. Below is a photo of this first system of a academic document on these efforts, showing the state of the art of telepresence research thirty years ago.

Mobile telepresence robot

This early system was extremely impressive at the time and was developed by some of the best researchers in the field at the time. But if you wanted to take the gear to a trade show, you’d probably need a good-sized van or maybe even a U-Haul truck. The idea of ​​shrinking such a system down to a size that a person could swallow and be checked by a doctor was beyond conception. So what were the fields of application proposed in the early days of telepresence?

In the beginning, the goal was to allow human operators to carry out work in dangerous places, for example, to clean up nuclear accidents, repair satellites and even fix leaking oil wells at the bottom of the ocean. My personal goal in the early 1990s was to add augmented reality and haptic feedback to the telepresence field with the goal of improving operator performance. It was cutting-edge research at the time, but I still hadn’t considered the idea of ​​scaling the technology down to such a size that it could be ‘stolen’ inside the human body in order to diagnose and treat patients from within.

“Fantastic Journey” directed

For these reasons, I was surprised to learn of the ambitious vision and recent technical successes of the Endiatx team. Founded in 2019, they have already created a tiny robotic drone that can be swallowed by human patients and piloted remotely inside the stomach and other parts of the digestive tract. Known as the PillBot™, their prototype system is essentially a tiny multivitamin-sized remote-controlled submarine that sends real-time video to a doctor’s computer or phone. And it works – they did tests on cadavers and living humans.

Pillbot™ robotic drone being swallowed by medical patients. Image courtesy of Endiatx

In a conversation with CEO Torrey Smith, he told me that he was inspired as a child by the sci-fi film Innerspace (1987) and has been thinking about this concept ever since. A few years ago, he finally took the plunge by founding Endiatx to bring this concept to life. And so far, it’s going well. Not only is his company poised to bring this capability to healthcare as a shipping product, but Torrey himself was the first person on the planet to have a robotic drone. flew into his stomach. He volunteered, swallowing the first prototype which made a real fantastic trip.

Since that initial test, others at the company have swallowed working prototypes on several occasions, capturing live video of the type that will one day be used to screen patients for ulcers, gastritis, cancers, and other cancers. other potential ailments. And that day isn’t far off – the team is currently performing cadaver tests with doctors at the Mayo Clinic and planning trials for FDA approval. If all goes well, the PillBot could diagnose patients around the world by 2024.

This could be a huge benefit for people going to the doctor with stomach pain. Instead of having a standard endoscopy procedure, which typically requires sedation and involves multiple visits, the small swallowable robot could save time, money and complexity, giving doctors a quick and easy way to look inside their patient. And it can provide more flexible control than a traditional endoscope, because the untethered Pillbot has the full 3D mobility of a tiny robotic submarine. It even looks like a tiny submarine – with micromotors and tiny propellers as well as a video camera, battery and wireless to send images back to doctors in real time.

Cutaway drawing of the Pillbot telepresence drone for human travel. Image courtesy of Endiatx

Faster, cheaper and more accurate screenings

Currently, doctors can fly the little drone using a standard Xbox game controller, but the company plans to enable control using the touchscreen of any mobile phone. In effect, their vision is of a disposable unit that is shipped to your home and swallowed during a telemedicine consultation with your doctor, who reviews the camera feed in real time from their PC or phone. Endiatx believes the robotic pills can be made for $25 each, sold for hundreds of dollars each, and save several thousand dollars in medical costs that endoscopic procedures cost under sedation. But more importantly, the company believes the PillBots will save many lives by enabling faster, cheaper screening that detects serious conditions earlier than would otherwise be practical.

The company plans to start the first clinical trials later this year, roll out an in-clinic version soon after, and launch a home version soon after. And once they have expedition products that allow doctors to look around inside the body, their next goal is to enable the device to take tissue samples and perform other tasks. surgical. In the long term, their plan is to shrink their robotic drone down to the size of a grain of rice, unlocking capabilities beyond the digestive tract. And their goal is to provide all of these features for home use, with doctors controlling the robots via telemedicine.

The future of telemedicine?

I was a little skeptical about the telemedicine angle for a little robot like this, since we’re talking about a powerful diagnostic tool delivered by mail. Will the medical profession embrace such a change or insist on retaining such capabilities in clinics and hospitals? That’s when I saw Amazon’s press release this week announcing the acquisition from healthcare provider One Medical for nearly $4 billion and stating that Amazon wants to reinvent the healthcare experience.

Neil Lindsay, SVP of Amazon Health Services is quoted in the statement: “Making an appointment, waiting weeks or even months to be seen, missing work, going to a clinic, finding a parking space, waiting in the waiting room and then the exam room for what is too often a few rushed minutes with a doctor, then making another trip to a pharmacy – we see many opportunities both to improve the quality of the experience and give people back valuable time in their days.

In this context, the sci-fi vision of tiny drones swallowed by patients and controlled remotely by doctors during telemedicine visits could really be part of our traditional medical experience. As a longtime technologist who often writes about the impending dangers of the metaverse, this is an emerging application area that really impresses me.

Authors biography : Louis Rosenberg, PhD is a pioneer in the fields of virtual reality, augmented reality and artificial intelligence. Thirty years ago, he developed the first working mixed reality system at the Air Force Research Laboratory. He went on to found the first virtual reality company Immersion Corporation (1993) and the first augmented reality company Outland Research (2004). He is currently CEO of Unanimous AI, a company that amplifies the intelligence of human groups. Rosenberg received his Ph.D. from Stanford University, was a professor at California State University, and has been awarded over 300 patents for his work in virtual reality, augmented reality, and AI.

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including data technicians, can share data insights and innovations.

If you want to learn more about cutting-edge insights and up-to-date information, best practices, and the future of data and data technology, join us at DataDecisionMakers.

You might even consider writing your own article!

Learn more about DataDecisionMakers

Comments are closed.