The Researcher Who Planned His Own Surgery in Virtual Reality

When professor Larry Smarr discovered he had Crohn's Disease, he tackled it the only way he knew how: with technology.

Larry Smarr, a polymath and computer science professor at the University of California, San Diego, is accustomed to self-analysis. If something in his body can be monitored, evaluated or picked apart–his sleeping habits, heart rate, DNA, stool–he’s done it. In fact, he does it, constantly. All of the data goes into what he calls the Quantified Self, a digital collection of regularly updated figures and measurements of his physical makeup. Nearly everything that enters and exits his body is analyzed. So to those who know him it was no surprise when he decided to digitally map his colon.

At the time, Smarr had been quantifying himself for years, using for-profit labs that he found on the Internet, when, in 2008, he discovered something unusual. From one of his regular blood tests, he found that his body’s inflammation levels were through the roof. “Something terrible,” he told his doctor, “is going on inside of me.” Exhibiting no symptoms, the doctor sent him away, telling him to come back when he felt pain–all he had at that point was numbers. But the data hadn’t lied and soon, after symptoms appeared in agonizing droves, Smarr learned that he had Crohn’s, an inflammatory bowel disease. When it was determined that the best course of action was to remove part of his colon, Smarr presented an odd request to his surgeon: to plan the surgery using a virtual reality map of his insides.

Though Smarr has a background in astrophysics, he is also a leading expert in virtual reality. All of that data from his years of quantifying himself he had converted into what he calls a Transparent Larry, a digital reproduction of his insides produced from MRI scans which could be explored in virtual reality. The surgery, he proposed, could be meticulously planned through using both these virtual reality maps and a 3D-printed model of part of his colon which he had had made. His surgeon, Dr. Sonia Ramamoorthy, a leader in integrating robotics into surgery, took to the idea. In the end, the operation was a major success. Smarr calls it “the quantified surgery” and now he and Ramamoorthy want to standardize the approach.

Their logic is simple: we use virtual reality in planning all sorts of things, from architectural projects and product design to space travel. Why not in medicine too? We reached out to Smarr to hear more.


Larry Smarr.

Take me to the beginning. How did your idea of the Quantified Self come about?

When I came to San Diego after teaching at the University of Illinois for twenty years I looked like the average American: overweight or obese. Two-thirds of American, at least, are that way. I looked around and everybody here in San Diego was healthy and thin. I thought, how does that happen? In most of the country, everybody looked like me. It was the frog in the boiling water scenario, with the water heating up without it noticing–that’s how the obesity epidemic has gone on for four decades. So I decided to make some changes. I got an exercise coach, started questioning my diet, reading books on nutrition. I realized, “Gee, there’s a science to this.”

As I began to lose weight, I wanted to know how well I was doing. As a scientist, if you’re going to do an experiment it’s important that you measure the rate of a change in a longitudinal time series. So I started recording my weight every day. Whenever I went to the doctor for a check-up, I asked for the data from my blood test. I started accumulating it. I realized there were all kinds of things I could track. One example is you can do a test that looks inside your red blood cells and get twenty or thirty different chemicals, like magnesium or chromium, recorded. I was trying to make sure that I was putting together a diet that had all the right micronutrients. I realized that I could start titrating my diet by the value in my blood: if a vitamin was low I could take a supplement or eat more spinach for iron. Simple things really. But the idea that as an individual you would do that seemed to be very rare in the way people operated. Yet it seemed totally logical to me.

With the success of Fitbit and other health apps, quantifying one’s health is a sort of movement these days, though, correct?

Well, it’s like how everybody thinks the internet showed up in 1995 but in fact is came about in ‘72. There’s been a growth curve. The Quantified Self movement has been slowly growing for decades and it’s gotten finally to the point where you actually know somebody who does it. It hasn’t just been all of the sudden.

How does, “being the CEO of your body,” as you’ve called it, affect your daily life?

When you realize that there is nobody other than you who is going to take care of your body that changes a lot about your lif

When you realize that there is nobody other than you who is going to take care of your body that changes a lot about your life. You eat better, you exercise, you sleep more. Most people are causing their own future disease by neglecting to think about what they eat or do, things that will have a downstream negative impact on your health. It’s just an abrogation of responsibility, fundamentally. I find it so weird that people think quantifying your body is strange. We quantify everything. How many people think that a quantified approach to investment is crazy? The thing that I find bizarre is that in our culture people think that knowing what goes on inside of your body is not something that’s their responsibility. They know more about what’s on Twitter than about what’s going on in their body. How crazy is that?

Is Quantified Self your term?

I don’t know whether I coined it or not. But I’m one of the few people who have actually quantified themselves through this many variables. I keep track of over one hundred things, between blood and stool variables. Then I’ve also started doing 3D quantification of my body using MRI scans. They put the data on a CD for me and then in my institute Calit2, at UCSD, we can put that into computer graphic software and build a Transparent Larry. But it can be a Transparent Anybody who has the MRI data.

What is “the Cave”?

My colleagues and I have been pioneers in over thirty years in building VR systems. The Cave is a walk-in VR room where you just put polarizing glasses on, you don’t have a head-mounted display, and can see a digital map of my body.

In the operation on your colon, this was very helpful?

Absolutely. It turns out that we all differ in detail with our internal anatomy. My surgeon Dr. Sonia Ramamoorthy is fantastic but this was the first time she had ever done 3D pre-surgical planning. It’s sort of amazing to think about. In computer science, it’s been 25 years at least since we knew how to take the slices of MRI and put them back into a transparent 3D model. The fact that it has not been adopted in most fields of surgery–they do use it some in brain surgery and some cardiovascular surgery–but it’s not at all standard in abdominal surgery.

What do you think was more helpful to your surgeon, Transparent Larry, or the 3D printed colon?

Transparent Larry. My colon differs in shape than the textbook colon and as I told the surgeon, “You’re not going to do surgery on a textbook, you’re doing it on an individual.” Without the 3D planning, Sonia wouldn’t have been able to enter my abdomen at the right place. She would have had to do a lot more work, probably an extra hour and a half of work while I’m under anesthesia And the length of time you’re under is the main thing that correlates with patient outcome after surgery. So it made a big difference.

Larry showing off “Transparent Larry,” a 3D model of his guts.

The field of surgery is notoriously resistant to change. How do you convince other surgeons to adopt this practice?

The market will bring it to them. People who make the scanners are perfectly capable of creating a 3D model from the data; it just hasn’t been requested by the customers before. I talk to some of the companies that make the scanners and they all have in-house versions of this. But the radiologists who they view as their customers are used to looking at everything in 2D so they don’t really have a need for it. But the surgeons, who work only in 3D, actually don’t get too much out of the 2D slices.

Are you actively lobbying for this technology to be integrated into medicine?

As a professor, I’m putting together a research program with my surgeon, radiologist and computer graphics people and any private sector companies that want to be part of this. Then the idea is to gradually improve the capability of this and get it used on more patients. Sonia would like to do a dozen patients like this right away.

You’re hopeful that VR pre-surgery planning will eventually be standardized?

Absolutely. There’s no question about it.

By the way, how is your health these days?