During a minimally invasive procedure such as angioplasty, a clinician threads a catheter or endoscope through a small incision and watches its progress on a live X-ray feed. The image is flat. It shows shadows moving across a two-dimensional plane, not a three-dimensional map of where the instrument tip actually sits inside the body. Working out precise position and orientation from that feed alone is hard, and getting it wrong raises the risk of complications during the procedure.
Clinicians compensate by aligning the live X-ray with a scan taken before surgery, usually a CT or MRI, in a step called registration. Today that alignment is done by hand: typing coordinate estimates into a workstation, or clicking anatomical landmarks on a screen to nudge the two images into agreement. Interpreting raw 2D projections well enough to do this reliably takes clinicians decades of training, according to researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), and the manual process itself is slow.
Automating registration with machine learning has been tried before, but existing 2D/3D registration tools have struggled to generalise across patients. Human anatomy varies enough, and annotated medical imaging data is scarce enough, that a single model trained to handle everyone tends to perform unevenly from one patient to the next.
A model built for one patient, not for every patient
The MIT team, working with collaborators at Harvard Medical School, Massachusetts General Hospital, Brigham and Women’s Hospital, Boston Children’s Hospital, the St. Luke’s Marion Bloch Neuroscience Institute and Shriners Children’s Hospital, took a different approach. Rather than build one general-purpose registration model, they built a system called xvr, for X-ray volume registration, that trains a model specific to each patient’s own anatomy.
It starts from a scan the patient already has: the preoperative CT or MRI. xvr feeds that volume through a physics-based simulation of how X-rays form, generating thousands of synthetic X-ray images from many different angles and viewpoints, at a rate of roughly 1,000 synthetic images per second. Because the simulation models the physics of radiographic projection directly from real patient data, rather than generating imagery through a generative model, the resulting images are not hallucinated approximations. They are the projections that patient’s own anatomy would actually produce from those angles.
Those synthetic X-rays then train a neural network tailored to that one patient. Once trained, the network can take a real, live 2D X-ray captured during surgery and align it with the patient’s 3D scan within seconds, at sub-millimetre precision.
Getting the adaptation time down from hours to minutes
Training a registration network from scratch for a single patient, as described above, takes around 12 hours. That is far too slow to be useful in an emergency, where a registration system needs to be ready before or during the procedure, not half a day later.
To close that gap, the researchers used xvr’s synthetic-data pipeline to pre-train a more general foundation model first. They assembled whole-body 3D scans from more than 2,000 patients, spanning a range of ages, imaging modalities and body regions, and used that pool to train a model that already understands the broad structure of human anatomy before it ever sees a specific patient. Starting from that foundation, adapting the model to a new patient’s own scan takes about five minutes, while matching the accuracy of a model trained from scratch for that patient over 12 hours.
Tested against the largest available real-world dataset
The team evaluated xvr on the largest available collection of real 2D/3D registration cases, drawing on data from five hospitals and covering dozens of bones and organ systems in both adult and paediatric patients. Against that benchmark, xvr outperformed existing AI-based registration methods by roughly an order of magnitude in accuracy and reliability across cases, according to MIT News, while running fast enough to support emergency surgical timelines.
Vivek Gopalakrishnan, a postdoc at CSAIL and the paper’s first author, points to stroke care as a concrete case where that speed and accuracy matter. Most Americans live more than an hour from a centre equipped to perform non-invasive procedures such as urgent stroke interventions, and in stroke care an hour is a substantial amount of time to lose. Combining 2D and 3D information, in his account, is what makes it feasible to bring these specialised, life-saving procedures to a wider range of hospitals and patients.
The paper, “Rapid patient-specific neural networks for X-ray to volume registration”, appeared in Nature on 16 September 2026. Gopalakrishnan carried out the work as part of the Harvard-MIT Program in Health Sciences and Technology, advised by Polina Golland, an MIT professor of electrical engineering and computer science who leads the Medical Vision Group at CSAIL and is a co-senior author on the paper. Neel Dey, a former postdoc in that group now at Harvard Medical School and Massachusetts General Hospital, is the other co-senior author.
What comes next
The researchers describe several open problems ahead of clinical deployment. They want to make xvr faster still, to support real-time use during surgery, and they plan further studies to confirm that its accuracy holds up across a wider range of clinical situations. They also intend to extend the approach to more complex scenarios involving moving anatomy, such as organs that shift with breathing or heartbeat, which is a harder registration problem than the largely static anatomy the current dataset covers. The team is working with robotic surgery companies and clinical groups to move the underlying research toward navigation tools that could eventually reach operating rooms, alongside potential applications in robotic surgery more broadly.
The work was funded in part by the National Institutes of Health, the MIT CSAIL-Wistron Program, the MIT-IBM Computing Research Lab, the MIT Jameel Clinic, the MIT Health and Life Sciences Collaborative, and the Chou Family Transformative Research Fund.
Sources
Adam Zewe, “New AI technique could make minimally invasive surgeries safer, more precise”, MIT News, 16 September 2026: https://news.mit.edu/2026/new-ai-technique-could-make-minimally-invasive-surgeries-safer-more-precise-0916
Vivek Gopalakrishnan et al., “Rapid patient-specific neural networks for X-ray to volume registration”, Nature, 16 September 2026: https://www.nature.com/articles/s41586-026-11045-x
read0nly 
Leave a comment