Measuring errors in neuro-interventional pointing tasks is critical to better evaluating human experts as well as machine learning algorithms. If the target may be highly ambiguous, different experts may fundamentally select different targets, believing them to refer to the same region, a phenomenon called an error of type. This paper investigates the effects of changing the prior distribution on a Bayesian model for errors of type specific to transcranial magnetic stimulation (TMS) planning. Our results show that a particular prior can be chosen which is analytically solvable, removes spurious modes, and returns estimates that are coherent with the TMS literature. This is a step towards a fully rigorous model that can be used in system evaluation and machine learning.
Purpose: Deep brain stimulation (DBS) is an interventional treatment for some neurological and neurodegenerative diseases. For example, in Parkinson’s disease, DBS electrodes are positioned at particular locations within the basal ganglia to alleviate the patient’s motor symptoms. These interventions depend greatly on a preoperative planning stage in which potential targets and electrode trajectories are identified in a preoperative MRI. Due to the small size and low contrast of targets such as the subthalamic nucleus (STN), their segmentation is a difficult task. Machine learning provides a potential avenue for development, but it has difficulty in segmenting such small structures in volumetric images due to additional problems such as segmentation class imbalance.
Approach: We present a two-stage separable learning workflow for STN segmentation consisting of a localization step that detects the STN and crops the image to a small region and a segmentation step that delineates the structure within that region. The goal of this decoupling is to improve accuracy and efficiency and to provide an intermediate representation that can be easily corrected by a clinical user. This correction capability was then studied through a human–computer interaction experiment with seven novice participants and one expert neurosurgeon.
Results: Our two-step segmentation significantly outperforms the comparative registration-based method currently used in clinic and approaches the fundamental limit on variability due to the image resolution. In addition, the human–computer interaction experiment shows that the additional interaction mechanism allowed by separating STN segmentation into two steps significantly improves the users’ ability to correct errors and further improves performance.
Conclusions: Our method shows that separable learning not only is feasible for fully automatic STN segmentation but also leads to improved interactivity that can ease its translation into clinical use.
KEYWORDS: Computer simulations, Magnetic resonance imaging, Data modeling, Probability theory, Machine learning, 3D image processing, Neuroimaging, Human-computer interaction
Transcranial magnetic stimulation is a non-invasive therapeutic procedure in which specific cortical brain regions are stimulated in order to disrupt abnormal neural behaviour. This procedure requires the annotation of a number of cortical point targets which is often performed by a human expert. Nevertheless, there is a large degree of variability between experts that cannot be described readily using the existing zero-mean uni-modal error model common in computer-assisted interventions. This is due to the error between experts arising from a difference of type rather than a difference of degree, that is, experts are not necessarily picking the same point with some error, but are picking fundamentally different points. In order to model these types of localisation errors, this paper proposes a simple probabilistic model that uses the agreement between annotations as a basis, not requiring a ground-truth annotation to be strictly known. This work will allow for the localisation error in transcranial magnetic stimulation to be better described which may spur further developments in clinical training as well as machine learning for cortical point localisation.
Parkinson's disease is a neurodegenerative disorder affecting the basal ganglia and resulting in characteristic motor and non-motor symptoms. Although pharmocological treatments are often used, deep brain stimulation can be used either to complement these treatments or replace them if ineffective. Deep brain stimulation involves the implantation of electrodes into the patient's subcortical anatomy at particular regions of interest, such as the subthalamic nucleus, in order to control or alleviate abnormal neural behaviour. For these interventions to be successful, precise pre-operative segmentation of these structures in MRI is of paramount importance. This paper presents a convolutional neural network that is capable of learning the process of subthalamic nucleus segmentation from pre-operative clinical strength MR images with an accuracy of 58:2 ± 12:1% Dice which is within the Dice range of a one-voxel translation or dilation from the reference manual segmentation. This is the final step in a combined localisation/segmentation framework for small anatomy such as the STN which is computationally efficient (avoiding deformable registration) while simultaneously being easier for the user to correct in the presence of errors.
Deep brain stimulation (DBS) is an interventional treatment for Parkinson’s disease in which electrodes are placed into specific locations in the basal ganglia to alleviate symptoms such as tremor and dyskinesia. Due to the small size and low contrast of specific targets of interest, such as the subthalamic nucleus (STN), localisation of these structures from pre-operative MRI is of great value. These localisation approaches are often atlas-based, using registration algorithms to align patient images with a prior annotated atlas. However, these methods require a large amount of time, especially for correctly estimating deformation fields, or are prone to error for smaller structures such as the STN. This paper investigates two deep learning frameworks for the localisation of the stn from T1- and T2-weighted MRI using convolutional neural networks which estimate its centroid. These methods are compared against an atlas-based segmentation using the ParkMedAtlis v3 atlas, showing an improvement in localisation error in the range of ≈0.5-1.3 mm with a reduction of orders of magnitude of computation time. This method of STN localisation will allow us in future to automatically identify the STN for DBS surgical planning as well as define a greatly reduced region-of-interest for more accurate segmentation of the STN.
One of the recent developments in deep learning is the ability to train extremely deep residual neural networks, knowing that each residual block produces only marginal changes to the data. The accumulation of these changes result in the network’s improved performance, analogous to a complex but trainable iterative algorithm. This intuition can be merged with the underlying theory of probabilistic graphical models in which these iterative algorithms are common and share the underlying probabilistic and information theoretic basis as deep learning. Prior models have been proposed with limitations on the number of iterations allowed in the solution algorithm due to the linear memory growth during the training process. This paper presents a structured activation layer which implements a conditional random field along with an arbitrary iteration message-passing marginal probability estimation algorithm which requires constant, rather than linear, memory with respect to the number of iterations. In this activation layer, the segmentation labels can be specified hierarchically, incorporating a level of abstract structure and some basic geometrical knowledge directly and easily into the network. Thus, this layer allows for the separation of abstract knowledge brought in by the network designer (in the form of the hierarchical structure) from probabilistic priors learned by the neural network. Preliminary results comparing this activation function to softmax and a similar non-hierarchical activation function indicate that it significantly improves performance in segmentation problems.
Ultrasound (US)-guided interventions are often enhanced via integration with an augmented reality environment, a necessary component of which is US calibration. Calibration requires the segmentation of fiducials, i.e., a phantom, in US images. Fiducial localization error (FLE) can decrease US calibration accuracy, which fundamentally affects the total accuracy of the interventional guidance system. Here, we investigate the effects of US image reconstruction techniques as well as phantom material and geometry on US calibration. It was shown that the FLE was reduced by 29% with synthetic transmit aperture imaging compared with conventional B-mode imaging in a Z-bar calibration, resulting in a 10% reduction of calibration error. In addition, an evaluation of a variety of calibration phantoms with different geometrical and material properties was performed. The phantoms included braided wire, plastic straws, and polyvinyl alcohol cryogel tubes with different diameters. It was shown that these properties have a significant effect on calibration error, which is a variable based on US beamforming techniques. These results would have important implications for calibration procedures and their feasibility in the context of image-guided procedures.
Optimization-based segmentation approaches deriving from discrete graph-cuts and continuous max-flow have become increasingly nuanced, allowing for topological and geometric constraints on the resulting segmentation while retaining global optimality. However, these two considerations, topological and geometric, have yet to be combined in a unified manner. The concept of “shape complexes,” which combine geodesic star convexity with extendable continuous max-flow solvers, is presented. These shape complexes allow more complicated shapes to be created through the use of multiple labels and super-labels, with geodesic star convexity governed by a topological ordering. These problems can be optimized using extendable continuous max-flow solvers. Previous approaches required computationally expensive coordinate system warping, which are ill-defined and ambiguous in the general case. These shape complexes are demonstrated in a set of synthetic images as well as vessel segmentation in ultrasound, valve segmentation in ultrasound, and atrial wall segmentation from contrast-enhanced CT. Shape complexes represent an extendable tool alongside other continuous max-flow methods that may be suitable for a wide range of medical image segmentation problems.
KEYWORDS: Image segmentation, Human-machine interfaces, Interfaces, Brain, Data modeling, Blood, Medical imaging, Computed tomography, Magnetic resonance imaging, Image processing algorithms and systems
Interactive segmentation is becoming of increasing interest to the medical imaging community in that it combines the positive aspects of both manual and automated segmentation. However, general-purpose tools have been lacking in terms of segmenting multiple regions simultaneously with a high degree of coupling between groups of labels. Hierarchical max-flow segmentation has taken advantage of this coupling for individual applications, but until recently, these algorithms were constrained to a particular hierarchy and could not be considered general-purpose. In a generalized form, the hierarchy for any given segmentation problem is specified in run-time, allowing different hierarchies to be quickly explored. We present an interactive segmentation interface, which uses generalized hierarchical max-flow for optimization-based multiregion segmentation guided by user-defined seeds. Applications in cardiac and neonatal brain segmentation are given as example applications of its generality.
Optimization-based segmentation approaches deriving from discrete graph-cuts and continuous max-flow have become increasingly nuanced, allowing for topological and geometric constraints on the resulting segmentation while retaining global optimality. However, these two considerations, topological and geometric, have yet to be combined in a unified manner. This paper presents the concept of shape complexes, which combine geodesic star convexity with extendable continuous max-flow solvers. These shape complexes allow more complicated shapes to be created through the use of multiple labels and super-labels, with geodesic star convexity governed by a topological ordering. These problems can be optimized using extendable continuous max-flow solvers. Previous work required computationally expensive co-ordinate system warping which are ill-defined and ambiguous in the general case. These shape complexes are validated in a set of synthetic images as well as atrial wall segmentation from contrast-enhanced CT. Shape complexes represent a new, extendable tool alongside other continuous max-flow methods that may be suitable for a wide range of medical image segmentation problems.
Image-based ultrasound to magnetic resonance image (US-MRI) registration can be an invaluable tool in image-guided neuronavigation systems. State-of-the-art commercial and research systems utilize image-based registration to assist in functions such as brain-shift correction, image fusion, and probe calibration.
Since traditional US-MRI registration techniques use reconstructed US volumes or a series of tracked US slices, the functionality of this approach can be compromised by the limitations of optical or magnetic tracking systems in the neurosurgical operating room. These drawbacks include ergonomic issues, line-of-sight/magnetic interference, and maintenance of the sterile field. For those seeking a US vendor-agnostic system, these issues are compounded with the challenge of instrumenting the probe without permanent modification and calibrating the probe face to the tracking tool.
To address these challenges, this paper explores the feasibility of a real-time US-MRI volume registration in a small virtual craniotomy site using a single slice. We employ the Linear Correlation of Linear Combination (LC2) similarity metric in its patch-based form on data from MNI’s Brain Images for Tumour Evaluation (BITE) dataset as a PyCUDA enabled Python module in Slicer. By retaining the original orientation information, we are able to improve on the poses using this approach. To further assist the challenge of US-MRI registration, we also present the BOXLC2 metric which demonstrates a speed improvement to LC2, while retaining a similar accuracy in this context.
Interactive segmentation is becoming of increasing interest in medical imaging, combining the positive aspects of manual and automated segmentation. However, general purpose tools have been lacking in terms of segmenting multiple regions with a high degree of coupling simultaneously. Hierarchical max-flow segmentation has taken advantage of this coupling for individual applications, but until recently these algorithms were constrained to a particular hierarchy and could not be considered general-purpose. With generalized hierarchical max-flow solvers, the hierarchy is specified in run-time, allowing different hierarchies to be explored. This paper presents a novel interactive segmentation interface, using generalized hierarchical max-flow for multi-region segmentation.
Ultrasound calibration is a necessary procedure in many image-guided interventions, relating the position of tools and anatomical structures in the ultrasound image to a common coordinate system. This is a necessary component of augmented reality environments in image-guided interventions as it allows for a 3D visualization where other surgical tools outside the imaging plane can be found. Accuracy of ultrasound calibration fundamentally affects the total accuracy of this interventional guidance system. Many ultrasound calibration procedures have been proposed based on a variety of phantom materials and geometries. These differences lead to differences in representation of the phantom on the ultrasound image which subsequently affect the ability to accurately and automatically segment the phantom. For example, taut wires are commonly used as line fiducials in ultrasound calibration. However, at large depths or oblique angles, the fiducials appear blurred and smeared in ultrasound images making it hard to localize their cross-section with the ultrasound image plane. Intuitively, larger diameter phantoms with lower echogenicity are more accurately segmented in ultrasound images in comparison to highly reflective thin phantoms. In this work, an evaluation of a variety of calibration phantoms with different geometrical and material properties for the phantomless calibration procedure was performed. The phantoms used in this study include braided wire, plastic straws, and polyvinyl alcohol cryogel tubes with different diameters. Conventional B-mode and synthetic aperture images of the phantoms at different positions were obtained. The phantoms were automatically segmented from the ultrasound images using an ellipse fitting algorithm, the centroid of which is subsequently used as a fiducial for calibration. Calibration accuracy was evaluated for these procedures based on the leave-one-out target registration error. It was shown that larger diameter phantoms with lower echogenicity are more accurately segmented in comparison to highly reflective thin phantoms. This improvement in segmentation accuracy leads to a lower fiducial localization error, which ultimately results in low target registration error. This would have a profound effect on calibration procedures and the feasibility of different calibration procedures in the context of image-guided procedures.
Simultaneous segmentation of multiple anatomical objects from medical images has become of increasing interest to the medical imaging community, especially when information concerning these objects such as grouping or hierarchical relationships can facilitate segmentation. Single parameter Potts models have often been used to address these multi-region problems, but such parameterization is not sufficient when regions have largely different regularization requirements. These problems can be addressed by introducing smoothing hierarchies with capture grouping relationships at the expense of additional parameterization. Tuning of these parameters to provide optimal segmentation accuracy efficiently is still an open problem in optimal image segmentation. This paper presents two mechanisms, one iterative and one more computationally efficient, for estimating optimal smoothness parameters for any arbitrary hierarchical model based on multi-objective optimization theory. These methods are evaluated using 5 segmentations of the brain from the IBSR database containing 35 distinct regions. The iterative estimator provides equivalent performance to the downhill simplex method, but takes significantly less computation time (93 vs. 431 minutes), allowing for more complicated models to be used without worry as to prohibitive parameter tuning procedures.
Endoscopic and laparoscopic surgeries are used for many minimally invasive procedures but limit the visual and haptic feedback available to the surgeon. This can make vessel sparing procedures particularly challenging to perform. Previous approaches have focused on hardware intensive intraoperative imaging or augmented reality systems that are difficult to integrate into the operating room. This paper presents a simple approach in which motion is visually enhanced in the endoscopic video to reveal pulsating arteries. This is accomplished by amplifying subtle, periodic changes in intensity coinciding with the patient’s pulse. This method is then applied to two procedures to illustrate its potential. The first, endoscopic third ventriculostomy, is a neurosurgical procedure where the floor of the third ventricle must be fenestrated without injury to the basilar artery. The second, nerve-sparing robotic prostatectomy, involves removing the prostate while limiting damage to the neurovascular bundles. In both procedures, motion magnification can enhance subtle pulsation in these structures to aid in identifying and avoiding them.
Ultrasound calibration allows for ultrasound images to be incorporated into a variety of interventional applica tions. Traditional Z- bar calibration procedures rely on wired phantoms with an a priori known geometry. The line fiducials produce small, localized echoes which are then segmented from an array of ultrasound images from different tracked probe positions. In conventional B-mode ultrasound, the wires at greater depths appear blurred and are difficult to segment accurately, limiting the accuracy of ultrasound calibration. This paper presents a novel ultrasound calibration procedure that takes advantage of synthetic aperture imaging to reconstruct high resolution ultrasound images at arbitrary depths. In these images, line fiducials are much more readily and accu rately segmented, leading to decreased calibration error. The proposed calibration technique is compared to one based on B-mode ultrasound. The fiducial localization error was improved from 0.21mm in conventional B-mode images to 0.15mm in synthetic aperture images corresponding to an improvement of 29%. This resulted in an overall reduction of calibration error from a target registration error of 2.00mm to 1.78mm, an improvement of 11%. Synthetic aperture images display greatly improved segmentation capabilities due to their improved resolution and interpretability resulting in improved calibration.
The importance of presenting medical images in an intuitive and usable manner during a procedure is essential.
However, most medical visualization interfaces, particularly those designed for minimally-invasive surgery, suffer
from a number of issues as a consequence of disregarding the human perceptual, cognitive, and motor system's
limitations. This matter is even more prominent when human visual system is overlooked during the design cycle.
One example is the visualization of the neuro-vascular structures in MR angiography (MRA) images. This study
investigates perceptual performance in the usability of a display to visualize blood vessels in MRA volumes
using a contour enhancement technique. Our results show that when contours are enhanced, our participants,
in general, can perform faster with higher level of accuracy when judging the connectivity of different vessels.
One clinical outcome of such perceptual enhancement is improvement of spatial reasoning needed for planning
complex neuro-vascular operations such as treating Arteriovenous Malformations (AVMs). The success of an
AVM intervention greatly depends on fully understanding the anatomy of vascular structures. However, poor
visualization of pre-operative MRA images makes the planning of such a treatment quite challenging.
Needle biopsies are standard protocols that are commonly performed under ultrasound (US) guidance or computed
tomography (CT)1. Vascular access such as central line insertions, and many spinal needle therapies also rely on US
guidance. Phantoms for these procedures are crucial as both training tools for clinicians and research tools for developing
new guidance systems. Realistic imaging properties and material longevity are critical qualities for needle guidance
phantoms. However, current commercially available phantoms for use with US guidance have many limitations, the most
detrimental of which include harsh needle tracks obfuscating US images and a membrane comparable to human skin that
does not allow seepage of inner media. To overcome these difficulties, we tested a variety of readily available media and
membranes to evaluate optimal materials to fit our current needs. It was concluded that liquid hand soap was the best
medium, as it instantly left no needle tracks, had an acceptable depth of US penetration and portrayed realistic imaging
conditions, while because of its low leakage, low cost, acceptable durability and transparency, the optimal membrane
was 10 gauge vinyl.
One of the fundamental components in all Image Guided Surgery (IGS) applications is a method for presenting
information to the surgeon in a simple, effective manner. This paper describes the first steps in our new
Augmented Reality (AR) information delivery program. The system makes use of new "off the shelf" AR glasses
that are both light-weight and unobtrusive, with adequate resolution for many IGS applications. Our first
application is perioperative planning of minimally invasive robot-assisted cardiac surgery. In this procedure,
a combination of tracking technologies and intraoperative ultrasound is used to map the migration of cardiac
targets prior to selection of port locations for trocars that enter the chest. The AR glasses will then be used to
present this heart migration data to the surgeon, overlaid onto the patients chest. The current paper describes
the calibration process for the AR glasses, their integration into our IGS framework for minimally invasive robotic
cardiac surgery, and preliminary validation of the system. Validation results indicate a mean 3D triangulation
error of 2.9 ± 3.3mm, 2D projection error of 2.1 ± 2.1 pixels, and Normalized Stereo Calibration Error of 3.3.
A unified framework for voxel classification and triangulation for medical images is presented. Given volumetric
data, each voxel is labeled by a two-dimensional classification function based on voxel intensity and gradient. A
modified Constrained Elastic Surface Net is integrated into the classification function, allowing the surface mesh
to be generated in a single step. The modification to the Constrained Elastic Surface Net includes additional
triangulation cases which reduce visual artifacts, and a surface-node relaxation criterion based on linear regression
which improves visual appearance and preserves the enclosed volume. By carefully designing the two-dimensional
classification function, surface meshes for different anatomical structures can be generated in a single process.
This framework is implemented on the GPU, allowing rendition of the voxel classification to be visualized in
near real-time.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.