In ultrasound (US)-guided medical procedures, accurate tracking of interventional tools is crucial to patient safety and clinical outcome. This requires a calibration procedure to recover the relationship between the US image and the tracking coordinate system. In literature, calibration has been performed on passive phantoms, which depend on image quality and parameters, such as frequency, depth, and beam-thickness as well as in-plane assumptions. In this work, we introduce an active phantom for US calibration. This phantom actively detects and responds to the US beams transmitted from the imaging probe. This active echo (AE) approach allows identification of the US image midplane independent of image quality. Both target localization and segmentation can be done automatically, minimizing user dependency. The AE phantom is compared with a crosswire phantom in a robotic US setup. An out-of-plane estimation US calibration method is also demonstrated through simulation and experiments to compensate for remaining elevational uncertainty. The results indicate that the AE calibration phantom can have more consistent results across experiments with varying image configurations. Automatic segmentation is also shown to have similar performance to manual segmentation.
Tone mapping operators compress high dynamic range images to improve the picture quality on a digital display when the dynamic range of the display is lower than that of the image. However, tone mapping operators have been largely designed and evaluated based on the aesthetic quality of the resulting displayed image or how perceptually similar the compressed image appears relative to the original scene. They also often require per image tuning of parameters depending on the content of the image. In military operations, however, the amount of information that can be perceived is more important than the aesthetic quality of the image and any parameter adjustment needs to be as automated as possible regardless of the content of the image. We have conducted two studies to evaluate the perceivable detail of a set of tone mapping algorithms, and we apply our findings to develop and test an automated tone mapping algorithm that demonstrates a consistent improvement in the amount of perceived detail. An automated, and thereby predictable, tone mapping method enables a consistent presentation of perceivable features, can reduce the bandwidth required to transmit the imagery, and can improve the accessibility of the data by reducing the needed expertise of the analyst(s) viewing the imagery.
Image-guided surgery systems are often used to provide surgeons with informational support. Due to several unique
advantages such as ease of use, real-time image acquisition, and no ionizing radiation, ultrasound is a common
intraoperative medical imaging modality used in image-guided surgery systems. To perform advanced forms of guidance
with ultrasound, such as virtual image overlays or automated robotic actuation, an ultrasound calibration process must be
performed. This process recovers the rigid body transformation between a tracked marker attached to the transducer and
the ultrasound image. Point-based phantoms are considered to be accurate, but their calibration framework assumes that
the point is in the image plane. In this work, we present the use of an active point phantom and a calibration framework
that accounts for the elevational uncertainty of the point. Given the lateral and axial position of the point in the
ultrasound image, we approximate a circle in the axial-elevational plane with a radius equal to the axial position. The
standard approach transforms all of the imaged points to be a single physical point. In our approach, we minimize the
distances between the circular subsets of each image, with them ideally intersecting at a single point. We simulated in
noiseless and noisy cases, presenting results on out-of-plane estimation errors, calibration estimation errors, and point
reconstruction precision. We also performed an experiment using a robot arm as the tracker, resulting in a point
reconstruction precision of 0.64mm.
In recent years, various methods have been developed to improve ultrasound based interventional tool tracking. However, none of them has yet provided a solution that effectively solves the tool visualization and mid-plane localization accuracy problem and fully meets the clinical requirements. Our previous work has demonstrated a new active ultrasound pattern injection system (AUSPIS), which integrates active ultrasound transducers with the interventional tool, actively monitors the beacon signals and transmits ultrasound pulses back to the US probe with the correct timing. Ex vivo and in vivo experiments have proved that AUSPIS greatly improved tool visualization, and provided tool-tip localization accuracy of less than 300 μm. In the previous work, the active elements were made of piezoelectric materials. However, in some applications the high driving voltage of the piezoelectric element raises safety concerns. In addition, the metallic electrical wires connecting the piezoelectric element may also cause artifacts in CT and MR imaging. This work explicitly focuses on an all-optical active ultrasound element approach to overcome these problems. In this approach, the active ultrasound element is composed of two optical fibers - one for transmission and one for reception. The transmission fiber delivers a laser beam from a pulsed laser diode and excites a photoacoustic target to generate ultrasound pulses. The reception fiber is a Fabry–Pérot hydrophone. We have made a prototype catheter and performed phantom experiments. Catheter tip localization, mid-plan detection and arbitrary pattern injection functions have been demonstrated using the all-optical AUSPIS.
We describe wireless networking systems for close proximity biological sensors, as would be encountered in artificial
skin. The sensors communicate to a "base station" that interprets the data and decodes its origin. Using a large bundle
of ultra thin metal wires from the sensors to the "base station" introduces significant technological hurdles for both the
construction and maintenance of the system. Fortunately, the Address Event Representation (AER) protocol provides an
elegant and biomorphic method for transmitting many impulses (i.e. neural spikes) down a single wire/channel.
However, AER does not communicate any sensory information within each spike, other that the address of the
origination of the spike. Therefore, each sensor must provide a number of spikes to communicate its data, typically in
the form of the inter-spike intervals or spike rate. Furthermore, complex circuitry is required to arbitrate access to the
channel when multiple sensors communicate simultaneously, which results in spike delay. This error is exacerbated as
the number of sensors per channel increases, mandating more channels and more wires.
We contend that despite the effectiveness of the wire-based AER protocol, its natural evolution will be the wireless
AER protocol. A wireless AER system: (1) does not require arbitration to handle multiple simultaneous access of the
channel, (2) uses cross-correlation delay to encode sensor data in every spike (eliminating the error due to arbitration
delay), and (3) can be reorganized and expanded with little consequence to the network. The system uses spread
spectrum communications principles, implemented with a low-power integrate-and-fire neurons. This paper discusses
the design, operation and capabilities of such a system. We show that integrate-and-fire neurons can be used to both
decode the origination of each spike and extract the data contained within in. We also show that there are many
technical obstacles to overcome before this version of wireless AER can be practical.
A 128(H) x 64(V) x RGB CMOS imager is integrated with region-of-interest selection, RGB-to-HSI transformation, HSI-based pixel segmentation, 36-bins x 12bits HSI histogramming and sum-of-absolute-difference (SAD) template matching. 32 learned color templates are stored and compared to each image. The chip captures the R, G and B images using in-pixel storage before passing the pixel content to a multiplying digital-to-analog converter (DAC) for white balancing. The DAC can also be used to pipe in images for a PC. The color processing uses a biologically inspired color opponent representation and an analog look-up table to determine the Hue (H) of each pixel. Saturation (S) is computed using a loser-take-all circuit. Intensity (I) is given by the sum color components. A histogram of the segments of the image, constructed by counting the number of pixels falling into 36 Hue intervals of 10 degrees, is stored on chip and compared against the histograms of new segments using SAD comparisons. We demonstrate color-based image segmentation and object recognition with this chip. Running at 30fps, it uses 1mW. To our knowledge, this is the first chip that integrates imaging, color segmentation and color-based object recognition at the focal plane.
Extracting relevant visual information about the operating environment of a roving robot at the focal plane is a challenging problem. We present two image processing architectures for motion computation at the focal plane. The first imaging architecture, composed of 250 x 250 active pixel sensors, has spatiotemporal difference computation capabilities at the focal plane. This spatiotemporal difference imager, fabricated in a 0.35μ process, contains in-pixel storage elements for previous and current frames and difference computational units outside the imaging array. A novel scan out technique allows for parallel computation of spatial and temporal 1-D derivatives on the read out. The final motion estimation based on the image brightness constancy equation, which is approximated as the ratio of the temporal and spatial derivates, is computed off chip, but it can be easily implemented on-chip. This approach can only determine the motion component normal to the spatial gradient. In order to address this ill-posed problem, known as the aperture problem, we propose a second imaging architecture, which is an extension of the first imaging architecture. The latter imaging architecture contains in-pixel memory for storing partial computational results. Hence, the motion vectors are stored back into the pixel memories after an initial scan; iterative algorithms can be applied to the stored computational results to solve the aperture problem. Experimental data from both imaging architectures are presented to validate the accuracy of the magnitude and angle of the computed target velocities.
In this paper, we present the algorithm and operation of an aVLSI chip that can extract normal optical flow by using the gradient approach without interfering with the imaging process. This approach is feasible for scaling to larger arrays without affecting the processing or the processing area. Our system has a 92 x 52 photosensitive array of APS pixels at the core with processing circuits on the periphery. We discuss the approach and the different blocks in the design and then demonstrate the working of the individual blocks and of the system as a whole. The chip outputs the image, the spatial and temporal gradients and the normal flow at the read-out frame-rate with no penalty to the imaging process. The chip occupies an area of 4.5 mm2 and consumes 2.6 mW (at Vdd = 5V). Once normal flow is obtained, the chip can be used to compute focus of expansion, time to contact and many other motion properties of images that can be used to control robots. Tracking systems can use the velocity and segmentation of moving objects can be realized using motion discontinuity.
Two types of focal plane image processing chips are presented. They address the two extremes of the application spectrum: general purpose and application specific designs. They both exploit the promise of focal-plane computation offered by CMOS technology. The general-purpose computational sensor, a 16x16 pixels prototype (easily scalable to larger arrays), has been fabricated in a standard 1.2 μ CMOS process, and its spatio-temporal filtering capabilities have been successfully tested. An array larger than 300x300 array will use only 0.5% of the chip area for the processing unit while providing multiple spatio-temporally processed images in parallel. The 16x16 chip performs 1 GOPS/mW (5.5-bit scale-accumulate) while computing four spatio-temporal images in parallel. The application specific system realizes a hybrid imaging system by combining a 120 X 36 low-noise active pixel sensor (APS) array with a 60x36 current mode motion detection and centroid localization array. These two arrays are spatially interleaved. The APS array, which integrates photo-generated charges on a capacitor in each pixel, includes column parallel correlated double sampling for fixed pattern noise reduction. The current mode array operates in continuous time, however, the programmable motion detection circuit indicates if the intensity of light at pixel is time varying. The centroid, x and y position, of all time varying pixels is computed using circuits located at the edges of the array. Clocked at greater than 60 fps, the chip consumes less than 2 mW.
Traditional robotics revolves around the microprocessor. All well-known demonstrations of sensory guided motor control, such as jugglers and mobile robots, require at least one CPU. Recently, the availability of fast CPUs have made real-time sensory-motor control possible, however, problems with high power consumption and lack of autonomy still remain. In fact, the best examples of real-time robotics are usually tethered or require large batteries. We present a new paradigm for robotics control that uses no explicit CPU. We use computational sensors that are directly interfaced with adaptive actuation units. The units perform motor control and have learning capabilities. This architecture distributes computation over the entire body of the robot, in every sensor and actuator. Clearly, this is similar to biological sensory- motor systems. Some researchers have tried to model the latter in software, again using CPUs. We demonstrate this idea in with an adaptive locomotion controller chip. The locomotory controller for walking, running, swimming and flying animals is based on a Central Pattern Generator (CPG). CPGs are modeled as systems of coupled non-linear oscillators that control muscles responsible for movement. Here we describe an adaptive CPG model, implemented in a custom VLSI chip, which is used to control an under-actuated and asymmetric robotic leg.
We discuss vibration control of a cantilevered plate with multiple sensors and actuators. An architecture is chosen to minimize the number of control and sensing wires required. A custom VLSI chip, integrated with the sensor/actuator elements, controls the local behavior of the plate. All the actuators are addressed in parallel; local decode logic selects which actuator is stimulated. Downloaded binary data controls the applied voltage and modulation frequency for each actuator, and High Voltage MOSFETs are used to activate them. The sensors, which are independent adjacent piezoelectric ceramic elements, can be accessed in a random or sequential manner. An A/D card and GPIB interconnected test equipment allow a PC to read the sensors' outputs and dictate the actuation procedure. A visual programming environment is used to integrate the sensors, controller and actuators. Based on the constitutive relations for the piezoelectric material, simple models for the sensors and actuators are derived. A two level hierarchical robust controller is derived for motion control and for damping of vibrations.
Two systems for velocity-based visual target tracking are presented. The first two computational layers of both implementations are composed of VLSI photoreceptors (logarithmic compression) and edge detection (difference-of-Gaussians) arrays that mimic the outer-plexiform layer of mammalian retinas. The subsequent processing layers for measuring the target velocity and to realize smooth pursuit tracking are implemented in software and at the focal plane in the two versions, respectively. One implentation uses a hybrid of a PC and a silicon retina (39 X 38 pixels) operating at 333 frames/second. The software implementation of a real-time optical flow measurement algorithm is used to determine the target velocity, and a closed-loop control system zeroes the relative velocity of the target and retina. The second implementation is a single VLSI chip, which contains a linear array of photoreceptors, edge detectors and motion detectors at the focal plane. The closed-loop control system is also included on chip. This chip realizes all the computational properties of the hybrid system. The effects of background motion, target occlusion, and disappearance are studied as a function of retinal size and spatial distribution of the measured motion vectors (i.e. foveal/peripheral and diverging/converging measurement schemes). The hybrid system, which tested successfully, tracks targets moving as fast as 3 m/s at 1.3 meters from the camera and it can compensate for external arbitrary movements in its mounting platform. The single chip version, whose circuits tested successfully, can handle targets moving at 10 m/s.
This paper gives an overview of the principles and hardware realizations of artificial neural networks. The first section describes the operation of neural networks, using simple examples to illustrate some of its key properties. Next the different architectures are described, including single and multiple perceptron networks, Hopfield and Kohonen nets. A brief discussion of the learning rules employed in feedforward and feedback networks follows. The final section discusses hardware implementations of neural systems with emphasis on analog VLSI. Different approaches for the realizations of neurons and synapses are described. A brief comparison between analog and digital techniques is given.