This paper presents a selection of image processing methods and algorithms, which are needed to enable the reliable
automation of robotic tasks at the micro and nanoscale. Application examples are automatic assembly of new nanoscale
electronic elements or automatic testing of material properties. Due to the very small object dimensions targeted here, the
scanning electron microscope is the appropriate image sensor. The methods described in this paper can be categorized
into procedures of object recognition and object tracking. Object recognition deals with the problem of finding and
labeling nanoscale objects in an image scene, whereas tracking is the process of continuously following the movement of
a specific object. Both methods carried out subsequently enable fully automated robotic tasks at the micro- and
nanoscale. A selection of algorithms is demonstrated and found suitable.
This paper describes the implementation of several key components that were used to build a prototype for a
versatile camera system operating in a scanning electron microscope. For the precise alignment of the camera
inside the vacuum chamber, a stick-slip-based actuator was developed, that can create a high torque while having
small dimensions. The camera is mounted on a rail and carriage system and the implemented combination of
absolute and relative optical sensors is described. Finally, several object tracking scenarios are defined and first
results of implemented tracking algorithms are given.
The atomic force microscope (AFM) has proven to be a valuable instrument for the characterization and manipulation of biological objects. When using the AFM as a nanomanipulation tool, two principal problems arise. First, when manipulating with the AFM, the manipulation process has to be performed in a blind way. This can partially be solved by using virtual imaging and force feedback techniques. A second, more challenging problem is caused by tip contamination and the selection of the AFM tip. If the same probe is used for manipulation and imaging, tip contamination can result in decreased image quality. Furthermore, requirements on both tip shape and material may vary for manipulation and imaging. Addressing both problems, an automated microrobot station is proposed, utilizing nanomanipulation robots equipped with self-sensing AFM tips (piezoresistive cantilevers) working in cooperation with a conventional AFM. The system will not only benefit from a decoupling of imaging and manipulation, it will also allow simultaneous measurements (electrical, mechanical and thermal conduction) in different points of the sample. Due to spatial uncertainties arising from thermal drift, hysteresis and creep afflicted actuators, the development of a control system for the cooperation of microrobot and AFM is challenging. Current research efforts towards a nanohandling robot station combining both an AFM cantilever equipped microrobot and an AFM are presented.
This paper presents a nanohandling robot cell with flexible visual feedback designed to work inside an SEM's vacuum
chamber in order to support teleoperated and fully automated nanohandling. Rail-based robots position miniature video
microscopes that observe the handling from different angles and with different magnifications. Image processing
techniques can be used to recognize and track objects and three-dimensional information can be obtained by stereo
vision and by the microscope's focus. The feasibility and advantages of the CameraMan concept are analyzed by the
implementation of a robot cell prototype. A self-learning controller is used to control the non-linear parts of the system,
challenges for cooperatively controlling the multi-robot system are outlined and high-level automation is discussed.
Current research work on the development of automated microrobot-based nanohandling stations (AMNSs) using the probe of an atomic force microscope (AFM) as an endeffector is presented. The manipulation of individual multiwalled carbon nanotubes (MWCNTs) and the characterization of eukaryotic cells are aspired applications. For this reason, the developed AMNSs have to be integrated both into a scanning electron microscope (SEM) for the nanomanipulation of carbon nanotubes (CNTs) and into an optical microscope for the cell characterization. Such an AMNS combines different micro- and nanomanipulators, each offering three degrees of freedom (DoF), in order to perform the coarse and fine positioning between object and endeffector. Piezoresistive AFM probes are applied as an endeffector allowing to measure the acting forces and to realize a force feedback for the station's control system. First investigations have been carried out by bending of MWCNTs and calculating the Young's modulus of a MWCNT. Electrically conductive adhesives (ECAs) have been developed for the microelectronics industry, and their mechanical properties have to be determined. Therefore an AMNS for the mechanical characterization of thin ECA coatings by nanoindentation inside an SEM is presented as well, showing first experimental results.
This paper presents a reference pattern-based two-dimensional (2D) measurement method. In the method, surface structure patterns obtained from a four-beam laser interference lithography (LIL) process were used as reference patterns for 2D measurement. The reference patterns played the role of 2D rulers in the measurement. The nano resolution of the measurement was achieved by feature counting and pattern matching techniques. A statistical analysis indicates that the measurement made by pattern matching has the advantage of averaging noise. For reference pattern-based 2D measurement, the reference patterns can be regular or irregular. This approach is potentially useful for micro and nano manipulation in the processes of assembly, packaging and manufacturing of nano and micro-systems when relative nano positioning accuracy is required.
In this paper, two image processing approaches are presented, which are used to gain vision feedback for automatic nanohandling inside a Scanning Electron Microscope (SEM). The first one is a vision-based force measurement that makes use of an active contours tracking algorithm for real-time tracking of the bending line of micro- and nanoobjects. With this algorithm, it is possible to calculate applied forces in real-time with respect to the image acquisition time. This approach is validated using a piezo-resistive force sensor. In a second experiment the force applied to a Si nanowire (d ≈ 470 nm) is measured. The second visual measurement approach deals with the calculations of depth information inside an SEM by means of stereoscopic images. Therefore, a new 3D-imaging system that uses a stereo algorithm based on a biologically motivated energy model is proposed. The system provides a sharp and high density disparity map in sub-pixel accuracy and a 3D-plot for the user.
This paper describes the implementation of smart materials in actuators which are employed by mobile microrobots. A versatile microrobot is able to perform complex handling and joining procedures of microobjects with high precision. Such a microrobot requires precise actuators made of specific smart materials. These materials enable the microrobots to fulfill movements in coarse- and fine-positioning mode.
The actuator material, currently most frequently used for the coarse-positioning mode, is a piezoelectric ceramic. It is important for the coarse-positioning actuators to be able to carry the entire weight of the microrobots, to provide a velocity of several cm/s and a positioning accuracy in a range of a few μm.
The actuators for a fine-positioning unit have to provide an accuracy in the range of a few nanometers. The workspace of a fine-positioning unit depends on a task and is in between a few μm3 and several cm3.
For this use, piezoelectric ceramics are suitable as well. Furthermore, the employment of ferrofluids is promising.
Microrobots are the result of increasing research activities at the border between microsystem technology and robotics. Today already, robots with dimensions of a few cubic- centimeters can be developed. Like conventional robots, microrobots represent a complex system that usually contains several different types of actuators and sensors. The measurement of gripping forces is the most important sensor application in micromanipulation besides visual servoing to protect the parts from too high surface pressures and thereby damage during the assembly process. Very small forces in the range of 200 (mu) N down to 0.1 (mu) N or even less have to be sensed. Thus, the aim of our current research activities is the development of a high-resolution integrated force microsensor for measuring gripping forces in a microhandling robot. On the one hand, the sensor should be a device for teleoperated manipulation tasks in a flexible microhandling station. On the other hand, typical microhandling operations should to a large extend be automated with the aid of computer-based signal processing of sensor information. The user should be provided with an interface for teleoperated manipulation and an interface for partially automated manipulation of microobjects. In this paper, a concept for the measurement of gripping forces in microrobotics using piezoresistive AFM (atomic force microscope) cantilevers is introduced. Further on, the concept of a microrobot-based SEM station and its applications are presented.
In the scanning electron microscope (SEM), specially designed microrobots can act as a flexible assembly facility for prototype microsystems, as probing devices for in-situ tests in various applications or just as a helpful teleoperated tool for the SEM operator when examining a few samples. Several flexible microrobots of this kind have been developed and tested. Driven by piezoactuators, these few cubic centimeters small mobile robots perform manipulations with a precision of up to 20 nm and transport the gripped objects at speeds of up to 3 cm/s. New microrobot prototypes being employed in the SEM are described in this paper. The SEM's vacuum chamber has been equipped with various elements to enable the robots to operate. In order ot use the SEM image for automatic real-time control of the robots, the SEM's electron beam is actively controlled by a PC. The latter submits the images to the robots' control computer s ystem. For obtaining three- dimensional information in real time, a triangulation method with the luminescent spot of the SEM's electron beam is being investigated. Finally, the strategies of a micro force sensing and control methods required for handling techniques with two robots are discussed.
KEYWORDS: Scanning electron microscopy, Robots, Microscopes, Control systems, Electron beams, Light emitting diodes, Sensors, Electron microscopes, CCD cameras, Computing systems
In the scanning electron microscope (SEM), specially designed microrobots can act as a flexible assembly facility for hybrid microsystems, as probing devices for in-situ tests on IC structures or just as a helpful teleoperated tool for the SEM operator when examining samples. Several flexible microrobots of this kind have been developed and tested. Driven by piezoactuators, these few cubic centimeters small mobile robots perform manipulations with a precision of up to 10 nm and transport the gripped objects at speeds of up to 3 cm/s. In accuracy, flexibility and price they are superior to conventional precision robots. A new SEM-suited microrobot prototype is described in this paper. The SEM's vacuum chamber has been equipped with various elements like flanges and CCD cameras to enable the robot to operate. In order to use the SEM image for the automatic real-time control of the robots, the SEM's electron beam is actively controlled by a PC. The latter submits the images to the robots' control computer system. For obtaining three-dimensional information in real time, especially for the closed-loop control of a robot endeffector, e.g. microgripper, a triangulation method with the luminescent spot of the SEM's electron beam is being investigated.
The assembly of complex microsystems consisting of several single components (i.e. hybrid microsystems) is a difficult task that is seen to be a real challenge for the robotics research community. It is necessary to conceive flexible, highly precise and fast microassembly devices and methods. In this paper, the development of a microrobot-based microassembly station is presented. Mobile piezoelectric microrobots with dimensions of some cm3 and with at least 5 DOF can perform various manipulations either under a light microscope or within the vacuum chamber of a scanning electron microscope. The control system of the microassembly station is described. The main attention is given to a vision-based sensor system for automatic robot control and a re-configurable parallel computer array enabling the station to work in real-time.
There is an increasing interest in performing assembly of microsystems (i.e. non-destructive transportation, precise manipulation, and exact positioning of microcomponents) by flexible microrobots. A microrobot-based microassembly desktop station is being developed at the University of Karlsruhe. Several prototypes of piezoelectric driven microrobots and a design of the flexible microassembly desktop station were already presented at the last years' SPIE-meetings. In this paper, some implementation results of the station's planning and control system are presented. On the planning level, a common microassembly model for a computer-aided assembly planning is suggested-which is based on geometric reasoning--and its components are discussed in detail. The feasibility criteria for the generation of feasible assembly sequences and the optimization criteria for selecting the optimal assembly plan are described. For stations employing several microrobots, a method for decomposition of an assembly plan is suggested. Since the station's microrobots are rather complicated systems, it is very hard to find a useful robot model for the control purposes. For this reason, control methods have to be used for positioning of a microrobot, which do not require an exact system model and which allow a reasonable compromise between the real-time processing and the exactness. An intelligent neural controller for positioning a microrobot has been developed.
There is an increasing interest in performing microsystem assembly using flexible microrobots. A new concept of a flexible robot-based micro-assembly desktop station and tow prototypes of piezoelectric microassembly robots, MINIMAN and PROHAM, were already presented at the last year's SPIE meeting. In this paper, the motion control approach of these robots is discussed. This control approach is based on the geometric description of the robot platform and aims at following the optimal motion trajectory to minimize the operation time and to keep the robot end effector under microscope supervision. Besides excellent abilities both robots have some disadvantages such as the relatively high drive voltage of the piezoactuators or the instability of grasp-and-hold operations. For this reason, several new piezoelectric microrobots that employ different locomotion and object handling principle have lately been developed. The design and functions of these microrobots are shown.
One of the main problems of present-day research on microsystem technology (MST) is to assemble a whole micro- system from different microcomponents. This paper presents a new concept of an automated micromanipulation desktop- station including piezoelectrically driven microrobots placed on a high-precise x-y-stage of a light microscope, a CCD-camera as a local sensor subsystem, a laser sensor unit as a global sensor subsystem, a parallel computer system with C167 microcontrollers, and a Pentium PC equipped additionally with an optical grabber. The microrobots can perform high-precise manipulations (with an accuracy of up to 10 nm) and a nondestructive transport (at a speed of about 3 cm/sec) of very small objects under the microscope. To control the desktop-station automatically, an advanced control system that includes a task planning level and a real-time execution level is being developed. The main function of the task planning sub-system is to interpret the implicit action plan and to generate a sequence of explicit operations which are sent to the execution level of the control system. The main functions of the execution control level are the object recognition, image processing and feedback position control of the microrobot and the microscope stage.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.