Benchmarking system for document analysis algorithms

Sami Nieminen; Jaakko J. Sauvola; Tapio Seppanen; Matti Pietikainen

doi:10.1117/12.304623

1 April 1998 Benchmarking system for document analysis algorithms

Sami Nieminen, Jaakko J. Sauvola, Tapio Seppanen, Matti Pietikainen

Proceedings Volume 3305, Document Recognition V; (1998) https://doi.org/10.1117/12.304623
Event: Photonics West '98 Electronic Imaging, 1998, San Jose, CA, United States

Abstract

With the increasing interest in document analysis research the number of available OCR, segmentation, noise removal and various other document analysis algorithms has grown considerably. However, algorithms are still purpose- specific, and to obtain optimal results, different algorithms for different situations are usually needed. The problem is to reliably evaluate the performance of an algorithm in a given situation. A framework for a benchmarking system for document analysis algorithms is presented. The system consists of a set of test cases for measuring the performance of different document analysis algorithms. The system is expandable, new algorithm types to be tested can be added by creating new test cases and benchmarking methods. The whole benchmarking process can be automated to allow mass performance testing with numerous algorithms. A set of weights is used to adjust the relative significance of the different aspects of a test case. The results of the benchmarking are expressed as a single value, which presents the performance of the algorithm in a given test case. The result can be easily compared with the results of other algorithms, which enables the ranking of the tested algorithms. Experiments with benchmarking system show promising results. The performance ranking also complies well with subjective human evaluation.

Citation Download Citation

Sami Nieminen, Jaakko J. Sauvola, Tapio Seppanen, and Matti Pietikainen "Benchmarking system for document analysis algorithms", Proc. SPIE 3305, Document Recognition V, (1 April 1998); https://doi.org/10.1117/12.304623

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available