a unified evaluation scheme
Stefania Calarasanu (1), Jonathan Fabrizio (1) and Séverine Dubuisson (2)
(1) LRDE-EPITA, 14-16, rue Voltaire, F-94276, Le Kremlin
Bicêtre, France
(2) CNRS, UMR 7222, ISIR, F-75005, Paris, France
Abstract:
Current text segmentation evaluation protocols are often incapable of
properly handling different scenarios (broken/merged/partial characters).
This leads to scores that incorrectly reflect the segmentation accuracy.
In this article we propose a new evaluation scheme that overcomes most
of the existent drawbacks by extending the EvaLTex protocol (initially
designed to evaluate text detection at region level). This new unified platform
has numerous advantages: it is able to evaluate a text understanding system
at every detection stage and granularity level (paragraph/line/word and now
character) by using the same metrics and matching rules; it is robust to all
segmentation scenarios; it provides a qualitative and quantitative evaluation
and a visual score representation that captures the whole behavior of a
segmentation algorithm. Experimental results on nine segmentation algorithms
using different evaluation frameworks are also provided to emphasize the
interest of our method.