28 Dec
2015
28 Dec
'15
10:55 a.m.
I am happy to announce that the following paper has been accepted
for publication in the Image and Vision Computing Journal (IVC):
What is a good evaluation protocol for text localization systems?
Concerns, arguments, comparisons and solutions
Stefania Calarasanu (1), Jonathan Fabrizio (1) and Séverine Dubuisson (2)
(1) LRDE-EPITA, 14-16, rue Voltaire, F-94276, Le Kremlin
Bicêtre, France
(2) CNRS, UMR 7222, ISIR, F-75005, Paris, France
Abstract:
A trustworthy protocole is essential to evaluate a text detection algorithm
in order to, first measure its efficiency and adjust its parameters and,
second to compare its performances with those of other algorithms.
However, current protocols do not give precise enough evaluations
because they use coarse evaluation metrics, and deal with inconsistent
matchings between the output of detection algorithms and the ground truth,
both often limited to rectangular shapes. In this paper, we propose a new
evaluation protocol, named EvaLTex, that solves some of the current problems
associated with classical metrics and matching strategies. Our system deals
with different kinds of annotations and detection shapes. It also considers
different kinds of granularity between detections and ground truth objects
and hence provides more realistic and accurate evaluation measures.
We use this protocol to evaluate text detection algorithms and highlight
some key examples that show that the provided scores are more relevant than
those of currently used evaluation protocols.
Ana Stefania Calarasanu
___________________________________________________
PhD Engineer
EPITA Research and Development Laboratory (LRDE)
14-16 rue Voltaire, 94276 Le Kremlin-Bicêtre CEDEX, France
https://www.lrde.epita.fr/wiki/User:Calarasanu
Ana Stefania Calarasanu
___________________________________________________
PhD Student
EPITA Research and Development Laboratory (LRDE)
14-16 rue Voltaire, 94276 Le Kremlin-Bicêtre CEDEX, France
https://www.lrde.epita.fr/wiki/User:Calarasanu