AMMCS-2011 Plenary Talk:

Mathematical Modelling in Mathematical Handwriting Recognition

by Stephen M. Watt

University of Western Ontario

Accurate computer recognition of handwritten mathematics offers to provide a natural interface for mathematical computing, document creation and collaboration. Mathematical handwriting, however, provides a number of challenges beyond what is required for the recognition of handwritten natural languages. On one hand, it is usual to use symbols from a range of different alphabets and there are many similar-looking symbols. Mathematical notation is two-dimensional and size and placement information is important. Additionally, there is no fixed vocabulary of mathematical ``words'' that can be used to disambiguate symbol sequences. On the other hand there are some simplifications. For example, symbols do tend to be well segmented. With these characteristics, new methods of character recognition are important for accurate handwritten mathematics input.

We present a geometric theory that we have found useful for recognizing mathematical symbols. Characters are represented as parametric curves approximated by certain truncated orthogonal series. This maps symbols to a low dimensional vector space of series coefficients in which the Euclidean distance is closely related to the variational integral between two curves. This can be used to find similar symbols very efficiently. We describe some properties of mathematical handwriting data sets when mapped into this space and compare classification methods and their confidence measures. We also show how, by choosing the functional basis appropriately, the series coefficients can be computed in real-time, as the symbol is being written and, by using integral invariant functions, orientation-independent recognition is achieved. The beauty of this theory is that a single, coherent view provides several related geometric techniques that give a high recognition rate and that do not rely on peculiarities of the symbol set.

Stephen Watt is a Professor of Computer Science at the University of Western Ontario. He received his PhD from the University of Waterloo in 1986 and held positions at IBM Research (Yorktown Heights) and the University of Nice-Sophia Antipolis/INRIA prior to joining Western. His research lies at the intersection of mathematics and computer science, principally in the areas of computer algebra, pen-based computing and programming languages. He is one of the original authors of the Maple and Axiom computer algebra systems as well as the MathML and InkML web standards.