Tree-based BLSTM for mathematical expression recognition

Ting Zhang (1) , Harold Mouchère (1) , Christian Viard-Gaudin (1)

1 LS2N - Laboratoire des Sciences du Numérique de Nantes

In this study, we extend the chain-structured BLSTM to tree structure topology and apply this new network model for online math expression recognition. The proposed system addresses the recognition task as a graph building problem. The input expression is a sequence of strokes from which an intermediate graph is derived using temporal and spatial relations among strokes. In this graph, a node corresponds to a stroke and an edge denotes the relationship between a pair of strokes. Then several trees are derived from the graph and labeled with Tree-based BLSTM. The last step is to merge these labeled trees to build an admissible label graph (LG) modeling 2-D formulas uniquely. The proposed system achieves competitive results in online math expression recognition domain.

Mots clés: Handwritten mathematical recognition; Recurrent neural network; BLSTM

Domaines: Interface homme-machine [cs.HC]; Intelligence artificielle [cs.AI]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Réseau de neurones [cs.NE]

Ting Zhang, Harold Mouchère, Christian Viard-Gaudin. Tree-based BLSTM for mathematical expression recognition. International Conference on Document Analysis and Recognition (ICDAR), Nov 2017, Kyoto, Japan. ⟨10.1109/ICDAR.2017.154⟩. ⟨hal-01576305⟩