## Abstract

Distance between sequences is structural by nature because it needs to establish the temporal alignments among the temporally correlated vectors in sequences with varying lengths. Generally, distances for sequences heavily depend on the ground metric between the vectors in sequences to infer the alignments and hence can be viewed as meta-distances upon the ground metric. Learning such meta-distance from multi-dimensional sequences is appealing but challenging. We propose to learn the meta-distance through learning a ground metric for the vectors in sequences. The learning samples are sequences of vectors for which how the ground metric between vectors induces the meta-distance is given. The objective is that the meta-distance induced by the learned ground metric produces large values for sequences from different classes and small values for those from the same class. We formulate the ground metric as a parameter of the meta-distance and regress each sequence to an associated pre-generated virtual sequence w.r.t. the meta-distance, where the virtual sequences for sequences of different classes are well-separated. Wedevelop general iterative solutions to learn both the Mahalanobis metric and the deep metric induced by a neural network for any ground-metric-based sequence distance. Experiments on several sequence datasets demonstrate the effectiveness and efficiency of the proposed methods.

Original language | English (US) |
---|---|

Pages (from-to) | 286-301 |

Number of pages | 16 |

Journal | IEEE Transactions on Pattern Analysis and Machine Intelligence |

Volume | 44 |

Issue number | 1 |

DOIs | |

State | Published - Jan 2022 |

### Funding

The authors would like to thank the associate editor and anonymous reviewers for their valuable comments. This work was supported in part by the National Natural Science Foundation of China No. 61976206, No. 61832017, and No. 61603373, Beijing Outstanding Young Scientist Program NO. BJJWZYJH012019100020098, Youth Innovation Promotion Association CAS No. 2019110, and US National Science Foundation grant IIS-1619078, IIS-1815561.

## Keywords

- Metric learning
- optimal transport
- temporal alignment
- virtual sequence regression

## ASJC Scopus subject areas

- Software
- Artificial Intelligence
- Applied Mathematics
- Computer Vision and Pattern Recognition
- Computational Theory and Mathematics