Skip to main content

Geometric Cross-Modal Comparison of Heterogeneous Sensor Data

Publication ,  Conference
Tralie, CJ; Smith, A; Borggren, N; Hineman, J; Bendich, P; Zulch, P; Harer, J
Published in: Proceedings of the 39th IEEE Aerospace Conference
March 2018

In this work, we address the problem of cross-modal comparison of aerial data streams. A variety of simulated automobile trajectories are sensed using two different modalities: full-motion video, and radio-frequency (RF) signals received by detectors at various locations. The information represented by the two modalities is compared using self-similarity matrices (SSMs) corresponding to time-ordered point clouds in feature spaces of each of these data sources; we note that these feature spaces can be of entirely different scale and dimensionality. Several metrics for comparing SSMs are explored, including a cutting-edge time-warping technique that can simultaneously handle local time warping and partial matches, while also controlling for the change in geometry between feature spaces of the two modalities. We note that this technique is quite general, and does not depend on the choice of modalities. In this particular setting, we demonstrate that the cross-modal distance between SSMs corresponding to the same trajectory type is smaller than the cross-modal distance between SSMs corresponding to distinct trajectory types, and we formalize this observation via precision-recall metrics in experiments. Finally, we comment on promising implications of these ideas for future integration into multiple-hypothesis tracking systems.

Duke Scholars

Published In

Proceedings of the 39th IEEE Aerospace Conference

Publication Date

March 2018
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Tralie, C. J., Smith, A., Borggren, N., Hineman, J., Bendich, P., Zulch, P., & Harer, J. (2018). Geometric Cross-Modal Comparison of Heterogeneous Sensor Data. In Proceedings of the 39th IEEE Aerospace Conference.
Tralie, C. J., A. Smith, N. Borggren, J. Hineman, P. Bendich, P. Zulch, and J. Harer. “Geometric Cross-Modal Comparison of Heterogeneous Sensor Data.” In Proceedings of the 39th IEEE Aerospace Conference, 2018.
Tralie CJ, Smith A, Borggren N, Hineman J, Bendich P, Zulch P, et al. Geometric Cross-Modal Comparison of Heterogeneous Sensor Data. In: Proceedings of the 39th IEEE Aerospace Conference. 2018.
Tralie, C. J., et al. “Geometric Cross-Modal Comparison of Heterogeneous Sensor Data.” Proceedings of the 39th IEEE Aerospace Conference, 2018.
Tralie CJ, Smith A, Borggren N, Hineman J, Bendich P, Zulch P, Harer J. Geometric Cross-Modal Comparison of Heterogeneous Sensor Data. Proceedings of the 39th IEEE Aerospace Conference. 2018.

Published In

Proceedings of the 39th IEEE Aerospace Conference

Publication Date

March 2018