Skip to main content

Multilabel Image Classification via Feature/Label Co-Projection

Publication ,  Journal Article
Wen, S; Liu, W; Yang, Y; Zhou, P; Guo, Z; Yan, Z; Chen, Y; Huang, T
Published in: IEEE Transactions on Systems, Man, and Cybernetics: Systems
November 1, 2021

This article presents a simple and intuitive solution for multilabel image classification, which achieves the competitive performance on the popular COCO and PASCAL VOC benchmarks. The main idea is to capture how humans perform this task: We recognize both labels (i.e., objects and attributes) and the correlation of labels at the same time. Here, label recognition is performed by a standard ConvNet pipeline, whereas label correlation modeling is done by projecting both labels and image features extracted by the ConvNet to a common latent vector space. Specifically, we carefully design the loss function to ensure that: 1) labels and features that co-appear frequently are close to each other in the latent space and 2) conversely, labels/features that do not appear together are far apart. This information is then combined with the original ConvNet outputs to form the final prediction. The whole model is trained end-to-end, with no additional supervised information other than the image-level supervised information. Experiments show that the proposed method consistently outperforms previous approaches on COCO and PASCAL VOC in terms of mAP, macro/micro precision, recall, and F-measure. Further, our model is highly efficient at test time, with only a small number of additional weights compared to the base model for direct label recognition.

Duke Scholars

Published In

IEEE Transactions on Systems, Man, and Cybernetics: Systems

DOI

EISSN

2168-2232

ISSN

2168-2216

Publication Date

November 1, 2021

Volume

51

Issue

11

Start / End Page

7250 / 7259
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Wen, S., Liu, W., Yang, Y., Zhou, P., Guo, Z., Yan, Z., … Huang, T. (2021). Multilabel Image Classification via Feature/Label Co-Projection. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(11), 7250–7259. https://doi.org/10.1109/TSMC.2020.2967071
Wen, S., W. Liu, Y. Yang, P. Zhou, Z. Guo, Z. Yan, Y. Chen, and T. Huang. “Multilabel Image Classification via Feature/Label Co-Projection.” IEEE Transactions on Systems, Man, and Cybernetics: Systems 51, no. 11 (November 1, 2021): 7250–59. https://doi.org/10.1109/TSMC.2020.2967071.
Wen S, Liu W, Yang Y, Zhou P, Guo Z, Yan Z, et al. Multilabel Image Classification via Feature/Label Co-Projection. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2021 Nov 1;51(11):7250–9.
Wen, S., et al. “Multilabel Image Classification via Feature/Label Co-Projection.” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 11, Nov. 2021, pp. 7250–59. Scopus, doi:10.1109/TSMC.2020.2967071.
Wen S, Liu W, Yang Y, Zhou P, Guo Z, Yan Z, Chen Y, Huang T. Multilabel Image Classification via Feature/Label Co-Projection. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2021 Nov 1;51(11):7250–7259.

Published In

IEEE Transactions on Systems, Man, and Cybernetics: Systems

DOI

EISSN

2168-2232

ISSN

2168-2216

Publication Date

November 1, 2021

Volume

51

Issue

11

Start / End Page

7250 / 7259