Academic Journal of Computing & Information Science, 2021, 4(7); doi: 10.25236/AJCIS.2021.040703.
Kuiliang Gao, Bing Liu, Zhixiang Xue, Xibing Zuo, Yifan Sun, Mofan Dai
PLA Strategic Support Force Information Engineering University, Zhengzhou, China
Different from the existing CNN-based models, a novel method based on the transformer model is proposed in this paper, to further improve the classification accuracy of hyperspectral image (HSI). Specifically, a deep network model is constructed with the Transformer-iN-Transformer (TNT) modules, to carry out end-to-end classification. The outer and inner transformer models in the TNT module can extract the patch-level and pixel-level features respectively, to make full use of the global and local information in the input cubes. Experimental results show that the proposed method can achieve better classification performance than the existing CNN-based models. In addition, using the transformer-baesd deep model without convolution to classify HSI provides a new idea for the related researches.
Hyperspectral image classification, deep learning, transformer, TNT, self-attention
Kuiliang Gao, Bing Liu, Zhixiang Xue, Xibing Zuo, Yifan Sun, Mofan Dai. Deep Transformer Network for Hyperspectral Image Classification. Academic Journal of Computing & Information Science (2021), Vol. 4, Issue 7: 11-17. https://doi.org/10.25236/AJCIS.2021.040703.
[1] N. Audebert, B. L. Saux, and S. Lefevre, "Deep Learning for Classification of Hyperspectral Data: A Comparative Review," IEEE Geoscience and Remote Sensing Magazine, vol. 7, no. 2, pp. 159-173, 2019, doi: 10.1109/MGRS.2019.2912563.
[2] P. Ghamisi, J. Plaza, Y. Chen, J. Li, and A. J. Plaza, "Advanced Spectral Classifiers for Hyperspectral Images: A review," IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 1, pp. 8-32, 2017.
[3] M. J. Khan, H. S. Khan, A. Yousaf, K. Khurshid, and A. Abbas, "Modern Trends in Hyperspectral Image Analysis: A Review," IEEE Access, vol. 6, pp. 14118-14129, 2018.
[4] M. E. Paoletti, J. M. Haut, J. Plaza, and A. Plaza, "Deep learning classifiers for hyperspectral imaging: A review," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 158, no. Dec., pp. 279-317, 2019.
[5] H. Lee and H. Kwon, "Going Deeper With Contextual CNN for Hyperspectral Image Classification," IEEE Trans Image Process, vol. 26, no. 10, pp. 4843-4855, 2017.
[6] Y. X. LIU Bing, ZHANG Pengqiang, TAN Xiong, "Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification," Acta Geodaetica et Cartographica Sinica, vol. 48, no. 1, pp. 53-63, 2019-01-20 2019, doi: 10.11947/j.AGCS.2019.20170578.
[7] S. Shabbir and M. Ahmad, Hyperspectral Image Classification -- Traditional to Deep Models: A Survey for Future Prospects. 2021.
[8] A. Vaswani et al., "Attention Is All You Need," 06/12 2017.
[9] A. Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. 2020.
[10] K. Han, A. Xiao, E. Wu, J. Guo, C. Xu, and Y. Wang, Transformer in Transformer. 2021.
[11] W. Li, G. Wu, F. Zhang, and Q. Du, "Hyperspectral Image Classification Using Deep Pixel-Pair Features," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 2, pp. 844-853, 2017, doi: 10.1109/TGRS.2016.2616355.
[12] H. Zhang, Y. Li, Y. Zhang, and Q. Shen, "Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network," Remote Sensing Letters, vol. 8, no. 5, pp. 438-447, 2017/05/04 2017.
[13] B. Liu, X. Yu, P. Zhang, A. Yu, Q. Fu, and X. Wei, "Supervised Deep Feature Extraction for Hyperspectral Image Classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 4, pp. 1909-1921, 2018.