Enabling machines to read, process, and understand natural language documents is a coveted goal of artificial intelligence. However, this task is extremely challenging, and most existing models lack the ability to perform complex reasoning. Considering that humans often read documents roughly first when understanding a problem, this paper proposes a new model that attempts to mimic the reasoning process of human readers. Our model performs a extensive read and a intensive read of the document separately, and then combines the information obtained from both reading methods to finally find a satisfactory answer. Finally, by experimenting within RACE dataset and comparing with the baseline model BERT, the feasibility and effectiveness of our proposed model can be illustrated.

Guoqi Zhang, Chunlong Yao. Extensive and Intensive: A BERT-based machine reading comprehension model with two reading strategies. Academic Journal of Computing & Information Science (2021), Vol. 4, Issue 2: 66-71. https://doi.org/10.25236/AJCIS.2021.040212.

[1] Hermann K M, Kočiský T, Grefenstette E, et al. Teaching machines to read and comprehend [J]. arXiv preprint arXiv:1506.03340, 2015.

[2] Rajpurkar P, Zhang J, Lopyrev K, et al. Squad: 100,000+ questions for machine comprehension of text [J]. arXiv preprint arXiv:1606.05250, 2016.

[3] Nguyen T, Rosenberg M, Song X, et al. MS MARCO: A human generated machine reading comprehension dataset [C]// CoCo@ NIPS. 2016.

[4] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate [J]. arXiv preprint arXiv:1409.0473, 2014.

[5] Yu A W, Dohan D, Luong M T, et al. Qanet: Combining local convolution with global self-attention for reading comprehension [J]. arXiv preprint arXiv:1804.09541, 2018.

[6] Cui Y, Chen Z, Wei S, et al. Attention-over-attention neural networks for reading comprehension [J]. arXiv preprint arXiv:1607.04423, 2016.

[7] Seo M, Kembhavi A, Farhadi A, et al. Bidirectional attention flow for machine comprehension [J]. arXiv preprint arXiv:1611.01603, 2016.

[8] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding [J]. arXiv preprint arXiv:1810.04805, 2018.

[9] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality [J]. arXiv preprint arXiv:1310.4546, 2013.

[10] Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation [C]// Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014: 1532-1543.

[11] Chen Z, Wu K. ForceReader: a BERT-based Interactive Machine Reading Comprehension Model with Attention Separation [C]//Proceedings of the 28th International Conference on Computational Linguistics. 2020: 2676-2686.

[12] Lai G, Xie Q, Liu H, et al. Race: Large-scale reading comprehension dataset from examinations [J]. arXiv preprint arXiv:1704.04683, 2017.

[13] Chen D, Bolton J, Manning C D. A thorough examination of the cnn/daily mail reading comprehension task [J]. arXiv preprint arXiv:1606.02858, 2016.

[14] Dhingra B, Liu H, Yang Z, et al. Gated-attention readers for text comprehension [J]. arXiv preprint arXiv:1606.01549, 2016.

[15] Wang S, Yu M, Chang S, et al. A co-matching model for multi-choice reading comprehension[J]. arXiv preprint arXiv:1806.04068, 2018.

[16] Tay Y, Tuan L A, Hui S C. Multi-range reasoning for machine comprehension [J]. arXiv preprint arXiv:1803.09074, 2018.