(Publisher of Peer Reviewed Open Access Journals)

International Journal of Advanced Technology and Engineering Exploration (IJATEE)

ISSN (Print):2394-5443    ISSN (Online):2394-7454
Volume-11 Issue-113 April-2024
Full-Text PDF
Paper Title : Bloom’s Taxonomy based automatic Marathi question generation
Author Name : Pushpa M. Patil, R. P. Bhavsar and B. V. Pawar
Abstract :

In the current era of automation, various fields, including education, are undergoing transformations to enhance their existing processes. One crucial aspect in the field of education is examination management. Automatic question generation (AQG) for creating evaluation systems and question papers represents a significant transformation that schools, colleges, and universities are experiencing. Although significant research has been conducted in AQG for foreign languages, there is a scarcity of such work in Indian regional languages. Considering this, a novel working model for AQG for Marathi language texts was presented. The proposed research generates a diverse set of questions automatically through various natural language processing (NLP) pipeline activities, including tokenization, parts of speech (POS) tagging, stemming, named entity recognition (NER), shallow parsing, and dependency parsing. The generated questions fall into the categories of context-based and grammar-based questions, each elaborated in detail with scientific interpretation. This process contributes to the validation and refinement of our question generation methodology. A benchmarking approach using Bloom's Taxonomy was employed to validate the accuracy of the generated questions, ensuring they were aligned with educational objectives and targeted the desired levels of cognitive complexity. The empirical evaluation of the proposed methodology is conducted using the bilingual evaluation understudy (BLEU) score and manual scoring. The accuracy achieved using the BLEU score is 90.37% for the 'wh' questions, based on the corpus created from the sixth standard science textbook published by the Maharashtra State Board, Maharashtra, India. A diverse set of high-quality Marathi language questions has been successfully curated, suitable for compiling question papers aligned with Bloom's taxonomy levels.

Keywords : Context based questions, Grammar based questions, NLP pipeline, POS, Parsing.
Cite this article : Patil PM, Bhavsar RP, Pawar BV. Bloom’s Taxonomy based automatic Marathi question generation. International Journal of Advanced Technology and Engineering Exploration. 2024; 11(113):575-603. DOI:10.19101/IJATEE.2023.10102163.
References :
[1]Craig J, Cairo IIIL. Assessing the relationship between questioning and understanding to improve learning and thinking (QUILT) and student achievement in mathematics: a pilot study. Appalachia Educational Laboratory at Edvantia (NJ1); 2005.
[Google Scholar]
[2]No TC. 12: deeper learning through questioning. Teaching Excellence in Adult Literacy. 2013.
[Google Scholar]
[3]Nguyen HA, Bhat S, Moore S, Bier N, Stamper J. Towards generalized methods for automatic question generation in educational domains. In European conference on technology enhanced learning 2022 (pp. 272-84). Cham: Springer International Publishing.
[Crossref] [Google Scholar]
[4]Dugan L, Miltsakaki E, Upadhyay S, Ginsberg E, Gonzalez H, Choi D, et al. A feasibility study of answer-agnostic question generation for education. In findings of the association for computational linguistics: ACL 2022 (pp. 1919–26).
[Crossref] [Google Scholar]
[5]Gaikwad DK, Kadam V, Mahender CN. Question based text summarization under rule based framework. International conference on cognitive knowledge engineering 2016 (pp. 332-5).
[Google Scholar]
[6]Shah R, Shah D, Kurup L. Automatic question generation for intelligent tutoring systems. In 2nd international conference on communication systems, computing and IT applications 2017 (pp. 127-32). IEEE.
[Google Scholar]
[7]Heilman M, Smith NA. Good question statistical ranking for question generation. In human language technologies: the annual conference of the North American chapter of the association for computational linguistics 2010 (pp. 609-17). ACL.
[Google Scholar]
[8]Wang Z, Valdez J, Basu MD, Baraniuk RG. Towards human-like educational question generation with large language models. In international conference on artificial intelligence in education 2022 (pp. 153-66). Cham: Springer International Publishing.
[Crossref] [Google Scholar]
[9]Sheu FR, Grissett JO, Hsu YL. Examining students’ perceptions and preferences for traditional paper vs. multiple-choice question generation assignments. In the annual convention of the association for educational communications and technology 2022 (pp. 298-306).
[Google Scholar]
[10]Wang Y, Rong W, Zhang J, Zhou S, Xiong Z. Multi-turn dialogue-oriented pretrained question generation model. Complex & Intelligent Systems. 2020; 6:493-505.
[Crossref] [Google Scholar]
[11]Langi AZ, Rosmansyah Y. Generation quiz with genetic algorithm based on blooms taxonomy classification in serious game based virtual environments. In international conference on ICT for smart society 2016 (pp. 42-8). IEEE.
[Crossref] [Google Scholar]
[12]Das R, Ray A, Mondal S, Das D. A rule based question generation framework to deal with simple and complex sentences. In international conference on advances in computing, communications and informatics 2016 (pp. 542-8). IEEE.
[Crossref] [Google Scholar]
[13]Chali Y, Hasan SA. Towards topic-to-question generation. Computational Linguistics. 2015; 41(1):1-20.
[Crossref] [Google Scholar]
[14]Gaikwad DK, Mahender CN, Namrata C. Question generation system for Marathi text. International Journal of Scientific Research in Computer Science, Engineering and Information Technology. 2018; 3(3): 246-9.
[Google Scholar]
[15]Chen Y, Wu L, Zaki MJ. Toward subgraph-guided knowledge graph question generation with graph neural networks. IEEE Transactions on Neural Networks and Learning Systems. 2023; 1-12.
[Crossref] [Google Scholar]
[16]Yin X, Zhou L, Small K, May J. Summary-oriented question generation for informational queries. In proceedings of the 1st workshop on document-grounded dialogue and conversational question answering 2020 (pp. 81-97). Association for Computational Linguistics.
[Google Scholar]
[17]Fei Z, Zhang Q, Zhou Y. Iterative GNN-based decoder for question generation. In proceedings of the conference on empirical methods in natural language processing 2021 (pp. 2573-82).
[Crossref] [Google Scholar]
[18]Xu Y, Wang D, Yu M, Ritchie D, Yao B, Wu T, et al. Fantastic questions and where to find them:fairytaleqa-an authentic dataset for narrative comprehension. In proceedings of ACL 2022 (pp. 447–60). Association for Computational Linguistics.
[Crossref] [Google Scholar]
[19]Gopal A. Automatic question generation for Hindi and Marathi. In international conference on advanced learning technologies 2022 (pp. 19-21). IEEE.
[Crossref] [Google Scholar]
[20]Fahad AR, Al NN, Islam MA, Rahman RM. Answer agnostic question generation in Bangla language. International Journal of Networked and Distributed Computing. 2024: 1-26.
[Crossref] [Google Scholar]
[21]Jayaseelan KA. Questions and question‐word incorporating quantifiers in Malayalam. Syntax. 2001; 4(2):63-93.
[Crossref] [Google Scholar]
[22]Huber F, Hagel G. Semi-automatic generation of textual exercises for software engineering education. In global engineering education conference 2022 (pp. 51-6). IEEE.
[Crossref] [Google Scholar]
[23]Hou Z, Bi S, Qi G, Zheng Y, Ren Z, Li Y. Syntax-guided question generation using prompt learning. Neural Computing and Applications. 2024: 1-12.
[Crossref] [Google Scholar]
[24]Fei Z, Zhang Q, Gui T, Liang D, Wang S, Wu W, et al. CQG: a simple and effective controlled generation framework for multi-hop question generation. In proceedings of the 60th annual meeting of the association for computational linguistics 2022 (pp. 6896-906). Association for Computational Linguistics.
[Crossref] [Google Scholar]
[25]Steuer T, Filighera A, Tregel T. Investigating educational and noneducational answer selection for educational question generation. IEEE Access. 2022; 10:63522-31.
[Crossref] [Google Scholar]
[26]Wijanarko BD, Heryadi Y, Toba H, Budiharto W. Automated question generating method based on derived keyphrase structures from bloom’s taxonomy. ICIC Express Letters. 2020; 14(11):1059-67.
[Google Scholar]
[27]Liu M, Rus V, Liu L. Automatic Chinese factual question generation. IEEE Transactions on Learning Technologies. 2016; 10(2):194-204.
[Crossref] [Google Scholar]
[28]Leite B, Cardoso HL, Reis LP, Soares C. Factual question generation for the Portuguese language. In international conference on innovations in intelligent systems and applications 2020 (pp. 1-7). IEEE.
[Crossref] [Google Scholar]
[29]Morón M, Scocozza J, Chiruzzo L, Rosá A. A tool for automatic question generation for teaching English to beginner students. In 40th international conference of the Chilean computer science society 2021 (pp. 1-5). IEEE.
[Crossref] [Google Scholar]
[30]Garimella A, Chiticariu L, Li Y. Domain-aware dependency parsing for questions. In findings of the association for computational linguistics: ACL-IJCNLP 2021 (pp. 4562-8).
[Google Scholar]
[31]Kumar NA, Fernandez N, Wang Z, Lan A. Improving reading comprehension question generation with data augmentation and overgenerate-and-rank. In proceedings of the 18th workshop on innovative use of NLP for building educational applications (BEA 2023) (pp. 247–59). Association for Computational Linguistics.
[Google Scholar]
[32]Maheshwari H, Shekhar S, Saxena A, Chhaya N. Open-world factually consistent question generation. In findings of the association for computational linguistics: ACL 2023 (pp. 2390-404).
[Crossref] [Google Scholar]
[33]Das B, Majumder M, Sekh AA, Phadikar S. Automatic question generation and answer assessment for subjective examination. Cognitive Systems Research. 2022; 72:14-22.
[Crossref] [Google Scholar]
[34]Rathi S, Wawage P, Kulkarni A. Automatic question generation from textual data using NLP techniques. In international conference on emerging smart computing and informatics 2023 (pp. 1-7). IEEE.
[Crossref] [Google Scholar]
[35]Patil N, Patil AS, Pawar BV. Issues and challenges in Marathi named entity recognition. International Journal on Natural Language Computing. 2016; 5(1):15-30.
[Google Scholar]
[36]https://www.nltk.org/. Accessed 23 March 2024.
[37]Carlberger J, Kann V. Implementing an efficient part‐of‐speech tagger. Software: Practice and Experience. 1999; 29(9):815-32.
[Crossref] [Google Scholar]
[38]Olney AM, Graesser AC, Person NK. Question generation from concept maps. Dialogue & Discourse. 2012; 3(2):75-99.
[Crossref] [Google Scholar]
[39]Brown J, Frishkoff G, Eskenazi M. Automatic question generation for vocabulary assessment. In proceedings of human language technology conference and conference on empirical methods in natural language processing 2005 (pp. 819-26). Association for Computational Linguistics.
[Google Scholar]
[40]Patil PM, Bhavsar RP, Pawar BV. Automatic question generation for subordinate conjunctions of Marathi. In 8th international conference on advanced computing and communication systems 2022 (pp. 169-173). IEEE.
[Crossref] [Google Scholar]
[41]Agarwal M, Shah R, Mannem P. Automatic question generation using discourse cues. In proceedings of the sixth workshop on innovative use of NLP for building educational applications 2011 (pp. 1-9). Association for Computational Linguistics.
[Google Scholar]
[42]Kalady S, Elikkottil A, Das R. Natural language question generation using syntax and keywords. In proceedings of QG2010: the third workshop on question generation 2010 (pp. 1-10).
[Google Scholar]
[43]Gašpar A, Grubišić A, Šarić-grgić I. Evaluation of a rule-based approach to automatic factual question generation using syntactic and semantic analysis. Language Resources and Evaluation. 2023; 57(4):1431-61.
[Crossref] [Google Scholar]
[44]https://spacy.io/. Accessed 30 June 2024.
[45]Afzal N, Mitkov R. Automatic generation of multiple choice questions using dependency-based semantic relations. Soft Computing. 2014; 18:1269-81.
[Crossref] [Google Scholar]
[46]Bloom BS. Taxonomy of educational objectives: the classification of educational goals: by a committee of college and university examiners. David McKay; 1971.
[Google Scholar]
[47]Karamustafaoglu S, Karamustafaoglu O, Bacanak A, Degirmenci S. Classification of biology exam questions as to bloom. Energy Education Science and Technology Part B: Social and Educational Studies. 2011; 3(4):579-88.
[Google Scholar]
[48]Setyaningsih ER, Listiowarni I. Categorization of exam questions based on bloom taxonomy using naïve bayes and laplace smoothing. In 3rd east Indonesia conference on computer and information technology 2021 (pp. 330-3). IEEE.
[Crossref] [Google Scholar]
[49]Ifham M, Banujan K, Kumara BS, Wijeratne PM. Automatic classification of questions based on blooms taxonomy using artificial neural network. In international conference on decision aid sciences and applications 2022 (pp. 311-5). IEEE.
[Crossref] [Google Scholar]
[50]Gani MO, Ayyasamy RK, Sangodiah A, Fui YT. Bloom’s taxonomy-based exam question classification: the outcome of CNN and optimal pre-trained word embedding technique. Education and Information Technologies. 2023; 28(12):15893-914.
[Crossref] [Google Scholar]
[51]Romadhony A, Abdurohman R. Primary and high school question classification based on blooms taxonomy. In 10th international conference on information and communication technology 2022 (pp. 234-9). IEEE.
[Crossref] [Google Scholar]
[52]Omar N, Haris SS, Hassan R, Arshad H, Rahmat M, Zainal NF, et al. Automated analysis of exam questions according to Blooms taxonomy. Procedia-Social and Behavioral Sciences. 2012; 59:297-303.
[Crossref] [Google Scholar]
[53]Papineni K, Roukos S, Ward T, Zhu WJ. Bleu: a method for automatic evaluation of machine translation. In proceedings of the 40th annual meeting of the association for computational linguistics 2002 (pp. 311-8). ACL.
[Google Scholar]