intechnguВестник НГУ. Серия: Информационные технологииVestnik NSU. Series: Information Technologies1818-79002410-0420НГУ10.25205/1818-7900-2026-24-1-55-66intechngu-367Research ArticleСтатьиМетоды генерации критических вопросов для анализа аргументации на русском языкеMethods for Generating Critical Questions for Analyzing Russian-Language ArgumentsХайдарД.HaidarD.

Диаб Хайдар, магистрант

Новосибирск

Diab Haidar, Master’s Student

Novosibirsk

k.diab1@g.nsu.ruНовосибирский государственный университетРоссияNovosibirsk State UniversityRussian Federation2026300620262415566Copyright © Хайдар Д., 20262026Хайдар Д.Haidar D.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://intechngu.elpub.ru/jour/article/view/367

Исследование посвящено разработке и реализации модуля генерации критических вопросов к текстам, содержащим аргументацию, с использованием схем аргументации Д. Уолтона и больших языковых моделей. В рамках исследования для проведения экспериментов было разработано два корпуса текстов: англоязычный корпус, состоящий из 200 аргументов, и русскоязычный корпус из 92 текстов, из которых 54 текста являются новостными статьями, а 38 текстов – эссе или аналитическими статьями. Для тестирования подхода к генерации критических вопросов было использовано несколько больших языковых моделей: mistralai/Mistral-7B-Instruct-v0.3, Qwen/ Qwen2.5-7B-Instruct и google/fl an-t5-large. Результаты исследования показали, что модель google/fl an-t5-large имеет лучшие результаты на англоязычном корпусе (совокупный балл – 0,361), а модель mistralai/Mistral-7BInstruct-v0.3 – на русскоязычном корпусе (совокупный балл – 0,285; BERTScore – 0,722). Реализованный модуль генерации критических вопросов может быть использован для построения компоненты интерпретации результатов анализа достоверности информации. 

In this paper, we aim to develop an efficient workflow for critical questions generation from argumentative texts based on argumentation schemes and modern large language models. We have prepared two corpora for this task: an English corpus with 200 argumentative texts and a Russian corpus with 92 texts (54 news texts, 38 essays/analytics). We have also carried out an experiment with three different models for critical questions generation: mistralai/Mistral-7B-Instruct-v0.3, Qwen/Qwen2.5-7B-Instruct, google/flan-t5-large. The results have shown that the google/flan-t5-large model is the best for critical questions generation for the English corpus (composite score 0.361), while for the Russian corpus, it was the best performance of the mistralai/Mistral-7B-Instruct-v0.3 model (composite score 0.285, BERTScore 0.722). The critical questions generation module can be used as a component in information reliability analysis systems. 

генерация критических вопросованализ аргументациибольшие языковые моделисхемы аргументации Д. Уолтонапроверка достоверности информацииобработка естественного языкаинтерпретируемые модели ИИcritical question generationargumentationlarge language modelsWalton’s argumentation schemesargumentation analysisinformation reliability verificationnatural language processinginterpretable AI modelsReferencesWalton D., Reed C., Macagno F. Argumentation Schemes. Cambridge: Cambridge University Press, 2008.Walton D., Reed C., Macagno F. Argumentation Schemes. Cambridge: Cambridge University Press, 2008.Song Y., Heilman M., Beigman Klebanov B., Deane P. Applying argumentation schemes for essay scoring // Proceedings of the First Workshop on Argumentation Mining. Baltimore, Maryland: Association for Computational Linguistics, 2014. P. 69–78.Song Y., Heilman M., Beigman Klebanov B., Deane P. Applying argumentation schemes for essay scoring // Proceedings of the First Workshop on Argumentation Mining. Baltimore, Maryland: Association for Computational Linguistics, 2014. P. 69–78.Gao Y., Xiong Y., Gao X., Jia K., Pan J., Bi Y., Dai Y., Sun J., Wang H. Retrieval-Augmented Generation for Large Language Models: A Survey // arXiv preprint. 2023. arXiv:2312.10997Gao Y., Xiong Y., Gao X., Jia K., Pan J., Bi Y., Dai Y., Sun J., Wang H. Retrieval-Augmented Generation for Large Language Models: A Survey // arXiv preprint. 2023. arXiv:2312.10997Thorne J., Vlachos A., Christodoulopoulos C., Mittal A. FEVER: a large-scale dataset for fact extraction and verifi cation // Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). New Orleans, Louisiana: Association for Computational Linguistics, 2018. P. 809–819.Thorne J., Vlachos A., Christodoulopoulos C., Mittal A. FEVER: a large-scale dataset for fact extraction and verifi cation // Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). New Orleans, Louisiana: Association for Computational Linguistics, 2018. P. 809–819.Soleimani A., Monz C., Worring M. BERT for evidence retrieval and claim verification // Advances in Information Retrieval. Cham: Springer International Publishing, 2020. P. 359–366.Soleimani A., Monz C., Worring M. BERT for evidence retrieval and claim verification // Advances in Information Retrieval. Cham: Springer International Publishing, 2020. P. 359–366.Ruiz-Dolz R., Lawrence J. Detecting argumentative fallacies in the wild: problems and limitations of large language models // Proceedings of the 10th Workshop on Argument Mining. Singapore: Association for Computational Linguistics, 2023. P. 1–10.Ruiz-Dolz R., Lawrence J. Detecting argumentative fallacies in the wild: problems and limitations of large language models // Proceedings of the 10th Workshop on Argument Mining. Singapore: Association for Computational Linguistics, 2023. P. 1–10.Ruiz-Dolz R., Taverner J., Lawrence J., Reed C. NLAS-multi: a multilingual corpus of automatically generated natural language argumentation schemes // arXiv preprint. 2024. arXiv:2402.144.Ruiz-Dolz R., Taverner J., Lawrence J., Reed C. NLAS-multi: a multilingual corpus of automatically generated natural language argumentation schemes // arXiv preprint. 2024. arXiv:2402.144.Chung Y.-L., Kuzmenko E., Tekiroglu S.S., Guerini M. CONAN – COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019. P. 2819–2829.Chung Y.-L., Kuzmenko E., Tekiroglu S.S., Guerini M. CONAN – COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech // Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019. P. 2819–2829.Visser J., Lawrence J., Reed C., Wagemans J., Walton D. Annotating argument schemes // Argumentation. 2021. Vol. 35, No. 1. P. 101Visser J., Lawrence J., Reed C., Wagemans J., Walton D. Annotating argument schemes // Argumentation. 2021. Vol. 35, No. 1. P. 101Lawrence J., Visser J., Reed C. BBC Moral Maze: test your argument // In: COMMA. 2018.Lawrence J., Visser J., Reed C. BBC Moral Maze: test your argument // In: COMMA. 2018.Maertens R., Roozenbeek J., Basol M., van der Linden S. Long-term effectiveness of inoculation against misinformation: three longitudinal experiments // Journal of Experimental Psychology: Applied. 2021. Vol. 27, No. 1. P. 1–16.Maertens R., Roozenbeek J., Basol M., van der Linden S. Long-term effectiveness of inoculation against misinformation: three longitudinal experiments // Journal of Experimental Psychology: Applied. 2021. Vol. 27, No. 1. P. 1–16.Xu F., Lin Q., Han J., Zhao T., Liu J., Cambria E. Are large language models really good logical reasoners? A comprehensive evaluation and beyond // arXiv preprint. 2023. arXiv:2306.09841 cs.Xu F., Lin Q., Han J., Zhao T., Liu J., Cambria E. Are large language models really good logical reasoners? A comprehensive evaluation and beyond // arXiv preprint. 2023. arXiv:2306.09841 cs.Musi E., Carmi E., Reed C., Yates S., O’Halloran K. Developing misinformation immunity: How to reason-check fallacious news in a human–computer interaction environment // Social Media + Society. 2023. Vol. 9, No. 1. Article 20563051221150407.Musi E., Carmi E., Reed C., Yates S., O’Halloran K. Developing misinformation immunity: How to reason-check fallacious news in a human–computer interaction environment // Social Media + Society. 2023. Vol. 9, No. 1. Article 20563051221150407.

The authors declare that there are no conflicts of interest present.