Kanygin A.V. Construction of a Model for the Task of Reasoning Text Classification

Details

Hits: 8

https://doi.org/10.15688/mpcm.jvolsu.2025.1.3

Alexander V. Kanygin
Postgraduate Student, Department of Computer Sciences and Experimental Mathematics, Volgograd State University
This email address is being protected from spambots. You need JavaScript enabled to view it.
Prosp. Universitetsky, 100, 400062 Volgograd, Russian Federation

Abstract. The article addresses the task of classifying texts for the presence of reasoning (logical links, argumentation, cause-and-effect relationships). The aim of the study is to develop a method that allows for highly accurate determination of the “reasoning” nature of a text fragment using modern machine learning algorithms. Particular attention is paid to an ensemble approach based on stacking: strong models (XGBoost, CatBoost, Random Forest, etc.) are considered as base classifiers, while logistic regression serves as the meta-model. To justify the choice of stacking, we present the results of a comparative analysis of more than ten popular algorithms (Logistic Regression, SVC, Random Forest, CatBoost, XGBoost, etc.) by Accuracy, Precision, Recall, F1-score, ROC AUC, and PR AUC. The main stages of the study include the generation and annotation of the training dataset, preliminary text processing (tokenization, lemmatization, stop-word removal), feature vectorization (TF-IDF), and experimental comparison of the models on a control sample. The proposed stacking model showed the best overall performance across all metrics, enabling us to increase the accuracy of reasoning text classification to F1 equal to 0.905 at ROC AUC equal to 0.887.

Key words: machine learning, ensemble methods, stacking, TF-IDF, argumentation, text processing.

Kanygin A.V. Construction of a Model for the Task of Reasoning Text Classification is licensed under a Creative Commons Attribution 4.0 International License.

Citation in English: Mathematical Physics and Computer Simulation. Vol. 28 No. 1 2025, pp. 27-39

Attachments:

kanygin.pdf URL: https://mp.jvolsu.com/index.php/en/component/attachments/download/1248

4 Downloads