Assessing psychometric properties of the serbian version of the metacognitive awareness inventory (MAI)

Introduction: Metacognition, which refers to the ability to oversee and regulate one’s cognitive activities, plays a crucial role in medical education. Evaluating metacognitive awareness is particularly important for university students, as it impacts their learning strategies and academic performance. The objective of this study was to examine the psychometric characteristics of the Serbian adaptation of the Metacognitive Awareness Inventory (MAI), with an emphasis on its construct validity and internal consistency.

Materials and methods: This was a cross-sectional study conducted at the Faculty of Medicine University of Belgrade, on third-year medical students during October 2024. The psychometric properties of the Serbian version of the MAI were evaluated by assessing its factorial structure and internal consistency. The MAI consists of 52 items, comprising two primary components: ‘Knowledge of Cognition’ and ‘Regulation of Cognition’. The subcomponent of “knowledge of cognition” were categorized into declarative knowledge, procedural knowledge, and conditional knowledge, whereas “regulation of cognition” subcomponent were classified into planning, information management strategies, comprehension monitoring, debugging, and evaluation.

Results: A total of 426 medical students were included in the study. The mean age of the respondents was 21.3 ± 1.4 years, and the majority were females (68.5%). The mean score of the scale was 199.0. The internal consistency analysis of the Serbian version of the MAI questionnaire yielded a Cronbach’s α of 0.94, and a McDonald’s ω of 0.95 for the entire scale, which indicates excellent scale reliability. The Serbian version of the MAI questionnaire was validated using confirmatory factor analysis, supporting both the hypothesized eight-factor and two-factor structures. The eight-factor model showed good fit indices (IFI = 0.908, CFI = 0.905, RMSEA = 0.042), with statistically significant factor loadings (p < 0.05). Similarly, the two-factor model demonstrated adequate fit (IFI = 0.921, CFI = 0.918, RMSEA = 0.039), with all loadings also significant (p < 0.05).

Conclusion: The present study revealed the Serbian version of MAI as reliable and valid tool for identifying medical students’ metaconginive awareness.

INTRODUCTION

In recent decades, metacognition has garnered substantial attention within the field of cognitive psychology, emerging as a central topic of research and discussion. Initially, the concept of metacognition remained largely unexplored; however, recent advancements in cognitive science have underscored its crucial role in various cognitive processes [1]. Metacognition refers to the ability to reflect upon, understand, and regulate one’s cognitive processes [2]. Metacognition can be defined as: ‘cognition about cognition’ [3], ‘thinking about thinking’ [4], or ‘knowing about knowing’ [5]. The concept of metacognition includes two main components: ‘Knowledge of Cognition’ and ‘Regulation of Cognition’ [6].

Individuals demonstrating higher metacognitive awareness and skills excel in various cognitive tasks, especially those necessitating effective planning, information management, monitoring, debugging, and evaluation. They manage information more efficiently by actively selecting, organizing, and prioritizing relevant data, which enhances their problem-solving capabilities [7].

The metacognitive process plays a pivotal role in medical education, as it significantly enhances critical thinking, improves the accuracy of clinical judgment, reduces diagnostic errors, and fosters reflective practices [8]. Furthermore, research indicates that students with higher metacognitive awareness achieve superior outcomes across a range of academic domains, including problem-based learning, language acquisition, collaboration, collaborative learning, and communication – skills essential for medical training [9],[10]. Consequently, assessing metacognitive awareness becomes a critical concern for university students, as it directly influences their ability to manage learning effectively and achieve their academic goals [11].

To address this, Schraw and Dennison introduced the Metacognitive Awareness Inventory (MAI) – a simple and efficient tool for evaluating metacognitive awareness, moving beyond traditional metacognitive judgments as a means of assessing metacognitive skills [2]. However, despite the growing recognition of the importance of metacognitive skills in higher education, no standardized tool exists to specifically measure the metacognitive awareness of Serbian university students. Given that the MAI has not yet been validated for the Serbian population, the objective of this study was to evaluate the psychometric properties of the Serbian version of the MAI, with particular focus on its construct validity and its internal consistency.

MATERIALS AND METHODS

Study design

This cross-sectional study was conducted at the Institute for Medical Statistics and Informatics of the University of Belgrade Faculty of Medicine in October 2024. Participation in the study was voluntary, and the confidentiality of all respondents was observed. Ethical approval (No: 25/IX-3) was obtained from the University of Belgrade Faculty of Medicine Ethics Committee. The study was conducted within the ERASMUS+ project: Development and Implementation of Metacognitive Problem-based Modules in Blended Learning Courses in Medical Sciences (ProBLeMS).

Study participants The study population comprised third-year medical students enrolled at the University of Belgrade Faculty of Medicine. The sample size for analyzing the psychometric properties of the metacognitive awareness assessment questionnaire was determined using the N:q rule (N: number of respondents; q: number of questionnaire items). For the defined value of q = 52 and the value of 5 for the N:q ratio, a minimum of 260 respondents were planned to be included in the research.

Instrument

The Metacognitive Awareness Inventory (MAI) is a 52- item self-report tool comprising two primary dimensions: ‘Knowledge of Cognition’ (17 items) and ‘Regulation of Cognition’ (35 items) [2]. The subcomponents of ‘Knowledge of Cognition’ were categorized as follows: declarative knowledge (DK – self-awareness and strategies; 8 items), procedural knowledge (PK – application of procedures; 4 items), and conditional knowledge (CK – knowledge of when and how to apply strategies; 5 items). The subcomponents of ‘Regulation of Cognition’ were classified as: planning (P – goal-setting; 7 items), information management strategies (IMS – implementation strategies; 10 items), comprehension monitoring (CM – self-monitoring, 7 items), debugging strategies (DS – error correction; 5 items), and evaluation (E – performance evaluation; 6 items). The instrument employs a five-point Likert scale, with values spanning from 1 (always false) to 5 (always true). The total score ranges from 52 to 260, with higher scores indicating higher awareness of cognition and its regulation [2]. Before the beginning of the study, we obtained formal approval from the author of the questionnaire, who authorized the validation and use of the instrument. A comprehensive structural description of the MAI questionnaire is available in Table 1.

Table 1. A comprehensive structural description of the MAI questionnaire

Translation and cultural adaptation

The translation and cultural adaptation of the MAI questionnaire were carried out according to well-established, internationally recognized guidelines, which encompass several stages, so as to ensure the validity and reliability of the translated instrument [12]. The first stage involved the initial translation of the questionnaire from its original English version into the version in Serbian. This was followed by a harmonization process wherein the translated version was reviewed and refined to ensure clarity and appropriateness of the language for the Serbian-speaking population. Subsequently, the Serbian version was translated back into English to check for consistency and alignment with the original content, and another round of harmonization was conducted to address any discrepancies. To ensure the cultural and conceptual relevance of the final version of this instrument, a group of experts, including linguists and subject-matter specialists, engaged in a detailed discussion of the translations. Their collective input led to the preparation of the final version of the questionnaire, which was deemed suitable for use in the research context. The evaluation of the psychometric properties of the MAI questionnaire in the Serbian language was based on the evaluation of the following: the acceptability of the questionnaire, the application of factor analysis, and the analysis of the reliability of the scale.

The questionnaire was administered through the Moodle platform as part of the Medical Statistics and Informatics course. Prior to participation, students were thoroughly informed about the objectives and scope of the study, ensuring that they understood the context and significance of the research. Following ethical research practices, participants were asked to provide informed consent, confirming their voluntary participation in the study and their understanding of their rights, including confidentiality and the option to withdraw at any stage without consequence. Once informed consent was obtained, students proceeded to complete the questionnaire, which was designed to collect data relevant to the research objectives.

Statistical analysis

Categorical variables were summarized using frequencies and percentages, while continuous variables were described using mean values and values of standard deviation (SD), for data following a normal distribution, or as median values, with minimum and maximum values for data not normally distributed. The psychometric evaluation of the Serbian version of the MAI included assessments of validity and reliability. Confirmatory factor analysis (CFA) was conducted to verify the original eight-factor structure of the MAI. The model fit was assessed using the chi-square test (with p > 0.05 suggesting a good fit) along with three additional indices: the comparative fit index (CFI), the incremental fit index (IFI), and the root mean square error of approximation (RMSEA). An adequate model fit was determined by CFI and IFI values exceeding 0.9 and RMSEA values below 0.06. Reliability was analyzed by calculating internal consistency using Cronbach’s α and McDonald’s ω coefficients (values closer to 1 denote higher reliability). Two-tailed statistical tests were performed, with significance set at p < 0.05. Analyses were carried out using the Amos 21, IBM SPSS Statistics 24 software (IBM Corporation, Armonk, NY, USA), and the R software (Version 4.3.3, R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

A total of 426 medical students, who provided complete responses to the questionnaire, were included in the study. The mean age of the respondents was 21.3 ± 1.4 years, and the majority were females (68.5%).

Table 2. presents the average scores and reliability statistics for each item, as well as the overall measurement of metacognitive awareness among medical students. The mean score on the scale was 199.0. The mean scores of the subcategories were as follows: 30.3 for declarative, 15.2 for procedural, and 19.0 for conditional knowledge; 26.2 for the planning subscale, 39.5 for information management strategies, 26.5 for comprehension monitoring, 20.5 for debugging strategies, and 21.9 for the evaluation subscale. The Serbian version of the MAI questionnaire demonstrated excellent reliability, with Cronbach’s α of 0.94 for the total scale. Subscale reliability ranged from 0.57 for comprehension monitoring to 0.81 for information management strategies. Similarly, McDonald’s ω for the overall scale was 0.95, confirming high reliability, with subscale values ranging between 0.67 (comprehension monitoring) and 0.86 (information management strategies) (Table 2).

Table 2. Average scores and reliability statistics

The Serbian version of the MAI questionnaire was evaluated using maximum likelihood confirmatory factor analysis for both the eight-factor and two-factor models. The findings supported a good fit for the hypothesized eight-factor structure. Although the chisquare test indicated a lack of fit for the eight-factor model (χ² = 1,851.5; p < 0.001; χ²/df = 1.73), this was anticipated due to the large sample size. Fit indices, including IFI (0.908) and CFI (0.905), were above the recommended threshold of 0.90, while the RMSEA value (0.042; 90% CI: 0.038–0.045) was within the acceptable range of ≤ 0.06. All standardized factor loadings were significant (p < 0.05), (Figure 1). Similarly, the two-factor model showed comparable results. The chi-square test again suggested а poor fit (χ² = 1,742.4; p < 0.001; χ²/df = 1.64), which was attributed to the sample size, while the IFI (0.921) and CFI (0.918) values exceeded 0.90, and the RMSEA value (0.039; 90% CI: 0.035–0.042) met the recommended standard. All factor loadings in this model were also statistically significant (p < 0.05), (Figure 2).

Figure 1. Standardized factor loadings for the MAI questionnaire in the Serbian language for the eight-factor model
Legend: DK – declarative knowledge; PK – procedural knowledge; CK – conditional knowledge; P – planning; IMS – information management strategies; CM – comprehension monitoring; DS – debugging strategies; E – evaluation

Figure 2. Standardized factor loadings for the MAI questionnaire in the Serbian language for the two-factor model

DISCUSSION

The Serbian adaptation of the MAI questionnaire was validated in a sample of undergraduate medical students. The findings demonstrated that the instrument possesses adequate validity and reliability. These results indicate that the Serbian version of the MAI is a reliable and effective tool for assessing metacognitive awareness in this population.

The International Group on Metacognition conducted a validation study of the MAI with a standardized Spanish version, involving 1,622 undergraduate students from 12 Spanish-speaking countries. The study sought to evaluate the reliability and cross-cultural applicability of the instrument. Findings supported a robust baseline confirmatory factor analysis model, consistent across all participating countries, reinforcing the original two-factor structure identified in English-speaking samples from the United States. Multigroup measurement invariance analyses were carried out to assess the equivalence of the MAI across diverse cultural contexts. Although minor variations were observed in five parameters among some countries, chi-square difference tests showed that allowing these parameters to vary did not significantly improve the model fit, as compared to a fully constrained model. This confirmed the measurement invariance of the MAI across Spanish-speaking populations, indicating that the instrument can be reliably used in different cultural settings without significant bias or alterations in its properties [13].

In comparison with the study by Omprakash et al. [14], which examined metacognitive awareness among first-year medical and dental students in India and identified a six-factor structure after removing 12 low-loading items, our analysis of the Serbian version of the MAI questionnaire using maximum likelihood estimation revealed an eight-factor structure. While Omprakash et al. confirmed an acceptable, though not ideal model, using exploratory factor analysis (EFA), with somewhat lower fit indices (CFI = 0.78; GFI = 0.80; AGFI = 0.77), our findings showed that, although the chi-square test rejected the eight-factor model (χ² = 1,851.5; p < 0.001; χ²/df = 1.73), as anticipated due to the large sample size, other fit indices were satisfactory. Specifically, the IFI (0.908) and CFI (0.905) exceeded the recommended cutoff value of ≥ 0.90, and the RMSEA amounting to 0.042 (0.038–0.045) fell below the suggested threshold of ≤ 0.06. These results indicate that, compared to the previous study, our Serbian version of the MAI exhibits better fit characteristics and a more stable multidimensional structure, thereby supporting the hypothesis that metacognitive awareness may be organized in a more complex manner within this particular sample.

A systematic review of studies utilizing the MAI questionnaire reveals a wide range of findings regarding factor structures, ranging from unidimensional structures to those with eight components. The results confirm that self-reports are most suitable for measuring two primary factors of metacognition – knowledge and regulation [15]. Schraw and Denison’s original study indicates stronger evidence supporting the two-factor structure (knowledge of cognition and regulation of cognition) than the structure with eight subcomponents [2].

Li et al. recently conducted a study involving 592 nursing students from four universities in different regions of China [16]. Their findings demonstrated that the 52-item Chinese version of the Metacognitive Awareness Scale (C-MAS) is a valid and reliable instrument. The scale showed strong content validity, acceptable internal consistency, and solid construct validity, confirming its utility for assessing metacognitive awareness among Chinese nursing students. According to the study, the 52 items of the C-MAS were organized into eight factors, forming a first-order model. This model exhibited excellent internal consistency, with a Cronbach’s α of 0.963 and factor loadings ranging from 0.875 to 0.992. The eight factors were further grouped into two broader dimensions, resulting in a second-order model. For this model, a Cronbach’s α of 0.800 and factor loadings between 0.756 and 0.788 were reported, indicating good reliability. Both the first-order and second-order models of the C-MAS were found to be robust, providing reliable tools for evaluating metacognitive awareness.

Similarly, in our study, the Serbian version of the MAI demonstrated excellent reliability, with a Cronbach’s α of 0.94. Subscale reliability ranged from 0.57 (comprehension monitoring) to 0.81 (information management), thus further confirming the scale’s strong internal consistency. The high internal consistency of the total score confirms that the items measure a unified construct. Similar findings have been reported in other studies [2],[15]. Lower values of internal consistency estimated using Cronbach’s α for individual factors may be attributed to linguistic and cultural differences between the original and translated versions of the questionnaire, which is common when adapting instruments for new populations. Additionally, the number of items within the factors varied between 4 and 10. It is well-known that a smaller number of items tends to produce lower internal consistency values, which is a limitation of the applied metric. An alternative metric, McDonald’s ω, which is more robust in estimating internal consistency, demonstrated satisfactory values [17],[18],[19].

Cross-cultural translation and psychometric assessment of a questionnaire play a crucial role in advancing scientific understanding of metacognitive awareness. Not only does this process improve the accuracy and validity of assessment tools but it also provides a foundation for exploring metacognitive awareness in various fields and professions. The results of our study align with the findings of Li et al. [15], which emphasize the importance of explicitly defining and quantifying cognitive knowledge and cognitive regulation. This is essential for the development of effective educational intervention plans and strategies.

Future studies can apply the Serbian version of the MAI questionnaire to assess the metacognitive awareness of medical students across different regions of Serbia and students majoring in other areas. With appropriate adaptations for variations in dialect, the instrument could also serve as a reliable and valid tool in the Montenegrin, Bosnian, and Croatian educational systems. Such studies would enhance our understanding of the similarities and differences in metacognitive awareness among various student populations. Moreover, the use of the Serbian version of the MAI questionnaire as a reliable assessment tool can inform the improvement and refinement of educational programs with the purpose of enhancing metacognitive skills, as well as help researchers in designing and implementing more targeted and effective educational strategies, ultimately improving learning outcomes.

CONCLUSION

The present study showed the Serbian version of the MAI questionnaire to be a reliable and valid tool for assessment the metacognitive awareness of medical students.

APPENDIX 1

UPITNIK ZA PROCENU METAKOGNITIVNE SVESNOSTI

Zamislite sebe dok učite. Pažljivo pročitajte svaku od navedenih tvrdnji. Svaku od njih označite u kojoj meri se odnosi na vas kada učite, tako što ćete zaokružiti odgovarajući broj: (1) Uvek netačno (2) Ponekad netačno (3) Neutralno (4) Ponekad tačno (5) Uvek tačno