www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Original scientific paper

Received: November 10, 2023.

Revised: March 12, 2024.

Accepted: March 20, 2024.

UDC:

378.147.091.33-027.22

159.9.072-057.875

10.23947/2334-8496-2024-12-1-41-55

Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

*Corresponding author: maravena@ucm.cl

Noemí Cárcamo-Mansilla1 , María D. Aravena-Díaz1*

Metacognitive Strategies for Mathematical Modeling with

Engineering Groups of Students: Adaptation and Validation of a

Questionnaire

Faculty of Basic Sciences, Catholic University of Maule, Talca, Chile; Mathematical and Statistical Education Research

Center, Catholic University of Maule, Talca, Chile

e-mail:

ncarcamo@ucm.cl

maravena@ucm.cl

Abstract: A sequential exploratory mixed-methods study is implemented to develop an instrument that allows for the

evaluation of the metacognitive strategies used by engineering groups of students when solving mathematical modeling problems.

The findings of the qualitative study guided by observations and interviews reveal the use of metacognitive strategies of ‘planning’,

‘monitoring and, if necessary, regulation’, and ‘evaluation’. In this article, we present the final categories of the qualitative analysis and

discuss how these data were shaped into a theoretical construct and items of an instrument to measure metacognitive strategies. The

psychometric properties of the instrument are analyzed, and it is argued that it has a similar interpretation among males and females,

as there are no significant differences in these results. The development of the present study demonstrates how the qualitative

method can support the adaptation of an instrument to measure metacognitive strategies, thus contributing to validity and applicability.

Keywords: modeling, metacognitive strategies, engineering education, questionnaire, gender invariance.

Introduction

Several authors suggest that the future of work is uncertain, but this uncertainty does not refer

to the lack of employment, but rather to the skills that new jobs will demand (Dirksen, 2019; Weller et

al., 2019; World Economic Forum, 2020). The case of engineering is particular, as professionals in this

field are expected to be able to handle large volumes of data, work in teams with machines, possess

communication skills, solve complex and open-ended problems, among other skills (Richert et al., 2016;

Humberto and Rojas, 2017). The skills to solve complex and open-ended problems underlies much of

the standards for 21st-century engineering education (Kent and Noss, 2003; International Engineering

Alliance, 2014; ABET, 2017; Alpers, 2021). However, the literature has suggested that the deployment of

metacognitive skills is required to successfully solve this type of task (Blum, 2011; Borromeo, 2006; Maaß,

2006; Stillman, 2011; Vorhölter et al., 2019). Therefore, metacognition influences the way complex and

open problems are approached and modeled in engineering (Wengrowicz et al., 2018; Hidayat, Zulnaidi,

et al., 2018; Wedelin et al., 2015; Yildirim, 2010).

Modeling tasks concentrate essential features of common engineering professional environments’

problems since they are complex, open-ended, and have few structure (Jonassen et al., 2006; Gainsburg,

2013; Lyon and Magana, 2020). When students in engineering training solve this type of tasks, they

understand the connection between engineering contexts and mathematics (Aravena et al., 2022; Soon

et al., 2011). Therefore, mathematical modeling in engineering is relevant due to its role in the connection

between mathematics and industry problems (Li, 2013; Palmer et al., 2013). However, addressing

modeling tasks is challenging because they are complex, open-ended, and it is not evident where to

commence and where one is expected to arrive (Kaiser and Schwarz, 2010). Usually, students face

obstacles in successfully solving modeling problems due to their own beliefs, limited self-regulation,

inappropriate attitudes, and expectations (Wedelin et al., 2015). Incorporating this type of tasks in the

classroom requires careful consideration of metacognition due to its significant impact.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Metacognition plays an essential role in solving complex engineering problems as it promotes the

understanding of mathematical concepts, encourages organizational behavior, and enhances teamwork.

Metacognition enables the evaluation of local and global effects in engineering problem-solving, the

organization of resolution strategies, and the effective monitoring of the calculation process (Hegedus,

2001). Furthermore, it has been found that metacognition positively influences the development of

mathematical modeling skills among engineering students, fostering deliberate behaviors and a deeper

understanding of the theoretical implications of application (Tzohar-Rozen and Kramarski, 2014; Wedelin

et al., 2015; Hidayat et al., 2018). Collaboration in the resolution of modeling tasks also stimulates the

use of metacognitive strategies in small groups, allowing efficient monitoring of progress, assessment

of viability, and critical analysis of innovative ideas and results (Newell et al., 2004; Cardella, 2008;

Gainsburg, 2013). Together, these findings highlight the importance of metacognition in the context of

mathematical modeling in engineering.

However, at this educational level, there are few instruments for measuring metacognitive strategies

in mathematical modeling problems (Yildirim, 2010; Czocher, 2018; Hidayat et al., 2018; Hidayat, Zulnaidi,

et al., 2018), especially for measuring metacognitive strategies with engineering groups of students. The

study contributes to the research field by addressing the need to construct instruments as didactic tools to

measure the metacognitive strategies used by engineering students in mathematical modeling processes.

In this article, we present the development of the metacognitive strategies instrument guided by a mixed

methods study, which was conducted over an academic semester through observations, interviews, and

the application of a questionnaire.

To address the study, two research questions are sequentially answered. The first question

was: How do the metacognitive strategies of ‘planning,’ ‘monitoring and, if necessary, regulation,’ and

‘evaluation’ in the domain of mathematical modeling documented in the literature relate to the observed

metacognitive strategies in engineering groups of students in a multiple case study? Based on the results

of the qualitative research and theoretical aspects, an instrument was adapted and validated to measure

metacognitive strategies in mathematical modeling among engineering groups of students. Based on the

development of the instrument, the second research question was: What are the psychometric properties

of the instrument, such as content validity, reliability, and measurement invariance? To answer this

question, a quantitative correlational research design is utilized. The gender variable is used to investigate

measurement invariance.

The stereotypes that exist regarding gender differences (Correl, 2004; Cheryan and Plaut,

2010; Cheryan, 2012) could play a significant role in shaping how engineering students employ their

metacognitive strategies to solve mathematical modeling problems. Beliefs based on gender stereotypes

have shaped the interest and performance in engineering, science, technology, or mathematics careers

(Charles and Bradley, 2009; Noel-Levitz, 2014). This has led to, on the one hand, women and other

minorities being underrepresented in engineering careers (Wang and Degol, 2017; National Center

for Science and Engineering Statistics, 2023), that is to say, there is low entry into such careers. On

the other hand, some women who enter engineering do not persist in their trajectory in response to

negative feedback associated with performance in introductory mathematics or science courses,

cultural stereotypes, limited peer networks, or discrimination (Riegle-Crumb, 2006; Cech et al., 2011;

Mann and DiPetre, 2013; Sanabria and Penner, 2017). This situation generates a global-scale problem,

as industries and the economic sector of countries require the presence of men, women, and diverse

individuals to develop innovative and creative solutions to complex problems, drive the economy, and

achieve sustainable development (Woetzel et al., 2015; Smith-Doerr et al., 2017).

Inquiry into gender disparities regarding metacognitive skills in complex problem-solving has been

the subject of study for a long period, revealing diverse results concerning the gender effect on self-

efficacy, self-regulation, and metacognitive skills (Ciascai and Lavinia, 2011; Hong et al., 2021). It has

been demonstrated that women tend to use self-regulation strategies more effectively, adjusting their

efforts according to learning tasks and showing superior performance compared to men when solving

complex problems (Bidjerano, 2005). However, there are studies suggesting the absence of gender

differences in this field (Fitzpatrick, 1994; Bembenutty, 2007; Aravena-Díaz et al., in press), highlighting

the lack of clarity in the relationship between gender and the control of one’s own learning processes, that

is, metacognition.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Metacognitive strategies in mathematical modeling

A first approach to the analysis of metacognitive strategies in contextualized problems in real-

world environments is carried out by Stillman and Galbraith (1998), who take into account the cognitive-

metacognitive framework of Garofalo and Lester (1985), to categorize the knowledge of the metacognitive

strategy for understanding the problem, organizing information, developing and executing plans,

monitoring progress, and verifying the final result. Later, Stillman (2011) highlights the importance of

metacognitive activity in the development of mathematical modeling processes, considering the presence

of metacognition in each transition between the stages of the modeling cycle. Furthermore, the work of

Schukajlow and Krug (2013) reveals the positive influence of metacognitive planning and monitoring

strategies on modeling problems that require assuming missing data. Continuing the line of research,

Vorhölter (2018, 2019) implemented a project aimed at stimulating ninth and tenth-grade students in this

domain, from which he developed an analytical framework for analyzing metacognition in the context of

mathematical modeling.

According to Vorhölter et al. (2019), declarative metacognition involves understanding the

characteristics of individuals, strategies, and tasks related to modeling. In contrast, procedural

metacognitive strategies focus on ‘planning’, ‘monitoring’ and, if necessary, regulation’, as well as the

‘evaluation’ of actions and performance in the process of solving mathematical modeling problems.

Metacognitive strategies play an essential role in the successful development of modeling tasks. The

metacognitive strategies for ‘planning’ allow for an understanding of the task’s objectives, the prediction

of outcomes, and the exploration of various resolution approaches. Decision-making regarding which

approach to employ for resolution is also considered part of metacognition. “Monitoring” involves the

constant review of one’s own cognitive processes during the task, posing questions to verify the accuracy

of the steps being followed. ‘Regulation’ is activated when students encounter obstacles and seek support.

‘Evaluation’ analyzes the processes undertaken and the team’s performance, considering the strengths,

weaknesses, and team dynamics. These classifications in dimensions of metacognitive strategies are

applicable to both individual and group metacognition (Vorhölter, 2019; Vorhölter and Krüger, 2021).

Materials and Methods

The design of this research is of a mixed, sequential exploratory type (Creswell and Plano, 2018),

in which the findings from four months of observation and interviews were integrated into an instrument

to measure metacognitive strategies (Tashakkori and Teddlie, 2003). This research method allowed

for a profound understanding of student’s metacognitive strategies and assisted in formulating items

for the instrument. This design is developed in two phases: the first phase identifies and classifies the

metacognitive strategies of the engineering groups of students. In the second phase, the instrument is

adapted and validated based on the categories established in the first phase and a review of the existing

literature. The development of the instrument allowed for the application of the study to a larger and

more diverse sample, thus extending the scope of the results. For both phases, the analytical framework

of specific metacognitive strategies in the mathematical modeling process (Vorhölter et al., 2019) was

employed, along with studies that reveal metacognitive aspects of engineering students.

Qualitative component

In the qualitative phase, research techniques guided by multiple case studies were used, including

non-participant observations and interviews (Yin, 2014). The observations were carried out while the

students were participating in a mathematical modeling program in a one-variable calculus course for

engineering. In this initial research phase, a qualitative approach was adopted with the purpose of

exploring the processes of mathematical modeling in engineering students to deeply understand the

metacognitive strategies that may contribute to the development of this type of problems (Creswell, 2009).

Participants

In the phase that adopts a qualitative design, four engineering student cases with experience in

mathematical modeling problem solving are selected to observe the metacognitive behavior of students

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

encompassing greater diversity (Simons, 2013). The cases consist of four groups, each composed of three

students who had participated in a mathematical modeling program within the context of an engineering

calculus course, collaborating on modeling problems. For a detailed description, see Cárcamo Mansilla

et al., (in press). To distinguish the groups in description of Table 1, we will use the letter G1, G2, G3 and

G4 and the students with the letters s1, s2, s3 respectively.

Data collection

Non-participant observations were made of four groups of students as they worked on a sequence

of mathematical modeling problems in the classroom. The sessions were recorded using twelve video

recordings, and interviews were conducted with the groups after solving the modeling tasks to record

retrospective knowledge of the metacognitive strategies they had used in their own process of mathematical

modeling. The interviews were organized around metacognitive strategies for ‘planning,’ ‘monitoring and,

if necessary, regulation,’ and ‘evaluation,’ based on the work of Vorhölter et al. (2019). The interview script

was validated by experts according to the criteria of Escobar-Pérez and Cuervo-Martínez (2008).

Data analysis

A deductive-inductive content analysis (Mayring, 2014; Gläser-Zikuda et al., 2020) is used to

categorize metacognitive strategies. For this purpose, a categorization was carried out considering codes

based on the literature, taking into account the metacognitive strategies of ‘planning’, ‘monitoring and if

necessary regulation’, and ‘evaluation’ (ME1 to ME3) based on the theoretical framework of Vorhölter

et al. (2019). Besides the literature-based codes, the research team allowed for the emergence of

additional phenomena (Kuckartz, 2019; Kohlbacher, 2006) related to the metacognitive strategies of

engineering students, such as the necessity to make assumptions, as it prompted engineering students

to comprehensively address the context of the problem. Additionally, the organization of groups, the

definition of roles, and the allocation of time fostered autonomy. Such skills are crucial for future engineers

because they will face new challenges ahead (Newell et al., 2004; Penagos, 2011).

Qualitative results

A set of topics organized into categories and subcategories emerged from the observations that

were contrasted with the interviews conducted, and are summarized in Table 1. These topics were used to

form the final constructs of the instrument to measure metacognitive strategies, which are discussed later.

Table 1. Qualitative categories and subcategories

Category/ subcategories Description

Planning metacognitive strategies (ME1)

• They discuss the information among all [p1]

• Establish demands and objectives among everyone [p2]

• Discuss solution strategies [p3]

• Realize the need to establish assumptions [p4]

• Consider the resources available to solve the task [p5]

It reflects the active participation of students in the discus-

sion and the joint establishment of demands, objectives, and

solution strategies, as well as the awareness of the impor-

tance of making assumptions and considering the available

resources to effectively address the tasks.

Monitoring and regulation metacognitive strategies (ME2)

• Monitor the progress of the modeling process [m1]

• Validates the modeling process [m2]

• Question the evolving mathematical model [m3]

• Discuss successes and difficulties in the process [r1]

• Establish a course of action at the local level [r2]

• Discuss action plans at a global level [r3]

• Select a new solution plan [r4]

It refers to the constant awareness and reflection on prog-

ress and difficulties in the modeling process, where students

demonstrate their ability to monitor, question, and adjust

evolving mathematical models.

Evaluation metacognitive strategies (ME3)

• Evaluate the effectiveness of resources [e1]

• Evaluate the effectiveness of the process [e2]

• Evaluate the effectiveness of the strategies [e3]

They demonstrate their skills to analyze the effectiveness of

available resources, as well as to critically assess the mod-

eling process itself, considering approaches, methods, suc-

cesses, and challenges. Furthermore, the evaluation of the

strategies employed highlights their skill in identifying those

that prove effective and applicable in future contexts.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Several of these categories of metacognitive strategies have been previously studied, specifically

in the contexts of high school students (Schukajlow and Krug, 2013; Vorhölter, 2017, 2018; Vorhölter,

2019), but have not been explored with engineering groups of students.

Development of the instrument to measure metacognitive strategies

The qualitative data analysis phase reported categories and subcategories of metacognitive

strategies, serving as constructs for the instrument to measure metacognitive strategies in mathematical

modeling among groups of engineering students. These constructs were operationalized using the

dimensions from the analytical framework of Vorhölter et al. (2019) and the transcriptions of video episodes

and retrospective interviews to create items that formed part of the assessment tool. For instance, if a

phrase appeared in multiple video episodes and interviews, it was used to create an item addressing

this theme in the assessment instrument and added to the corresponding dimension. Table 2 presents

examples of qualitative data transformed into assessment instrument items.

Table 2. Item creation from qualitative data

Qualitative Data Category/ subcategories Survey Item

Video episodes transcription:

“What I want is for the four of us

to do it to see if we are doing it

equally and well” (G4_S1)

“Let’s all four do the data for the

cylinder and if they give us the

same it’s because the cylinder is

good” (G2_S2)

Interview transcription:

“We try to see everything together

and then compare and follow the

same path” (G4_S2)

Planning metacognitive strategies

(ME1)

• Discuss solution strategies [p3]

i14: … We agreed on the same

approach to solving the problem

individually, to then compare and

review results or calculations

together

Video episodes transcription:

“Wait, let’s check again” (G3_S1)

“With these numbers, what we are

getting is the capacity that we are

going to have inside, do you under-

stand me?” (G1_S3)

Interview transcription:

“I thank my colleagues because

sometimes I miss a sign…” (G4_

S1)

Monitoring and, if necessary,

regulation metacognitive strategies

(ME2)

• Monitor the progress of the mod-

eling process [m1]

i10: … The groups ensures that

everyone understand individu-

ally in order to proceed with the

resolution of the issue

Video episodes transcription:

“Yes, if you look at the photo I sent,

ehm, it actually has an interval of

thirteen points, it has an interval of

thirteen point sixty-five” (G4_S3)

“Why do you give us 10 bottles at

the base, it seems that it is bad?”

(G3_S1)

“Did you see the measurements?

It’s bad, we have to check it”

(G1_S2)

Interview transcription:

“Then we saw that it should not be

easy but it is important to try to ap-

ply mathematics to something that

one recognizes” (G4_S1)

“At one point, when we calculated

the number of bottles it didn’t fit

us and we realized it was wrong

and we had to calculate everything

again” (G3_S3)

Monitoring and, if necessary,

regulation metacognitive strategies

(ME2)

• Select a new solution plan [r3]

i2: … In some occasions, we no-

tice that it is necessary to change

the work plan (due to confusion

or errors)

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Additionally, theoretical items were incorporated into each dimension of the instrument. The

theoretical references in specialized literature served as a guide. The scale developed by Rakoczy et al.

(2005) was used and translated to adapt the instrument, with reported reliability exceeding a Cronbach’s

alpha of .70 in each of its dimensions. The Schukajlow and Krug (2013) scale is also employed, who

reported reliability levels of .74 and .84 for the planning and monitoring dimensions, respectively. Finally,

the items are supplemented with the findings of those who have measured metacognitive strategies in the

specific domain of mathematical modeling (Vorhölter, 2017, 2018, 2019).

Before administering the instrument, an expert validity test was conducted. Specialists in

mathematics education and cognitive psychology evaluated the instrument by reviewing the used scale

and items obtaining a score of .9 and guaranteeing content validity (Tristán-López, 2008). Then, a pilot

study is conducted to carry out a preliminary analysis of the instrument with a sample of 30 engineering

students (Johanson and Brooks, 2010) who had experience in modeling processes. The overall reliability of

the scale is analyzed through the Cronbach’s alpha coefficient, which reaches a value of 0.83, surpassing

the minimum acceptable internal consistency criterion (Allen et al., 2008).

Then, an instrument with 23 items and three subconstructs is obtained (Appendix A): metacognitive

strategies for ‘planning’ (8 items), ‘monitoring and, if necessary, regulating’ (12 items) and ‘evaluating’ (3

items). To measure the metacognitive strategies with the instrument, a 5-point Likert scale is used (1 =

totally disagree, 5 = totally agree).

Quantitative component

After the creation of the instrument, the quantitative phase of the cross-sectional study is conducted.

This phase should be treated as a follow-up to the initial qualitative phase, as suggested by the sequential

exploratory research design (Creswell and Plano, 2018). This phase has been designed to analyze the

psychometric properties of the metacognitive strategies assessment instrument and its invariance across

male and female.

Participants

According to the guidelines outlined by Creswell and Plano (2018) for mixed methods, this phase

involved different participants from the same student population. Engineering students with experiences

in mathematical modeling problem solving were selected. To recognize such expertise, it is considered

that they have participated in a modeling program or courses such as calculus, linear algebra, structural

analysis, differential equations, among others, where students confront problems in the field of engineering,

applying knowledge of mathematics and engineering, with critical thinking and analytical capacity. A total

of 104 students (random selection) participated, who had experience in mathematical modeling problem

solving acquired through various courses, including 71 men and 33 women. The age ranged from 18 to

47 years (SD = 11.89). They belonged to various engineering fields. The statistical power of the sample

was calculated to be .80, considering a confidence level of .05; a root mean square error of approximation

(RMSEA) index approximation of 0.080, and 227 degrees of freedom (Preacher and Coffman, 2006).

Data collection

The instrument was administered in 2022, after obtaining approval from the authorities of

the institutions involved in the study and informed consent from the participants. The students took

approximately 20 minutes to complete the instrument. Once the data were collected through the

application of the instrument, they were transcribed in Excel format, and a spreadsheet was prepared for

further analysis in the statistical programs M-Plus (Muthén and Muthén, 2007) and R Studio, respectively.

Data analysis

A descriptive analysis of the items was carried out, considering the mean, standard deviation, and

skewness in order to assess the performance of each questionnaire item, ensuring that none of them

exhibit an extreme mean or an excessively low standard deviation. Confirmatory factor analysis (CFA)

procedures were used to assess the construct validity of the instrument (Ferrando & Anguiano-Carrasco,

2010). The internal consistency values were measured using ordinal data alpha (Oliden and Zumbo,

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

2008) to estimate the reliability of the scale’s dimensions. To identify reliability by factor, the criterion of

Ferrando and Anguiano-Carrasco (2010) is followed, that is, if there is a factor loading less than 0.3 in

magnitude, the item’s inclusion is evaluated.

The measurement invariance between male and female students was also evaluated. Due to the

lack of normality assumption in the variables, confirmed by the Kolmogorov-Smirnov test (p < 0.05),

the non-parametric method of the Mann-Whitney U test was used to compare two independent groups

(Flores-Ruiz, 2017).

Results

Quantitative results

The mean scores, variance, and skewness were obtained for each of the items. As shown in

Table 3, participants exhibit a relatively high level of metacognitive strategies in the specific domain of

modeling, with an average of 4.22 on a scale of 1 to 5. The items showed a slight negative skewness within

acceptable ranges (Ferrando and Anguiano-Carrasco (2010)) for the development of a CFA (Confirmatory

Factor Analysis). The skewness indicates that the responses for each item are clustered around the mean,

consistent with the high level of reported metacognitive strategies. The standard deviations demonstrate

that all items were able to discriminate among subjects.

The information presented in Table 3 does not justify the elimination of any item due to low

discrimination capacity, a high grouping of responses in very low or very high scores, or asymmetries

outside acceptable ranges. This is confirmed by the high significant correlations among the items and

the corrected total score, which ranged from 0.30 to 0.70, which has been acceptable in studies in the

educational field (Assis Gomes et al., 2017). With the factors composed of 23 items, organized according

to the defined theoretical perspective, a confirmatory factor analysis for categorical variables was

conducted. Upon evaluating the validity of the proposed construct, the following statistics are observed.

RMSEA, CFI, TLI: The factorial model exhibited a good fit to the data (χ2 (N=108, df=227) = 289.426,

p=.003; CFI=.947; TLI=.940; RMSEA=.051 (CI90%=.031, .068).

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Table 3. Descriptive statistics of items from the metacognitive strategies questionnaire and factorial loads

resulting from the confirmatory factor analysis.

N Items Mean SD Skewness Factor

i1 Metacognitive strategies for planning

… We ensure to have clarity on what the problem is about 4.61 0.58 -1.16 0.38

i19 … We realize that we can establish assumptions (hypotheses) to solve the

problem 4.16 0.95 -1.15 0.80

i15 … First, we plan how to deal with the problem 4.09 0.98 -1.15 0.70

i13 … We make sure to focus on the objective requested in the problem 4.63 0.58 -1.30 0.46

i18 … We realize that it is necessary to organize time, roles, and responsibilities to

respond to what the professor has requested 4.29 0.97 -1.36 0.72

i7 … We discussed the steps that we must take to solve the task (strategies) 4.35 0.81 -1.25 0.66

i5 … We discussed different ways to solve the problem with the aim to choose

then the best one 4.36 0.91 -1.45 0.58

i14 … We agreed on the same approach to solving the problem individually, to then

compare and review results or calculations together 3.79 1.25 -0.72 0.50

i12 Metacognitive strategies for monitoring and, if necessary, regulation

… Throughout the problem solving process, we continuously check whether we

are on the right track 4.44 0.72 -1.19 0.55

i10 … The groups ensures that everyone understand individually in order to pro-

ceed with the resolution of the issue 3.94 1.13 -0.89 0.70

i4 … Every now and then we monitor the time we have left to finish solving the

problem 4.37 0.90 -1.57 0.30

i9 … We questioning the evolving mathematical model 4.32 0.85 -1.22 0.75

i6 … We questioning whether mathematical procedures (equation solving,

variable manipulation, matrix resolution, application of theorems) are well-

developed

4.36 0.93 -1.55 0.63

i3 … We endeavor to try to clarify if the solution is reasonable according to the

real problem 4.54 0.64 -1.04 0.61

i8 … When difficulties arose, we discussed them together as a groups 4.38 0.84 -1.19 0.62

i11 … We realized the need to ask the teacher for support when we encountered

difficulties 4.49 0.79 -1.82 0.56

i20 … In some occasions during the resolution process, we realized that we

needed to change the resolution strategy 4.33 0.70 -1.40 0.67

i2 … In some occasions, we notice that it is necessary to change the work plan

(due to confusion or errors) 4.21 0.88 -0.85 0.46

i23 … When we notice that we are facing difficulties in reaching a solution to the

problem, we undertake a new plan 4.32 0.84 -1.64 0.73

i17 … When there are incorrect solutions, we review the resolution process, and if

deemed necessary, we propose a new work plan 4.41 0.80 -1.34 0.63

i16 Metacognitive strategies for evaluation

… At the final of process, we collectively reflect on the quality of our perfor-

mance, the fulfillment of roles we undertook, time management, and agree-

ments

3.47 1.30 -0.43 0.78

i21 … At the final of process, we reflect on the entire problem-solving process to

enhance our performance in future opportunities, considering successes, er-

rors, strategies, achievements, and obstacles.

3.96 1.00 -1.03 0.81

i22 … At the final of process, we discuss whether there was alternative approach

that could have led to the solution. 3.63 1.30 -0.70 0.72

TOTAL 4.23 0.89 -1.19 NA

The Mann-Whitney U test results indicated that there are no statistically significant differences

associated with gender in the metacognitive strategies of (1) ‘planning,’ (2) ‘monitoring and, if necessary,

regulation,’ and (3) ‘evaluation’ (p > 0.05).

Figures 1a and 1b of metacognitive strategies for planning related to (i13) task objective and (i1)

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

clarity of the problem, did not reveal students in disagreement. However, 7% of male students expressed

indifference compared to females who predominantly chose agreement or complete agreement.

Nevertheless, these differences are not statistically significant (i13: U = 1320, p = 0.20; i1: U = 1072.5,

p = 0.41). Only the strategy related to (i14) agreeing on the same way to solve the problem and then

comparing together showed higher disagreements, which were not statistically significant, and a lower

frequency of usage (U = 1171.5, p = 1).

While female students use metacognitive strategies at a slightly higher level, there were no

significant differences between the groups regarding (i7) discussing steps (U = 1178.5, p = 0.96), (i19)

realizing the need to establish assumptions (U = 1036.5, p = 0.31), and (i18) discussing time organization,

roles, or other available resources to solve the task (U = 1151.5, p = 0.88). Likewise, no significant

differences were found in the use of metacognitive strategies for (i5) discussing different ways to solve

a problem to choose the best one (U = 1157, p = 0.91) and for (i15) planning first (U = 1339.5, p = 0.21);

even though men have slightly higher scores than women.

Figures 2a and 2b of metacognitive monitoring and, if necessary, regulation strategies show that

they are, on average, higher than the total scale. Students predominantly tend to agree or strongly agree

with the statements.

Figure 1a. The percentage for likert scale ‘planning’,

mean and standard deviation of the student male Figure 1b. The percentage for likert scale ‘planning’,

mean and standard deviation of the student female

Figure 2a. The percentage for likert scale ‘monitoring

and if necessary regulation’, mean and standard

deviation of the student male.

Figure 2b. The percentage for likert scale ‘monitoring

and if necessary regulation’, mean and standard devia-

tion of the student female.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

The strategy related to ensuring that (i10) everyone understands individually before continuing with

the problem resolution shows a slightly higher frequency of use in women (M = 4.12) compared to men

(M = 3.86). However, there were no statistically significant differences (U = 1042, p = 0.34). The strategy

(i3) of trying to clarify if the solution is reasonable according to the real problem concentrates more than

90% of preferences on agreement or strong agreement, with a slight advantage towards men (M = 4.58)

compared to women (M = 4.45), which is not statistically significant (U = 1291.5, p = 0.33). Similarly,

metacognitive strategies associated with (i9) questioning the evolving mathematical model (U = 947, p

= 0.08), (i23) undertaking a new plan in the face of difficulties (U = 1003, p = 0.19), (i12) checking if one

is on the right track (U = 1020.5, p = 0.24), and (i11) ask the teacher for support when we encountered

difficulties (U = 1193, p = 0.86) concentrate 90% of responses in the agreement or strong agreement

preference, with a slight advantage towards women that is not significant.

Figures 3a and 3b indicate a moderately lower average compared to the overall scale average,

suggesting that students use metacognitive evaluation strategies less frequently.

While the (i16) team reflection on performance, roles, time, and agreements strategy is slightly

higher in women (M = 3.52) compared to men (M = 3.45), there are no statistically significant differences

(U = 1140.5, p = 0.83). Even though male students more frequently use metacognitive evaluation

strategies for (i21) improving performance in a future opportunity (M = 3.99) compared to women (M =

3.91) and (i22) analyzing other solution approach (M = 3.66) compared to women (M = 3.58), there are no

statistically significant differences (i21: U = 1178.5, p = 0.96; i22: U = 1242, p = 0.61).

Discussions

The qualitative results reveal the presence of specific metacognitive strategies in engineering

students, divided into planning categories (ME1) involving active discussion and joint decision-making on

objectives, strategies, and available resources; monitoring and if necessary regulation categories (ME2)

involving constant reflection and adjustment of the modeling process based on progress and encountered

difficulties; and evaluation categories (ME3) highlighting students’ ability to critically analyze the modeling

process, resources used, and the effectiveness of employed strategies. Regarding the first research

question, it can be concluded that the observed metacognitive strategies in engineering students align

with existing literature based on the analytical framework of metacognitive strategies in the domain of

mathematical modeling (Vorhölter et al., 2019).

Furthermore, the quantitative results support these qualitative findings. There is evidence of a

generally high level of use of all metacognitive strategies. Previous research has highlighted the crucial

role of metacognition in solving complex mathematical modeling problems among engineering students

(Li, 2013; Palmer et al., 2013; Hidayat et al., 2018). Team collaboration has also been identified as a key

factor that stimulates the effective use of metacognitive strategies, reinforcing the importance of social

interactions in the development of metacognitive competence (Gainsburg, 2006; Newell et al., 2004;

Cardella, 2008).

Regarding the second research question, it can be concluded that the instrument exhibits adequate

psychometric properties. The analytical framework aligns appropriately with the observed data, as

indicated by the results of the construct validity test. Additionally, it can be concluded that there were no

statistically significant differences associated with gender male or female in terms of the metacognitive

Figure 3a. The percentage for likert scale ‘evaluation’,

mean and standard deviation of the student male.

Figure 3b. The percentage for likert scale ‘evaluation’,

mean and standard deviation of the student female.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

strategies used in mathematical modeling processes. Therefore, the instrument is invariant across

gender groups. The existing literature has shown various results in terms of gender disparities in the

metacognitive dimension, especially in the context of engineering careers and the use of metacognitive

strategies to address complex problems (Ciascai and Lavinia, 2011; Hong et al., 2021; Aravena-Díaz et

al., in press). However, our findings coincide with the results obtained by Aravena-Díaz et al. (in press),

indicating that no significant gender differences are observed when students face mathematical modeling

tasks, especially in relation to the metacognitive strategies they use.

Because addressing modeling challenges is generally not the task of students alone, but is done in

small groups, it is crucial not only to consider individual metacognitive strategies, but also to pay special

attention to group metacognitive strategies (Vorhölter and Krüger, 2021). For this reason, an instrument

has been developed to measure metacognitive strategies at the group level in this study.

Conclusions

Based on the qualitative and quantitative results obtained in this study on metacognitive strategies

in the context of mathematical modeling among engineering students, clear patterns of behavior have

been identified. Qualitative findings revealed the presence of three main categories of metacognitive

strategies, namely planning strategies (ME1), monitoring and if necessary regulation strategies (ME2),

and evaluation strategies (ME3). These categories reflect students’ active involvement in the modeling

process, highlighting their ability to collaborate, critically reflect, and effectively evaluate their own

performance and team progress.

It is worth noting that this study provides additional evidence on the issue of gender disparities in

the use of metacognitive strategies in the context of mathematical modeling, demonstrating that there

were no statistically significant differences based on the gender variable. These findings contradict some

previous research suggesting possible gender differences in the application of metacognitive strategies.

The refutation is grounded in the strengths of mathematical modeling as an educational innovation in

engineering training since, at the metacognitive level, it does not reproduce gender differences.

In conclusion, this study provides a deeper understanding of the dynamics of metacognitive

strategies among groups of engineering students in the context of mathematical modeling and offers an

instrument to measure metacognitive strategies that can be useful for future research exploring in-depth

interactions between metacognitive strategies, mathematical performance, and other relevant factors in

the field of mathematics and engineering education.

Acknowledgements

This research was supported by the Mathematical and Statistical Education Research Center

(CIEMAE, for its name in Spanish), Catholic University of Maule, Talca, Chile.

The study has been approved by the Scientific Ethics Committee of the Catholic University of Maule

in Act Nº41/2021. All study participants expressed their willingness to participate freely, consensually and

voluntarily.

Conflict of interests

The authors declare no conflict of interest.

Author Contributions

Conceptualization, N.C.M., M.A.D.; Investigation, N.C.M., M.A.D.; Methodology, N.C.M., M.A.D.;

Software, N.C.M.; Formal analysis, N.C.M., M.A.D.; Visualization, N.C.M.; Writing—original draft

preparation, N.C.M.; Writing—review and editing, M.A.D.. All authors have read and agreed to the

published version of the manuscript.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Appendix A

Items included in the Metacognitive Strategies Questionnaire for Mathematical Modeling

When I solve problems that come from the real world...

i1 … We ensure to have clarity on what the problem is about

i2 …

In some occasions, we notice that it is necessary to change the work plan (due to confusion or errors)

i3 … We endeavor to try to clarify if the solution is reasonable according to the real problem

i4 … Every now and then we monitor the time we have left to finish solving the problem

i5 … We discussed different ways to solve the problem with the aim to choose then the best one

i6 … We questioning whether mathematical procedures (equation solving, variable manipulation,

matrix resolution, application of theorems) are well-developed

i7 … We discussed the steps that we must take to solve the task (strategies)

i8 … When difficulties arose, we discussed them together as a groups

i9 … We questioning the evolving mathematical model

i10 … The groups ensures that everyone understand individually in order to proceed with the resolution

of the issue

i11 … We realized the need to ask the teacher for support when we encountered difficulties

i12 … Throughout the problem solving process, we continuously check whether we are on the right track

i13 … We make sure to focus on the objective requested in the problem

i14 … We agreed on the same approach to solving the problem individually, to then compare and

review results or calculations together

i15 … First, we plan how to deal with the problem

i16 … At the final of process, we collectively reflect on the quality of our performance, the fulfillment of

roles we undertook, time management, and agreements

i17 … When there are incorrect solutions, we review the resolution process, and if deemed necessary,

we propose a new work plan

i18 … We realize that it is necessary to organize time, roles, and responsibilities to respond to what the

professor has requested

i19 … We realize that we can establish assumptions (hypotheses) to solve the problem

i20 … In some occasions during the resolution process, we realized that we needed to change the

resolution strategy

i21 …

At the final of process, we reflect on the entire problem-solving process to enhance our performance

in future opportunities, considering successes, errors, strategies, achievements, and obstacles.

i22 … At the final of process, we discuss whether there was alternative approach that could have led

to the solution.

i23 … When we notice that we are facing difficulties in reaching a solution to the problem, we undertake

a new plan

References

ABET. (2017). Engineering Accreditation Commission Criteria for Accrediting. Engineerings programs. ABET. http://www.abet.org

Allen, K., Reed-Rhoads, T., Terry, R. A., Murphy, T. J., & Stone, A. D. (2008). Coefcient alpha: An engineer’s interpretation of

test reliability. Journal of Engineering Education, 97, 87-94. https://doi.org/10.1002/j.2168-9830.2008.tb00956.x

Alpers, B. (2021). Making Sense of Engineering Workplace Mathematics to Inform Engineering Mathematics Education. A

Report for the Mathematics Interest Group. European Society for Engineering Education (SEFI). https://www.se.be/

wp-content/uploads/2021/04/Workplace_Mathematics_SEFI_nal.pdf

Aravena Díaz, M. D., Díaz Levicoy, D., Rodríguez Alveal, F., & Cárcamo Mansilla, N. (2022). Case study and mathematical

modeling in the training of engineers. Characterization of STEM skills. Ingeniare. Revista chilena de ingeniería, 30(1),

37-56. http://dx.doi.org/10.4067/S0718-33052022000100037.

Aravena-Díaz, M. D., Sanhueza Henríquez, S., Rodriguez Gallardo, M., & Cárcamo Mansilla, N. (in press). Mathematical

modeling to reduce gender gaps in STEM: characterization of STEM skills in high school students. In V. Geiger, G.

Kaiser & H. Siller (Eds.), Researching Mathematical Modelling Education in Disruptive Times, International Perspectives

on the Teaching and Learning of Mathematical Modelling.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Assis Gomes, C., Almeida, L. S., & Núñez, J. C. (2017). Rationale and Applicability of Exploratory Structural Equation Modeling

(ESEM) in psychoeducational contexts. Psicothema, 29(3), 396–401. https://doi.org/10.7334/psicothema2016.369

Bembenutty, H. (2007). Self-Regulation of Learning and Academic Delay of Gratication: Gender and ethnic differences among

college students. Journal of Advanced Academics, 18(4), 586–616. https://doi.org/10.4219/jaa-2007-553

Bidjerano, T. (2005). Gender differences in self-regulated learning [Paper presentation]. Annual Meeting of the Northeastern

Educational Research Association, Kerhonkson, New York. https://eric.ed.gov/?id=ED490777

Blum, W. (2011). Can Modelling Be Taught and Learnt? Some Answers from Empirical Research. In G. Kaiser, W. Blum,

R. Borromeo Ferri, & G. Stillman (Eds.), Trends in Teaching and Learning of Mathematical Modelling. International

Perspectives on the Teaching and Learning of Mathematical Modelling (pp. 15–30). https://doi.org/10.1007/978-94-

007-0910-2_3

Borromeo, R. (2006). Theoretical and empirical differentiations of phases in the modelling process. ZDM - International Journal

on Mathematics Education, 38(2), 86–95. https://doi.org/10.1007/BF02655883

Cardella, M. E. (2008). Which mathematics should we teach engineering students? An empirically grounded case for a broad

notion of mathematical thinking. Teaching Mathematics and Its Applications, 27(3), 150–159. https://doi.org/10.1093/

teamat/hrn008

Cárcamo Mansilla, N., Aravena-Díaz, M. D., & Berres, S. (in press). Metacognitive Strategies in Mathematical Modelling

with Groups of Engineering Students. In V. Geiger, G. Kaiser & H. Siller (Eds.), Researching Mathematical Modelling

Education in Disruptive Times, International Perspectives on the Teaching and Learning of Mathematical Modelling.

Cech, E., Rubineau, B., Silbey, S., & Seron, C. (2011). Professional role condence and gendered persistence in engineering.

American sociological review, 76(5), 641-666. https://doi.org/10.1177/0003122411420815

Charles, M., & Bradley, K. (2009). Indulging our gendered selves? Sex segregation by eld of study in 44 countries. American

journal of sociology, 114(4), 924-976. https://doi.org/10.1086/595942

Cheryan, S. (2012). Understanding the paradox in math-related elds: Why do some gender gaps remain while others do not?.

Sex roles, 66, 184-190. https://doi.org/10.1007/s11199-011-0060-z

Ciascai, L., & Lavinia, H. (2011). Gender differences in metacognitive skills. A study of the 8th grade pupils in Romania.

Procedia - Social and Behavioral Sciences, 29, 396–401. https://doi.org/10.1016/j.sbspro.2011.11.255

Correll, S. J. (2004). Constraints into preferences: Gender, status, and emerging career aspirations. American sociological

review, 69(1), 93-113.

Creswell, J. (2009). Research Design Qualitative, Quantitative and Mixed Methods Approaches. SAGE Publications.

Creswell, J., & Plano, V. (2018). Designing and Conducting Mixed Methods Research. SAGE Publications.

Dirksen, U. (2019). Trabajo del fututo y futuro del trabajo. Nueva Sociedad, 279. https://static.nuso.org/media/articles/

downloads/3.TC_Dirksen_279.pdf

Escobar-Pérez, J., & Cuervo-Martínez, Á. (2008). Validez de contenido y juicio de expertos: Una aproximación a su

utilización. Avances En Medición, 6(1), 27–36. http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid

=S1665-61802017000300042

Ferrando, P. J., & Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología. Papeles del

psicólogo, 31(1), 18–33. https://www.papelesdelpsicologo.es/pdf/1793.pdf

Fitzpatrick, C. (1994). Adolescent Mathematical Problem Solving: The Role of Metacognition, Strategies and Beliefs [paper

presentation]. Annual Meeting of the American Educational Research Association. New Orleans. https://les.eric.

ed.gov/fulltext/ED374969.pdf

Flores-Ruiz, E., Miranda-Novales, M. G., & Villasís-Keever, M. A. (2017). The research protocol VI: How to choose the

appropriate statistical test. Inferential statistics. Revista Alergia México, 64(3), 364-370. https://doi.org/10.29262/ram.

v64i3.304

Gainsburg, J. (2013). Learning to Model in Engineering. Mathematical Thinking and Learning, 15(4), 259–290. https://doi.org/

10.1080/10986065.2013.830947

Garofalo, J., & Lester, F. (1985). Metacognition, Cognitive Monitoring, and Mathematical Performance. Journal for Research in

Mathematics Education, 16(3), 163. https://doi.org/10.2307/748391

Gläser-Zikuda, M., Hagenauer, G., & Stephan, M. (2020). The potential of qualitative content analysis for empirical educational

research. Forum Qualitative Sozialforschung, 21(1), 35–61. https://doi.org/10.17169/fqs-21.1.3443

Hegedus, S. J. (2001). Problem Solving in Integral Calculus: One Role of Metacognitive Thinking. In R. Speiser, C. A. Maher, &

C. N. Walter (Eds.), Proceedings of the 23rd Annual Meeting of the North American Chapter of the International Group

for the Psychology of Mathematics Education. (pp. 491–500). ERIC Clearinghouse for Science, Mathematics and

Environmental Education. https://les.eric.ed.gov/fulltext/ED476613.pdf

Hidayat, R., Zamri, S., & Zulnaidi, H. (2018). Does Mastery of Goal Components Mediate the Relationship between

Metacognition and Mathematical Modelling Competency?. Educational Sciences: Theory & Practice, 18(3). https://doi.

org/10.12738/estp.2018.3.0108

Hidayat, R., Zulnaidi, H., & Zamri, S. (2018). Roles of metacognition and achievement goals in mathematical modeling

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

competency: A structural equation modeling analysis. PLoS ONE, 13(11). https://doi.org/10.1371/journal.pone.0206211

Hong, E., O’Neil, H. F., Jr., & Feldon, D. (2005). Gender effects on mathematics achievement: Mediating role of state and trait

self-regulation. In A. M. Gallagher, & J. C. Kaufman (Eds.), Gender differences in mathematics (pp. 264–293). New

York, NY: Cambridge University Press. http://ndl.ethernet.edu.et/bitstream/123456789/16205/1/4.pdf#page=281

Humberto, J., & Rojas, C. (2017). La Cuarta Revolución Industrial o Industria 4.0 y su Impacto en la Educación Superior en

Ingeniería en Latinoamérica y el Caribe. 15 Th LACCEI International Multi-Conference for Engineering, Education, and

Technology. http://www.laccei.org/LACCEI2017-BocaRaton/work_in_progress/WP386.pdf

International Engineering Alliance. (2014). 25 years of the Washington Accord. International Engineering Alliance. http://www.

ieagreements.org/25_years/

Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological

Measurement, 70(3), 394–400. https://doi.org/10.1177/0013164409355692

Jonassen, D., Strobel, J., & Lee, C. (2006). Everyday Problem Solving in Engineering: Lessons for Engineering Educators.

Journal of Engineering Education, 9(2), 139–151. https://doi.org/https://doi.org/10.1002/j.2168-9830.2006.tb00885.x

Kaiser, G., & Schwarz, B. (2010). Authentic Modelling Problems in Mathematics Education-Examples and Experiences. J Math

Didakt, 31, 51–76. https://doi.org/10.1007/s13138-010-0001-3

Kent, P., & Noss, R. (2003). Mathematics in the University Education of Engineers A Report to the Ove Arup Foundation. The

Ove Arup Foundation. https://www.ovearupfoundation.org/library/media-reports

Kohlbacher, F. (2006). The use of qualitative content analysis in case study research. Forum Qualitative Sozialforschung, 7(1).

https://doi.org/10.17169/fqs-7.1.75

Kuckartz, U. (2019). Qualitative Text Analysis: A Systematic Approach. In: Kaiser, G., Presmeg, N. (Eds.) Compendium for Early

Career Researchers in Mathematics Education. ICME-13 Monographs. Springer, Cham. https://doi.org/10.1007/978-3-

030-15636-7_8

Li, T. (2013). Mathematical Modeling Education is the Most Important Educational Interface Between Mathematics and

Industry. In A. Damlamian, J. Rodrigues, & R. Sträßer (Eds.), New ICMI Study Series (pp. 51–58). Springer. https://doi.

org/10.1007/978-3-319-02270-3_5

Lyon. J. A., & Magana, A. J. (2020). A Review of Mathematical Modeling in Engineering Education. International Journal of

Engineering Education, 36(1), 101–116. https://www.ijee.ie/1atestissues/Vol36-1A/09_ijee3860.pdf

Mann, A., & DiPrete, T. A. (2013). Trends in gender segregation in the choice of science and engineering majors. Social science

research, 42(6), 1519-1541. https://doi.org/10.1016/j.ssresearch.2013.07.002

Maaß, K. (2006). What are modelling competencies?, ZDM, 38(2), 113–142. https://doi.org/10.1007/BF02655885

Mayring, P. (2014). Qualitative Content Analysis. Theoretical Foundation, Basic Procedures and Software Solution. https://

www.ssoar.info/ssoar/handle/document/39517

Muthén, L. K., & Muthén, B. O. (2017). Mplus User’s Guide. Muthén & Muthén.

National Center for Science and Engineering Statistics (NCSES). 2023. Diversity and STEM: Women, Minorities, and Persons

with Disabilities 2023. National Science Foundation. https://ncses.nsf.gov/wmpd

Newell, J., Dahm, K., Harvey, R., & Newell, H. (2004). Development metacognitive engineering teams. Chemical Engineering

Education, 38(4), 106–129. https://doi.org/10.4018/978-1-5225-2212-6.ch006

Oliden, P. E., & Zumbo, B. D. (2008). Coecientes de abilidad para escalas de respuesta categórica ordenada. Psicothema,

20(4), 896–901. http://www.psicothema.com/psicothema.asp?id=3572

Palmer, A., Amat, S., Busquier, S., Romero, P., & Tejada, J. (2013). Mathematics for Engineering and Engineering for

Mathematics. In A. Damlamian, J. Rodrigues, & R. Sträßer (Eds.), New ICMI Study Series (pp. 185–198). Springer.

https://doi.org/10.1007/978-3-319-02270-3_17

Penagos, H. P. (2011). How can metacognition be developed through problem-solving in higher education?. Ingeniería e

Investigación, 31(1), 213-223. https://repositorio.unal.edu.co/handle/unal/33495

Preacher, K., & Coffman, D. (2006). Computing power and minimum sample size for RMSEA. http://quantpsy.org/

Rakoczy, K., Buff, A., & Lipowsky, F. (2005). Dokumentation der Erhebungs-und Auswertungsinstrumente zur schweizerisch-

deutschen Videostudie.” Unterrichtsqualität, Lernverhalten und mathematisches Verständnis”. 1. Frankfurt: Main:

GFPF ua.

Richert, A., Shehadeh, M., Willicks, F., & Jeschke, S. (2016). Digital Transformation of Engineering Education - Empirical

Insights from Virtual Worlds and Human-Robot-Collaboration. International Journal of Engineering Pedagogy (IJEP),

6(4), 23. https://doi.org/10.3991/ijep.v6i4.6023

Riegle-Crumb, C. (2006). The path through math: Course sequences and academic performance at the intersection of race-

ethnicity and gender. American Journal of Education, 113(1), 101-122. https://doi.org/10.1086/506495

Schoenfeld, A. (1992). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics. In

D. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning (pp. 334–370). New York: MacMillan.

http://hplengr.engr.wisc.edu/Math_Schoenfeld.pdf

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Schukajlow, S., & Krug, A. (2013). Planning, monitoring and multiple solutions while solving modeling problems. In A. M.

Lindmeier & A. Heinze (Eds.), Proceedings of the 37th Conference of the International 4 - 177 Group for the Psychology

of Mathematics Education (pp. 177–184). Kiel, Germany: PME. https://www.researchgate.net/publication/274257874_

PLANNING_MONITORING_AND_MULTIPLE_SOLUTIONS_WHILE_SOLVING_MODELLING_PROBLEMS

Sanabria, T., & Penner, A. (2017). Weeded out? Gendered responses to failing calculus. Social Sciences, 6(2), 47. https://doi.

org/10.3390/socsci6020047

Simons, H. (2013). El estudio de caso: Teoría y práctica. Ediciones Morata, S. L. Madrid

Smith-Doerr, L., Alegria, S. N., & Sacco, T. (2017). How diversity matters in the US science and engineering workforce: A

critical review considering integration in teams, elds, and organizational contexts. Engaging Science, Technology, and

Society, 3, 139-153. https://doi.org/10.17351/ests2017.142

Soon, W., Lioe, L. T., & McInnes, B. (2011). Understanding the difculties faced by engineering undergraduates in learning

mathematical modelling. International Journal of Mathematical Education in Science and Technology, 42(8), 1023–

1039. https://doi.org/10.1080/0020739X.2011.573867

Stillman, G. (2011). Applying Metacognitive Knowledge and Strategies in Applications and Modelling Tasks at Secondary

School. In G. Kaiser, W. Blum, R. Borromeo Ferri, & G. Stillman (Eds.), Trends in Teaching and Learning of Mathematical

Modelling. International Perspectives on the Teaching and Learning of Mathematical Modelling (pp. 165–180). https://

doi.org/10.1007/978-94-007-0910-2_18

Stillman, G., & Galbraith, P. (1998). Applying mathematics with real world connections: metacognitive characteristics of

secondary students. Educational Studies in Mathematics, 36(2), 157–194. https://doi.org/10.1023/A:1003246329257

Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA:

Sage. https://doi.org/10.4135/9781506335193

Tristán-López, a. (2008). Modicación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un

instrumento objetivo. Avances En Medición, 6, 37–48. https://dialnet.unirioja.es/servlet/articulo?codigo=2981185

Tzohar-Rozen, M., & Kramarski, B. (2014). Metacognition, Motivation and Emotions: Contribution of Self-Regulated Learning

to Solving Mathematical Problems. Global Education Review, 1(4), 76–95. http://ger.mercy.edu/index.php/ger/article/

view/63

Vorhölter, K. (2017). Measuring Metacognitive Modelling Competencies. In G Stillman, W. Blum, & G. Kaiser (Eds.),

Mathematical Modelling and Applications. International Perspectives on the Teaching and Learning of Mathematical

Modelling (pp. 175–185). Springer, Cham

Vorhölter, K. (2018). Conceptualization and measuring of metacognitive modelling competencies: empirical verication of

theoretical assumptions. ZDM Mathematics Education, 50(1), 343–354. https://doi.org/10.1007/s11858-017-0909-x

Vorhölter, K. (2019). Enhancing metacognitive group strategies for modelling. ZDM Mathematics Education, 51, 703–716.

https://doi.org/10.1007/s11858-019-01055-7

Vorhölter, K., Krüger, A., & Wendt, L. (2019). Chapter 2: Metacognition in Mathematical Modeling – An Overview. In S.

Chamberlin & B. Sriraman (Eds.), Affect in Mathematical Modeling. Advances in Mathematics Education (pp. 29–51).

Springer. https://doi.org/10.1007/978-3-030-04432-9_3

Vorhölter, K., & Krüger, A. (2021). Metacognitive strategies in modeling: Comparison of the results achieved with the help of

different methods. Quadrante, 30(1), 178-197. https://doi.org/10.48489/quadrante.23653

Wang, M. T., & Degol, J. L. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current

knowledge, implications for practice, policy, and future directions. Educational psychology review, 29, 119-140. https://

doi.org/10.1007/s10648-015-9355-x

Wedelin, D., Adawi, T., Jahan, T., & Andersson, S. (2015). Investigating and developing engineering students’ mathematical

modelling and problem-solving skills. European Journal of Engineering Education, 40(5), 557–572. https://doi.org/10.

1080/03043797.2014.987648

Weller, J., Gontero, S., & Campbell, S. (2019). Cambio tecnológico y empleo: una perspectiva latinoamericana. Riesgos de la

sustitución tecnológica del trabajo humano y desafíos de la generación de nuevos puestos de trabajo. Macroeconomía

del Desarrollo, N° 201 (LC/TS.2019/37), Santiago, Comisión Económica para América Latina y el Caribe (CEPAL).

www.cepal.org/apps

Wengrowicz, N., Dori, Y. J., & Dori, D. (2018). Metacognition and Meta-assessment in Engineering Education. In Y. J. Dori, Z.

R. Mevarech, & D. R. Baker (Eds.), Cognition, Metacognition, and Culture in STEM Education. Innovations in Science

Education and Technology (pp. 191–216). Springer, Cham. https://doi.org/10.1007/978-3-319-66659-4_9

World Economic Forum, (2020). The Future of Jobs Report 2020. https://www.weforum.org/reports/the-future-of-jobs-

report-2020

Woetzel, J., Madgavkar, A., Ellingrud, K., Labaye, E., Devillard, S., Kutcher, E., Manyika, J., Dobbs, R., & Krishnan, M. (2015).

The power of parity: How advancing women’s equality can add $12 trillion to global growth. McKinsey Global Institute.

https://www.mckinsey.com/featured-insights/employment-and-growth/how-advancing-womens-equality-can-add-12-

trillion-to-global-growth

Yin, R. K. (2014). Case study research design and methods (5th ed.). Thousand Oaks, CA: Sage.

www.ijcrsee.com

Cárcamo-Mansilla N., & Aravena-Díaz M. D. (2024). Metacognitive strategies for mathematical modeling with engineering

groups of students: Adaptation and validation of a questionnaire, International Journal of Cognitive Research in Science,

Engineering and Education (IJCRSEE), 12(1), 41-55.

Yildirim, T. Pinar. (2010). Understanding the modelling skills shift in engineering: The impact of self-efcacy, epistemology, and

metacognition [Master’s thesis, University of Pittsburgh]. University of Pittsburgh ProQuest Dissertations Publishing.

https://www.proquest.com/docview/858073953