www.ijcrsee.com
73
Al-Balushi, M. S. et al. (2022). Metacognitive awareness perceptions of students with high and low scores on TIMSS-Like
science tests, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), 73-82.
Introduction
Omani students have participated in the Trends in International Mathematics and Science Study
(TIMSS) since 2007, but have consistently scored lower than the cut-off point throughout that time
(Mullis et al., 2020). While little is known about the factors affecting Omani students’ performance in
TIMSS, internationally the TIMSS has gained the interest of a number of researchers (Kang and Cogan,
2020), resulting in a growing body of research exploring the factors associated with TIMSS test-taking
(Chiu, 2012; Kang and Cogan, 2022; Ruthven, 2011; Tighezza, 2014; Wang and Liou, 2018). According
to several studies, one of the main factors affecting student performance in science is metacognition
(Casselman and Atwood, 2017; Hong, Bernacki and Perera, 2020; Oyelekan, Jolayemi and Upahi, 2019;
Wang and Chen, 2014; Wang et al., 2014).
Metacognition has attracted a range of researchers since the term rst appeared in the literature
in the 1970s. The scientist who rst developed metacognitive theory dened it as individuals’ awareness
of their mental and cognitive processes and the way they affect performance (Flavell, 1979); other
researchers have described it as involving the ability to reect on one’s own thinking, and to monitor and
control its progress in order to achieve the goals one desires (Brown et al., 1983; Efklides, 2011; Larkin,
2006; Tang et al., 2016). Metacognition has also been explained as the process of thinking about one’s
own thinking, a skill which is believed to promote higher-order thinking (Adey, 1999).
Research into the brain has shown that there is an executive control mechanism in the pre-frontal
cortex; this is known as “inhibitory control” and is responsible for metacognition. This mental mechanism
Metacognitive Awareness Perceptions of Students with High and Low
Scores on TIMSS-Like Science Tests
Sulaiman M. Al-Balushi
1*
, Ibrahim S. Al-Harthy
2
, Rashid S. Almehrizi
1
, Abdullah K. Ambusiaidi
3
,
Khadija A. Al-Balushi
3
, Khalid K. Al-Saadi
1
, Mohammed Al-Aghbari
1
, Moza Al-Balushi
3
1
College of Education, Sultan Qaboos University, Muscat, Oman,
e-mail: sbalushi@squ.edu.om, mehrzi@squ.edu.om, saadi@squ.edu.om, malaghbari@squ.edu.om
2
Quality Assurance & Enhancement, National University of Science & Technology, Muscat, Oman,
e-mail: ibrahimsa@nu.edu.om
3
Ministry of Education, Muscat, Oman, e-mail: ambusaidi40@hotmail.com, khadija.belushi@gmail.com, al_balushim@hotmail.com
Abstract: The current study explores the differences in metacognitive awareness perceptions of students who had high
and low scores on TIMSS-like science tests. The sample consisted of 937 Omani students, 478 in Grade Five and 459 in Grade
Nine. TIMSS-like tests were specially designed for both grade levels, and students also completed a metacognitive awareness
perceptions inventory which explored their use of four main skills: planning, information management strategies, debugging
strategies and evaluation. MANOVA was used to analyze the data. The ndings indicated that students with high scores in the
TIMSS-like test out-performed students with low scores in the test on all four metacognitive skills surveyed. This was true for all
three performance areas analysed: performance in the TIMSS-like test as a whole, performance in lower-level test questions and
performance in higher-level test questions. These ndings highlight the extent to which students’ metacognitive skills inuence
their performance in science tests. The study recommends that students be trained to improve their metacognitive skills, reviews
several methods for doing this, and suggests that such training might better prepare them for taking science tests. However,
it also notes that further research is needed to explore the impact of metacognitive training on student performance in specic
science examinations such as TIMSS.
Keywords: debugging, evaluation, information management, metacognition, planning, science, TIMSS.
Original scientic paper
Received: June, 06.2022.
Revised: August, 24.2022.
Accepted: September, 07.2022.
UDK:
159.922-057.875
159.955.07-057.875
10.23947/2334-8496-2022-10-3-73-82
© 2022 by the authors. This article is an open access article distributed under the terms and conditions of the
Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
*
Corresponding author: sbalushi@squ.edu.om
www.ijcrsee.com
74
Al-Balushi, M. S. et al. (2022). Metacognitive awareness perceptions of students with high and low scores on TIMSS-Like
science tests, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), 73-82.
is believed to play a crucial role in inhibiting any initially ineffective or inappropriate response we might
have and in guiding us to accept a more fruitful and powerful one (Larkin, 2010). The value of this ability
to inhibit initial responses allows an individual to plan effectively. It does this by enhancing their capacity
to assess prior knowledge retrieved from long-term memory, to mediate the construction of meaning in the
working memory, and also to accommodate new knowledge into existing knowledge networks (Al-Harthy,
2016; Larkin, 2010; Wang and Chen, 2014).
The impact of metacognition on learning and academic achievement has been solidly established
in research literature; it has been positively linked to improved levels of learning, intelligence, problem-
solving, and decision-making (Al-Harthy, 2016; Balashov et al., 2020; Larkin, 2006; Lee et al., 2012;
Mahdavi, 2014; Sari Faradiba et al., 2019; Was and Al-Harthy, 2015). Several studies argue that students
with a high level of metacognition are able to decide what they need to learn, and can also control their
thinking processes and act in ways that will help them achieve their intended goals. In addition, when a
course of study or a topic area emphasizes different metacognitive practices from those they know, they
are able to develop reexive problem-solving strategies (Al-Harthy, Was and Hassan, 2015; Balashov et
al., 2020; Rahman and Hussan, 2017; Efklides, 2011; Jaleel and Premachandran, 2016; Joseph, 2010;
Lee et al., 2012; Liu and Liu , 2020; Mahdavi, 2014; Shubber, Udin and Minghat, 2015; Sutiyatno and
Sukarno, 2019; Tok, Özgan and Döş, 2010). Metacognitive ability also makes students aware of their
learning progress; they are thus able to reect on what they have accomplished and decide how they
need to go about completing their learning tasks, a process which requires the use of mental skills such as
planning, monitoring and evaluation (Efklides, 2011; Wiley and Guss, 2007; Hong, Bernacki and Perera,
2020; Jaleel and Premachandran, 2016; Joseph, 2010; Miller and Geraci, 2011; Santelmann, Stevens
and Martin, 2018; Sutiyatno and Sukarno, 2019).
A good deal of research has examined the relationship between students’ metacognition and their
attainment of academic goals, and has shown clearly that greater use of metacognitive skills is associated
with an improved awareness of what students are studying. It has also been linked with greater attainment
of their learning goals, improved reading comprehension, and enhanced independent learning skills (Al-
Harthy, Was and Hassan, 2015; Coutinho, 2007; Jaleel and Premachandran, 2016; Rahman and Hussan,
2017; López-Vargas Ibáñez-Ibáñez and Racines-Prada, 2017; Meniado, 2016; Moir, Boyle and Woolfson,
2020; Shubber, Udin and Minghat, 2015; Sutiyatno and Sukarno, 2019; Zhao, 2014). As far back as
1983, Brown et al. (1983) had demonstrated signicantly that several metacognitive strategies, including
self-regulation, planning, evaluating, and monitoring, are relevant to reading comprehension. Baker and
Brown (1984) argued similarly that anyone engaging in reading comprehension must be able to process
their cognitive activities, and that most of this involves metacognition. Another interesting point arising
from the research is that metacognitive scaffolding seems to lower the cognitive load during the time that
an individual is involved in learning tasks (López-Vargas Ibáñez-Ibáñez and Racines-Prada, 2017); by the
same token, experiencing learning anxiety during a problem-solving process might lead to metacognitive
blindness (Sari Faradiba et al., 2019).
In science education, studies have indicated a positive association between metacognitive
awareness and student achievement (Hong, Bernacki and Perera, 2020; Oyelekan, Jolayemi and Upahi,
2019). Metacognition has also been linked to improvements in problem-solving skills (Akben, 2020; Aurah,
Cassady and McConnell, 2014), in reective thinking skills (Antonio, 2020), in science inquiry learning
(Tang et al., 2016), in comprehension of science texts (Wang and Chen, 2014; Wang et al., 2014), and
in performance on exams (Casselman and Atwood, 2017). There have been other interesting ndings;
if metacognitive training is carried out continually throughout a semester, it is more likely to improve
students’ ability to assess their test scores (Al-Harthy, Was and Hassan, 2015; Casselman and Atwood,
2017), and the use of metacognitive prompts during test-taking improves student test scores (Aurah,
Cassady and McConnell, 2014). Research also shows that students with high test scores are better able
to accurately predict their scores, an important metacognitive skill, than are lower-performing students
(Hawker, Dysleski and Rickey., 2016). While students are actually taking a test, they use a range of
metacognitive strategies to answer test questions; these include eliminating incorrect options, underlining
the clues found in the question text, and re-examining their answers. The metacognitive strategies used
vary according to the nature and features of the test items (e.g. narrative, gures and graphics) (Diken,
2020). However, in spite of the growing body of research supporting the positive impact of metacognition on
learning, some studies indicate that students do not regularly practice meta-cognitive skills while actually
carrying out learning tasks (McCabe, 2011; Santelmann, Stevens and Martin, 2018; Siagian, Saragih and
Sinaga 2019; Saenz, Geraci and Tirso, 2019). This could be partly because there is little focus on these
skills within the classrooms where they study, and a lack of coverage of metacognition in the curriculum
materials used (Jaleel and Premachandran, 2016; Joseph, 2010). Overall, though, research shows that
www.ijcrsee.com
75
Al-Balushi, M. S. et al. (2022). Metacognitive awareness perceptions of students with high and low scores on TIMSS-Like
science tests, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), 73-82.
metacognitive mental skills characterize high-performing achievers and more advanced students (Larkin,
2006). Other research also reports, interestingly, that metacognition develops with practice and also
through interaction with others, and that it is these factors which impact individuals’ consciousness of
their cognitive processing (Balashov et al., 2020; Larkin, 2006).
There are several general learning theories which have a bearing on the role of metacognitive
thinking, which is sometimes seen as a kind of private speech that takes place in children’s minds.
One of these is Vygotsky’s sociocultural theory, which stresses the importance of private speech to
the development of learning. Another is Piaget’s cognitive theory, which emphasizes the importance of
meta-thinking in the comprehension of abstract concepts and phenomena; this meta-thinking is a type of
metacognition that happens in a later stage of a learner’s development (Bates, 2019). A third theoretical
perspective that helps to explain the association between learning and metacognitive skills is the cognitive
acceleration theory proposed by Shayer and Adey (2002), and heavily reliant on the ideas introduced by
Piaget and Vygotsky. A key principle of cognitive acceleration theory is the emphasis on asking learners
to explain their thinking process to their peers, telling them how they solve the problems at hand. This
lays the ground for the belief that social interaction accelerates learning (Shayer and Adey, 2002), with
the group conversation allowing learners to articulate how they think, organize their thoughts, and benet
from others’ techniques of tackling different problems, so that they come to visualize their thinking in a
sophisticated way (Oliver and Venville, 2015). Other research also shows that the social construction of
metacognition is best done while students are actually engaging in the task, rather than when reecting
on their thinking after the task has been completed (Larkin, 2010).
Purpose of study
Understanding the cognitive factors which affect Omani students’ performance in the TIMSS has
become particularly important to science education research in the country; metacognition is one of these
factors, and, if investigated, will provide a better understanding of its role in student test performance in
general. It is with this purpose in mind that the current study explores the differences in metacognitive
awareness perceptions of students with high and low scores in TIMSS-like science tests. Also, although
there have been a growing number of studies investigating student metacognition while they carry out
learning tasks, studies investigating the association between student metacognition during test taking
and their performance on tests have been very rare, especially in the area of science education. To
the knowledge of the authors, there has been no study that explored the association between student
metacognition and performance in TIMSS; the current study is an attempt to address this gap in the
research literature.
The main research question of the current study is therefore:
What are the differences in the metacognitive awareness perceptions of students with high and low
scores on TIMSS-like science tests?
Three sub-questions stem from this main research question:
1. Do metacognitive perceptions differ for students with high and low scores in TIMSS-like science
tests for Grades Five and Nine?
2. Do metacognitive perceptions differ for students with high and low scores in lower-level questions
of TIMSS-like science test for Grades Five and Nine?
3. Do metacognitive perceptions differ for students with high and low scores in higher-level questions
of TIMSS-like science test for Grades Five and Nine?
Materials and Methods
Participants and Context
The participants were 937 Omani students, 478 in Grade Five and 459 in Grade Nine. The
participants were from eleven schools located in three important governorates in Oman. The Omani
school system is composed of three stages: Cycle I (Grades 1-4), Cycle II (Grades 5-10) and Cycle III
(Grades 11 and 12). The study proposal went through an ethics checking review by both the funding body
and the Ministry of Education. After their approval was obtained, the schools conducted their own ethics
review before approving the participation of their teachers and students; they also obtained parental
consent required for the students who would be in the study.
www.ijcrsee.com
76
Al-Balushi, M. S. et al. (2022). Metacognitive awareness perceptions of students with high and low scores on TIMSS-Like
science tests, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), 73-82.
Instruments
Metacognitive awareness perceptions
The current study used the Metacognitive Awareness Inventory (MAI) created by Schraw and
Dennison (1994), and a widely used research instrument. The MAI contains four sections, each of which
addresses one type of metacognitive skill: planning, information management strategies, debugging
strategies and evaluation. The original instrument contained 52 items, but this was reduced to 22 because
the original was thought to be too lengthy for young children, and may have negatively affected the data
collected.The shorter version had a balanced number of items in each section. The content validity of
the selected items was validated by a panel of seven science educators and educational psychologists,
who assessed its appropriateness to measure the metacognitive awareness perceptions of young school
students. The items were then translated into Arabic, with the translation veried by two psychologists
uent in both Arabic and English. The modied instrument was then piloted on 120 students, and the
calculated Cronbach Alpha coefcient was 0.95.
TIMSS-like tests
As our aim was to conduct the test in the study in the same way as the actual TIMSS test is
conducted, multiple versions of the test were designed; the actual tests are administered using separate
but matching booklets that are randomly assigned to students. We followed the TIMSS Framework (Mullis
and Martin, 2017) and designed 18 versions of a TIMSS-like science test for Grade Nine and 20 versions of
a Grade Five test. The tests were written by fteen science educators who were trained by an international
expert to design TIMSS-like questions. The tests were then reviewed by six local TIMSS assessment
experts and by an international expert; they were then piloted on 1,163 students. The resulting Cronbach
alpha reliability coefcients ranged between 0.661 and 0.885 with an average of 0.779 for Grade Nine,
and between 0.655 and 0.885 with an average of 0.778 for Grade Five. These values were found to be
acceptable when compared to the reliability values for the actual TIMSS science tests (Martin, Mullis and
Hooper, 2016). More details about the construction of the test, its items, the validation of the matching
versions, and other details can be found in Al-Balushi, Al-Harthy and Almehrizi (2022).
Data collection
The instruments were uploaded into a mobile application designed specially to collect the data
needed for this study; the reader can nd more details about this application elsewhere (Al-Balushi,
Al-Harthy and Almehrizi, 2022). The mobile application, called the Trends in Oman Science Study
(TOSS), was designed by the Sas for Entrepreneurship Center under the Omani Ministry of Transport,
Communications and Information Technology; it was piloted on 120 students in order to ensure that
there were no technical or administrative malfunctions. The study also hired twelve science teachers as
research assistants; their role was to administer the study instruments in their classrooms in each of the
selected schools. The administration of TIMSS-like test lasted for 40 minutes, while the administration of
the metacognitive awareness perceptions instrument lasted for 10 minutes.
Data Analysis
We used means and standard deviations to describe participants’ scores in the four sections of
the metacognitive awareness perceptions instrument: planning, information management strategies,
debugging strategies and evaluation. We also used the Multivariate Analysis of Variance (MANOVA) to
answer the research questions and to discover whether there were any differences between high- and
low-scoring students in the four sections of the metacognitive awareness perceptions instrument.
Results
Tables 1-4 illustrate the results of the study, and show signicant Wilks’ Lambda values for all
MANOVA analyses conducted on the data. The ndings also indicate that there were signicant differences
in metacognitive awareness perceptions between high- and low-scoring students, showing that, in both
Grades Five and Nine, students with higher overall performances on the TIMSS-like test also had higher
metacognitive awareness perceptions. When data was analysed separately for performance on lower-
level and higher-level test questions, the results were found to be the same.
www.ijcrsee.com
77
Al-Balushi, M. S. et al. (2022). Metacognitive awareness perceptions of students with high and low scores on TIMSS-Like
science tests, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), 73-82.
Table 1.
Means and standard deviations of metacognitive awareness perceptions for Grade Five
Table 2.
MANOVA results of between-subjects effects for metacognitive awareness perceptions and TIMSS-
like total performance of Grade Five (df=1, 475)
a: Wilks’ Lambda= 0.96, F= 5.16, P= 0.000
b: Wilks’ Lambda= 0.97, F= 3.44, P= 0.009
c: Wilks’ Lambda= 0.97, F= 3.07, P= 0.016
Table 3.
Means and standard deviations of metacognitive awareness perceptions for Grade Nine
www.ijcrsee.com
78
Al-Balushi, M. S. et al. (2022). Metacognitive awareness perceptions of students with high and low scores on TIMSS-Like
science tests, International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 10(3), 73-82.
Table 4.
MANOVA results of between-subjects effects for metacognitive awareness perceptions and TIMSS-
like total performance of Grade Nine (df=1, 454)
a: Wilks’ Lambda= 0.92, F= 9.73, P= 0.000
b: Wilks’ Lambda= 0.94, F= 7.21, P= 0.000
c: Wilks’ Lambda= 0.93, F= 9.06, P= 0.000
Discussions
The ndings of the current study showed that the metacognitive awareness of the higher-performing
participants in Grades Five and Nine was consistently and signicantly higher than that of the lower-
performing participants. This was the case for the overall test scores, as well as for the scores in lower-
level questions and the scores in higher-level questions. These ndings emphasize the importance of
metacognitive awareness in student test performance on both lower-level and higher-level questions.
These ndings also conrm the existence of a undeniable association between metacognition and
academic performance, which has been emphasized consistently by previous research (Efklides, 2011;
Wiley and Guss, 2007; Hong, Bernacki and Perera, 2020; Jaleel and Premachandran, 2016; Joseph, 2010;
Mahdavi, 2014; Miller and Geraci, 2011; Santelmann, Stevens and Martin, 2018; Sutiyatno and Sukarno,
2019; Tok, Özgan and Döş, 2010). This and other studies have come to a number of conclusions about
the role of metacognitive thinking in test-taking. The ndings of the current study indeed indicate that
answering TIMSS test items is a cognitively demanding process, and that metacognitive thinking skills are
essential for students taking these tests. Looking at the issue from a practical point of view, Zimmerman
and Moylan (2009) have argued that such demanding contexts require test-takers to use their initiative and
to be self-regulated, resourceful and persistent, all of which require the use of metacognitive skills. It has
also been argued by proponents of the social cognitive model of self-regulation that test-takers engage
in a three-phase cyclical mental process. These phases are the forethought phase, the performance
phase and the self-reection phase, and all require metacognitive abilities if they are to be successful.
For instance, during the performance phase, test-takers must use self-observation strategies such as
time management, imagery that enhances the processing of information into the memory system, and
self-instruction; failure to mobilise these metacognitive strategies could result in poor test performance
(Panadero and Alonso-Tapia, 2014). The current study provides added evidence that strong levels of
metacognitive processing are an essential factor for students to perform well in the TIMSS science test;
lack of these metacognitive skills is a key factor in reducing their ability to conduct the mental processes
required to succeed in the test.
Science education literature offers little information about the link between students’ metacognitive
awareness and their performance in examinations, and we are aware of no study that has specically
investigated this association in the context of student performance in international science tests such
as TIMSS. However, research in other areas might help to explain our ndings, and their linking of the
performance of high-achieving students to their metacognitive awareness. A number of previous research
studies suggest that the possession of a high level of metacognitive awareness allows students to reect
on their thinking and monitor its progress; it also helps them to control their thinking processes and
direct them towards achieving the goals for which they strive (Efklides, 2011; Jaleel and Premachandran,
2016; Joseph, 2010; Larkin, 2006; Mahdavi, 2014; Sutiyatno and Sukarno, 2019; Tok, Özgan and Döş,
2010). Thus, since researchers have widely reported a positive association between metacognition and