International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE)

IJCRSEE Editorial Team; Noemí Cárcamo Mansilla; María Aravena Díaz

doi:10.23947/2334-8496-2024-12-1-41-55

Authors

Noemí Cárcamo Mansilla Faculty of Basic Sciences, Catholic University of Maule, Talca, Chile; Mathematical and Statistical Education Research Center, Catholic University of Maule, Talca, Chile https://orcid.org/0000-0002-9271-6000
María Aravena Díaz Faculty of Basic Sciences, Catholic University of Maule, Talca, Chile; Mathematical and Statistical Education Research Center, Catholic University of Maule, Talca, Chile https://orcid.org/0000-0002-6796-6366

DOI:

https://doi.org/10.23947/2334-8496-2024-12-1-41-55

Keywords:

Modeling, metacognitive strategies, Engineering Education, questionnaire, Gender Invariance

Abstract

A sequential exploratory mixed-methods study is implemented to develop an instrument that allows for the evaluation of the metacognitive strategies used by engineering groups of students when solving mathematical modeling problems. The findings of the qualitative study guided by observations and interviews reveal the use of metacognitive strategies of ‘planning’, ‘monitoring and, if necessary, regulation’, and ‘evaluation’. In this article, we present the final categories of the qualitative analysis and discuss how these data were shaped into a theoretical construct and items of an instrument to measure metacognitive strategies. The psychometric properties of the instrument are analyzed, and it is argued that it has a similar interpretation among males and females, as there are no significant differences in these results. The development of the present study demonstrates how the qualitative method can support the adaptation of an instrument to measure metacognitive strategies, thus contributing to validity and applicability.

Downloads

Download data is not yet available.

References

ABET. (2017). Engineering Accreditation Commission Criteria for Accrediting. Engineerings programs. ABET. http://www.abet.org

Allen, K., Reed-Rhoads, T., Terry, R. A., Murphy, T. J., & Stone, A. D. (2008). Coefficient alpha: An engineer’s interpretation of test reliability. Journal of Engineering Education, 97, 87-94. https://doi.org/10.1002/j.2168-9830.2008.tb00956.x DOI: https://doi.org/10.1002/j.2168-9830.2008.tb00956.x

Alpers, B. (2021). Making Sense of Engineering Workplace Mathematics to Inform Engineering Mathematics Education. A Report for the Mathematics Interest Group. European Society for Engineering Education (SEFI). https://www.sefi.be/wp-content/uploads/2021/04/Workplace_Mathematics_SEFI_final.pdf

Aravena Díaz, M. D., Díaz Levicoy, D., Rodríguez Alveal, F., & Cárcamo Mansilla, N. (2022). Case study and mathematical modeling in the training of engineers. Characterization of STEM skills. Ingeniare. Revista chilena de ingeniería, 30(1), 37-56. http://dx.doi.org/10.4067/S0718-33052022000100037. DOI: https://doi.org/10.4067/S0718-33052022000100037

Aravena-Díaz, M. D., Sanhueza Henríquez, S., Rodriguez Gallardo, M., & Cárcamo Mansilla, N. (in press). Mathematical modeling to reduce gender gaps in STEM: characterization of STEM skills in high school students. In V. Geiger, G. Kaiser & H. Siller (Eds.), Researching Mathematical Modelling Education in Disruptive Times, International Perspectives on the Teaching and Learning of Mathematical Modelling.

Assis Gomes, C., Almeida, L. S., & Núñez, J. C. (2017). Rationale and Applicability of Exploratory Structural Equation Modeling (ESEM) in psychoeducational contexts. Psicothema, 29(3), 396–401. https://doi.org/10.7334/psicothema2016.369

Bembenutty, H. (2007). Self-Regulation of Learning and Academic Delay of Gratification: Gender and ethnic differences among college students. Journal of Advanced Academics, 18(4), 586–616. https://doi.org/10.4219/jaa-2007-553 DOI: https://doi.org/10.4219/jaa-2007-553

Bidjerano, T. (2005). Gender differences in self-regulated learning [Paper presentation]. Annual Meeting of the Northeastern Educational Research Association, Kerhonkson, New York. https://eric.ed.gov/?id=ED490777

Blum, W. (2011). Can Modelling Be Taught and Learnt? Some Answers from Empirical Research. In G. Kaiser, W. Blum, R. Borromeo Ferri, & G. Stillman (Eds.), Trends in Teaching and Learning of Mathematical Modelling. International Perspectives on the Teaching and Learning of Mathematical Modelling (pp. 15–30). https://doi.org/10.1007/978-94-007-0910-2_3 DOI: https://doi.org/10.1007/978-94-007-0910-2_3

Borromeo, R. (2006). Theoretical and empirical differentiations of phases in the modelling process. ZDM - International Journal on Mathematics Education, 38(2), 86–95. https://doi.org/10.1007/BF02655883 DOI: https://doi.org/10.1007/BF02655883

Cardella, M. E. (2008). Which mathematics should we teach engineering students? An empirically grounded case for a broad notion of mathematical thinking. Teaching Mathematics and Its Applications, 27(3), 150–159. https://doi.org/10.1093/teamat/hrn008 DOI: https://doi.org/10.1093/teamat/hrn008

Cárcamo Mansilla, N., Aravena-Díaz, M. D., & Berres, S. (in press). Metacognitive Strategies in Mathematical Modelling with Groups of Engineering Students. In V. Geiger, G. Kaiser & H. Siller (Eds.), Researching Mathematical Modelling Education in Disruptive Times, International Perspectives on the Teaching and Learning of Mathematical Modelling.

Cech, E., Rubineau, B., Silbey, S., & Seron, C. (2011). Professional role confidence and gendered persistence in engineering. American sociological review, 76(5), 641-666. https://doi.org/10.1177/0003122411420815 DOI: https://doi.org/10.1177/0003122411420815

Charles, M., & Bradley, K. (2009). Indulging our gendered selves? Sex segregation by field of study in 44 countries. American journal of sociology, 114(4), 924-976. https://doi.org/10.1086/595942 DOI: https://doi.org/10.1086/595942

Cheryan, S. (2012). Understanding the paradox in math-related fields: Why do some gender gaps remain while others do not?. Sex roles, 66, 184-190. https://doi.org/10.1007/s11199-011-0060-z DOI: https://doi.org/10.1007/s11199-011-0060-z

Ciascai, L., & Lavinia, H. (2011). Gender differences in metacognitive skills. A study of the 8th grade pupils in Romania. Procedia - Social and Behavioral Sciences, 29, 396–401. https://doi.org/10.1016/j.sbspro.2011.11.255 DOI: https://doi.org/10.1016/j.sbspro.2011.11.255

Correll, S. J. (2004). Constraints into preferences: Gender, status, and emerging career aspirations. American sociological review, 69(1), 93-113. DOI: https://doi.org/10.1177/000312240406900106

Creswell, J. (2009). Research Design Qualitative, Quantitative and Mixed Methods Approaches. SAGE Publications.

Creswell, J., & Plano, V. (2018). Designing and Conducting Mixed Methods Research. SAGE Publications.

Dirksen, U. (2019). Trabajo del fututo y futuro del trabajo. Nueva Sociedad, 279. https://static.nuso.org/media/articles/downloads/3.TC_Dirksen_279.pdf

Escobar-Pérez, J., & Cuervo-Martínez, Á. (2008). Validez de contenido y juicio de expertos: Una aproximación a su utilización. Avances En Medición, 6(1), 27–36. http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S1665-61802017000300042

Ferrando, P. J., & Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología. Papeles del psicólogo, 31(1), 18–33. https://www.papelesdelpsicologo.es/pdf/1793.pdf

Fitzpatrick, C. (1994). Adolescent Mathematical Problem Solving: The Role of Metacognition, Strategies and Beliefs [paper presentation]. Annual Meeting of the American Educational Research Association. New Orleans. https://files.eric.ed.gov/fulltext/ED374969.pdf

Flores-Ruiz, E., Miranda-Novales, M. G., & Villasís-Keever, M. A. (2017). The research protocol VI: How to choose the appropriate statistical test. Inferential statistics. Revista Alergia México, 64(3), 364-370. https://doi.org/10.29262/ram.v64i3.304 DOI: https://doi.org/10.29262/ram.v64i3.304

Gainsburg, J. (2013). Learning to Model in Engineering. Mathematical Thinking and Learning, 15(4), 259–290. https://doi.org/10.1080/10986065.2013.830947 DOI: https://doi.org/10.1080/10986065.2013.830947

Garofalo, J., & Lester, F. (1985). Metacognition, Cognitive Monitoring, and Mathematical Performance. Journal for Research in Mathematics Education, 16(3), 163. https://doi.org/10.2307/748391 DOI: https://doi.org/10.2307/748391

Gläser-Zikuda, M., Hagenauer, G., & Stephan, M. (2020). The potential of qualitative content analysis for empirical educational research. Forum Qualitative Sozialforschung, 21(1), 35–61. https://doi.org/10.17169/fqs-21.1.3443

Hegedus, S. J. (2001). Problem Solving in Integral Calculus: One Role of Metacognitive Thinking. In R. Speiser, C. A. Maher, & C. N. Walter (Eds.), Proceedings of the 23rd Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education. (pp. 491–500). ERIC Clearinghouse for Science, Mathematics and Environmental Education. https://files.eric.ed.gov/fulltext/ED476613.pdf

Hidayat, R., Zamri, S., & Zulnaidi, H. (2018). Does Mastery of Goal Components Mediate the Relationship between Metacognition and Mathematical Modelling Competency?. Educational Sciences: Theory & Practice, 18(3). https://doi.org/10.12738/estp.2018.3.0108 DOI: https://doi.org/10.12738/estp.2018.3.0108

Hidayat, R., Zulnaidi, H., & Zamri, S. (2018). Roles of metacognition and achievement goals in mathematical modeling competency: A structural equation modeling analysis. PLoS ONE, 13(11). https://doi.org/10.1371/journal.pone.0206211 DOI: https://doi.org/10.1371/journal.pone.0206211

Hong, E., O’Neil, H. F., Jr., & Feldon, D. (2005). Gender effects on mathematics achievement: Mediating role of state and trait self-regulation. In A. M. Gallagher, & J. C. Kaufman (Eds.), Gender differences in mathematics (pp. 264–293). New York, NY: Cambridge University Press. http://ndl.ethernet.edu.et/bitstream/123456789/16205/1/4.pdf#page=281 DOI: https://doi.org/10.1017/CBO9780511614446.014

Humberto, J., & Rojas, C. (2017). La Cuarta Revolución Industrial o Industria 4.0 y su Impacto en la Educación Superior en Ingeniería en Latinoamérica y el Caribe. 15 Th LACCEI International Multi-Conference for Engineering, Education, and Technology. http://www.laccei.org/LACCEI2017-BocaRaton/work_in_progress/WP386.pdf

International Engineering Alliance. (2014). 25 years of the Washington Accord. International Engineering Alliance. http://www.ieagreements.org/25_years/

Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70(3), 394–400. https://doi.org/10.1177/0013164409355692 DOI: https://doi.org/10.1177/0013164409355692

Jonassen, D., Strobel, J., & Lee, C. (2006). Everyday Problem Solving in Engineering: Lessons for Engineering Educators. Journal of Engineering Education, 9(2), 139–151. https://doi.org/https://doi.org/10.1002/j.2168-9830.2006.tb00885.x DOI: https://doi.org/10.1002/j.2168-9830.2006.tb00885.x

Kaiser, G., & Schwarz, B. (2010). Authentic Modelling Problems in Mathematics Education-Examples and Experiences. J Math Didakt, 31, 51–76. https://doi.org/10.1007/s13138-010-0001-3 DOI: https://doi.org/10.1007/s13138-010-0001-3

Kent, P., & Noss, R. (2003). Mathematics in the University Education of Engineers A Report to the Ove Arup Foundation. The Ove Arup Foundation. https://www.ovearupfoundation.org/library/media-reports

Kohlbacher, F. (2006). The use of qualitative content analysis in case study research. Forum Qualitative Sozialforschung, 7(1). https://doi.org/10.17169/fqs-7.1.75

Kuckartz, U. (2019). Qualitative Text Analysis: A Systematic Approach. In: Kaiser, G., Presmeg, N. (Eds.) Compendium for Early Career Researchers in Mathematics Education. ICME-13 Monographs. Springer, Cham. https://doi.org/10.1007/978-3-030-15636-7_8 DOI: https://doi.org/10.1007/978-3-030-15636-7_8

Li, T. (2013). Mathematical Modeling Education is the Most Important Educational Interface Between Mathematics and Industry. In A. Damlamian, J. Rodrigues, & R. Sträßer (Eds.), New ICMI Study Series (pp. 51–58). Springer. https://doi.org/10.1007/978-3-319-02270-3_5 DOI: https://doi.org/10.1007/978-3-319-02270-3_5

Lyon. J. A., & Magana, A. J. (2020). A Review of Mathematical Modeling in Engineering Education. International Journal of Engineering Education, 36(1), 101–116. https://www.ijee.ie/1atestissues/Vol36-1A/09_ijee3860.pdf

Mann, A., & DiPrete, T. A. (2013). Trends in gender segregation in the choice of science and engineering majors. Social science research, 42(6), 1519-1541. https://doi.org/10.1016/j.ssresearch.2013.07.002 DOI: https://doi.org/10.1016/j.ssresearch.2013.07.002

Maaß, K. (2006). What are modelling competencies?, ZDM, 38(2), 113–142. https://doi.org/10.1007/BF02655885 DOI: https://doi.org/10.1007/BF02655885

Mayring, P. (2014). Qualitative Content Analysis. Theoretical Foundation, Basic Procedures and Software Solution. https://www.ssoar.info/ssoar/handle/document/39517 DOI: https://doi.org/10.1007/978-94-017-9181-6_13

Muthén, L. K., & Muthén, B. O. (2017). Mplus User’s Guide. Muthén & Muthén.

National Center for Science and Engineering Statistics (NCSES). 2023. Diversity and STEM: Women, Minorities, and Persons with Disabilities 2023. National Science Foundation. https://ncses.nsf.gov/wmpd

Newell, J., Dahm, K., Harvey, R., & Newell, H. (2004). Development metacognitive engineering teams. Chemical Engineering Education, 38(4), 106–129. https://doi.org/10.4018/978-1-5225-2212-6.ch006 DOI: https://doi.org/10.4018/978-1-5225-2212-6.ch006

Oliden, P. E., & Zumbo, B. D. (2008). Coeficientes de fiabilidad para escalas de respuesta categórica ordenada. Psicothema, 20(4), 896–901. http://www.psicothema.com/psicothema.asp?id=3572

Palmer, A., Amat, S., Busquier, S., Romero, P., & Tejada, J. (2013). Mathematics for Engineering and Engineering for Mathematics. In A. Damlamian, J. Rodrigues, & R. Sträßer (Eds.), New ICMI Study Series (pp. 185–198). Springer. https://doi.org/10.1007/978-3-319-02270-3_17 DOI: https://doi.org/10.1007/978-3-319-02270-3_17

Penagos, H. P. (2011). How can metacognition be developed through problem-solving in higher education?. Ingeniería e Investigación, 31(1), 213-223. https://repositorio.unal.edu.co/handle/unal/33495 DOI: https://doi.org/10.15446/ing.investig.v31n1.20557

Preacher, K., & Coffman, D. (2006). Computing power and minimum sample size for RMSEA. http://quantpsy.org/

Rakoczy, K., Buff, A., & Lipowsky, F. (2005). Dokumentation der Erhebungs-und Auswertungsinstrumente zur schweizerisch-deutschen Videostudie.” Unterrichtsqualität, Lernverhalten und mathematisches Verständnis”. 1. Frankfurt: Main: GFPF ua.

Richert, A., Shehadeh, M., Willicks, F., & Jeschke, S. (2016). Digital Transformation of Engineering Education - Empirical Insights from Virtual Worlds and Human-Robot-Collaboration. International Journal of Engineering Pedagogy (IJEP), 6(4), 23. https://doi.org/10.3991/ijep.v6i4.6023 DOI: https://doi.org/10.3991/ijep.v6i4.6023

Riegle-Crumb, C. (2006). The path through math: Course sequences and academic performance at the intersection of race-ethnicity and gender. American Journal of Education, 113(1), 101-122. https://doi.org/10.1086/506495 DOI: https://doi.org/10.1086/506495

Schoenfeld, A. (1992). Learning to think mathematically: Problem solving, metacognition, and sense making in mathematics. In D. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning (pp. 334–370). New York: MacMillan. http://hplengr.engr.wisc.edu/Math_Schoenfeld.pdf

Schukajlow, S., & Krug, A. (2013). Planning, monitoring and multiple solutions while solving modeling problems. In A. M. Lindmeier & A. Heinze (Eds.), Proceedings of the 37th Conference of the International 4 - 177 Group for the Psychology of Mathematics Education (pp. 177–184). Kiel, Germany: PME. https://www.researchgate.net/publication/274257874_PLANNING_MONITORING_AND_MULTIPLE_SOLUTIONS_WHILE_SOLVING_MODELLING_PROBLEMS

Sanabria, T., & Penner, A. (2017). Weeded out? Gendered responses to failing calculus. Social Sciences, 6(2), 47. https://doi.org/10.3390/socsci6020047 DOI: https://doi.org/10.3390/socsci6020047

Simons, H. (2013). El estudio de caso: Teoría y práctica. Ediciones Morata, S. L. Madrid

Smith-Doerr, L., Alegria, S. N., & Sacco, T. (2017). How diversity matters in the US science and engineering workforce: A critical review considering integration in teams, fields, and organizational contexts. Engaging Science, Technology, and Society, 3, 139-153. https://doi.org/10.17351/ests2017.142 DOI: https://doi.org/10.17351/ests2017.142

Soon, W., Lioe, L. T., & McInnes, B. (2011). Understanding the difficulties faced by engineering undergraduates in learning mathematical modelling. International Journal of Mathematical Education in Science and Technology, 42(8), 1023–1039. https://doi.org/10.1080/0020739X.2011.573867 DOI: https://doi.org/10.1080/0020739X.2011.573867

Stillman, G. (2011). Applying Metacognitive Knowledge and Strategies in Applications and Modelling Tasks at Secondary School. In G. Kaiser, W. Blum, R. Borromeo Ferri, & G. Stillman (Eds.), Trends in Teaching and Learning of Mathematical Modelling. International Perspectives on the Teaching and Learning of Mathematical Modelling (pp. 165–180). https://doi.org/10.1007/978-94-007-0910-2_18 DOI: https://doi.org/10.1007/978-94-007-0910-2_18

Stillman, G., & Galbraith, P. (1998). Applying mathematics with real world connections: metacognitive characteristics of secondary students. Educational Studies in Mathematics, 36(2), 157–194. https://doi.org/10.1023/A:1003246329257 DOI: https://doi.org/10.1023/A:1003246329257

Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781506335193 DOI: https://doi.org/10.4135/9781506335193

Tristán-López, a. (2008). Modificación al modelo de Lawshe para el dictamen cuantitativo de la validez de contenido de un instrumento objetivo. Avances En Medición, 6, 37–48. https://dialnet.unirioja.es/servlet/articulo?codigo=2981185

Tzohar-Rozen, M., & Kramarski, B. (2014). Metacognition, Motivation and Emotions: Contribution of Self-Regulated Learning to Solving Mathematical Problems. Global Education Review, 1(4), 76–95. http://ger.mercy.edu/index.php/ger/article/view/63

Vorhölter, K. (2017). Measuring Metacognitive Modelling Competencies. In G Stillman, W. Blum, & G. Kaiser (Eds.), Mathematical Modelling and Applications. International Perspectives on the Teaching and Learning of Mathematical Modelling (pp. 175–185). Springer, Cham DOI: https://doi.org/10.1007/978-3-319-62968-1_15

Vorhölter, K. (2018). Conceptualization and measuring of metacognitive modelling competencies: empirical verification of theoretical assumptions. ZDM Mathematics Education, 50(1), 343–354. https://doi.org/10.1007/s11858-017-0909-x DOI: https://doi.org/10.1007/s11858-017-0909-x

Vorhölter, K. (2019). Enhancing metacognitive group strategies for modelling. ZDM Mathematics Education, 51, 703–716. https://doi.org/10.1007/s11858-019-01055-7 DOI: https://doi.org/10.1007/s11858-019-01055-7

Vorhölter, K., Krüger, A., & Wendt, L. (2019). Chapter 2: Metacognition in Mathematical Modeling – An Overview. In S. Chamberlin & B. Sriraman (Eds.), Affect in Mathematical Modeling. Advances in Mathematics Education (pp. 29–51). Springer. https://doi.org/10.1007/978-3-030-04432-9_3 DOI: https://doi.org/10.1007/978-3-030-04432-9_3

Vorhölter, K., & Krüger, A. (2021). Metacognitive strategies in modeling: Comparison of the results achieved with the help of different methods. Quadrante, 30(1), 178-197. https://doi.org/10.48489/quadrante.23653

Wang, M. T., & Degol, J. L. (2017). Gender gap in science, technology, engineering, and mathematics (STEM): Current knowledge, implications for practice, policy, and future directions. Educational psychology review, 29, 119-140. https://doi.org/10.1007/s10648-015-9355-x DOI: https://doi.org/10.1007/s10648-015-9355-x

Wedelin, D., Adawi, T., Jahan, T., & Andersson, S. (2015). Investigating and developing engineering students’ mathematical modelling and problem-solving skills. European Journal of Engineering Education, 40(5), 557–572. https://doi.org/10.1080/03043797.2014.987648 DOI: https://doi.org/10.1080/03043797.2014.987648

Weller, J., Gontero, S., & Campbell, S. (2019). Cambio tecnológico y empleo: una perspectiva latinoamericana. Riesgos de la sustitución tecnológica del trabajo humano y desafíos de la generación de nuevos puestos de trabajo. Macroeconomía del Desarrollo, N° 201 (LC/TS.2019/37), Santiago, Comisión Económica para América Latina y el Caribe (CEPAL). www.cepal.org/apps

Wengrowicz, N., Dori, Y. J., & Dori, D. (2018). Metacognition and Meta-assessment in Engineering Education. In Y. J. Dori, Z. R. Mevarech, & D. R. Baker (Eds.), Cognition, Metacognition, and Culture in STEM Education. Innovations in Science Education and Technology (pp. 191–216). Springer, Cham. https://doi.org/10.1007/978-3-319-66659-4_9 DOI: https://doi.org/10.1007/978-3-319-66659-4_9

World Economic Forum, (2020). The Future of Jobs Report 2020. https://www.weforum.org/reports/the-future-of-jobs-report-2020

Woetzel, J., Madgavkar, A., Ellingrud, K., Labaye, E., Devillard, S., Kutcher, E., Manyika, J., Dobbs, R., & Krishnan, M. (2015). The power of parity: How advancing women’s equality can add $12 trillion to global growth. McKinsey Global Institute. https://www.mckinsey.com/featured-insights/employment-and-growth/how-advancing-womens-equality-can-add-12-trillion-to-global-growth

Yin, R. K. (2014). Case study research design and methods (5th ed.). Thousand Oaks, CA: Sage.

Yildirim, T. Pinar. (2010). Understanding the modelling skills shift in engineering: The impact of self-efficacy, epistemology, and metacognition [Master’s thesis, University of Pittsburgh]. University of Pittsburgh ProQuest Dissertations Publishing. https://www.proquest.com/docview/858073953