literature review about language teaching

• Reference Manager
• Simple TEXT file

People also looked at

Review article, studies of teaching and learning english-speaking skills: a review and bibliometric analysis.

• School of Educational Studies, Universiti Sains Malaysia, Penang, Malaysia

This study conducted a comprehensive historical review and bibliometric analysis of the literature on English-speaking (ES) education and mapped the current state of the field, trends, and emerging topics, as well as identified gaps where further research is needed. We retrieved 361 sample documents on ES teaching and learning in Scopus (2010–2021) under certain conditions and analyzed the extracted data using Excel and VOSviewer 1.6.17 from the perspectives of the number of yearly publications, countries, authors, citation numbers, and keywords. The findings show that the number of publications on ES education increased from 2010 to 2021, but there was a lack of sustained engagement with this topic by researchers. Countries with an ESL or EFL context focused more on the subject of ES, although studies from native ES countries were more influential. The research topics showed a multidimensional trend, covering communicative skills, language knowledge, assessment, teaching or learning methods, ICT-related applications, and cognitive factors, of which ICT-related applications (such as flipped classrooms, blended learning, and e-learning) and cognitive factors (such as motivation, anxiety, and affect) were the areas of focus. Students in higher institutions, rather than children, became the main research subject of ES education over the period studied.

Introduction

A considerable amount of time and money has been invested in English language education (ELE) around the world, especially in countries where English is a Foreign (EFL) or Second Language (ESL). For example, ELE in East Asian countries such as China, South Korea, and Japan has been identified as a necessary skill, which has motivated the development of various approaches and policies ( Song, 2011 ; Hu and McKay, 2012 ). There have been at least three phases of English curriculum reforms by the Malaysian Ministry of Education directed toward improving students’ English proficiency and teachers’ professional development (TPD) ( Rashid et al., 2017 ; Kummin et al., 2020 ).

Despite unremitting efforts in many countries, ELE is still facing the problem of low average English skills. For example, students’ English skills in Turkey are not as good as expected ( Coskun, 2016 ; Özmen et al., 2016 ; Umunĉ and Raw, 2017 ). The survey by Wei and Su (2015) clearly showed that the subjects’ ES proficiency was generally low.

Many terms have been used to refer to the speaking aspects of the English language, e.g., “oral English,” “spoken English,” or “English speaking.” “English speaking” is the term used in this article. Speaking is different from writing, although both are productive skills, in that it is “transient, unplanned, context-dependent, oral/aural, and dynamic” ( Hughes, 2017 ). English-speaking (ES) has been treated as an indivisible language skill for learners in the language education fields of TESOL, EFL, and ESL.

How to improve ES ability, including teaching and learning approaches, influencing factors, and other related issues, have always been a focus of researchers. Thus, in view of the profound changes in society, politics, economics, and technologies, this article aims to give an overview of the current situation and trends regarding ES studies based on Scopus from 2010 to 2021. Moreover, it seeks to provide useful information for further ES teaching and learning research through visualized data analysis using VOSviewer 1.6.17 and Micro Excel.

Thus, the research questions (RQs) of this article are as follows:

RQ1. What is the bibliometric information regarding publications about ES teaching and learning in Scopus (2010–2021), including the number of yearly publications, authors, citations, country contributions, and keywords?

RQ2. What is the status of ES teaching and learning?

RQ3. What are the most influential authors in the field of ES teaching and learning?

RQ4. What are the trends in ES teaching and learning?

RQ5. What are the gaps in ES teaching and learning from the bibliometric information?

Materials and Methods

Bibliometric analysis refers to the cross-science of quantitative analysis of all carriers of knowledge by means of mathematics and statistics ( Broadus, 1987 ). The development of bibliometric software such as VOSviewer, Citespace, and Gephi, and the foundation of the big databases for academic documents such as Scopus, Web of Science, and Taylor and Francis make bibliometric analysis more feasible and practical ( Donthu et al., 2021 ). Meanwhile, according to Rogers et al. (2020) , the recommended minimum sample size for a bibliometric analysis is 200 entries.

Article Selection and Identification

Scopus was chosen as the database for this historical review and bibliometric analysis of ES education. This is because Scopus, as one of the world’s largest databases, covers a wide range of academic journals, conference proceedings, books, and other related publications with relatively high citation indexes and quality, much like the Web of Science ( Pham et al., 2018 ; Baas et al., 2020 ). Scopus is user-friendly in the sense that information can be conveniently retrieved through string retrieval. This study replicated the methodologies used by Lázaro (2022) and Kaya and Erbay (2020) . This article was conducted around RQs after the identification of some keywords as conditions for data mining.

Thus, 23,633 sample documents were first strictly extracted under the condition [TITLE-ABS-KEY (“English speaking” OR “English-speaking” OR “oral English” OR “spoken English”)] AND (“TESOL” OR “EFL” OR “ESL”). Then, the conditions of time span and document type were added for filtering from 2010 to 2021. Then, the articles, conference papers, reviews, book chapters, and books were chosen as the target document types. The detailed conditions can be seen in Table 1 .

Table 1. Retrieval conditions.

Finally, 1,893 documents were obtained. These were exported in the form of an Excel document with citation information, bibliographic information, abstract and keywords, funding details, and other information.

After strict data cleaning through thematic analysis of the abstracts by three researchers for more than three times, 361 sample documents remained, which were classified into four types of documents: journal articles (256; 70.91%), conference papers (79; 21.88%), book chapters (16; 4.43%), and reviews (10; 2.77%), covering more than 10 subject areas, such as social sciences, computer sciences, medicine, engineering, and arts and humanities.

Research Framework and Instruments

In the data selection step, sample documents were screened for information about authors, titles, years, citations, author keywords, index keywords, publishers, document types, countries, and author affiliations from Scopus under strict conditions. The sample documents were then uploaded to Excel and VOSviewer 1.6.17 during the data-processing step. Excel and VOSviewer 1.6.17 were used to perform the visualized bibliometric analysis of the number of publications per year, contributions of authors and countries, and keywords ( Chen, 2016 ; Van Eck and Waltman, 2017 ). Finally, the current situation, developing trends, research gaps, and lessons we can learn about ES teaching were sorted. Thus, the research framework is divided into four main steps, as shown in Figure 1 .

Figure 1. The conceptual framework.

Compared with studies on English writing and reading, studies on ES education are relatively very small in scale. Although only 361 sample documents conforming to the screening conditions were identified, it was still feasible to conduct a bibliometric analysis from the perspectives of the number of yearly publications, countries, authors, citations, and keywords.

Number of Publications by Year

According to the linear trend line in Figure 2 , the overall trend of the ES education literature in Scopus was on the rise from 2010 to 2021. The number of publications in 2021 was six times more than that in 2010, indicating that ES education was gradually beginning to be taken seriously by researchers.

Figure 2. Yearly publications on English-speaking teaching and learning (2010–2021).

However, there were some tortuous changes. In 2010, only 10 studies were identified, but the percentage of the high citation index occupied 50%. From 2011 to 2018, the number of documents published in this area presented an up-and-down curve. The number of publications was slightly lower in 2012 than in 2021. This might be due to the decreased demand for ES education as a result of the economic downturn in many emerging economies such as China, South Korea, and Brazil ( Reid, 2013 ). Yearly publications in this field increased from 2012 to 2013, but decreased again from 2013 to 2015, which was again in line with the global economic situation ( Mau and Ulyukaev, 2015 ). In 2015, the number of publications was more or less the same as in 2010. The reasons for this might be that world trade reduced during the global crisis from 2014 to 2015 ( Baber, 2015 ; Xu and Carey, 2015 ) or that no new research directions were explored during that time. After 2015, there was a continuous increase until 2017. After a subtle decrease in 2018, there was a significant accumulation in the number of publications from 2018 to 2021, showing a new growth trend. Especially in 2020 and 2021, when the COVID-19 pandemic brought disaster to the whole world, publications on ES education increased, reflecting the increasing requirement for ES communication during this time of global cooperation ( Sun and Lan, 2021 ). The influence of the date on the extraction of the sample documents was not very great, as it was 12 December 2021.

Contributions and Collaborations by Country/Region

The 361 sample records extracted in Scopus from 2010 to 2021 were associated with around 40 countries, showing the global distribution of interest by country in ES education.

Figure 3 shows the top 20 countries/regions publishing articles in this field, and they were responsible for 344 ES education publications (2010–2021) (accounting for 95.29% of the total). The countries with big and bright circles were the ones with the large number of the publications. Apart from the 75 publications contributed by the United States, the United Kingdom, Australia, Canada, Spain, and New Zealand, the remaining 269 publications were published by 10 Asian countries, accounting for 74.52% of the total sample documents, which implied the huge demand for the improvement of the learners’ ES skills in those countries. Mainland China contributed 128 publications, accounting for 40.44% of the total, followed by the United States, with 38 documents, accounting for 10.53%.

Figure 3. Density map of the top 10 countries.

Indonesia, Malaysia, and Thailand, countries in South-East Asia, occupied the third, fourth, and fifth positions, with 22, 22, and 18 publications, respectively. Thailand, Vietnam, the Philippines, and Bangladesh began to participate in country collaborations in recent years, in contrast to countries such as the United States, Australia, the United Kingdom, and Singapore where English is the native language or first language.

The citation network in Figure 4 shows only countries with more than five publications, which reflected the passive collaboration among the countries. As one of the native ES countries, home to many ELE approaches and English assessment tools such as TOFEL, the publications by the United States were cited 518 times (total link strength = 10). Meanwhile, the United Kingdom, home to IELTS, contributed 12 publications, which were cited 72 times (total link strength = 5). Malaysia, where ESL, contributed 22 publications, which were cited 90 times (total link strength = 17). Meanwhile, China, with an EFL context, ranked second with 128 publications, which were cited 395 times, and the total link strength achieved 24.

Figure 4. Country co-authorship in the field of English speaking teaching and learning.

The total citation number of the 361 sample publications was 1,828. Table 2 provides detailed information on the 15 countries that published the most cited articles. The publication and the corresponding citation rate of the United States, Australia, New Zealand, and Singapore showed huge contrasts, respectively, 12.92, 19.45, 19.5, and 13.83. The high citation rate may to a certain extent represent a high reference value, although it may also be influenced by some highly cited papers ( Schubert and Braun, 1986 ; Aksnes et al., 2012 ; Brika et al., 2021 ). Thus, it was concluded that the United States, Australia, New Zealand, and Singapore, where English was the official language, were the leading countries with high citation rates in the field of ES education studies. Similarly, native ES countries—the United Kingdom and Canada—showed relatively high citation rates of 5.83 and 6.14, respectively. Meanwhile, the citation rates of Asian countries such as China (4.73), Japan (6.35), South Korea (5.08), Vietnam (8.38), and Oman (6.75) indicated the progress and the relatively high reference value of publications on ES education studies in those countries. The non-ES European countries such as Spain received 4.57 in citation rates, which were much lower than those of the native ES countries.

Table 2. Description of the 15 countries that published the most cited articles in the field of English-speaking (ES) education studies in Scopus (2010–2021).

Co-authorship among the countries is shown in Figure 5 , which is a presentation of active collaborations. The co-authorship links among Malaysia, India, China, and the United States were linear. However, the collaboration in the map showed a tendency toward a partial focus. For instance, the United States was the main collaborating country for Indonesia, South Korea, Japan, Thailand, Canada, Australia, and Singapore. While China collaborated mainly with the United Kingdom, the Philippines, Turkey, Vietnam, and New Zealand. Thus, there was a need for an omnidirectional and multi-angle collaboration among the countries for ES teaching and learning research across the world for further studies.

Figure 5. The map of the co-authorship among the countries on English-speaking teaching and learning.

Author Contributions

Table 3 shows general information about the citations for the 361 sample documents in Scopus (2010–2021). As can be seen in Table 4 , the topics of the top 10 most frequently cited articles were concerned with the assessment of ES proficiency and fluency, teachers’ influence, lexical acquisition, and the facilitation of mobile social networks. The total citation number was 1,828. On average, each document was cited 5.06 times. An experimental study by Kang et al. (2010) that proposed suprasegmental measurement for pronunciation assessment from the perspective of accent and equipment use was the most frequently cited article, which was cited 134 times. A qualitative study by Ma (2012) was cited 66 times, ranking second among the top 10 most frequently cited. It focused on ES teaching methods and investigated the advantages and disadvantages of native and non-native ES teachers in practice. The third most frequently cited article, which analyzed the academic lexical demands and academic word list coverage for ES communications by means of corpus, was cited 61 times ( Dang and Webb, 2014 ). Obviously, most of the top 10 most frequently cited articles were published before 2016, except for the experimental studies by Sun and Lan (2021) on the application of e-learning to develop young learners’ ES competence, implying the emergence of new research topics after 2016 in ES teaching and learning studies.

Table 3. General citations of English-speaking (ES) education publications in Scopus (2010–2021).

Table 4. Top 10 frequently cited authors on English-speaking (ES) education in Scopus (2010–2021).

The top 10 authors with more than three articles in order, were Ismail, K. (6), Abdullah, M. Y. (5), Hussin, S. (5), Liu, M. (5), Habil, H. (4), Chen, Z. (3), Hasan, M. K. (3), Hwang, G. J. (3), Rao, Z. (3), and Seraj, P. M. I. (4), and the co-authorship relationships can be seen in Figure 6 . Seraj, P. M. I published four articles (one in 2020 and three in 2021) focusing on the topic of a flipped classroom. The other author with four publications was Liu M., studying the problem of Chinese EFL students’ anxiety, respectively, in 2013, 2018, 2018, and 2021. Rao Z. made three publications on the issues of native and non-native English teachers in China in 2010, 2016, and 2020.

Figure 6. Density map of the key words.

Keyword Analysis

There were 1,049 keywords among the 361 sample documents, and only 49 keywords (2.88%) appeared more than five times after merging synonyms and deleting extraneous words. This indicates that the number of high-frequency keywords was relatively small, which reflects the relatively extensive content of ES research in the field of language education. Table 5 lists the top 10 keywords ordered by the frequency of occurrence apart from the retrieval words, among which the frequency of “ES skill” was the highest, accounting for 3.43%. The remaining keywords with a frequency greater than 10 were “speech recognition” (32), “College English” (26), “e-learning” (22), “computer-aided instruction” (19), “learning system” (14), “native-English speaking teachers” (13), anxiety (13), “oral communication” (12), “virtual reality” (11), and “artificial intelligence” (10). Obviously, the gap in frequency among keywords is not very large.

Table 5. The top 10 most frequently occurring keywords on English-speaking (ES) education in Scopus (2010–2021).

The bibliometric co-occurrence analysis of keywords provided a convenient way to assess the state of the research field and spot hot issues ( Chen, 2016 ; Mutira et al., 2021 ; Sun and Lan, 2021 ). Meanwhile, importantly, keyword co-occurrence analysis can reflect the viewpoints of core academic articles and may be beneficial for researchers trying to keep up with research trends in a certain area ( Li et al., 2016 ; Shoaib et al., 2021 ). Figure 6 shows the density of keywords that appeared more than 10 times; the brightness of the color represents the heat color of the keyword studied. The more studies, the brighter the color ( Van Eck and Waltman, 2020 ). The colors of the keywords “English speaking skill,” “college English,” “computer-aided learning instruction,” and “speech recognition” were brighter than others. The other keywords, such as “speaking anxiety” and “e-learning,” were also brighter. To some extent, these brighter keywords reflected the research hotspots in the field of ES education from 2010 to 2021 in Scopus.

Keyword cluster analysis reflected the topics to some extent ( Yang et al., 2017 ). After combining synonyms (e.g., oral English and spoken English; computer-aided learning and computer-aided instruction; and native and non-native ES teacher) and the deletion of non-sense words (e.g., human, priority journal, and education), the keywords except the retrieval terms “English speaking,” “oral English,” “English-speaking,” “spoken English,” “EFL,” “TESOL,” and “ESL” were categorized into seven clusters with three main topics, as seen in Figure 7 . The keywords with red color dealt with the application of ICT in ES education, including items such as artificial intelligence, automatic speech recognition, computer-aided instruction, correlation methods, deep learning, information science, learning system, machine learning, quality control, correlation methods, corrective feedback, ES learning, oral communication, etc. Cluster 2 dealt with the cognitive factors influencing students’ ES skills or performance, such as attitude, EFL, English speaking performance, ES skill, the flipped classroom, motivation, speaking anxiety, and teaching methods, of which flipped classroom as a teaching method had the highest frequency of occurrence. Clusters 3 and 4 dealt with the application of ICT in college ES education, covering topics such as e-learning, engineering education, English speaking, learning, virtual reality, big data, college English, and educational computing. Cluster 7 dealt with the assessment of pronunciation or others.

Figure 7. The network visualization map of co-occurrence of keywords.

Keyword Changes and the Enlightenment to Research Topics

Figure 8 reveals a change in the time distribution of topics. It was obvious that most of the light-colored nodes were close to the keyword “college English,” while there were only a few around the keywords “child,” “preschool,” and “adolescent” after 2016. This shows that college students had become the main subjects of ES education studies instead of young learners.

Figure 8. The overlay visualization map of keywords according to year.

Meanwhile, studies with keywords related to the application of ICT, such as “big data,” “artificial intelligence,” “flipped classroom,” “speech recognition system,” and “virtual reality” in ES education, were emerging as a focus of research. Academic ES also began to attract researchers’ attention. Some researchers started to consider the development of twenty first-century skills during ES education. In addition, light-colored nodes of the keywords concerning teaching and learning modes (“continuous development,” “teaching method,” “EMI,” “error correction,” etc.), cognitive factors (“students’ interests,” “anxiety,” “motivation,” etc.), language skills (“ES performance,” “communicative skills,” “accuracy,” “fluency,” etc.), and language knowledge (“pronunciation,” “grammar,” etc.) remained the focus of research.

Limitations

The interpretation of the review should be very cautious due to some limitations. First, bibliometric analysis is a literature review method based on big data technology rather than synthesized thematic analysis. The data were collected and analyzed through the software. Thus, the accuracy of this analysis method is highly dependent on that of the software. The second limitation refers to the database. Though Scopus has covered the majority of the publications on ES teaching and learning worldwide, there are still some publications that were not included in the research.

This historical review and bibliometric analysis sought to better understand the current state of the research field, trends, and emerging research topics on ES education from 2010 to 2021. The results show that there was an increasing trend in the number of publications in this area from 2010 to 2021 in Scopus, indicating that ES education studies remained a necessary research topic, although the research population was not large. Countries with an ESL or EFL context, such as China, Japan, South Korea, Malaysia, Indonesia, and Saudi Arabia, paid more attention to the development of learners’ ES abilities and contributed more to ES education studies. However, the citation analysis revealed that native ES countries such as the United States, Australia, New Zealand, the United Kingdom, and Canada, were the major authorities or origins of ES education studies, which can also be seen by author contributions. Country collaboration analysis showed that the United States, China, India, and Malaysia acted as hubs of contact, establishing overall relationships within the collaboration network. In addition, the analysis of author distribution and collaboration revealed that there were constantly new researchers entering this field, but the lack of authors focusing on ES education over the long term and sustained research was still a problem. Further exploration of keywords revealed that the hot research issues encompass communicative skills, language knowledge, assessment, teaching or learning methods, ICT-related applications, and cognitive factors. Rather than focusing on ES education for young and adolescent learners, researchers showed a preference for investigating ES education for college students, catering to the increasing requirements of oral international communication. Meanwhile, topics on ICT application, autonomous learning, academic ES ability, and twenty first-century learning skills are gradually becoming hot areas for the improvement of ES teaching and learning worldwide.

JW was the research designer and executor of this study, participated in and completed the data analysis, and wrote the first draft of the manuscript. RA and L-ML gave suggestions when necessary. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

JW would like to express their gratitude to RA and L-ML who participated in this project.

Aksnes, D. W., Schneider, J. W., and Gunnarsson, M. (2012). Ranking national research systems by citation indicators: a comparative analysis using whole and fractionalized counting methods. J. Informetr. 6, 36–43. doi: 10.1016/j.joi.2011.08.002

CrossRef Full Text | Google Scholar

Baas, J., Schotten, M., Plume, A., Côté, G., and Karimi, R. (2020). Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies. Quant. Sci. Stud. 1, 377–386. doi: 10.1162/qss_a_00019

Baber, G. (2015). “The European Union’s legislative response to the financial crisis: a perspective taken from 2015,” in Global Financial Crisis: Causes, Consequences and Impact on Economic Growth , 89–158.

Google Scholar

Brika, S. K. M., Algamdi, A., Chergui, K., Musa, A. A., and Zouaghi, R. (2021). Quality of higher education: a bibliometric review study. Front. Educ. 6:666087. doi: 10.3389/feduc.2021.666087

Broadus, R. N. (1987). Toward a definition of “bibliometrics”. Scientometrics 12, 373–379. doi: 10.1007/BF02016680

Chen, M.-L. (2016). Development of corpus-based studies in second/foreign language acquisition and pedagogy from 1990 to 2015: a bibliometric analysis. Engl. Teach. Learn. 40, 1–38. doi: 10.6330/ETL.2016.40.4.01

Chen, Z., Goh, C., and Chuen, M. (2011). Teaching oral English in higher education: challenges to EFL teachers. Teach. High. Educ. 16, 333–343. doi: 10.1080/09500782.2011.609281

Coskun, A. (2016). Causes of the ‘i can understand english, but i can’t speak’ syndrome in turkey. J. Engl. Lang. Teach. 6, 1–12. doi: 10.26634/jelt.6.3.8174

Dang, T. N. Y., and Webb, S. (2014). The lexical profile of academic spoken English. Eng. Specif. Purp. 33, 66–76. doi: 10.1016/j.esp.2013.08.001

Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., and Lim, W. M. (2021). How to conduct a bibliometric analysis: an overview and guidelines. J. Bus. Res. 133, 285–296. doi: 10.1016/j.jbusres.2021.04.070

Ginther, A., Dimova, S., and Yang, R. (2010). Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Lang. Test. 27, 379–399. doi: 10.1177/0265532210364407

Hu, G., and McKay, S. L. (2012). English language education in East Asia: some recent developments. J. Multiling. Multicult. Dev. 33, 345–362. doi: 10.1080/01434632.2012.661434

Hughes, R. (2017). Teaching and Researching Speaking , 3rd Edn. New York, NY: Routledge.

Kang, O., Rubin, D., and Pickering, L. (2010). Suprasegmental measures of accentedness and judgments of language learner proficiency in oral English. Mod. Lang. J. 94, 554–566.

Kaya, M., and Erbay, E. (2020). Global trends of the research on COVID-19: a bibliometric analysis via VOSviewer. J. Ankara Health Sci. 9, 201–216.

Kummin, S., Surat, S., Amir, R., Maslawati, M., and Md Melor, Y. (2020). The effects of meta-discussion strategies toward low english proficiency students in oral english performance. Asia Pacific J. Educ. Educ. 35, 75–91. doi: 10.21315/apjee2020.35.1.5

PubMed Abstract | CrossRef Full Text | Google Scholar

Lázaro, I. G. (2022). Integration and management of technologies through practicum experiences: a review in preservice teacher education (2010–2020). Contemp. Educ. Technol. 14:e352.

Li, H., An, H., Wang, Y., and Huang, J. (2016). Evolutionary features of academic articles co-keyword network and keywords co-occurrence network: based on two-mode affiliation network. Phys. A 450, 657–669. doi: 10.1016/j.physa.2016.01.017

Ma, F., and Ping, L. (2012). Advantages and disadvantages of native- and nonnative-English-speaking teachers: student perceptions in Hong Kong. TESOL Q. 46, 280–305. doi: 10.1002/tesq.21

Mau, V., and Ulyukaev, A. (2015). Global crisis and challenges for Russian economic development. Russian J. Econ. 1, 4–29. doi: 10.1016/j.ruje.2015.05.003

Mutira, P., Meutia, Y. H., and Bastian, E. (2021). A bibliometrics analysis of management control system. Rev. Int. Geogr. Educ. Online 11, 2634–2649. doi: 10.48047/rigeo.11.05.160

Ojima, S. Matsuba-Kurita, H., Nakamura, N., Hoshino, T., and Hagiwara, H. (2011). Age and amount of exposure to a foreign language during childhood: behavioral and ERP data on the semantic comprehension of spoken English by Japanese children. Neurosci. Res. 70, 197–205. doi: 10.1016/j.neures.2011.01.018

Oliver, R., Vanderford, S., and Grote, E. (2012). Evidence of English language proficiency and academic achievement of non-English-speaking background students. High. Educ. Res. Dev. 31, 541–555. doi: 10.1080/07294360.2011.653958

Özmen, K. S., Cephe, P. T., and Kınık, B. (2016). Trends in doctoral research on English language teaching in Turkey. Kuram Uygulamada Egitim Bilimleri 16, 1737–1759. doi: 10.12738/estp.2016.5.0069

Pham, X. L., Nguyen, T. H., and Chen, G. D. (2018). Research through the app store: understanding participant behavior on a mobile English learning app. J. Educ. Comput. Res. 56, 1076–1098. doi: 10.1177/0735633117727599

Rahimi, M., and Zhang, L. J. (2015). Exploring non-native English-speaking teachers’ cognitions about corrective feedback in teaching English oral communication. System 55, 111–122. doi: 10.1016/j.system.2015.09.006

Rashid, R. A. B., Abdul Rahman, S. B., and Yunus, K. (2017). Reforms in the policy of English language teaching in Malaysia. Policy Futures Educ. 15, 100–112. doi: 10.1177/1478210316679069

Reid, C. D. (2013). World economic outlook April 2013: hopes, realities, risks. Ref. Rev. 28, 23–24. doi: 10.1108/RR-02-2014-0042

Rogers, G., Szomszor, M., and Adams, J. (2020). Sample size in bibliometric analysis. Scientometrics 125, 777–794. doi: 10.1007/s11192-020-03647-7

Schubert, A., and Braun, T. (1986). Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 9, 281–291. doi: 10.1007/BF02017249

Shoaib, M., Ali, N., Anwar, B., Rasool, S., Mustafa, R. E., and Shi, Z. (2021). Research visualization on teaching, language, learning of English and higher education institutions from 2011 to 2020: bibliometric evidence. Libr. Philos. Pract. 2021, 1–27.

Song, J. J. (2011). English as an official language in South Korea: global English or social malady? Lang. Prob. Lang. Plan. 35, 35–55. doi: 10.1075/lplp.35.1.03son

Sun, X., Xie, B., and Zhu, R. (2021). “A review of oral english teaching methodologies in chinese colleges,” in Proceedings of the 3rd International Conference on Literature, Art and Human Development (ICLAHD 2021) , (Amsterdam: Atlantis Press), 368–372. doi: 10.2991/assehr.k.211120.069

Sun, Y., and Lan, G. (2021). Research trends in ‘trans-’ studies on writing: a bibliometric analysis. System 103, 1–12. doi: 10.1016/j.system.2021.102640

Umunĉ, H., and Raw, L. (2017). Reassessing english studies in turkey. Ariel 48, 137–145. doi: 10.1353/ari.2017.0005

Van Eck, N. J., and Waltman, L. (2017). Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics 111, 1053–1070. doi: 10.1007/s11192-017-2300-7

Van Eck, N. J., and Waltman, L. (2020). VOSviewer Manual Version 1.6.16. Leiden: Univeristeit Leiden.

Wei, R., and Su, J. (2015). Surveying the English language across China. World Engl. 34, 175–189. doi: 10.1002/jid.3120

Xu, J., and Carey, R. (2015). Post-2015 Global governance of official development finance: harnessing the renaissance of public entrepreneurship. J. Int. Dev. 27, 856–880.

Yang, L., Sun, T., and Liu, Y. (2017). A bibliometric investigation of flipped classroom research during 2000–2015. Int. J. Emerg. Technol. Learn. 12, 178–186. doi: 10.3991/ijet.v12i06.7095

Keywords : English-speaking skills, bibliometric analysis, research trends, enlightenment, research state

Citation: Wang J, Abdullah R and Leong L-M (2022) Studies of Teaching and Learning English-Speaking Skills: A Review and Bibliometric Analysis. Front. Educ. 7:880990. doi: 10.3389/feduc.2022.880990

Received: 22 February 2022; Accepted: 01 June 2022; Published: 06 July 2022.

Reviewed by:

Copyright © 2022 Wang, Abdullah and Leong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Rohaya Abdullah, [email protected]

• Publisher Home

• About the Journal
• Editorial Team
• Article Processing Fee
• Privacy Statement
• Crossmark Policy
• Copyright Statement
• GDPR Policy
• Open Access Policy
• Publication Ethics Statement
• Author Guidelines
• Announcements

Systematic Review of Communicative Language Teaching (CLT) in Language Education: A Balanced Perspective

• Lhoussine Qasserras  

Lhoussine Qasserras

Search for the other articles from the author in:

Abstract Views 2711

Downloads 392

##plugins.themes.bootstrap3.article.sidebar##

##plugins.themes.bootstrap3.article.main##

This article presents a comprehensive review of communicative language teaching (CLT) in language education, considering both the positive impact of the approach and the criticisms raised by scholars. CLT has garnered significant attention in language education due to its emphasis on promoting effective communication and meaningful language use. This systematic review presents an impartial and well-rounded analysis of the pros and cons of CLT, citing previous studies in the literature review. It highlights the positive results of past research, demonstrating how CLT can enhance students’ communicative competence, language proficiency, cultural awareness, learner autonomy, critical thinking, and problem-solving skills. However, other studies have criticized CLT, raising concerns about its lack of explicit grammar and vocabulary instruction, insufficient preparation for formal writing, standardized exams, insufficient attention to individual learner differences, and emphasis on fluency over accuracy. By presenting both perspectives, this article aims to provide educators with a balanced understanding of CLT’s potential strengths and limitations, guiding them in designing effective language teaching practices.

Introduction

The field of language teaching has experienced the creation and growth of diverse approaches over the course of its historical progression. Several approaches and methods have gained recognition in the English Language Teaching (ELT) community, including the Grammar Translation Method, the Direct Method, the Audio-lingual Method, Suggestopedia, and Total Physical Response. These approaches or methods have undergone thorough examination and study by researchers and scholars. Every approach or method demonstrates specific areas of emphasis, weaknesses, and strengths, all solidly grounded in clearly defined theoretical frameworks. These approaches are frequently developed by integrating theoretical paradigms such as behaviourism, structuralism, constructivism, and universal grammar.

In this scholarly investigation, we embark on an examination of CLT, a pedagogical methodology that is subject to the same rigorous investigation as its precursors. CLT gives much value to meaning since communication is the ultimate goal of second language acquisition ( Magnan, 2007 ). This approach is also referred to as the notional functional strategy. It was established by the European Council in the 1970s, building on the work of language experts such as Wilkins, who analyzed the communicative meanings that a learner needs to express and understand. Instead of the traditional system of teaching grammar, he focused on meaning. For further illustration, the core idea behind CLT is that language instruction should not only focus on teaching grammar and vocabulary but also include the study of the meanings and functions of utterances in a variety of authentic contexts.

CLT is a combination of various language concepts, including sociolinguistics, functional linguistics, semantics, and pragmatics. Through the CLT approach, learners are encouraged to develop their communicative competence by using the language in meaningful and real-life situations. This approach emphasizes the importance of fluency over accuracy, making it a popular choice for language instruction in many contexts. The CLT approach, which originated in Britain, has been widely adopted in English as a Second or Foreign Language (ESL/EFL) classrooms worldwide ( Liao & Zhao, 2012 ; Ozsevik, 2010 ). This global acceptance has been extensively documented by Ellis (1996) . The aforementioned approach holds a crucial position within the domain of English language teaching (ELT), receiving acknowledgement from both linguists and language educators due to its pedagogical merit.

Methodology

Research problem.

To what extent does the implementation of communicative language teaching (CLT) in English language teaching (ELT) contexts contribute to the development of students’ language proficiency and effective communication skills, and what challenges and limitations may arise in balancing its benefits and potential drawbacks?

This research problem seeks to explore the dual nature of CLT in ELT by investigating its positive influence on language proficiency and communication skills development while also examining the challenges and limitations it may pose in terms of ensuring accuracy and addressing potential shortcomings in explicit language instruction. Understanding the overall impact of CLT on ELT will provide valuable insights for educators and policymakers seeking to optimize language teaching methods and curricula.

Research Objectives

This academic paper aims to clarify the principles and practices of the CLT approach, offering a thorough comprehension of its theoretical underpinnings and outlining its main distinguishing features. Moreover, our objective is to analyze the key benefits and constraints linked to the adoption of this instructional approach. This study serves as a vital contribution to the ongoing academic conversation on language instruction approaches, providing insightful perspectives on the dynamic nature of language education.

Research Questions

How does the implementation of communicative language teaching (CLT) in English language teaching (ELT) contexts affect the development of students’ communication skills and overall language proficiency, considering its merits as a student-centered approach?

What challenges and limitations arise in the application of communicative language teaching (CLT) in English language teaching (ELT), particularly in terms of accuracy and potential issues related to neglecting explicit language instruction?

Data Extraction

A thorough and organized method was used to collect data in order to do a full review of all the past and present studies that looked at the pros and cons of using CLT to teach English. A comprehensive literature review of scholarly papers, journal articles, research reports, and dissertations from reputable databases and academic repositories was conducted. To assure relevance and currency, we will concentrate on studies published between 1972 and 2023, with particular emphasis on studies from the last five decades.

The studies selected for analysis were carefully vetted based on their relevance and methodological rigor. Key findings about the pros and cons of using CLT in ELT were found by extracting data. These findings focused on things like student-centered learning, improving communication skills, implementation challenges, and possible limits in teaching language accuracy. By synthesizing the insights from these studies, a comprehensive understanding of the multifaceted nature of CLT in ELT was achieved. This way of collecting data made it possible for a fair and fact-based evaluation of CLT, which showed its good points and areas that might need more work.

The communicative language teaching approach has garnered significant attention in language education due to its stress on improving effective communication and meaningful language use. The results below provide a balanced analysis of the merits and demerits of CLT, drawing insights from past studies mentioned in the literature review.

Merits of CLT

Enhanced communicative competence and language proficiency.

Richards and Rodgers (2001) highlight that the CLT approach has prioritized real-life communication as a means of developing language competencies in authentic contexts. Evidence also shows that engaging students in communication tasks and activities allows students to fine-tune their ability to transmit and interpret messages proficiently, thereby enhancing their communicative competence ( Krashen, 1981 ). This approach has underscored oral communication and active engagement, helping students cultivate an advanced level of fluency and precision in language use. In this respect, once students practice the language, especially in meaningful contexts, their language proficiency is profoundly improved ( Savignon, 1972 ).

Additionally, Chaudhury (2015) conducted a study that is in accordance with the principles of CLT. This approach focuses on the improvement of communicative competence and language proficiency. The author sought to establish an interactive learning environment wherein students could actively participate in real-life communication scenarios by implementing communicative language activities, as opposed to the conventional ‘chalk and talk’ approach. The above-mentioned technique is compatible with the fundamental objective of CLT, which is to impart language learning that is both meaningful and practical. The use of a quasi-experimental, non-randomized pre-test and post-test control group design enhances the study by providing a research-supported framework for assessing the true effects of communicative language activities on learners’ communicative competence over a period of time. The results indicating a substantial disparity in post-test scores provide robust evidence for the efficacy of CLT-focused approaches in improving language proficiency and communicative competence. These findings highlight the impact of Chaudhury ’s (2015) research in advocating for the implementation of communicative language teaching methods in language education.

Fostering Learner Autonomy and Motivation

Ellis (1999) posits that the CLT approach, grounded on the idea of learner-centeredness, endeavours to facilitate students’ autonomy in assuming responsibility for their language acquisition process. Previous research has substantiated the assertion that CLT is a pedagogical technique that fosters the active participation of students, encouraging them to become engaged, creative, and self-directed learners ( Long, 1996 ; Snider & Schnurer, 2002 ). In this regard, the CLT approach facilitates the development of learner autonomy by affording students increased control and responsibility in their learning process.

In order to accomplish this objective, CLT offers students opportunities to engage in authentic communicative tasks, including but not limited to role plays, debates, dialogues, and puzzles. The primary objective of these exercises is to enhance students’ opportunities for verbal expression while concurrently reducing the teacher’s talking time. Furthermore, when students participate in these activities, whether in pairs or groups, working together towards a common goal, they assume complete responsibility for their own educational development ( Benson, 2011 ; Little, 1991 ).

Development of Cultural Competence

The CLT approach is widely recognized for its effectiveness in fostering cultural competence among language learners. This approach cultivates intercultural communication and understanding, as pointed out by Nunan (1991) , and encourages learners to engage with authentic language materials from diverse cultural contexts. When students interact with their peers from different backgrounds, they develop sensitivity and awareness of different cultures, leading to a deeper understanding of the language and its cultural implications.

Research has shown that CLT enables learners to communicate more effectively in multicultural settings by understanding and respecting different cultural norms and practices ( Byram, 1997 ; Kramsch, 1993 ). This approach’s emphasis on authentic material and peer interaction facilitates cultural immersion and provides occasions for learners to experience different cultures firsthand. Besides, the CLT approach encourages learners to use the language in its natural context, leading to a more inclusive understanding of the language’s cultural implications.

The connection of Arroussi (2014) with the concepts of communicative language teaching in relation to the promotion of cultural competency underscores the need to include culture in language instruction. The fact that the researcher saw that culture had a big effect on improving communication skills is in line with the basic ideas of CLT, which stress how important it is to use language in real, appropriate situations. From Arroussi’s point of view, language learners’ success in communication situations depends on how well they understand and value culture. This highlights the close connection between learning a language and communicating culturally. The above alignment accentuates the need to integrate cultural components into language instruction approaches, cultivating a more profound grasp of the desired language and boosting learners’ ability to communicate effectively.

The experimental study conducted by Hua (2011) presents persuasive evidence in favour of the relationship between the implementation of CLT and the development of cultural competence. The results of the study stress the efficacy of integrating cultural education into an English as a Foreign Language (EFL) course, as it led to a notable enhancement in students’ cultural consciousness and ability to communicate effectively ( Wu, 2008 ). This is in perfect agreement with the concepts of CLT, which prioritize the incorporation of authentic cultural contexts into language acquisition in order to improve learners’ proficiency in varied communication settings. Hua’s (2011) study provides evidence that being exposed to and comprehending diverse cultures not only increases students’ cultural awareness but also improves their communication skills, which is an ultimate principle of CLT ( Lee et al ., 2023 ). These findings offer significant support for the incorporation of cultural components into language instruction, which is consistent with the comprehensive approach to language acquisition advocated by CLT. This approach aims to equip students with the necessary skills to engage in meaningful and successful cross-cultural conversation.

Enhancing Critical Thinking and Problem-solving Skills

According to a study by Hasibuan and Batubara (2012) , the CLT approach is effective in improving learners’ critical thinking and problem-solving skills. The approach involves the use of communicative tasks that require learners to negotiate meaning in various contexts. By engaging in these tasks, learners are encouraged to think critically while communicating, leading to the development of cognitive abilities that extend beyond just language learning. Additionally, CLT promotes critical thinking about language itself, which can help students gain a deeper understanding of the nuances and implications of the language they are using.

Furthermore, the development of critical thinking skills and oral communication proficiency is widely recognized as a fundamental necessity in the context of 21st-century education. Numerous studies have been conducted to examine several methodologies and techniques aimed at enhancing critical thinking and oral communication abilities within the classroom setting ( Halpern, 2003 ). Existing literature provides evidence that the incorporation of communicative language teaching practices, such as in-class debates, discussions, public-speaking exercises, and role-play activities, enhances an atmosphere conducive to understanding, introspection, inquiry, negotiation, and logical evaluation. Conversely, the implementation of critical thinking exercises in language education can help in the development of oral competency skills.

Demerits of CLT

Neglect of explicit grammar and vocabulary instruction.

One of the primary criticisms directed at the communicative language teaching approach is its alleged neglect of explicit grammar and vocabulary instruction ( Larsen-Freeman, 2000 ; Spada & Lightbown, 2008 ). Nassaji (2000) stresses the importance of integrating form-focused activities within an integrative approach to L2 teaching that balances meaning with form. The author argues that an overemphasis on meaning, as seen in traditional communicative language teaching, can affect learners negatively if they neglect the language structure. Consequently, to ensure the development of proficient language learners, form-focused activities should be incorporated within a communicative and meaningful context. The article provides various pedagogical strategies, such as the use of corrective feedback, explicit instruction, and consciousness-raising tasks, to help learners develop their form-focused abilities. These strategies aim to enhance learners’ capacity to notice and analyze the form of language, leading to greater accuracy and fluency in language production.

Likewise, in his work, Grim (2008) places much emphasis on the significance of form, particularly for beginning-level learners and cultural lessons. The author argues that integrating language structures into cultural lessons at the early stages of French L2 classes can be more effective for learning second language grammar, vocabulary, and cultural content in intermediate French L2 classes. Grim’s approach departs from traditional communicative language teaching, which gives much more value to meaning than form. Instead, Grim contends that giving equal importance to form and meaning can boost language learning outcomes.

El-Dakhs (2015) argues that without systematic and explicit instruction, students may struggle to develop a solid foundation in language structure and vocabulary knowledge. This limitation may hinder learners’ ability to produce accurate and grammatically correct language. For example, the author’s critique of communicative language teaching addresses significant language accuracy issues. CLT stresses meaning and communication, while El-Dakhs’ research shows the risks of emphasizing meaning alone in language training. This view is supported by research that suggests pure CLT learners may lack linguistic accuracy, a key component of language mastery ( Ahmad & Rao, 2013 ). El-Dakhs’ criticism focuses on the necessity for a balanced approach to language development that includes communicative competency and formal teaching. Recognizing the importance of language accuracy allows educators to take a more holistic approach that ensures students communicate successfully and have the grammatical and linguistic precision needed for language competency. This perspective enhances CLT discourse by highlighting its benefits and weaknesses in language instruction.

Limited Preparation for Formal Writing and Standardized Exams

According to Bachman (1990) , some researchers express concerns that the CLT approach may not provide students with sufficient preparation for formal writing and standardized exams that require precise grammar and vocabulary usage. While CLT’s major focus on communicative activities is critical for effective communication, it may not fully address the intricacies required for formal language production. Consequently, students may lack the necessary skills to perform well in language assessment contexts that demand precise grammar and vocabulary usage. Furthermore, some sceptics argue against implementing CLT, as exemplified by Norland and Pruett-Said (2006) . These critics express apprehension regarding the potential neglect of reading and writing abilities within the framework of CLT. They elucidate that if educators fail to instruct grammatical regulations or vocabulary, learners may ultimately experience difficulty expressing themselves clearly and effectively throughout interpersonal exchanges. Finally, a limitation of communicative language teaching is its applicability mainly to small class sizes. In the context of a large classroom, teachers may allocate an important amount of time towards assessing the language proficiency of individual students and rectifying any errors they may have made ( Wilkins, 1976 ).

Prioritization of Fluency over Accuracy

While fluency is an essential component in language production, it is important to maintain a balance between fluency and accuracy. The Communicative Language Teaching approach accentuates fluency, which may contribute to the development of fossilized errors if accuracy-focused activities are neglected ( Ellis, 2003 ). These errors could become ingrained and difficult to correct, which can negatively impact language learning. Thus, it is crucial to include accuracy-focused activities in language learning to ensure that learners are not only fluent but also accurate in their language production. Striking a balance between fluency and accuracy can help learners develop a well-rounded and effective communication skill set.

Similarly, according to Brumfit and Johnson’s (1979) publication, the communicative approach to language learning, which gives emphasis to the ability to communicate effectively in a language, should not be prioritized to the detriment of language accuracy. The authors suggest that a balanced approach that incorporates both communicative competence and language accuracy is essential to achieving optimal language learning outcomes. This implies that a singular focus on communicative competence may not suffice to equip learners with the necessary language skills required for effective communication in real-life situations. As such, educators and language learners alike must strive to achieve a balance between the two approaches in order to maximize language learning outcomes.

In addition, Gerges’ (2016) research regarding the potential weakness of CLT brings attention to an essential aspect of language education. Although this approach is highly effective in improving oral fluency and communicative competence, it is essential to recognize that language proficiency encompasses more than just speaking abilities. The findings of Gerges draw attention to the significance of maintaining a balanced approach in the training of language. By placing exclusive emphasis on oral fluency, there is a possibility of unintended repercussions, including a potential decrease in the development of grammar skills and writing proficiency. This underscores the imperative for an all-inclusive strategy that incorporates both fluency and accuracy in language use.

Over the years, the communicative language teaching (CLT) approach has become increasingly popular in language education. This approach emphasizes the importance of promoting effective communication and meaningful language use in the classroom. This discussion aims to provide a broader analysis of both the merits and drawbacks of CLT, drawing upon relevant findings from the literature review mentioned above.

Research Question 1

Enhancing communication skills.

One of the main advantages of CLT is its rigorous emphasis on honing students’ communication proficiencies. This approach encourages learners to apply language skills in real-life scenarios, enabling them to develop effective communication abilities across diverse contexts. This pedagogical approach fits well with modern theories about how people learn languages, like Krashen’s Input Hypothesis ( Krashen, 1981 ), which says that people learn languages best when they are exposed to input that is just a little bit above their current level of proficiency. This helps them communicate meaningfully.

Promoting Student-centered Learning

Another advantage of CLT is that it promotes student-centered learning. In a CLT classroom, the teacher acts as a facilitator who guides and supports student interactions. This approach encourages students to take responsibility for their own learning, making it more engaging and motivating for them. For instance, Long’s Interaction Hypothesis ( Long, 1996 ) complements CLT by underscoring that interaction and negotiation of meaning during language use promote language acquisition, echoing the fundamental concept in CLT of student-centred, communicative learning. Savignon’s Communicative Competence Theory ( Tarone, 1983 ) further supports CLT’s objectives by highlighting the importance of teaching language within real-life contexts, a core principle of CLT. Additionally, Hasibuan and Batubara (2012) agree with CLT’s principles, confirming that this approach allows learners to develop their critical thinking and problem-solving skills as they collaborate with their peers. These results show that CLT goes hand in hand with language acquisition theory, making it clear that it is a way to teach that focuses on building good communication skills.

Research Question 2

Difficulties in implementation.

Nevertheless, CLT is not without its limitations. One of the main criticisms of this approach is that it can be difficult to implement in practice. For instance, it can be challenging for teachers to design and manage activities that meet the diverse needs of their students. While the acceptance of CLT is widespread among educators in English-speaking nations in the Western hemisphere, its adoption in Asia and non-native English countries is not as prevalent. According to Kustati (2013) , the efficacy of the strategy in Southeast Asian nations such as Vietnam, Cambodia, Indonesia, Thailand, Myanmar, Laos, and Malaysia is limited. The degree of success observed in these nations is constrained and subject to variation based on the specific country and socio-cultural circumstances. Similarly, Koosha and Yakhabi (2013) highlight that EFL instructors encounter problems and challenges while attempting to implement the CLT approach and optimize the learning experience in their classrooms.

Neglecting Language Accuracy

Some research suggests potential drawbacks. For example, Spada and Lightbown’s study (2008) indicates that explicit instruction, especially in grammar, can be beneficial alongside CLT. This raises questions about whether CLT, which primarily focuses on communication, might neglect the role of explicit language instruction, potentially hindering learners’ language development.

To address this criticism, language learning programmes could incorporate explicit grammar and vocabulary instruction into their curriculum. Explicit instruction can help learners develop a solid foundation in language structure and vocabulary knowledge, which is essential for effective communication. According to Canale and Swain (1980) , linguistic and communicative competence is an essential aspect of language instruction and acquisition, particularly in the context of ESL/EFL education. The authors further elaborate that communicative competence is predicated upon the interplay between grammatical competence and knowledge, as well as socio-linguistic competence and knowledge. In order to facilitate effective communication, it is essential to possess knowledge of the appropriate content, delivery, and timing of one’s message.

In 1985, Swain put forward the Output Hypothesis, which disagrees with the receptive approach of CLT and supports active language production. This displays how significant it is to include language production activities in the CLT framework. Moreover, Nation and Newton (2009) accentuate the significance of vocabulary instruction alongside communicative activities, pointing out a potential gap in CLT’s focus on vocabulary acquisition. These criticisms reveal that CLT may need to strike a better balance between communication-centred teaching and addressing these potential limitations. To avoid such negative consequences, it is crucial to incorporate accuracy-focused activities into language learning programmes.

In short, the study of communicative language teaching (CLT) demonstrates that it fits with current theories of how people learn languages because it focuses on real communication, interaction with others, and real-life situations. The efficacy of this pedagogical technique has been demonstrated in its ability to enhance learners’ communication abilities and promote a student-centered learning environment. However, it is critical to acknowledge potential constraints, such as the requirement for explicit instruction in language, the necessity for active production of language, and the acquisition of vocabulary. These factors have the potential to affect the overall effectiveness of language learning within the Communicative Language Teaching (CLT) framework.

In summary, the discourse surrounding communicative language teaching features its prominence and pertinence within the realm of language education. The CLT approach, which is based on learner-centeredness and meaningful dialogue, presents a number of benefits. This approach promotes active participation from students, facilitates the application of language in authentic contexts, and improves their ability to communicate effectively in accordance with current theories of language learning. Furthermore, the implementation of the CLT approach fosters the development of student autonomy, enhances critical thinking skills, and cultivates problem-solving ability.

However, it is imperative to recognize that CLT is not devoid of constraints. The challenges encompass obstacles encountered during the implementation process, particularly in countries where English is not the local language, as well as the possibility of disregarding language accuracy and explicit instruction. Achieving a harmonious equilibrium between pedagogical approaches that stress communication and the need to acknowledge and overcome these constraints is crucial to facilitating successful language acquisition.

Implications

The implications of the findings presented in this discourse hold substantial importance for practitioners and scholars in the field of language education. Educators ought to persist in leveraging the advantages of CLT by prioritizing the promotion of meaningful conversation, interactive engagement, and the application of language skills in authentic, real-world contexts. Yet, it is imperative for educators to take into account the incorporation of explicit language instruction, engaging language production activities, and vocabulary teaching within their communicative language teaching methodologies. This approach is fundamental in order to effectively tackle any potential constraints and offer a comprehensive language learning experience for students.

Future Research

In order to augment our understanding of the efficacy of communicative language teaching and explore prospective avenues for development, it is recommended that future research endeavours concentrate on various domains:

• Comparative Studies: Conduct comparative studies to analyze the outcomes of CLT-based instruction in comparison to other language teaching methodologies. These studies aim to measure the effectiveness of CLT in various circumstances.
• Language Production Activities: Examine the effects of integrating additional language production activities inside the CLT framework. Furthermore, it seeks to explore the ideal equilibrium between the enhancement of receptive and productive language skills.
• Vocabulary Acquisition : Look into new ways to teach vocabulary within the framework of communicative language teaching. The goal is to help students build a strong vocabulary foundation along with their communication skills.
• Teacher Training: Examine the effects of offering teacher training programmes with an emphasis on CLT in order to enrich educators’ proficiency in efficiently implementing CLT methodologies.

In conclusion, it can be argued that CLT holds significant pedagogical value. However, it is crucial to emphasize the prominence of ongoing study and adaptation in order to fully optimize its advantages and effectively address any potential limitations. This approach ultimately aims to offer learners a whole language learning experience.

Most read articles by the same author(s)

• Najib Slimani, Lhoussine Qasserras, Teacher Trainees’ Stance Towards Distance Learning During COVID-19: An Experiential Outlook , European Journal of Education and Pedagogy: Vol. 3 No. 4 (2022)

Similar Articles

• Angeliki Ypsilanti, Ioannis Karras, “Coursebook or No Coursebook”? A False Dilemma , European Journal of Education and Pedagogy: Vol. 5 No. 2 (2024)

You may also start an advanced similarity search for this article.

• Open access
• Published: 02 May 2024

Teaching foundational language equity concepts in the pre-clinical curriculum

• Maria Gabriela Valle Coto 1 ,
• Reniell X. Iñiguez 2 ,
• Marina A. Lentskevich 3 ,
• Syeda Akila Ally 3 ,
• Julia F. Farfan 3 ,
• Yoon Soo Park 3 ,
• Ananya G. Gangopadhyaya 3 &
• Pilar Ortega   ORCID: orcid.org/0000-0002-5136-1805 3 , 4  

BMC Medical Education volume  24 , Article number:  485 ( 2024 ) Cite this article

177 Accesses

Metrics details

Despite the prevalence of non-English languages in the US population, existing medical training to teach communication with linguistically diverse communities is limited to electives or solely focuses on medical interpreting. Language-appropriate communication skills are seldom comprehensively integrated in medical education. This study describes the development and evaluation of an intervention to teach foundational language equity concepts.

The authors implemented a pre-clinical language equity course at three medical school campuses between August 2020 and March 2022. Sessions focused on the impact of language in health, physician language proficiency standards, and working with medical interpreters. The study sought to (1) understand students’ language skills and prior clinical experiences with patients with non-English language preference and (2) evaluate the curriculum’s impact. Students self-reported their language skills and experiences as part of a voluntary pre-questionnaire. Pre and post-questionnaires evaluated knowledge, attitudes, and intent to apply language equity concepts. Descriptive statistics and chi-squared tests were used to examine trends; themes were identified from free-text responses.

Overall, 301 students completed the course, 252 (83%) completed at least one questionnaire; for each session, between 35% and 46% of learners completed both pre and post-questionnaires. Three quarters (189/252) reported non-English languages. Over half (138/252) reported previous non-English language patient care, and 28% (62/224) had served as ad hoc (untrained) interpreters. Only two students (< 1%) had ever been assessed for medical language abilities. Students demonstrated improved post-course language equity knowledge, strategies for interpreter-mediated encounters, and likelihood to report a plan for language skills assessment (all p  < .001). Most plans were multifaceted (61%, 38/62), involving goals like completing a language course, taking a proficiency exam, openly discussing skills and uncertainties with team members, and increasing professional interpreter utilization.

Conclusions

A longitudinal language equity curriculum can be feasibly integrated in pre-clinical education, highlight the linguistic diversity of the student body, and serve as a first step in ensuring that all students have a strong language equity foundation prior to clinical rotations. Future steps include evaluating the intervention’s potential long-term effects on professional interpreter utilization, student clinical performance, and institutional culture that promotes multilingualism.

Peer Review reports

Over 67 million individuals in the United States speak a non-English language at home [ 1 ]. At least 38% of these individuals speak English less than very well and are labeled as having limited English proficiency (LEP) [ 2 ]. Recently, a more inclusive term, non-English language preference (NELP), has emerged to describe individuals who can best communicate in a language other than English with respect to a particular type of service or encounter, such as medical care [ 3 ]. When individuals with NELP seek healthcare, language discordance between clinicians and patients often leads to suboptimal communication and poor outcomes [ 4 ] Language discordance can be successfully addressed either through a professional interpreter (who does not eliminate the language discordance but serves as a mediator for effective communication between clinician and patient) [ 5 ] or by matching the patient with a language-concordant clinician (thereby achieving language-concordant care) [ 6 ]. While mandates from the federal government require meaningful healthcare access in a patient’s preferred language [ 7 ], implementation and utilization of language services (e.g., medical interpreters) and confirmation of non-English skills for clinical use vary significantly across states and healthcare systems [ 8 , 9 ].

Data show that hospitals frequently underutilize professional interpreters [ 10 , 11 , 12 , 13 ] and medical students and physicians use their non-English skills in patient care even when they recognize their skills are limited [ 10 , 14 ]. A recent risk assessment study evaluated why physicians choose to ‘get by’ with limited language abilities and identified “lack of physician knowledge and skills” related to language-appropriate communication strategies, such as working with interpreters, as the most common factor and the most amenable to intervention [ 15 ].

To improve clinicians’ skills in language-appropriate care, some prior work has explored medical education curricula related to language equity. A 2017 US survey by Himmelstein et al. gathered responses from a quarter of US allopathic medical schools and found that 29 out of 38 schools “provided specific instruction addressing how to work with medical interpreters and/or patients with LEP [ 16 ].” The rest of the survey focused on training related to working with interpreters and did not address provision of language-concordant care or student recognition of their own language skills/limitations. Himmelstein et al. point out that “a few schools reported having a large bilingual student population and therefore did not see the need for this instruction” – an alarming statement that illustrates how institutions may view language-appropriate care as the responsibility of some but not all clinicians. In stark contrast to the hundreds of hours dedicated to English clinical skills education, institutions commonly check the box on teaching language-appropriate care by categorizing all non-dominant linguistic groups (of which more than 350 exist in the US) as LEP and providing a one-time educational intervention. Even clinicians with full proficiency in one or more non-English languages are likely to encounter patients with whom they are language-discordant.

Some published curricula [ 3 , 17 , 18 , 19 , 20 , 21 , 22 ] have sought to increase physician trainee knowledge on how to work with professional interpreters. Unfortunately, data demonstrate that students and clinicians persistently underutilize medical interpreters and use their own language skills to “get by” in taking care of linguistically minoritized patients.10 Hence, it is not enough to teach student how to work with interpreters; they also need to know when to do so – a clinical decision-making skill that should factor in the clinician’s language skills and limitations, the medical complexity and urgency of the situation, and management of limited resources (e.g., requesting an onsite interpreter versus using a telephonic interpreter).

Medical language courses (e.g., medical Spanish) aim to improve language concordance for direct patient-clinician communication and teach learners to recognize their limitations in the target language [ 23 ]. However, these opportunities are typically offered as electives, focus on one language, and require a minimum language proficiency level; [ 24 ] as a result, they are only accessible to a small subset of trainees. Additionally, multilingual students or physicians may feel overburdened by frequent requests to serve as ad hoc interpreters themselves [ 14 ]. Efforts to improve language-appropriate healthcare through medical education should target all students rather than only those eligible for a language course, and should do more than teach students how to work with interpreters. A comprehensive and nuanced approach to language equity is needed to prepare future physicians for effective communication with all patients.

To address current gaps, we developed and evaluated an intervention to longitudinally integrate foundational language equity concepts within the required pre-clinical curriculum at three medical school campuses.

The University of Illinois College of Medicine (UICOM) has historically been home to one of the most diverse student populations in the mainland US [ 25 ] and has three campuses: Chicago, Peoria, and Rockford each in an urban, suburban, and rural setting, respectively. UICOM’s required Doctoring and Clinical Skills (DoCS) longitudinal course spans the 18-month pre-clinical curriculum.

Educational intervention

We developed three language equity sessions to be imparted within the DoCS curriculum at all three UICOM campuses. To develop each session, we created a coalition comprised of a faculty member with content expertise in language-appropriate healthcare, a DoCS faculty member, and several medical students. The group was linguistically diverse, representing eight non-English languages (Belarusian, Bengali, Hindi, Polish, Russian, Spanish, Ukrainian, and Urdu). The coalition met monthly between August 2020 and March 2022 to refine the content, learning activities, and questionnaires for each session.

We reviewed conceptual frameworks in the literature, seeking approaches that would allow us to best address the identified gaps in language equity education [ 26 ], and ultimately selected two: First, self-determination theory, proposes that addressing student competence, autonomy, and relatedness improves learning, and has been applied in medical education through small group activities and case-based learning [ 27 ]. Understanding how to build linguistic competence, graduated autonomy in language-concordant care, and the interdependence/intersection of clinician skills and patient language preference are aspects of language equity that have not been formally addressed through prior curricula and can be explored through a self-determination lens. Second, community cultural wealth [ 28 ] is a framework based on social capital theory [ 29 ] that posits that individuals’ cultural—including linguistic—assets can be sources of power and social mobility. Medical students’ pre-existing language skills intersect with their early childhood lived experiences, personal and familial national origin, ethnicity, and cultural heritage. If acknowledged and promoted, this linguistic capital can not only enrich the classroom but also healthcare interactions with patients.

Drawing on these conceptual frameworks and based on previously identified gaps in physician knowledge [ 15 ] and structural barriers to language-appropriate care [ 30 ], each session (Table  1 ) focused on one of three main areas: (1) the role and impact of language in health, (2) physician language proficiency standards for clinical use, and (3) working effectively with medical interpreters. One faculty member led all three sessions, and the coalition recruited and trained faculty, residents, and senior medical students with prior experience caring for NELP patients to serve as small group facilitators. Each session involved two hours of pre-session student preparation, consisting of three or four articles and a 90-minute live virtual meeting with several interactive components (detailed in Table  1 ).

Participants

All 301 medical students in the class of 2024 at the three medical school campuses (182, 65, and 54 at Chicago, Peoria, and Rockford campuses, respectively) participated in the language equity curriculum and were invited to complete voluntary pre- and post-session questionnaires. Informed consent was obtained from all participants. Pre-questionnaires were sent approximately one week prior to the session, and a reminder and opportunity to complete the survey was provided during the first few minutes of the session. Post-questionnaire links were provided in the final five minutes of the session, and students received an email reminder within one week after the session. The University of Illinois Institutional Review Board approved this study on August 15, 2017 (protocol# 2017 − 0482).

Questionnaires

Coalition leaders created questionnaires by reviewing the available literature and applying their experience as multilingual clinicians and trainees. We identified one validated tool to classify student language proficiency, the Interagency Language Roundtable scale for healthcare, or ILR-H, a self-reporting tool validated for use by health professionals [ 37 ]. Due to the paucity of questionnaires in the literature to gather and track data regarding language use in healthcare, other items were developed using research team member expertise following guidance from Artino et al. [ 38 ] Questionnaires were piloted with student members of the coalition prior to implementation.

Pre-session questionnaires asked students to indicate their language skills and prior experiences related to language in healthcare. Specifically, we asked about any prior training or exposure related to medical interpreting and caring for patients with NELP. Items with multiple choice responses allowed for optional free-text responses. For the free-text items that would be used for qualitative analysis, an ethnographic approach was used since the goal was to enable respondents to describe prior experiences relevant to the care of linguistically diverse populations.

Some items aimed to assess changes in learner knowledge and attitudes pertaining to each session’s topic. Knowledge questions were multiple choice or true/false items. Attitudes questions with 4-point Likert scale response options (Strongly Agree to Strongly Disagree) asked about student confidence in several common situations, such as using their non-English language skills for patient care, or recognizing potential communication errors that may occur when providing medical care for patients with NELP. Finally, the post-session questionnaires elicited student intent to apply the concepts learned and gathered feedback, including suggestions for future improvements.

Statistical analysis

Aggregate data on self-reported race and ethnicity for the class of 2024 were used to inform overall descriptive statistics about the cohort of participants. We used descriptive statistics (frequencies and proportions) to examine trends in questionnaire responses. To evaluate internal structure validity evidence of survey responses, we used Cronbach’s alpha to examine internal-consistency reliability. To evaluate validity evidence supporting relations to other variables, we examined pre-post changes to learners’ responses. More specifically, responses were dichotomized (Agree/Strongly Agree v. Disagree/Strongly Disagree) to facilitate interpretation; learner responses between pre- and post-session changes in knowledge and attitudes were compared using chi-squared tests. We conducted analyses using both paired pre-post data (restricting analysis to learners who completed both pre- and post-questionnaires) and unpaired data (using all data collected); we opted to display the results of both analytical methods to maximize the inclusion of data from all respondents. Data compilation and analyses were conducted using Stata 17 (College Station, TX).

Once data were fully deidentified, we reviewed qualitative responses using Microsoft Excel. Coding and inductive analysis followed the Standards of Reporting Qualitative Research [ 39 ]. Two research team members (M.V.C and J.F.) reviewed qualitative responses from selected questionnaire items with free-text response opportunities and identified codes and sub-themes individually for each item. We further analyzed sub-themes and grouped them together in overarching themes.

Each of the three live language equity sessions were held at the three sites between December 2020 and May 2021. Across the three sites, all 301 students completed the full curriculum, and 83% (252/301) responded to at least one of the questionnaires; paired response rates, indicating students who completed both pre- and post-questionnaires, were 46% (session one), 42% (session two), and 35% (session three). The overall racial/ethnic demographic distribution of students was as follows: 38% White (115/301), 24% Asian (73/301), 15% Hispanic/Latinx (46/301), 12% Black/African American (36/301), < 1% American Indian/Alaska Native (1/301), 5% Multi-race (16/301), and 5% Unknown (14/301). Response rate per site was 96% (175/182), 69% (45/65), and 81% (44/54) at Chicago, Peoria, and Rockford campuses, respectively.

Descriptive statistics

Overall, 252 (83%, 252/301), 158 (86%, 158/182), and 224 (74%, 224/301) students completed the pre-questionnaires for the first, second, and third sessions, respectively. Following each session, 117 (38%, 117/301), 67 (36%, 67/182), and 79 (26%, 79/301) students completed the post-questionnaires for the first, second, and third sessions, respectively. Internal-consistency reliability (Cronbach’s alpha) of survey items (24 questions with rating responses) was 0.73, demonstrating good reproducibility. As previously noted, we ran all analyses both using unpaired data to maximize use of all student responses, and using paired data that restricted the analyses to respondents who responded to both pre and post-questionnaires. Depending on the session, the paired response rate ranged between 35 and 46%. Overall, while there are modest changes in effect sizes when analyzing paired versus unpaired data, our findings indicated no changes in statistical inference (i.e., statistical significance remained the same across the vast majority of items). Additionally, pre and post-subgroups in the unpaired data did not differ in their linguistic profile ( p  = .097).

Most respondents (75%; 189/252) indicated having skills in a language besides English, with over 38 languages represented. The most common non-English language spoken across all three campuses was Spanish (46%, 116/252), followed by French (8%, 19/252), and Mandarin (4%, 11/252). When analyzed separately by campus, Arabic was reported by four Peoria students (9% of 45 respondents), making it the second most common non-English language reported on that campus, and Urdu was as common as Mandarin in the Rockford campus (each reported by 2/44 respondents, tying for third most common language reported).

Students’ prior language-related experiences in healthcare

Students reported experiences in three specific categories: (1) interactions with linguistically diverse patients, (2) assessment of non-English language skills, and (3) any prior training related to working with medical interpreters.

Interactions with linguistically diverse patients

More than half of session one respondents reported previous experience providing medical care to patients with NELP (55%, 138/252), which was true across respondents who were multilingual as well as those who were monolingual English-speaking: 56% (104/186) of multilingual students and 53% (34/64) of monolingual students reported previously caring for this population ( p  = .699) and regardless of campus ( p  = .095).

We asked students to select and describe the nature of their prior interactions with NELP patients. The largest subset (28%, 62/224) had been asked to serve as interpreters, of whom ten (5% of 224) reported having received any training on how to interpret and none had been certified. The majority who reported serving as ad hoc interpreters indicated playing this role with their own family members, and some in clinic/hospital settings when volunteering or shadowing or when working in other healthcare jobs prior to medical school.

When analyzing associated free-text responses, we identified six themes (Table  2 ). Most (113/172, 66%) described experiences an observer in the care of patients with NELP. Thirty-four students (20% of 172) elaborated on their experiences as ad hoc interpreters. For example, one described that “Growing up I had to often translate for my mother whenever she took us to the pediatrician or when she needed to see her PCP.” A few (22/72, 13%) described the direct care of patients with NELP, and three students had done so as part of research working with patients enrolled in clinical trials.

Prior experiences assessing non-english language skills

In session two, we asked students to report whether they had ever been assessed for their non-English language skills in healthcare and whether they had been taught self-assessment techniques. Two students reported having had medically contextualized language assessment; one described having been interviewed to confirm language skills in Spanish prior to volunteering as an assistant at a clinic and another as a community health educator. Twenty-six students (16% of 158) reported having taken and passed some form of general (non-medical) language assessment; most of these respondents described advanced placement language courses or exams in high school or minors/certificates in college (e.g., Spanish, Chinese, Japanese, French, and Korean). Very few (3/158, 2%) reported being exposed to any language tools for progressive self-assessment.

Prior training related to working with medical interpreters

In session three, over half of the respondents (144/224) reported previous exposure to patient care mediated by a professional medical interpreter. Most reported having received partial training to work with interpreters (153/224, 68%), but when invited to elaborate through free-text responses, many of the respondents (26/54, 48%) explained that they were referring to our preceding language sessions given as part of this language equity curriculum. One student explained, “Previous sessions have discussed the importance of using a medical interpreter to communicate and understand the nuances of explanations and to prevent missing vital details in patient care.” Other training themes identified included self-study opportunities to learn these skills (6/54, 11%) and partial training in working with interpreters as part of research experiences (1/54, 2%).

Post-session change in knowledge and attitudes

Across the three sessions, student knowledge and attitudes improved for most questionnaire items (Table  3 ). In the first session, students reported increased confidence in explaining the role of language in health and in addressing language issues that may arise in healthcare settings ( p  < .001). Students were less likely to indicate that “language is a barrier” to quality medical care ( p  < .001; p  = .015 for unpaired and paired responses, respectively), suggesting that the session helped some students gain a more nuanced understanding of language as an opportunity to improve quality medical care rather than as a barrier. However, after the first session, students were more likely to respond incorrectly to the definition of medical interpreter ( p  = .003 [unpaired]; p  < .001 [paired]) and to report discomfort with the prospect of mixing languages during communication with patients ( p  = .010 [unpaired]; p  = .004 [paired]), suggesting that one session about language and healthcare alone is insufficient at gaining the necessary skills to provide language-appropriate care.

Following session two, students were more likely to correctly define “false fluency” and “medical language proficiency” and describe the accuracy limitations of language self-assessment (all p  < .001). They were also more confident in discussing language skills with peers. Although more than 75% of pre-session respondents reported having skills in at least one non-English language, only 16% reported a self-perceived ability to use non-English language skills with patients without an interpreter, and this percentage did not significantly change after the session (18%, p  = .808[unpaired]; p  = .651 [paired]).

After the final session, when looking at the full data set (unpaired responses), students were more likely to successfully define the role and value of medical interpreters following the session ( p  = .016), though when comparing paired data only, the difference was not significant ( p  = .147) as the number of students that had this knowledge prior to the session was high. This was the only item that differed in statistical significance when examining unpaired versus paired data. Importantly, students were more likely to recognize common pitfalls during interpreter-mediated encounters ( p  < .001). They also reported increased confidence in the logistical process of requesting a medical interpreter at their clinical sites, as well as the steps for collaborating with a professional medical interpreter via in person, through video, and via phone (all p  < .001).

Intent to apply concepts learned

After each session, students had the option to provide free-text responses to describe the ways they plan to apply the new knowledge into their practice. One-hundred and two students provided such responses for session one (87% of 117 post-session respondents), 62 for session two (93% of 67 post-session respondents), and 56 students for session three (71% of 79 post-session respondents). Table  4 summarizes the themes identified across responses.

Following session one, the most commonly recurring themes for skills application were focused on advocacy and systems-based practice with 29 students (28% of 102) sharing their plans to advocate for patients’ language rights and 29 (28% of 102) planning to increase their use of professional interpreters: “I plan to always make it a top priority for the patient to have interpretive resources and for us both to be able to fully understand one another.”

After session two, more students reported a plan for progressive non-English language self-assessment than before the session (91% of post-session respondents [61/67] v. 24% of pre-session respondents [38/158], p  < .001). Most post-session respondents indicated a plan to pursue multiple approaches to advance or assess their language skills (61%, 38/62), including taking a language course (44%, 27/62), taking a formal assessment exam (42%, 26/62), scheduling time for progressive self-assessment using the validated tools provided during the session (56%, 35/62), and discussing medical language skills, uncertainties, or questions with peers and supervisors (63%, 39/62). Students could use a free-text box elaborate on ways they plan to apply what they learned, and 54 did so. From those responses, we identified two major themes: most discussed how they will assess their second language skills (61%, 33/54) and others wrote about increasing their recognition of false fluency (39%, 21/54). One participant described this knowledge gain about false fluency as being aware that “ perceived language skills are different than actual skills.”

Following session three, we identified four themes across free-text responses about knowledge application. Twenty-four students (43% of 56) referenced their increased practical knowledge about hospital policies regarding language use and how they will use that information to better access professional interpreters in the future. One student shared, “I feel more confident in understanding the process of requesting and utilizing an interpreter, making it more likely that I will advocate for their use whenever needed.”

Student feedback and curricular updates

Most students consistently reported that the didactic components were useful in helping them achieve the session learning objectives (94% [110/117], 100% [64/64], and 85% [67/79] for each session, respectively). Respondents rated the interactive elements similarly positively with regards to usefulness toward achieving learning objectives (90% [105/117], 90% [58/64], and 86% [68/79] for each session, respectively). When asked about feedback for future improvements, almost half of session one respondents suggested to shorten the breakout session duration (44%, 44/99); this informed our planning for the second session. Some students proposed expanding the scope to include more facilitator examples of lived experiences working with NELP populations, as well as education on nonverbal communication strategies (25%, 25/99). After session two, students suggested topics that could be explored for additional content (29%, 14/48), such as information about available resources for learning a non-English language for medical use. Following the third session, the most common opportunity for improvement noted by students was to the desire for hands-on practice experience in working with medical interpreters (43%, 31/72).

Based on this feedback, following the successful implementation of the three-part language equity series, the medical school implemented several curricular updates. First, a fourth educational intervention was developed during a required course that takes place after the first few months of clinical clerkships. This fourth session was added to enable students to reflect about language-appropriate care in the context of patient safety following their initial clinical experiences. Second, the medical school improved the accessibility of language services information to students in the clinical years by creating an informational tag that could be attached to student identification badges. Third, the coalition has begun working with the institution’s simulation center on recruitment of a linguistically diverse pool of standardized patients. This is a first step in planning for standardized patient encounters where formative and summative evaluation of students’ language-appropriate communication can be more broadly incorporated.

We developed a longitudinal language equity curriculum to equip medical students with foundational language equity concepts and skills. One of the strengths of our curriculum is its implementation across urban, suburban, and rural sites. Over half of respondents indicated past exposure to care of NELP patients regardless of their own non-English language skills. This finding supports the need for language equity education for all students, not just those whose skills are proficient enough to provide language-concordant care nor those who are internally motivated to sign up for language electives or extracurricular experiences. Notably, by the third session, both multilingual and monolingual students across all three campuses described a plan to advocate on behalf of their patients with NELP if they observed poor communication practices. Similarly, many students planned on increasing their own use of professional interpreters.

By collecting language skills data, our intervention facilitated recognition of the rich linguistic diversity of the student body in participating schools; three quarters of respondents reported skills in at least one non-English language. Language skills are invisible characteristics that may intersect with other elements of diversity such as race, ethnicity, immigration story, and nationality, among others [ 40 , 41 ]. While many institutions offer some form of medical Spanish education [ 24 ], and Spanish is the language of greatest need in most areas of the US, [ 1 , 2 ] the linguistic diversity of our study’s participants highlights the need for preparing all learners (not just Spanish speakers) to appropriately use their language skills clinically. Students with skills in less common languages may have difficulty identifying educational resources for advancing or assessing their proficiency, making language equity education an important and foundational way to engage students in language-appropriate care regardless of languages spoken. Institutionalizing language equity education is an opportunity to highlight language – an understudied facet of student diversity – and engage learners and faculty in active discussions in which multilingual experiences are explicitly valued. Embedding language equity concepts, including skills for working with interpreters [ 42 ] as part of core, required clinical skills training sends an important message that skills for language-appropriate care are a key part of a comprehensive toolbox for all physicians.

Students learned the importance of progressively self-assessing their skills in languages besides English to accurately determine when they should partner with a qualified medical interpreter. For monolingual English-speaking students or multilingual students with intermediate or lower skill levels in a language, they should always partner with an interpreter via remote or in-person modes [ 10 ]. The choice of mode of interpretation depends on availability (which can vary by clinical site) and encounter complexity. Students received a list of their local campus’ clinical sites and information on how to access language services at each site. For multilingual students with advanced or higher skills in a language, proficiency testing is recommended to certify their skills; additionally, self-assessment should be continually applied since some clinical situations might pose unexpected or complex linguistic challenges with which even an advanced speaker may need additional language support [ 35 ].

This study sheds light on persistent structural barriers that disproportionately affect multilingual trainees and contribute to language-related health disparities for patients [ 43 ]. A concerning number of students reported having been asked to serve as ad hoc interpreters. These findings are consistent with prior literature about ad hoc interpreting by trainees, [ 14 ,  44 ] and support the need to revisit hospital policies and training for all healthcare staff (e.g., resident and attending physicians, nurses, etc.) who may be unaware of language-related legal requirements or best practices. Importantly, our longitudinal intervention resulted in many respondents developing a plan to apply strategies to improve and/or assess their language skills and to openly discuss language issues with peers, staff, or supervisors. Future research should explore long-term outcomes by evaluating the rates of working with interpreters, participating in language courses, or taking language assessment examinations for students exposed to the foundational pre-clinical language equity curriculum.

Our study had some limitations. While overall student engagement in the course was excellent, we observed attrition in the number of students who completed the questionnaires, with 25% of participants completing the final session’s post-questionnaire, potentially resulting in sampling bias. Also, a small subset of students (17%) participated in the course but did not complete the questionnaires. Since language data is not routinely collected, we have no way of knowing whether this subset of students differed from respondents with regards to their linguistic profile. All students were from the same institution despite being situated on three different campuses with unique patient populations. Student feedback prompted some course improvements, such as creating a role-play video to illustrate an example of an ethical dilemma that may arise during a medical encounter between a patient with NELP and a partially fluent medical student. In future courses, the impact of these changes should be evaluated. Secondly, our primary outcomes were self-perceived attitudes, confidence, and intent to apply concepts learned; it would be important to correlate these findings with students’ performance on experiential opportunities, such as standardized patient encounters, to assess the clinical skills taught in the course and receive formative feedback. Moreover, it would be valuable to track learners’ progress throughout the clinical years of medical school and residency through metrics such as interpreter utilization, periodic language proficiency assessments, clinical outcomes, and patient satisfaction.

Incorporating a pre-clinical undergraduate medical curriculum is a strategy for exposing all medical students to foundational education about improving health equity through language-appropriate care. Next steps should include exploring methods for evaluating these skills, including students’ communication with linguistically diverse populations and interprofessional collaboration with medical interpreters during clinical clerkships. Future research should also consider the potential indirect impact on language services utilization by other members of the healthcare team who might learn about language-appropriate care from medical students who took the course and explore whether language equity education improves belonging for students and clinicians from linguistically diverse backgrounds.

Data availability

Data is provided within the manuscript; additional data requests should be directed to the corresponding author.

Abbreviations

Doctoring and Clinical Skills

Limited English Proficiency

Non-English Language Preference

University of Illinois College of Medicine

American Community Survey. S1601 Language Spoken at Home: ACS 1-Year Estimates Subject Tables [Internet]. Washington, DC: United States Census Bureau; 2021 [cited 2023 July 11]. https://data.census.gov/table?q=Language+&y=2021&tid=ACSST1Y2021.S1601 .

American Community Survey. B16001 Language Spoken at Home by Ability to Speak English for the Population 5 Years and Over: ACS 1-Year Estimates Subject Tables [Internet]. Washington, DC: United States Census Bureau; 2021[cited 2023 July 11]. https://data.census.gov/table?q=B16001:+LANGUAGE+SPOKEN+AT+HOME+BY+ABILITY+TO+SPEAK+ENGLISH+FOR+THE+POPULATION+5+YEARS+AND+OVER&g=0100000US&tid=ACSDT1Y2021.B16001 .

Ortega P, Shin TM, Martínez GA. Rethinking the term Limited English proficiency to Improve Language-Appropriate Healthcare for all. J Immigr Minor Health. 2022;24(3):799–805. https://doi.org/10.1007/s10903-021-01257-w . Epub 2021 Jul 30. PMID: 34328602; PMCID: PMC8323079.

Article   Google Scholar  

Parker MM, Fernández A, Moffet HH, Grant RW, Torreblanca A, Karter AJ. Association of Patient-Physician Language Concordance and Glycemic Control for Limited-English Proficiency Latinos With Type 2 Diabetes. JAMA Intern Med. 2017;177(3):380–387. https://doi.org/10.1001/jamainternmed.2016.8648 . Erratum in: JAMA Intern Med. 2017;177(3):449. PMID: 28114680; PMCID: PMC5339062.

Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Serv Res. 2007;42(2):727–54. https://doi.org/10.1111/j.1475-6773.2006.00629.x . PMID: 17362215; PMCID: PMC1955368.

Diamond L, Izquierdo K, Canfield D, Matsoukas K, Gany F. A systematic review of the impact of patient-physician Non-english Language Concordance on Quality of Care and outcomes. J Gen Intern Med. 2019;34(8):1591–606. https://doi.org/10.1007/s11606-019-04847-5 . Epub 2019 May 30. PMID: 31147980; PMCID: PMC6667611.

Executive order No. 13166, Improving Access to Services for Persons with Limited English Proficiencies [Internet]. Washington, DC: Executive Office of the President; 2000 August 10 [cited 2023 July 11]. https://www.federalregister.gov/documents/2000/08/16/00-20938/improving-access-to-services-for-persons-with-limited-english-proficiency .

Jacobs B, Ryan AM, Henrichs KS, Weiss BD. Medical interpreters in Outpatient Practice. Ann Fam Med. 2018;16(1):70–6. https://doi.org/10.1370/afm.2154 . PMID: 29311179; PMCID: PMC5758324.

Showstack RE, Guzman K, Chesser AK, Woods NK. Improving latino Health Equity through Spanish Language Interpreter Advocacy in Kansas. Hisp Health Care Int. 2019;17(1):18–22. Epub 2018 Dec 21. PMID: 30572724.

Diamond LC, Schenker Y, Curry L, Bradley EH, Fernandez A. Getting by: underuse of interpreters by resident physicians. J Gen Intern Med. 2009;24(2):256 – 62. doi: 10.1007/s11606-008-0875-7. Epub 2008 Dec 17. PMID: 19089503; PMCID: PMC2628994.

Hudelson P. Improving patient-provider communication: insights from interpreters. Fam Pract. 2005;22(3):311-6. https://doi.org/10.1093/fampra/cmi015 . Epub 2005 Apr 1. PMID: 15805131.

Hsieh E, Kramer EM. Medical interpreters as tools: dangers and challenges in the utilitarian approach to interpreters’ roles and functions. Patient Educ Couns. 2012;89(1):158–62. https://doi.org/10.1016/j.pec.2012.07.001 . Epub 2012 Jul 31. PMID: 22857777; PMCID: PMC3462307.

Hsieh E. Not just getting by: factors influencing providers’ choice of interpreters. J Gen Intern Med. 2015;30(1):75–82. https://doi.org/10.1007/s11606-014-3066-8 . Epub 2014 Oct 23. PMID: 25338731; PMCID: PMC4284281.

Vela MB, Fritz C, Press VG, Girotti J. Medical students’ experiences and perspectives on interpreting for LEP patients at two US Medical Schools. J Racial Ethn Health Disparities. 2016;3(2):245–9. https://doi.org/10.1007/s40615-015-0134-7 . Epub 2015 May 28. PMID: 27271065.

Maul L, Regenstein M, Andres E, Wright R, Wynia MK. Using a risk assessment approach to determine which factors influence whether partially bilingual physicians rely on their non-English language skills or call an interpreter. Jt Comm J Qual Patient Saf. 2012;38(7):328 – 36. https://doi.org/10.1016/s1553-7250(12)38043-4 . PMID: 22852193.

Himmelstein J, Wright WS, Wiederman MW. U.S. medical school curricula on working with medical interpreters and/or patients with limited English proficiency. Adv Med Educ Pract. 2018;9:729–33. https://doi.org/10.2147/AMEP.S176028 . PMID: 30319306; PMCID: PMC6168005.

Pinto Taylor E, Mulenos A, Chatterjee A, Talwalkar JS. Partnering with interpreter services: standardized patient cases to Improve Communication with Limited English proficiency patients. MedEdPORTAL. 2019;15:10826. https://doi.org/10.15766/mep_2374-8265.10826 . PMID: 31161138; PMCID: PMC6543860.

Trial J, Elliott D, Lauzon V, Lie D, Chvira E. Interpretation at the OB/GYN Bedside - Cultural Competence in the Third-Year Clerkships. MedEdPORTAL [Internet]. 2010 June [cited 2023 July 11]; 6:1148. https://doi.org/10.15766/mep_2374-8265.1148 .

Jones J, Rice K, Cueto V, Mojica CDV, Stawitcke M, Salem S, Talley E, Blankenburg R. Increasing Health Care workers’ proficiency with using Professional Medical Interpretation: a workshop. MedEdPORTAL. 2020;16:11017. https://doi.org/10.15766/mep_2374-8265.11017 . PMID: 33204841; PMCID: PMC7666837.

Callahan E, Garcia E, Rehm J, Talk. Louder? Communicating With Your Spanish Speaking Patients. MedEdPORTAL [Internet]. 2011 June 20 [cited 2023 July 11];7:8427. https://doi.org/10.15766/mep_2374-8265.8427 .

TeamSTEPPS 3. 0 Video Training Tools [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; [2023 July, cited 2023 July 11] https://www.ahrq.gov/teamstepps/instructor/videos/index.html .

Lie D. Interpreter Cases for Cultural Competency Instruction. MedEdPORTAL [Internet]. 2006 March 27 [2023 July 11]; 2:205. https://doi.org/10.15766/mep_2374-8265.205 .

Ortega P, Diamond L, Alemán MA, Fatás-Cabeza J, Magaña D, Pazo V, Pérez N, Girotti JA, Ríos E. Medical Spanish Summit. Medical Spanish Standardization in U.S. Medical Schools: Consensus Statement From a Multidisciplinary Expert Panel. Acad Med. 2020;95(1):22–31. https://doi.org/10.1097/ACM.0000000000002917 . PMID: 31365394.

Ortega P, Francone NO, Santos MP, Girotti JA, Shin TM, Varjavand N, Park YS. Medical Spanish in US Medical schools: a National Survey to examine existing Programs. J Gen Intern Med. 2021;36(9):2724–30. https://doi.org/10.1007/s11606-021-06735-3 . Epub 2021 Mar 29. PMID: 33782890; PMCID: PMC8390604.

Total Enrollment by U.S. MD-Granting Medical School and Race/Ethnicity. (Alone), 2022–2023 [Internet]. Association of American Medical Colleges; 2022 November 4 [cited 2023 July 11]. https://www.aamc.org/media/6131/download .

Zackoff MW, Real FJ, Abramson EL, Li ST, Klein MD, Gusic ME. Enhancing Educational Scholarship through conceptual frameworks: a challenge and Roadmap for Medical Educators. Acad Pediatr. 2019;19(2):135–41. Epub 2018 Aug 20. PMID: 30138745.

Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55(1):68–78.

Yosso TJ. Whose culture has capital? A critical race theory discussion of community cultural wealth. Race Ethn Educ. 2005;8(1):69–91. https://doi.org/10.1080/1361332052000341006 .

Bourdieu P. Outline of a theory of practice. The new social theory reader. Routledge; 2020. pp. 80–6.

Ortega P, Shin TM. Language is not a barrier—it is an opportunity to improve health equity through education. Health Affairs Forefr. 2021. https://doi.org/10.1377/hblog20210726.579549 .

Ortega P. Spanish Language Concordance in U.S. Medical Care: A Multifaceted Challenge and Call to Action. Acad Med. 2018;93(9):1276–1280. https://doi.org/10.1097/ACM.0000000000002307 . PMID: 29877912.

US Department of Health and Human Serivces Office of Minoirty Health. National CLAS Standards - Think Cultural Health. [Internet]. Think Culture Health; [cited 2023 Jul 11]. https://thinkculturalhealth.hhs.gov/assets/pdfs/EnhancedNationalCLASStandards.pdf .

Rivera C, Pulsevoices. 2021 [cited 2023 Jul 11]. El Jugo Me Hizo Daño. https://pulsevoices.org/stories/el-jugo-me-hizo-dano/ .

Diamond L, Toro Bejarano M, Chung S, Ferguson W, Gonzalez J, Genoff Garzon M, Mujawar I, Gany F. Factors Associated with Accuracy of Self-Assessment compared with tested Non-english Language Proficiency among Primary Care Providers. Med Care. 2019;57(5):385–90. PMID: 30844905; PMCID: PMC6459717.

Regenstein M, Andres E, Wynia MK. Appropriate use of non-English-language skills in clinical care. JAMA. 2013;309(2):145-6. https://doi.org/10.1001/jama.2012.116984 . PMID: 23299604.

99Statesmen. Using a Medical Interpreter [Internet]. 2012 [cited 2023 Jul 11]. https://www.youtube.com/watch?v=D9s3sl5AoMg .

Diamond LC, Luft HS, Chung S, Jacobs EA. Does this doctor speak my language? Improving the characterization of physician non-english language skills. Health Serv Res. 2012;47(1 Pt 2):556–69. https://doi.org/10.1111/j.1475-6773.2011.01338.x . Epub 2011 Oct 27. PMID: 22091825; PMCID: PMC3393012.

Artino AR Jr, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach. 2014;36(6):463 – 74. doi: 10.3109/0142159X.2014.889814. Epub 2014 Mar 24. PMID: 24661014; PMCID: PMC4059192.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-51. https://doi.org/10.1097/ACM.0000000000000388 . PMID: 24979285.

Ortega P, Martínez G, Alemán MA, Zapién-Hidalgo A, Shin TM. Recognizing and Dismantling Raciolinguistic Hierarchies in Latinx Health. AMA J Ethics. 2022;24(4):E296-304. English, Spanish. https://doi.org/10.1001/amajethics.2022.296 . PMID: 35405056.

Diamond L, Grbic D, Genoff M, Gonzalez J, Sharaf R, Mikesell C, Gany F. Non-english-language proficiency of applicants to US residency programs. JAMA. 2014;312(22):2405–7. https://doi.org/10.1001/jama.2014.15444 . PMID: 25490332; PMCID: PMC4512180.

Jacobs EA, Diamond LC, Stevak L. The importance of teaching clinicians when and how to work with interpreters. Patient Educ Couns. 2010;78(2):149–53. https://doi.org/10.1016/j.pec.2009.12.001 . Epub 2009 Dec 29. PMID: 20036480.

Divi C, Koss RG, Schmaltz SP, Loeb JM. Language proficiency and adverse events in US hospitals: a pilot study. Int J Qual Health Care. 2007;19(2):60–7. https://doi.org/10.1093/intqhc/mzl069 . Epub 2007 Feb 2. PMID: 17277013.

Flores G, Laws MB, Mayo SJ, Zuckerman B, Abreu M, Medina L, Hardt EJ. Errors in medical interpretation and their potential clinical consequences in pediatric encounters. Pediatrics. 2003;111(1):6–14. https://doi.org/10.1542/peds.111.1.6 . PMID: 12509547.

Download references

Acknowledgements

The authors thank the Doctoring and Clinical Skills team at the University of Illinois College of Medicine, specifically, Shila Dauner, MPA, Angelica Bevelle, Euna Chi, MD, Julie Loza, MD, and María Isabel Angulo, MD for their invaluable support with implementation of the course and questionnaires. The authors also thank the many senior medical students, resident physicians, staff, and attending physicians who served as facilitators and provided feedback about the curriculum. Lastly, the authors are deeply grateful to Amy De La Torre, Alondra Díaz, and Christopher Llerena, MS, who, as medical students, contributed to course development and facilitator recruitment as core members of the language equity task force.

Not applicable.

Author information

Authors and affiliations.

Yale New Haven Hospital, New Haven, CT, USA

Maria Gabriela Valle Coto

McGaw Medical Center of Northwestern University, Chicago, IL, USA

Reniell X. Iñiguez

Department of Medical Education, University of Illinois College of Medicine, 808 S. Wood Street, MC 591, Chicago, IL, 606012, USA

Marina A. Lentskevich, Syeda Akila Ally, Julia F. Farfan, Yoon Soo Park, Ananya G. Gangopadhyaya & Pilar Ortega

Department of Diversity, Equity, and Inclusion, Accreditation Council for Graduate Medical Education, Chicago, IL, USA

Pilar Ortega

You can also search for this author in PubMed   Google Scholar

Contributions

MGVC co-led the project’s conceptualization, investigation, and formal analysis, and led the writing of the original draft, review, and editing. RXI co-led the project’s conceptualization and investigation, and participated in manuscript draft preparation, review, and editing. MAL, SAA, and JF participated in the project’s conceptualization, investigation, and manuscript draft preparation. JF also contributed to the formal analysis. YSP led the formal analysis and participated in critical manuscript review. AGG participated in project conceptualization, investigation, and critical manuscript review. PO was the primary lead in project conceptualization, investigation, and methodology and supervised the manuscript draft preparation, review, and editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Pilar Ortega .

Ethics declarations

Ethics approval and consent to participate.

The University of Illinois Institutional Review Board approved this study on August 15, 2017 (protocol #2017 − 0482). Informed consent was obtained from all participants.

Consent for publication

Competing interests.

Dr. Ortega receives textbook author royalties from Elsevier. The authors have no other conflicts of interest to report.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Valle Coto, M.G., Iñiguez, R.X., Lentskevich, M.A. et al. Teaching foundational language equity concepts in the pre-clinical curriculum. BMC Med Educ 24 , 485 (2024). https://doi.org/10.1186/s12909-024-05474-3

Download citation

Received : 07 December 2023

Accepted : 26 April 2024

Published : 02 May 2024

DOI : https://doi.org/10.1186/s12909-024-05474-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Language equity
• Undergraduate medical education
• Language-appropriate care
• Health equity
• Non-English language preference
• Limited English proficiency

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Literature, Language, and Elitism in Higher Education

Emily Martin

Emily has a PhD in English from the University of Southern Mississippi, MS, and she has an MFA in Creative Writing from GCSU in Milledgeville, GA, home of Flannery O’Connor. She spends her free time reading, watching horror movies and musicals, cuddling cats, Instagramming pictures of cats, and blogging/podcasting about books with the ladies over at #BookSquadGoals (www.booksquadgoals.com). She can be reached at [email protected].

View All posts by Emily Martin

Have you ever dreamed of being fluent in other languages? What if you could become fluent in another language in just about two weeks? This book was such a wish-fulfillment moment for me, but also, be careful what you wish for…

The Centre by Ayesha Manazir Siddiqi

Anisa Ellahi wants to be a translator of great literature, but that world doesn’t really open up to her until she meets Adam. When Anisa invites Adam to meet her parents, she’s immediately shocked by how quickly he’s able to pick up Urdu and speak the language fluently to her family. In fact, he speaks Urdu better than she does. How is this possible? What is his secret?

His secret, friends, is The Centre. Have you ever wished you could learn to speak a language fluently in less than a month? Read it, write it, listen to it, speak it, everything? Well, then set Duolingo aside because The Centre is your answer. Of course, there’s a catch. Or a few catches, really. First of all, you can’t just walk into the Centre. The program is extremely elite and invite-only. It also costs upwards of $20k to take a course. If you’re able to get in and you’re able to pay the exorbitant price, you’ll be forced to cut yourself off from the outside world. Every day, you’ll listen to language recordings and meditate without any contact with any other of The Centre’s learners. If you can do all of that, you will leave speaking the language of your choice fluently.

During Anisa’s time in the Centre, she chooses to learn German, and she spends her days listening to recordings of a man speaking German, telling his full life story. At first, she can’t understand a word he’s saying. Then, suddenly, when she enters the listening booth, she realizes she understands every word he’s saying, and she remembers everything he’s told her since the beginning of the recordings. After Anisa leaves the circle, she’s able to translate works of German literature, and her translation career becomes so successful that she decides to come back to the Centre to learn Russian.

Sound good? Well, Anisa soon learns that it might just be too good to be true. As Anisa becomes more enthralled by The Centre and what it’s capable of doing, she starts to look into how the program works. And the answers to her questions will surprise you. You won’t see this ending coming.

As someone who has always loved learning languages, I found this premise so intriguing, and the story definitely delivered. The Centre is a book that celebrates literature and the joy of learning languages. But it’s also a dark, unsettling novel that will give you the creeps and leave you with so many thoughts about classism, the elitism of higher learning, and more. Read this book, friends! You won’t regret it.

Want reading recs in your inbox? Sign up for our Read This Book newsletter !

You Might Also Like

Teaching and learning artificial intelligence: Insights from the literature

• Published: 02 May 2024

Cite this article

• Bahar Memarian   ORCID: orcid.org/0000-0003-0671-3127 1 &
• Tenzin Doleck 1  

102 Accesses

Explore all metrics

Artificial Intelligence (AI) has been around for nearly a century, yet in recent years the rapid advancement and public access to AI applications and algorithms have led to increased attention to the role of AI in higher education. An equally important but overlooked topic is the study of AI teaching and learning in higher education. We wish to examine the overview of the study, pedagogical outcomes, challenges, and limitations through a systematic review process amidst the COVID-19 pandemic and public access to ChatGPT. Twelve articles from 2020 to 2023 focused on AI pedagogy are explored in this systematic literature review. We find in-depth analysis and comparison of work post-COVID and AI teaching and learning era is needed to have a more focused lens on the current state of AI pedagogy. Findings reveal that the use of self-reported surveys in a pre-and post-design form is most prevalent in the reviewed studies. A diverse set of constructs are used to conceptualize AI literacy and their associated metrics and scales of measure are defined based on the work of specific authors rather than a universally accepted framework. There remains work and consensus on what learning objectives, levels of thinking skills, and associated activities lead to the advanced development of AI literacy. An overview of the studies, pedagogical outcomes, and challenges are provided. Further implications of the studies are also shared. The contribution of this work is to open discussions on the overlooked topic of AI teaching and learning in higher education.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Introducing Artificial Intelligence Literacy in Schools: A Review of Competence Areas, Pedagogical Approaches, Contexts and Formats

AI’s Role and Application in Education: Systematic Review

Artificial intelligence (AI) learning tools in K-12 education: A scoping review

Data availability.

Data sharing does not apply to this article as no datasets were generated or analyzed during the current study.

Ali, M., & Abdel-Haq, M. K. (2021). Bibliographical analysis of artificial intelligence learning in Higher Education: is the role of the human educator and educated a thing of the past? Fostering Communication and Learning With Underutilized Technologies in Higher Education , 36–52.

Anderson, J., Rainie, L., & Luchsinger, A. (2018). Artificial intelligence and the future of humans. Pew Research Center , 10 (12).

Bates, T., Cobo, C., Mariño, O., & Wheeler, S. (2020). No can artificial intelligence transform higher education? International Journal of Educational Technology in Higher Education , 17 (1), 1–12.

Article   Google Scholar  

Biggs, J., & Tang, C. (2011). Train-the-trainers: Implementing outcomes-based teaching and learning in Malaysian higher education. Malaysian Journal of Learning and Instruction , 8 , 1–19.

Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research , 3 , 993–1022.

Google Scholar  

Chang, Y. S., Wang, Y. Y., & Ku, Y. T. (2023). Influence of online STEAM hands-on learning on AI learning, creativity, and creative emotions. Interact Learn Environ PG . https://doi.org/10.1080/10494820.2023.2205898 .

Chen, X., Xie, H., Zou, D., & Hwang, G. J. (2020). Application and theory gaps during the rise of artificial intelligence in education. Computers and Education: Artificial Intelligence , 1 , 100002.

Clark, D. (2023). PedAIgogy – New Era of Knowledge and Learning Where AI changes Everything .

Covidence (2023). Covidence systematic review software . www.covidence.org.

Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: The state of the field. International Journal of Educational Technology in Higher Education , 20 (1), 1–22.

Cumming, G., & McDougall, A. (2000). Mainstreaming AIED into education. International Journal of Artificial Intelligence in Education , 11 (2), 197–207.

Dignum, V. (2020). AI is multidisciplinary. AI Matters , 5 (4), 18–21.

Fuller, C. M., Simmering, M. J., Atinc, G., Atinc, Y., & Babin, B. J. (2016). Common methods variance detection in business research. Journal of Business Research , 69 (8), 3192–3198. https://doi.org/10.1016/j.jbusres.2015.12.008 .

Gao, Z., Wanyama, T., & Singh, I. (2020). Project and practice centered learning: A systematic methodology and strategy to cultivate future full stack artificial intelligence engineers. Int. J. Eng. Educ , 36 (6 PG-1760–1772), 1760–1772. https://www.scopus.com/inward/record.uri?eid=2s2.085096038454&partnerID=40&md5=b4778257adf34da504fdf1f97ebb7fe9NS .

Halic, O., Lee, D., Paulus, T., & Spence, M. (2010). To blog or not to blog: Student perceptions of blog effectiveness for learning in a college-level course. The Internet and Higher Education , 13 (4), 206–213.

Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development , 48 (3), 23–48.

Hsu, T. C., & Chen, M. S. (2022). The engagement of students when learning to use a personal audio classifier to control robot cars in a computational thinking board game. Res Pract Technol Enhanc Learn , 17 (1 PG-). https://doi.org/10.1186/s41039-022-00202-1 .

Hwang, G. J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence , 1 , 100001.

Jaramillo, F., Locander, W. B., Spector, P. E., & Harris, E. G. (2007). Getting the job done: The moderating role of initiative on the relationship between intrinsic motivation and adaptive selling. Journal of Personal Selling and Sales Management , 27 (1), 59–74.

Javed, R. T., Nasir, O., Borit, M., Vanhée, L., Zea, E., Gupta, S., Vinuesa, R., & Qadir, J. (2022). Get out of the BAG! Silos in AI Ethics Education: Unsupervised topic modeling analysis of global AI Curricula. Journal of Artificial Intelligence Research , 73 , 933–965. https://doi.org/10.1613/jair.1.13550 .

Jiang, L. (2021). Virtual reality Action interactive teaching Artificial Intelligence Education System. Complexity , 2021 (PG-). https://doi.org/10.1155/2021/5553211 .

Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence , 1 (9), 389–399.

Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development , 10 (3), 2–10.

Article   MathSciNet   Google Scholar  

Kim, S., Jang, Y., Choi, S., Kim, W., Jung, H., Kim, S., & Kim, H. (2021). Analyzing teacher competency with TPACK for K-12 AI education. KI-Künstliche Intelligenz , 35 (2), 139–151.

Koh, J. H., Chai, L., Wong, C. S., & Hong, B., H.-Y (2015). Design thinking for education: Conceptions and applications in teaching and learning . Springer.

Kong, S. C., Cheung, W. M. Y., & Zhang, G. (2023). Evaluating an Artificial Intelligence Literacy Programme for developing University students’ conceptual understanding, literacy, empowerment and ethical awareness. Educational Technology and Society , 26 (1), 16–30. https://doi.org/10.30191/ETS.202301_26(1).0002 .

Korkmaz, Ö., & Xuemei, B. A. I. (2019). Adapting computational thinking scale (CTS) for Chinese high school students and their thinking scale skills level. Participatory Educational Research , 6 (1), 10–26.

Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice , 41 (4), 212–218. https://doi.org/10.1207/s15430421tip4104_2 .

Lin, C. H., Wu, L. Y., Wang, W. C., Wu, P. L., & Cheng, S. Y. (2020). Development and validation of an instrument for AI-Literacy. 3rd Eurasian Conference on Educational Innovation (ECEI 2020) .

Lin, X. F., Chen, L., Chan, K. K., Peng, S. Q., Chen, X. F., Xie, S. Q., Liu, J. C., & Hu, Q. T. (2022). Teachers’ perceptions of teaching sustainable Artificial Intelligence: A Design Frame Perspective. SUSTAINABILITY , 14 (13 PG-). https://doi.org/10.3390/su14137811 .

Martín-Núñez, J. L., Ar, A. Y., Fernández, R. P., Abbas, A., & Radovanović, D. (2023). Does intrinsic motivation mediate perceived artificial intelligence (AI) learning and computational thinking of students during the COVID-19 pandemic? Computers & Education , 4(PG-) . https://doi.org/10.1016/j.caeai.2023.100128 .

Meek, T., Barham, H., Beltaif, N., Kaadoor, A., & Akhter, T. (2016). Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review. 2016 Portland International Conference on Management of Engineering and Technology (PICMET) , 682–693.

Miriyev, A., & Kovač, M. (2020). Skills for physical artificial intelligence. Nature Machine Intelligence , 2 (11), 658–660.

Ng, D. T. K., Lee, M., Tan, R. J. Y., Hu, X., Downie, J. S., & Chu, S. K. W. (2022). A review of AI teaching and learning from 2000 to 2020. EDUCATION AND INFORMATION TECHNOLOGIES . https://doi.org/10.1007/s10639-022-11491-w .

Pinkwart, N. (2016). Another 25 years of AIED? Challenges and opportunities for intelligent educational technologies of the future. International Journal of Artificial Intelligence in Education , 26 , 771–783.

Popenici, S. A., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning , 12 (1), 1–13.

Rizvi, S., Waite, J., & Sentance, S. (2023). Artificial Intelligence teaching and learning in K-12 from 2019 to 2022: A systematic literature review. Computers and Education: Artificial Intelligence , 100145.

Shih, P. K., Lin, C. H., Wu, L. Y., & Yu, C. C. (2021). Learning ethics in AI—teaching non-engineering undergraduates through situated learning. Sustainability , 13 (7), 3718.

Southworth, J., Migliaccio, K., Glover, J., Glover, J. N., Reed, D., McCarty, C., Brendemuhl, J., & Thomas, A. (2023). Developing a model for AI across the curriculum: Transforming the higher education landscape via innovation in AI literacy. Computers & Education , 4 , PG–. https://doi.org/10.1016/j.caeai.2023.100127 .

Su, J., Guo, K., Chen, X., & Chu, S. K. W. (2023). Teaching artificial intelligence in K–12 classrooms: A scoping review. Interactive Learning Environments , 1–20.

Tedre, M., Toivonen, T., Kahila, J., Vartiainen, H., Valtonen, T., Jormanainen, I., & Pears, A. (2021). Teaching machine learning in K–12 classroom: Pedagogical and technological trajectories for artificial intelligence education. Ieee Access : Practical Innovations, Open Solutions , 9 , 110558–110572.

UNESCO (2023). 10 Innovative Learning Strategies For Modern Pedagogy . TeachThought Staff. https://policytoolbox.iiep.unesco.org/library/BHABKKH6 .

Wong, M. K., Wu, J., Ong, Z. Y., Goh, J. L., Cheong, C. W. S., Tay, K. T., Tan, L. H. S., & Krishna, L. K. R (2019). Teaching ethics in medical schools: A systematic review from 2000 to 2018. Journal of Medical Education , 18 , 226–250.

Xu, B. (2021). Artificial Intelligence Teaching System and Data Processing Method based on Big Data. Complexity , 2021(PG-) . https://doi.org/10.1155/2021/9919401 .

Yağcı, M. (2019). A valid and reliable tool for examining computational thinking skills. Education and Information Technologies , 24 (1), 929–951.

Yi, Y. (2021). Establishing the concept of AI literacy. European Journal of Bioethics , 12 (2), 353–368.

Yue, M., Jong, M. S. Y., & Dai, Y. (2022). Pedagogical design of K-12 artificial intelligence education: A systematic review. Sustainability , 14 (23), 15620.

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education , 16 (1), 1–27.

Zeide, E. (2019). Artificial intelligence in higher education: Applications, promise and perils, and ethical questions. Educause Review , 54 (3).

Zhang, K., & Aslan, A. B. (2021). AI technologies for education: Recent research & future directions. Computers and Education: Artificial Intelligence , 2 , 100025.

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

Faculty of Education, Simon Fraser University, Vancouver, BC, Canada

Bahar Memarian & Tenzin Doleck

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Bahar Memarian .

Ethics declarations

Conflict of interest, additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Memarian, B., Doleck, T. Teaching and learning artificial intelligence: Insights from the literature. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12679-y

Download citation

Received : 03 August 2023

Accepted : 10 April 2024

Published : 02 May 2024

DOI : https://doi.org/10.1007/s10639-024-12679-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Higher education
• Artificial intelligence (AI)
• Find a journal
• Publish with us
• Track your research

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

Navigation Menu

Weizhi Yang Mahasarakham University Thailand

Faculty of Education 

Chowwalit Chookhampaeng Mahasarakham University Thailand

Jiraporn Chano Mahasarakham University Thailand

Faculty of Education

AUTHOR GUIDELINES

• Other Journals

IJoST QUALITY

IJoST is now at Q1 Scimago Journal Ranking

• For Readers
• For Authors
• For Librarians

Journal Visitor

• Announcements
• REVIEW PROCESS

Spatial Visualization Ability Assessment for Analyzing Differences and Exploring Influencing Factors: Literature Review with Bibliometrics and Experiment

Abdussemiu, A. (2022). Problems of teaching practical biology in senior secondary schools. ASEAN Journal of Science and Engineering Education, 2(3), 199-206.

Abosede, P.J., Onasanya, S.A., and Ngozi, O.C. (2024). Students self-assessment of demonstration-based flipped classroom on senior secondary school students’ performance in physics. Indonesian Journal of Teaching in Science, 4(1), 27-40.

Ajenikoko, G.A., and Ogunwuyi, O.J. (2022). A mathematical model for estimating the end-of-life of power transformers: From literature review to development analysis. ASEAN Journal of Science and Engineering, 2(3), 285-294.

Akinoso, S.O. (2023). Motivation and ICT in secondary school mathematics using unified theory of acceptance and use of technology model. Indonesian Journal of Educational Research and Technology, 3(1), 79-90.

Al Husaeni, D. F., Al Husaeni, D. N., Ragadhita, R., Bilad, M. R., Al-Obaidi, A. S. M., Abduh, A., and Nandiyanto, A. B. D. (2022). How language and technology can improve student learning quality in engineering? Definition, factors for enhancing students comprehension, and computational bibliometric analysis. International Journal of Language Education, 6(4), 445-476.

Al Husaeni, D. F., and Al Husaeni, D. N. (2022). Computational bibliometric analysis of research on science and Islam with VOSviewer: Scopus database in 2012 to 2022. ASEAN Journal of Religion, Education, and Society, 1(1), 39-48.

Al Husaeni, D. N. (2022). Bibliometric Analysis of briquette research trends during the covid-19 pandemic. ASEAN Journal for Science and Engineering in Materials, 1(2), 99-106.

Al Husaeni, D.F., and Nandiyanto, A.B.D. (2022). Bibliometric using VOSviewer with publish or perish (using google scholar data): From step-by-step processing for users to the practical examples in the analysis of digital learning articles in pre and post covid-19 pandemic. ASEAN Journal of Science and Engineering, 2(1), 19-46.

Al Husaeni, D.N. (2022). Development analysis research on physics education by mapping keywords using the VOSviewer application. ASEAN Journal of Physical Education and Sport Science, 1(1), 9-18.

Al Husaeni, D.N., and Al Husaeni, D.F. (2022). How to calculate bibliometric using VOSviewer with Publish or Perish (using Scopus data): Science education keywords. Indonesian Journal of Educational Research and Technology, 2(3), 247-274.

Ala, N.A., Onojah, A.O., Ishyaku, A.M., and Adamu, S.B. (2022). Development of an animation package in biology for teaching vertebrate, anatomy, and physiology. ASEAN Journal for Science Education, 1(2), 117-130.

Alhassan, N., Alhassan, A., and Chioma, A.F. (2024). Examining the role of biology teachers' beliefs, motivations, and self-reported practices in constructing curves for biology class. Indonesian Journal of Teaching in Science, 4(1), 11-26.

Amevor, G., Bayaga, A., and Bossé, M. J. (2021). Assessing the impact of dynamic software environments (MATLAB) on rural-based pre-service teachers’ spatial-visualisation skills. Contemporary Educational Technology, 13(4), ep327.

Andika, R., and Putra, Z.A. (2022). Teaching programming to chemical engineering students. ASEAN Journal of Science and Engineering Education, 2(1), 51-60.

Awofala, A.O.A. (2023). Examining sources of mathematics self- efficacy beliefs of senior secondary school students. ASEAN Journal of Science and Engineering Education, 3(3), 229-244.

Awofala, A.O.A., Akinoso, S.O., Adeniyi, C.O., Jega, S.H., Fatade, A.O., and Arigbabu, A.A. (2024). Primary teachers’ mathematics anxiety and mathematics teaching anxiety as predictors of students’ performance in mathematics. ASEAN Journal of Science and Engineering Education, 4(1), 9-24.

Awofala, A.O.A., and Agbolade, F.O.O. (2024). Effect of peer-tutoring strategy on senior secondary school students’ achievement in mathematics. ASEAN Journal for Science Education, 3(1), 1-12.

Awofala, A.O.A., and Olaniyi, A.O. (2023). Assessing teachers’ formative evaluation strategy as related to senior secondary school students’ achievement in mathematics. ASEAN Journal for Science Education, 2(2), 77-86.

Awofala, A.O.A., Olabiyi, O.S., Ojo, O.T., Oladipo, A.J., Fatade, A.O., and Udeani, U.N. (2023). Personal and contextual factors as correlates of entrepreneurial intentions among pre-service science, technology, and mathematics teachers. ASEAN Journal of Science and Engineering Education, 3(3), 265-278.

Azizah, E.V., Nandiyanto, A.B.D., Kurniawan, T., and Bilad, M.R. (2022). The effectiveness of using a virtual laboratory in distance learning on the measurement materials of the natural sciences of physics for junior high school students. ASEAN Journal of Science and Engineering Education, 2(3), 207-214.

Azizah, N.N., Maryanti, R., and Nandiyanto, A.B.D. (2021). How to search and manage references with a specific referencing style using google scholar: From step-by-step processing for users to the practical examples in the referencing education. Indonesian Journal of Multidiciplinary Research, 1(2), 267-294.

Babalola, E.O. (2022). Design and development of 3-dimensional model of human circulatory system to teach a concept of biology in senior secondary schools. Indonesian Journal of Teaching in Science, 2(1), 17-28.

Babalola, E.O., Ayoola, D.A., and Omolafe, E.V. (2023). Analysis of experts' opinion on the human excretory system model for teaching biology in Nigeria. ASEAN Journal of Science and Engineering Education, 3(1), 19-26.

Barke, H.D. (2023). Broensted acids and bases: History, misconception, and application today. ASEAN Journal for Science Education, 2(1), 23-32.

Barke, H.D., and Buechter, J. (2023). Laboratory jargon and misconceptions in chemistry an empirical study. ASEAN Journal of Science and Engineering Education, 3(1), 65-70.

Barke, H.D., and Harsch, N. (2022). Chemical bonding: From simple bond forces to electron pairs. Indonesian Journal of Educational Research and Technology, 2(3), 275-284.

Bartlett, K. A., and Camba, J. D. (2023). Gender differences in spatial ability: A critical review. Educational Psychology Review, 35(1), 8.

Bayaga, A., and Kok, P. J. (2019). Enhancing graphic communication and design student teachers’ spatial visualisation skills through 3D solid computer modelling. African Journal of Research in Mathematics, Science and Technology Education, 23(1), 52-63.

Bilad, M. R. (2022). Bibliometric analysis for understanding the correlation between chemistry and special needs education using vosviewer indexed by google. ASEAN Journal of Community and Special Needs Education, 1(2), 61-68.

Bilad, M.R., and Prayogi, S. (2021). Portfolio workbook as an effective method for student-centered learning of chemical engineering principles. ASEAN Journal of Science and Engineering Education, 1(1), 31-36.

Branoff, T. J, Dobelis, and M. (2012). The relationship between spatial visualization ability and students' ability to model 3D objects from engineering assembly drawings. Journal Articles Reports Research, 76(3), 37-43.

Camenda, D.Y., Gaba, C.A., Lacord, N., Natango, D., Pabl, A., and Abusam, H. (2021). How difficult is 1+1? A phenomenological study of high school students struggling in mathematics. ASEAN Journal of Science and Engineering Education, 1(2), 111-116.

Contero, M., Naya, F., Company, P., Saorin, J. L., and Conesa, J. (2005). Improving visualization skills in engineering education. IEEE Computer Graphics and Applications, 25(5), 24–31-24–31.

Cooke-Simpson, A., and Voyer, D. (2007). Confidence and gender differences on the Mental rotations test. Learning and Individual Differences, 17(2), 181–186.

Dallyono, R., Sukyadi, D., and Hakim, L. (2020). A mathematical model of the cognitive semantics of the English preposition on. Indonesian Journal of Science and Technology, 5(1), 133-153.

Deno, J. A. (1995). The relationship of previous experiences to spatial visualization ability. Engineering Design Graphics Journal, 59(3), 5–17.

Dermawan, R., Muktiarni, M., and Mupita, J. (2022). Efforts to increase the interest of junior high school students in mathematics lessons using the tik tok learning tool. ASEAN Journal for Science Education, 1(2), 81-88.

Devon, R., Engel, R., Foster, R., Sathianathan, D., and Turner, G. (1994). The effect of solid modeling software on 3-D visualization skills. Engineering Design Graphics Journal, 58(2), 4-11.

Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., and Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285–296.

Effiong, J.B., and Aya, C.F. (2022). Rural-urban migration among women farmers: Science education, survey, and implication for food crop production in Cross River State, Nigeria. Indonesian Journal of Teaching in Science, 2(1), 75-80.

Ernst, J. V., Williams, T. O., Clark, A. C., and Kelly, D. P. (2017). Factors of spatial visualization: An analysis of the PSVT: R. The Engineering Design Graphics Journal, 81(1), 1-10.

Fauziah, S. P., Suherman, I., Sya, M. F., Roestamy, M., Abduh, A., and Nandiyanto, A. B. D. (2021). Strategies in language education to improve science student understanding during practicum in laboratory: Review and computational bibliometric analysis. International Journal of Language Education, 5(4), 409-425.

Febriandi, R., Herman, T., Turmudi, T., Farokhah, L., Abidin, Z., Alman, A., and Supriyadi, E. (2023). Research on algebraic thinking in elementary school is reduced: a bibliometric analysis. Journal of Engineering Science and Technology, 18, 97-104.

Feng, J., Spence, I., and Pratt, J. (2007). Playing an action video game reduces gender differences in spatial cognition. Psychological Science, 18(10), 850–855.

Francis, T.T., and Baba, S.J. (2023). Effect of concept mapping teaching approach on students’ academic performance in chemistry in senior secondary schools. Indonesian Journal of Educational Research and Technology, 3(1), 69-78.

French, J. W. (1951). The description of aptitude and achievement tests in terms of rotated factors. The University of Chicago Press, 1951, 278.

Froiland, J. M., and Davison, M. L. (2020). Social perception: relationships with general intelligence, working memory, processing speed, visual-spatial ability, and verbal comprehension. Educational Psychology, 40(6), 750–766.

Glorifica, I. (2021). Media analysis of biology teaching book grade xii: A study based on science literation category. Indonesian Journal of Educational Research and Technology, 1(2), 17-22.

Gobert, J. D., and Buckley, B. C. (2000). Introduction to model-based teaching and learning in science education. International Journal of Science Education, 22(9), 891–894.

Guilford, J. P. (1956). The structure of intellect. Psychological Bulletin, 53(4), 267–293.

Hao, X., Huang, Y., Li, X., Song, Y., Kong, X., Wang, X., and Liu, J. (2016). Structural and functional neural correlates of spatial navigation: a combined voxel‐based morphometry and functional connectivity study. Brain and Behavior, 6(12), e00572.

Harith, K. (2023). Exploring historical seismic events through secondary data analysis: Implications for understanding submarine earthquakes in marine geophysics for educational purposes. Indonesian Journal of Multidiciplinary Research, 3(2), 349-370.

Hashim, S., Masek, A., Mahthir, B.N.S.M., Rashid, A.H.A., and Nincarean, D. (2021). Association of interest, attitude and learning habit in mathematics learning towards enhancing students’ achievement. Indonesian Journal of Science and Technology, 6(1), 113-122.

Hausmann, M., Slabbekoorn, D., Van Goozen, S. H. M., Cohen-Kettenis, P. T., and Güntürkün, O. (2000). Sex hormones affect spatial abilities during the menstrual cycle. Behavioral Neuroscience, 114(6), 1245–1250.

Heyden, K. M. V., Huizinga, M., and Jolles, J. (2017). Effects of a classroom intervention with spatial play materials on children’s object and viewer transformation abilities. Developmental Psychology, 53(2), 290–305.

Hofifah, S.N., and Sumiati, S. (2023). The effectiveness of the practicum video guide on distance learning in improving biology learning outcomes in enzyme content. Indonesian Journal of Teaching in Science, 3(2), 201-212.

Husnah, A.U., Hidayat, M.A., and Jannah, M. (2021). The journey of a math: As a mathematics learning innovation. Indonesian Journal of Multidiciplinary Research, 1(1), 129-136.

Hyde, J. S., and Linn, M. C. (1989). The psychology of gender: Advances through meta-analysis. American Journal of Psychology, 102(2), 285-285.

Ibrahim, A.O. (2023). Impact of blended learning method on secondary school physics students’ achievement and retention in Lokoja, Nigeria. ASEAN Journal for Science Education, 2(2), 57-66.

Jansen-Osmann, P., and Heil, M. (2007). Suitable stimuli to obtain (no) gender differences in the speed of cognitive processes involved in mental rotation. Brain and Cognition, 64(3), 217–227.

Jose, M.T.N.S. (2022). Factors that affect the performance of selected high school students from the third district of Albay in International Mathematics Competitions. ASEAN Journal for Science Education, 1(1), 9-16.

Karakus, F., and Aydin, B. (2017). The effects of computer algebra system on undergraduate students' spatial visualization skills in a calculus course. Malaysian Online Journal of Educational Technology, 5(3), 54-69.

Koch, and Douglas. (2011). The effects of solid modeling and visualization on technical problem solving. Journal Articles Reports Evaluative, 22(2), 3-21.

Kolakoti, A., Setiyo, M., Al Husaeni, D. N., and Nandiyanto, A. B. D. (2023). Enhancing heat transfer performance of automotive car radiator using camphor nanoparticles: experimental study with bibliometric analysis. Teknomekanik, 6(2), 47-66.

Kösa, T., and Karakuş, F. (2018). The effects of computer-aided design software on engineering students’ spatial visualisation skills. European Journal of Engineering Education, 43(2), 296-308.

Kritchevsky, M. (2022). The elementary spatial functions of the brain. Spatial Cognition, 2022, 111-140.

Kurul, R., Ögün, M. N., Neriman Narin, A., Avci, Ş., and Yazgan, B. (2020). An alternative method for anatomy training: Immersive virtual reality. Anatomical Sciences Education, 13(5), 648-656.

Lagcao, Y.G.D., Dechavez, J.P.A.D., Goleng, D.J.G., Lagca, Y.G.D., Tangkli, K.Y.M., and Vicera, W.J.C. (2023). Math readiness and its Effect on the online academic performance of science, technology, engineering, and mathematics students. ASEAN Journal for Science Education, 2(1), 33-38.

Lasisi, A.K., Hassan, A.A., and Abdulkareem, H.B. (2024). Impact of single parenting on academic performance of junior secondary school students in mathematics. ASEAN Journal for Science Education, 3(2), 129-138.

Lathifah, N.N., and Maryanti, R. (2021). Basic arithmetic learning through math online games for elementary school students during the pandemic. Indonesian Journal of Multidiciplinary Research, 1(2), 379-384.

Lauer, J. E., Yhang, E., and Lourenco, S. F. (2019). The development of gender differences in spatial reasoning: A meta-analytic review. Psychological Bulletin, 145(6), 537-565.

Lee, Y., Capraro, R. M., and Bicer, A. (2019). Gender difference on spatial visualization by college students’ major types as STEM and non-STEM: A meta-analysis. International Journal of Mathematical Education in Science and Technology, 50(8), 1241–1255.

Lehmann, J., Quaiser-Pohl, C., and Jansen, P. (2014). Correlation of motor skill, mental rotation, and working memory in 3- to 6-year-old children. European Journal of Developmental Psychology, 11(5), 560–573.

Lennon-Maslin, M., Quaiser-Pohl, C. M., Ruthsatz, V., and Saunders, M. (2023). Under My skin: Reducing bias in STEM through new approaches to assessment of spatial abilities considering the role of emotional regulation. Social Sciences, 12(6), 356.

Lestari, D.A., Suwarma, I.R., and Suhendi, E. (2024). Feasibility analysis of the development of STEM-based physics e-book with self-regulated learning on global warming topics. Indonesian Journal of Teaching in Science, 4(1), 1-10.

Li, Y., Kong, F., Ji, M., Luo, Y., Lan, J., and You, X. (2019). Shared and distinct neural bases of large-and small-scale spatial ability: A coordinate-based activation likelihood estimation meta-analysis. Frontiers in Neuroscience, 12, 1021.

Linn, M. C., and Petersen, A. C. (1985). Emergence and characterization of sex differences in spatial ability: A meta-analysis. Child Development, 56(6), 1479-1479.

Lohman, D. F. (1979). Spatial ability: A review and reanalysis of the correlational literature. Stanford, CA: School of Education, Stanford University, 8, 226.

Lohman, D. F. (2005). The role of nonverbal ability tests in identifying academically gifted students: An aptitude perspective. Gifted Child Quarterly, 49(2), 111–138.

Maeda, Y., and Yoon, S. Y. (2012). A meta-analysis on gender differences in mental rotation ability measured by the purdue spatial visualization tests: Visualization of rotations (PSVT:R). Educational Psychology Review, 25(1), 69–94.

Maeda, Y., and Yoon, S. Y. (2016). Are gender differences in spatial ability real or an artifact? Evaluation of measurement invariance on the revised PSVT: R. Journal of Psychoeducational Assessment, 34(4), 397-403.

Marasabessy, R. (2021). Study of mathematical reasoning ability for mathematics learning in schools: A literature review. Indonesian Journal of Teaching in Science, 1(2), 79-90.

Maryanti, R. (2021). Assessment of mathematical abilities of students with intellectual disabilities during the covid-19 pandemic. Indonesian Journal of Community and Special Needs Education, 1(2), 47-52.

Maryanti, R., and Asjjari, M. (2022). Family as an educational environment that can provide science education for children with special needs. Indonesian Journal of Community and Special Needs Education, 2(2), 149-154.

Maryanti, R., and Nandiyanto, A.B.D. (2021). Curriculum development in science education in vocational school. ASEAN Journal of Science and Engineering Education, 2(1), 151-156.

Maryanti, R., Nandiyanto, A.B.D., Hufad, A., and Sunardi, S. (2021). Science education for students with special needs in Indonesia: From definition, systematic review, education system, to curriculum. Indonesian Journal of Community and Special Needs Education, 1(1), 1-8.

Maryanti, R., Rahayu, N. I., Muktiarni, M., Al Husaeni, D. F., Hufad, A., Sunardi, S., and Nandiyanto, A. B. D. (2022). Sustainable development goals (SDGs) in science education: Definition, literature review, and bibliometric analysis. Journal of Engineering Science and Technology, 17, 161-181.

Maryati, W.E., Retnowati, E., and Thoe, N, K. (2022). Learning mathematics formulas by listening and reading worked examples. Indonesian Journal of Teaching in Science, 2(1), 61-74.

McGee, M. G. (1979). Human spatial abilities: Psychometric studies and environmental, genetic, hormonal, and neurological influences. Psychological Bulletin, 86(5), 889–918.

Mirzabek, R. (2023). The science education and history of ulugh beg: Astronomer and mathematician from Samarkand, Uzbekistan. ASEAN Journal of Science and Engineering Education, 3(1), 59-64.

Mitrayana, M., and Nurlaelah, E. (2023). Computational thinking in mathematics learning: Systematic literature review. Indonesian Journal of Teaching in Science, 3(2), 133-142.

Moè, A. (2009). Are males always better than females in mental rotation? Exploring a gender belief explanation. Learning and Individual Differences, 19(1), 21–27.

Mohler, J. L. (2008). A review of Spatial Ability Research. Engineering Design Graphics Journal, 72(2), 19–30.

Muffato, V., Meneghetti, C., Di Ruocco, V., and De Beni, R. (2017). When young and older adults learn a map: The influence of individual visuo-spatial factors. Learning and Individual Differences, 53, 114–121.

Naglieri, J. A., and Ford, D. Y. (2003). Addressing underrepresentation of gifted minority children using the naglieri nonverbal ability test (NNAT). Gifted Child Quarterly, 47(2), 155–160.

Nancy, E. (2011). Long-term impact of improving visualization abilities of minority engineering and technology students: Preliminary results. Engineering Design Graphics Journal, 75, 2-8.

Nandiyanto, A. B. D., Al Husaeni, D. F., and Ragadhita, R. (2023c). Bibliometric data analysis of research on resin-based brake-pads from 2012 to 2021 using vosviewer mapping analysis computations. ASEAN Journal for Science and Engineering in Materials, 2(1), 35-44.

Nandiyanto, A. B. D., Al Husaeni, D. N., and Al Husaeni, D. F. (2021). A bibliometric analysis of chemical engineering research using vosviewer and its correlation with covid-19 pandemic condition. Journal of Engineering Science and Technology, 16(6), 4414-4422.

Nandiyanto, A. B. D., and Al Husaeni, D. F. (2021a). A bibliometric analysis of materials research in Indonesian journal using VOSviewer. Journal of Engineering Research, 2021, 2307-1877.

Nandiyanto, A. B. D., Ragadhita, R., and Aziz, M. (2023b). Involving particle technology in computational fluid dynamics research: A bibliometric analysis. CFD Letters, 15(11), 92-109.

Nandiyanto, A. B. D., Ragadhita, R., Setiyo, M., Al Obaidi, A. S. M., and Hidayat, A. (2023a). Particulate matter emission from combustion and non-combustion automotive engine process: Review and computational bibliometric analysis on its source, sizes, and health and lung impact. Automotive Experiences, 6(3), 599-623.

Newcombe, N. S. (2010). Picture this: Increasing math and science learning by improving spatial thinking. American Educator, 34(2), 29.

Nordin, N. A. H. M. (2022). A bibliometric analysis of computational mapping on publishing teaching science engineering using VOSviewer application and correlation. Indonesian Journal of Teaching in Science, 2(2), 127-138.

Nordin, N. A. H. M. (2022a). Correlation between process engineering and special needs from bibliometric analysis perspectives. ASEAN Journal of Community and Special Needs Education, 1(1), 9-16.

Nursaniah, S.S.J. (2023). Analysis of science education curriculum for students with special needs in special schools: the curriculum of 2013. Indonesian Journal of Community and Special Needs Education, 3(2), 113-130.

Obafemi, K.E. (2024). Enhancing pupils’ academic performance in mathematics using brainstorming instructional strategy. ASEAN Journal of Science and Engineering Education, 4(2), 99-106.

Obafemi, K.E., Fajonyomi, A., and Ola-Alani, E.K. (2023). Effect of reversed jigsaw instructional strategy on pupils academic achievement in mathematics. ASEAN Journal of Science and Engineering Education, 3(3), 297-304.

Obafemi, K.E., Saadu, U.T., Adesokan, A., Yahaya, O., Sulaimon, J.T., Obafemi, T.O., and Yakubu, F.M. (2023). Self-efficacy as a correlate of pupils’ academic achievement in mathematics. Indonesian Journal of Teaching in Science, 3(2), 113-120.

Obafemi, K.E., Saadu, U.T., Yahaya, O., Obafemi, T.O., and Yakubu, F.M. (2024). Exploration of the effect of scaffolding instructional strategy on pupils’ academic performance in mathematics. ASEAN Journal for Science Education, 2(2), 121-128.

Ogunjimi, M.O., and Gbadeyanka, T.A. (2023). Effect of guided inquiry and explicit-instructional strategies on lower basic students’ academic performance in mathematics. Indonesian Journal of Teaching in Science, 3(1), 23-32.

Olumorin, C.O., Babalola, E.O., Aladesusi, G.A., Issa, A.I., and Omolafe, E.V. (2021). Experts’ validation of the developed 3-dimensional automated model of the human heart to teach a biology concept in Ilorin, Nigeria. Indonesian Journal of Multidiciplinary Research, 1(2), 299-308.

Olumorin, C.O., Babalola, E.O., and Ayoola, D.A. (2022). Design and development of human excretory system model to teach a biology concept in Ilorin, Nigeria. Indonesian Journal of Teaching in Science, 2(2), 107-116.

Omolafe.E.V. (2021). Primary educators’ experts’ validation of the developed mathematics mobile application to enhance the teaching of mathematics in Nigeria primary schools. ASEAN Journal of Science and Engineering Education, 2(1), 157-166.

Padmore, E.A., and Ali, C.A. (2024). Exploring effective differentiated instruction in the teaching and learning of mathematics. ASEAN Journal for Science Education, 3(1), 41-54.

Patkin, D., and Dayan, E. (2013). The intelligence of observation: improving high school students’ spatial ability by means of intervention unit. International Journal of Mathematical Education in Science and Technology, 44(2), 179-195.

Peters, M. (2005). Sex differences and the factor of time in solving vandenberg and kuse mental rotation problems. Brain and Cognition, 57(2), 176–184.

Putri, S.R., Hofifah, S.N., Girsang, G.C.S., and Nandiyanto, A.B.D. (2022). How to identify misconception using certainty of response index (CRI): A study case of mathematical chemistry subject by experimental demonstration of adsorption. Indonesian Journal of Multidiciplinary Research, 2(1), 143-158.

Quaiser-Pohl, C., Geiser, C., and Lehmann, W. (2006). The relationship between computer-game preference, gender, and mental-rotation ability. Personality and Individual Differences, 40(3), 609–619.

Radiamoda, A.A. (2024). Difficulties encountered by the students in learning mathematics. Indonesian Journal of Educational Research and Technology, 4(1), 63-70.

Ragadhita, R., and Nandiyanto, A. B. D. (2022). Computational bibliometric analysis on publication of techno-economic education. Indonesian Journal of Multidiciplinary Research, 2(1), 213-222.

Ramdhani, M.R., Kholik, A., Fauziah, S.P., Roestamy, M., Suherman, I., Nandiyanto, A.B.D. (2023). A comprehensive study on biochar production, bibliometric analysis, and collaborative teaching practicum for sustainable development goals (SDGs) in islamic schools. Jurnal Pendidikan Islam, 9(2), 123-144.

Ramdhani, M.R., Kholik, A., Fauziah, S.P., Roestamy, M., Suherman, I., Nandiyanto, A.B.D. (2023). A comprehensive study on biochar production, bibliometric analysis, and collaborative teaching practicum for sustainable development goals (SDGs) in Islamic schools. Jurnal Pendidikan Islam, 9(2), 123-144.

Riandi, R., Permanasari, A., and Novia, N. (2022). Implementation of biotechnology in education towards green chemistry teaching: A bibliometrics study and research trends. Moroccan Journal of Chemistry, 10(3), 10-13.

Ruzmetov, A., and Ibragimov, A. (2023). Past, current and future trends of salicylic acid and its derivatives: A bibliometric review of papers from the Scopus database published from 2000 to 2021. ASEAN Journal for Science and Engineering in Materials, 2(1), 53-68.

Sambudi, N.S., and Ramli, R.M. (2021). Integrated project as innovative assessment to enhance learning experience in thermodynamics class. ASEAN Journal of Science and Engineering Education, 2(1), 167-176.

Samsuri, S., and Wan Osman, W.N.A.B. (2024). Teaching chemical engineering thermodynamics using substituted blended learning techniques. ASEAN Journal of Science and Engineering Education, 4(2), 143-162.

Scali, R. M., Brownlow, S., and Hicks, J. L. (2000). Gender differences in spatial task performance as a function of speed or accuracy orientation. Sex Roles, 43, 359-376.

Serra, E.J.P., Senope, N.J.R., and Lariosa, C.M. (2021). Potholes in the implementation of printed module in mathematics and feedbacks of learners in Lambayong national high school during covid-19 pandemic. ASEAN Journal of Science and Engineering Education, 2(1), 177-182.

Sharobeam, M. H. (2016). The variation in spatial visualization abilities of college male and female students in STEM fields versus non-STEM fields. Journal of College Science Teaching, 46(2), 93.

Shea, D. L., Lubinski, D., and Benbow, C. P. (2001). Importance of assessing spatial ability in intellectually talented young adolescents: A 20-year longitudinal study. Journal of Educational Psychology, 93(3), 604–614.

Shidiq, A. S., Permanasari, A., and Hernani, S. H. (2021). The use of simple spectrophotometer in STEM education: A bibliometric analysis. Moroccan Journal of Chemistry, 9(2), 9-2.

Snowden, A. W., Warren, C. M., Goodridge, W., and Fang, N. (2021). Neural efficiency and spatial task difficulty: A road forward to mapping students’ neural engagement in spatial cognition. Engineering Design Graphics Journal, 85, 1-9.

Sombria, K.J.F., Celestial, D.L., Jalagat, C.G.M., and Valdez, A.G. (2023). Online learning through google classroom: Effects on students critical thinking skills in chemistry. ASEAN Journal of Science and Engineering Education, 3(2), 193-210.

Sorby, S. A., and Baartmans, B. J. (1996). A course for the development of 3-D spatial visualization skills. Engineering Design Graphics Journal, 60(1), 13-20.

Study, N. E. (2002). The effectiveness of using the successive perception test i to measure visual-haptic tendencies in engineering students. Engineering Design Graphics Journal, 66(2), 17-26.

Suherman, A., Komaro, M., and Ana, A. (2023). e-book multimedia animation implementation on concept mastery and problem-solving skills of crystal structure subjects in engineering materials course. Indonesian Journal of Science and Technology, 8(2), 259-280.

Suherman, I., Fauziah, S. P., Roestamy, M., Bilad, M. R., Abduh, A., and Nandiyanto, A. B. D. (2023). How to improve student understanding in learning science by regulating strategy in language education? Definition, factors for enhancing students comprehension, and computational bibliometric review analysis. International Journal of Language Education, 7(3), 527-562.

Supriyadi, E., Turmudi, T., Dahlan, J. A., Juandi, D., Istikomah, E., Febriandi, R., and Sholikhakh, R. A. (2023). Global trend of ethnoscience research: A bibliometric analysis using scopus database. Journal of Engineering Science and Technology, 18, 1-8.

Susilowati, N.I., Liliawati, W., and Rusdiana, D. (2023). Science process skills test instruments in the new indonesian curriculum (merdeka): Physics subject in renewable energy topic. Indonesian Journal of Teaching in Science, 3(2), 121-132.

Swafiyah, B., Muhammad, B.A., and Yamusa, A.Z. (2023). Effect of conceptual change instructional strategy on chemistry students' performance in acids and bases concepts. ASEAN Journal for Science Education, 2(1), 47-54.

Tartre, L. A. (1990). Spatial skills, gender, and mathematics. Mathematics and Gender, 1990, 27-59.

Thomas, H. R., and Kail, R. V. (1991). Sex differences in speed of mental rotation and the X-linked genetic hypothesis. Intelligence, 15(1), 17–32.

Thornton, T., and Lammi, M. (2021). An exploration of augmented reality in an introductory engineering graphics course. Journal of Technology Education, 32(2), 38-55.

Tian, Y., Yang, T., and Dong, W. (2022). Does culture shape our spatial ability? an investigation based on eye tracking. ISPRS International Journal of Geo-Information, 11(11), 578.

Tipmontiane, K., and Williams, P.J. (2022). The integration of the engineering design process in biology-related STEM activity: A review of Thai secondary education. ASEAN Journal of Science and Engineering Education, 2(1), 1-10.

Trumble, J., and Dailey, D. (2019). Change in spatial visualization mental rotation abilities of intermediate elementary students. Journal of Computers in Mathematics and Science Teaching, 38(1), 77-90.

Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., and Newcombe, N. S. (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin, 139(2), 352–402.

Veurink, N., and Sorby, S. A. (2012). Comparison of spatial skills of students entering different engineering majors. The Engineering Design Graphics Journal, 76(3), 49-54.

Veurink, N., Hamlin, A., Kampe, Sorby, S. A., Blasko, D. G., Holliday-Darr, K., and Knott, T. (2009). Enhancing visualization skills-improving options and success (EnViSIONS) of engineering and technology students. Engineering Design Graphics Journal, 73(2), 1–17.

Voyer, D., Voyer, S. D., and Bryden, M. P. (1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117(2), 250–270.

Wagiran, W., Mujiyono, M., Setiadi, B.R., Wibowo, Y.E., Surahmanto, F., Agata, D.A., and Areeprasert, C. (2023). Temperature distribution in bio stove using saw dust: An integrated project-based learning. Indonesian Journal of Science and Technology, 8(1), 127-140.

Wahab, N.A., Mahmood, N.H.N., and Minghat, A.D. (2023). Science education research methodology: A case study investigating the correlation between construction, safety, accident, and the effectiveness construction industry development board (CIDB) green card training program. ASEAN Journal for Science Education, 2(1), 7-16.

Wai, J., Lubinski, D., and Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817–835.

Wang, L., and Carr, M. (2014). Working memory and strategy use contribute to gender differences in spatial ability. Educational Psychologist, 49(4), 261–282.

Webb, R. M., Lubinski, D., and Benbow, C. P. (2007). Spatial ability: A neglected dimension in talent searches for intellectually precocious youth. Journal of Educational Psychology, 99(2), 397–420.

Wijaya, H., Maryanti, R., Wulandary, V., and Irawan, A.R. (2022). Numerical minimum competence assessment for increasing students’ interest in mathematics. ASEAN Journal of Science and Engineering Education, 2(3), 183-192.

Williams Jr, T. O., Ernst, J. V., Kelly, D. P., and Clark, A. C. (2019). Confirmatory factor analyses of the PSVT: R with data from engineering design graphics students. The Engineering Design Graphics Journal, 83, 27-34.

Wirzal, M. D. H., and Putra, Z. A. (2022). What is the correlation between chemical engineering and special needs education from the perspective of bibliometric analysis using vosviewer indexed by google scholar. Indonesian Journal of Community and Special Needs Education, 2(2), 103-110.

Wirzal, M.D.H., and Halim, N.S.A. (2022). Short play approach for analytical chemistry class. ASEAN Journal of Science and Engineering Education, 2(2), 163-168.

Yan, Z., Yu, X., Cheng, D., Ma, C., and Zhou, X. (2023). Spatial ability and science achievement more closely associated in eighth-graders than in fourth-graders: A large-scale study. International Journal of Science Education, 45, 1-22.

Yue, J. (2008). Spatial visualization by realistic 3D views. The Engineering Design Graphics Journal, 72(1), 28-38.

• There are currently no refbacks.