chapter 5 research about modular learning

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Behav Sci (Basel)

Assessing Cognitive Factors of Modular Distance Learning of K-12 Students Amidst the COVID-19 Pandemic towards Academic Achievements and Satisfaction

Yung-tsan jou.

1 Department of Industrial and Systems Engineering, Chung Yuan Christian University, Taoyuan 320, Taiwan; wt.ude.ucyc@uojty (Y.-T.J.); moc.oohay@enimrahcrolfas (C.S.S.)

Klint Allen Mariñas

2 School of Industrial Engineering and Engineering Management, Mapua University, Manila 1002, Philippines

3 Department of Industrial Engineering, Occidental Mindoro State College, San Jose 5100, Philippines

Charmine Sheena Saflor

Associated data.

Not applicable.

The COVID-19 pandemic brought extraordinary challenges to K-12 students in using modular distance learning. According to Transactional Distance Theory (TDT), which is defined as understanding the effects of distance learning in the cognitive domain, the current study constructs a theoretical framework to measure student satisfaction and Bloom’s Taxonomy Theory (BTT) to measure students’ academic achievements. This study aims to evaluate and identify the possible cognitive capacity influencing K-12 students’ academic achievements and satisfaction with modular distance learning during this new phenomenon. A survey questionnaire was completed through an online form by 252 K-12 students from the different institutions of Occidental Mindoro. Using Structural Equation Modeling (SEM), the researcher analyses the relationship between the dependent and independent variables. The model used in this research illustrates cognitive factors associated with adopting modular distance learning based on students’ academic achievements and satisfaction. The study revealed that students’ background, experience, behavior, and instructor interaction positively affected their satisfaction. While the effects of the students’ performance, understanding, and perceived effectiveness were wholly aligned with their academic achievements. The findings of the model with solid support of the integrative association between TDT and BTT theories could guide decision-makers in institutions to implement, evaluate, and utilize modular distance learning in their education systems.

1. Introduction

The 2019 coronavirus is the latest infectious disease to develop rapidly worldwide [ 1 ], affecting economic stability, global health, and education. Most countries have suspended thee-to-face classes in order to curb the spread of the virus and reduce infections [ 2 ]. One of the sectors impacted has been education, resulting in the suspension of face-to-face classes to avoid spreading the virus. The Department of Education (DepEd) has introduced modular distance learning for K-12 students to ensure continuity of learning during the COVID-19 pandemic. According to Malipot (2020), modular learning is one of the most popular sorts of distance learning alternatives to traditional face-to-face learning [ 3 ]. As per DepEd’s Learner Enrolment and Survey Forms, 7.2 million enrollees preferred “modular” remote learning, TV and radio-based practice, and other modalities, while two million enrollees preferred online learning. It is a method of learning that is currently being used based on the preferred distance learning mode of the students and parents through the survey conducted by the Department of Education (DepEd); this learning method is mainly done through the use of printed and digital modules [ 4 ]. It also concerns first-year students in rural areas; the place net is no longer available for online learning. Supporting the findings of Ambayon (2020), modular teaching within the teach-learn method is more practical than traditional educational methods because students learn at their own pace during this modular approach. This educational platform allows K-12 students to interact in self-paced textual matter or digital copy modules. With these COVID-19 outbreaks, some issues concerned students’ academic, and the factors associated with students’ psychological status during the COVID-19 lockdown [ 5 ].

Additionally, this new learning platform, modular distance learning, seems to have impacted students’ ability to discover and challenged their learning skills. Scholars have also paid close attention to learner satisfaction and academic achievement when it involves distance learning studies and have used a spread of theoretical frameworks to assess learner satisfaction and educational outcomes [ 6 , 7 ]. Because this study aimed to boost academic achievement and satisfaction in K-12 students, the researcher thoroughly applied transactional distance theory (TDT) to understand the consequences of distance in relationships in education. The TDT was utilized since it has the capability to establish the psychological and communication factors between the learners and the instructors in distance education that could eventually help researchers in identifying the variables that might affect students’ academic achievement and satisfaction [ 8 ]. In this view, distance learning is primarily determined by the number of dialogues between student and teacher and the degree of structuring of the course design. It contributes to the core objective of the degree to boost students’ modular learning experiences in terms of satisfaction. On the other hand, Bloom’s Taxonomy Theory (BTT) was applied to investigate the students’ academic achievements through modular distance learning [ 6 ]. Bloom’s theory was employed in addition to TDT during this study to enhance students’ modular educational experiences. Moreover, TDT was utilized to check students’ modular learning experiences in conjuction with enhacing students’ achievements.

This study aimed to detect the impact of modular distance learning on K-12 students during the COVID-19 pandemic and assess the cognitive factors affecting academic achievement and student satisfaction. Despite the challenging status of the COVID-19 outbreak, the researcher anticipated a relevant result of modular distance learning and pedagogical changes in students, including the cognitive factors identified during this paper as latent variables as possible predictors for the utilization of K-12 student academic achievements and satisfaction.

1.1. Theoretical Research Framework

This study used TDT to assess student satisfaction and Bloom’s theory to quantify academic achievement. It aimed to assess the impact of modular distance learning on academic achievement and student satisfaction among K-12 students. The Transactional Distance Theory (TDT) was selected for this study since it refers to student-instructor distance learning. TDT Moore (1993) states that distance education is “the universe of teacher-learner connections when learners and teachers are separated by place and time.” Moore’s (1990) concept of ”Transactional Distance” adopts the distance that occurs in all linkages in education, according to TDT Moore (1993). Transactional distance theory is theoretically critical because it states that the most important distance is transactional in distance education, rather than geographical or temporal [ 9 , 10 ]. According to Garrison (2000), transactional distance theory is essential in directing the complicated experience of a cognitive process such as distance teaching and learning. TDT evaluates the role of each of these factors (student perception, discourse, and class organization), which can help with student satisfaction research [ 11 ]. Bloom’s Taxonomy is a theoretical framework for learning created by Benjamin Bloom that distinguishes three learning domains: Cognitive domain skills center on knowledge, comprehension, and critical thinking on a particular subject. Bloom recognized three components of educational activities: cognitive knowledge (or mental abilities), affective attitude (or emotions), and psychomotor skills (or physical skills), all of which can be used to assess K-12 students’ academic achievement. According to Jung (2001), “Transactional distance theory provides a significant conceptual framework for defining and comprehending distance education in general and a source of research hypotheses in particular,” shown in Figure 1 [ 12 ].

Theoretical Research Framework.

1.2. Hypothesis Developments and Literature Review

This section will discuss the study hypothesis and relate each hypothesis to its related studies from the literature.

There is a significant relationship between students’ background and students’ behavior .

The teacher’s guidance is essential for students’ preparedness and readiness to adapt to a new educational environment. Most students opt for the Department of Education’s “modular” distance learning options [ 3 ]. Analyzing students’ study time is critical for behavioral engagement because it establishes if academic performance is the product of student choice or historical factors [ 13 ].

There is a significant relationship between students’ background and students’ experience .

Modules provide goals, experiences, and educational activities that assist students in gaining self-sufficiency at their speed. It also boosts brain activity, encourages motivation, consolidates self-satisfaction, and enables students to remember what they have learned [ 14 ]. Despite its success, many families face difficulties due to their parents’ lack of skills and time [ 15 ].

There is a significant relationship between students’ behavior and students’ instructor interaction .

Students’ capacity to answer problems reflects their overall information awareness [ 5 ]. Learning outcomes can either cause or result in students and instructors behavior. Students’ reading issues are due to the success of online courses [ 16 ].

There is a significant relationship between students’ experience and students’ instructor interaction .

The words “student experience” relate to classroom participation. They establish a connection between students and their school, teachers, classmates, curriculum, and teaching methods [ 17 ]. The three types of student engagement are behavioral, emotional, and cognitive. Behavioral engagement refers to a student’s enthusiasm for academic and extracurricular activities. On the other hand, emotional participation is linked to how children react to their peers, teachers, and school. Motivational engagement refers to a learner’s desire to learn new abilities [ 18 ].

There is a significant relationship between students’ behavior and students’ understanding .

Individualized learning connections, outstanding training, and learning culture are all priorities at the Institute [ 19 , 20 ]. The modular technique of online learning offers additional flexibility. The use of modules allows students to investigate alternatives to the professor’s session [ 21 ].

There is a significant relationship between students’ experience and students’ performance .

Student conduct is also vital in academic accomplishment since it may affect a student’s capacity to study as well as the learning environment for other students. Students are self-assured because they understand what is expected [ 22 ]. They are more aware of their actions and take greater responsibility for their learning.

There is a significant relationship between students’ instructor interaction and students’ understanding .

Modular learning benefits students by enabling them to absorb and study material independently and on different courses. Students are more likely to give favorable reviews to courses and instructors if they believe their professors communicated effectively and facilitated or supported their learning [ 23 ].

There is a significant relationship between students’ instructor interaction and students’ performance.

Students are more engaged and active in their studies when they feel in command and protected in the classroom. Teachers play an essential role in influencing student academic motivation, school commitment, and disengagement. In studies on K-12 education, teacher-student relationships have been identified [ 24 ]. Positive teacher-student connections improve both teacher attitudes and academic performance.

There is a significant relationship between students’ understanding and students’ satisfaction .

Instructors must create well-structured courses, regularly present in their classes, and encourage student participation. When learning objectives are completed, students better understand the course’s success and learning expectations. “Constructing meaning from verbal, written, and graphic signals by interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining” is how understanding is characterized [ 25 ].

There is a significant relationship between students’ performance and student’s academic achievement .

Academic emotions are linked to students’ performance, academic success, personality, and classroom background [ 26 ]. Understanding the elements that may influence student performance has long been a goal for educational institutions, students, and teachers.

There is a significant relationship between students’ understanding and students’ academic achievement .

Modular education views each student as an individual with distinct abilities and interests. To provide an excellent education, a teacher must adapt and individualize the educational curriculum for each student. Individual learning may aid in developing a variety of exceptional and self-reliant attributes [ 27 ]. Academic achievement is the current level of learning in the Philippines [ 28 ].

There is a significant relationship between students’ performance and students’ satisfaction .

Academic success is defined as a student’s intellectual development, including formative and summative assessment data, coursework, teacher observations, student interaction, and time on a task [ 29 ]. Students were happier with course technology, the promptness with which content was shared with the teacher, and their overall wellbeing [ 30 ].

There is a significant relationship between students’ academic achievement and students’ perceived effectiveness .

Student satisfaction is a short-term mindset based on assessing students’ educational experiences [ 29 ]. The link between student satisfaction and academic achievement is crucial in today’s higher education: we discovered that student satisfaction with course technical components was linked to a higher relative performance level [ 31 ].

There is a significant relationship between students’ satisfaction and students’ perceived effectiveness.

There is a strong link between student satisfaction and their overall perception of learning. A satisfied student is a direct effect of a positive learning experience. Perceived learning results had a favorable impact on student satisfaction in the classroom [ 32 ].

2. Materials and Methods

2.1. participants.

The principal area under study was San Jose, Occidental Mindoro, although other locations were also accepted. The survey took place between February and March 2022, with the target population of K-12 students in Junior and Senior High Schools from grades 7 to 12, aged 12 to 20, who are now implementing the Modular Approach in their studies during the COVID-19 pandemic. A 45-item questionnaire was created and circulated online to collect the information. A total of 300 online surveys was sent out and 252 online forms were received, a total of 84% response rate [ 33 ]. According to several experts, the sample size for Structural Equation Modeling (SEM) should be between 200 and 500 [ 34 ].

2.2. Questionnaire

The theoretical framework developed a self-administered test. The researcher created the questionnaire to examine and discover the probable cognitive capacity influencing K-12 students’ academic achievement in different parts of Occidental Mindoro during this pandemic as well as their satisfaction with modular distance learning. The questionnaire was designed through Google drive as people’s interactions are limited due to the effect of the COVID-19 pandemic. The questionnaire’s link was sent via email, Facebook, and other popular social media platforms.

The respondents had to complete two sections of the questionnaire. The first is their demographic information, including their age, gender, and grade level. The second is about their perceptions of modular learning. The questionnaire is divided into 12 variables: (1) Student’s Background, (2) Student’s Experience, (3) Student’s Behavior, (4) Student’s Instructor Interaction, (5) Student’s Performance, (6) Student’s Understanding, (7) Student’s Satisfaction, (8) Student’s Academic Achievement, and (9) Student’s Perceived Effectiveness. A 5-point Likert scale was used to assess all latent components contained in the SEM shown in Table 1 .

The construct and measurement items.

2.3. Structural Equation Modeling (SEM)

All the variables have been adapted from a variety of research in the literature. The observable factors were scored on a Likert scale of 1–5, with one indicating “strongly disagree” and five indicating “strongly agree”, and the data were analyzed using AMOS software. Theoretical model data were confirmed by Structural Equation Modeling (SEM). SEM is more suitable for testing the hypothesis than other methods [ 53 ]. There are many fit indices in the literature, of which the most commonly used are: CMIN/DF, Comparative Fit Index (CFI), AGFI, GFI, and Root Mean Square Error (RMSEA). Table 2 demonstrates the Good Fit Values and Acceptable Fit Values of the fit indices, respectively. AGFI and GFI are based on residuals; when sample size increases, the value of the AGFI also increase. It takes a value between 0 and 1. The fit is good if the value is more significant than 0.80. GFI is a model index that spans from 0 to 1, with values above 0.80 deemed acceptable. An RMSEA of 0.08 or less suggests a good fit [ 54 ], and a value of 0.05 to 0.08 indicates an adequate fit [ 55 ].

Acceptable Fit Values.

3. Results and Discussion

Figure 2 demonstrates the initial SEM for the cognitive factors of Modular Distance learning towards academic achievements and satisfaction of K-12 students during the COVID-19 pandemic. According to the figure below, three hypotheses were not significant: Students’ Behavior to Students’ Instructor Interaction (Hypothesis 3), Students’ Understanding of Students’ Academic Achievement (Hypothesis 11), and Students’ Performance to Students’ Satisfaction (Hypothesis 12). Therefore, a revised SEM was derived by removing this hypothesis in Figure 3 . We modified some indices to enhance the model fit based on previous studies using the SEM approach [ 47 ]. Figure 3 demonstrates the final SEM for evaluating cognitive factors affecting academic achievements and satisfaction and the perceived effectiveness of K-12 students’ response to Modular Learning during COVID-19, shown in Table 3 . Moreover, Table 4 demonstrates the descriptive statistical results of each indicator.

Initial SEM with indicators for evaluating the cognitive factors of modular distance learning towards academic achievements and satisfaction of K-12 students during COVID-19 pandemic.

Revised SEM with indicators for evaluating the cognitive factors of modular distance learning towards academic achievements and satisfaction of K-12 students during the COVID-19 pandemic.

Summary of the Results.

Descriptive statistic results.

The current study was improved by Moore’s transactional distance theory (TDT) and Bloom’s taxonomy theory (BTT) to evaluate cognitive factors affecting academic achievements and satisfaction and the perceived effectiveness of K-12 students’ response toward modular learning during COVID-19. SEM was utilized to analyze the correlation between Student Background (SB), Student Experience (SE), Student Behavior (SBE), Student Instructor Interaction (SI), Student Performance (SP), Student Understanding (SAU), Student Satisfaction (SS), Student’s Academic achievement (SAA), and Student’s Perceived effectiveness (SPE). A total of 252 data samples were acquired through an online questionnaire.

According to the findings of the SEM, the students’ background in modular learning had a favorable and significant direct effect on SE (β: 0.848, p = 0.009). K-12 students should have a background and knowledge in modular systems to better experience this new education platform. Putting the students through such an experience would support them in overcoming all difficulties that arise due to the limitations of the modular platforms. Furthermore, SEM revealed that SE had a significant adverse impact on SI (β: 0.843, p = 0.009). The study shows that students who had previous experience with modular education had more positive perceptions of modular platforms. Additionally, students’ experience with modular distance learning offers various benefits to them and their instructors to enhance students’ learning experiences, particularly for isolated learners.

Regarding the Students’ Interaction—Instructor, it positively impacts SAU (β: 0.873, p = 0.007). Communication helps students experience positive emotions such as comfort, satisfaction, and excitement, which aim to enhance their understanding and help them attain their educational goals [ 62 ]. The results revealed that SP substantially impacted SI (β: 0.765; p = 0.005). A student becomes more academically motivated and engaged by creating and maintaining strong teacher-student connections, which leads to successful academic performance.

Regarding the Students’ Understanding Response, the results revealed that SAA (β: 0.307; p = 0.052) and SS (β: 0.699; p = 0.008) had a substantial impact on SAU. Modular teaching is concerned with each student as an individual and with their specific capability and interest to assist each K-12 student in learning and provide quality education by allowing individuality to each learner. According to the Department of Education, academic achievement is the new level for student learning [ 63 ]. Meanwhile, SAA was significantly affected by the Students’ Performance Response (β: 0.754; p = 0.014). It implies that a positive performance can give positive results in student’s academic achievement, and that a negative performance can also give negative results [ 64 ]. Pekrun et al. (2010) discovered that students’ academic emotions are linked to their performance, academic achievement, personality, and classroom circumstances [ 26 ].

Results showed that students’ academic achievement significantly positively affects SPE (β: 0.237; p = 0.024). Prior knowledge has had an indirect effect on academic accomplishment. It influences the amount and type of current learning system where students must obtain a high degree of mastery [ 65 ]. According to the student’s opinion, modular distance learning is an alternative solution for providing adequate education for all learners and at all levels in the current scenario under the new education policy [ 66 ]. However, the SEM revealed that SS significantly affected SPE (β: 0.868; p = 0.009). Students’ perceptions of learning and satisfaction, when combined, can provide a better knowledge of learning achievement [ 44 ]. Students’ perceptions of learning outcomes are an excellent predictor of student satisfaction.

Since p -values and the indicators in Students’ Behavior are below 0.5, therefore two paths connecting SBE to students’ interaction—instructor (0.155) and students’ understanding (0.212) are not significant; thus, the latent variable Students’ Behavior has no effect on the latent variable Students’ Satisfaction and academic achievement as well as perceived effectiveness on modular distance learning of K12 students. This result is supported by Samsen-Bronsveld et al. (2022), who revealed that the environment has no direct influence on the student’s satisfaction, behavior engagement, and motivation to study [ 67 ]. On the other hand, the results also showed no significant relationship between Students’ Performance and Students’ Satisfaction (0.602) because the correlation p -values are greater than 0.5. Interestingly, this result opposed the other related studies. According to Bossman & Agyei (2022), satisfaction significantly affects performance or learning outcomes [ 68 ]. In addition, it was discovered that the main drivers of the students’ performance are the students’ satisfaction [ 64 , 69 ].

The result of the study implies that the students’ satisfaction serves as the mediator between the students’ performance and the student-instructor interaction in modular distance learning for K-12 students [ 70 ].

Table 5 The reliabilities of the scales used, i.e., Cronbach’s alphas, ranged from 0.568 to 0.745, which were in line with those found in other studies [ 71 ]. As presented in Table 6 , the IFI, TLI, and CFI values were greater than the suggested cutoff of 0.80, indicating that the specified model’s hypothesized construct accurately represented the observed data. In addition, the GFI and AGFI values were 0.828 and 0.801, respectively, indicating that the model was also good. The RMSEA value was 0.074, lower than the recommended value. Finally, the direct, indirect, and total effects are presented in Table 7 .

Construct Validity Model.

Direct effect, indirect effect, and total effect.

Table 6 shows that the five parameters, namely the Incremental Fit Index, Tucker Lewis Index, the Comparative Fit Index, Goodness of Fit Index, and Adjusted Goodness Fit Index, are all acceptable with parameter estimates greater than 0.8, whereas mean square error is excellent with parameter estimates less than 0.08.

4. Conclusions

The education system has been affected by the 2019 coronavirus disease; face-to-face classes are suspended to control and reduce the spread of the virus and infections [ 2 ]. The suspension of face-to-face classes results in the application of modular distance learning for K-12 students according to continuity of learning during the COVID-19 pandemic. With the outbreak of COVID-19, some issues concerning students’ academic Performance and factors associated with students’ psychological status are starting to emerge, which impacted the students’ ability to learn. This study aimed to perceive the impact of Modular Distance learning on the K-12 students amid the COVID-19 pandemic and assess cognitive factors affecting students’ academic achievement and satisfaction.

This study applied Transactional Distance Theory (TDT) and Bloom Taxonomy Theory (BTT) to evaluate cognitive factors affecting students’ academic achievements and satisfaction and evaluate the perceived effectiveness of K-12 students in response to modular learning. This study applied Structural Equation Modeling (SEM) to test hypotheses. The application of SEM analyzed the correlation among students’ background, experience, behavior, instructor interaction, performance, understanding, satisfaction, academic achievement, and student perceived effectiveness.

A total of 252 data samples were gathered through an online questionnaire. Based on findings, this study concludes that students’ background in modular distance learning affects their behavior and experience. Students’ experiences had significant effects on the performance and understanding of students in modular distance learning. Student instructor interaction had a substantial impact on performance and learning; it explains how vital interaction with the instructor is. The student interacting with the instructor shows that the student may receive feedback and guidance from the instructor. Understanding has a significant influence on students’ satisfaction and academic achievement. Student performance has a substantial impact on students’ academic achievement and satisfaction. Perceived effectiveness was significantly influenced by students’ academic achievement and student satisfaction. However, students’ behavior had no considerable effect on students’ instructor interaction, and students’ understanding while student performance equally had no significant impact on student satisfaction. From this study, students are likely to manifest good performance, behavior, and cognition when they have prior knowledge with regard to modular distance learning. This study will help the government, teachers, and students take the necessary steps to improve and enhance modular distance learning that will benefit students for effective learning.

The modular learning system has been in place since its inception. One of its founding metaphoric pillars is student satisfaction with modular learning. The organization demonstrated its dedication to the student’s voice as a component of understanding effective teaching and learning. Student satisfaction research has been transformed by modular learning. It has caused the education research community to rethink long-held assumptions that learning occurs primarily within a metaphorical container known as a “course.” When reviewing studies on student satisfaction from a factor analytic perspective, one thing becomes clear: this is a complex system with little consensus. Even the most recent factor analytical studies have done little to address the lack of understanding of the dimensions underlying satisfaction with modular learning. Items about student satisfaction with modular distance learning correspond to forming a psychological contract in factor analytic studies. The survey responses are reconfigured into a smaller number of latent (non-observable) dimensions that the students never really articulate but are fully expected to satisfy. Of course, instructors have contracts with their students. Studies such as this one identify the student’s psychological contact after the fact, rather than before the class. The most important aspect is the rapid adoption of this teaching and learning mode in Senior High School. Another balancing factor is the growing sense of student agency in the educational process. Students can express their opinions about their educational experiences in formats ranging from end-of-course evaluation protocols to various social networks, making their voices more critical.

Furthermore, they all agreed with latent trait theory, which holds that the critical dimensions that students differentiate when expressing their opinions about modular learning are formed by the combination of the original items that cannot be directly observed—which underpins student satisfaction. As stated in the literature, the relationship between student satisfaction and the characteristic of a psychological contract is illustrated. Each element is translated into how it might be expressed in the student’s voice, and then a contract feature and an assessment strategy are added. The most significant contributor to the factor pattern, engaged learning, indicates that students expect instructors to play a facilitative role in their teaching. This dimension corresponds to the relational contract, in which the learning environment is stable and well organized, with a clear path to success.

5. Limitations and Future Work

This study was focused on the cognitive capacity of modular distance learning towards academic achievements and satisfaction of K-12 students during the COVID-19 pandemic. The sample size in this study was small, at only 252. If this study is repeated with a larger sample size, it will improve the results. The study’s restriction was to the province of Occidental Mindoro; Structural Equation Modeling (SEM) was used to measure all the variables. Thus, this will give an adequate solution to the problem in the study.

The current study underlines that combining TDT and BTT can positively impact the research outcome. The contribution the current study might make to the field of modular distance learning has been discussed and explained. Based on this research model, the nine (9) factors could broadly clarify the students’ adoption of new learning environment platform features. Thus, the current research suggests that more investigation be carried out to examine relationships among the complexity of modular distance learning.

Funding Statement

This research received no external funding.

Author Contributions

Data collection, methodology, writing and editing, K.A.M.; data collection, writing—review and editing, Y.-T.J. and C.S.S. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Informed consent statement.

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Effect of Modular Distance Learning Approach to Academic Performance in Mathematics of Students in Mindanao State University-Sulu Senior High School Amidst COVID-19 Pandemic

Article sidebar.

Main Article Content

This study investigated the instructional competencies of teachers in The existence of COVID-19 pandemic brought extraordinary challenges to the stakeholders, teachers, parents, and students. Thus, the researcher believed that there is an effect of teaching-learning process in new normal education to students’ performance most especially using modular type of learning in Mathematics. With this, the study sought to determine the perception of the students regarding modular distance learning approach (MDLA) in Mathematics, identify the challenges of the students, examine the effect of MDLA to academic performance of students in Mathematics, determine the level of academic performance of students, determine the significant difference on perceptions when they grouped according to their gender and age, and determine the relationship of students’ perceptions regarding MDLA to their academic performance in Mathematics. The descriptive research design was utilized in this study. The researcher gathered one hundred seventy eight (178) grade 11 STEM students currently enrolled in MSU-Sulu Senior High School through the use of purposive random sampling. The survey questionnaire was applied in the study. Mean, frequency counts and percentage, t-test for independent samples, one-way analysis of variance (ANOVA), and person product-moment correlation were used to analyze and interpret the data. Based on the result, the study revealed that students’ perceptions agreed on using modular distance learning approach (MDLA). It means the students had positive perceptions regarding MDLA in Mathematics. The study also revealed that students agreed on using modular distance learning approach (MDLA) in Math have little challenges. It had also a positive effect to students’ performance in which students performed very satisfactory in Mathematics which means they had good quality performance. However, the study also revealed that it has no significant difference on their perceptions when they are grouped by gender and age which means the students had the same perceptions.

Article Details

Open Access Indonesia Journal of Social Sciences (OAIJSS) allow the author(s) to hold the copyright without restrictions and  allow the author(s) to retain publishing rights without restrictions, also the owner of the commercial rights to the article  is  the author.

Modular Online Learning Design: A Flexible Approach for Diverse Learning Needs— eEditions e-book

The download link for this product can be found on the final confirmation screen after you complete your purchase, and may also be accessed from your Account Profile. For more information about ALA eEditions file types and how to view them on eReaders, desktop computers, and other devices,  see this page .

Primary tabs

You don't need to be an ALA Member to purchase from the ALA Store, but you'll be asked to create an online account/profile during the checkout to proceed. This Web Account is for both members and non-members.

• Description
• Table of Contents
• About the author

Does your online instruction program sometimes feel like a constant scramble to keep pace with requests and deadlines? Modular design is the answer. Approaching projects, whether large and small, with an eye towards future uses will put you on the path to accomplishing broader, organizational goals. And by intentionally building documentation and structure into your process, you will create content that can easily be scaled, modified, adapted, and transformed to meet different learner needs. Hess, experienced in online instruction in both K-12 and academic libraries, shows you how, using project examples of various sizes to illustrate each chapter’s concepts. Her resource guides you through such topics as

• the eight components of modular online learning design;
• key considerations for choosing the design model that best fits your organization and project;
• techniques for connecting your online learning goals with institutional strategy; 
• using the IDEA process to align OER content with your instructional needs;
• documenting your planning with checklists, scaffolds, and templates;
• ensuring equity of access with all content formats using the Accessibility Inventory Index;
• principles for scaling up, down, or laterally;
• three models for more meaningful and functional collaboration with internal or external partners; and
• formative testing as a foundation for ongoing evaluation and assessment.

Using this book as a roadmap, you'll learn how to more intentionally and strategically develop online learning objects to meet different learning needs both now and in the future.

List of Figures Preface Acknowledgments

Chapter 1: Libraries and Online Learning Design Chapter 2: Choosing the Best-Fit Instructional Design Approach  Chapter 3: Tasks of the Design Process Chapter 4: Identifying Stakeholders and Partners  Chapter 5: Modifying and Adapting Existing Content Chapter 6: Designing for Accessibility and Equitable Experiences  Chapter 7: Meaningful Measurement Chapter 8: Flexibility in Action Chapter 9: Forward Thinking for Future Modularity  

Bibliography Index

Amanda Nichols Hess

Amanda Nichols Hess is the e-learning, instructional technology, and education librarian at Oakland University in Rochester, Michigan. She holds a PhD in educational leadership, an Education Specialist certificate in instructional technology, and an MS in information. Her research focuses on information literacy, instructional design, online learning, and the intersections of these topics, particularly in library-centric professional learning. Her work has been published in College and Research Libraries , Communications in Information Literacy , Journal of Academic Librarianship , and portal: Libraries and the Academy , among other venues. Amanda also authored Transforming Academic Library Instruction: Changing Practices to Reflect Changed Perspectives (Rowman and Littlefield, 2019).

"Relevant to library staff in any setting who create learning objects such as video tutorials, self-paced modules, instructional handouts and subject guides. The example projects used and web resources suggested in the text are specific to academic libraries, but the structure of the design process and the planning tools are applicable to special, public and school libraries as well ... As a health sciences librarian in a hospital, I intend to apply some of the concepts and planning tools to the online learning resources that I create.” — Journal of the Canadian Health Libraries Association

"Handy charts, checklists, and workflow models round out each chapter, and the comprehensive bibliography and index will be helpful ... Emphasizing flexibility and functionality, Hess's book will aid librarians who want to save time and energy when creating online learning content." — Library Journal

"Various instructional design models are discussed, including ADDIE, ARCS, and the waterfall design. Also included are helpful charts and figures that make it easy for a visual learner to understand the concepts presented ... Recommended for librarians and instructional designers in academic libraries." — Choice

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

MODULAR DISTANCE LEARNING AMIDST OF COVID-19 PANDEMIC: CHALLENGES AND OPPORTUNITIES

2021, IOER International Multidisciplinary Research Journal

Education is one of the relevant industries caught in the middle of this pandemic and the Philippines has millions of affected learners all over the country. Incidentally, it is necessary to safeguard the education sector through strategies that guarantee the continuous flow of learning integrating online with offline approaches. The researcher aimed to present the difficulties and experiences faced by the learners on Modular Distance Learning. A descriptive, qualitative research was conducted and used an online survey, interview, and observation as tools to gather data and to find out the problems encountered of the learners on this mode of learning. Moore's theory on Transactional Distance Learning served as the framework of analysis and the researcher analyzed the results by thematic coding. A total of 45 learners participated in the online survey and 10 learners participated on online interview. Questions in the survey elicit the situations of the learners and how they managed to study on their own in the absence of learning facilitators to guide them. The result of the survey conducted to section HUMMS 11-Kohlberg determine the accessibility and availability of the gadgets that will be used for modular distance learning, it was revealed that most of the learners' used cellphones to access FB messenger, group chat and google meet for online classes. Learners engaged themselves in understanding the concepts presented in the module as they developed a sense of responsibility in learning on their own and in accomplishing the tasks provided in the module, with limited assistance from the teacher, these learners progress on their own. Today, as the country is at the state of emergency health crisis, these SLMs for Modular Distance Learning were the most convenient, and appropriate to use for our learners to continue learning amidst of Covid-19 pandemic.

Related Papers

IOER International Multidisciplinary Research Journal

IOER International Multidisciplinary Research Journal ( IIMRJ) , Kerwin Paul Gonzales

Schools' stakeholders are the most affected during this time of the pandemic. They are mostly the ones at a loss and are the ones sacrificing, may it be either academically, financially, or both. The different gathered data aimed to provide clarity on the issues and provide propositions on how to conduct the schools' organic functions, possibly during and after the pandemic. In this study, a total of 220 participants came from 44 different schools. The study employed a concurrent-triangulation research design in which an online survey was sent to the participants. Also, teachers coming from international schools and schools outside the Philippines were contacted to have them share their experiences in regards to how their schools handle the situation. Lastly, document analysis was also utilized as a data-gathering procedure.

IOER International Multidisciplinary Research Journal ( IIMRJ) , RYAN V LANSANGAN , Kerwin Paul Gonzales

The outbreak of COVID-19 pandemic as a massive global concern has brought unprecedented challenges in different sectors of the world. One of it is education which posed as one of the most vulnerable sectors significantly impacted by it. This phenomenon changed the mode of instructional delivery and the viewpoint of education stakeholders on the kind of learning continuity applicable to the learners amidst the looming uncertainty brought about by the health crisis. Using phenomenology, this study explored the voices of public Science school teachers regarding their instructional dilemmas to adapt in the demands of the new normal teaching and learning. Findings uncovered seven emerging emotional themes capped as HOPEFUL: Hard-working and dedicated; Optimistic amidst uncertainty; Problematic yet reflective; Evenhandedness in responsibilities; Frightened but ready; Undisruptive desire to reach; and Lifelong learner. Despite the evident uncertainties of the situation, this paper describes the experiences of the Science teachers in their response to their mission of shaping today's generation towards undisruptive education.

IOER Inernational Multidisciplinary Research Journal

IOER International Multidisciplinary Research Journal ( IIMRJ)

COVID-19 pandemic has resulted drastic changes in education. Part of it is the shift from face-to-face classes to different learning modalities which include distance learning. Since education is believed to continue despite the circumstances, teachers started to prepare for modular and online distance learning. Teaching is possible, but, has challenges as well. Hence, this phenomenological research explored the lived experiences of secondary teachers in the Division of San Pablo City in the pre-implementation of distance learning in the new normal. The participants were selected through purposive sampling and underwent one-on-one actual in-depth interview through video conference. The documented interviews were transcribed and coded. Categories were clustered; then, emerging themes were derived. Results identified three core themes related to preparation such as gathering resources and establishing practices, profiling learners, and capacity building for continuous learning and development; three core themes related to challenges such as complexity of assessment, difficulty in instructional delivery and digital divide; and five core themes related to coping mechanisms which include positive well-being, time management, openness to change, peer mentoring, and collaboration. Findings revealed that as education migrates to a New Normal, teachers make necessary preparations to equip themselves with distance learning. Though they face challenges which may hamper their work, they still manage to cope with the new normal to continue their tasks. The higher offices and school authorities should work with teachers at the pre-implementation of distance learning to address their needs in resources and training to effectively facilitate the delivery of quality education for students.

IOER International Multidisciplinary Research Journal ( IIMRJ) , Karl Joseph Sanmocte

Organizational culture and leadership competencies are rarely discussed topics in public organizations. However, by shifting the focus to these two, public organizations are given a leeway to arrive at a holistic view of their entity. CALABARZON PESOs, a public organization, had been encountering the same challenge of addressing unemployment and underemployment. Their performance, on a monthly basis, is fluctuating. This study then sought to improve and stabilize that by determining the relationship between organizational culture and leadership competencies, and the organizational performance of CALABARZON PESOs. This descriptive-correlational study was limited to institutionalized City PESOs and utilized survey questionnaire and focus-group discussion among 15 PESO managers and 100 PESO staff. Using independent t-test and multiple regression analysis, the researcher arrived at the following significant findings: that CALABARZON PESOs have a very strong organizational culture; that CALABARZON PESO managers have above average to excellent leadership competencies, and; that CALABARZON PESOs extraordinarily exceeded the performance standards and expectations. Furthermore, PESO managers and staff have a uniformed perception of their organizational culture and the leadership competency of their PESO managers. Also, there was a significant relationship, at 0.05 level of significance, between organizational culture and organizational performance in terms of efficiency. Among the former's five sub-dimensions, it was only employee-participation that maintained a significant relationship. There was also a significant relationship, at 0.05 level of significance, between leadership competency, taken only as whole, and organizational performance. Considering the said results, this paper had come out with a supplemental manual for a value-added PESO.

Changing school culture requires building professional learning communities that aim to improve and empower teacher's competence, complete well-being, and impact on student learning. This qualitative, descriptive phenomenological study was conducted to determine the purposes, expectations, challenges, and learning and sharing experiences of teachers educating indigenous learners of Mindoro, Philippines. Data were obtained from pre-structured interview of faculty handling multi-grade levels of IP learners delivered essences and emerged themes. Study indicates that teachers' main goal is to transfer understanding and make a difference, they strongly affirmed to deliver significant influence among learners. Meanwhile, learners divergent behavior and learning styles were amongst shared challenges that steered faculty to become progressivists implementing learner-centered approaches. Teachers best realization posited that teaching is a never-ending commitment and education must be inclusive. Based on these results, improving various sources of learning, providing a holistic program for teachers and strengthening the implementation of inclusive education reforms are recommended.

IOER International Multidisciplinary Research Journal ( IIMRJ) , Prodip Chandra Bishwas , Mubin Ul Hakim

This paper aims at presenting online class and its psychological impact relating to satisfaction on University students in Bangladesh during COVID-19 pandemic. A non-experimental survey was conducted using 5-point Likert Scale. Questionnaire was given to the students via internet and 382 students participated the survey. Linear regression analysis and one sample statistics analysis were performed to estimate the students' satisfaction towards online class and its psychological impact on university students in Bangladesh via SPSS version 25. This study revealed some challenges of the students in their online classes due to COVID-19. Results found that that 59.68 percent students are dissatisfied with online classes because of poor internet connection and load-shading. The level of satisfaction from online classes among the students is low and the students do not think that they are getting proper education. Most of the students are suffering from stress, depression, and anxiety and therefore they may struggle for jobs in the future as the job market is getting shrunk due to COVID-19 pandemic. Henceforth, the findings of the study might be useful for the planners and policy makers who are thinking to attain quality online education in Bangladesh.

IOER International Multidisciplinary Research Journal ( IIMRJ) , ELIZABETH NOCHE- ENRIQUEZ

Most of the world's non-English language teachers speak English as a second or third language rather than as their first language. For many, their level of proficiency in English may not reach established by their school heads, colleagues, and students, raising the issue that is the focus of this research. Using the descriptive method of research, the researcher determined the perception of school heads, teachers, and students on the English proficiency level of teachers in terms of academic language and language comprehension. This also determined the level of performance of students in English core subjects, thus, the subject teachers could be encouraged to strive to become better educator to provide a venue for students to continuously dream of becoming better learners. The statistical formulas used for treating the data obtained were Frequency Count, Percentage, T Test of Difference between Unequal Samples, Weighted Mean, Kruskal-Wallis Test of Change, and Pearson R Product-Moment Correlation. The findings showed that the level of English proficiency of teachers and the students' performance on academic language and language comprehension were very satisfactory. Moreover, the findings showed that there was a significant relationship that exists between the English proficiency Level of teachers and the students' performance based on the components considered in the study. With this, a training design to enhance the level of English proficiency of teachers was proposed.

IOER International Multidisciplinary Research Journal ( IIMRJ) , Robin Parojenog , ROBIN C . PAROJENOG

Teachers are known to be versatile. They are equipped with different knowledge and skills for them to be prepared in handling the diversity of learners. It is a fact that each day, they are faced with this challenge as each learner is noted to be unique. Thus, they have different ways of how they should be taught in school. One of the challenges being encountered by educators is on maximizing students' comprehension, especially on literature subjects. Therefore, different approaches have been proposed to accommodate the needs of every learner when it comes to studying literature. This study aimed to analyze the process of teaching and learning of literature among senior high school teachers based on the K to 12 Curriculum for the English subject. Specifically, it sought to find out the teaching learning activities being conducted by the four senior high school literature teachers. It also looked into the teaching approaches mostly preferred by the four senior high school literature teachers. As an offshoot of this study, an instructional module for literature was designed. The study utilized both quantitative and qualitative methodologies. The questionnaire, classroom observation, and interview schedule were used to answer the research questions. This study was limited to the approaches in teaching literature employed by selected senior high school teachers who were handling literature subjects in senior high school in the second district of Ubay, Bohol. The aforementioned findings led the researcher to conclude that the ultimate goal in learning literature is to let the students appreciate its content by using a variety of approaches gleaned from the teaching techniques used by the teachers. Thus, an instructional module was designed to enhance learning by applying the different approaches in teaching literature. This book will serve as a guide to Senior High School literature teachers for them to use varied approaches in teaching the subject for the enhancement of learning. It is recommended that a variety of strategies may be used by teachers in teaching literature to develop not only the cognitive but also the affective and psychomotor domains.

RD Deo , Abraham Deogracias

Knowing that education has been inaccessible and ineffective to some students in the Philippines, it is crucial to know the efficacy and success of the new learning setup during this time of the pandemic to assess if these new modalities are beneficial to the students or not. This paper reviewed and studied the various perceptions and lived experiences of selected students in a private university in Manila regarding online classes amidst the pandemic. Using a phenomenological approach, the researchers gathered data by interviewing 15 students who are attending online classes. Findings and results were drawn on the themes created by the researchers. It can be seen that majority of the students find online classes ineffective and only cater the privileged. Furthermore, lessons and discussions are difficult to digest using the current mode of learning. These findings signify a need to improve the mode of learning in order to provide a quality and effective education to students despite the current crisis faced by the country. Keywords: distance learning, online learning, online classes

IOER INTERNATIONAL MULTIDISCIPLINARY RESEARCH JOURNAL ( IIMRJ )

ABSTRACT Quality education considered as a crucial factor to produce a competent professional to build a strong nation and to bring out the best way to get along with global competition. Thus, this study aimed to determine the current practices in using Web 2.0 tools in 15 selected Higher Education Institutions in CALABARZON, Philippines concerning communication and collaboration, creativity and innovation, and instructional design. It also considered its level of acceptability for classroom instruction as assessed by administrators, teachers, and students. The level of seriousness of the problems met in the integration relative to teachers’ preparation, curriculum content and administrative support were also evaluated. The research design was descriptive survey method with the use of a researcher-constructed questionnaire as the data gathering instrument. The method and instrument employed were deemed appropriate to determine the viability of providing the students with an alternative delivery of learning through Web 2.0 tools for instruction. Weighted mean, T-test and Probability values, Percentage and Standard Deviation were the statistical tools used to test the hypothesis posited in this study. The hypothesis tested the significant differences between two groups of respondents regarding the extent of use of Web 2.0 tools in classroom instruction. Results revealed that to a very great extent, integration of Web 2.0 tools in the classroom promotes learner to interact, build a learning community and promotes student active participation in the classroom and increases student’s productivity. Based on the findings and conclusions, the researcher developed an offline game-based interactive instructional material that supports instruction and collaboration and could be used to enhance students’ critical thinking and problem-solving skills to achieve better learning outcomes. Keywords: Web 2.0 tools, 21st Century students, Communication and Collaboration, Descriptive method, Higher Education, Philippines

RELATED PAPERS

Belo Horizonte: UFMG/ …

Edson Domingues

Alfas Pliura

European Journal of Obstetrics & Gynecology and Reproductive Biology

Elīna Gelderiņa

Rudolf Muhr

Revista Brasileira de Ciência do Solo

Fábio Olivares

Sensors and Actuators B: Chemical

Ashok Ranjan

Malaria Journal

Turkish Journal of Trauma and Emergency Surgery

volkan baytaş

Terahertz Emitters, Receivers, and Applications XI

Andrei Gorodetsky

Ogunuyo Ejiro

The Plant Pathology Journal

Mahmud Mohamed

Carlos Ortega Elduque

IEICE Electronics Express

Saroj Tripathi

Cristiane Santos

Questões Polêmicas do Regime Tributário de ISS das Sociedades Profissionais

Michell Przepiorka

IEEE Transactions on Signal Processing

Vijay Madisetti

Wardah Ebrahim

Gastroenterology

John Abraham

ACS Applied Energy Materials

akmal abbas

Paul Auwaerter

Revista chilena de pediatría

Fabian Stiven Villero Vasquez

Wesleyan Economics WP

Daniel Ortega

Environmental Toxicology

Asok Dasmahapatra

International Journal of Cancer

Gary Beecher

See More Documents Like This

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Science and Information Conference

SAI 2023: Intelligent Computing pp 561–595 Cite as

Modularity in Deep Learning: A Survey

• Haozhe Sun 10 &
• Isabelle Guyon 10 , 11  
• Conference paper
• First Online: 20 August 2023

608 Accesses

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 739))

Modularity is a general principle present in many fields. It offers attractive advantages, including, among others, ease of conceptualization, interpretability, scalability, module combinability, and module reusability. The deep learning community has long sought to take inspiration from the modularity principle, either implicitly or explicitly. This interest has been increasing over recent years. We review the notion of modularity in deep learning around three axes: data, task, and model, which characterize the life cycle of deep learning. Data modularity refers to the observation or creation of data groups for various purposes. Task modularity refers to the decomposition of tasks into sub-tasks. Model modularity means that the architecture of a neural network system can be decomposed into identifiable modules. We describe different instantiations of the modularity principle, and we contextualize their advantages in different deep learning sub-fields. Finally, we conclude the paper with a discussion of the definition of modularity and directions for future research.

• Deep Learning
• Neural Networks

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Accelerate Fast Math with Intel® oneAPI Math Kernel Library. https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html

Abbas, A., Abdelsamea, M.M., Gaber, M.M.: DeTraC: transfer learning of class decomposed medical images in convolutional neural networks. IEEE Access 8 , 74901–74913 (2020)

Article   Google Scholar  

Abdrakhmanova, M., et al.: Speakingfaces: a large-scale multimodal dataset of voice commands with visual and thermal video streams. Sensors 21 (10), 3465 (2021)

Abraham, W.C., Robins, A.: Memory retention - the synaptic stability versus plasticity dilemma. Trends Neurosci. 28 (2), 73–78 (2005)

Alain, G., Bengio, Y.: Understanding intermediate layers using linear classifier probes. arXiv preprint: arXiv:1610.01644 (2016)

Alet, F., Lozano-Pérez, T., Kaelbling, L.P.: Modular meta-learning. arXiv:1806.10166 [cs, stat], May 2019

Alias Parth Goyal, A.G., et al.: Neural production systems. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Wortman Vaughan, J. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 25673–25687. Curran Associates, Inc., (2021)

Google Scholar  

Almeida, F., Xexéo, G.: Word embeddings: a survey, January 2019

Amer, M., Maul, T.: A review of modularization techniques in artificial neural networks. Artif. Intell. Rev. 52 , 527–561 (2019)

Anderson, A., Shaffer, K., Yankov, A., Corley, C.D., Hodas, N.O.: Beyond fine tuning: a modular approach to learning on small data, November 2016

Andreas, J.: Measuring compositionality in representation learning. In: International Conference on Learning Representations (2019)

Andreas, J., Rohrbach, M., Darrell, T., Klein, D.: Neural module networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 39-48, IEEE, June 2016

Auda, G., Kamel, M.: Modular neural networks a survey. Int. J. Neural Syst. 9 (2), 129–51 (1999)

Avigad, J.: Modularity in mathematics. Rev. Symbolic Logic 13 (1), 47–79 (2020)

Article   MathSciNet   MATH   Google Scholar  

Azam, F.: Biologically Inspired Modular Neural Networks. PhD thesis, Virginia Tech, May 2000

Bahdanau, D., Murty, S., Noukhovitch, M., Nguyen, T.H., de Vries, H. and Courville, A.: Systematic generalization: what is required and can it be learned? In: International Conference on Learning Representations (2019)

Baldwin, C.Y., Clark, K.B.: Design Rules: The Power of Modularity, vol. 1, 1st edn. MIT Press, Cambridge (1999)

Balestriero, R., LeCun, Y.: POLICE: Provably optimal linear constraint enforcement for deep neural networks, November 2022

Baltrušaitis, T., Ahuja, C., Morency, L.-P.: Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41 (2), 423–443 (2018)

Bao, H., Dong, L., Piao, S., Wei, F.: BEiT: BERT pre-training of image transformers. In: International Conference on Learning Representations (2022)

Barham, P., et al.: Pathways: asynchronous Distributed Dataflow for ML. arXiv:2203.12533 [cs], March 2022

Barrett, H.C., Kurzban, R.: Modularity in cognition: framing the debate. Psychol. Rev. 113 (3), 628–647 (2006)

Battaglia, P.W., et al.: Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261 [cs, stat], October 2018

Bekkers, E.J., Lafarge, M.W., Veta, M., Eppenhof, K.A., Pluim, J.P., Duits, R.: Roto-translation covariant convolutional networks for medical image analysis. arXiv:1804.03393 [cs, math], June 2018

Belay, B., Habtegebrial, T., Liwicki, M., Belay, G., Stricker, D.: Factored convolutional neural network for amharic character image recognition. In: 2019 IEEE International Conference on Image Processing (ICIP) , pp. 2906–2910 (2019)

Béna, G., Goodman, D.F.M.: Extreme sparsity gives rise to functional specialization. arXiv:2106.02626 [cs, q-bio], June 2021

Bengio, E., Bacon, P.L., Pineau, J., Precup, D.: Conditional Computation in Neural Networks for faster models. arXiv:1511.06297 [cs], January 2016

Bengio, Y., Leonard, N., Courville, A.: Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv:1308.3432 [cs], August 2013

Bongard, J.: Evolving modular genetic regulatory networks. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC2002 (Cat. No.02TH8600), vol. 2, pp. 1872–1877, May 2002

Bouchacourt, D., Ibrahim, M., Morcos, A.: Grounding inductive biases in natural images: Invariance stems from variations in data. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 19566–19579. Curran Associates, Inc., (2021)

Bourbaki, N.: The architecture of mathematics. Am. Math. Mon. 57 (4), 221–232 (1950)

Brandes, U., et al.: On modularity clustering. IEEE Trans. Knowl. Data Eng. 20 (2), 172–188 (2007)

Braylan, A., Hollenbeck, M., Meyerson, E., Miikkulainen, R.: Reuse of neural modules for general video game playing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)

Bromley, J., Guyon, I., LeCun, Y., Sackinger, E., Shah, R.: Signature verification using a “Siamese” time delay neural network. In: Cowan, J., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6. Morgan-Kaufmann (1994)

Brown, T., et al.: Language models are few-shot learners. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901. Curran Associates, Inc., (2020)

Cao, Q., Trivedi, H., Balasubramanian, A., Balasubramanian, N.: DeFormer: decomposing pre-trained transformers for faster question answering. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4487–4497. Association for Computational Linguistics, July 2020

Casey, R.G., Lecolinet, E.: A survey of methods and strategies in character segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18 (7), 690–706 (1996)

Chang, M.B., Gupta, A., Levine, S., Griffiths, T.L.: Automatically composing representation transformations as a means for generalization. In: International Conference on Learning Representations (2019)

Chang, M., Kaushik, S., Levine, S., Griffiths, T.: Modularity in reinforcement learning via algorithmic independence in credit assignment. In: International Conference on Machine Learning, pp. 1452–1462. PMLR, July 2021

Chen, S., Dobriban, E., Lee, J.H.: A group-theoretic framework for data augmentation. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 21321–21333. Curran Associates, Inc., (2020)

Chen, X., Jin, L., Zhu, Y., Luo, C., Wang, T.: Text recognition in the wild: a survey. arXiv:2005.03492 [cs], December 2020

Chen, X., He, K.: Exploring simple siamese representation learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15745–15753, Nashville, TN, USA. IEEE, June 2021

Chen, Y., et al.: Modular meta-learning with shrinkage. In: Advances in Neural Information Processing Systems, vol. 33, pp. 2858–2869 (2020)

Chevalier, G.: Long short-term memory (LSTM cell). Wikipedia, September 2022

Cho, K., Van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder–decoder approaches. In: Syntax, Semantics and Structure in Statistical Translation, p. 103 (2014)

Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

Chomsky, N.: Aspects of the Theory of Syntax. MIT Press, Cambridge (1965)

Choudhary, A., Rishi, R., Ahlawat, S.: A new character segmentation approach for off-line cursive handwritten words. Procedia Comput. Sci. 17 , 88–95 (2013)

Chowdhery, A., et al.: PaLM: scaling language modeling with pathways. arXiv:2204.02311 [cs], April 2022

Chu, B., Madhavan, V., Beijbom, O., Hoffman, J., Darrell, T.: Best practices for fine-tuning visual classifiers to new domains. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9915, pp. 435–442. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49409-8_34

Chapter   Google Scholar  

Clavera, I., Held, D., Abbeel, P.: Policy transfer via modularity and reward guiding. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1537–1544. IEEE (2017)

Clune, J., Mouret, J.-B., Lipson, H.: The evolutionary origins of modularity. Proc. R. Soc. b: Biol. Sci. 280 (1755), 20122863 (2013)

Cohen, T., Welling, M.: Group equivariant convolutional networks. arXiv:1602.07576 [cs, stat], June 2016

Cohen, T.S., Welling, M.: Steerable CNNs. arXiv:1612.08498 [cs, stat], December 2016

Cohen-Boulakia, S., et al.: Scientific workflows for computational reproducibility in the life sciences: status, challenges and opportunities. Futur. Gener. Comput. Syst. 75 , 284–298 (2017)

CColas, C., Fournier, P., Chetouani, M., Sigaud, O., Oudeyer, P.Y.: Curious: Intrinsically motivated modular multi-goal reinforcement learning. In: International Conference on Machine Learning, pp. 1331–1340. PMLR (2019)

Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms (2022)

Cosmides, L., Tooby, J.: Cognitive adaptations for social exchange. Undefined, 163–228 (1992)

Cosmides, L., Tooby, J.: Origins of domain specificity: the evolution of functional organization. In: Hirschfeld, L.A., Gelman, S.A. (eds.) Mapping the Mind: Domain Specificity in Cognition and Culture, pp. 85–116. Cambridge University Press, Cambridge (1994)

Csordás, R., Irie, K., Schmidhuber, J.: CTL++: evaluating generalization on never-seen compositional patterns of known functions, and compatibility of neural representations. In: Proceedings Conference on Empirical Methods in Natural Language Processing (EMNLP), December 2022

Csordás, R., van Steenkiste, S., Schmidhuber, J.: Are neural nets modular? inspecting functional modularity through differentiable weight masks. In: International Conference on Learning Representations (2021)

D’Amario, V., Sasaki, T., Boix, X.: How modular should neural module networks be for systematic generalization? In: Thirty-Fifth Conference on Neural Information Processing Systems (2021)

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

Devin, C., Gupta, A., Darrell, T., Abbeel, P., Levine, S.: Learning modular neural network policies for multi-task and multi-robot transfer. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2169–2176. IEEE (2017)

Dieleman, S., De Fauw, J., Kavukcuoglu, K.: Exploiting cyclic symmetry in convolutional neural networks. arXiv:1602.02660 [cs], May 2016

Ding, C., Tao, D.: Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40 (4), 1002–1014 (2017)

Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (2021)

Du, N., et al.: Glam: efficient scaling of language models with mixture-of-experts. In: International Conference on Machine Learning, pp. 5547–5569. PMLR (2022)

Eastwood, C., Williams, C.K.: A framework for the quantitative evaluation of disentangled representations. In: Sixth International Conference on Learning Representations (ICLR 2018), May 2018

Eigen, D., Ranzato, M.A., Sutskever, I.: Learning factored representations in a deep mixture of experts. In: ICLR Workshop (2014)

El Baz, A., et al.: Lessons learned from the NeurIPS 2021 MetaDL challenge: backbone fine-tuning without episodic meta-learning dominates for few-shot learning image classification. In: Kiela, D., Ciccone, M., Caputo, B. (eds.) Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, volume 176 of Proceedings of Machine Learning Research, pp. 80–96. PMLR, December 2022

Ellefsen, K.O., Mouret, J.B., Clune, J.: Neural modularity helps organisms evolve to learn new skills without forgetting old skills. PLoS Comput. Biol. 11 (4), e1004128 (2015)

Elsayed, G.F., Ramachandran, P., Shlens, J., Kornblith, S.: Revisiting spatial invariance with low-rank local connectivity. arXiv:2002.02959 [cs, stat], August 2020

Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search. pp. 69–86

Fedus, W., Dean, J., Zoph, B.: A review of sparse expert models in deep learning, September 2022

Fedus, W., Zoph, B., Shazeer, N.: Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res. 23 (120), 1–39 (2022)

MathSciNet   Google Scholar  

Fernando, C., et al.: PathNet: evolution channels gradient descent in super neural networks. arXiv:1701.08734 [cs], January 2017

Filan, D., Casper, S., Hod, S., Wild, C., Critch, A., Russell, S.: Clusterability in neural networks. arXiv:2103.03386 [cs], March 2021

Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. arXiv:1703.03400 [cs], July 2017

Fodor, J.A.: The modularity of mind, April 1983

Fodor, J.A.: The Mind Doesn’t Work That Way: The Scope and Limits of Computational Psychology. MIT Press, Cambridge (2000)

Book   Google Scholar  

Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28 (1–2), 3–71 (1988)

Ford, M.: Architects of Intelligence: The Truth about AI from the People Building It. Packt Publishing, Birmingham, first published: November 2018 edition (2018)

Frankenhuis, W.E., Ploeger, A.: Evolutionary psychology versus fodor: arguments for and against the massive modularity hypothesis. Philos. Psychol. 20 (6), 687–710 (2007)

French, R.: Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks (1991)

Fuengfusin, N., Tamukoh, H.: Network with sub-networks: layer-wise detachable neural network. J. Robot., Netw. Artif. Life 7 (4), 240–244 (2020)

Galanti, T., Wolf, L.: On the modularity of hypernetworks. arXiv:2002.10006 [cs, stat], November 2020

Gao, H., Ji, S.: Efficient and invariant convolutional neural networks for dense prediction. In: 2017 IEEE International Conference on Data Mining (ICDM), pp. 871–876 (2017)

Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 2414–2423. IEEE, June 2016

Gavali, P., Banu, J.S.: Chapter 6 - deep convolutional neural network for image classification on CUDA platform. In: Sangaiah, AK. (ed.) Deep Learning and Parallel Computing Environment for Bioengineering Systems, pp. 99–122. Academic Press (2019)

Gentile, P.: Theory of modularity, a hypothesis. Procedia Comput. Sci. 20 (2013)

Ghazi, B., Panigrahy, R., Wang, J.: Recursive sketches for modular deep learning. In: Proceedings of the 36th International Conference on Machine Learning, pp. 2211–2220. PMLR, May 2019

Gómez, D., Rodríguez, J.T., Yáñez, J., Montero, J.: A new modularity measure for Fuzzy community detection problems based on overlap and grouping functions. Int. J. Approximate Reasoning 74 , 88–107 (2016)

Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

MATH   Google Scholar  

Goodfellow, I.J., et al.: Generative adversarial networks. arXiv:1406.2661 [cs, stat], June 2014

Goyal, A., et al.: Recurrent independent mechanisms. In: International Conference on Learning Representations (2021)

Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., Parikh, D.: Making the V in VQA matter: elevating the role of image understanding in visual question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6904–6913 (2017)

Gray, S., Radford, A., Kingma, D.P.; GPU Kernels for Block-Sparse Weights. Technical report

Ha, D., Dai, A., Le, Q.V.: HyperNetworks. arXiv:1609.09106 [cs], December 2016

Hacohen, G., Weinshall, D.: On the power of curriculum learning in training deep networks. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 2535–2544. PMLR, June 2019

Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference and Prediction, 2nd edn. Springer, Cham (2009)

Book   MATH   Google Scholar  

He, J., et al.: FasterMoE: modeling and optimizing training of large-scale dynamic pre-trained models. In: Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 120–134 (2022)

He, K., Chen, X., Xie, S., Li, Y., Dollar, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)

He, K., Girshick, R., Dollar, P.: Rethinking ImageNet pre-training. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4918–4927 (2019)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313 (5786), 504–507 (2006)

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9 (8), 1735–1780 (1997)

Hofman, M.A.: Evolution of the human brain: when bigger is better. Front. Neuroanat. 8 , 15 (2014)

Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint: arXiv:1704.04861 (2017)

Hu, G., et al.: Deep stock representation learning: from candlestick charts to investment decisions. arXiv:1709.03803 [q-fin], February 2018

Hu, R., Andreas, J., Rohrbach, M., Darrell, T., Saenko, K.: Learning to reason: end-to-end module networks for visual question answering. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)

Huang, J., et al.: A multiplexed network for end-to-end, multilingual OCR. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4547–4557 (2021)

Huizinga, J., Clune, J., Mouret, J.B.: Evolving neural networks that are both modular and regular: Hyperneat plus the connection cost technique. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 697–704 (2014)

Hupkes, D., Dankers, V., Mul, M., Bruni, E.: Compositionality decomposed: how do neural networks Generalise? J. Artif. Intell. Res. 67 , 757–795 (2020)

Article   MathSciNet   Google Scholar  

Hupkes, D., et al.: State-of-the-art generalisation research in NLP: a taxonomy and review, October 2022

Hutter, F., Kotthoff, L., Vanschoren, J. (eds.): Automatic Machine Learning: Methods, Systems, Challenges. Springer, Cham (2019)

Islam, R., et al.: Discrete factorial representations as an abstraction for goal conditioned reinforcement learning, October 2022

Jacobs, R.A., Jordan, M.I., Barto, A.G.: Task decomposition through competition in a modular connectionist architecture: the what and where vision tasks. Cogn. Sci. 15 (2), 219–250 (1991)

Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3 (1), 79–87 (1991)

Javed, K., White, M.: Meta-learning representations for continual learning. arXiv:1905.12588 [cs, stat], October 2019

Jin, T., Hong, S.: Split-CNN: splitting window-based operations in convolutional neural networks for memory system optimization. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS 2019, New York, NY, USA, pp. 835–847. Association for Computing Machinery (2019)

Jing, L., Zhu, J., LeCun, Y.: Masked siamese ConvNets, June 2022

Jordan, M.I., Jacobs, R.A.: Hierarchical mixtures of experts and the EM algorithm. Neural Comput. 6 (2), 181–214 (1994)

Cheng, J., Bibaut, A., van der Laan, M.: The relative performance of ensemble methods with deep convolutional neural networks for image classification. J. Appl. Stat. 45 (15), 2800–2818 (2018)

Jurafsky, D., Martin, J.H.: Speech and Language Processing. (3rd draft ed.) (2019)

Kanakis, M., Bruggemann, D., Saha, S., Georgoulis, S., Obukhov, A., Van Gool, L.: Reparameterizing convolutions for incremental multi-task learning without task interference. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12365, pp. 689–707. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58565-5_41

Kassner, N., Tafjord, O., Schutze, H., Clark, P.: BeliefBank: adding memory to a pre-trained language model for a systematic notion of belief. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8849–8861 (2021)

Kaur, A., Baghla, S., Kumar, S.: Study of various character segmentation techniques for handwritten off-line cursive words: a review. Int. J. Adv. Sci. Eng. Technol. 3 (3), 154–158 (2015)

Ke, Z., Liu, B., Nianzu Ma, H.X., Shu, L.: Achieving forgetting prevention and knowledge transfer in continual learning. In: Advances Neural Information Processing System, vol. 34, pp. 22443–22456 (2021)

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.; BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)

Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On large-batch training for deep learning: generalization gap and sharp minima. In: ICLR (2017)

Keysers, D., et al.: Measuring compositional generalization: a comprehensive method on realistic data. In: International Conference on Learning Representations (2020)

Kim, J., Park, Y., Kim, G., Hwang, S.J.: SplitNet: learning to semantically split deep networks for parameter reduction and model parallelization. In: Proceedings of the 34th International Conference on Machine Learning, pp. 1866–1874. PMLR, July 2017

Kingetsu, H., Kobayashi, K., Suzuki, T.: Neural network module decomposition and recomposition, December 2021

Kirsch, L., Kunze, J., Barber, D.: Modular networks: learning to decompose neural computation. In: Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc., (2018)

Koh, E., Dubnov, S.: Comparison and analysis of deep audio embeddings for music emotion recognition, April 2021

Yamuna Krishnamurthy and Chris Watkins. Interpretability in gated modular neural networks. In eXplainable AI Approaches for Debugging and Diagnosis. , 2021

Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

Krueger, D., et al.: Zoneout: regularizing RNNs by randomly preserving hidden activations. In: International Conference on Learning Representations (2017)

Kurzweil, R.: How to Create a Mind: The Secret of Human Thought Revealed. Penguin Books, USA (2013)

Laenen, S., Bertinetto, L.: On episodes, prototypical networks, and few-shot learning. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Wortman Vaughan, J. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 24581–24592. Curran Associates, Inc., (2021)

Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350 (6266), 1332–1338 (2015)

Lake, B., Baroni, M.: Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In: International Conference on Machine Learning, pp. 2873–2882. PMLR (2018)

Lake, B.M.: Compositional generalization through meta sequence-to-sequence learning. arXiv:1906.05381 [cs], October 2019

LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II–104, June 2004

LeCun, Y.: A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27 (2022)

LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86 (11), 2278–2324 (1998)

LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. In: Advances in Neural Information Processing Systems, vol. 2 (1989)

Li, M., Vitanyi, P.: An Introduction to Kolmogorov Complexity and Its Applications, 3rd edn. Springer Publishing Company, Incorporated, Cham (2008)

Li, N., Liu, S., Liu, Y., Zhao, S., Liu, M.: Neural speech synthesis with transformer network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6706–6713 (2019)

Li, Z., Wu, B., Liu, Q., Wu, L., Zhao, H., Mei, T.: Learning the compositional visual coherence for complementary recommendations. In: Bessiere, C. (ed.) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 3536–3543. International Joint Conferences on Artificial Intelligence Organization, July 2020

Liu, H., Simonyan, K., Vinyals, O., Fernando, C., Kavukcuoglu, K.: Hierarchical representations for efficient architecture search. In: International Conference on Learning Representations (2018)

Loula, J., Baroni, M., Lake, B.M.: Lake. Rearranging the familiar: testing compositional generalization in recurrent networks. In: BlackboxNLP@EMNLP, pp. 108–114 (2018)

Ma, J., Cui, P., Kuang, K., Wang, X., Zhu, W.: Disentangled graph convolutional networks. In: Chaudhuri, K., Salakhutdinov, R. (ed.) Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 4212–4221. PMLR, June 2019

Maninis, K.K., Radosavovic, I., Kokkinos, I.: Attentive single-tasking of multiple tasks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 1851–1860. IEEE, June 2019

Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015)

Masse, N.Y., Grant, G.D., Freedman, D.J.: Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. Proc. Natl. Acad. Sci. 115 (44), E10467–E10475 (2018)

Mazzia, V., Salvetti, F., Chiaberge, M.: Efficient-CapsNet: capsule network with self-attention routing. Sci. Rep. 11 (1), 1–13 (2021)

McNeely-White, D., Beveridge, J.R., Draper, B.A.: Inception and ResNet features are (almost) equivalent. Cogn. Syst. Res. 59 , 312–318 (2020)

Meng, K., Bau, D., Andonian, A., Belinkov, Y.: Locating and editing factual associations in GPT, February 2022

Meyerson, E., Miikkulainen, R.: Modular universal reparameterization: Deep multi-task learning across diverse domains. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

Mitchell, E., Lin, C., Bosselut, A., Finn, C., Manning, C.D.: Fast model editing at scale. arXiv:2110.11309 [cs], October 2021

Mitchell, E., Lin, C., Bosselut, A., Manning, C.D., Finn, C.: Memory-based model editing at scale. In: International Conference on Machine Learning (2022)

Mittal, S., Bengio, Y., Lajoie, G.: Is a Modular architecture enough? (2022)

Mittal, S., Raparthy, S.C., Rish, I., Bengio, Y., Lajoie, G.: Compositional attention: disentangling search and retrieval. In: International Conference on Learning Representations (2022)

Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. arXiv e-prints , page arXiv:1602.01783 , February 2016

Modrak, V., Soltysova, Z.: Development of the modularity measure for assembly process structures. Math. Probl. Eng. 2021 , e4900748 (2021)

Muff, S., Rao, F., Caflisch, A.: Local modularity measure for network clusterizations. Phys. Rev. E 72 (5), 056107 (2005)

Murty, S., Sharma, P., Andreas, J., Manning, C.D.: Manning. Characterizing intrinsic compositionality in transformers with tree projections, November 2022

Newman, M.E.J.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. 103 (23), 8577–8582 (2006)

Opitz, M., Possegger, H., Bischof, H.: Efficient model averaging for deep neural networks. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10112, pp. 205–220. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54184-6_13

Ostapenko, O., Rodriguez, P., Caccia, M., Charlin, L.: Continual learning via local module composition. In: Advances in Neural Information Processing Systems, vol. 34, pp. 30298–30312 (2021)

Ostapenko, O., Rodriguez, P., Lacoste, A., Charlin, L.: Attention for compositional modularity. In: NeurIPS 2022 Workshop on All Things Attention: Bridging Different Perspectives on Attention (2022)

Pan, R., Rajan, H.: On decomposing a deep neural network into modules. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, New York, NY, USA, pp. 889–900. Association for Computing Machinery (2020)

Pan, R., Rajan, H.: Decomposing convolutional neural networks into reusable and replaceable modules. In: Proceedings of The 44th International Conference on Software Engineering (ICSE 2022), December 2021

Parascandolo, G., Kilbertus, N., Rojas-Carulla, M., Schölkopf, B.: Learning independent causal mechanisms. In: International Conference on Machine Learning, pp. 4036–4044. PMLR (2018)

Parnas, D.L.: On the criteria to be used in decomposing systems into modules. Commun. ACM 15 (12), 1053–1058 (1972)

Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A.,. d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc., (2019)

Pathak, D., Lu, C., Darrell, T., Isola, P., Efros, A.A.: Learning to control self-assembling morphologies: a study of generalization via modularity. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

Pereira-Leal, J.B., Levy, E.D., Teichmann, S.A.: The origins and evolution of functional modules: lessons from protein complexes. Philos. Trans. R. Soc. B: Biol. Sci. 361 (1467), 507–517 (2006)

Peters, J., Janzing, D., Schölkopf, B.: Elements of Causal Inference: Foundations and Learning Algorithms. Adaptive Computation and Machine Learning Series. MIT Press, Cambridge (2017)

Poisot, T.: An a posteriori measure of network modularity. F1000Research 2 , 130 (2013)

Ponti, E.: Inductive Bias and Modular Design for Sample-Efficient Neural Language Learning. PhD thesis, University of Cambridge (2021)

Ponti, E.M., Sordoni, A., Bengio, Y., Reddy, S.: Combining modular skills in multitask learning, March 2022

Purushwalkam, S., Nickel, M., Gupta, A., Ranzato, M.A.: Task-driven modular networks for zero-shot compositional learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3593–3602 (2019)

Pylyshyn, Z.: Is vision continuous with cognition?: The case for cognitive impenetrability of visual perception. Behav. Brain Sci. 22 (3), 341–365 (1999)

Qiao, J.-F., Meng, X., Li, W.-J., Wilamowski, B.M.: A novel modular RBF neural network based on a brain-like partition method. Neural Comput. Appl. 32 (3), 899–911 (2020)

Rahaman, N., et al.: Dynamic inference with neural interpreters. In: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems, vol. 34, pp. 10985–10998. Curran Associates, Inc., (2021)

Ramachandran, P., Le, Q.V.: Diversity and depth in per-example routing models. In: International Conference on Learning Representations (2019)

Ranganathan, G., et al.: A study to find facts behind preprocessing on deep learning algorithms. J. Innov. Image Process. (JIIP) 3 (01), 66–74 (2021)

Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: International Conference on Learning Representations (2017)

Reisinger, J., Stanley, K.O., Miikkulainen, R.: Evolving reusable neural modules. In: Deb, K. (ed.) GECCO 2004. LNCS, vol. 3103, pp. 69–81. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24855-2_7

Ren, P., et al.: A survey of deep active learning. ACM Comput. Surv. (CSUR) 54 (9), 1–40 (2021)

Ridgeway, K., Mozer, M.C.: Learning deep disentangled embeddings with the f-statistic loss. In: Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc., (2018)

Robbins, P.: Modularity of mind. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, winter 2017 edition (2017)

Rose, J.S.: A Course on Group Theory. Courier Corporation, Massachusetts (1994)

Rosenbaum, C., Cases, I., Riemer, M., Klinger, T.: Routing networks and the challenges of modular and compositional computation, April 2019

Rosenbaum, C., Klinger, T., Riemer, M.: Routing networks: adaptive selection of non-linear functions for multi-task learning. In: International Conference on Learning Representations (2018)

Ruder, S.: An overview of gradient descent optimization algorithms. arXiv preprint: arXiv:1609.04747 (2016)

Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science (1985)

Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115 , 211–252 (2015)

Rusu, A.A., et al.: Progressive neural networks. arXiv:1606.04671 [cs], September 2016

Salha-Galvan, G., Lutzeyer, J.F., Dasoulas, G., Hennequin, R., Vazirgiannis, M.: Modularity-aware graph autoencoders for joint community detection and link prediction, June 2022

Schenkel, M., Weissman, H., Guyon, I., Nohl, C., Henderson, D.: Recognition-based segmentation of on-line hand-printed words. In: Hanson, S., Cowan, J., Giles, C. (eds.) Advances in Neural Information Processing Systems, vol. 5. Morgan-Kaufmann (1992)

Schilling, M.: Toward a general modular systems theory and its application to interfirm product modularity. Acad. Manag. Rev. 25 (2000)

Schmidhuber, J.: Towards compositional learning in dynamic networks (1990)

Schmidt, A.L., Bandar, Z.U.: Modularity - a concept for new neural network architectures. November 2001

Shao, Y., Zavala, V.M.: Modularity measures: concepts, computation, and applications to manufacturing systems. AIChE J. 66 (6), e16965 (2020)

Shazeer, N., et al.: Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net (2017)

Shi, B., Bai, X., Yao, C.: Script identification in the wild via discriminative convolutional neural network. Pattern Recogn. 52 , 448–458 (2016)

Shin, H., Lee, J.K., Kim, J., Kim, J.: Continual learning with deep generative replay. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (edis.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., (2017)

Shiokawa, H., Fujiwara, Y., Onizuka, M.: Fast algorithm for modularity-based graph clustering. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 27, pp. 1170–1176 (2013)

Laurent, S.: Rigid-Motion Scattering for Image Classification [PhD Thesis]. PhD thesis (2014)

Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529 (7587), 484–489 (2016)

Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550 (7676), 354–359 (2017)

Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: Seventh International Conference on Document Analysis and Recognition, Proceedings , pp. 958–963, August 2003

Simon, H.A.: The architecture of complexity. Proc. Am. Philos. Soc. 106 (6), 467–482 (1962)

Simon, H.A., Ando, A.: Aggregation of variables in dynamic systems. Econometrica 29 (2), 111–138 (1961)

Article   MATH   Google Scholar  

Simpkins, C., Isbell, C.: Composable modular reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4975–4982 (2019)

Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., Babenko, A.: Editable neural networks. In: International Conference on Learning Representations (2019)

Smith, S.L., Kindermans, P.J., Ying, C., Le, Q.V.: Don’t decay the learning rate, increase the batch size. In: International Conference on Learning Representations (2018)

Smith, S., et al.: Using DeepSpeed and Megatron to train Megatron-Turing NLG 530B, a large-scale generative language model, February 2022

Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. arXiv:1703.05175 [cs, stat], June 2017

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15 (56), 1929–1958 (2014)

MathSciNet   MATH   Google Scholar  

Sun, C., Shrivastava, A., Singh, S., Gupta, A.: Revisiting unreasonable effectiveness of data in deep learning era. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 843–852 (2017)

Sun, G., et al.: Task switching network for multi-task learning. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, pp. 8271–8280. IEEE, October 2021

Sun, H., Tu, W.W., Guyon, I.: OmniPrint: a configurable printed character synthesizer. In: Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) (2021)

Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)

Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. arXiv:1703.01780 [cs, stat], April 2018

Teerapittayanon, S., McDanel, B., Kung, H.T.: BranchyNet: fast inference via early exiting from deep neural networks. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2464–2469 (2016)

Terekhov, A.V., Montone, G., O’Regan, J.K.: Knowledge transfer in deep block-modular neural networks. In: Wilson, S.P., Verschure, P.F.M.J., Mura, A., Prescott, T.J. (eds.) LIVINGMACHINES 2015. LNCS (LNAI), vol. 9222, pp. 268–279. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22979-9_27

Tishby, N., Zaslavsky, N.: Deep learning and the information bottleneck principle. In: 2015 IEEE Information Theory Workshop (ITW), pp. 1–5 (2015)

Triantafillou, E., et al.: Meta-dataset: a dataset of datasets for learning to learn from few examples. In: International Conference on Learning Representations (2019)

Ullah, I., et al.: Meta-album: multi-domain meta-dataset for few-shot image classification (2022)

Vankov, I.I., Bowers, J.S.: Training neural networks to encode symbols enables combinatorial generalization. Philos. Trans. R. Soc. B 375 (1791), 20190309 (2020)

Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., (2017)

Veniat, T., Denoyer, L., Ranzato, M.A.: Efficient continual learning with modular networks and task-driven priors. In: 9th International Conference on Learning Representations, ICLR 2021 (2021)

Von Luxburg, U., Williamson, R.C., Guyon, I.: Clustering: science or art? In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning. JMLR Workshop and Conference Proceedings, pp. 65–79, June 2012

Wagner, G.P., Altenberg, L.: Perspective: complex adaptations and the evolution of evolvability. Evolution 50 (3), 967–976 (1996)

Wang, H., Zhao, H., Li, B.: Bridging multi-task learning and meta-learning: towards efficient training and effective adaptation. In: International Conference on Machine Learning, pp. 10991–11002. PMLR (2021)

Wang, J., Sezener, E., Budden, D., Hutter, M., Veness, J.: A Combinatorial perspective on transfer learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 918–929. Curran Associates, Inc., (2020)

Wang, R., Pontil, M., Ciliberto, C.: The role of global labels in few-shot classification and how to infer them. In: Advances in Neural Information Processing Systems, vol. 34, pp. 27160–27170 (2021)

Watanabe, C., Hiramatsu, K., Kashino, K.: Modular representation of layered neural networks. Neural Netw. 97 , 62–73 (2018)

Weiler, M., Cesa, G.: General \$E(2)\$-equivariant steerable CNNs. arXiv:1911.08251 [cs, eess], April 2021

Weiler, M., Hamprecht, F.A., Storath, M.: Learning steerable filters for rotation equivariant CNNs. arXiv:1711.07289 [cs], March 2018

Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Harmonic networks: deep translation and rotation equivariance. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5028–5037 (2017)

Wu, L., et al.: Learning the implicit semantic representation on graph-structured data. In: Jensen, C.S., et al. (eds.) DASFAA 2021. LNCS, vol. 12681, pp. 3–19. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73194-6_1

Wu, Y., Mansimov, E., Grosse, R.B., Liao, S., Ba, J.: Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation. arXiv e-prints: arXiv:1708.05144 , August 2017

Xie, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

Xie, S., Kirillov, A., Girshick, R., He, K.: Exploring randomly wired neural networks for image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

Xiong, C., Zhao, X., Tang, D., Jayashree, K., Yan, S., Kim, T.K.: Conditional convolutional neural network for modality-aware face recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3667–3675 (2015)

Yalniz, I.Z., Jegou, H., Chen, K., Paluri, M., Mahajan, D.: Billion-scale semi-supervised learning for image classification. CoRR, abs/1905.00546 (2019)

Yang, S., Yu, X., Zhou, Y.: LSTM and GRU neural network performance comparison study: taking yelp review dataset as an example. In: 2020 International Workshop on Electronic Communication and Artificial Intelligence (IWECAI), pp. 98–101 (2020)

Yao, B., Walther, D., Beck, D., Fei-Fei, L.: Hierarchical mixture of classification experts uncovers interactions between brain regions. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C., Culotta, A. (eds.) Advances in Neural Information Processing Systems, vol. 22. Curran Associates, Inc., (2009)

Ying, C., Klein, A., Christiansen, E., Real, E., Murphy, K., Hutter, F.: NAS-bench-101: towards reproducible neural architecture search. In: International Conference on Machine Learning, pp. 7105–7114. PMLR (2019)

Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? arXiv:1411.1792 [cs], November 2014

Yu, J., Yang, L., Xu, N., Yang, J., Huang, T.L.: Slimmable neural networks. In: International Conference on Learning Representations (2019)

Yu, L., et al.: MAttNet: modular attention network for referring expression comprehension. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1307–1315 (2018)

Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., Finn, C.: Gradient surgery for multi-task learning. In: Advances in Neural Information Processing Systems, vol. 33, pp. 5824–5836. Curran Associates, Inc., (2020)

Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint: arXiv:1605.07146 (2016)

Zaidi, J., Boilard, J., Gagnon, G., Carbonneau, M.-A.: Measuring disentanglement: a review of metrics. arXiv:2012.09276 [cs], January 2021

Zhang, Q., Yang, Y., Yu, Q., Wu, Y.N.: Network transplanting. arXiv:1804.10272 [cs, stat], December 2018

Zhang, Y., Yang, Q.: A survey on multi-task learning. IEEE Trans. Knowl. Data Eng. (2021)

Zhou, A., Knowles, T., Finn, C.: Meta-learning symmetries by reparameterization. arXiv:2007.02933 [cs, stat], October 2020

Zhou, T., Wang, S., Bilmes, J.A.: Diverse ensemble evolution: curriculum data-model marriage. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc., (2018)

Zhou, Z.-H.: Ensemble Methods: Foundations and Algorithms. CRC Press, Boca Raton (2012)

Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Computer Vision (ICCV), 2017 IEEE International Conference On (2017)

Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. arXiv:1707.07012 [cs, stat], April 2018

Download references

Acknowledgments

We gratefully acknowledge constructive feedback and suggestions from Birhanu Hailu Belay, Romain Egele, Felix Mohr, Hedi Tabia, and the reviewers. This work was supported by ChaLearn and the ANR (Agence Nationale de la Recherche, National Agency for Research) under AI chair of excellence HUMANIA, grant number ANR-19-CHIA-0022.

Author information

Authors and affiliations.

LISN/CNRS/INRIA, Université Paris-Saclay, Gif-sur-Yvette, France

Haozhe Sun & Isabelle Guyon

ChaLearn, Berkeley, USA

Isabelle Guyon

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Haozhe Sun .

Editor information

Editors and affiliations.

Faculty of Science and Engineering, Saga University, Saga, Japan

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Sun, H., Guyon, I. (2023). Modularity in Deep Learning: A Survey. In: Arai, K. (eds) Intelligent Computing. SAI 2023. Lecture Notes in Networks and Systems, vol 739. Springer, Cham. https://doi.org/10.1007/978-3-031-37963-5_40

Download citation

DOI : https://doi.org/10.1007/978-3-031-37963-5_40

Published : 20 August 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-37962-8

Online ISBN : 978-3-031-37963-5

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research