literature review website development

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
HHS Author Manuscripts

A Literature Review: Website Design and User Engagement

Renee garett.

1 ElevateU, Los Angeles, CA, USA

Sean D. Young

2 University of California Institute for Prediction Technology, Department of Family Medicine, University of California, Los Angeles, Los Angeles, CA, USA

3 UCLA Center for Digital Behavior, Department of Family Medicine, University of California, Los Angeles, Los Angeles, CA, USA

Proper design has become a critical element needed to engage website and mobile application users. However, little research has been conducted to define the specific elements used in effective website and mobile application design. We attempt to review and consolidate research on effective design and to define a short list of elements frequently used in research. The design elements mentioned most frequently in the reviewed literature were navigation, graphical representation, organization, content utility, purpose, simplicity, and readability. We discuss how previous studies define and evaluate these seven elements. This review and the resulting short list of design elements may be used to help designers and researchers to operationalize best practices for facilitating and predicting user engagement.

1. INTRODUCTION

Internet usage has increased tremendously and rapidly in the past decade ( “Internet Use Over Time,” 2014 ). Websites have become the most important public communication portal for most, if not all, businesses and organizations. As of 2014, 87% of American adults aged 18 or older are Internet users ( “Internet User Demographics,” 2013 ). Because business-to-consumer interactions mainly occur online, website design is critical in engaging users ( Flavián, Guinalíu, & Gurrea, 2006 ; Lee & Kozar, 2012 ; Petre, Minocha, & Roberts, 2006 ). Poorly designed websites may frustrate users and result in a high “bounce rate”, or people visiting the entrance page without exploring other pages within the site ( Google.com, 2015 ). On the other hand, a well-designed website with high usability has been found to positively influence visitor retention (revisit rates) and purchasing behavior ( Avouris, Tselios, Fidas, & Papachristos, 2003 ; Flavián et al., 2006 ; Lee & Kozar, 2012 ).

Little research, however, has been conducted to define the specific elements that constitute effective website design. One of the key design measures is usability ( International Standardization Organization, 1998 ). The International Standardized Organization (ISO) defines usability as the extent to which users can achieve desired tasks (e.g., access desired information or place a purchase) with effectiveness (completeness and accuracy of the task), efficiency (time spent on the task), and satisfaction (user experience) within a system. However, there is currently no consensus on how to properly operationalize and assess website usability ( Lee & Kozar, 2012 ). For example, Nielson associates usability with learnability, efficiency, memorability, errors, and satisfaction ( Nielsen, 2012 ). Yet, Palmer (2002) postulates that usability is determined by download time, navigation, content, interactivity, and responsiveness. Similar to usability, many other key design elements, such as scannability, readability, and visual aesthetics, have not yet been clearly defined ( Bevan, 1997 ; Brady & Phillips, 2003 ; Kim, Lee, Han, & Lee, 2002 ), and there are no clear guidelines that individuals can follow when designing websites to increase engagement.

This review sought to address that question by identifying and consolidating the key website design elements that influence user engagement according to prior research studies. This review aimed to determine the website design elements that are most commonly shown or suggested to increase user engagement. Based on these findings, we listed and defined a short list of website design elements that best facilitate and predict user engagement. The work is thus an exploratory research providing definitions for these elements of website design and a starting point for future research to reference.

2. MATERIALS AND METHODS

2.1. selection criteria and data extraction.

We searched for articles relating to website design on Google Scholar (scholar.google.com) because Google Scholar consolidates papers across research databases (e.g., Pubmed) and research on design is listed in multiple databases. We used the following combination of keywords: design, usability, and websites. Google Scholar yielded 115,000 total hits. However, due to the large list of studies generated, we decided to only review the top 100 listed research studies for this exploratory study. Our inclusion criteria for the studies was: (1) publication in a peer-reviewed academic journal, (2) publication in English, and (3) publication in or after 2000. Year of publication was chosen as a limiting factor so that we would have enough years of research to identify relevant studies but also have results that relate to similar styles of websites after the year 2000. We included studies that were experimental or theoretical (review papers and commentaries) in nature. Resulting studies represented a diverse range of disciplines, including human-computer interaction, marketing, e-commerce, interface design, cognitive science, and library science. Based on these selection criteria, thirty-five unique studies remained and were included in this review.

2.2. Final Search Term

(design) and (usability) and (websites).

The search terms were kept simple to capture the higher level design/usability papers and allow Google scholar’s ranking method to filter out the most popular studies. This method also allowed studies from a large range of fields to be searched.

2.3. Analysis

The literature review uncovered 20 distinct design elements commonly discussed in research that affect user engagement. They were (1) organization – is the website logically organized, (2) content utility – is the information provided useful or interesting, (3) navigation – is the website easy to navigate, (4) graphical representation – does the website utilize icons, contrasting colors, and multimedia content, (5) purpose – does the website clearly state its purpose (i.e. personal, commercial, or educational), (6) memorable elements – does the website facilitate returning users to navigate the site effectively (e.g., through layout or graphics), (7) valid links – does the website provide valid links, (8) simplicity – is the design of the website simple, (9) impartiality – is the information provided fair and objective, (10) credibility – is the information provided credible, (11) consistency/reliability – is the website consistently designed (i.e., no changes in page layout throughout the site), (12) accuracy – is the information accurate, (13) loading speed – does the website take a long time to load, (14) security/privacy – does the website securely transmit, store, and display personal information/data, (15) interactive – can the user interact with the website (e.g., post comments or receive recommendations for similar purchases), (16) strong user control capabilities– does the website allow individuals to customize their experiences (such as the order of information they access and speed at which they browse the website), (17) readability – is the website easy to read and understand (e.g., no grammatical/spelling errors), (18) efficiency – is the information presented in a way that users can find the information they need quickly, (19) scannability – can users pick out relevant information quickly, and (20) learnability – how steep is the learning curve for using the website. For each of the above, we calculated the proportion of studies mentioning the element. In this review, we provide a threshold value of 30%. We identified elements that were used in at least 30% of the studies and include these elements that are above the threshold on a short list of elements used in research on proper website design. The 30% value was an arbitrary threshold picked that would provide researchers and designers with a guideline list of elements described in research on effective web design. To provide further information on how to apply this list, we present specific details on how each of these elements was discussed in research so that it can be defined and operationalized.

3.1. Popular website design elements ( Table 1 )

Frequency of website design elements used in research (2000–2014)

Seven of the website design elements met our threshold requirement for review. Navigation was the most frequently discussed element, mentioned in 22 articles (62.86%). Twenty-one studies (60%) highlighted the importance of graphics. Fifteen studies (42.86%) emphasized good organization. Four other elements also exceeded the threshold level, and they were content utility (n=13, 37.14%), purpose (n=11, 31.43%), simplicity (n=11, 31.43%), and readability (n=11, 31.43%).

Elements below our minimum requirement for review include memorable features (n=5, 14.29%), links (n=10, 28.57%), impartiality (n=1, 2.86%), credibility (n=7, 20%), consistency/reliability (n=8. 22.86%), accuracy (n=5, 14.29%), loading speed (n=10, 28.57%), security/privacy (n=2, 5.71%), interactive features (n=9, 25.71%), strong user control capabilities (n=8, 22.86%), efficiency (n=6, 17.14%), scannability (n=1, 2.86%), and learnability (n=2, 5.71%).

3.2. Defining key design elements for user engagement ( Table 2 )

Definitions of Key Design Elements

In defining and operationalizing each of these elements, the research studies suggested that effective navigation is the presence of salient and consistent menu/navigation bars, aids for navigation (e.g., visible links), search features, and easy access to pages (multiple pathways and limited clicks/backtracking). Engaging graphical presentation entails 1) inclusion of images, 2) proper size and resolution of images, 3) multimedia content, 4) proper color, font, and size of text, 5) use of logos and icons, 6) attractive visual layout, 7) color schemes, and 8) effective use of white space. Optimal organization includes 1) cognitive architecture, 2) logical, understandable, and hierarchical structure, 3) information arrangement and categorization, 4) meaningful labels/headings/titles, and 5) use of keywords. Content utility is determined by 1) sufficient amount of information to attract repeat visitors, 2) arousal/motivation (keeps visitors interested and motivates users to continue exploring the site), 3) content quality, 4) information relevant to the purpose of the site, and 5) perceived utility based on user needs/requirements. The purpose of a website is clear when it 1) establishes a unique and visible brand/identity, 2) addresses visitors’ intended purpose and expectations for visiting the site, and 3) provides information about the organization and/or services. Simplicity is achieved by using 1) simple subject headings, 2) transparency of information (reduce search time), 3) website design optimized for computer screens, 4) uncluttered layout, 5) consistency in design throughout website, 6) ease of using (including first-time users), 7) minimize redundant features, and 8) easily understandable functions. Readability is optimized by content that is 1) easy to read, 2) well-written, 3) grammatically correct, 4) understandable, 5) presented in readable blocks, and 6) reading level appropriate.

4. DISCUSSION

The seven website design elements most often discussed in relation to user engagement in the reviewed studies were navigation (62.86%), graphical representation (60%), organization (42.86%), content utility (37.14%), purpose (31.43%), simplicity (31.43%), and readability (31.43%). These seven elements exceeded our threshold level of 30% representation in the literature and were included into a short list of website design elements to operationalize effective website design. For further analysis, we reviewed how studies defined and evaluated these seven elements. This may allow designers and researchers to determine and follow best practices for facilitating or predicting user engagement.

A remaining challenge is that the definitions of website design elements often overlap. For example, several studies evaluated organization by how well a website incorporates cognitive architecture, logical and hierarchical structure, systematic information arrangement and categorization, meaningful headings and labels, and keywords. However, these features are also crucial in navigation design. Also, the implications of using distinct logos and icons go beyond graphical representation. Logos and icons also establish unique brand/identity for the organization (purpose) and can serve as visual aids for navigation. Future studies are needed to develop distinct and objective measures to assess these elements and how they affect user engagement ( Lee & Kozar, 2012 ).

Given the rapid increase in both mobile technology and social media use, it is surprising that no studies mentioned cross-platform compatibility and social media integration. In 2013, 34% of cellphone owners primarily use their cellphones to access the Internet, and this number continues to grow ( “Mobile Technology Factsheet,” 2013 ). With the rise of different mobile devices, users are also diversifying their web browser use. Internet Explorer (IE) was once the leading web browser. However, in recent years, FireFox, Safari, and Chrome have gained significant traction ( W3schools.com, 2015 ). Website designers and researchers must be mindful of different platforms and browsers to minimize the risk of losing users due to compatibility issues. In addition, roughly 74% of American Internet users use some form of social media ( Duggan, Ellison, Lampe, Lenhart, & Smith, 2015 ), and social media has emerged as an effective platform for organizations to target and interact with users. Integrating social media into website design may increase user engagement by facilitating participation and interactivity.

There are several limitations to the current review. First, due to the large number of studies published in this area and due to this study being exploratory, we selected from the first 100 research publications on Google Scholar search results. Future studies may benefit from defining design to a specific topic, set of years, or other area to limit the number of search results. Second, we did not quantitatively evaluate the effectiveness of these website design elements. Additional research can help to better quantify these elements.

It should also be noted that different disciplines and industries have different objectives in designing websites and should thus prioritize different website design elements. For example, online businesses and marketers seek to design websites that optimize brand loyalty, purchase, and profit ( Petre et al., 2006 ). Others, such as academic researchers or healthcare providers, are more likely to prioritize privacy/confidentiality, and content accuracy in building websites ( Horvath, Ecklund, Hunt, Nelson, & Toomey, 2015 ). Ultimately, we advise website designers and researchers to consider the design elements delineated in this review, along with their unique needs, when developing user engagement strategies.

Arroyo Ernesto, Selker Ted, Wei Willy. Usability tool for analysis of web designs using mouse tracks. Paper presented at the CHI’06 Extended Abstracts on Human Factors in Computing Systems.2006. [ Google Scholar ]
Atterer Richard, Wnuk Monika, Schmidt Albrecht. Knowing the user’s every move: user activity tracking for website usability evaluation and implicit interaction. Paper presented at the Proceedings of the 15th international conference on World Wide Web.2006. [ Google Scholar ]
Auger Pat. The impact of interactivity and design sophistication on the performance of commercial websites for small businesses. Journal of Small Business Management. 2005; 43 (2):119–137. [ Google Scholar ]
Avouris Nikolaos, Tselios Nikolaos, Fidas Christos, Papachristos Eleftherios. Advances in Informatics. Springer; 2003. Website evaluation: A usability-based perspective; pp. 217–231. [ Google Scholar ]
Banati Hema, Bedi Punam, Grover PS. Evaluating web usability from the user’s perspective. Journal of Computer Science. 2006; 2 (4):314. [ Google Scholar ]
Belanche Daniel, Casaló Luis V, Guinalíu Miguel. Website usability, consumer satisfaction and the intention to use a website: The moderating effect of perceived risk. Journal of retailing and consumer services. 2012; 19 (1):124–132. [ Google Scholar ]
Bevan Nigel. Usability issues in web site design. Paper presented at the HCI; 1997. [ Google Scholar ]
Blackmon Marilyn Hughes, Kitajima Muneo, Polson Peter G. Repairing usability problems identified by the cognitive walkthrough for the web. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems.2003. [ Google Scholar ]
Blackmon Marilyn Hughes, Polson Peter G, Kitajima Muneo, Lewis Clayton. Cognitive walkthrough for the web. Paper presented at the Proceedings of the SIGCHI conference on human factors in computing systems.2002. [ Google Scholar ]
Braddy Phillip W, Meade Adam W, Kroustalis Christina M. Online recruiting: The effects of organizational familiarity, website usability, and website attractiveness on viewers’ impressions of organizations. Computers in Human Behavior. 2008; 24 (6):2992–3001. [ Google Scholar ]
Brady Laurie, Phillips Christine. Aesthetics and usability: A look at color and balance. Usability News. 2003; 5 (1) [ Google Scholar ]
Cyr Dianne, Head Milena, Larios Hector. Colour appeal in website design within and across cultures: A multi-method evaluation. International journal of human-computer studies. 2010; 68 (1):1–21. [ Google Scholar ]
Cyr Dianne, Ilsever Joe, Bonanni Carole, Bowes John. Website Design and Culture: An Empirical Investigation. Paper presented at the IWIPS.2004. [ Google Scholar ]
Dastidar Surajit Ghosh. Impact of the factors influencing website usability on user satisfaction. 2009. [ Google Scholar ]
De Angeli Antonella, Sutcliffe Alistair, Hartmann Jan. Interaction, usability and aesthetics: what influences users’ preferences?. Paper presented at the Proceedings of the 6th conference on Designing Interactive systems.2006. [ Google Scholar ]
Djamasbi Soussan, Siegel Marisa, Tullis Tom. Generation Y, web design, and eye tracking. International journal of human-computer studies. 2010; 68 (5):307–323. [ Google Scholar ]
Djonov Emilia. Website hierarchy and the interaction between content organization, webpage and navigation design: A systemic functional hypermedia discourse analysis perspective. Information Design Journal. 2007; 15 (2):144–162. [ Google Scholar ]
Duggan M, Ellison N, Lampe C, Lenhart A, Smith A. Social Media update 2014. Washington, D.C: Pew Research Center; 2015. [ Google Scholar ]
Flavián Carlos, Guinalíu Miguel, Gurrea Raquel. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Information & Management. 2006; 43 (1):1–14. [ Google Scholar ]
George Carole A. Usability testing and design of a library website: an iterative approach. OCLC Systems & Services: International digital library perspectives. 2005; 21 (3):167–180. [ Google Scholar ]
Google.com. Bounce Rate. Analyrics Help. 2015 Retrieved 2/11, 2015, from https://support.google.com/analytics/answer/1009409?hl=en .
Green D, Pearson JM. Development of a web site usability instrument based on ISO 9241-11. Journal of Computer Information Systems. 2006 Fall [ Google Scholar ]
Horvath Keith J, Ecklund Alexandra M, Hunt Shanda L, Nelson Toben F, Toomey Traci L. Developing Internet-Based Health Interventions: A Guide for Public Health Researchers and Practitioners. J Med Internet Res. 2015; 17 (1):e28. doi: 10.2196/jmir.3770. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
International Standardization Organization. ISO 2941-11:1998 Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability: International Standardization Organization (ISO) 1998. [ Google Scholar ]
Internet Use Over Time. 2014 Jan 2; Retrieved February 15, 2015, from http://www.pewinternet.org/data-trend/internet-use/internet-use-over-time/
Internet User Demographics. 2013 Nov 14; Retrieved February 11, 2015, from http://www.pewinternet.org/data-trend/internet-use/latest-stats/
Kim Jinwoo, Lee Jungwon, Han Kwanghee, Lee Moonkyu. Businesses as Buildings: Metrics for the Architectural Quality of Internet Businesses. Information Systems Research. 2002; 13 (3):239–254. doi: 10.1287/isre.13.3.239.79. [ CrossRef ] [ Google Scholar ]
Lee Younghwa, Kozar Kenneth A. Understanding of website usability: Specifying and measuring constructs and their relationships. Decision Support Systems. 2012; 52 (2):450–463. [ Google Scholar ]
Lim Sun. The Self-Confrontation Interview: Towards an Enhanced Understanding of Human Factors in Web-based Interaction for Improved Website Usability. J Electron Commerce Res. 2002; 3 (3):162–173. [ Google Scholar ]
Lowry Paul Benjamin, Spaulding Trent, Wells Taylor, Moody Greg, Moffit Kevin, Madariaga Sebastian. A theoretical model and empirical results linking website interactivity and usability satisfaction. Paper presented at the System Sciences, 2006. HICSS’06. Proceedings of the 39th Annual Hawaii International Conference on.2006. [ Google Scholar ]
Maurer Steven D, Liu Yuping. Developing effective e-recruiting websites: Insights for managers from marketers. Business Horizons. 2007; 50 (4):305–314. [ Google Scholar ]
Mobile Technology Fact Sheet. 2013 Dec 27; Retrieved August 5, 2015, from http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/
Nielsen Jakob. Usability 101: introduction to Usability. 2012 Retrieved 2/11, 2015, from http://www.nngroup.com/articles/usability-101-introduction-to-usability/
Palmer Jonathan W. Web Site Usability, Design, and Performance Metrics. Information Systems Research. 2002; 13 (2):151–167. doi: 10.1287/isre.13.2.151.88. [ CrossRef ] [ Google Scholar ]
Petre Marian, Minocha Shailey, Roberts Dave. Usability beyond the website: an empirically-grounded e-commerce evaluation instrument for the total customer experience. Behaviour & Information Technology. 2006; 25 (2):189–203. [ Google Scholar ]
Petrie Helen, Hamilton Fraser, King Neil. Tension, what tension?: Website accessibility and visual design. Paper presented at the Proceedings of the 2004 international cross-disciplinary workshop on Web accessibility (W4A).2004. [ Google Scholar ]
Raward Roslyn. Academic library website design principles: development of a checklist. Australian Academic & Research Libraries. 2001; 32 (2):123–136. [ Google Scholar ]
Rosen Deborah E, Purinton Elizabeth. Website design: Viewing the web as a cognitive landscape. Journal of Business Research. 2004; 57 (7):787–794. [ Google Scholar ]
Shneiderman Ben, Hochheiser Harry. Universal usability as a stimulus to advanced interface design. Behaviour & Information Technology. 2001; 20 (5):367–376. [ Google Scholar ]
Song Jaeki, Zahedi Fatemeh “Mariam”. A theoretical approach to web design in e-commerce: a belief reinforcement model. Management Science. 2005; 51 (8):1219–1235. [ Google Scholar ]
Sutcliffe Alistair. Interactive systems: design, specification, and verification. Springer; 2001. Heuristic evaluation of website attractiveness and usability; pp. 183–198. [ Google Scholar ]
Tan Gek Woo, Wei Kwok Kee. An empirical study of Web browsing behaviour: Towards an effective Website design. Electronic Commerce Research and Applications. 2007; 5 (4):261–271. [ Google Scholar ]
Tarafdar Monideepa, Zhang Jie. Determinants of reach and loyalty-a study of Website performance and implications for Website design. Journal of Computer Information Systems. 2008; 48 (2):16. [ Google Scholar ]
Thompson Lori Foster, Braddy Phillip W, Wuensch Karl L. E-recruitment and the benefits of organizational web appeal. Computers in Human Behavior. 2008; 24 (5):2384–2398. [ Google Scholar ]
W3schools.com. Browser Statistics and Trends. Retrieved 1/15, 2015, from http://www.w3schools.com/browsers/browsers_stats.asp .
Williamson Ian O, Lepak David P, King James. The effect of company recruitment web site orientation on individuals’ perceptions of organizational attractiveness. Journal of Vocational Behavior. 2003; 63 (2):242–263. [ Google Scholar ]
Zhang Ping, Small Ruth V, Von Dran Gisela M, Barcellos Silvia. A two factor theory for website design. Paper presented at the System Sciences, 2000. Proceedings of the 33rd Annual Hawaii International Conference on.2000. [ Google Scholar ]
Zhang Ping, Von Dran Gisela M. Satisfiers and dissatisfiers: A two-factor model for website design and evaluation. Journal of the American society for information science. 2000; 51 (14):1253–1268. [ Google Scholar ]

A Literature Review: Website Design and User Engagement

Affiliations.

1 ElevateU, Los Angeles, CA, USA.
2 University of California Institute for Prediction Technology, Department of Family Medicine, University of California, Los Angeles, Los Angeles, CA, USA; UCLA Center for Digital Behavior, Department of Family Medicine, University of California, Los Angeles, Los Angeles, CA, USA.
PMID: 27499833
PMCID: PMC4974011

Keywords: Website design; navigation; organization; simplicity; usability.

Grants and funding

K01 MH090884/MH/NIMH NIH HHS/United States
R01 MH106415/MH/NIMH NIH HHS/United States
R21 DA039458/DA/NIDA NIH HHS/United States

A Literature Review: Website Design and User Engagement.

Garett, Renee ;
Chiu, Jason ;
Zhang, Ly ;
Young, Sean D

Published Web Location

Many UC-authored scholarly publications are freely available on this site because of the UC's open access policies . Let us know how this access is important for you.

Enter the password to open this PDF file:

A Systematic Review of Adaptive and Responsive Design Approaches for World Wide Web

Conference paper
First Online: 27 December 2018
Cite this conference paper

Nazish Yousaf 17 ,
Wasi Haider Butt 17 ,
Farooque Azam 17 &
Muhammad Waseem Anwar 17

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 887))

Included in the following conference series:

Future of Information and Communication Conference

1102 Accesses

1 Citations

World Wide Web (WWW) in today’s age is used on different devices. These devices communicate with each other thus creating a design problem of layouts and data mismatch. Consequently, there is a strong need of visualizing and envisioning web data with proper customization suiting different types of users. The purpose of this Systematic Literature Review (SLR) is to investigate the two widely used web based designs i.e. Adaptive Web Design (AWD) and Responsive Web Design (RWD). Particularly, a systematic literature review has been used to identify 58 research works, published during 2009–2017. Consequently, we identify 23 research works regarding AWD, 14 research works related to RWD and 21 research works pertaining to both AWD and RWD. Moreover, 4 significant tools and 13 leading techniques have been identified in the context of AWD and RWD implementation. Finally, significant aspects of two web development approaches (AWD and RWD) are also compared with the traditional web design. It has been concluded that the traditional web design is not sufficient to fulfill the needs of ever-growing web users all around the globe. Therefore, the combination of both AWD and RWD is essential to meet the technological advancements in WWW.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Available as PDF
Read on any device
Instant download
Own it forever
Available as EPUB and PDF
Compact, lightweight edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

WEBAPIK: a body of structured knowledge on designing web APIs

Paving the Path to Content-Centric and Device-Agnostic Web Design

Quality of Web Mashups: A Systematic Mapping Study

Roberto, V.: AML: a modeling language for designing adaptive web applications. J. Pers. Ubiquit. Comput. 16 (5), 527–541 (2012)

Article MathSciNet Google Scholar

Stewart, C.B.: Making it work for everyone HTML5 and CSS Level3 for responsive accessible design on your library website. J. Libr. Inf. Serv. Distance Learn. 8 (3–4), 118–136 (2014)

Google Scholar

Changbo, K., Zhiqiu, H.: Self-adaptive semantic web service matching method. J. Knowl. Based Syst. 35 , 41–48 (2012)

Article Google Scholar

Jia-Jiunn, L., Ya-Chen, C., Shiou-Wen, Y.: Designing an adaptive web-based learning system based on students’ cognitive styles identified online. J. Comput. Educ. 59 (1), 209–222 (2012)

Zhenlong, L., Binghua C.: The design of adaptive web-learning system base on learning behavior monitoring. In: Proceedings of 7th International Conference on Computer Science and Education (ICCSE), Melbourne. IEEE (2012)

Manvi, A.D., Komal K.B.: Design of an ontology based adaptive crawler for hidden web. In: International Conference in Communication Systems and Network Technologies (CSNT 2013), Gwalior. IEEE (2013)

Carlos, G., Carlos, J., Ramon P.: Improving web cache performance via adaptive content fragmentation design. In: 10th IEEE International Symposium on Network Computing and Applications (NCA 2011), Cambridge. IEEE (2011)

Bin, Z.: Responsive design e-learning site transformation. In: 4th International Conference on Networking and Distributed Computing (ICNDC 2013), Los Angeles. IEEE (2013)

Bijan, B.G.: Culturally responsive educational web sites. J. Educ. Media Int. 37 (3), 185–195 (2010)

Luca, L.: Formalising human mental workload as non-monotonic concept for adaptive and personalised web-design. In: International Conference on User Modeling, Adaptation, and Personalization (UMAP) 2012. LNCS, vol. 7379, pp 369–373. Springer (2012)

Nancy, R.G., Phil, S.: One site fits all: responsive web design. J. Electron. Resour. Med. Libr. 11 (2), 78–90 (2014)

Ioannis, M., Stavros, D., Anastasios, K.: Adaptive and intelligent systems for collaborative learning support: a review of the field. IEEE Trans. Learn. Technol. 4 (1), 5–20 (2011)

Mohorovičić, S.: Implementing responsive web design for enhanced web presence. In: 36th International Convention on Information and Communication Technology Electronics and Microelectronics (MIPRO 2013), Opatija. IEEE (2013)

Jari-Pekka, V., Jaakko, S., Tommi M.: On the design of a responsive user interface for a multi-device web service. In: 2nd ACM International Conference on Mobile Software Engineering and Systems (MOBILEsoft 2015), Florence. ACM (2015)

Mohammed, S., Mohammed, R., Mostafa, B.: Towards minimizing author’s effort during designing the domain model of adaptive web-based educational system. In: 8th International Conference on Intelligent Systems: Theories and Applications (SITA 2013), Rabat. IEEE (2013)

Gu, W., Liu, J.: Web-based adaptive intelligent learning system design and implementation. In: International Conference on Information, Networking and Automation (ICINA), Kunming. IEEE (2010)

Aqueasha, M.H., Abdullah, A., Catherine, H., Amy K.H.: Understanding design considerations for adaptive user interfaces for accessible pointing with older and younger adults. In: Proceedings of the 12th Web for All Conference (W4A 2015), New York. ACM (2015)

Ramón, H., José, B.: Towards the ubiquitous visualization: adaptive user-interfaces based on the semantic web. J. Interact. Comput. 23 , 40–56 (2011)

Vlad, S., Yu, Y.: Experiences designing hypermedia-driven and self-adaptive web-based AR authoring tools. In: Proceedings of the Third International Workshop on RESTful Design (WS-REST 2012), New York. ACM (2012)

Federica, C., Antonina, D., Pasquale, L., Julita, V.: Perspectives in semantic adaptive social web. ACM Trans. Intell. Syst. Technol. (TIST) 4 (4), 59 (2013)

Bilal, L.B., Gede, P., Halizah, B.B.: ULUL-ILM the design of web-based adaptive educational hypermedia system based on learning style. In: 13th International Conference on Intelligent Systems Design and Applications (ISDA 2013), Bangi. IEEE (2013)

Liu, X., Han, H.: The design and realization of web-based adaptive learning system. In: 2nd International Conference on Networking and Digital Society (ICNDS 2010), Wenzhou. IEEE (2010)

Wei, J., Meng, Z., Bin, Z., Yujian, J., Yingwei, Z.: Responsive web design mode and application. In: IEEE Workshop on Advanced Research and Technology in Industry Applications (WARTIA 2014), Ottawa. IEEE (2014)

Jeonghyun, L., Imsu, L., Iseul, K., Hyejin, Y., Jongwon, L., Mansung, J., Hyenki, K.: Responsive web design according to the resolution. In: 8th International Conference on u- and e-Service, Science and Technology (UNESST 2015), Jeju. IEEE (2015)

Jacob, D., Bert, V., Tele, T.: Enhancing accessibility to web mapping systems with technology aligned adaptive profiles. Int. J. Dig. Earth 7 (4), 256–278 (2014)

Fabio, P., Christian, S.: Model-based customizable adaptation of web applications for vocal browsing. In: Proceedings of the 29th ACM International Conference on Design of Communication (SIGDOC 2011), pp. 83–90, Pisa. ACM (2011)

Anne, S.: Responsive web design for libraries A LITA guide. J. Hosp. Librariansh. 64 (2), 159–160 (2015)

Eva, H., Paul, K., Steven, L., Norbert, S.: Responsive Web Design. Elsevier, Amsterdam (2011)

Ezwan, S.A.M., Zulkefli, M., Norfadillah, K.: Adaptation of usability principles in responsive web design technique for e-commerce development. In: 5th International Conference on Electrical Engineering and Informatics (ICEEI 2015), Denpasar. IEEE (2015)

Wang, H., Liu, X.: Adaptive site design based on web mining and topology. In: World Congress on Computer Science and Information Engineering WRI 2009, Los Angeles. IEEE (2009)

Katerina, P., Stephanos, M.: Developing adaptive e-Learning systems based on Web 2.0. In: International Conference on Information Society (i-Society 2012), London. IEEE (2012)

Guo, L., Hua, Q.Y., Wang, P., Yu, K., Wang, K., Xu, J.: A model-based responsive web user interface. In: 6th International Conference on Software Engineering and Service Science (ICSESS 2015), Beijing. IEEE (2015)

Evgeny, K., Paul, D.B., Mykola, P.: AH 12 years later: a comprehensive survey of adaptive hypermedia methods and techniques. J. New Rev. Hypermed. Multimed. 15 (1), 5–38 (2009)

Yoann, B., Marianne, H., Michel, M.: Reconciling user and designer preferences in adapting web pages for people with low vision. In: Proceedings of the 12th Web for All Conference (W4A 2015), Florence. ACM (2015)

Kunyanuth, K., Pubet, K., Pattarapan, R.: Developing an adaptive web-based intelligent tutoring system using mastery learning technique. Procedia Soc. Behav. Sci. 191 , 686–691 (2015)

Quan, Z.S., Boualem, B., Zakaria, M., Anne, H.N.: Configurable composition and adaptive provisioning of web services. IEEE Trans. Serv. Comput. 2 (1), 34–49 (2009)

Jàn, A.: Beyond adaptive web design. IEEE (2014)

Bujar, R., Mexhid, F., Xhemal, Z., Jaumin, A., Florije, I.: Methods and techniques of adaptive web accessibility for the blind and visually impaired. Procedia Soc. Behav. Sci. 195 , 1999–2007 (2015)

Bo, M., Bin, C., Xiaoying, B., Junfei, H.: Design of BDI agent for adaptive performance testing of web services. In: Proceedings of 10th International Conference on Quality Software, Zhangjiajie. IEEE (2010)

Meltem, H.B., Murat, B.: Responsive web design-a new type of design for web-based instructional content. Procedia Soc. Behav. Sci. 106 , 2275–2279 (2013)

Jian, Y., Quan, Z.S., Joshua, K.Y.S., Han, J., Liu, C., Noor, T.N.: Model-driven development of adaptive web service processes with aspects and rules. J. Comput. Syst. Sci. 81 , 533–552 (2015)

Cristóbal, R., Sebastián, V., Amelia, Z., de Paul, B.: Applying web usage mining for personalizing hyperlinks in web-based adaptive educational systems. J. Comput. Educ. 53 , 828–840 (2009)

Nicolas, P., Toni, M., Josep, L.B., Ricard, G., Jordi, T.: Self-adaptive utility-based web session management. J. Comput. Netw. 53 , 1712–1721 (2009)

Evangelos, S., Antoniou, D., Poulia, A., Athanasios, T.: A web personalizing technique using adaptive data structures: the case of bursts in web visits. J. Syst. Softw. 83 (11), 2200–2210 (2010)

Quang-Dung, V., Danh-Viet, V., Ha-Thanh, N.P.B., Viet-Ha, N., Nobuyasu, N.: Adaptive web page layout for mobile devices. In: International Conference on Computing, Management and Telecommunications (ComManTel 2014), Da Nang. IEEE (2014)

Sowmya, S., Ravitheja, T., Joy, B.: Rendering on browsers responsive to user head. In: Annual India Conference (INDICON 2014), Pune. IEEE (2014)

Shengbo, C., Huaikou, M., Bo, S., Yihai, C.: Towards practical modeling of web applications and generating tests. In: IEEE 4th International Symposium on Theoretical Aspects of Software Engineering (TASE 2010), Taipei. IEEE (2010)

Sowmya, S., Ravitheja, T., Joy, B.: Responsive, adaptive and user personalized rendering on mobile browsers. In: International Conference on Advances in Computing, Communications and Informatics (ICACCI 2014), New Delhi. IEEE (2014)

Stephen, S.Y., Nong, Y., Hessam, S.S., Dazhi, H., Auttawut, R., Mustafa, G.B., Mohammed, A.M.: Toward development of adaptive service-based software systems. IEEE Trans. Serv. Comput. 2 (3), 247–260 (2009)

Tiago, C.N., Deller, J.F., Sérgio, T.C., Luciana, O.B.: Evaluating responsive web design’s impact on blind users. IEEE Multimedia 24 (2), 86–95 (2017)

Maryam, A.A., John, H.C., Carla, B.Z., Dimitrios, P.: An adaptive educational web application for engineering students. IEEE Access 5 , 359–365 (2017)

Wenhui, P., Yaling, Z.: The design and research of responsive web supporting mobile learning devices. In: International Symposium on Educational Technology (ISET 2015), Wuhan. IEEE (2015)

Yahya, M.T., Majd, S., Manar, F., Walaa, S., Mohammad, A.A.: Adaptive e-learning web-based English tutor using data mining techniques and Jackson’s learning styles. In: 8th International Conference on Information and Communication Systems (ICICS 2017), Irbid. IEEE (2017)

Yingzi, L., Jeffrey, B., Mura, I., David, S.: An adaptive interface design (AID) for enhanced computer accessibility and rehabilitation. Int. J. Hum. Comput. Stud. 98 (C), 14–23 (2017)

Xuan-zhe, L., Mengwei, X., Teng, T., Gang, H., Hong, M.: MUIT: a domain-specific language and its middleware for adaptive mobile web-based user interfaces in WS-BPEL. IEEE Trans. Serv. Comput. PP (99), 1 (2016)

Azlan, Y., Moamin, A.M., Mohd, S.A.: A conceptual multi-agent semantic web model of a self-adaptive website for intelligent strategic marketing in learning institutions. In: 2nd International Symposium on Agent, Multi-Agent Systems and Robotics (ISAMSR 2016), Bangi. IEEE (2016)

Tilman, D., Ted, S.: On the importance of spatial perception for the design of adaptive user interfaces. In: IEEE 10th International Conference on Self-Adaptive and Self-Organizing Systems (SASO 2016), Augsburg. IEEE (2016)

Kitchenham, B: Procedures for Performing Systematic Reviews, Keele University TR/SE-0401/NICTA, Technical Report 0400011T (2004)

Brusilovsky, P.: Adaptive Hypermedia (2001)

Search Process Search terms. http://ceme.nust.edu.pk/ISEGROUP/Resources/slra/slr.html

Download references

Author information

Authors and affiliations.

Department of Computer and Software Engineering, College of Electrical and Mechanical Engineering (EME), National University of Sciences and Technology (NUST), Islamabad, Pakistan

Nazish Yousaf, Wasi Haider Butt, Farooque Azam & Muhammad Waseem Anwar

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nazish Yousaf .

Editor information

Editors and affiliations.

Faculty of Science and Engineering, Saga University, Saga, Japan

The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

Supriya Kapoor

The Science and Information (SAI) Organization, New Delhi, India

Rahul Bhatia

Rights and permissions

Reprints and permissions

Copyright information

About this paper

Cite this paper.

Yousaf, N., Butt, W.H., Azam, F., Anwar, M.W. (2019). A Systematic Review of Adaptive and Responsive Design Approaches for World Wide Web. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Advances in Information and Communication Networks. FICC 2018. Advances in Intelligent Systems and Computing, vol 887. Springer, Cham. https://doi.org/10.1007/978-3-030-03405-4_50

Download citation

DOI : https://doi.org/10.1007/978-3-030-03405-4_50

Published : 27 December 2018

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-03404-7

Online ISBN : 978-3-030-03405-4

eBook Packages : Engineering Engineering (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

This paper is in the following e-collection/theme issue:

Published on 24.10.2019 in Vol 3 , No 4 (2019) : Oct-Dec

A Comprehensive Framework to Evaluate Websites: Literature Review and Development of GoodWeb

Authors of this article:

Rosalie Allison, BSc, MSc ;
Catherine Hayes, BSc ;
Cliodna A M McNulty, MBBS, FRCP ;
Vicki Young, BSc, PhD

Public Health England, Gloucester, United Kingdom

Corresponding Author:

Rosalie Allison, BSc, MSc

Public Health England

Primary Care and Interventions Unit

Gloucester, GL1 1DQ

United Kingdom

Phone: 44 0208 495 3258

Email: [email protected]

Background: Attention is turning toward increasing the quality of websites and quality evaluation to attract new users and retain existing users.

Objective: This scoping study aimed to review and define existing worldwide methodologies and techniques to evaluate websites and provide a framework of appropriate website attributes that could be applied to any future website evaluations.

Methods: We systematically searched electronic databases and gray literature for studies of website evaluation. The results were exported to EndNote software, duplicates were removed, and eligible studies were identified. The results have been presented in narrative form.

Results: A total of 69 studies met the inclusion criteria. The extracted data included type of website, aim or purpose of the study, study populations (users and experts), sample size, setting (controlled environment and remotely assessed), website attributes evaluated, process of methodology, and process of analysis. Methods of evaluation varied and included questionnaires, observed website browsing, interviews or focus groups, and Web usage analysis. Evaluations using both users and experts and controlled and remote settings are represented. Website attributes that were examined included usability or ease of use, content, design criteria, functionality, appearance, interactivity, satisfaction, and loyalty. Website evaluation methods should be tailored to the needs of specific websites and individual aims of evaluations. GoodWeb, a website evaluation guide, has been presented with a case scenario.

Conclusions: This scoping study supports the open debate of defining the quality of websites, and there are numerous approaches and models to evaluate it. However, as this study provides a framework of the existing literature of website evaluation, it presents a guide of options for evaluating websites, including which attributes to analyze and options for appropriate methods.

Introduction

Since its conception in the early 1990s, there has been an explosion in the use of the internet, with websites taking a central role in diverse ﬁelds such as ﬁnance, education, medicine, industry, and business. Organizations are increasingly attempting to exploit the benefits of the World Wide Web and its features as an interface for internet-enabled businesses, information provision, and promotional activities [ 1 , 2 ]. As the environment becomes more competitive and websites become more sophisticated, attention is turning toward increasing the quality of the website itself and quality evaluation to attract new and retain existing users [ 3 , 4 ]. What determines website quality has not been conclusively established, and there are many different deﬁnitions and meanings of the term quality, mainly in relation to the website’s purpose [ 5 ]. Traditionally, website evaluations have focused on usability, deﬁned as “the extent to which a product can be used by speciﬁed users to achieve speciﬁed goals with effectiveness, efficiency and satisfaction in a speciﬁed context of use [ 6 ].” The design of websites and users’ needs go beyond pure usability, as increased engagement and pleasure experienced during interactions with websites can be more important predictors of website preference than usability [ 7 - 10 ]. Therefore, in the last decade, website evaluations have shifted their focus to users’ experience, employing various assessment techniques [ 11 ], with no universally accepted method or procedure for website evaluation.

This scoping study aimed to review and define existing worldwide methodologies and techniques to evaluate websites and provide a simple framework of appropriate website attributes, which could be applied to future website evaluations.

A scoping study is similar to a systematic review as it collects and reviews content in a field of interest. However, scoping studies cover a broader question and do not rigorously evaluate the quality of the studies included [ 12 ]. Scoping studies are commonly used in the fields of public services such as health and education, as they are more rapid to perform and less costly in terms of staff costs [ 13 ]. Scoping studies can be precursors to a systematic review or stand-alone studies to examine the range of research around a particular topic.

The following research question is based on the need to gain knowledge and insight from worldwide website evaluation to inform the future study design of website evaluations: what website evaluation methodologies can be robustly used to assess users’ experience?

To show how the framework of attributes and methods can be applied to evaluating a website, e-Bug, an international educational health website, will be used as a case scenario [ 14 ].

This scoping study followed a 5-stage framework and methodology, as outlined by Arksey and O’Malley [ 12 ], involving the following: (1) identifying the research question, as above; (2) identifying relevant studies; (3) study selection; (4) charting the data; and (5) collating, summarizing, and reporting the results.

Identifying Relevant Studies

Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [ 15 ], studies for consideration in the review were located by searching the following electronic databases: Excerpta Medica dataBASE, PsycINFO, Cochrane, Cumulative Index to Nursing and Allied Health Literature, Scopus, ACM digital library, and IEEE Xplore SPORTDiscus. The keywords used referred to the following:

Population: websites
Intervention: evaluation methodologies
Outcome: user’s experience.

Table 1 shows the specific search criteria for each database. These keywords were also used to search gray literature for unpublished or working documents to minimize publication bias.

a EMBASE: Excerpta Medica database.

b CINAHL: Cumulative Index to Nursing and Allied Health Literature.

c ACM: Association for Computing Machinery.

d IEEE: Institute of Electrical and Electronics Engineers.

Study Selection

Once all sources had been systematically searched, the list of citations was exported to EndNote software to identify eligible studies. By scanning the title, and abstract if necessary, studies that did not fit the inclusion criteria were removed by 2 researchers (RA and CH). As abstracts are not always representative of the full study that follows or capture the full scope [ 16 ], if the title and abstract did not provide sufficient information, the full manuscript was examined to ascertain whether they met all the inclusion criteria, which included (1) studies focused on websites, (2) studies of evaluative methods (eg, use of questionnaire and task completion), (3) studies that reported outcomes that affect the user’s experience (eg, quality, satisfaction, efficiency, effectiveness without necessarily focusing on methodology), (4) studies carried out between 2006 and 2016, (5) studies published in English, and (6) type of study (any study design that is appropriate).

Exclusion criteria included (1) studies that focus on evaluations using solely experts and are not transferrable to user evaluations; (2) studies that are in the form of electronic book or are not freely available on the Web or through OpenAthens, the University of Bath library, or the University of the West of England library; (3)studies that evaluate banking, electronic commerce (e-commerce), or online libraries’ websites and do not have transferrable measures to a range of other websites; (4) studies that report exclusively on minority or special needs groups (eg, blind or deaf users); and (5) studies that do not meet all the inclusion criteria.

Charting the Data

The next stage involved charting key items of information obtained from studies being reviewed. Charting [ 17 ] describes a technique for synthesizing and interpreting qualitative data by sifting, charting, and sorting material according to key issues and themes. This is similar to a systematic review in which the process is called data extraction. The data extracted included general information about the study and specific information relating to, for instance, the study population or target, the type of intervention, outcome measures employed, and the study design.

The information of interest included the following: type of website, aim or purpose of the study, study populations (users and experts), sample size, setting (laboratory, real life, and remotely assessed), website attributes evaluated, process of methodology, and process of analysis.

NVivo version 10.0 software was used for this stage by 2 researchers (RA and CH) to chart the data.

Collating, Summarizing, and Reporting the Results

Although the scoping study does not seek to assess the quality of evidence, it does present an overview of all material reviewed with a narrative account of findings.

Ethics Approval and Consent to Participate

As no primary research was carried out, no ethical approval was required to undertake this scoping study. No specific reference was made to any of the participants in the individual studies, nor does this study infringe on their rights in any way.

The electronic database searches produced 6657 papers; a further 7 papers were identified through other sources. After removing duplicates (n=1058), 5606 publications remained. After titles and abstracts were examined, 784 full-text papers were read and assessed further for eligibility. Of those, 69 articles were identified as suitable by meeting all the inclusion criteria ( Figure 1 ).

Study Characteristics

Studies referred to or used a mixture of users (72%) and experts (39%) to evaluate their websites; 54% used a controlled environment, and 26% evaluated websites remotely ( Multimedia Appendix 1 [ 2 - 4 , 11 , 18 - 85 ]). Remote usability, in its most basic form, involves working with participants who are not in the same physical location as the researcher, employing techniques such as live screen sharing or questionnaires. Advantages to remote website evaluations include the ability to evaluate using a larger number of participants as travel time and costs are not a factor, and participants are able to partake at a time that is appropriate to them, increasing the likelihood of participation and the possibility of a greater diversity of participants [ 18 ]. However, the disadvantages of remote website evaluations, in comparison with a controlled setting, are that system performance, network traffic, and the participant’s computer setup can all affect the results.

A variety of types of websites evaluated were included in this review including government (9%), online news (6%), education (1%), university (12%), and sports organizations (4%). The aspects of quality considered, and their relative importance varied according to the type of website and the goals to be achieved by the users. For example, criteria such as ease of paying or security are not very important to educational websites, whereas they are especially important for online shopping. In this sense, much attention must be paid when evaluating the quality of a website, establishing a speciﬁc context of use and purpose [ 19 ].

The context of the participants was also discussed, in relation to the generalizability of results. For example, when evaluations used potential or current users of their website, it was important that computer literacy was reflective of all users [ 20 ]. This could mean ensuring that participants with a range of computer abilities and experiences were used so that results were not biased to the most or least experienced users.

Intervention

A total of 43 evaluation methodologies were identified in the 69 studies in this review. Most of them were variations of similar methodologies, and a brief description of each is provided in Multimedia Appendix 2 . Multimedia Appendix 3 shows the methods used or described in each study.

Questionnaire

Use of questionnaires was the most common methodology referred to (37/69, 54%), including questions to rank or rate attributes and open questions to allow text feedback and suggested improvements. Questionnaires were used in a combination of before or after usability testing to assess usability and overall user experience.

Observed Browsing the Website

Browsing the website using a form of task completion with the participant, such as cognitive walkthrough, was used in 33/69 studies (48%), whereby an expert evaluator used a detailed procedure to simulate task execution and browse all particular solution paths, examining each action while determining if expected user’s goals and memory content would lead to choosing a correct option [ 30 ]. Screen capture was often used (n=6) to record participants’ navigation through the website, and eye tracking was used (n=7) to assess where the eye focuses on each page or the motion of the eye as an individual views a Web page. The think-aloud protocol was used (n=10) to encourage users to express out loud what they were looking at, thinking, doing, and feeling, as they performed tasks. This allows observers to see and understand the cognitive processes associated with task completion. Recording the time to complete tasks (n=6) and mouse movement or clicks (n=8) were used to assess the efficiency of the websites.

Qualitative Data Collection

Several forms of qualitative data collection were used in 27/69 studies (39%). Observed browsing, interviews, and focus groups were used either before or after the use of the website. Pre-website-use, qualitative research was often used to collect details of which website attributes were important for participants or what weighting participants would give to each attribute. Postevaluation, qualitative techniques were used to collate feedback on the quality of the website and any suggestions for improvements.

Automated Usability Evaluation Software

In 9/69 studies (13%), automated usability evaluation focused on developing software, tools, and techniques to speed evaluation (rapid), tools that reach a wider audience for usability testing (remote), and tools that have built-in analyses features (automated). The latter can involve assessing server logs, website coding, and simulations of user experience to assess usability [ 42 ].

Card Sorting

A technique that is often linked with assessing navigability of a website, card sorting, is useful for discovering the logical structure of an unsorted list of statements or ideas by exploring how people group items and structures that maximize the probability of users finding items (5/69 studies, 7%). This can assist with determining effective website structure.

Web Usage Analysis

Of 69 studies, 3 studies used Web usage analysis or Web analytics to identify browsing patterns by analyzing the participants’ navigational behavior. This could include tracking at the widget level, that is, combining knowledge of the mouse coordinates with elements such as buttons and links, with the layout of the HTML pages, enabling complete tracking of all user activity.

Outcomes (Attributes Used to Evaluate Websites)

Often, different terminology for website attributes was used to describe the same or similar concepts ( Multimedia Appendix 4 ). The most used website attributes that were assessed can be broken down into 8 broad categories and further subcategories:

Usability or ease of use is the degree to which a website can be used to achieve given goals (n=58). It includes navigation such as intuitiveness, learnability, memorability, and information architecture; effectiveness such as errors; and efficiency.
Content (n=41) includes completeness, accuracy, relevancy, timeliness, and understandability of the information.
Web design criteria (n=29) include use of media, search engines, help resources, originality of the website, site map, user interface, multilanguage, and maintainability.
Functionality (n=31) includes links, website speed, security, and compatibility with devices and browsers.
Appearance (n=26) includes layout, font, colors, and page length.
Interactivity (n=25) includes sense of community, such as ability to leave feedback and comments and email or share with a friend option or forum discussion boards; personalization; help options such as frequently answered questions or customer services; and background music.
Satisfaction (n=26) includes usefulness, entertainment, look and feel, and pleasure.
Loyalty (n=8) includes first impression of the website.

GoodWeb: Website Evaluation Guide

As there was such a range of methods used, a suggested guide of options for evaluating websites is presented below ( Figure 2 ), coined GoodWeb, and applied to an evaluation of e-Bug, an international educational health website [ 14 ]. Allison at al [ 86 ] show the full details of how GoodWeb has been applied and outcomes of the e-Bug website evaluation.

Step 1. What Are the Important Website Attributes That Affect User's Experience of the Chosen Website?

Usability or ease of use, content, Web design criteria, functionality, appearance, interactivity, satisfaction, and loyalty were the umbrella terms that encompassed the website attributes identified or evaluated in the 69 studies in this scoping study. Multimedia Appendix 4 contains a summary of the most used website attributes that have been assessed. Recent website evaluations have shifted focus from usability of websites to an overall user’s experience of website use. A decision on which website attributes to evaluate for specific websites could come from interviews or focus groups with users or experts or a literature search of attributes used in similar evaluations.

Application

In the scenario of evaluating e-Bug or similar educational health websites, the attributes chosen to assess could be the following:

Appearance: colors, fonts, media or graphics, page length, style consistency, and first impression
Content: clarity, completeness, current and timely information, relevance, reliability, and uniqueness
Interactivity: sense of community and modern features
Ease of use: home page indication, navigation, guidance, and multilanguage support
Technical adequacy: compatibility with other devices, load time, valid links, and limited use of special plug-ins
Satisfaction: loyalty

These cover the main website attributes appropriate for an educational health website. If the website did not currently have features such as search engines, site map, background music, it may not be appropriate to evaluate these, but may be better suited to question whether they would be suitable additions to the website; or these could be combined under the heading modern features . Furthermore, security may not be a necessary attribute to evaluate if participant identifiable information or bank details are not needed to use the website.

Step 2. What Is the Best Way to Evaluate These Attributes?

Often, a combination of methods is suitable to evaluate a website, as 1 method may not be appropriate to assess all attributes of interest [ 29 ] (see Multimedia Appendix 3 for a summary of the most used methods for evaluating websites). For example, screen capture of task completion may be appropriate to assess the efficiency of a website but would not be the chosen method to assess loyalty. A questionnaire or qualitative interview may be more appropriate for this attribute.

In the scenario of evaluating e-Bug, a questionnaire before browsing the website would be appropriate to rank the importance of the selected website attributes, chosen in step 1. It would then be appropriate to observe browsing of the website, collecting data on completion of typical task scenarios, using the screen capture function for future reference. This method could be used to evaluate the effectiveness (number of tasks successfully completed), efficiency (whether the most direct route through the website was used to complete the task), and learnability (whether task completion is more efficient or effective second time of trying). It may then be suitable to use a follow-up questionnaire to rate e-Bug against the website attributes previously ranked. The attribute ranking and rating could then be combined to indicate where the website performs well and areas for improvement.

Step 3: Who Should Evaluate the Website?

Both users and experts can be used to evaluate websites. Experts are able to identify areas for improvements, in relation to usability; whereas, users are able to appraise quality as well as identify areas for improvement. In this respect, users are able to fully evaluate user’s experience, where experts may not be able to.

For this reason, it may be more appropriate to use current or potential users of the website for the scenario of evaluating e-Bug.

Step 4: What Setting Should Be Used?

A combination of controlled and remote settings can be used, depending on the methods chosen. For example, it may be appropriate to collect data via a questionnaire, remotely, to increase sample size and reach a more diverse audience, whereas a controlled setting may be more appropriate for task completion using eye-tracking methods.

Strengths and Limitations

A scoping study differs from a systematic review, in that it does not critically appraise the quality of the studies before extracting or charting the data. Therefore, this study cannot compare the effectiveness of the different methods or methodologies in evaluating the website attributes. However, what it does do is review and summarize a huge amount of literature, from different sources, in a format that is understandable and informative for future designs of website evaluations.

Furthermore, studies that evaluate banking, e-commerce, or online libraries’ websites and do not have transferrable measures to a range of other websites were excluded from this study. This decision was made to limit the number of studies that met the remaining inclusion criteria, and it was deemed that the website attributes for these websites would be too specialist and not necessarily transferable to a range of websites. Therefore, the findings of this study may not be generalizable to all types of website. However, Multimedia Appendix 1 shows that data were extracted from a very broad range of websites when it was deemed that the information was transferrable to a range of other websites.

A robust website evaluation can identify areas for improvement to both fulfill the goals and desires of its users [ 62 ] and influence their perception of the organization and overall quality of resources [ 48 ]. An improved website could attract and retain more online users; therefore, an evidence-based website evaluation guide is essential.

Conclusions

This scoping study emphasizes the fact that the debate about how to define the quality of websites remains open, and there are numerous approaches and models to evaluate it. Multimedia Appendix 2 shows existing methodologies or tools that can be used to evaluate websites. Many of these are variations of similar approaches; therefore, it is not strictly necessary to use these tools at face value; however, some could be used to guide analysis, following data collection. By following steps 1 to 4 of GoodWeb, the framework suggested in this study, taking into account the desired participants and setting and website evaluation methods, can be tailored to the needs of specific websites and individual aims of evaluations.

Acknowledgments

This work was supported by the Primary Care Unit, Public Health England. This study is not applicable as secondary research.

Authors' Contributions

RA wrote the protocol with input from CH, CM, and VY. RA and CH conducted the scoping review. RA wrote the final manuscript with input from CH, CM, and VY. All authors reviewed and approved the final manuscript.

Conflicts of Interest

None declared.

Summary of included studies, including information on the participant.

Interventions: methodologies and tools to evaluate websites.

Methods used or described in each study.

Summary of the most used website attributes evaluated.

Straub DW, Watson RT. Research Commentary: Transformational Issues in Researching IS and Net-Enabled Organizations. Info Syst Res 2001;12(4):337-345. [ CrossRef ]
Bairamzadeh S, Bolhari A. Investigating factors affecting students' satisfaction of university websites. In: 2010 3rd International Conference on Computer Science and Information Technology. 2010 Presented at: ICCSIT'10; July 9-11, 2010; Chengdu, China p. 469-473. [ CrossRef ]
Fink D, Nyaga C. Evaluating web site quality: the value of a multi paradigm approach. Benchmarking 2009;16(2):259-273. [ CrossRef ]
Markaki OI, Charilas DE, Askounis D. Application of Fuzzy Analytic Hierarchy Process to Evaluate the Quality of E-Government Web Sites. In: Proceedings of the 2010 Developments in E-systems Engineering. 2010 Presented at: DeSE'10; September 6-8, 2010; London, UK p. 219-224. [ CrossRef ]
Eysenbach G, Powell J, Kuss O, Sa E. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. J Am Med Assoc 2002;287(20):2691-2700. [ CrossRef ] [ Medline ]
International Organization for Standardization. ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs): Part 11: Guidance on Usability. Switzerland: International Organization for Standardization; 1998.
Hartmann J, Sutcliffe A, Angeli AD. Towards a theory of user judgment of aesthetics and user interface quality. ACM Trans Comput-Hum Interact 2008;15(4):1-30. [ CrossRef ]
Bargas-Avila JA, Hornbæk K. Old Wine in New Bottles or Novel Challenges: A Critical Analysis of Empirical Studies of User Experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2011 Presented at: CHI'11; May 7-12, 2011; Vancouver, BC, Canada p. 2689-2698. [ CrossRef ]
Hassenzahl M, Tractinsky N. User experience - a research agenda. Behav Info Technol 2006;25(2):91-97. [ CrossRef ]
Aranyi G, van Schaik P. Testing a model of user-experience with news websites. J Assoc Soc Inf Sci Technol 2016;67(7):1555-1575. [ CrossRef ]
Tsai W, Chou W, Lai C. An effective evaluation model and improvement analysis for national park websites: a case study of Taiwan. Tour Manag 2010;31(6):936-952. [ CrossRef ]
Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Method 2005;8(1):19-32. [ CrossRef ]
Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Syst 2008;6:7 [ FREE Full text ] [ CrossRef ] [ Medline ]
e-Bug. 2018. Welcome to the e-Bug Teachers Area! URL: https://e-bug.eu/eng_home.aspx?cc=eng&ss=1&t=Welcome%20to%20e-Bug [accessed 2019-08-23]
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009 Jul 21;6(7):e1000100 [ FREE Full text ] [ CrossRef ] [ Medline ]
Badger D, Nursten J, Williams P, Woodward M. Should All Literature Reviews be Systematic? Eval Res Educ 2000;14(3-4):220-230. [ CrossRef ]
Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess B, editors. Analyzing Qualitative Data. Abingdon-on-Thames: Routledge; 2002:187-208.
Thomsett-Scott BC. Web site usability with remote users: Formal usability studies and focus groups. J Libr Adm 2006;45(3-4):517-547. [ CrossRef ]
Moreno JM, Morales del Castillo JM, Porcel C, Herrera-Viedma E. A quality evaluation methodology for health-related websites based on a 2-tuple fuzzy linguistic approach. Soft Comput 2010;14(8):887-897. [ CrossRef ]
Alva M, Martínez A, Labra Gayo J, Del Carmen Suárez M, Cueva J, Sagástegui H. Proposal of a tool of support to the evaluation of user in educative web sites. 2008 Presented at: 1st World Summit on the Knowledge Society, WSKS 2008; 2008; Athens p. 149-157. [ CrossRef ]
Usability: Home. Usability Evaluation Basics URL: https://www.usability.gov/what-and-why/usability-evaluation.html [accessed 2019-08-24]
AddThis: Get more likes, shares and follows with smart. 10 Criteria for Better Website Usability: Heuristics Cheat Sheet URL: http://www.addthis.com/blog/2015/02/17/10-criteria-for-better-website-usability-heuristics-cheat-sheet/#.V712QfkrJD8 [accessed 2019-08-24]
Akgül Y. Quality evaluation of E-government websites of Turkey. In: Proceedings of the 2016 11th Iberian Conference on Information Systems and Technologies. 2016 Presented at: CISTI'16; June 15-18 2016; Las Palmas, Spain p. 1-7. [ CrossRef ]
Al Zaghoul FA, Al Nsour AJ, Rababah OM. Ranking Quality Factors for Measuring Web Service Quality. In: Proceedings of the 1st International Conference on Intelligent Semantic Web-Services and Applications. 2010 Presented at: 1st ACM Jordan Professional Chapter ISWSA Annual - International Conference on Intelligent Semantic Web-Services and Applications, ISWSA'10; June 14-16, 2010; Amman, Jordan. [ CrossRef ]
Alharbi A, Mayhew P. Users' Performance in Lab and Non-lab Enviornments Through Online Usability Testing: A Case of Evaluating the Usability of Digital Academic Libraries' Websites. In: 2015 Science and Information Conference. 2015 Presented at: SAI'15; July 28-30, 2015; London, UK p. 151-161. [ CrossRef ]
Aliyu M, Mahmud M, Md Tap AO. Preliminary Investigation of Islamic Websites Design & Content Feature: A Heuristic Evaluation From User Perspective. In: Proceedings of the 2010 International Conference on User Science and Engineering. 2010 Presented at: iUSEr'10; December 13-15, 2010; Shah Alam, Malaysia p. 262-267. [ CrossRef ]
Aliyu M, Mahmud M, Tap AO, Nassr RM. Evaluating Design Features of Islamic Websites: A Muslim User Perception. In: Proceedings of the 2013 5th International Conference on Information and Communication Technology for the Muslim World. 2013 Presented at: ICT4M'13; March 26-27, 2013; Rabat, Morocco. [ CrossRef ]
Al-Radaideh QA, Abu-Shanab E, Hamam S, Abu-Salem H. Usability evaluation of online news websites: a user perspective approach. World Acad Sci Eng Technol 2011;74:1058-1066 [ FREE Full text ]
Aranyi G, van Schaik P, Barker P. Using think-aloud and psychometrics to explore users’ experience with a news web site. Interact Comput 2012;24(2):69-77. [ CrossRef ]
Arrue M, Fajardo I, Lopez JM, Vigo M. Interdependence between technical web accessibility and usability: its influence on web quality models. Int J Web Eng Technol 2007;3(3):307-328. [ CrossRef ]
Arrue M, Vigo M, Abascal J. Quantitative metrics for web accessibility evaluation. 2005 Presented at: Proceedings of the ICWE 2005 Workshop on Web Metrics and Measurement; 2005; Sydney.
Atterer R, Wnuk M, Schmidt A. Knowing the User's Every Move: User Activity Tracking for Website Usability Evaluation and Implicit Interaction. In: Proceedings of the 15th international conference on World Wide. 2006 Presented at: WWW'06; May 23-26, 2006; Edinburgh, Scotland p. 203-212. [ CrossRef ]
Bahry F, Masrom M, Masrek M. Website evaluation measures, website user engagement and website credibility for municipal website. ARPN J Eng Appl Sci 2015;10(23):18228-18238. [ CrossRef ]
Bañón-Gomis A, Tomás-Miquel JV, Expósito-Langa M. Improving user experience: a methodology proposal for web usability measurement. In: Strategies in E-Business: Positioning and Social Networking in Online Markets. New York City: Springer US; 2014:123-145.
Barnes SJ, Vidgen RT. Data triangulation and web quality metrics: a case study in e-government. Inform Manag 2006;43(6):767-777. [ CrossRef ]
Bolchini D, Garzotto F. Quality of Web Usability Evaluation Methods: An Empirical Study on MiLE+. In: Proceedings of the 2007 international conference on Web information systems engineering. 2007 Presented at: WISE'07; December 3-3, 2007; Nancy, France p. 481-492. [ CrossRef ]
Chen FH, Tzeng G, Chang CC. Evaluating the enhancement of corporate social responsibility websites quality based on a new hybrid MADM model. Int J Inf Technol Decis Mak 2015;14(03):697-724. [ CrossRef ]
Cherfi SS, Tuan AD, Comyn-Wattiau I. An Exploratory Study on Websites Quality Assessment. In: Proceedings of the 32nd International Conference on Conceptual Modeling Workshops. 2013 Presented at: ER'13; November 11-13, 2014; Hong Kong, China p. 170-179. [ CrossRef ]
Chou W, Cheng Y. A hybrid fuzzy MCDM approach for evaluating website quality of professional accounting firms. Expert Sys Appl 2012;39(3):2783-2793. [ CrossRef ]
Churm T. Usability Geek. 2012 Jul 9. An Introduction To Website Usability Testing URL: http://usabilitygeek.com/an-introduction-to-website-usability-testing/ [accessed 2019-08-24]
Demir Y, Gozum S. Evaluation of quality, content, and use of the web site prepared for family members giving care to stroke patients. Comput Inform Nurs 2015 Sep;33(9):396-403. [ CrossRef ] [ Medline ]
Dominic P, Jati H, Hanim S. University website quality comparison by using non-parametric statistical test: a case study from Malaysia. Int J Oper Res 2013;16(3):349-374. [ CrossRef ]
Elling S, Lentz L, de Jong M, van den Bergh H. Measuring the quality of governmental websites in a controlled versus an online setting with the ‘Website Evaluation Questionnaire’. Gov Inf Q 2012;29(3):383-393. [ CrossRef ]
Fang X, Holsapple CW. Impacts of navigation structure, task complexity, and users’ domain knowledge on web site usability—an empirical study. Inf Syst Front 2011;13(4):453-469. [ CrossRef ]
Fernandez A, Abrahão S, Insfran E. A systematic review on the effectiveness of web usability evaluation methods. In: Proceedings of the 16th International Conference on Evaluation & Assessment in Software Engineering. 2012 Presented at: EASE'12; May 14-15, 2012; Ciudad Real, Spain. [ CrossRef ]
Flavián C, Guinalíu M, Gurrea R. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inform Manag 2006;43(1):1-14. [ CrossRef ]
Flavián C, Guinalíu M, Gurrea R. The influence of familiarity and usability on loyalty to online journalistic services: the role of user experience. J Retail Consum Serv 2006;13(5):363-375. [ CrossRef ]
Gonzalez ME, Quesada G, Davis J, Mora-Monge C. Application of quality management tools in the evaluation of websites: the case of sports organizations. Qual Manage J 2015;22(1):30-46. [ CrossRef ]
Harrison C, Pétrie H. Deconstructing Web Experience: More Than Just Usability and Good Design. In: Proceedings of the 12th international conference on Human-computer interaction: applications and services. 2007 Presented at: HCI'07; July 22-27, 2007; Beijing, China p. 889-898. [ CrossRef ]
Hart D, Portwood DM. Usability Testing of Web Sites Designed for Communities of Practice: Tests of the IEEE Professional Communication Society (PCS) Web Site Combining Specialized Heuristic Evaluation and Task-based User Testing. In: Proceedings of the 2009 IEEE International Professional Communication Conference. 2009 Presented at: 2009 IEEE International Professional Communication Conference; July 19-22, 2009; Waikiki, HI, USA. [ CrossRef ]
Hedegaard S, Simonsen JG. Extracting Usability and User Experience Information From Online User Reviews. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2013 Presented at: CHI'13; April 27-May 2, 2013; Paris, France p. 2089-2098. [ CrossRef ]
Herrera F, Herrera-Viedma E, Martínez L, Ferez LG, López-Herrera AG, Alonso S. A multi-granular linguistic hierarchical model to evaluate the quality of web site services. In: Mathew S, Mordeson JN, Malik DS, editors. Studies in Fuzziness and Soft Computing. New York City: Springer; 2006:247-274.
Hinchliffe A, Mummery WK. Applying usability testing techniques to improve a health promotion website. Health Promot J Austr 2008 Apr;19(1):29-35. [ CrossRef ] [ Medline ]
Ijaz T, Andlib F. Impact of Usability on Non-technical Users: Usability Testing Through Websites. In: Proceedings of the 2014 National Software Engineering Conference. 2014 Presented at: 2014 National Software Engineering Conference, NSEC 2014; November 11-12, 2014; Rawalpindi, Pakistan. [ CrossRef ]
Janiak E, Rhodes E, Foster AM. Translating access into utilization: lessons from the design and evaluation of a health insurance Web site to promote reproductive health care for young women in Massachusetts. Contraception 2013 Dec;88(6):684-690. [ CrossRef ] [ Medline ]
Kaya T. Multi-attribute evaluation of website quality in e-business using an integrated fuzzy AHP-TOPSIS methodology. Int J Comput Intell Syst 2010;3(3):301-314. [ CrossRef ]
Kincl T, Štrach P. Measuring website quality: asymmetric effect of user satisfaction. Behav Inform Technol 2012;31(7):647-657. [ CrossRef ]
Koutsabasis P, Istikopoulou TG. Perceived website aesthetics by users and designers: implications for evaluation practice. Int J Technol Human Interact 2014;10(2):21-34. [ CrossRef ]
Leuthold S, Schmutz P, Bargas-Avila JA, Tuch AN, Opwis K. Vertical versus dynamic menus on the world wide web: eye tracking study measuring the influence of menu design and task complexity on user performance and subjective preference. Comput Human Behav 2011;27(1):459-472. [ CrossRef ]
Longstreet P. Evaluating Website Quality: Applying Cue Utilization Theory to WebQual. In: Proceedings of the 2010 43rd Hawaii International Conference on System Sciences. 2010 Presented at: HICSS'10; January 5-8, 2010; Honolulu, HI, USA. [ CrossRef ]
Manzoor M. Measuring user experience of usability tool, designed for higher educational websites. Middle East J Sci Res 2013;14(3):347-353. [ CrossRef ]
Mashable India. 22 Essential Tools for Testing Your Website's Usability URL: http://mashable.com/2011/09/30/website-usability-tools/#cNv8ckxZsmqw [accessed 2019-08-16]
Matera M, Rizzo F, Carughi GT. Web usability: principles and evaluation methods. In: Web Engineering. New York City: Springer; 2006:143-180.
McClellan MA, Karumur RP, Vogel RI, Petzel SV, Cragg J, Chan D, et al. Designing an educational website to improve quality of supportive oncology care for women with ovarian cancer: an expert usability review and analysis. Int J Hum Comput Interact 2016;32(4):297-307 [ FREE Full text ] [ CrossRef ] [ Medline ]
Nakamichi N, Shima K, Sakai M, Matsumoto KI. Detecting Low Usability Web Pages Using Quantitative Data of Users' Behavior. In: Proceedings of the 28th international conference on Software engineering. 2006 Presented at: ICSE'06; May 20-28, 2006; Shanghai, China p. 569-576. [ CrossRef ]
Nathan RJ, Yeow PH. An empirical study of factors affecting the perceived usability of websites for student internet users. Univ Access Inf Soc 2009;8(3):165-184. [ CrossRef ]
Oliver H, Diallo G, de Quincey E, Alexopoulou D, Habermann B, Kostkova P, et al. A user-centred evaluation framework for the Sealife semantic web browsers. BMC Bioinform 2009;10(S10). [ CrossRef ]
Paul A, Yadamsuren B, Erdelez S. An Experience With Measuring Multi-User Online Task Performance. In: Proceedings of the 2012 World Congress on Information and Communication Technologies. 2012 Presented at: WICT'12; October 30-November 2, 2012; Trivandrum, India p. 639-644. [ CrossRef ]
Petrie H, Power C. What Do Users Really Care About?: A Comparison of Usability Problems Found by Users and Experts on Highly Interactive Websites. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2012 Presented at: CHI'12; May 5-10, 2012; Austin, Texas, USA p. 2107-2116. [ CrossRef ]
Rekik R, Kallel I. Fuzzy Reduced Method for Evaluating the Quality of Institutional Web Sites. In: Proceedings of the 2011 7th International Conference on Next Generation Web Services Practices. 2011 Presented at: NWeSP'11; October 19-21, 2011; Salamanca, Spain p. 296-301. [ CrossRef ]
Reynolds E. The secret to patron-centered Web design: cheap, easy, and powerful usability techniques. Comput Librar 2008;28(6):44-47 [ FREE Full text ]
Sheng H, Lockwood NS, Dahal S. Eyes Don't Lie: Understanding Users' First Impressions on Websites Using Eye Tracking. In: Proceedings of the 15th International Conference on Human Interface and the Management of Information: Information and Interaction Design. 2013 Presented at: HCI'13; July 21-26, 2013; Las Vegas, NV, USA p. 635-641. [ CrossRef ]
Swaid SI, Wigand RT. Measuring Web-based Service Quality: The Online Customer Point of View. In: Proceedings of the 13th Americas Conference on Information Systems. 2007 Presented at: AMCIS'07; August 9-12, 2007; Keystone, Colorado, USA p. 778-790.
Tan GW, Wei KK. An empirical study of Web browsing behaviour: towards an effective Website design. Elect Commer Res Appl 2006;5(4):261-271. [ CrossRef ]
Tan W, Liu D, Bishu R. Web evaluation: heuristic evaluation vs user testing. Int J Ind Ergonom 2009;39(4):621-627. [ CrossRef ]
Tao D, LeRouge CM, Deckard G, de Leo G. Consumer Perspectives on Quality Attributes in Evaluating Health Websites. In: Proceedings of the 2012 45th Hawaii International Conference on System Sciences. 2012 Presented at: HICSS'12; January 4-7, 2012; Maui, HI, USA. [ CrossRef ]
The Whole Brain Group. 2011. Conducting a Quick & Dirty Evaluation of Your Website's Usability URL: http://blog.thewholebraingroup.com/conducting-quick-dirty-evaluation-websites-usability [accessed 2019-08-24]
Thielsch MT, Blotenberg I, Jaron R. User evaluation of websites: From first impression to recommendation. Interact Comput 2014;26(1):89-102. [ CrossRef ]
Tung LL, Xu Y, Tan FB. Attributes of web site usability: a study of web users with the repertory grid technique. Int J Elect Commer 2009;13(4):97-126. [ CrossRef ]
Agarwal R, Venkatesh V. Assessing a firm's Web presence: a heuristic evaluation procedure for the measurement of usability. Inform Syst Res 2002;13(2):168-186. [ CrossRef ]
Venkatesh V, Ramesh V. Web and wireless site usability: understanding differences and modeling use. Manag Inf Syst Q 2006;30(1):181-206. [ CrossRef ]
Vaananen-Vainio-Mattila K, Wäljas M. Development of Evaluation Heuristics for Web Service User Experience. In: Proceedings of the Extended Abstracts on Human Factors in Computing Systems. 2009 Presented at: CHI'09; April 4-9, 2009; Boston, MA, USA p. 3679-3684. [ CrossRef ]
Wang WT, Wang B, Wei YT. Examining the Impacts of Website Complexities on User Satisfaction Based on the Task-technology Fit Model: An Experimental Research Using an Eyetracking Device. In: Proceedings of the 18th Pacific Asia Conference on Information Systems. 2014 Presented at: PACIS'14; June 18-22, 2014; Jeju Island, South Korea.
Yen B, Hu PJ, Wang M. Toward an analytical approach for effective web site design: a framework for modeling, evaluation and enhancement. Elect Commer Res Appl 2007;6(2):159-170. [ CrossRef ]
Yen PY, Bakken S. A comparison of usability evaluation methods: heuristic evaluation versus end-user think-aloud protocol - an example from a web-based communication tool for nurse scheduling. AMIA Annu Symp Proc 2009 Nov 14;2009:714-718 [ FREE Full text ] [ Medline ]
Allison R, Hayes C, Young V, McNulty CAM. Evaluation of an educational health website on infections and antibiotics: a mixed-methods, user-centred approach in England. JMIR Formative Research 2019 (forthcoming).

Abbreviations

Edited by G Eysenbach; submitted 26.04.19; peer-reviewed by C Eley, C Brown; comments to author 31.05.19; revised version received 24.06.19; accepted 18.08.19; published 24.10.19

©Rosalie Allison, Catherine Hayes, Cliodna A M McNulty, Vicki Young. Originally published in JMIR Formative Research (http://formative.jmir.org), 24.10.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.

Literature review: your definitive guide

Joanna Wilkinson

This is our ultimate guide on how to write a narrative literature review. It forms part of our Research Smarter series .

How do you write a narrative literature review?

Researchers worldwide are increasingly reliant on literature reviews. That’s because review articles provide you with a broad picture of the field, and help to synthesize published research that’s expanding at a rapid pace .

In some academic fields, researchers publish more literature reviews than original research papers. The graph below shows the substantial growth of narrative literature reviews in the Web of Science™, alongside the percentage increase of reviews when compared to all document types.

It’s critical that researchers across all career levels understand how to produce an objective, critical summary of published research. This is no easy feat, but a necessary one. Professionally constructed literature reviews – whether written by a student in class or an experienced researcher for publication – should aim to add to the literature rather than detract from it.

To help you write a narrative literature review, we’ve put together some top tips in this blog post.

Best practice tips to write a narrative literature review:

Don’t miss a paper: tips for a thorough topic search
Identify key papers (and know how to use them)
Tips for working with co-authors
Find the right journal for your literature review using actual data
Discover literature review examples and templates

We’ll also provide an overview of all the products helpful for your next narrative review, including the Web of Science, EndNote™ and Journal Citation Reports™.

1. Don’t miss a paper: tips for a thorough topic search

Once you’ve settled on your research question, coming up with a good set of keywords to find papers on your topic can be daunting. This isn’t surprising. Put simply, if you fail to include a relevant paper when you write a narrative literature review, the omission will probably get picked up by your professor or peer reviewers. The end result will likely be a low mark or an unpublished manuscript, neither of which will do justice to your many months of hard work.

Research databases and search engines are an integral part of any literature search. It’s important you utilize as many options available through your library as possible. This will help you search an entire discipline (as well as across disciplines) for a thorough narrative review.

We provide a short summary of the various databases and search engines in an earlier Research Smarter blog . These include the Web of Science , Science.gov and the Directory of Open Access Journals (DOAJ).

Like what you see? Share it with others on Twitter:

[bctt tweet=”Writing a #LiteratureReview? Check out the latest @clarivateAG blog for top tips (from topic searches to working with coauthors), examples, templates and more”]

Searching the Web of Science

The Web of Science is a multidisciplinary research engine that contains over 170 million papers from more than 250 academic disciplines. All of the papers in the database are interconnected via citations. That means once you get started with your keyword search, you can follow the trail of cited and citing papers to efficiently find all the relevant literature. This is a great way to ensure you’re not missing anything important when you write a narrative literature review.

We recommend starting your search in the Web of Science Core Collection™. This database covers more than 21,000 carefully selected journals. It is a trusted source to find research papers, and discover top authors and journals (read more about its coverage here ).

Learn more about exploring the Core Collection in our blog, How to find research papers: five tips every researcher should know . Our blog covers various tips, including how to:

Perform a topic search (and select your keywords)
Explore the citation network
Refine your results (refining your search results by reviews, for example, will help you avoid duplication of work, as well as identify trends and gaps in the literature)
Save your search and set up email alerts

Try our tips on the Web of Science now.

2. Identify key papers (and know how to use them)

As you explore the Web of Science, you may notice that certain papers are marked as “Highly Cited.” These papers can play a significant role when you write a narrative literature review.

Highly Cited papers are recently published papers getting the most attention in your field right now. They form the top 1% of papers based on the number of citations received, compared to other papers published in the same field in the same year.

You will want to identify Highly Cited research as a group of papers. This group will help guide your analysis of the future of the field and opportunities for future research. This is an important component of your conclusion.

Writing reviews is hard work…[it] not only organizes published papers, but also positions t hem in the academic process and presents the future direction. Prof. Susumu Kitagawa, Highly Cited Researcher, Kyoto University

3. Tips for working with co-authors

Writing a narrative review on your own is hard, but it can be even more challenging if you’re collaborating with a team, especially if your coauthors are working across multiple locations. Luckily, reference management software can improve the coordination between you and your co-authors—both around the department and around the world.

We’ve written about how to use EndNote’s Cite While You Write feature, which will help you save hundreds of hours when writing research . Here, we discuss the features that give you greater ease and control when collaborating with your colleagues.

Use EndNote for narrative reviews

Sharing references is essential for successful collaboration. With EndNote, you can store and share as many references, documents and files as you need with up to 100 people using the software.

You can share simultaneous access to one reference library, regardless of your colleague’s location or organization. You can also choose the type of access each user has on an individual basis. For example, Read-Write access means a select colleague can add and delete references, annotate PDF articles and create custom groups. They’ll also be able to see up to 500 of the team’s most recent changes to the reference library. Read-only is also an option for individuals who don’t need that level of access.

EndNote helps you overcome research limitations by synchronizing library changes every 15 minutes. That means your team can stay up-to-date at any time of the day, supporting an easier, more successful collaboration.

Start your free EndNote trial today .

4.Finding a journal for your literature review

Finding the right journal for your literature review can be a particular pain point for those of you who want to publish. The expansion of scholarly journals has made the task extremely difficult, and can potentially delay the publication of your work by many months.

We’ve written a blog about how you can find the right journal for your manuscript using a rich array of data. You can read our blog here , or head straight to Endnote’s Manuscript Matcher or Journal Citation Report s to try out the best tools for the job.

5. Discover literature review examples and templates

There are a few tips we haven’t covered in this blog, including how to decide on an area of research, develop an interesting storyline, and highlight gaps in the literature. We’ve listed a few blogs here that might help you with this, alongside some literature review examples and outlines to get you started.

Literature Review examples:

Aggregation-induced emission
Development and applications of CRISPR-Cas9 for genome engineering
Object based image analysis for remote sensing

(Make sure you download the free EndNote™ Click browser plugin to access the full-text PDFs).

Templates and outlines:

Learn how to write a review of literature , Univ. of Wisconsin – Madison
Structuring a literature review , Australian National University
Matrix Method for Literature Review: The Review Matrix , Duquesne University

Additional resources:

Ten simple rules for writing a literature review , Editor, PLoS Computational Biology
Video: How to write a literature review , UC San Diego Psychology

Journal citation reports 2024 preview: unified rankings for more inclusive journal assessment.

Introducing the Clarivate Academic AI Platform

Reimagining research impact: Introducing Web of Science Research Intelligence

How To Write An A-Grade Literature Review

3 straightforward steps (with examples) + free template.

By: Derek Jansen (MBA) | Expert Reviewed By: Dr. Eunice Rautenbach | October 2019

Quality research is about building onto the existing work of others , “standing on the shoulders of giants”, as Newton put it. The literature review chapter of your dissertation, thesis or research project is where you synthesise this prior work and lay the theoretical foundation for your own research.

Long story short, this chapter is a pretty big deal, which is why you want to make sure you get it right . In this post, I’ll show you exactly how to write a literature review in three straightforward steps, so you can conquer this vital chapter (the smart way).

Overview: The Literature Review Process

Understanding the “ why “
Finding the relevant literature
Cataloguing and synthesising the information
Outlining & writing up your literature review
Example of a literature review

But first, the “why”…

Before we unpack how to write the literature review chapter, we’ve got to look at the why . To put it bluntly, if you don’t understand the function and purpose of the literature review process, there’s no way you can pull it off well. So, what exactly is the purpose of the literature review?

Well, there are (at least) four core functions:

For you to gain an understanding (and demonstrate this understanding) of where the research is at currently, what the key arguments and disagreements are.
For you to identify the gap(s) in the literature and then use this as justification for your own research topic.
To help you build a conceptual framework for empirical testing (if applicable to your research topic).
To inform your methodological choices and help you source tried and tested questionnaires (for interviews ) and measurement instruments (for surveys ).

Most students understand the first point but don’t give any thought to the rest. To get the most from the literature review process, you must keep all four points front of mind as you review the literature (more on this shortly), or you’ll land up with a wonky foundation.

Okay – with the why out the way, let’s move on to the how . As mentioned above, writing your literature review is a process, which I’ll break down into three steps:

Finding the most suitable literature
Understanding , distilling and organising the literature
Planning and writing up your literature review chapter

Importantly, you must complete steps one and two before you start writing up your chapter. I know it’s very tempting, but don’t try to kill two birds with one stone and write as you read. You’ll invariably end up wasting huge amounts of time re-writing and re-shaping, or you’ll just land up with a disjointed, hard-to-digest mess . Instead, you need to read first and distil the information, then plan and execute the writing.

Step 1: Find the relevant literature

Naturally, the first step in the literature review journey is to hunt down the existing research that’s relevant to your topic. While you probably already have a decent base of this from your research proposal , you need to expand on this substantially in the dissertation or thesis itself.

Essentially, you need to be looking for any existing literature that potentially helps you answer your research question (or develop it, if that’s not yet pinned down). There are numerous ways to find relevant literature, but I’ll cover my top four tactics here. I’d suggest combining all four methods to ensure that nothing slips past you:

Method 1 – Google Scholar Scrubbing

Google’s academic search engine, Google Scholar , is a great starting point as it provides a good high-level view of the relevant journal articles for whatever keyword you throw at it. Most valuably, it tells you how many times each article has been cited, which gives you an idea of how credible (or at least, popular) it is. Some articles will be free to access, while others will require an account, which brings us to the next method.

Method 2 – University Database Scrounging

Generally, universities provide students with access to an online library, which provides access to many (but not all) of the major journals.

So, if you find an article using Google Scholar that requires paid access (which is quite likely), search for that article in your university’s database – if it’s listed there, you’ll have access. Note that, generally, the search engine capabilities of these databases are poor, so make sure you search for the exact article name, or you might not find it.

Method 3 – Journal Article Snowballing

At the end of every academic journal article, you’ll find a list of references. As with any academic writing, these references are the building blocks of the article, so if the article is relevant to your topic, there’s a good chance a portion of the referenced works will be too. Do a quick scan of the titles and see what seems relevant, then search for the relevant ones in your university’s database.

Method 4 – Dissertation Scavenging

Similar to Method 3 above, you can leverage other students’ dissertations. All you have to do is skim through literature review chapters of existing dissertations related to your topic and you’ll find a gold mine of potential literature. Usually, your university will provide you with access to previous students’ dissertations, but you can also find a much larger selection in the following databases:

Open Access Theses & Dissertations
Stanford SearchWorks

Keep in mind that dissertations and theses are not as academically sound as published, peer-reviewed journal articles (because they’re written by students, not professionals), so be sure to check the credibility of any sources you find using this method. You can do this by assessing the citation count of any given article in Google Scholar. If you need help with assessing the credibility of any article, or with finding relevant research in general, you can chat with one of our Research Specialists .

Alright – with a good base of literature firmly under your belt, it’s time to move onto the next step.

Need a helping hand?

Step 2: Log, catalogue and synthesise

Once you’ve built a little treasure trove of articles, it’s time to get reading and start digesting the information – what does it all mean?

While I present steps one and two (hunting and digesting) as sequential, in reality, it’s more of a back-and-forth tango – you’ll read a little , then have an idea, spot a new citation, or a new potential variable, and then go back to searching for articles. This is perfectly natural – through the reading process, your thoughts will develop , new avenues might crop up, and directional adjustments might arise. This is, after all, one of the main purposes of the literature review process (i.e. to familiarise yourself with the current state of research in your field).

As you’re working through your treasure chest, it’s essential that you simultaneously start organising the information. There are three aspects to this:

Logging reference information
Building an organised catalogue
Distilling and synthesising the information

I’ll discuss each of these below:

2.1 – Log the reference information

As you read each article, you should add it to your reference management software. I usually recommend Mendeley for this purpose (see the Mendeley 101 video below), but you can use whichever software you’re comfortable with. Most importantly, make sure you load EVERY article you read into your reference manager, even if it doesn’t seem very relevant at the time.

2.2 – Build an organised catalogue

In the beginning, you might feel confident that you can remember who said what, where, and what their main arguments were. Trust me, you won’t. If you do a thorough review of the relevant literature (as you must!), you’re going to read many, many articles, and it’s simply impossible to remember who said what, when, and in what context . Also, without the bird’s eye view that a catalogue provides, you’ll miss connections between various articles, and have no view of how the research developed over time. Simply put, it’s essential to build your own catalogue of the literature.

I would suggest using Excel to build your catalogue, as it allows you to run filters, colour code and sort – all very useful when your list grows large (which it will). How you lay your spreadsheet out is up to you, but I’d suggest you have the following columns (at minimum):

Author, date, title – Start with three columns containing this core information. This will make it easy for you to search for titles with certain words, order research by date, or group by author.
Categories or keywords – You can either create multiple columns, one for each category/theme and then tick the relevant categories, or you can have one column with keywords.
Key arguments/points – Use this column to succinctly convey the essence of the article, the key arguments and implications thereof for your research.
Context – Note the socioeconomic context in which the research was undertaken. For example, US-based, respondents aged 25-35, lower- income, etc. This will be useful for making an argument about gaps in the research.
Methodology – Note which methodology was used and why. Also, note any issues you feel arise due to the methodology. Again, you can use this to make an argument about gaps in the research.
Quotations – Note down any quoteworthy lines you feel might be useful later.
Notes – Make notes about anything not already covered. For example, linkages to or disagreements with other theories, questions raised but unanswered, shortcomings or limitations, and so forth.

If you’d like, you can try out our free catalog template here (see screenshot below).

2.3 – Digest and synthesise

Most importantly, as you work through the literature and build your catalogue, you need to synthesise all the information in your own mind – how does it all fit together? Look for links between the various articles and try to develop a bigger picture view of the state of the research. Some important questions to ask yourself are:

What answers does the existing research provide to my own research questions ?
Which points do the researchers agree (and disagree) on?
How has the research developed over time?
Where do the gaps in the current research lie?

To help you develop a big-picture view and synthesise all the information, you might find mind mapping software such as Freemind useful. Alternatively, if you’re a fan of physical note-taking, investing in a large whiteboard might work for you.

Mind mapping is a useful way to plan your literature review.

Step 3: Outline and write it up!

Once you’re satisfied that you have digested and distilled all the relevant literature in your mind, it’s time to put pen to paper (or rather, fingers to keyboard). There are two steps here – outlining and writing:

3.1 – Draw up your outline

Having spent so much time reading, it might be tempting to just start writing up without a clear structure in mind. However, it’s critically important to decide on your structure and develop a detailed outline before you write anything. Your literature review chapter needs to present a clear, logical and an easy to follow narrative – and that requires some planning. Don’t try to wing it!

Naturally, you won’t always follow the plan to the letter, but without a detailed outline, you’re more than likely going to end up with a disjointed pile of waffle , and then you’re going to spend a far greater amount of time re-writing, hacking and patching. The adage, “measure twice, cut once” is very suitable here.

In terms of structure, the first decision you’ll have to make is whether you’ll lay out your review thematically (into themes) or chronologically (by date/period). The right choice depends on your topic, research objectives and research questions, which we discuss in this article .

Once that’s decided, you need to draw up an outline of your entire chapter in bullet point format. Try to get as detailed as possible, so that you know exactly what you’ll cover where, how each section will connect to the next, and how your entire argument will develop throughout the chapter. Also, at this stage, it’s a good idea to allocate rough word count limits for each section, so that you can identify word count problems before you’ve spent weeks or months writing!

PS – check out our free literature review chapter template…

3.2 – Get writing

With a detailed outline at your side, it’s time to start writing up (finally!). At this stage, it’s common to feel a bit of writer’s block and find yourself procrastinating under the pressure of finally having to put something on paper. To help with this, remember that the objective of the first draft is not perfection – it’s simply to get your thoughts out of your head and onto paper, after which you can refine them. The structure might change a little, the word count allocations might shift and shuffle, and you might add or remove a section – that’s all okay. Don’t worry about all this on your first draft – just get your thoughts down on paper.

Once you’ve got a full first draft (however rough it may be), step away from it for a day or two (longer if you can) and then come back at it with fresh eyes. Pay particular attention to the flow and narrative – does it fall fit together and flow from one section to another smoothly? Now’s the time to try to improve the linkage from each section to the next, tighten up the writing to be more concise, trim down word count and sand it down into a more digestible read.

Once you’ve done that, give your writing to a friend or colleague who is not a subject matter expert and ask them if they understand the overall discussion. The best way to assess this is to ask them to explain the chapter back to you. This technique will give you a strong indication of which points were clearly communicated and which weren’t. If you’re working with Grad Coach, this is a good time to have your Research Specialist review your chapter.

Finally, tighten it up and send it off to your supervisor for comment. Some might argue that you should be sending your work to your supervisor sooner than this (indeed your university might formally require this), but in my experience, supervisors are extremely short on time (and often patience), so, the more refined your chapter is, the less time they’ll waste on addressing basic issues (which you know about already) and the more time they’ll spend on valuable feedback that will increase your mark-earning potential.

Literature Review Example

In the video below, we unpack an actual literature review so that you can see how all the core components come together in reality.

Let’s Recap

In this post, we’ve covered how to research and write up a high-quality literature review chapter. Let’s do a quick recap of the key takeaways:

It is essential to understand the WHY of the literature review before you read or write anything. Make sure you understand the 4 core functions of the process.
The first step is to hunt down the relevant literature . You can do this using Google Scholar, your university database, the snowballing technique and by reviewing other dissertations and theses.
Next, you need to log all the articles in your reference manager , build your own catalogue of literature and synthesise all the research.
Following that, you need to develop a detailed outline of your entire chapter – the more detail the better. Don’t start writing without a clear outline (on paper, not in your head!)
Write up your first draft in rough form – don’t aim for perfection. Remember, done beats perfect.
Refine your second draft and get a layman’s perspective on it . Then tighten it up and submit it to your supervisor.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

38 Comments

Thank you very much. This page is an eye opener and easy to comprehend.

This is awesome!

I wish I come across GradCoach earlier enough.

But all the same I’ll make use of this opportunity to the fullest.

Thank you for this good job.

Keep it up!

You’re welcome, Yinka. Thank you for the kind words. All the best writing your literature review.

Thank you for a very useful literature review session. Although I am doing most of the steps…it being my first masters an Mphil is a self study and one not sure you are on the right track. I have an amazing supervisor but one also knows they are super busy. So not wanting to bother on the minutae. Thank you.

You’re most welcome, Renee. Good luck with your literature review 🙂

This has been really helpful. Will make full use of it. 🙂

Thank you Gradcoach.

Really agreed. Admirable effort

thank you for this beautiful well explained recap.

Thank you so much for your guide of video and other instructions for the dissertation writing.

It is instrumental. It encouraged me to write a dissertation now.

Thank you the video was great – from someone that knows nothing thankyou

an amazing and very constructive way of presetting a topic, very useful, thanks for the effort,

It is timely

It is very good video of guidance for writing a research proposal and a dissertation. Since I have been watching and reading instructions, I have started my research proposal to write. I appreciate to Mr Jansen hugely.

I learn a lot from your videos. Very comprehensive and detailed.

Thank you for sharing your knowledge. As a research student, you learn better with your learning tips in research

I was really stuck in reading and gathering information but after watching these things are cleared thanks, it is so helpful.

Really helpful, Thank you for the effort in showing such information

This is super helpful thank you very much.

Thank you for this whole literature writing review.You have simplified the process.

I’m so glad I found GradCoach. Excellent information, Clear explanation, and Easy to follow, Many thanks Derek!

You’re welcome, Maithe. Good luck writing your literature review 🙂

Thank you Coach, you have greatly enriched and improved my knowledge

Great piece, so enriching and it is going to help me a great lot in my project and thesis, thanks so much

This is THE BEST site for ANYONE doing a masters or doctorate! Thank you for the sound advice and templates. You rock!

Thanks, Stephanie 🙂

This is mind blowing, the detailed explanation and simplicity is perfect.

I am doing two papers on my final year thesis, and I must stay I feel very confident to face both headlong after reading this article.

thank you so much.

if anyone is to get a paper done on time and in the best way possible, GRADCOACH is certainly the go to area!

This is very good video which is well explained with detailed explanation

Thank you excellent piece of work and great mentoring

Thanks, it was useful

Thank you very much. the video and the information were very helpful.

Good morning scholar. I’m delighted coming to know you even before the commencement of my dissertation which hopefully is expected in not more than six months from now. I would love to engage my study under your guidance from the beginning to the end. I love to know how to do good job

Thank you so much Derek for such useful information on writing up a good literature review. I am at a stage where I need to start writing my one. My proposal was accepted late last year but I honestly did not know where to start

Like the name of your YouTube implies you are GRAD (great,resource person, about dissertation). In short you are smart enough in coaching research work.

This is a very well thought out webpage. Very informative and a great read.

Very timely.

I appreciate.

Very comprehensive and eye opener for me as beginner in postgraduate study. Well explained and easy to understand. Appreciate and good reference in guiding me in my research journey. Thank you

Thank you. I requested to download the free literature review template, however, your website wouldn’t allow me to complete the request or complete a download. May I request that you email me the free template? Thank you.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Print Friendly

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

We're Hiring!
Help Center

A LITERATURE REVIEW ON WEB DESIGN Directed Study

Related Papers

Curtis Kelly

In the consumer-ruled environment of the Web, where competition is just a click away, good Web design is essential for keeping users on site. The key factor in good site design is usability, which means the ability of a user to accomplish tasks with ease, efficiency, and accuracy. The basic principles of usability can be organized into to three categories, those related to: 1) page design, 2) content design and 3) site design. The two most important factors in usability are speed and information architecture.

proceedings of Advances in Computing …

Andres Baravalle

This paper is made available online in accordance with publisher policies. Please scroll down to view the document itself. Please refer to the repository record for this item and our policy information available from the repository home page for further information. ... Author(s): Rukshan, ...

Cathy Cavanaugh

Web sites are important for schools to support teachers, administrators, counselors, students, parents, and the community. Redesigning a school's web site can become a complex process and requires careful planning. Studies with web users (Yale Style Guide, Nielsen, Siegel) have produced work that informs school web site revisers of ways to make a site more usable and enjoyable. The US government and the W3C consortium have accessibility guidelines for web sites to assist site reviewers in adapting their site so users of all abilities have equitable access. A school must reconcile time and budget limitations with the need to serve a diverse audience. This paper offers guidelines and tools for streamlining the process of web site review and redesign. Recommendations address the web site redesign team and their roles, supporting the process, identifying the site's purposes, user surveys, content audits, task lists, storyboards, color palettes, style guide, usability and accessibility. Web sites are becoming increasingly important for schools as support for teachers, administrators, counselors, students, parents, and the community. According to the Webb66 online school web site directory, over 13,000 of the nation's 108,000 schools have web sites. Jamie Mackenzie (1997) offers four reasons for maintaining effective school web sites. Reasons include introducing visitors to the school, pointing students to useful web resources, publishing student work, and collecting data on curriculum projects. Developing or redeveloping a school's web site can become a complex process and it requires careful planning. Judi Harris (1997) suggests exploring who will use the site, what information users will require or appreciate, and maximally useful ways to present the information. Years of study with web users (Yale Style Guide, Nielsen, Siegel) have produced an extensive body of work that informs school web site revisers of ways to make a site more usable and enjoyable. The US government and the W3C consortium recently released accessibility guidelines for web sites that will assist school web site reviewers in adapting their site so users of all abilities have equitable access. School web site review and redesign may be a limited, target process for making basic improvements and updates, or it may be an ambitious complete overhaul. Extensive web redesign projects can be facilitated using summer writing teams, grants, and class projects done by high school or college web design students. The following tools are presented sequentially as guidelines to assist in the school web site redesign process. Schools have unique needs, capabilities, and characters, and each school's uniqueness is reflected in its web site, as it is in the redesign process. Schools have widely varying resources to commit to web site redesign. Therefore the tools presented here should be considered a menu of options, any of which can be used to streamline the process. 1. Site redesign team

ACM Transactions on the Web

Harald Weinreich

James Cullin

Proceedings of the 15th …

steward mwale

Proceedings of the 15th international conference on World Wide Web - WWW '06

Malaysian Journal of Computer Science

Thiam Kian Chiew

Usability is one of the major factors that determines the successfulness of a website. It is important therefore to have certain measurement methods to assess the usability of websites. The methods could be used to help website designers make their websites more usable. This research focuses on website usability issues and implements a tool for evaluating the usability of websites, called WEBUSE (WEBsite USability Evaluation Tool). Based on literature research, a 24-question evaluation questionnaire has been ...

ACSIJ Journal , Dr. Sandeep Kumar Panda

We investigate the usability problems of e-commerce online shopping websites from user’s preferences and determine the relative importance of factors such as navigability, content, design, ease of use, and structure through user survey. The main intent of this ranking of web site characteristics is that a designer can relatively give higher efforts on designing features that may lead to higher merit and better usability. As such, our research work help us capture the data by involving user testing (usability testing) and open source automated tools such as Camtasia. Hence, the outcomes of the above approach show the navigation, content, design were the first, second, and third priority for evaluating the usability of e-commerce websites whereas ease of use and structure were the fourth and fifth features from the overall usability value calculation. There is a significant statistical difference between novice and expert users only for navigation feature. The maximum number of users feels satisfied with navigation, content, and design features whereas they are dissatisfied with ease of use and structural features of the websites.

Peabody Library

Literature reviews.

Developing a Literature Review
Steps to Success: The Literature Review Process
Literature Reviews Webinar Recording
Literature Review Table
Writing Like an Academic
Publishing in Academic Journals
Managing Citations This link opens in a new window

The Literature Review

What Is a Literature Review? According to the seventh edition of the APA Publication Manual, a literature review is "a critical evaluation of material that has already been published." As one embarks on creating a literature review, it is important to note that the grouping of components within a literature review can be arranged according to the author's discretion. However, it is important for the author to ensure the review reflects current APA publication standards.

Next: Steps to Success: The Literature Review Process >>
Last Updated: Feb 29, 2024 4:20 PM
URL: https://researchguides.library.vanderbilt.edu/peabody/litreviews

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

An introductory paragraph that explains what your working topic and thesis is
A forecast of key topics or texts that will appear in the review
Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
Critically Evaluate: Mention the strengths and weaknesses of your sources
Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

Summarize the key findings you have taken from the literature and emphasize their significance
Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
Qualitative versus quantitative research
Empirical versus theoretical scholarship
Divide the research by sociological, historical, or cultural sources
Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Best Web Development Bootcamp (2024): Udemy Complete Python Pro Bootcamp Review by Compare Before Buying

Boston - May 23, 2024 —

Compare Before Buying, a trusted source for online reviews and comparisons, has published an in-depth analysis of Udemy’s 100 Days of Code: The Complete Python Pro Bootcamp , highlighting the course as the best web development bootcamp for 2024. The article provides a closer look at the bootcamp’s key components and benefits for beginner and advanced learners alike.

Top-Rated Web Development Bootcamp:

100 Days of Code: The Complete Python Pro Bootcamp - click here and explore the content and resources included in the comprehensive course

Compare Before Buying recommends Udemy’s 100 Days of Code: The Complete Python Pro Bootcamp for its systematic approach, cost-effectiveness, and expert guidance.

Bootcamps for Web Development

Bootcamps offer significant value for individuals pursuing web development skills. They often provide a condensed and immersive learning experience that can fast-track one's career in the field. Udemy's 100 Days of Code: The Complete Python Pro Bootcamp structured curriculum and actual projects gives a comprehensive pathway for learners to master Python programming and web development concepts.

The course's emphasis on practical skills and real-world projects equips students with the knowledge and expertise needed to succeed in the industry. Additionally, Udemy's platform has affordable access to high-quality instruction, which can help those seeking to enter or advance in the field of web development.

Becoming a Web Developer Web development bootcamps, such as the Udemy's 100 Days of Code, are intensive training programs designed to equip individuals with the skills and knowledge needed to pursue a career in web development. These typically cover a range of topics, including HTML, CSS, JavaScript, frameworks like React or Angular, back-end development languages like Python or Ruby, databases, and more.

Bootcamps often offer hands-on projects, coding challenges, and mentorship to help learners build skills and gain experience. While a traditional computer science degree may provide a more comprehensive understanding of computer science principles, bootcamps offer a more focused and accelerated path to becoming a web developer.

Beginners in Bootcamps Coding bootcamps are designed to accommodate learners at all levels, from absolute beginners to experienced developers. In fact, many bootcamps specifically cater to beginners by providing foundational courses and resources to help them build a solid understanding of programming concepts.

Udemy's 100 Days of Code, for example, is designed to guide beginners through the fundamentals of Python programming and web development over the course of 100 days. With step-by-step instruction, hands-on projects, and community support, beginners can gradually build their skills and confidence in coding.

Joining a coding bootcamp as a beginner can be a great way to kickstart a journey into the world of programming. These programs offer structured learning paths, mentorship, and practical experience that can fast-track progress and prepare individuals for a career in web development.

Click here and explore Udemy’s 100 Days of Code: The Complete Python Pro Bootcamp. For a more comprehensive review of the course, please visit the Compare Before Buying website .

About the company: Compare Before Buying provides news and reviews of consumer products and services. As an affiliate, Compare Before Buying may earn commissions from sales generated using links provided.

Contact Info: Name: Andy Mathews Email: Send Email Organization: Compare Before Buying Website: https://www.comparebeforebuying.com

Release ID: 89130641

Should there be any problems, inaccuracies, or doubts arising from the content provided in this press release that require attention or if a press release needs to be taken down, we urge you to notify us immediately by contacting [email protected] (it is important to note that this email is the authorized channel for such matters, sending multiple emails to multiple addresses does not necessarily help expedite your request). Our efficient team will promptly address your concerns within 8 hours, taking necessary steps to rectify identified issues or assist with the removal process. Providing accurate and dependable information is central to our commitment.

Open access
Published: 20 May 2024

Targeted temperature control following traumatic brain injury: ESICM/NACCS best practice consensus recommendations

Andrea Lavinio 1 , 2 ,
Jonathan P. Coles 1 , 2 ,
Chiara Robba 3 ,
Marcel Aries 4 , 5 ,
Pierre Bouzat 6 ,
Dara Chean 7 ,
Shirin Frisvold 8 , 9 ,
Laura Galarza 10 ,
Raimund Helbok 11 , 12 ,
Jeroen Hermanides 13 ,
Mathieu van der Jagt 14 ,
David K. Menon 1 , 2 ,
Geert Meyfroidt 15 ,
Jean-Francois Payen 6 ,
Daniele Poole 16 ,
Frank Rasulo 17 ,
Jonathan Rhodes 18 ,
Emily Sidlow 19 ,
Luzius A. Steiner 20 ,
Fabio Silvio Taccone 21 , 22 &
Riikka Takala 23 , 24

Critical Care volume 28 , Article number: 170 ( 2024 ) Cite this article

3762 Accesses

16 Altmetric

Metrics details

Aims and scope

The aim of this panel was to develop consensus recommendations on targeted temperature control (TTC) in patients with severe traumatic brain injury (TBI) and in patients with moderate TBI who deteriorate and require admission to the intensive care unit for intracranial pressure (ICP) management.

A group of 18 international neuro-intensive care experts in the acute management of TBI participated in a modified Delphi process. An online anonymised survey based on a systematic literature review was completed ahead of the meeting, before the group convened to explore the level of consensus on TTC following TBI. Outputs from the meeting were combined into a further anonymous online survey round to finalise recommendations. Thresholds of ≥ 16 out of 18 panel members in agreement (≥ 88%) for strong consensus and ≥ 14 out of 18 (≥ 78%) for moderate consensus were prospectively set for all statements.

Strong consensus was reached on TTC being essential for high-quality TBI care. It was recommended that temperature should be monitored continuously, and that fever should be promptly identified and managed in patients perceived to be at risk of secondary brain injury. Controlled normothermia (36.0–37.5 °C) was strongly recommended as a therapeutic option to be considered in tier 1 and 2 of the Seattle International Severe Traumatic Brain Injury Consensus Conference ICP management protocol. Temperature control targets should be individualised based on the perceived risk of secondary brain injury and fever aetiology.

Conclusions

Based on a modified Delphi expert consensus process, this report aims to inform on best practices for TTC delivery for patients following TBI, and to highlight areas of need for further research to improve clinical guidelines in this setting.

Introduction

Traumatic brain injury (TBI) is a complex and heterogeneous disease, and a major cause of death and disability globally [ 1 , 2 , 3 ]. Amongst other common neurological diseases, TBI is estimated to have the highest prevalence and incidence, impacting up to 60 million people worldwide annually and representing a substantial public health burden [ 4 ].

TBI is defined as an alteration in brain function or other evidence of brain pathology caused by an external force [ 5 ], and requires immediate and sustained management strategies to optimise clinical outcome. The injury processes that follow from a TBI are divided into two stages: primary and secondary [ 6 ], where primary injury refers to the damage caused by the original physical impact, which can trigger a pathophysiological cascade resulting in secondary injury with deleterious effects on neurological outcome and survival [ 7 , 8 ]. In order to prevent or mitigate secondary injury, immediate treatment following severe TBI focuses on the prevention of further brain damage. As the brain remains susceptible to secondary injury from processes that extend beyond the zone of primary injury such as ischaemia, oedema, herniation, seizures and altered metabolism [ 9 ], immediate treatment following severe TBI focuses on prevention or mitigation of such injury. This is achieved through the control of intracranial pressure (ICP), and prompt treatment of systemic insults such as hypoxia, hypercapnia, and systemic hypotension [ 10 ].

In the neuro-intensive care unit (NICU), fever is a prevalent occurrence with heterogenous underlying causes, and it may contribute to secondary injury. Across patients with TBI, subarachnoid haemorrhage and stroke [ 11 , 12 , 13 ], hyperthermia has been found to increase the risk of complications and is believed to be associated with unfavourable clinical outcome including death [ 9 , 11 , 14 , 15 ].

Targeted temperature control (TTC) is a complex intervention that aims to control body or brain temperature to prevent further brain injury and improve neurological outcome [ 9 ]. The term TTC may refer to different degrees of temperature control, from fever prevention, maintenance of normothermia to the induction of hypothermia, at different levels [ 9 , 16 ]. In TBI, TTC can be used to modulate a range of important physiological parameters such as cerebral metabolism and ICP. However, its role in improving long-term outcome, as well as the appropriate indications, targets and duration of TTC in severe or moderate TBI are currently unknown.

This work aims to utilise a Delphi approach to develop best-practice consensus recommendations from international experts for the real-world application of TTC in severe TBI with ICP guided treatments.

Review of the literature and evidence quality assessment

Statements and questions were informed by a systematic review of the literature, which identified observational studies, meta-analyses and randomised controlled trials (RCTs) relevant to the topics under discussion. This review search focused on evidence released since 2013. Following this first review, the methodology group of ESICM conducted an independent systematic review of the literature, considering only published RCTs regarding TTC in TBI patients with ICP monitoring. This review confirmed the paucity of RCTs and the substantial clinical heterogeneity between them, which precluded meta-analytical combination. The outputs from the reviews were shared with the expert panel members ahead of the Delphi process. A detailed reporting of the literature reviews is provided as Additional files 1 and 4 .

Participants

The 18 expert attendees for the Delphi process were chosen from members of three professional societies: the Neuro Anaesthesia and Critical Care Society (NACCS), the European Society of Intensive Care Medicine (ESICM), and the European Society of Anaesthesiology and Intensive Care (ESAIC). Selection was based on a documented history of publications in the fields of traumatic brain injury and/or targeted temperature management, as well as their established professional profiles and expertise as leading intensive care practitioners in teaching university hospitals. We endeavoured to ensure balanced representation, covering the geographic areas of the EU, Switzerland, and the UK.

Delphi rounds

A modified Delphi consensus method was employed, involving a combination of an online survey (Round 1), a face-to-face meeting (Round 2), an additional online survey containing the refined questions from the previous steps, (Round 3) and post-meeting reviews of the consensus results. The questions asked at Round 1 can be found in the Additional file 2 , and the results following Round 3 are shown in Table 1 . Round 1 was conducted via the SmartSurvey® online platform, and Round 2 was held as a hybrid meeting in London, UK, on Tuesday 10th October 2023. AL acted as Chair, with an independent facilitator (ES) moderating the meeting. After the results from the final survey of Round 3 were received, the recommendations and final manuscript were developed, with documents shared by e-mail and feedback collected independently from each participant by the facilitator. The predefined agreed cut-off for strong consensus was to have ≥ 16 out of 18 (≥ 88%) of panel members in agreement, and for moderate consensus was to have ≥ 14 out of 18 (≥ 78%) of panel members in agreement. The Delphi methodology and process was adopted from the manuscript published by Lavinio et al. [ 17 ]. In a Delphi process, conflicting opinions are addressed through a structured framework that promotes consensus-building among experts. Initially, participants are asked to provide their views anonymously, which are then summarised and shared with the group. This approach facilitates open and unbiased input, as the anonymity helps mitigate the influence of dominant personalities or hierarchical pressures. When conflicting opinions emerge, they are documented and presented back to the participants, along with any common ground that has been identified. In subsequent rounds, individuals are encouraged to reconsider their positions in light of the collective feedback, which often leads to a convergence of opinions. If discrepancies persist, these are explored through further iterative rounds, with an emphasis on clarifying rationale and seeking areas of agreement. The Delphi method's iterative nature, combined with the feedback mechanism, effectively manages conflicting opinions by fostering a gradual move towards consensus, or at least a clearer understanding of the points of divergence. The process for the Delphi panel and subsequent manuscript development is visualised in Fig. 1 . A detailed overview of the iterative Delphi process is provided in the Additional files 2 and 3 .

Summary of the Delphi process. ESAIC European Society of Anaesthesiology and Intensive Care, ESICM European Society of Intensive Care Medicine, NACCS Neuro Anaesthesia and Critical Care Society

Definitions

To guide discussions during the Delphi process, clinical terms were defined with the values as shown below.

Declarations and conflicts of interest

The face-to face meeting in London was supported by Becton, Dickinson and Company (“BD”) through the provision of travel costs, meeting space and refreshments. Representatives from BD were allowed to silently observe the conference, without any interaction with the panellists or the process. No donors or other outside parties influenced any portion of these recommendations. There was no industry input into recommendation development, and no panel member received honoraria for their involvement. Panellists completed conflict of interest forms relevant to TBI management. There were no conflicts mandating recusal of any participant. No funding was provided by the societies involved.

The results of the final consensus are presented in Table 1 . We highlight and expand upon statements in which consensus was reached in the discussion section. Some consideration is added to statements in which consensus was not reached, proposing them as potential areas for valuable future research.

To date, there is a lack of definitive evidence regarding the use of TTC with an automated feedback-controlled device for managing temperature in severe TBI. This underlines the importance of consensus discussion in identifying areas of uncertainty where evidence is lacking, and in encouraging harmonised care delivery across different settings.

Pathophysiology

Temperature measurement and control is an essential aspect of high-quality care in patients with severe TBI

In patients with impending cerebral herniation, temperature control is essential

As an introduction to the discussions, the group debated the recommendation for temperature measurement and control following severe TBI and, after extensive discussion, concluded that core temperature measurement and control is essential for the provision of high-quality care, especially in patients perceived to be at high risk of secondary brain injury. Noting the phrasing of ‘temperature control’ in the recent guidelines for temperature control following cardiac arrest [ 18 ], the group agreed that as an entry point into high-quality care following TBI, the notion of temperature measurement and control is key, opening the door to the full practice of targeted temperature management. This nuanced phrasing was intended to set the scene for the group’s work, with the specifics of the TTC process such as temperature ranges and duration of control being addressed throughout the remainder of the discussions.

Highlighting the wealth of physiological data available on the management of temperature in stroke and cardiac arrest, the group noted that the guidelines for temperature management in TBI are less specific. Fundamentally, the group agreed that high-quality TBI care does include monitoring temperature and implementing some form of temperature control, recognising its potential role in optimising outcome. The group highlighted the importance of treatment titration based on an individualised risk–benefit assessment and stratification. In particular, it was noted that in patients with exhausted intracranial compensatory reserve and at risk of cerebral herniation or ischaemia—there exists an extreme susceptibility to secondary brain injury precipitated by suboptimal temperature control.

Cerebral herniation is a life-threatening event that requires early diagnosis and prompt management in order to prevent irreversible pathological cascades that can lead to death [ 19 ]. Increases in brain temperature have been linked to a linear rise in ICP, with the relationships between temperature, ICP and cerebral perfusion pressure (CPP) becoming more apparent with rapid temperature changes. The impact of temperature on ICP supports the recommendation from the group that temperature control is an essential aspect of care in patients at risk of herniation [ 20 ]. The group agreed that while control of ICP and prevention of herniation were important reasons for TTC in TBI, benefits of TTC in the acute phase of TBI also extended to patients without intracranial hypertension.

During the discussions the group highlighted that different pathologies often dictate different patient management. For example, patients in whom fluctuations in ICP are well-tolerated (e.g., patients with high intracranial compliance) will be managed differently to patients with obliterated basal cisterns, obliterated cortical sulci, and midline shift (e.g., intracranial mass effect). In patients with exhausted intracranial volume-buffering reserve, strict control of physiological parameters such as CO 2 and temperature, is strongly recommended.

Continuous temperature monitoring is preferable over intermittent temperature measurements in patients with severe TBI.

Monitoring core temperature (e.g., bladder, oesophageal, brain) is strongly recommended over measuring or monitoring superficial temperature (e.g., skin, tympanic) in severe TBI.

When brain temperature monitoring is in place, it is advisable to assess an additional source of core temperature monitoring (i.e. oesophageal, bladder).

The group widely agreed, in line with supporting literature, that continuous temperature monitoring is preferable over intermittent temperature measurements with severe TBI. Intermittent monitoring and recording of temperature can result in large fluctuations in temperature being missed, as highlighted by supporting literature investigating the use of TTC following cardiac arrest, TBI and stroke [ 17 , 21 , 22 ].

Discussions amongst the group drew attention to the fact that inaccurately measured temperatures can negatively impact patient care and outcome. Several temperature monitoring sites are available for TTC, and the group widely agreed that core temperature measurements, i.e., bladder and oesophageal sites, are strongly preferred over superficial measurements such as those taken at skin and tympanic sites. Following acknowledgement of their limitations [ 23 ], bladder and oesophageal were singled out as favoured core temperature measurements. The group acknowledged the widespread use of oesophageal probes due to their relative ease of insertion and the challenges of finding MRI compatible bladder probes. Confirmation of preference between the two was acknowledged as being beyond the scope of the group due to these nuances. Rectal temperature monitoring was widely regarded as impractical for reasons such as the lag time and a high rate of dislocation [ 16 , 23 ]. Peripheral sites were unanimously deemed to be insufficiently accurate to guide temperature treatment [ 16 ].

Some panel members argued that monitoring target organ (i.e. brain) temperature could add a layer of clinical safety, improve pathophysiological understanding and allow selective and individualised titration of treatment (i.e. selective brain cooling). It was, however, agreed by the group that more research is needed into optimum methods for measuring brain temperature and its interpretation from both a clinical and resource-availability perspective. In particular, it was highlighted that temperature thresholds for harm are less well defined for brain temperature than core temperature. When brain temperature monitoring is available and in place, the group advised that core temperature should also be assessed with bladder or oesophageal probes since this is part of routine practice and has been studied to a greater extent than brain temperature. The group noted the importance of having a dual source of temperature monitoring when using automated TTC devices to reduce the risk of probe malfunction and subsequent over or undercooling [ 24 ].

After TBI, brain temperature has often been shown to be higher than systemic temperature and can vary independently, with literature noting a difference of as much as 2 °C depending on the individual characteristics of brain pathology and/or probe location, making a consistent and accurate link between the two challenging and possibly inaccurate [ 25 , 26 ]. The group highlighted that targeting brain temperature may allow precise titration of treatment dose, including titration of selective brain cooling with brain temperature management technologies, theoretically reducing side effects associated with systemic hypothermia, whilst delivering neuroprotection and brain temperature management. However, it was concluded that further research is needed in this regard and that not enough evidence exists to support practical recommendations.

ICP management

Temperature control is a key component of ICP management in severe TBI.

Controlled normothermia (i.e., target core temperature 36.0–37.5 °C) should be included as an addition to the Tier 1 and Tier 2 treatments defined within the Seattle International Severe Traumatic Brain Injury Consensus Conference (SIBICC) 2019 guidelines.

Therapeutic hypothermia (i.e., target core temperature ≤ 36.0 °C) should be considered in cases where tier 1 and 2 treatments (as per SIBICC guidance) have failed to control ICP.

If hypothermia is considered to control ICP, target temperature should be managed as close to normothermia as possible.

ICP monitoring remains a critical component in the management of severe TBI [ 27 , 28 ]. The group unanimously agreed that temperature control is a key aspect of managing ICP, highlighting that an increase in temperature can lead to an increase in cerebral metabolism and augmented cerebral blood flow, and a simultaneous increase in cerebral blood volume. In cases of exhausted compensatory mechanisms, these factors can precipitate intracranial hypertension [ 20 ], which in turn can have a deleterious effect on overall outcome.

Because there is often no single pathophysiological pathway of ICP elevation, its management is complex. The most recent versions of the Brain Trauma Foundation TBI guidelines do not contain treatment protocols, in part due to a lack of solid evidence around the relative efficacy of available interventions [ 27 ]. To address this, the Seattle International Severe Traumatic Brain Injury Consensus Conference (SIBICC) developed a consensus-based practical algorithm for tiered management of severe TBI guided by ICP measurements [ 28 ].

One of the most impactful outcomes from this consensus meeting was the acknowledgement of the essential role of temperature control for ICP management in severe TBI, and the recommendation that controlled normothermia (i.e., target core temperature 36.0–37.5 °C) should be considered in addition to Tier 1 and Tier 2 treatments. The group was keen to harmonise this output with SIBICC by suggesting a more aggressive and specific management with the addition of controlled normothermia in Tiers 1 and 2, adding a layer of clinical safety beyond merely the avoidance of fever over 38.0 °C in Tier 0, as shown in Fig. 2 . In cases when hypothermia is considered (i.e., SIBICC Tier 3), the group recommended that target temperature be managed as close to normothermia as possible, based on an individualised risk–benefit assessment [ 29 ].

Intracranial pressure management algorithm for severe TBI edited from SIBICC 2019 [ 28 ]. * Including TTC in tiers 1 and 2 is the suggested addition from the TTC-TBI group to the original SIBICC tiers (green bars). *When possible, the lowest tier should be used. It is not necessary to use all modalities in a previous tier before moving to the next tier. Consider repeat CT and surgical options for space occupying lesions. CPP cerebral perfusion pressure, CT computed tomography, EEG electroencephalography, Hb haemoglobin, kPa kilopascal, mmHg milimetre of mercury, PaCO 2 arterial partial pressure of carbon dioxide, SpO 2 arterial oxygen saturation

No consensus was reached on whether hypothermia was a viable temporising strategy in patients with impending cerebral herniation, in patients awaiting haematoma evacuation or decompression, or before consideration of barbiturate coma. Whilst the group acknowledged that therapeutic hypothermia can be effective in reducing ICP, there was no consensus on whether this could be induced rapidly enough in these circumstances, and it was felt that insufficient evidence was available to provide pragmatic recommendations on its indication in these extreme clinical circumstances.

Whilst the majority of experts indicated 35.0 °C as the lowest target temperature to be considered in these circumstances, no consensus was reached. The discussion highlighted that insufficient evidence exists to support practical recommendations and highlighted the importance of an individualised risk–benefit assessment. It was also noted that centres might have a varying degree of familiarity with different therapeutic options, including ease of access to neurosurgical options (i.e. ventricular drainage, decompression) and this may have an impact on clinician preference for hypothermia as a temporising therapeutic modality.

The group also discussed the indication of barbiturates in the context of ICP control following severe TBI, not reaching consensus on whether therapeutic hypothermia should be attempted before considering barbiturates. The group noted that both barbiturate-induced burst-suppression and therapeutic hypothermia have distinctive side effects and concluded that no recommendations for standard clinical practice could be made beyond what was already stated in SIBICC guidance.

Neurogenic fever (core temperature > 37.5 °C) driven by neurological dysregulation in the absence of sepsis or a clinically significant systemic inflammatory process is relatively common in TBI, and it should be promptly detected and treated (i.e., with controlled normothermia targeting 36.0 °C to 37.5 °C), irrespective of ICP level.

Controlled normothermia should be considered when pyrexia is secondary to sepsis or inflammatory processes, and when the patient is perceived to be at risk of secondary brain injury, especially in the acute phase of TBI.

Uncontrolled fever (neurogenic or secondary to inflammation or infection) can precipitate secondary brain injury in patients with severe TBI.

It was widely agreed that neurogenic fever, defined here as core temperature > 37.5 °C driven by neurological dysregulation in the absence of sepsis or a clinically significant inflammatory process is common in intensive care and it has been found to be associated with an increased risk of complications and unfavourable outcome [ 9 , 14 , 15 ]. In the setting of neurogenic fever developing in comatose patients with acute traumatic encephalopathies, controlled normothermia targeting 36.0–37.5 °C was recommended in tier 1 and 2 of the ICP management algorithm.

Correctly differentiating central fever against fever of infectious origin is both challenging and clinically important due to the impact of failing to identify a treatable condition, the negative consequences of antibiotic overuse, and the detrimental effect of hyperthermia on brain-injured patients [ 17 , 30 , 31 ]. However, the group noted that physiological processes such as brain metabolic rate of oxygen, CO 2 control, brain tissue oxygenation (P bt O 2 ) and ICP are directly related to temperature, and that the deleterious effects and likelihood of secondary injury may occur irrespective of whether temperature is raised due to infection or impaired thermoregulation. This therefore highlights the need for acute management of temperature regardless of the source of the pyrexia, although added focus must be placed on the management of nuanced patient characteristics such as those with severe TBI with impending herniation and/or obliterated basal cisterns, as opposed those with low ICP and preserved intracranial compliance.

In line with current research [ 9 , 11 , 32 ], it was agreed that the development of fever is common in TBI cases, and that it can precipitate secondary brain injury and adversely affect patient outcome. It is therefore of utmost importance to prevent or promptly treat fever when detected. The group agreed that while some degree of controlled pyrexia may be allowed during the subacute phase of disease, ‘uncontrolled’ fever requires urgent management in the acute phase as long as the patient is still perceived to be at significant risk of secondary brain injury.

Fever control is recommended in patients with severe TBI who have seizures or are perceived to be at high risk of seizures.

In patients with severe TBI who are sedated and ventilated, controlled normothermia, irrespective of ICP, should be initiated reactively when fever is detected.

When neurogenic fever is detected in TBI cases, controlled normothermia should be continued for as long as the brain remains at risk of secondary brain damage.

The group strongly recommended that fever control and controlled normothermia are of particular relevance in patients perceived to be at high risk of seizures and, more in general, secondary brain injury. The assessment of whether an individual patient should be considered ‘at risk of seizures’ or ‘at risk of secondary brain injury’ remains the responsibility of the managing physician. The group defined risk factors for seizures as a history of seizures, the presence of temporal contusions or depressed skull fractures. Features associated with a higher ‘risk of secondary brain injury’ included labile ICP, obliterated basal cisterns, midline shift or subfalcine herniation, and other signs of exhausted intracranial volume buffering reserve. While no consensus was reached on a specific temperature range to target during controlled normothermia, the group agreed that the reactive initiation of temperature control was important in sedated and ventilated TBI patients, with agreement on a pragmatic setting of a target core temperature range of 36.0–37.5 °C to accommodate expected fluctuations of ± 0.5 °C while avoiding spikes over 38.0 °C [ 28 ].

Hypothermic TTC induction

It is recommended that the rapid induction of hypothermia in traumatic brain injury cases should be achieved with automated feedback-controlled temperature management devices.

In line with current research [ 17 ], the group widely agreed on the reactive use of an automated feedback-controlled device for the application of optimal TTC. The TTC process can be divided into three phases: induction, maintenance, and rewarming [ 9 , 16 ]. As explained in existing literature, varying availability of devices and financial aspects may dictate choice, and while non-automated methods of temperature control are cheaper and easier to apply, the level of control offered is poor and their use should be limited to the induction phase, as adjuncts to automated devices. [ 17 , 33 ] Whilst antipyretics such as acetaminophen (paracetamol) or nonsteroidal anti-inflammatory drugs (NSAIDs) are widely acknowledged in intensive care unit (ICU) settings for their role in fever management, it is recognised that in the context of severe TBI, the efficacy of antipyretics in controlling fever and minimising temperature variability is limited. The application of therapeutic hypothermia requires constant monitoring of core body temperature in order to achieve an accurate target temperature during induction to prevent overcooling, to assess variations during the maintenance phase, and to ensure a steady, controlled rewarming phase [ 16 ].

There was no agreed recommendation from the group as to whether ICUs should stock readily available ice-cold NaCl solutions of different concentrations for the management of ICP crises, citing a lack of clear evidence to draw upon. The group did however highlight the fact that the rapid infusion of ice-cold saline is an inexpensive and readily available option for lowering core body temperature [ 9 ], with the rapidity of response to ice-cold infusions being regarded as a valuable aspect of TTC induction.

TTC maintenance

An automated feedback-controlled TTC device that enables precise temperature control is desirable for the initiation of TTC and maintenance at target temperature in patients with severe TBI.

The maximum temperature variation that a patient should experience during normothermia is less than or equal to +/− 0.5 °C per hour and ≤ 1 °C per 24-hperiod

When hypothermia is indicated, treatment should be continued for as long as the brain is considered to be at risk of secondary brain injury.

Automated feedback-controlled devices for TTC are powerful tools, encouraging the delivery of quality care and aiming to improve neurological outcome [ 13 , 17 ], minimising the chances of temperature variability. Temperature variability is the deviation of patient temperature outside of the goal, typically reported as mean deviation or percent of time outside of target [ 9 ]. The group noted that there is a level of pragmatism to be adopted in TTC maintenance, discussing that while more time spent in fever can negatively impact neurological outcome, fluctuations in temperature may also affect outcome [ 17 ], and consensus was reached on the importance of maintaining temperature at as consistent a level as possible with the group settling on a fluctuation range of less than or equal to ± 0.5 °C per hour and ≤ 1 °C per 24-h period. In instances where an automated feedback-controlled device is not available, the group noted the importance of increased staff awareness of patient status to ensure fluctuations outside of this range are appropriately managed. The group highlighted that a dedicated protocol for sedation, analgesia and shivering management might be helpful to ensure consistent application of optimal TTC.

The group agreed that when indicated, hypothermia should be continued for as long as the individual practitioner considers the brain to be at risk of secondary injury. These considerations were supported with a suggestion that it should be maintained for as short a time as possible.

Rewarming following hypothermic TTC

Obtaining an interval scan and/or an alternative assessment of intracranial compliance, in addition to the absolute number of ICP, is recommended before rewarming.

Rebound hyperthermia should be prevented whenever possible or promptly treated in cases when the brain is perceived to be at risk of secondary brain injury.

In cases in which the patient is being rewarmed from therapeutic hypothermia (core temperature lower than 36.0 °C), the group agreed that once ICP has been maintained within controlled limits and de-escalation of treatment intensity is considered, it is sensible to ensure the patient has sufficient intracranial volume buffering reserve through the use of an interval scan and/or an alternative measure of intracranial compliance, before commencing the rewarming process. The group also noted the high prevalence and potential risks associated with rebound hyperthermia when TTC is discontinued following therapeutic hypothermia, highlighting the importance of continued vigilance and careful temperature control in the rewarming phase.

Whilst no consensus was reached on recommended rewarming rates, the group agreed that controlled rewarming with an automated feedback-controlled device may reduce the risk of rapid temperature variations and rebound pyrexia that can precipitate secondary brain injury and compromise care [ 16 , 33 ]. The group highlighted how controlled rewarming may improve the ability of clinicians to more effectively control important inter-dependent clinical variables such as PaCO 2 , ventilation settings and depth of sedation.

TTC for shivering

It is important to assess, document and manage shivering in severe TBI patients.

Whenever ICP is labile and shivering is detected, neuromuscular blockers should be considered after ensuring appropriate depth of sedation.

In self-ventilating patients in the subacute phase of severe TBI, an individualised risk–benefit assessment should be undertaken regarding the strict indications of controlled normothermia.

Permissive hyperthermia should be considered in cases where risk of secondary brain injury resulting from pyrexia is thought to be low, and when shivering cannot be controlled with first line treatments such as NSAIDs, opiates, magnesium or counter warming.

In line with current literature, it was widely agreed that shivering should be managed in patients following severe TBI. Shivering can reduce brain tissue oxygenation leading to cerebral metabolic stress, which may therefore negate the neuroprotective benefits of TTC [ 9 , 34 , 35 , 36 ].

Titration of sedation and the use of neuromuscular blocking agents provides intensivists with readily available and effective options for shivering control in critically ill patients [ 37 ]. To ensure appropriate and effective use however, treating staff must be aware of the nuances of selecting the correct agent, monitoring the depth of neuromuscular blockade, and ensuring adequate skeletal muscle recovery once therapy with neuromuscular blockers has ceased. In cases of shivering when ICP is labile, the group agreed in line with current literature that ensuring depth of sedation before administering neuromuscular blockers is of utmost importance [ 37 , 38 ]. When using pharmacologic agents for shivering management, treating staff must consider potential pharmacokinetic and pharmacodynamic variation and monitor for efficacy (i.e. shivering control) and safety (i.e. adverse events and drug-drug interactions) [ 9 ].

The group agreed that in patients who are perceived to be at relatively lower risk of secondary brain injury (i.e. self-ventilating patients in the sub-acute phase of severe TBI), permissive hyperthermia may be considered over TTC, especially if the latter therapeutic option would require sedation or other invasive interventions. The group agreed that an individualised risk–benefit assessment should ultimately be undertaken before commencing controlled normothermia in such patients.

‘Time within target range’, ‘burden of fever’ and similar metrics can be considered as indicators of quality of temperature management.

‘Time within target range’ and ‘burden of fever’ were considered by the group to be appropriate metrics of quality temperature management. It was widely acknowledged that these metrics should be weighed by patient length of stay and/or duration of monitoring for appropriate statistical interpretation. The group was also careful to note that the administrative burden on physicians is already high and acknowledged the fact that some centres may not have access to electronic patient data management systems, so it was agreed that it was unrealistic for this group to issue prescriptive recommendations on auditing practices. In light of the high heterogeneity across centres [ 9 ], here the group were keen to clarify that wherever possible, documenting metrics such as ‘time within target range’ and ‘burden of fever’ may improve their ability to deliver data-driven service improvement and temperature control.

This consensus review was undertaken to evaluate current evidence on the application of TTC in the management of severe TBI in a critical care setting, and to develop a set of practical recommendations to address identified gaps in current published evidence.

As highlighted by the SIBICC 2020 group, the gap between published evidence and management protocols is bridged by expert opinion [ 39 ]. The optimal method for the provision of high-quality TTC remains unknown, and barriers to its consistent implementation include the lack of evidence-based treatment protocols, knowledge deficiencies, limited access to equipment, lack of financial resources and staff workload. This document aims to address key practice gaps and optimise patient care through multimodal assessment following TBI.

Strengths and limitations

The Delphi process has a number of strengths. Participants are able to reconsider their views in light of the evolving discussions, allowing for an element of reflection that isn’t regularly seen in other studies involving a single time point such as interviews or focus groups [ 40 ]. The element of anonymity offered to the panellists in the survey rounds avoids group conformity and promotes honesty, and the controlled and iterative discussions offer a flexible approach to gathering expert viewpoints on the set research questions. The Delphi method is an iterative process allowing the anonymous inclusion of a number of individuals across diverse locations and areas of expertise and avoiding dominance by any one individual. It uses a systematic progression of repeated rounds of voting and is an effective process for determining expert group consensus where there is little or no definitive evidence and where opinion is important [ 41 , 42 ]. The modified Delphi approach used here combined the early flow of structured information and submission of anonymous responses with the (hybrid) face-to-face discussion and further voting to gain consensus (or establish lack thereof) and expert insight into usual practice regarding non-pharmacological TTC with an automated feedback-controlled device. As cited in existing literature however [ 13 , 17 ], the Delphi process has limitations. The process is vulnerable to drop-outs and technical issues, with the online voting process during our meeting seeing some participants unable to cast their votes on a number of questions, leading to the need for a final anonymous survey round. The group opinions during the meeting may have been impacted by social bias, and the voices across the in-person and online participants may not have been equally heard, highlighting a potential need to ensure consistency in attendance in the same format in future panel meetings.

Our recommendations for the use of automated feedback-controlled TTC devices are based on expert consensus and theoretical benefits, such as precise temperature control and reduced temperature variability, which are thought to potentially improve outcomes in severe TBI management. We acknowledge the current evidence gap and strongly emphasise the need for rigorous research to evaluate the effectiveness of these devices, especially in diverse healthcare settings, including lower-income countries where resource limitations are critical. Future updates to these best-practice recommendations will incorporate emerging evidence to ensure relevance and applicability across different healthcare contexts, aiming for the highest standards of care within the constraints of available resources. While automated feedback-controlled TTC devices represent a significant advancement in the management of temperature in severe TBI patients, offering potential benefits in terms of precision and consistency, it is imperative to recognise the value and applicability of a wide range of temperature management approaches. These include both manual methods and simpler devices, which remain vital in many clinical settings around the world. Our guidelines advocate for the adaptation and implementation of TTC principles based on the specific resources, capabilities, and needs of each clinical setting.

This report has been developed by an expert panel comprised of specialists in neuro-critical care experienced in the management of severe TBI, therefore the recommendations focus on patients managed in a critical care environment. An individualised risk–benefit assessment should be undertaken for each domain to accommodate the high levels of heterogeneity seen across TBI patients, local practice settings, staff training and equipment availability [ 9 ].

TTC is a therapy that has a role in ICP management and may reduce secondary injury and improve long-term neurological outcome for victims of TBI [ 9 ]. Appropriate methods for the implementation of TTC across widely heterogenous clinical settings and patient populations are relatively understudied, and due to a lack of consistent and high-quality evidence, remain largely unknown. Areas of consensus emerging from the Delphi process included TTC being recognised as an essential aspect of high-quality TBI care. Controlled normothermia (36.0–37.5 °C) was strongly recommended as a therapeutic option to be considered in Tier 1 and 2 of the SIBICC ICP management protocol. Temperature management targets should be individualised based on the perceived risk of secondary brain injury and fever aetiology.

Availability of data and materials

All data generated or analysed during this study are included in this article and its supplementary information files.

Abbreviations

Cerebral perfusion pressure

Computed tomography

Electroencephalography

European Society of Anaesthesiology and Intensive Care

European Society of Intensive Care Medicine

Haemoglobin

Intracranial pressure

Intensive care unit

Neuro Anaesthesia and Critical Care Society

Sodium chloride

Neuro-intensive care unit

Nonsteroidal anti-inflammatory drugs

Arterial partial pressure of carbon dioxide

Brain tissue oxygenation

Randomised controlled trial

Seattle International Severe Traumatic Brain Injury Consensus Conference

Arterial oxygen saturation

Traumatic brain injury
Targeted temperature control

Centers for Disease Control and Prevention: Moderate to Severe Traumatic Brain Injury is a Lifelong Condition; 2023. https://www.cdc.gov/traumaticbraininjury/pdf/moderate_to_severe_tbi_lifelong-a.pdf . Accessed Dec 2023.

Center TBI: Traumatic Brain Injury Fact sheets and Policy brief. Center TBI. https://www.centertbi.eu/files/news/21571f81-20b8-4860-a3dd-1f6e27d02b3d.pdf . Accessed Dec 2023.

Vrettou CS, Mentzelopoulos SD. Second-and third-tier therapies for severe traumatic brain injury. J Clin Med. 2022;11(16):4790.

Article CAS PubMed PubMed Central Google Scholar

Maas AI, Menon DK, Manley GT, Abrams M, Åkerlund C, Andelic N, Aries M, Bashford T, Bell MJ, Bodien YG, Brett BL. Traumatic brain injury: progress and challenges in prevention, clinical care, and research. Lancet Neurol. 2022;21(11):1004–60.

Article PubMed PubMed Central Google Scholar

Menon DK, Schwab K, Wright DW, Maas AI. Position statement: definition of traumatic brain injury. Arch Phys Med Rehabil. 2010;91(11):1637–40.

Article PubMed Google Scholar

Wu X, Tao Y, Marsons L, Dee P, Yu D, Guan Y, Zhou X. The effectiveness of early prophylactic hypothermia in adult patients with traumatic brain injury: a systematic review and meta-analysis. Aust Crit Care. 2021;34(1):83–91.

Hopkins Medicine: Traumatic Brain Injury; 2023. https://www.hopkinsmedicine.org/health/conditions-and-diseases/traumatic-brain-injury . Accessed Nov 2023.

The BMJ: Rapid response to: Prehospital management of severe traumatic brain injury; 2009. https://www.bmj.com/rapid-response/2011/11/02/classifying-types-brain-injury . Accessed Dec 2023.

Madden LK, Hill M, May TL, Human T, Guanci MM, Jacobi J, Moreda MV, Badjatia N. The implementation of targeted temperature management: an evidence-based guideline from the Neurocritical Care Society. Neurocrit Care. 2017;27:468–87.

Neurological Institute of Neurological Disorders and Stroke: Traumatic Brain Injury (TBI); 2023. https://www.ninds.nih.gov/health-information/disorders/traumatic-brain-injury-tbi . Accessed Dec 2023.

Hinson HE, Rowell S, Morris C, Lin AL, Schreiber MA. Early fever after trauma: does it matter? J Trauma Acute Care Surg. 2018;84(1):19–24.

Greer DM, Funk SE, Reaven NL, Ouzounelli M, Uman GC. Impact of fever on outcome in patients with stroke and neurologic injury: a comprehensive meta-analysis. Stroke. 2008;39(11):3029–35.

Andrews PJ, Verma V, Healy M, Lavinio A, Curtis C, Reddy U, Andrzejowski J, Foulkes A, Canestrini S. Targeted temperature management in patients with intracerebral haemorrhage, subarachnoid haemorrhage, or acute ischaemic stroke: consensus recommendations. Br J Anaesth. 2018;121(4):768–75.

Article CAS PubMed Google Scholar

Muengtaweepongsa S, Yodwisithsak P. Targeted temperature management in traumatic brain injury. intraumatic brain injury-pathobiology, advanced diagnostics and acute management 2017 Dec 20. IntechOpen.

Cariou A, Payen JF, Asehnoune K, Audibert G, Botte A, Brissaud O, Debaty G, Deltour S, Deye N, Engrand N, Francony G. Targeted temperature management in the ICU: guidelines from a French expert panel. Ann Intensive Care. 2017;7:1–4.

Article Google Scholar

Omairi AM, Pandey S. Targeted temperature management. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing. 2023.

Lavinio A, Andrzejowski J, Antonopoulou I, Coles J, Geoghegan P, Gibson K, Gudibande S, Lopez-Soto C, Mullhi R, Nair P, Pauliah VP. Targeted temperature management in patients with intracerebral haemorrhage, subarachnoid haemorrhage, or acute ischaemic stroke: updated consensus guideline recommendations by the Neuroprotective Therapy Consensus Review (NTCR) group. Br J Anaesth. 2023.

Sandroni C, Nolan JP, Andersen LW, Böttiger BW, Cariou A, Cronberg T, Friberg H, Genbrugge C, Lilja G, Morley PT, Nikolaou N. ERC-ESICM guidelines on temperature control after cardiac arrest in adults. Intensive Care Med. 2022;48(3):261–9.

Munakomi S. Brain herniation. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing. 2023.

Birg T, Ortolano F, Wiegers EJ, Smielewski P, Savchenko Y, Ianosi BA, Helbok R, Rossi S, Carbonara M, Zoerle T, Stocchetti N. Brain temperature influences intracranial pressure and cerebral perfusion pressure after traumatic brain injury: a CENTER-TBI study. Neurocrit Care. 2021;35:651–61.

Taccone FS, Picetti E, Vincent JL. High quality targeted temperature management (TTM) after cardiac arrest. Crit Care. 2020;24:1–7.

Jo KW. Target temperature management in traumatic brain injury with a focus on adverse events, recognition, and prevention. Acute Crit Care. 2022;37(4):483.

Paal P, Pasquier M, Darocha T, Lechner R, Kosinski S, Wallner B, Zafren K, Brugger H. Accidental hypothermia: 2021 update. Int J Environ Res Public Health. 2022;19(1):501.

Lavinio A. Therapeutic hypothermia: heat transfer from warmed endotracheal tubes to oesophageal temperature probes poses risk of life-threatening overcooling. Br J Anaesth. 2011;107(eLetters).

Mrozek S, Vardon F, Geeraerts T. Brain temperature: physiology and pathophysiology after brain injury. Anesthesiol Res Pract. 2012;2012.

Rossi S, Zanier ER, Mauri I, Columbo A, Stocchetti N. Brain temperature, body core temperature, and intracranial pressure in acute cerebral damage. J Neurol Neurosurg Psychiatry. 2001;71(4):448–54.

Brain Trauma Foundation: Guidelines for the Management of Severe TBI, 4th Edition; 2016. https://braintrauma.org/coma/guidelines/guidelines-forthe-management-of-severe-tbi-4th-ed . Accessed Dec 2023.

Hawryluk GW, Aguilera S, Buki A, Bulger E, Citerio G, Cooper DJ, Arrastia RD, Diringer M, Figaji A, Gao G, Geocadin R. A management algorithm for patients with intracranial pressure monitoring: the Seattle International Severe Traumatic Brain Injury Consensus Conference (SIBICC). Intensive Care Med. 2019;45:1783–94.

Hui J, Feng J, Tu Y, Zhang W, Zhong C, Liu M, Wang Y, Long L, Chen L, Liu J, Mou C. Safety and efficacy of long-term mild hypothermia for severe traumatic brain injury with refractory intracranial hypertension (LTH-1): a multicenter randomized controlled trial. EClinicalMedicine. 2021;1:32.

Google Scholar

Goyal K, Garg N, Bithal P. Central fever: a challenging clinical entity in neurocritical care. J Neurocrit Care. 2020;13(1):19–31.

Hocker SE, Tian L, Li G, Steckelberg JM, Mandrekar JN, Rabinstein AA. Indicators of central fever in the neurologic intensive care unit. JAMA Neurol. 2013;70(12):1499–504.

PubMed Google Scholar

Lee D, Ryu H, Jung E. Effect of fever on the clinical outcomes of traumatic brain injury by age. Medicina. 2022;58(12):1860.

Taccone FS, Donadello K, Mayer SA. Manipulating temperature: devices for targeted temperature management (TTM) in brain injury. Intensive Care Med. 2022;48(10):1409–12.

Jain A, Gray M, Slisz S, Haymore J, Badjatia N, Kulstad E. Shivering treatments for targeted temperature management: a review. J Neurosci Nurs. 2018;50(2):63–7.

Choi HA, Ko SB, Presciutti M, Fernandez L, Carpenter AM, Lesch C, Gilmore E, Malhotra R, Mayer SA, Lee K, Claassen J. Prevention of shivering during therapeutic temperature modulation: the Columbia anti-shivering protocol. Neurocrit Care. 2011;14:389–94.

Badjatia N, Strongilis E, Gordon E, Prescutti M, Fernandez L, Fernandez A, Buitrago M, Schmidt JM, Ostapkovich ND, Mayer SA. Metabolic impact of shivering during therapeutic temperature modulation: the Bedside Shivering Assessment Scale. Stroke. 2008;39(12):3242–7.

Renew JR, Ratzlaff R, Hernandez-Torres V, Brull SJ, Prielipp RC. Neuromuscular blockade management in the critically Ill patient. J Intensive Care. 2020;8:1–5.

Oddo M, Crippa IA, Mehta S, Menon D, Payen JF, Taccone FS, Citerio G. Optimizing sedation in patients with acute brain injury. Crit Care. 2016;20:1–1.

Chesnut R, Aguilera S, Buki A, Bulger E, Citerio G, Cooper DJ, Arrastia RD, Diringer M, Figaji A, Gao G, Geocadin R. A management algorithm for adult patients with both brain oxygen and intracranial pressure monitoring: the Seattle International Severe Traumatic Brain Injury Consensus Conference (SIBICC). Intensive Care Med. 2020;46:919–29.

Barrett D, Heale R. What are Delphi studies? Evid Based Nurs. 2020;23(3):68–9.

Meshkat B, Cowman S, Gethin G, Ryan K, Wiley M, Brick A, Clarke E, Mulligan E. Using an e-Delphi technique in achieving consensus across disciplines for developing best practice in day surgery in Ireland.

Eubank BH, Mohtadi NG, Lafave MR, Wiley JP, Bois AJ, Boorman RS, Sheps DM. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Med Res Methodol. 2016;16:1–5.

Download references

Acknowledgements

The group would like to acknowledge the support of Page & Page, London UK in facilitating the Delphi meeting.

The Delphi Panel meeting in October 2023 was facilitated (through the provision of travel costs, meeting space and refreshments) by Becton, Dickinson and Company. The development of these consensus recommendations was conducted with strict measures to ensure independence from its sponsor. The research team independently conducted all data analyses and drafted the manuscript. The role of BD was limited to providing logistical support for the Delphi panel meeting held in London, including travel costs, meeting space, and refreshments, without any influence over the study's content or conclusions. The Delphi voting process was conducted anonymously, ensuring that panel members could freely express their professional opinions without bias or influence from the sponsoring body or among panel members. The manuscript's drafting, review, and revision processes were carried out independently of BD. The sponsor had no editorial control, ensuring that the recommendations are based on the authors’ independent, professional expertise in targeted temperature management following traumatic brain injury. This article contains the personal and professional opinions of the individual authors and does not necessarily reflect the views and opinions of Becton, Dickinson and Company (“BD”) or any Business Unit or affiliate of BD. If drugs and/or medical devices are cited in the article, please consult package insert and instructions for use of them to know indications, contraindications, and any other more detailed safety information.

Author information

Authors and affiliations.

Department of Medicine, BOX 1 Addenbrooke’s Hospital, University of Cambridge, Long Road, Cambridge, CB2 0QQ, UK

Andrea Lavinio, Jonathan P. Coles & David K. Menon

Department of Anaesthesia and Critical Care, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK

IRCCS Policlinico San Martino, Genoa, Italy

Chiara Robba

Department of Intensive Care, Maastricht University Medical Center+, Maastricht, The Netherlands

Marcel Aries

School of Mental Health and Neurosciences, University Maastricht, Maastricht, The Netherlands

Inserm U1216, Department of Anesthesia and Critical Care, CHU Grenoble Alpes, Grenoble Institute Neurosciences, Université Grenoble Alpes, 38000, Grenoble, France

Pierre Bouzat & Jean-Francois Payen

Medical Intensive Care Unit, Saint-Louis Teaching Hospital, Paris, France

Department of Anaesthesia and Intensive Care, University Hospital of North Norway, Tromsö, Norway

Shirin Frisvold

Department of Clinical Medicine, UiT the Arctic University of Norway, Tromsö, Norway

Department of Intensive Care, Hospital General Universitario de Castellón, Castellón de la Plana, Spain

Laura Galarza

Department of Neurology, Kepler University Hospital, Johannes Kepler University, Linz, Austria

Raimund Helbok

Clinical Research Institute for Neuroscience, Johannes Kepler University, Linz, Austria

Department of Anaesthesiology, Amsterdam UMC, Amsterdam, The Netherlands

Jeroen Hermanides

Department of Intensive Care Adults, Erasmus MC, University Medical Centre, Rotterdam, The Netherlands

Mathieu van der Jagt

Department and Laboratory of Intensive Care Medicine, University Hospitals Leuven, Leuven, Belgium

Geert Meyfroidt

Anesthesia and Intensive Care Operative Unit, S. Martino Hospital, Belluno, Italy

Daniele Poole

Spedali Civili University Hospital of Brescia, Brescia, Italy

Frank Rasulo

Department of Anaesthesia, Critical Care and Pain Medicine, Royal Infirmary of Edinburgh, University of Edinburgh, Edinburgh, UK

Jonathan Rhodes

Page and Page Healthcare Communications, London, UK

Emily Sidlow

University Hospital Basel, Department of Clinical Research, University of Basel, Basel, Switzerland

Luzius A. Steiner

Department of Intensive Care, Brussels University Hospital, Brussels, Belgium

Fabio Silvio Taccone

Université Libre de Bruxelles, Brussels, Belgium

Perioperative Services, Intensive Care Medicine and Pain Management, Turku University Hospital, Turku, Finland

Riikka Takala

Department of Anaesthesiology and Intensive Care, University of Turku, Turku, Finland

You can also search for this author in PubMed Google Scholar

Contributions

All authors took part in the Delphi process. All authors read, revised and approved the manuscript.

Corresponding author

Correspondence to Andrea Lavinio .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

AL received consultancy and speaker fees from Beckton, Dickinson and Company (“BD”) for Chairing the Delphi panel and for contributing to the writing of the article. RH received speaker fees from BD and Zoll.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

. Evaluation of five randomized controlled trials by the ESICM Methodology Group evaluates evulating cooling strategies against traditional interventions. The evaluation highlights methodological heterogeneities and evidential challenges.

Additional file 2

. Delphi questionnaire: Round 1.

Additional file 3

. Delphi questionnaire. Round 3.

Additional file 4

. Systematic review of the literature on targeted temperature control in traumatic brain injury, covering clinical studies from 2013 to 2023.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lavinio, A., Coles, J.P., Robba, C. et al. Targeted temperature control following traumatic brain injury: ESICM/NACCS best practice consensus recommendations. Crit Care 28 , 170 (2024). https://doi.org/10.1186/s13054-024-04951-x

Download citation

Received : 08 April 2024

Accepted : 12 May 2024

Published : 20 May 2024

DOI : https://doi.org/10.1186/s13054-024-04951-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Normothermia

Critical Care

ISSN: 1364-8535

Submission enquiries: [email protected]

IMAGES

50 Smart Literature Review Templates (APA) ᐅ TemplateLab
Literature Review
examples of good literature review
50 Smart Literature Review Templates (APA) ᐅ TemplateLab
Research development based on quantitative...- Mind Map
how to write a chronological literature review

VIDEO

3_session2 Importance of literature review, types of literature review, Reference management tool
Essential Websites for Research!!!
Approaches to Literature Review
Literature review in research
How to Do a Good Literature Review for Research Paper and Thesis
A Comprehensive Guide to Systematic Literature Review (SLR)

COMMENTS

A Literature Review: Website Design and User Engagement
2.3. Analysis. The literature review uncovered 20 distinct design elements commonly discussed in research that affect user engagement. They were (1) organization - is the website logically organized, (2) content utility - is the information provided useful or interesting, (3) navigation - is the website easy to navigate, (4) graphical representation - does the website utilize icons ...
A Literature Review: Website Design and User Engagement
Ensuring the appropriate design of web-based resources is a critical component of website development 41 in which the use of iterative, co-designed methods is strongly advocated, 42 43 especially ...
(PDF) A Comprehensive Framework to Evaluate Websites: Literature Review
Allison et al. (2019) suggested a comprehensive website evaluation to improve the quality of websites. Besides, as claimed by Kettle et al. (2012), the evaluation results of ELL websites assisted ...
A Literature Review: Website Design and User Engagement
The design elements mentioned most frequently in the reviewed literature were navigation, graphical representation, organization, content utility, purpose, simplicity, and readability. We discuss how previous studies define and evaluate these seven elements. This review and the resulting short list of design elements may be used to help ...
How to Write a Literature Review
Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.
A Literature Review: Website Design and User Engagement.
A review and consolidate research on effective design and a short list of elements frequently used in research is defined to help designers and researchers to operationalize best practices for facilitating and predicting user engagement. Proper design has become a critical element needed to engage website and mobile application users. However, little research has been conducted to define the ...
A Literature Review: Website Design and User Engagement.
Author(s): Garett, Renee; Chiu, Jason; Zhang, Ly; Young, Sean D | Abstract: Proper design has become a critical element needed to engage website and mobile application users. However, little research has been conducted to define the specific elements used in effective website and mobile application design. We attempt to review and consolidate research on effective design and to define a short ...
13703 PDFs
Explore the latest full-text research PDFs, articles, conference papers, preprints and more on WEBSITE DEVELOPMENT. Find methods information, sources, references or conduct a literature review on ...
PDF A Literature Review: Website Design and User Engagement
The literature review uncovered 20 distinct design elements commonly discussed in research that affect user engagement. They were (1) organization - is the website logically organized, (2) content utility - is the information provided useful or interesting, (3) navigation - is the
PDF Conducting a Literature Review
Literature Review A literature review is a survey of scholarly sources that provides an overview of a particular topic. Literature reviews are a collection of the most relevant and significant publications regarding that topic in order to provide a comprehensive look at what has been said on the topic and by whom.
A Systematic Review of Adaptive and Responsive Design ...
The purpose of this Systematic Literature Review (SLR) is to investigate the two widely used web based designs i.e. Adaptive Web Design (AWD) and Responsive Web Design (RWD). Particularly, a systematic literature review has been used to identify 58 research works, published during 2009-2017.
A Comprehensive Framework to Evaluate Websites: Literature Review and
Background: Attention is turning toward increasing the quality of websites and quality evaluation to attract new users and retain existing users. Objective: This scoping study aimed to review and define existing worldwide methodologies and techniques to evaluate websites and provide a framework of appropriate website attributes that could be applied to any future website evaluations.
Development of an Approach to Evaluate Website Effectiveness
First, the literature review showed that there are 243 web elements/categories/factors to evaluate websites as proposed by different researchers. This number would increase if a further literature review was conducted. These elements were reviewed by the authors to avoid repetition, and the duplicated items were shortlisted.
Literature review: your definitive guide
Try our tips on the Web of Science now. 2. Identify key papers (and know how to use them) As you explore the Web of Science, you may notice that certain papers are marked as "Highly Cited.". These papers can play a significant role when you write a narrative literature review.
How To Write A Literature Review (+ Free Template)
Okay - with the why out the way, let's move on to the how. As mentioned above, writing your literature review is a process, which I'll break down into three steps: Finding the most suitable literature. Understanding, distilling and organising the literature. Planning and writing up your literature review chapter.
A LITERATURE REVIEW ON WEB DESIGN Directed Study
The following literature review will examine the theories, principles, and practices recommended for website planning and creation. Adopting a design strategy Most novices jump right into page construction with only a vague notion of a plan, which Nielsen (2000) considers one of the two great errors in site design.
Web application testing: A systematic literature review
In this paper, we present a systematic literature review (SLR) of the web application testing (WAT) research domain. In a recent work, we conducted a systematic mapping (SM) study ( Garousi et al., 2013) in which we reviewed 79 papers in the WAT domain.
Developing a Literature Review
According to the seventh edition of the APA Publication Manual, a literature review is "a critical evaluation of material that has already been published." As one embarks on creating a literature review, it is important to note that the grouping of components within a literature review can be arranged according to the author's discretion ...
Towards Adaptive and Responsive Web Design: A Systematic Literature Review
Detailed analysis of Responsive Web Design (RWD) and Adaptive Web Design (AWD) in terms of usage. A Systematic Literature Review (SLR) has been used to identify 58 research works, published during ...
Writing a Literature Review
The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say "literature review" or refer to "the literature," we are talking about the research (scholarship) in a given field. You will often see the terms "the research," "the ...
PDF LITERATURE REVIEWS
would go so far as to say that literature reviews are the quintessential site of identity work." Kamler, Barbara. and Pat Thomson. 2006. Helping Doctoral Students Write: Pedagogies for Supervision. London; New York: Routledge. p.29 The literature review is an opportunity to discover and craft your scholarly identity through the
Maternal depression and development of child executive functions
Yakupova, V. A., Anikeeva, M. A., & Suarez, A. D. (2024). Maternal depression and development of child executive functions: Foreign literature review. Theoretical and Experimental Psychology, 17(1), 123-142. Abstract. Background: An estimated 3 to 60% of mothers worldwide suffer from depression that often goes undiagnosed and untreated.
Best Web Development Bootcamp (2024): Udemy Complete Python Pro
Boston - May 23, 2024 — . Compare Before Buying, a trusted source for online reviews and comparisons, has published an in-depth analysis of Udemy's 100 Days of Code: The Complete Python Pro ...
Targeted temperature control following traumatic brain injury: ESICM
This review confirmed the paucity of RCTs and the substantial clinical heterogeneity between them, which precluded meta-analytical combination. The outputs from the reviews were shared with the expert panel members ahead of the Delphi process. A detailed reporting of the literature reviews is provided as Additional files 1 and 4. Participants
(PDF) A systematic review of e-commerce websites literature in 2010
As a topic searc hed on the Web of Science, E commerce has 17.945 papers between 2010 and 2020. However, 8.482 (47,3%) of these papers are articles that constitute the sample of this study ...
Site development Review 10 (docx)
Information-systems document from Wayne County Community College District, 1 page, Lesson 10 Review 1. User-friendly Web page interfaces, such as a pleasing layout and accessible navigation, are considered to be what type of issues? Front- end issues. 2. What is Internet marketing, and of what three components does it consist? Internet

A Literature Review: Website Design and User Engagement

Sean D. Young

1. INTRODUCTION

2. MATERIALS AND METHODS

2.2. Final Search Term

2.3. Analysis

3.1. Popular website design elements ( Table 1 )

3.2. Defining key design elements for user engagement ( Table 2 )

4. DISCUSSION

A Literature Review: Website Design and User Engagement

Grants and funding

A Literature Review: Website Design and User Engagement.

Published Web Location

A Systematic Review of Adaptive and Responsive Design Approaches for World Wide Web

Access this chapter

Similar content being viewed by others

WEBAPIK: a body of structured knowledge on designing web APIs

Paving the Path to Content-Centric and Device-Agnostic Web Design

Quality of Web Mashups: A Systematic Mapping Study

Author information

Corresponding author

Editor information

Rights and permissions

Copyright information

About this paper

Download citation

Share this paper

This paper is in the following e-collection/theme issue:

A Comprehensive Framework to Evaluate Websites: Literature Review and Development of GoodWeb

Corresponding Author:

Introduction

Identifying Relevant Studies

Study Selection

Charting the Data

Collating, Summarizing, and Reporting the Results

Ethics Approval and Consent to Participate

Study Characteristics

Intervention

Questionnaire

Observed Browsing the Website

Qualitative Data Collection

Automated Usability Evaluation Software

Card Sorting

Web Usage Analysis

Outcomes (Attributes Used to Evaluate Websites)

GoodWeb: Website Evaluation Guide

Step 1. What Are the Important Website Attributes That Affect User's Experience of the Chosen Website?

Application

Step 2. What Is the Best Way to Evaluate These Attributes?

Step 3: Who Should Evaluate the Website?

Step 4: What Setting Should Be Used?

Strengths and Limitations

Conclusions

Acknowledgments

Authors' Contributions

Conflicts of Interest

Abbreviations

Literature review: your definitive guide

How do you write a narrative literature review?

1. Don’t miss a paper: tips for a thorough topic search

Searching the Web of Science

2. Identify key papers (and know how to use them)

3. Tips for working with co-authors

Use EndNote for narrative reviews

4.Finding a journal for your literature review

5. Discover literature review examples and templates

Related posts

Introducing the Clarivate Academic AI Platform

Reimagining research impact: Introducing Web of Science Research Intelligence

How To Write An A-Grade Literature Review

Overview: The Literature Review Process

But first, the “why”…

Step 1: Find the relevant literature

Method 1 – Google Scholar Scrubbing

Method 2 – University Database Scrounging

Method 3 – Journal Article Snowballing

Method 4 – Dissertation Scavenging

Need a helping hand?

Step 2: Log, catalogue and synthesise

2.1 – Log the reference information