google for research articles

18 Google Scholar tips all students should know

Dec 13, 2022

[[read-time]] min read

Think of this guide as your personal research assistant.

“It’s hard to pick your favorite kid,” Anurag Acharya says when I ask him to talk about a favorite Google Scholar feature he’s worked on. “I work on product, engineering, operations, partnerships,” he says. He’s been doing it for 18 years, which as of this month, happens to be how long Google Scholar has been around.

Google Scholar is also one of Google’s longest-running services. The comprehensive database of research papers, legal cases and other scholarly publications was the fourth Search service Google launched, Anurag says. In honor of this very important tool’s 18th anniversary, I asked Anurag to share 18 things you can do in Google Scholar that you might have missed.

1. Copy article citations in the style of your choice.

With a simple click of the cite button (which sits below an article entry), Google Scholar will give you a ready-to-use citation for the article in five styles, including APA, MLA and Chicago. You can select and copy the one you prefer.

2. Dig deeper with related searches.

Google Scholar’s related searches can help you pinpoint your research; you’ll see them show up on a page in between article results. Anurag describes it like this: You start with a big topic — like “cancer” — and follow up with a related search like “lung cancer” or “colon cancer” to explore specific kinds of cancer.

Related searches can help you find what you’re looking for.

3. And don’t miss the related articles.

This is another great way to find more papers similar to one you found helpful — you can find this link right below an entry.

4. Read the papers you find.

Scholarly articles have long been available only by subscription. To keep you from having to log in every time you see a paper you’re interested in, Scholar works with libraries and publishers worldwide to integrate their subscriptions directly into its search results. Look for a link marked [PDF] or [HTML]. This also includes preprints and other free-to-read versions of papers.

5. Access Google Scholar tools from anywhere on the web with the Scholar Button browser extension.

The Scholar Button browser extension is sort of like a mini version of Scholar that can move around the web with you. If you’re searching for something, hitting the extension icon will show you studies about that topic, and if you’re reading a study, you can hit that same button to find a version you read, create a citation or to save it to your Scholar library.

Install the Scholar Button Chrome browser extension to access Google Scholar from anywhere on the web.

6. Learn more about authors through Scholar profiles.

There are many times when you’ll want to know more about the researchers behind the ideas you’re looking into. You can do this by clicking on an author’s name when it’s hyperlinked in a search result. You’ll find all of their work as well as co-authors, articles they’re cited in and so on. You can also follow authors from their Scholar profile to get email updates about their work, or about when and where their work is cited.

7. Easily find topic experts.

One last thing about author profiles: If there are topics listed below an author’s name on their profile, you can click on these areas of expertise and you’ll see a page of more authors who are researching and publishing on these topics, too.

8. Search for court opinions with the “Case law” button.

Scholar is the largest free database of U.S. court opinions. When you search for something using Google Scholar, you can select the “Case law” button below the search box to see legal cases your keywords are referenced in. You can read the opinions and a summary of what they established.

9. See how those court opinions have been cited.

If you want to better understand the impact of a particular piece of case law, you can select “How Cited,” which is below an entry, to see how and where the document has been cited. For example, here is the How Cited page for Marbury v. Madison , a landmark U.S. Supreme Court ruling that established that courts can strike down unconstitutional laws or statutes.

10. Understand how a legal opinion depends on another.

When you’re looking at how case laws are cited within Google Scholar, click on “Cited by” and check out the horizontal bars next to the different results. They indicate how relevant the cited opinion is in the court decision it’s cited within. You will see zero, one, two or three bars before each result. Those bars indicate the extent to which the new opinion depends on and refers to the cited case.

In the Cited by page for New York Times Company v. Sullivan, court cases with three bars next to their name heavily reference the original case. One bar indicates less reliance.

11. Sign up for Google Scholar alerts.

Want to stay up to date on a specific topic? Create an alert for a Google Scholar search for your topics and you’ll get email updates similar to Google Search alerts. Another way to keep up with research in your area is to follow new articles by leading researchers. Go to their profiles and click “Follow.” If you’re a junior grad student, you may consider following articles related to your advisor’s research topics, for instance.

12. Save interesting articles to your library.

It’s easy to go down fascinating rabbit hole after rabbit hole in Google Scholar. Don’t lose track of your research and use the save option that pops up under search results so articles will be in your library for later reading.

13. Keep your library organized with labels.

Labels aren’t only for Gmail! You can create labels within your Google Scholar library so you can keep your research organized. Click on “My library,” and then the “Manage labels…” option to create a new label.

14. If you’re a researcher, share your research with all your colleagues.

Many research funding agencies around the world now mandate that funded articles should become publicly free to read within a year of publication — or sooner. Scholar profiles list such articles to help researchers keep track of them and open up access to ones that are still locked down. That means you can immediately see what is currently available from researchers you’re interested in and how many of their papers will soon be publicly free to read.

15. Look through Scholar’s annual top publications and papers.

Every year, Google Scholar releases the top publications based on the most-cited papers. That list (available in 11 languages) will also take you to each publication’s top papers — this takes into account the “h index,” which measures how much impact an article has had. It’s an excellent place to start a research journey as well as get an idea about the ideas and discoveries researchers are currently focused on.

16. Get even more specific with Advanced Search.

Click on the hamburger icon on the upper left-hand corner and select Advanced Search to fine-tune your queries. For example, articles with exact words or a particular phrase in the title or articles from a particular journal and so on.

17. Find extra help on Google Scholar’s help page.

It might sound obvious, but there’s a wealth of useful information to be found here — like how often the database is updated, tips on formatting searches and how you can use your library subscriptions when you’re off-campus (looking at you, college students!). Oh, and you’ll even learn the origin of that quote on Google Scholar’s home page.

18. Keep up with Google Scholar news.

Don’t forget to check out the Google Scholar blog for updates on new features and tips for using this tool even better.

Related stories

Quiz: Do you know solar eclipse Search Trends?

4 ways to use Search to check facts, images and sources online

Get more personalized shopping options with these Google tools

6 ways to travel smarter this summer using Google tools

Empowering your team to build best-in-class mmms.

New ways we’re tackling spammy, low-quality content on Search

Let’s stay in touch. Get the latest news from Google in your inbox.

Faculty and researchers : We want to hear from you! We are launching a survey to learn more about your library collection needs for teaching, learning, and research. If you would like to participate, please complete the survey by May 17, 2024. Thank you for your participation!

• University of Massachusetts Lowell
• University Libraries

Google Scholar Search Strategies

• About Google Scholar
• Manage Settings
• Enable My Library
• Google Scholar Library
• Cite from Google Scholar
• Tracking Citations
• Add Articles Manually
• Refine your Profile Settings

Using Google Scholar for Research

Google Scholar is a powerful tool for researchers and students alike to access peer-reviewed papers. With Scholar, you are able to not only search for an article, author or journal of interest, you can also save and organize these articles, create email alerts, export citations and more. Below you will find some basic search tips that will prove useful.

This page also includes information on Google Scholar Library - a resource that allows you to save, organize and manage citations - as well as information on citing a paper on Google Scholar.

Search Tips

• Locate Full Text
• Sort by Date
• Related Articles
• Court Opinions
• Email Alerts
• Advanced Search

Abstracts are freely available for most of the articles and UMass Lowell holds many subscriptions to journals and online resources. The first step is make sure you are affiliated with the UML Library on and off campus by Managing your Settings, under Library Links. 

When searching in Google Scholar here are a few things to try to get full text:

• click a library link, e.g., "Full-text @ UML Library", to the right of the search result;
• click a link labeled [PDF] to the right of the search result;
• click "All versions" under the search result and check out the alternative sources;
• click "More" under the search result to see if there's an option for full-text;
• click "Related articles" or "Cited by" under the search result to explore similar articles.

Your search results are normally sorted by relevance, not by date. To find newer articles, try the following options in the left sidebar:

• click "Sort by date" to show just the new additions, sorted by date;  If you use this feature a lot, you may also find it useful to setup email alerts to have new results automatically sent to you.
• click the envelope icon to have new results periodically delivered by email.

Note: On smaller screens that don't show the sidebar, these options are available in the dropdown menu labeled "Any time" right below the search button .

The Related Articles option under the search result can be a useful tool when performing research on a specific topic. 

After clicking you will see articles from the same authors and with the same keywords.

You can select the jurisdiction from either the search results page or the home page as well; simply click "select courts". You can also refine your search by state courts or federal courts. 

To quickly search a frequently used selection of courts, bookmark a search results page with the desired selection. 

 How do I sign up for email alerts?

Do a search for the topic of interest, e.g., "M Theory"; click the envelope icon in the sidebar of the search  results page; enter your email address, and click " Create alert ". Google will periodically email you newly published papers that match your search criteria. You can use any email address for this; it does not need to be a Google Account. 

If you want to get alerts from new articles published in a specific journal; type in the name of this journal in the search bar and create an alert like you would a keyword. 

How do I get notified of new papers published by my colleagues, advisors or professors?

First, do a search for your their name, and see if they have a Citations profile. If they do, click on it, and click the "Follow new articles" link in the right sidebar under the search box.

If they don't have a profile, do a search by author, e.g., [author:s-hawking], and click on the mighty envelope in the left sidebar of the search results page. If you find that several different people share the same name, you may need to add co-author names or topical keywords to limit results to the author you wish to follow.

How do I change my alerts?

If you created alerts using a Google account, you can manage them all on the "Alerts" page . 

From here you can create, edit or delete alerts. Select cancel under the actions column to unsubscribe from an alert. 

This will pop-open the advanced search menu

Here you can search specific words/phrases as well as for author, title and journal. You can also limit your search results by date.

• << Previous: Enable My Library
• Next: Google Scholar Library >>
• Last Updated: Feb 14, 2024 2:55 PM
• URL: https://libguides.uml.edu/googlescholar

Using Google for Research

• Google Search
• Google Scholar
• Google Books

What is Google Scholar?

Google Scholar searches for scholarly literature in a simple, familiar way. You can search across many disciplines and sources at once to find articles, books, theses, court opinions, and content from academic publishers, professional societies, some academic web sites, and more. See the Google Scholar inclusion guidelines for more about what’s in Google Scholar.

Advanced Search Tips

For more precise searching, use Google's  Advanced Scholar Search Page

• To pull up the Advanced Scholar Search menu, go to the regular Google Scholar search page.
• In the upper left corner of the page, press the button made of three horizontal lines to open a new menu. 
• Advanced Search should be the second to last option in the newly-opened menu.

Or, try these tips:

Find content by an author:.

• Add the author's name to the search, or
• Use the "author:" operator (eg. aphasia author:jones finds articles about aphasia written by people named Jones)

Search for a phrase:

• Use "quotation marks" to find phrases (eg. "allegory of the cave" plato republic finds articles about Plato's cave allegory in The Republic )

Search by words in the title:

• Use the "intitle:" operator (eg. intitle:fellini finds articles with Fellini in the title]

Setting "Library Links" Preferences in Google Scholar

1. go to scholar.google.com , and click on the menu button (3 horizontal bars) in the upper left-hand corner of the screen..

2. In the menu that appears, click "Settings"

3. Click "Library links" in the left-hand menu. 

4. Search for NYU, and select only  "New York University Libraries - GetIt@NYU" then click "Save".

5. Conduct a new search in Google Scholar. Click the "GetIt@NYU" link next to each search result to get NYU Libraries-subscribed access to the article. If you are off campus, you will be prompted to log in with your NetID and password before being granted access to the full-text.

6. If you encounter a search result without a "GetIt@NYU" link next to it, try clicking on the "double arrow" button below it, and the link should appear.

• << Previous: Google Search
• Next: Google Books >>
• Last Updated: Mar 29, 2024 1:48 PM
• URL: https://guides.nyu.edu/google

Google Scholar

Using Google Scholar with your HarvardKey allows you to make the most of provided links, granting access to full text available through Harvard Library subscriptions.

Google Scholar can quickly surface highly cited peer-reviewed articles, abstracts, books, patents, scholarly web pages, and more. 

Explore Google Scholar

Connect Google Scholar To Your Library Access

Connecting Google Scholar to your Harvard Library access is a good way to make sure you get access to articles that Harvard Library subscribes to.

Here's how: 

• Go to Google Scholar and sign in to your Google account
• Look for the menu options
• Go into the settings and select "Library links"
• Type in Harvard and select: Harvard University - Try Harvard Library
• Deselect the box for WorldCat if shown
• Save your preferences
• Search your topic and look for the "Try Harvard Library" links to the right of the articles. This link should take you to Harvard's access to that item

Google Scholar Tips

• Like Google, Google Scholar allows searching of metadata terms, but unlike Google, it also indexes full text. 
• Choose the default search or select “Advanced search” to search by title, author, journal, and date.
• For more advanced researchers, it is possible to specify phrases in quotation marks, enter Boolean queries, or search within fields.
• You may also create an account to set up your author profile or sign up for alerts.
• In settings, you may elect to limit your search by language and show citation import links.
• Results are returned in relevance-ranked order, generally favoring entries when search terms appear in document titles and prioritizing documents with larger citation counts.

How to Use Google Scholar for Research: A Complete Guide

To remain competitive, Research and Development (R&D) teams must utilize all of the resources available to them. Google Scholar can be a powerful asset for R&D professionals who are looking to quickly find relevant sources related to their project.  With its sophisticated search engine capabilities, advanced filtering options, and alert notifications, using Google Scholar for research allows teams to easily locate reliable information in an efficient manner. Want to learn how to use google scholar for research? This blog post will cover how to use google scholar for research, how R&D professionals can exploit the potential of Google Scholar to uncover novel discoveries related to their projects, as well as remain apprised of advancements in their area.

Table of Contents

What is Google Scholar?

Overview of google scholar, searching with google scholar, finding relevant sources with google scholar, exploring related topics, evaluating sources found on google scholar, staying up to date with google scholar alerts, faqs in relation to how to use google scholar for research, how do i use google scholar for research, can you use google scholar for research papers, why is it important to use google scholar for research, are google scholar articles credible.

Google Scholar is a powerful research platform that enables users to quickly find, access, and evaluate scholarly information. It provides easy access to academic literature from all disciplines, including books, journal articles, conference papers, and more. Google Scholar offers researchers a wide range of tools for searching the web for the relevant content as well as ways to keep up with new developments in their field.

Google Scholar i s an online search engine designed specifically for finding scholarly literature on the internet. Google Scholar provides access to a vast array of scholarly literature from renowned universities and publishers around the world, simplifying the process of locating relevant material on any subject. In addition to its comprehensive indexing capabilities, Google Scholar also includes advanced search features such as citation tracking and alert notifications when new results are published in your chosen areas of interest.

The platform makes it a breeze for users to traverse multiple facets of a given topic by providing them with an array of different filters they can apply when conducting searches – these include things such as author name or publication date range; language; type (e.g., book chapter vs journal article); source material (e.g., open access only); etc Moreover, many results found through this platform come equipped with full-text PDFs available for download – so you don’t have to worry about pesky paywalls blocking your path while doing research.

Google Scholar is an invaluable resource for research and development teams, offering quick access to a wealth of scholarly information. Utilizing the proper search approaches, you can quickly locate precisely what you need by employing Google Scholar. Let’s look now at how to refine your results with advanced search techniques.

Key Takeaway:  Google Scholar is a powerful research platform that gives researchers an array of tools to quickly locate, access and evaluate scholarly information. It provides users with advanced search features such as citation tracking and alert notifications, along with easy-to-apply filters for narrowing down results by author name or publication date range – making it the go-to tool for any researcher looking to cut through the noise.

Exploring with Google Scholar can be a useful approach to quickly locate applicable scholarly material. There are several different strategies that can be used to get the most out of this powerful tool.

Basic google scholar search strategies involve entering a few keywords or phrases into the search bar and then refining your results using filters, sorting options, and related topics. This method is ideal for those who require a rapid search of information without needing to expend an excessive amount of time researching exact terms, especially for those unfamiliar with searching databases such as Google Scholar. It’s also useful for those who don’t have a lot of experience in searching databases like Google Scholar. 

Advanced search strategies allow users to take advantage of more sophisticated features such as Boolean operators , wildcards, and phrase searches. These tools make it easier to narrow down results by specifying exactly what you’re looking for or excluding irrelevant sources from your search results. Advanced searchers should also pay attention to synonyms when crafting their queries since these can help broaden the scope of their searches while still providing relevant results.

Finally, refining your results is key in order to ensure that you only see sources that are truly relevant and authoritative on the topic at hand. Filters such as date range, publication type, language, author name, etc., can help refine your query so that only high-quality sources appear in your list of results. Sorting options provide users with the ability to prioritize documents, enabling them to quickly locate relevant materials without needing to review a large number of irrelevant ones. 

Utilizing Google Scholar can be advantageous for swiftly finding pertinent research materials, but it is essential to comprehend the search strategies and filters at hand in order to maximize your searches. By understanding how to identify keywords and phrases, explore related topics, and utilize sorting options and filters, you can ensure that you are finding all of the relevant sources for your research project. 

Key Takeaway:  Google Scholar is a great tool for quickly locating relevant research sources. Advanced searchers can make use of Boolean operators, wildcards and phrase searches to narrow down their results while basic search strategies such as entering keywords into the search bar work just fine too. Additionally, refining your results with filters and sorting options helps ensure that you only see high-quality sources related to your topic at hand.

Locating applicable materials via Google Scholar can be a challenging endeavor, particularly for those unfamiliar with the research process. To facilitate the research process, employing various strategies can expedite and refine the search for relevant sources through Google Scholar. 

Making use of keywords and phrases is a powerful method for finding pertinent sources on Google Scholar. It is important to identify key terms related to your topic or research question so you can narrow down the results. Additionally, using quotation marks around multiple words will allow you to get more precise results as it searches for exact matches instead of individual words within a phrase.

Exploring related topics helps provide additional context when researching on Google Scholar. This includes looking at previous studies conducted on similar topics or areas of interest, which provides further insight into potential sources available from other researchers’ work in the field. Utilizing tools such as co-citation analysis also allows users to explore how different authors have been cited together over time by providing visualizations based on their connections and relationships with each other through citations.

Utilizing filters and sorting options such as language, date range, publication type, etc., enables users to refine their search even further so they only receive results that match their specific criteria. Sorting options like relevance ranking or date published also make it easier for them to find what they need without having to sift through hundreds of irrelevant documents manually. By utilizing these features effectively, researchers can save valuable time when searching for relevant sources in Google Scholar since all the information they need will already be organized accordingly right away, saving them an hour’s worth of manual labor.

By utilizing Google Scholar, research teams can quickly and easily find relevant sources for their projects. With the next heading, we will explore how to evaluate these sources for credibility and authority.

Key Takeaway:  Utilizing the right keywords and phrases, exploring related topics, and utilizing filters are essential techniques for finding relevant sources quickly with Google Scholar. By taking advantage of the available features, you can swiftly and accurately pinpoint documents that meet your criteria.

To assess the reliability and authority of each source, consider factors such as the publication’s reputation, author credentials in the field, and when it was published. To do this, look for publications from reputable journals or authors with credentials in the field. Furthermore, consider when the source was issued – more modern pieces may be more pertinent and exact than older ones.

It is advantageous to be aware of the distinct kinds of publications that can appear in search results, such as scholarly articles, books, conference papers, and dissertations; each offering various degrees of precision and accuracy depending on their intent and target audience. 

For example, a book chapter may provide an overview of a topic while a peer-reviewed journal article will contain more detailed information backed up by research evidence. Similarly, conference papers are typically shorter summaries of research projects whereas dissertations offer comprehensive coverage including methodology and analysis results. Understanding these differences helps you identify which sources are most suitable for your needs when conducting research using Google Scholar.

Evaluating sources found on Google Scholar is an important step to ensure the credibility and accuracy of research results. By setting up alerts with Google Scholar, you can stay informed about new research findings and manage your subscriptions accordingly.

Maximize your research efforts with Google Scholar. Assess credibility & authority, pay attention to the date of publication & understand different types of publications. #ResearchTips #GoogleScholar Click to Tweet

Google Scholar is an invaluable tool for staying up to date with the latest research in your field. With its alert feature, you can easily set up notifications so that you’re always on top of new developments. Setting up alerts and managing them effectively will help ensure that you never miss a beat when it comes to relevant information.

Begin your research by utilizing Google Scholar’s sophisticated search features such as keyword and phrase searches, sorting results according to relevance or date of publication, and excluding unrelated sources. Once you’ve identified the most pertinent topics related to your research interests, set up alerts for each one by clicking on the bell icon in the upper right corner of the page. This will allow Google Scholar to send notifications whenever new content is published about those specific topics.

When setting up alerts in Google Scholar, make sure that they are tailored specifically toward what matters most to you – this could include certain authors or journals whose work has particular relevance to your own research projects. You can also adjust how often these alerts are sent (daily or weekly) depending on how frequently new material is being published within those fields of study. Additionally, if there are any other sources outside of Google Scholar which may contain useful information (such as blogs), consider adding their RSS feeds into your alert system too so that all relevant updates appear in one place.

Finally, don’t forget to manage existing alerts regularly; this means keeping track of which ones are still relevant and deleting any no longer needed from time to time (this helps keep clutter down). Additionally, try experimenting with different combinations/filters within each alert until you find what works best for keeping yourself informed without getting overwhelmed with notifications.

Key Takeaway:  Utilize Google Scholar to stay up-to-date on the latest research in your field – create tailored alerts for specific topics and authors, adjust frequency of notifications as needed, and manage existing alerts regularly. Stay ahead of the curve by gathering all pertinent news in one location.

Google Scholar is a great tool for conducting research. It provides access to millions of scholarly articles, books, and other sources from across the web. Google scholar works by entering keywords related to your topic into the search bar at the top of the page to quickly locate relevant scholarly articles, books, and other sources from across the web. Then narrow down your results using filters such as date range or publication type.

Finally, skim through the abstracts and full texts to pinpoint useful information for your research project.

Yes, Google Scholar is a great resource for research papers. It offers access to an extensive range of scholarly literature from journals, books, and conference proceedings. The search engine provides a convenient way to locate the most recent research in any area by entering keywords or phrases.

Advanced capabilities, such as citation monitoring, can be utilized to track the latest citations of one’s own or others’ work.

Google Scholar is an invaluable tool for research, as it provides access to a vast range of scholarly literature from around the world. It allows researchers to quickly and easily search through millions of publications and journals in order to find relevant information.

Google Scholar also offers the ability to trace connections between different works, allowing researchers to stay abreast of recent developments in their field. With its user-friendly interface, Google Scholar makes researching easier than ever before.

Yes, Google Scholar articles are credible. They provide access to a wide range of academic literature from reliable sources such as peer-reviewed journals and conference proceedings. Expert scrutiny has been conducted to guarantee the accuracy and excellence of the articles before they are put up on Google Scholar. Additionally, each article includes information about its authorship and citation count which can help readers assess their credibility further.

Google Scholar provides a convenient way to uncover pertinent material, assess the quality of these sources with ease, and be informed about novel advancements in your area through notifications.  Thus, R&D supervisors should know how to use google scholar for research. Also, R&D supervisors considering utilizing Google Scholar for investigation ought to recall that this apparatus should not supplant customary techniques, for example, peer survey or manual searching; rather it should supplement them.

With its powerful search capabilities and ability to keep researchers informed about their fields of interest, using Google Scholar for research can save time while providing more accurate results than ever before.

Unlock the power of research with Cypris . Our platform provides rapid time to insights, enabling R&D and innovation teams to quickly access data sources for their projects.

Similar insights you might enjoy

Gallium Nitride Innovation Pulse

Carbon Capture & Storage Innovation Pulse

Sodium-Ion Batteries Innovation Pulse

Reference management. Clean and simple.

The top list of academic search engines

1. Google Scholar

4. science.gov, 5. semantic scholar, 6. baidu scholar, get the most out of academic search engines, frequently asked questions about academic search engines, related articles.

Academic search engines have become the number one resource to turn to in order to find research papers and other scholarly sources. While classic academic databases like Web of Science and Scopus are locked behind paywalls, Google Scholar and others can be accessed free of charge. In order to help you get your research done fast, we have compiled the top list of free academic search engines.

Google Scholar is the clear number one when it comes to academic search engines. It's the power of Google searches applied to research papers and patents. It not only lets you find research papers for all academic disciplines for free but also often provides links to full-text PDF files.

• Coverage: approx. 200 million articles
• Abstracts: only a snippet of the abstract is available
• Related articles: ✔
• References: ✔
• Cited by: ✔
• Links to full text: ✔
• Export formats: APA, MLA, Chicago, Harvard, Vancouver, RIS, BibTeX

BASE is hosted at Bielefeld University in Germany. That is also where its name stems from (Bielefeld Academic Search Engine).

• Coverage: approx. 136 million articles (contains duplicates)
• Abstracts: ✔
• Related articles: ✘
• References: ✘
• Cited by: ✘
• Export formats: RIS, BibTeX

CORE is an academic search engine dedicated to open-access research papers. For each search result, a link to the full-text PDF or full-text web page is provided.

• Coverage: approx. 136 million articles
• Links to full text: ✔ (all articles in CORE are open access)
• Export formats: BibTeX

Science.gov is a fantastic resource as it bundles and offers free access to search results from more than 15 U.S. federal agencies. There is no need anymore to query all those resources separately!

• Coverage: approx. 200 million articles and reports
• Links to full text: ✔ (available for some databases)
• Export formats: APA, MLA, RIS, BibTeX (available for some databases)

Semantic Scholar is the new kid on the block. Its mission is to provide more relevant and impactful search results using AI-powered algorithms that find hidden connections and links between research topics.

• Coverage: approx. 40 million articles
• Export formats: APA, MLA, Chicago, BibTeX

Although Baidu Scholar's interface is in Chinese, its index contains research papers in English as well as Chinese.

• Coverage: no detailed statistics available, approx. 100 million articles
• Abstracts: only snippets of the abstract are available
• Export formats: APA, MLA, RIS, BibTeX

RefSeek searches more than one billion documents from academic and organizational websites. Its clean interface makes it especially easy to use for students and new researchers.

• Coverage: no detailed statistics available, approx. 1 billion documents
• Abstracts: only snippets of the article are available
• Export formats: not available

Consider using a reference manager like Paperpile to save, organize, and cite your references. Paperpile integrates with Google Scholar and many popular databases, so you can save references and PDFs directly to your library using the Paperpile buttons:

Google Scholar is an academic search engine, and it is the clear number one when it comes to academic search engines. It's the power of Google searches applied to research papers and patents. It not only let's you find research papers for all academic disciplines for free, but also often provides links to full text PDF file.

Semantic Scholar is a free, AI-powered research tool for scientific literature developed at the Allen Institute for AI. Sematic Scholar was publicly released in 2015 and uses advances in natural language processing to provide summaries for scholarly papers.

BASE , as its name suggest is an academic search engine. It is hosted at Bielefeld University in Germany and that's where it name stems from (Bielefeld Academic Search Engine).

CORE is an academic search engine dedicated to open access research papers. For each search result a link to the full text PDF or full text web page is provided.

Science.gov is a fantastic resource as it bundles and offers free access to search results from more than 15 U.S. federal agencies. There is no need any more to query all those resources separately!

International Students' Library Resource Guide: Use Google Scholar Effectively for Research

• How to start research

Use Google Scholar Effectively for Research

• Avoiding Plagiarism & Creating Citations
• Language Resources
• Campus Resources

Google Scholar

• Connect Off Campus
• Connect to RefWorks

From the Google Scholar home page, click on the "Settings" gear on the top of the page.

• Select “Library Links” from the left column
• Search for “University of Wisconsin Milwaukee” and check the "UWM" box that appears

When you find a good article, click on the "Get IT! (UWM Libraries)" link -  not  the title 

To show links to import into RefWorks:

• Select "Search Results" from the left column
• Select "Show links to import citations into" radio button and select RefWorks

When you find a good article, save the citation by selecting "Import Into RefWorks"

Google Scholar finds academic materials such as peer-reviewed papers, theses, books, abstracts, and technical reports from broad areas of research. It searches a variety of undisclosed academic publishers, professional societies, and universities, as well as scholarly articles available across the web.

Advantages of Google Book Search

Many articles are available full-text.

Google Scholar can connect you to library resources.

The sources are more academic than those you would find through a standard search engine query, though you should still always evaluate the content you find.

It covers a wide range of academic content areas and is continually being updated.

It ranks and lists results according to how relevant they are to the search query. The most relevant references should theoretically appear at the top of the page.

You can access articles the library has on campus or off campus by logging in your student ID.

Clicking the Cited by link on Google Scholar will display a list of articles and documents that have cited the document originally retrieved in the search. This makes it possible to find other documents related by topic to the original document. However, Google Scholar only includes articles that are indexed within its database, and this is a much smaller subset of scholarly articles than those found in other UWM databases.

Interpreting Your Findings

Search results have a variety of formats. PDF and PostScript documents will have clickable title links that point to abstracts or full-text, but citations and books will not. Citations are items that have been extracted from the references, footnotes, or bibliographies of documents indexed by Google Scholar. They typically do not have clickable links, but the citation usually gives you enough information to track down the item. Clicking on “Web Search” will start a search for the citation using the standard Google search engine. This can help you determine the title of the book or journal the article was published in if it is abbreviated in the citation.

Advanced Search Tips

Use Google Scholar's Advanced Search to search by author, publication, and date.

Setting Library Links

From the Google Scholar Preferences page, search for UW Milwaukee and check UWM Libraries (Get Text from UW-Milwaukee) box. You will only need to do this the first time you search Google Scholar.

Off Campus? Not a Problem

You can still access all of the databases from the UWM Libraries and set Google Scholar preferences through a VPN.

• << Previous: How to start research
• Next: Avoiding Plagiarism & Creating Citations >>
• Last Updated: Aug 15, 2022 1:06 PM
• URL: https://libguides.stthomas.edu/c.php?g=516880

© 2023 University of St. Thomas, Minnesota

• {{link.text}}

Publications

Google publishes hundreds of research papers each year. Publishing is important to us; it enables us to collaborate and share ideas with, as well as learn from, the broader scientific community. Submissions are often made stronger by the fact that ideas have been tested through real product implementation by the time of publication.

We believe the formal structures of publishing today are changing - in computer science especially, there are multiple ways of disseminating information.  We encourage publication both in conventional scientific venues, and through other venues such as industry forums, standards bodies, and open source software and product feature releases.

Open Source

We understand the value of a collaborative ecosystem and love open source software .

Product and Feature Launches

With every launch, we're publishing progress and pushing functionality.

Industry Standards

Our researchers are often helping to define not just today's products but also tomorrow's.

"Resources" doesn't just mean tangible assets but also intellectual. Incredible datasets and a great team of colleagues foster a rich and collaborative research environment.

Couple big challenges with big resources and Google offers unprecedented research opportunities.

22 Research Areas

• Algorithms and Theory 608 Publications
• Data Management 116 Publications
• Data Mining and Modeling 214 Publications
• Distributed Systems and Parallel Computing 208 Publications
• Economics and Electronic Commerce 209 Publications
• Education Innovation 30 Publications
• General Science 158 Publications
• Hardware and Architecture 67 Publications
• Human-Computer Interaction and Visualization 444 Publications
• Information Retrieval and the Web 213 Publications
• Machine Intelligence 1019 Publications
• Machine Perception 454 Publications
• Machine Translation 48 Publications
• Mobile Systems 72 Publications
• Natural Language Processing 395 Publications
• Networking 210 Publications
• Quantum A.I. 30 Publications
• Robotics 37 Publications
• Security, Privacy and Abuse Prevention 289 Publications
• Software Engineering 100 Publications
• Software Systems 250 Publications
• Speech Processing 264 Publications

3 Collections

• Google AI Residency 60 Publications
• Google Brain Team 305 Publications
• Data Infrastructure and Analysis 10 Publications

• Ask a Librarian

Citation Databases

• Searching Scopus
• Researcher Profiles
• Researcher IDs
• Searching Web of Science
• Google Scholar
• Research Impact and Metrics

Introduction to Google Scholar

Google Scholar is the largest citation search-base in the world, though it suffers from some difficulties it is the most widely used of all such search engines. The reason for this is that author profiles in Google Scholar tend to index the entirety of an author’s output. The database is maintained by a powerful algorithm which searches the entire internet for citations, documents, and other research output. However, not all of the material which can be found in Google Scholar is peer-reviewed so remember that it is up to you to be a critical consumer of information. Google Scholar cannot sort by research field or type, browse by title, and cannot limit search results (except for by year).

Uses for Researchers

There are many use cases of Google Scholar for researchers, the most obvious cases are literature reviews and making note of research trends. Searching Google Scholar is easy, and by making use of some of the tips below you will be able to narrow your search results.

Search Tips

Here are some easy search tips for using Google Scholar.

Opening Advanced search: to open advanced search navigate to the three lines at the top left of the google scholar home page, click on that and then click on advanced search.

Using “Boolean Operators” such as ‘AND’ as well as ‘OR’ can allow you to search for things which include specific words

Using the command “intitle:” we can do an in title search, so for example “Intitle: “Romanitas”     Roman”' will search for articles with “Romanitas” in the title and the word “Roman” in the text of the article. This can help you to find articles with specific content.

Cited by References search is a powerful tool which allows you to search for articles that have cited a particular article. Search for an article and then click “cited by” under the entry.

Setting library links can be done by clicking “settings”, clicking on “library links” and then searching for “Purdue”, and finally clicking “save”. This will allow you to access materials you find on Google Scholar directly from your searches

Metrics in Google Scholar

Google Scholar maintains researcher level profiles which can be created or claimed. If you are already a prolific author, you might have an algorithmically generated profile already. These can be claimed, but others will find that they can create a profile themselves. Either way, once you have done this you can begin to access personalized metrics such as h-index and citations.

• << Previous: Searching Web of Science
• Next: Research Impact and Metrics >>
• Last Edited: Apr 11, 2024 9:42 AM
• URL: https://guides.lib.purdue.edu/citationdatabases

University Libraries

• How Do I...
• Search for Articles with Google Scholar
• Adding a Library Printer to Your Laptop

The Catholic University library catalog and many of the article databases Catholic University subscribes to are accessible through Google Scholar .

On-campus access

Visit https://scholar.google.com and begin searching. You're good to go!

Off-campus access

If you are off campus you will need to set the preferences so that Google will show you the resources that Catholic University provides.

• Go to https://scholar.google.com
• Look at the left corner menu icon and click Settings from the menu.
• Click on Library Links from the navbar along the side of the page.
• Enter CUA in the text field next to Library Links then click on the Search button.
• Check the box in the front of our university name, then click Save in the lower right corner.

Searching with Google Scholar

Within Google Scholar you may conduct searches by keyword, author and article title. There is also an advanced search with more options. In the result list, when you see ViewIt@CatholicU , that means we have access to the electronic copy for the article. Click on ViewIt@CatholicU , the next page will show that item in our SearchBox with a link to the full text.

Google Scholar is good for conducting simple searches across a broad number of databases. For complex or in depth searching we recommend that you search individual subject databases .

Google Scholar™ is a trademark of Google Inc.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Med Libr Assoc
• v.106(4); 2018 Oct

A systematic approach to searching: an efficient and complete method to develop literature searches

Associated data.

Creating search strategies for systematic reviews, finding the best balance between sensitivity and specificity, and translating search strategies between databases is challenging. Several methods describe standards for systematic search strategies, but a consistent approach for creating an exhaustive search strategy has not yet been fully described in enough detail to be fully replicable. The authors have established a method that describes step by step the process of developing a systematic search strategy as needed in the systematic review. This method describes how single-line search strategies can be prepared in a text document by typing search syntax (such as field codes, parentheses, and Boolean operators) before copying and pasting search terms (keywords and free-text synonyms) that are found in the thesaurus. To help ensure term completeness, we developed a novel optimization technique that is mainly based on comparing the results retrieved by thesaurus terms with those retrieved by the free-text search words to identify potentially relevant candidate search terms. Macros in Microsoft Word have been developed to convert syntaxes between databases and interfaces almost automatically. This method helps information specialists in developing librarian-mediated searches for systematic reviews as well as medical and health care practitioners who are searching for evidence to answer clinical questions. The described method can be used to create complex and comprehensive search strategies for different databases and interfaces, such as those that are needed when searching for relevant references for systematic reviews, and will assist both information specialists and practitioners when they are searching the biomedical literature.

INTRODUCTION

Librarians and information specialists are often involved in the process of preparing and completing systematic reviews (SRs), where one of their main tasks is to identify relevant references to include in the review [ 1 ]. Although several recommendations for the process of searching have been published [ 2 – 6 ], none describe the development of a systematic search strategy from start to finish.

Traditional methods of SR search strategy development and execution are highly time consuming, reportedly requiring up to 100 hours or more [ 7 , 8 ]. The authors wanted to develop systematic and exhaustive search strategies more efficiently, while preserving the high sensitivity that SR search strategies necessitate. In this article, we describe the method developed at Erasmus University Medical Center (MC) and demonstrate its use through an example search. The efficiency of the search method and outcome of 73 searches that have resulted in published reviews are described in a separate article [ 9 ].

As we aimed to describe the creation of systematic searches in full detail, the method starts at a basic level with the analysis of the research question and the creation of search terms. Readers who are new to SR searching are advised to follow all steps described. More experienced searchers can consider the basic steps to be existing knowledge that will already be part of their normal workflow, although step 4 probably differs from general practice. Experienced searchers will gain the most from reading about the novelties in the method as described in steps 10–13 and comparing the examples given in the supplementary appendix to their own practice.

CREATING A SYSTEMATIC SEARCH STRATEGY

Our methodology for planning and creating a multi-database search strategy consists of the following steps:

• Determine a clear and focused question
• Describe the articles that can answer the question
• Decide which key concepts address the different elements of the question
• Decide which elements should be used for the best results
• Choose an appropriate database and interface to start with
• Document the search process in a text document
• Identify appropriate index terms in the thesaurus of the first database
• Identify synonyms in the thesaurus
• Add variations in search terms
• Use database-appropriate syntax, with parentheses, Boolean operators, and field codes
• Optimize the search
• Evaluate the initial results
• Check for errors
• Translate to other databases
• Test and reiterate

Each step in the process is reflected by an example search described in the supplementary appendix .

1. Determine a clear and focused question

A systematic search can best be applied to a well-defined and precise research or clinical question. Questions that are too broad or too vague cannot be answered easily in a systematic way and will generally result in an overwhelming number of search results. On the other hand, a question that is too specific will result into too few or even zero search results. Various papers describe this process in more detail [ 10 – 12 ].

2. Describe the articles that can answer the question

Although not all clinical or research questions can be answered in the literature, the next step is to presume that the answer can indeed be found in published studies. A good starting point for a search is hypothesizing what the research that can answer the question would look like. These hypothetical (when possible, combined with known) articles can be used as guidance for constructing the search strategy.

3. Decide which key concepts address the different elements of the question

Key concepts are the topics or components that the desired articles should address, such as diseases or conditions, actions, substances, settings, domains (e.g., therapy, diagnosis, etiology), or study types. Key concepts from the research question can be grouped to create elements in the search strategy.

Elements in a search strategy do not necessarily follow the patient, intervention, comparison, outcome (PICO) structure or any other related structure. Using the PICO or another similar framework as guidance can be helpful to consider, especially in the inclusion and exclusion review stage of the SR, but this is not necessary for good search strategy development [ 13 – 15 ]. Sometimes concepts from different parts of the PICO structure can be grouped together into one search element, such as when the desired outcome is frequently described in a certain study type.

4. Decide which elements should be used for the best results

Not all elements of a research question should necessarily be used in the search strategy. Some elements are less important than others or may unnecessarily complicate or restrict a search strategy. Adding an element to a search strategy increases the chance of missing relevant references. Therefore, the number of elements in a search strategy should remain as low as possible to optimize recall.

Using the schema in Figure 1 , elements can be ordered by their specificity and importance to determine the best search approach. Whether an element is more specific or more general can be measured objectively by the number of hits retrieved in a database when searching for a key term representing that element. Depending on the research question, certain elements are more important than others. If articles (hypothetically or known) exist that can answer the question but lack a certain element in their titles, abstracts, or keywords, that element is unimportant to the question. An element can also be unimportant because of expected bias or an overlap with another element.

Schema for determining the optimal order of elements

Bias in elements

The choice of elements in a search strategy can introduce bias through use of overly specific terminology or terms often associated with positive outcomes. For the question “does prolonged breastfeeding improve intelligence outcomes in children?,” searching specifically for the element of duration will introduce bias, as articles that find a positive effect of prolonged breastfeeding will be much more likely to mention time factors in their titles or abstracts.

Overlapping elements

Elements in a question sometimes overlap in their meaning. Sometimes certain therapies are interventions for one specific disease. The Lichtenstein technique, for example, is a repair method for inguinal hernias. There is no need to include an element of “inguinal hernias” to a search for the effectiveness of the Lichtenstein therapy. Likewise, sometimes certain diseases are only found in certain populations. Adding such an overlapping element could lead to missing relevant references.

The elements to use in a search strategy can be found in the plot of elements in Figure 1 , by following the top row from left to right. For this method, we recommend starting with the most important and specific elements. Then, continue with more general and important elements until the number of results is acceptable for screening. Determining how many results are acceptable for screening is often a matter of negotiation with the SR team.

5. Choose an appropriate database and interface to start with

Important factors for choosing databases to use are the coverage and the presence of a thesaurus. For medically oriented searches, the coverage and recall of Embase, which includes the MEDLINE database, are superior to those of MEDLINE [ 16 ]. Each of these two databases has its own thesaurus with its own unique definitions and structure. Because of the complexity of the Embase thesaurus, Emtree, which contains much more specific thesaurus terms than the MEDLINE Medical Subject Headings (MeSH) thesaurus, translation from Emtree to MeSH is easier than the other way around. Therefore, we recommend starting in Embase.

MEDLINE and Embase are available through many different vendors and interfaces. The choice of an interface and primary database is often determined by the searcher’s accessibility. For our method, an interface that allows searching with proximity operators is desirable, and full functionality of the thesaurus, including explosion of narrower terms, is crucial. We recommend developing a personal workflow that always starts with one specific database and interface.

6. Document the search process in a text document

We advise designing and creating the complete search strategies in a log document, instead of directly in the database itself, to register the steps taken and to make searches accountable and reproducible. The developed search strategies can be copied and pasted into the desired databases from the log document. This way, the searcher is in control of the whole process. Any change to the search strategy should be done in the log document, assuring that the search strategy in the log is always the most recent.

7. Identify appropriate index terms in the thesaurus of the first database

Searches should start by identifying appropriate thesaurus terms for the desired elements. The thesaurus of the database is searched for matching index terms for each key concept. We advise restricting the initial terms to the most important and most relevant terms. Later in the process, more general terms can be added in the optimization process, in which the effect on the number of hits, and thus the desirability of adding these terms, can be evaluated more easily.

Several factors can complicate the identification of thesaurus terms. Sometimes, one thesaurus term is found that exactly describes a specific element. In contrast, especially in more general elements, multiple thesaurus terms can be found to describe one element. If no relevant thesaurus terms have been found for an element, free-text terms can be used, and possible thesaurus terms found in the resulting references can be added later (step 11).

Sometimes, no distinct thesaurus term is available for a specific key concept that describes the concept in enough detail. In Emtree, one thesaurus term often combines two or more elements. The easiest solution for combining these terms for a sensitive search is to use such a thesaurus term in all elements where it is relevant. Examples are given in the supplementary appendix .

8. Identify synonyms in the thesaurus

Most thesauri offer a list of synonyms on their term details page (named Synonyms in Emtree and Entry Terms in MeSH). To create a sensitive search strategy for SRs, these terms need to be searched as free-text keywords in the title and abstract fields, in addition to searching their associated thesaurus terms.

The Emtree thesaurus contains more synonyms (300,000) than MeSH does (220,000) [ 17 ]. The difference in number of terms is even higher considering that many synonyms in MeSH are permuted terms (i.e., inversions of phrases using commas).

Thesaurus terms are ordered in a tree structure. When searching for a more general thesaurus term, the more specific (narrower) terms in the branches below that term will also be searched (this is frequently referred to as “exploding” a thesaurus term). However, to perform a sensitive search, all relevant variations of the narrower terms must be searched as free-text keywords in the title or abstract, in addition to relying on the exploded thesaurus term. Thus, all articles that describe a certain narrower topic in their titles and abstracts will already be retrieved before MeSH terms are added.

9. Add variations in search terms (e.g., truncation, spelling differences, abbreviations, opposites)

Truncation allows a searcher to search for words beginning with the same word stem. A search for therap* will, thus, retrieve therapy, therapies, therapeutic, and all other words starting with “therap.” Do not truncate a word stem that is too short. Also, limitations of interfaces should be taken into account, especially in PubMed, where the number of search term variations that can be found by truncation is limited to 600.

Databases contain references to articles using both standard British and American English spellings. Both need to be searched as free-text terms in the title and abstract. Alternatively, many interfaces offer a certain code to replace zero or one characters, allowing a search for “pediatric” or “paediatric” as “p?ediatric.” Table 1 provides a detailed description of the syntax for different interfaces.

Field codes in five most used interfaces for biomedical literature searching

Searching for abbreviations can identify extra, relevant references and retrieve more irrelevant ones. The search can be more focused by combining the abbreviation with an important word that is relevant to its meaning or by using the Boolean “NOT” to exclude frequently observed, clearly irrelevant results. We advise that searchers do not exclude all possible irrelevant meanings, as it is very time consuming to identify all the variations, it will result in unnecessarily complicated search strategies, and it may lead to erroneously narrowing the search and, thereby, reduce recall.

Searching partial abbreviations can be useful for retrieving relevant references. For example, it is very likely that an article would mention osteoarthritis (OA) early in the abstract, replacing all further occurrences of osteoarthritis with OA . Therefore, it may not contain the phrase “hip osteoarthritis” but only “hip oa.”

It is also important to search for the opposites of search terms to avoid bias. When searching for “disease recurrence,” articles about “disease free” may be relevant as well. When the desired outcome is survival , articles about mortality may be relevant.

10. Use database-appropriate syntax, with parentheses, Boolean operators, and field codes

Different interfaces require different syntaxes, the special set of rules and symbols unique to each database that define how a correctly constructed search operates. Common syntax components include the use of parentheses and Boolean operators such as “AND,” “OR,” and “NOT,” which are available in all major interfaces. An overview of different syntaxes for four major interfaces for bibliographic medical databases (PubMed, Ovid, EBSCOhost, Embase.com, and ProQuest) is shown in Table 1 .

Creating the appropriate syntax for each database, in combination with the selected terms as described in steps 7–9, can be challenging. Following the method outlined below simplifies the process:

• Create single-line queries in a text document (not combining multiple record sets), which allows immediate checking of the relevance of retrieved references and efficient optimization.
• Type the syntax (Boolean operators, parentheses, and field codes) before adding terms, which reduces the chance that errors are made in the syntax, especially in the number of parentheses.
• Use predefined proximity structures including parentheses, such as (() ADJ3 ()) in Ovid, that can be reused in the query when necessary.
• Use thesaurus terms separately from free-text terms of each element. Start an element with all thesaurus terms (using “OR”) and follow with the free-text terms. This allows the unique optimization methods as described in step 11.
• When adding terms to an existing search strategy, pay close attention to the position of the cursor. Make sure to place it appropriately either in the thesaurus terms section, in the title/abstract section, or as an addition (broadening) to an existing proximity search.

The supplementary appendix explains the method of building a query in more detail, step by step for different interfaces: PubMed, Ovid, EBSCOhost, Embase.com, and ProQuest. This method results in a basic search strategy designed to retrieve some relevant references upon which a more thorough search strategy can be built with optimization such as described in step 11.

11. Optimize the search

The most important question when performing a systematic search is whether all (or most) potentially relevant articles have been retrieved by the search strategy. This is also the most difficult question to answer, since it is unknown which and how many articles are relevant. It is, therefore, wise first to broaden the initial search strategy, making the search more sensitive, and then check if new relevant articles are found by comparing the set results (i.e., search for Strategy #2 NOT Strategy #1 to see the unique results).

A search strategy should be tested for completeness. Therefore, it is necessary to identify extra, possibly relevant search terms and add them to the test search in an OR relationship with the already used search terms. A good place to start, and a well-known strategy, is scanning the top retrieved articles when sorted by relevance, looking for additional relevant synonyms that could be added to the search strategy.

We have developed a unique optimization method that has not been described before in the literature. This method often adds valuable extra terms to our search strategy and, therefore, extra, relevant references to our search results. Extra synonyms can be found in articles that have been assigned a certain set of thesaurus terms but that lack synonyms in the title and/or abstract that are already present in the current search strategy. Searching for thesaurus terms NOT free-text terms will help identify missed free-text terms in the title or abstract. Searching for free-text terms NOT thesaurus terms will help identify missed thesaurus terms. If this is done repeatedly for each element, leaving the rest of the query unchanged, this method will help add numerous relevant terms to the query. These steps are explained in detail for five different search platforms in the supplementary appendix .

12. Evaluate the initial results

The results should now contain relevant references. If the interface allows relevance ranking, use that in the evaluation. If you know some relevant references that should be included in the research, search for those references specifically; for example, combine a specific (first) author name with a page number and the publication year. Check whether those references are retrieved by the search. If the known relevant references are not retrieved by the search, adapt the search so that they are. If it is unclear which element should be adapted to retrieve a certain article, combine that article with each element separately.

Different outcomes are desired for different types of research questions. For instance, in the case of clinical question answering, the researcher will not be satisfied with many references that contain a lot of irrelevant references. A clinical search should be rather specific and is allowed to miss a relevant reference. In the case of an SR, the researchers do not want to miss any relevant reference and are willing to handle many irrelevant references to do so. The search for references to include in an SR should be very sensitive: no included reference should be missed. A search that is too specific or too sensitive for the intended goal can be adapted to become more sensitive or specific. Steps to increase sensitivity or specificity of a search strategy can be found in the supplementary appendix .

13. Check for errors

Errors might not be easily detected. Sometimes clues can be found in the number of results, either when the number of results is much higher or lower than expected or when many retrieved references are not relevant. However, the number expected is often unknown, and very sensitive search strategies will always retrieve many irrelevant articles. Each query should, therefore, be checked for errors.

One of the most frequently occurring errors is missing the Boolean operator “OR.” When no “OR” is added between two search terms, many interfaces automatically add an “AND,” which unintentionally reduces the number of results and likely misses relevant references. One good strategy to identify missing “OR”s is to go to the web page containing the full search strategy, as translated by the database, and using Ctrl-F search for “AND.” Check whether the occurrences of the “AND” operator are deliberate.

Ideally, search strategies should be checked by other information specialists [ 18 ]. The Peer Review of Electronic Search Strategies (PRESS) checklist offers good guidance for this process [ 4 ]. Apart from the syntax (especially Boolean operators and field codes) of the search strategy, it is wise to have the search terms checked by the clinician or researcher familiar with the topic. At Erasmus MC, researchers and clinicians are involved during the complete process of structuring and optimizing the search strategy. Each word is added after the combined decision of the searcher and the researcher, with the possibility of directly comparing results with and without the new term.

14. Translate to other databases

To retrieve as many relevant references as possible, one has to search multiple databases. Translation of complex and exhaustive queries between different databases can be very time consuming and cumbersome. The single-line search strategy approach detailed above allows quick translations using the find and replace method in Microsoft Word (<Ctrl-H>).

At Erasmus MC, macros based on the find-and-replace method in Microsoft Word have been developed for easy and fast translation between the most used databases for biomedical and health sciences questions. The schema that is followed for the translation between databases is shown in Figure 2 . Most databases simply follow the structure set by the Embase.com search strategy. The translation from Emtree terms to MeSH terms for MEDLINE in Ovid often identifies new terms that need to be added to the Embase.com search strategy before the translation to other databases.

Schematic representation of translation between databases used at Erasmus University Medical Center

Dotted lines represent databases that are used in less than 80% of the searches.

Using five different macros, a thoroughly optimized query in Embase.com can be relatively quickly translated into eight major databases. Basic search strategies will be created to use in many, mostly smaller, databases, because such niche databases often do not have extensive thesauri or advanced syntax options. Also, there is not much need to use extensive syntax because the number of hits and, therefore, the amount of noise in these databases is generally low. In MEDLINE (Ovid), PsycINFO (Ovid), and CINAHL (EBSCOhost), the thesaurus terms must be adapted manually, as each database has its own custom thesaurus. These macros and instructions for their installation, use, and adaptation are available at bit.ly/databasemacros.

15. Test and reiterate

Ideally, exhaustive search strategies should retrieve all references that are covered in a specific database. For SR search strategies, checking searches for their recall is advised. This can be done after included references have been determined by the authors of the systematic review. If additional papers have been identified through other non-database methods (i.e., checking references in included studies), results that were not identified by the database searches should be examined. If these results were available in the databases but not located by the search strategy, the search strategy should be adapted to try to retrieve these results, as they may contain terms that were omitted in the original search strategies. This may enable the identification of additional relevant results.

A methodology for creating exhaustive search strategies has been created that describes all steps of the search process, starting with a question and resulting in thorough search strategies in multiple databases. Many of the steps described are not new, but together, they form a strong method creating high-quality, robust searches in a relatively short time frame.

Our methodology is intended to create thoroughness for literature searches. The optimization method, as described in step 11, will identify missed synonyms or thesaurus terms, unlike any other method that largely depends on predetermined keywords and synonyms. Using this method results in a much quicker search process, compared to traditional methods, especially because of the easier translation between databases and interfaces (step 13). The method is not a guarantee for speed, since speed depends on many factors, including experience. However, by following the steps and using the tools as described above, searchers can gain confidence first and increase speed through practice.

What is new?

This method encourages searchers to start their search development process using empty syntax first and later adding the thesaurus terms and free-text synonyms. We feel this helps the searcher to focus on the search terms, instead of on the structure of the search query. The optimization method in which new terms are found in the already retrieved articles is used in some other institutes as well but has to our knowledge not been described in the literature. The macros to translate search strategies between interfaces are unique in this method.

What is different compared to common practice?

Traditionally, librarians and information specialists have focused on creating complex, multi-line (also called line-by-line) search strategies, consisting of multiple record sets, and this method is frequently advised in the literature and handbooks [ 2 , 19 – 21 ]. Our method, instead, uses single-line searches, which is critical to its success. Single-line search strategies can be easily adapted by adding or dropping a term without having to recode numbers of record sets, which would be necessary in multi-line searches. They can easily be saved in a text document and repeated by copying and pasting for search updates. Single-line search strategies also allow easy translation to other syntaxes using find-and-replace technology to update field codes and other syntax elements or using macros (step 13).

When constructing a search strategy, the searcher might experience that certain parentheses in the syntax are unnecessary, such as parentheses around all search terms in the title/abstract portion, if there is only one such term, there are double parentheses in the proximity statement, or one of the word groups exists for only one word. One might be tempted to omit those parentheses for ease of reading and management. However, during the optimization process, the searcher is likely to find extra synonyms that might consist of one word. To add those terms to the first query (with reduced parentheses) requires adding extra parentheses (meticulously placing and counting them), whereas, in the latter search, it only requires proper placement of those terms.

Many search methods highly depend on the PICO framework. Research states that often PICO or PICOS is not suitable for every question [ 22 , 23 ]. There are other acronyms than PICO—such as sample, phenomenon of interest, design, evaluation, research type (SPIDER) [ 24 ]—but each is just a variant. In our method, the most important and specific elements of a question are being analyzed for building the best search strategy.

Though it is generally recommended that searchers search both MEDLINE and Embase, most use MEDLINE as the starting point. It is considered the gold standard for biomedical searching, partially due to historical reasons, since it was the first of its kind, and more so now that it is freely available via the PubMed interface. Our method can be used with any database as a starting point, but we use Embase instead of MEDLINE or another database for a number of reasons. First, Embase provides both unique content and the complete content of MEDLINE. Therefore, searching Embase will be, by definition, more complete than searching MEDLINE only. Second, the number of terms in Emtree (the Embase thesaurus) is three times as high as that of MeSH (the MEDLINE thesaurus). It is easier to find MeSH terms after all relevant Emtree terms have been identified than to start with MeSH and translate to Emtree.

At Erasmus MC, the researchers sit next to the information specialist during most of the search strategy design process. This way, the researchers can deliver immediate feedback on the relevance of proposed search terms and retrieved references. The search team then combines knowledge about databases with knowledge about the research topic, which is an important condition to create the highest quality searches.

Limitations of the method

One disadvantage of single-line searches compared to multi-line search strategies is that errors are harder to recognize. However, with the methods for optimization as described (step 11), errors are recognized easily because missed synonyms and spelling errors will be identified during the process. Also problematic is that more parentheses are needed, making it more difficult for the searcher and others to assess the logic of the search strategy. However, as parentheses and field codes are typed before the search terms are added (step 10), errors in parentheses can be prevented.

Our methodology works best if used in an interface that allows proximity searching. It is recommended that searchers with access to an interface with proximity searching capabilities select one of those as the initial database to develop and optimize the search strategy. Because the PubMed interface does not allow proximity searches, phrases or Boolean “AND” combinations are required. Phrase searching complicates the process and is more specific, with the higher risk of missing relevant articles, and using Boolean “AND” combinations increases sensitivity but at an often high loss of specificity. Due to some searchers’ lack of access to expensive databases or interfaces, the freely available PubMed interface may be necessary to use, though it should never be the sole database used for an SR [ 2 , 16 , 25 ]. A limitation of our method is that it works best with subscription-based and licensed resources.

Another limitation is the customization of the macros to a specific institution’s resources. The macros for the translation between different database interfaces only work between the interfaces as described. To mitigate this, we recommend using the find-and-replace functionality of text editors like Microsoft Word to ease the translation of syntaxes between other databases. Depending on one’s institutional resources, custom macros can be developed using similar methods.

Results of the method

Whether this method results in exhaustive searches where no important article is missed is difficult to determine, because the number of relevant articles is unknown for any topic. A comparison of several parameters of 73 published reviews that were based on a search developed with this method to 258 reviews that acknowledged information specialists from other Dutch academic hospitals shows that the performance of the searches following our method is comparable to those performed in other institutes but that the time needed to develop the search strategies was much shorter than the time reported for the other reviews [ 9 ].

CONCLUSIONS

With the described method, searchers can gain confidence in their search strategies by finding many relevant words and creating exhaustive search strategies quickly. The approach can be used when performing SR searches or for other purposes such as answering clinical questions, with different expectations of the search’s precision and recall. This method, with practice, provides a stepwise approach that facilitates the search strategy development process from question clarification to final iteration and beyond.

SUPPLEMENTAL FILE

Acknowledgments.

We highly appreciate the work that was done by our former colleague Louis Volkers, who in his twenty years as an information specialist in Erasmus MC laid the basis for our method. We thank Professor Oscar Franco for reviewing earlier drafts of this article.

Explore millions of high-quality primary sources and images from around the world, including artworks, maps, photographs, and more.

Explore migration issues through a variety of media types

• Part of The Streets are Talking: Public Forms of Creative Expression from Around the World
• Part of The Journal of Economic Perspectives, Vol. 34, No. 1 (Winter 2020)
• Part of Cato Institute (Aug. 3, 2021)
• Part of University of California Press
• Part of Open: Smithsonian National Museum of African American History & Culture
• Part of Indiana Journal of Global Legal Studies, Vol. 19, No. 1 (Winter 2012)
• Part of R Street Institute (Nov. 1, 2020)
• Part of Leuven University Press
• Part of UN Secretary-General Papers: Ban Ki-moon (2007-2016)
• Part of Perspectives on Terrorism, Vol. 12, No. 4 (August 2018)
• Part of Leveraging Lives: Serbia and Illegal Tunisian Migration to Europe, Carnegie Endowment for International Peace (Mar. 1, 2023)
• Part of UCL Press

Harness the power of visual materials—explore more than 3 million images now on JSTOR.

Enhance your scholarly research with underground newspapers, magazines, and journals.

Explore collections in the arts, sciences, and literature from the world’s leading museums, archives, and scholars.

This paper is in the following e-collection/theme issue:

Published on 11.4.2024 in Vol 26 (2024)

This is a member publication of Imperial College London (Jisc)

Regulatory Standards and Guidance for the Use of Health Apps for Self-Management in Sub-Saharan Africa: Scoping Review

Authors of this article:

• Benard Ayaka Bene 1, 2 , MBBS, MPH   ; 
• Sunny Ibeneme 3 , MD, PhD   ; 
• Kayode Philip Fadahunsi 1 , MBBS, MPH   ; 
• Bala Isa Harri 4 , MBBS, MPH, MSc   ; 
• Nkiruka Ukor 5 , MSc   ; 
• Nikolaos Mastellos 1 , BSc, PhD   ; 
• Azeem Majeed 1 , MD   ; 
• Josip Car 1, 6 , MSc, MD, PhD  

1 Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, United Kingdom

2 Department of Public Health, Federal Ministry of Health, Abuja, Nigeria

3 Digital Health Specialist, UNICEF East Asia Pacific Regional Office, Bangkok, Thailand

4 Department of Health Planning, Research and Statistics, Federal Ministry of Health, Abuja, Nigeria

5 Strategic Health Information Cluster, World Health Organization, Abuja, Nigeria

6 School of Life Course & Population Sciences, King’s College London, London, United Kingdom

Corresponding Author:

Benard Ayaka Bene, MBBS, MPH

Department of Primary Care and Public Health

School of Public Health

Imperial College London

The Reynolds Building

St Dunstan’s Road

London, W6 8RP

United Kingdom

Phone: 44 7598439185

Email: [email protected]

Background: Health apps are increasingly recognized as crucial tools for enhancing health care delivery. Many countries, particularly those in sub-Saharan Africa, can substantially benefit from using health apps to support self-management and thus help to achieve universal health coverage and the third sustainable development goal. However, most health apps published in app stores are of unknown or poor quality, which poses a risk to patient safety. Regulatory standards and guidance can help address this risk and promote patient safety.

Objective: This review aims to assess the regulatory standards and guidance for health apps supporting evidence-based best practices in sub-Saharan Africa with a focus on self-management.

Methods: A methodological framework for scoping reviews was applied. A search strategy was built and applied across the following databases, gray literature sources, and institutional websites: PubMed, Scopus, World Health Organization (WHO) African Index Medicus, OpenGrey, WHO Regional Office for Africa Library, ICTworks, WHO Directory of eHealth policies, HIS Strengthening Resource Center, International Telecommunication Union, Ministry of Health websites, and Google. The search covered the period between January 2005 and January 2024. The findings were analyzed using a deductive descriptive content analysis. The policy analysis framework was adapted and used to organize the findings. The Reporting Items for Stakeholder Analysis tool guided the identification and mapping of key stakeholders based on their roles in regulating health apps for self-management.

Results: The study included 49 documents from 31 sub-Saharan African countries. While all the documents were relevant for stakeholder identification and mapping, only 3 regulatory standards and guidance contained relevant information on regulation of health apps. These standards and guidance primarily aimed to build mutual trust; promote integration, inclusion, and equitable access to services; and address implementation issues and poor coordination. They provided guidance on systems quality, software acquisition and maintenance, security measures, data exchange, interoperability and integration, involvement of relevant stakeholders, and equitable access to services. To enhance implementation, the standards highlight that legal authority, coordination of activities, building capacity, and monitoring and evaluation are required. A number of stakeholders, including governments, regulatory bodies, funders, intergovernmental and nongovernmental organizations, academia, and the health care community, were identified to play key roles in regulating health apps.

Conclusions: Health apps have huge potential to support self-management in sub-Saharan Africa, but the lack of regulatory standards and guidance constitutes a major barrier. Hence, for these apps to be safely and effectively integrated into health care, more attention should be given to regulation. Learning from countries with effective regulations can help sub-Saharan Africa build a more robust and responsive regulatory system, ensuring the safe and beneficial use of health apps across the region.

International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2018-025714

Introduction

Health apps are the most widely used digital health products globally [ 1 , 2 ]. Harnessing the potential of health apps creates a huge opportunity in providing support for health care delivery, including patient communication, patient education, and decision support for self-management [ 3 - 8 ]. Health apps can be an effective tool to strengthen health systems worldwide, especially in low- and middle-income countries including those in sub-Saharan Africa [ 4 , 5 , 9 ]. As a result, the attainment of universal health coverage (UHC) and sustainable development goal (SDG) 3, good health and well-being, can be accelerated [ 8 , 10 ].

Many health apps fall below the expected quality threshold [ 11 ]. Several studies have found that widely used health apps are often technically unreliable and clinically unsafe [ 12 - 14 ] and do not comply with ethical standards and the principles of confidentiality of information and data privacy [ 15 , 16 ]. In addition, many commercially available health apps were not developed using interoperability standards that are widely accepted in sub-Saharan Africa (eg, Fast Healthcare Interoperability Resources [FHIR]) [ 17 - 20 ]. Consequently, it becomes difficult to integrate these apps into a clinical workflow.

Hence, regulation through robust mechanisms is crucial to enhance the development, implementation, and adoption of health apps. Regulatory standards and guidance are essential for the safety of patients as they ensure quality assurance of any new technology in health care and contribute to building mutual trust while promoting the optimal use of the technology [ 21 - 23 ]. Therefore, to ensure that health apps that are used to support the self-management of patients are technically reliable and clinically safe, interoperable across systems, and compliant with the principles of confidentiality of information and data privacy, there is a need for effective regulatory standards. Furthermore, effective regulation can help ensure that health apps for self-management are culturally functional and competent and are accessible to those who need them regardless of gender, ethnicity, geographical location, or financial status [ 24 - 31 ].

Since 2005, there have been ongoing efforts to strengthen digital health governance at both the national and international levels [ 32 , 33 ]. In 2018, the World Health Organization (WHO) member states renewed their commitment to using digital health technologies (DHTs) to advance UHC and SDG 3 [ 33 ]. However, to date, the extent to which the use of health apps for self-management is regulated across countries within the WHO African Region (also known as sub-Saharan Africa) remains unclear. Therefore, this review was conducted to identify available regulatory standards and guidance and assess the extent to which they regulate health apps for self-management in sub-Saharan Africa. The review also mapped out the key stakeholders and their roles in regulating health apps for self-management across sub-Saharan Africa.

Review Questions

The review attempted to answer the following questions: (1) What regulatory standards and guidance are available for regulating health apps for self-management across sub-Saharan Africa? (2) To what extent do regulatory standards and guidance regulate health apps for self-management in terms of what aspects are regulated; why, how, and for whom; and what aspects are not regulated? (3) Who are the key stakeholders and what are their roles in regulating health apps for self-management?

Study Design

The process of this scoping review followed the methodological framework for conducting a scoping study originally described by Arksey and O’Malley [ 34 ] and the updated methodological guidance for conducting a Joanna Briggs Institute scoping review [ 34 - 37 ]. The reporting of the review was guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist [ 38 ]. A completed PRISMA-ScR checklist is provided in Multimedia Appendix 1 . The protocol of this scoping review was published in BMJ Open [ 30 ].

Identifying Relevant Documents

Two reviewers (BAB and SI) developed the search strategy with the assistance of a librarian and in consultation with other research team members (KPF, BIH, NU, NM, AM, and JC). The following key terms were included: policy, legislation, strategy, regulation, standard, criterion, framework, guidance, guideline, digital health, eHealth, app, WHO African Region, and sub-Saharan Africa, and the names of all sub-Saharan African countries.

Owing to the absence of regulatory standards and guidance in scientific databases, the search focus was narrowed down to gray literature sources and institutional websites, including OpenGrey, WHO Regional Office for Africa (AFRO) Library, repositories for digital health policies (ICTworks, WHO’s Directory of eHealth Policies, and Health Information System Strengthening Resource Center), as well as the websites of WHO, International Telecommunication Union (ITU), and Ministries of Health (MOHs). The only scientific databases searched were PubMed, Scopus, and WHO AIM. PubMed was not included in the protocol. We also conducted a systematic search on Google. We used truncation to increase the yield of the results. The search strategy was then applied across PubMed, Scopus, and WHO AIM databases using Boolean terms (mainly OR and AND ) to combine search results. Gray literature sources and institutional websites were searched using phrases containing ≥2 keywords such as “eHealth regulation,” “digital health regulatory standard,” “eHealth regulatory standard,” “digital health regulation,” “digital health policy,” “eHealth policy,” “digital health strategy,” and “eHealth strategy.” For Google search, we added the names of the country to the phrases (eg, “digital health regulation Nigeria”). The reference lists of the included documents were also searched, and key individuals at the MOHs, WHO Country Offices, and the WHO AFRO were contacted for related documents. When our search was conducted, the WHO Directory of eHealth policies website was unavailable, and the WHO AFRO Library was undergoing reconstruction. The search strategies for PubMed, Scopus, and WHO AIM are provided in Multimedia Appendix 2 . The search was conducted between 2005 and January 2024.

Study Selection

The search results obtained from PubMed, Scopus, and WHO AIM were imported into Mendeley (Elsevier) [ 39 ] to remove duplicates. The search conducted on OpenGrey did not yield any results, whereas relevant records obtained from institutional websites, repositories, and Google were downloaded as PDF copies and uploaded to Mendeley. After removing duplicates, the remaining results were imported into Covidence (Veritas Health Innovation) [ 40 ] for screening. Two reviewers (BAB and SI) applied the predefined eligibility criteria ( Textbox 1 ) to screen the documents in 2 stages (title and abstract or executive summary). All discrepancies were discussed until the reviewers reached agreement.

Inclusion criteria

• Type of document: Regulatory standards, guidance, policies, strategies, and committee or government reports that address regulatory issues related to the use of health apps for self-management
• Location: Documents developed and implemented in countries within sub-Saharan Africa
• Date of publication: Documents developed since 2005; the global efforts toward promoting standards to minimize variability and potential harms that could arise from poorly regulated use of digital health began in 2005 [ 33 ]
• Language: Documents written in English language and other official languages of sub-Saharan African countries (Portuguese and French)

Exclusion criteria

• Type of document: Standards, guidance, policies, strategies, and reports not related to regulation of health apps
• Location: Documents from countries outside sub-Saharan Africa
• Date of publication: Documents developed before 2005
• Language: None

Data Charting (Extraction)

Two reviewers (BAB and SI), in consultation with the other members of the research team, developed the data extraction forms using an iterative process that included piloting data extraction and refinement until a consensus was reached.

We proposed in the study protocol [ 30 ] that data extraction would be conducted by the 2 reviewers independently. However, owing to the approach adopted for data extraction (deductive qualitative content analysis), 1 reviewer, rather than 2, initially extracted data from the included documents, and any concerns were discussed with a second reviewer [ 41 ]. Unresolved issues were then discussed and resolved with a third reviewer in a steering group meeting.

Collating, Summarizing, and Reporting Results

To address the research questions (particularly question 2), we adopted a deductive descriptive qualitative content analysis method to analyze and report the key findings. The policy analysis framework by Walt and Gilson [ 42 ] was adapted and applied to ensure that there was a consistent way of organizing the key findings: (1) Content (which aspects are regulated and which aspects are not?)—these are the components that directly or indirectly address regulatory issues related to the use of health apps for self-management, including areas that have not been addressed. (2) Context (why are those aspects regulated?)—this characterizes the rationale indicated for addressing regulatory issues related to the use of health apps for self-management. (3) Process (how are the regulatory standards developed and implemented?)—this describes the methods or approaches used to develop and implement regulatory standards. (4) Actors (who are the regulatory standards targeted toward?)—these are the key actors targeted by the standards.

Using a deductive descriptive qualitative content analysis approach, we examined each included document to systematically identify texts for concepts, patterns, and other relevant information. We then categorized them under content, context, process, or actors in relation to regulating health apps for self-management. The findings under content and context were further organized based on 4 predefined regulatory categories or themes as documented in the literature, namely (1) technical and clinical safety [ 12 - 14 ], (2) data protection and security [ 15 , 16 ], (3) standards and interoperability [ 28 , 31 ], and (4) inclusion and equitable access [ 24 - 29 ].

To address the third research question, the Reporting Items for Stakeholder Analysis (RISA) tool [ 41 ] was used as a guide to group key stakeholders based on role categorization as recognized globally by the WHO, the ITU, and UNESCO [ 32 , 33 , 43 ].

Ethical Considerations

Primary data were not collected in this study. Therefore, no ethics approval was required.

Search Results

A total of 2900 records were obtained after removing duplicates. Although the literature search was conducted in English, the search also yielded documents written in French and Portuguese from the ICTworks repository [ 44 ]. Following the initial screening of the title and abstract (or executive summaries), 73 documents were retrieved for full-text assessment. After applying the inclusion criteria for the full-text assessment, 49 documents were found eligible for inclusion in the review.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram [ 45 ] showing the study selection process is presented in Figure 1 .

Types of Documents

On the basis of the inclusion criteria, 3 categories of documents were considered for this review, namely “stand-alone regulatory standards and guidance that potentially regulate health apps for self-management,” “national policies and strategies on digital health,” and “other national documents that relate to the regulation of health apps for self-management.” Table 1 presents the types of documents obtained for each country within sub-Saharan Africa.

Characteristics of the Included Documents

Stand-alone regulatory standards and guidance.

We identified and included 6 stand-alone regulatory standards [ 18 , 19 , 46 - 49 ] from 3 countries (Ethiopia, Kenya, and Nigeria). All 6 documents were written in English. The years of development ranged between 2013 and 2021, as indicated in Multimedia Appendix 3 . The years of implementation were not specifically stated.

Although none of the included regulatory standards were exclusively developed to regulate health apps for self-management, 3 of them (Kenya Standards and Guidelines for mHealth Systems [ 18 ], Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ], and Health Sector Information and Communications Technology Standards and Guidelines [ 48 ]) provided concept and information relevant to the regulation of health apps and were included in the qualitative content analysis. The Kenya Standards and Guidelines for mHealth Systems [ 18 ] provides standards and guidelines on the design, development, and implementation of mobile health (mHealth) solutions to ensure they are interoperable, scalable, and sustainable. The Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] outlines the principles, requirements, and standards for eHealth systems interoperability in Kenya. The Health Sector Information and Communications Technology Standards and Guidelines [ 48 ] provide guidance and a consistent approach across the health sector in Kenya for establishing, acquiring, and maintaining current and future information systems and information and communications technology (ICT) infrastructure that foster interoperability across systems. These 3 documents are a good combination of regulatory standards and guidance that provide content and context relevant to the regulation of health apps in sub-Saharan Africa.

The remaining 3 standards (standard for electronic health record [EHR] system in Ethiopia [ 19 ], standards and guidelines for electronic medical record systems in Kenya [ 46 ], and the health information exchange standard operating procedure and guideline [ 49 ]) were exclusively developed for EHRs or electronic medical records. However, they contain information relevant for mapping stakeholders with potential roles in regulating health apps for supporting self-management.

National Policies and Strategies on Digital Health

This review includes 35 national policies and strategies that are related to digital health (potentially covering health apps) [ 50 - 84 ] from 31 countries written in English, French, and Portuguese (Benin, Botswana, Burkina Faso, Burundi, Cameroon, Comoros, Côte d’Ivoire [Ivory Coast], Democratic Republic of the Congo, Eswatini, Ethiopia, Gabon, Ghana, Kenya, Liberia, Madagascar, Malawi, Mali, Mauritius, Mozambique, Namibia, Niger, Nigeria, Rwanda, Senegal, Sierra Leone, South Africa, Tanzania, Togo, Uganda, Zambia, and Zimbabwe). Although the literature search was conducted in English, it also yielded documents written in French and Portuguese from the ICTworks repository. The years of development and implementation range between 2005 and 2030. Policies and strategies written in French and Portuguese were translated into English using Google Translate. Documents labeled as national development plans, strategic plans, and strategic development plans were considered as national strategies.

National policies and strategies do not offer specific standards or guidance, but rather outline the country’s vision, policy directions, and strategies for using digital technologies in health care. They provide useful information for identifying digital health stakeholders who can play a role in regulating health apps for self-management. For example, Nigeria has a separate National Digital Health Policy [ 72 ] and a National Digital Health Strategy [ 71 ]. Both documents were developed by building on the lessons learned from the end-term evaluation of the previous National Health ICT Strategic Framework [ 85 ]. They describe Nigeria’s renewed vision, mission, goals, objectives, and strategies for the development and implementation of digital health with the aim to improve the quality, efficiency, and effectiveness of health service delivery and health outcomes.

It is worth noting that for countries with >1 policy or strategy, we included only the most recent versions. For instance, as mentioned earlier, Nigeria now has both a national digital health policy and a national digital health strategy. These 2 documents supersede and thus replace the old National Health ICT Strategic Framework [ 86 ]. Details of included documents are presented in Multimedia Appendix 3 .

Other Related National Documents

We included 8 other documents [ 20 , 85 , 87 - 92 ] from 6 countries (Ethiopia, Kenya, Liberia, Nigeria, South Africa, and Tanzania) that did not fall under either stand-alone regulatory standards and guidance or national policies and strategies. These were mostly frameworks, road maps, and reports that potentially provide information relevant to the use of health apps. The years of development and implementation range from 2016 to 2025. These documents do not provide standards or guidance, but they contain information that can help map the digital health stakeholders that potentially play a role in regulating health apps for self-management. When multiple versions of a document exist, only the latest version was taken into consideration. Multimedia Appendix 3 provides details of the included documents.

Content: Aspects That Are Regulated and Aspects That Are Not

Technical and clinical safety.

Technical and clinical safety standards are required to prevent or minimize the harm that may arise from the use of the health ICT systems (including mHealth systems) as well as to improve the health outcomes and user satisfaction. As shown in Figure 2 , two subthemes were generated from included standards [ 18 , 47 , 48 ] as content under technical and clinical safety: v(1) guidance on system quality and (2) guidance on software or app development, acquisition, support, and maintenance.

Notably, 2 of the included standards [ 18 , 47 ] provide guidance on system quality to ensure the quality, security, reliability, performance, and maintenance of eHealth and mHealth systems. The Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] recommend the implementation of a data quality protocol to ensure that the data collection, collation, analysis, interpretation, dissemination, and use are managed in accordance with the quality standards. Similarly, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] recommends the inclusion of the following requirements in the technical manual: (1) minimum hardware requirements that should incorporate the preferred hardware architecture, (2) minimum software requirements that should include the minimum version of the underlying operating system as well as acceptable versions of related software, and (3) a detailed list of software dependencies (external libraries) necessary for the system to function properly.

The included standards [ 18 , 48 ] cover guidance on software or app development, acquisition, support, and maintenance, which aim to ensure the efficiency and effectiveness of eHealth and mHealth systems. The Kenya Standards and Guidelines for mHealth Systems [ 18 ] recommends a technical manual to provide a detailed description of the system’s installation and maintenance processes for system administrators and implementers; a developer’s guide for software developers and programmers to provide them with an overview of the system, description of the software design methodologies, description of the system architecture, and technical design diagrams; and a user manual to aid users in understanding how the system works and how each feature operates; in addition, the technical manual contains instructions for operating the software; entering and updating data; and generating, saving, and printing reports.

Although the contents generated here provide guidance that is relevant to health apps, they are not specific to health apps. Moreover, there are no clear measures to enable individuals or organizations that use health apps to manage clinical risk appropriately.

Data Protection and Security

Data protection and security are crucial aspects of managing patient information, thus ensuring the confidentiality, integrity, and availability of data as well as the rights and interests of the patient. Two subthemes related to data protection and security are (1) security measures for adequate protection of patients’ digital records and (2) guidance on data exchange.

The included standards [ 18 , 48 ] provide security measures for eHealth or mHealth systems to ensure the adequate protection of digitally accessible patient records. These measures include authentication, accountability, identification, authorization, integrity, confidentiality, availability, security, administration, and audit. This will help to achieve confidentiality, integrity, availability, and nonrepudiation of patient data or health records. Additional levels of security such as data encryption are required when there is a need to store sensitive information on removable devices or media or outside the MOH premises.

The Kenya Standards and Guidelines for mHealth Systems [ 18 ] provide the following guidance on data exchange to ensure privacy: (1) anonymize client data as much as possible before they can be shared; (2) where possible, use pseudonyms for the client data before they can be shared; (3) aggregate client data before they can be shared to reduce possibilities of tracing the data back to the client; and (4) minimize data so that access is available only to the data set required for that particular use. With regard to privacy rules, the Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] propose that a notice of privacy practices should be given to patients describing how their information may be used or shared while also specifying their legal rights.

Standards and Interoperability

Standards and interoperability are essential concepts in the field of IT, especially for systems that need to communicate and exchange data, as seen in the use of health apps for self-management. Two subthemes related to standards and interoperability are (1) interoperability as a basic requirement and (2) minimum standards to enable integration.

All the regulatory standards [ 18 , 47 , 48 ] highlight the importance of having interoperability as a basic requirement when selecting software products or services for use within the health system. This facilitates interaction across systems. For instance, to facilitate seamless interaction between mHealth systems and primary information systems for data capture, reporting, and decision support in various domains of the health system, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] recommends the incorporation of at least 3 types of interoperability, namely, technical interoperability, semantic interoperability, and process interoperability.

Furthermore, 2 regulatory standards [ 18 , 47 ] proposed minimum interoperability standards to enable the integration of services and data exchange between various systems in health care. For instance, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] suggests standards (for interoperability) for mHealth systems that are consistent with the recommendations in internationally accepted standards. They include the following: (1) clinical messaging—ensuring mHealth systems conform to Health Level 7 (HL7) version 3 standards and corresponding implementation guideline; (2) clinical terminology—ensuring terminologies and classifications for clinical concepts (eg, International Classification of Diseases, tenth revision—for diseases; Systemized Nomenclature of Medicine—for clinical data coding; Logical Observation Identifiers Names and Codes—for laboratories; and RxNorm—for Pharmacies); (3) the mHealth system must use the latest versions of international standards, such as HL7 Clinical Document Architecture for electronic sharing of clinical documents; (4) concepts—mHealth systems will use the idea of “concepts” so that information can be transmitted between systems without losing meaning or context, and HL7 Reference Implementation Model or other appropriate standards are recommended for implementing concepts; (5) architecture—to develop mHealth systems, developers should define the system architecture that should include data elements and business logic. Furthermore, to define how mHealth systems interact with other systems, developers of mHealth solutions must provide application programming interfaces. FHIR is the preferred application programming interface interoperability standard.

Inclusion and Equitable Access

Inclusion and equitable access are essential principles to ensure that health apps are culturally appropriate and relevant and accessible to everyone, regardless of gender, ethnicity, location, or economic status.

All the included regulatory standards [ 18 , 47 , 48 ] indicate that they were developed based on a combination of participatory and consultative approaches involving multiple actors or stakeholders, thus promoting inclusion. However, there are no specific measures or guidance to ensure adequate engagement and representation of all the relevant stakeholders and to sustain that engagement.

The Kenya Standards and Guidelines for mHealth Systems [ 18 ] proposes the following systems attributes to ensure equitable access to mHealth services at all times and from anywhere: (1) allocation of adequate storage and bandwidth capacity; (2) fast response time; (3) fast recovery capabilities; (4) performance monitoring; (5) business continuity processes, for example, backups; and (6) redundant sites and links. Furthermore, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] prescribes the following metrics for measuring system availability: (1) downtime per year, (2) mean time between failure, (3) mean time to repair, and (4) failure in time.

Although the abovementioned systems attributes and metrics for measuring system availability are important, the included standards do not offer any concrete guidance or model for achieving a sustainable funding mechanism for health apps to ensure that they are readily available and accessible to those who need them.

Context: Reasons Why Those Aspects Are Regulated

The 3 standards [ 18 , 47 , 48 ] were developed to address unsafe, isolated, and inconsistent implementation. The Health Sector ICT Standards and Guidelines [ 48 ] suggest that although there has been a lot of ICT investment in the health sector leading to improvement in service delivery and information exchange, there remains the challenge of inconsistency in ICT implementation and harmonization of the health sector system requirements. Hence, there is a need to adopt global best practices for software development, acquisition, support, and maintenance by the MOH. In addition, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] indicates that standards and guidelines are necessary to ensure a consistent approach to the development of ICT systems. Similarly, the Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] recognize the need to ensure that the processes of collecting, collating, analyzing, interpreting, disseminating, and using data are consistent with data quality standards.

To build mutual trust and maximize the benefits of eHealth information exchange, the Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] reiterate that as health data are constantly being exchanged across health information systems, robust security standards are required to maintain their integrity and confidentiality. This will build the trust of service users and consequently help to maximize the benefits of eHealth information exchange such as in self-management.

Two of the included regulatory standards [ 47 , 48 ] indicate that the context for standards and interoperability was (1) to address poor coordination, duplication of efforts, and inefficient use of resources and (2) to promote the integration of ICT systems.

The Kenya Standards and Guidelines for E-Health Systems Interoperability [ 47 ] acknowledge that the absence of interoperability standards over the years has led to the duplication of efforts and the inefficient use of ICT resources in health care. Now that ICT has become increasingly relevant in improving efficiency in health service delivery, the Kenya MOH recognizes the need to adopt a standardized approach, hence the development of interoperability standards for eHealth systems. In addition, the Health Sector ICT Standards and Guidelines [ 48 ] emphasize the relevance of interoperability as a requirement for addressing the inconsistency in implementing ICT in the health sector.

The Health Sector ICT Standards and Guidelines [ 48 ] consider “integration of ICT systems” as one of its key guiding principles, acknowledging the lack of information systems integration as a challenge experienced by ICT services across Kenya.

The contexts for inclusion and equitable access as generated from included standards [ 18 , 47 , 48 ] were (1) to promote inclusion and (2) to promote equitable access to services.

To promote inclusion, the standards [ 18 , 47 , 48 ] highlight the importance of involving and engaging multiple actors and stakeholders during the development process. However, no emphasis was placed on the need to sustain stakeholder engagement during the implementation process.

Pertaining to equitable access, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] acknowledges that the public health care system is largely unavailable to most of the population in many developing countries because of geographical location, resource constraints, inefficiencies, and lack of awareness. Hence, it recognizes the importance of ensuring that mHealth services are always accessible by users and from anywhere as well as the need to put in place mechanisms to make this happen.

Process: How the Regulations Are Developed and Implemented

Two themes were generated from the included standards: development and implementation processes [ 18 , 47 , 48 ].

Development Process

All the included standards [ 18 , 47 , 48 ] indicate that they were developed through a participatory process and in consultation with a range of subject experts and interest groups. In addition, the standards [ 18 , 47 , 48 ] adopted a multisectoral approach to engage health-related stakeholders from government ministries or agencies and development partners and a range of subject experts and interest groups. It has also been reported that these standards [ 18 , 47 , 48 ] were developed based on international best practices and with reference to international standards. However, there is no indication that a stakeholder engagement strategy was adopted to sustain the engagement of stakeholders through the entire development and implementation process.

Implementation Process

The 3 regulatory standards [ 18 , 47 , 48 ] identify the key requirements to ensure effective implementation of IT services in the health sector. These are (1) legal authority, (2) coordination, (3) building capacity, and (4) monitoring and evaluation.

The included standards [ 18 , 47 , 48 ] were established based on the legal provisions enshrined in the health and other related acts and laws of Kenya as well as the relevant policies and strategies. Hence, it is expected that their implementation will comply with and be backed by those legal provisions. For example, the Health Sector ICT Standards and Guidelines [ 48 ] indicate that its implementation will be supported by the authority from the Kenya Communications Act 2009, E-Government Strategy, and National ICT Policy. Similarly, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] asserts that it will be implemented by complying with existing and relevant national policies, legal frameworks, strategies, and standards, including the Health Information Policy, ICT Standards, and System Interoperability Principles.

The included standards [ 18 , 47 , 48 ] report that the implementation of regulations will require robust coordination mechanisms. For instance, the Health Sector ICT Standards and Guidelines [ 48 ] indicate that, as the Ministry’s ICT resource manager, the principal secretary (also the head of ICT), in collaboration with the ICT Governance Committee, is responsible for coordinating the implementation of the standard. The ICT Governance Committee comprises representatives from the heads of departments and ICT development partners in the health sector. The committee’s responsibilities include overseeing, enforcing, and reviewing standards as well as initiating ICT projects.

The Health Sector ICT Standards and Guidelines [ 48 ] highlight the need for capacity building or training of the MOH staff and stakeholders who are the primary users of the Ministry’s ICT services. This will enhance their capacity to implement the guidelines provided in the document in line with the ministry’s human resource development policies, regulations, and rules. However, it is acknowledged that building capacity for health ICT is a challenge given that there is low adoption of ICT among health providers, and ICT is not routinely included in the course content of most training programs. The Kenya Standards and Guidelines for mHealth Systems [ 18 ] listed the “number of mHealth practitioners trained on the standards and guidelines” as one of the indicators for monitoring and evaluating mHealth interventions.

The Health Sector ICT Standards and Guidelines [ 48 ] assert that monitoring and evaluation is an essential role of the MOH to ensure efficiency, accountability, and transparency throughout the implementation period. It further stresses that all those who use the Ministry’s ICT services are required to adhere to the provisions in the standard as the MOH will carry out quarterly monitoring exercises on the use of the standard to ensure compliance based on clear indicators. Furthermore, the ICT Governance Committee will periodically review and amend the standard to keep it relevant and effective. Similarly, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] establishes the following key indicators for effectively monitoring and evaluating the implementation of the standards and guidelines: (1) the number of counties in which the MOH has disseminated the standards and guidelines, (2) the number of counties successfully implementing the standards and guidelines, (3) the number of mHealth practitioners trained on the standards and guidelines, (4) the number of mHealth practitioners accessing the standards and guidelines, (5) the number of mHealth practitioners who correctly understand the standards and guidelines, (6) the number of stakeholders who adhere to the standards and guidelines, (7) the number of mHealth systems that follow the required development steps, and (8) the number of mHealth practitioners who have implemented their systems by using the standards and guidelines. In addition, the Kenya Standards and Guidelines for mHealth Systems [ 18 ] indicates that the outlined standards will be reviewed every 3 years to ensure they are up to date with new changes including the changes in policies and systems upgrades.

Although all the abovementioned indicators are relevant, the implementation process is not explicit on the approach for regulating health apps and ensuring compliance with regulatory standards and guidance.

Actors: Those the Regulations Are Targeted at

The included standards [ 18 , 47 , 48 ] identified 2 main groups of actors for whom the regulations and guidance were targeted. They included (1) those who provide digital health services and (2) those who use the ICT infrastructure of the MOH.

Two of the standards [ 47 , 48 ] indicated that the regulations should be implemented by all individuals and organizations that provide ICT-related health care services to the public. Similarly, the Health Sector ICT Standards and Guidelines [ 48 ] state that all those who access or use the MOH ICT infrastructure are expected to adhere to the guidelines outlined in the document.

Mapping of Stakeholders

To address the third research question, we conducted a stakeholder mapping guided by the RISA tool [ 41 ].

A total of 11 categories of key stakeholders were identified from all 49 included documents (6 stand-alone regulatory standards and guidance, 35 national policies or strategies, and 8 other related documents). These categories are consistent with the digital health stakeholders recognized by the WHO, ITU, and UNESCO [ 32 , 33 , 43 ]. Table 2 presents the mapping of stakeholders according to their role categorization. A more detailed table with a potential role description with regard to regulating health apps for self-management is presented in Multimedia Appendix 4 .

a WHO: World Health Organization.

This paper presents the findings of a scoping review of regulatory standards and guidance for the use of health apps for self-management in sub-Saharan Africa. To the best of our knowledge, this is the first study that attempted to identify and assess the extent to which regulatory standards and guidance regulate and guide the use of health apps for self-management in sub-Saharan Africa as well as map out the key stakeholders and their potential roles.

Our findings reveal that only 1 country (Kenya) in sub-Saharan Africa currently has national regulatory standards that could potentially regulate the use of health apps for self-management. The included standards failed to adequately address adequate attention to inclusion and equitable access. This is concerning given the growing need to promote the adoption of culturally appropriate and relevant health apps and to ensure that they are available to those who need them regardless of gender, ethnicity, geographical location, or financial status [ 24 - 29 ]. Consequently, this review provides insights into the regulation of health apps for self-management in sub-Saharan Africa, which needs to be given more attention if the potential of these apps is to be harnessed in the region.

Principal Findings

We identified 49 documents from 31 countries in sub-Saharan Africa. Although none of the included standards provided a specific set of regulations on health apps for self-management, we identified 3 standards [ 18 , 47 , 48 ] that provided relevant information regarding the regulation of health apps. The included national policies and strategies, in contrast, only outline the goals and commitments made by national governments to promote the adoption of digital technologies in the health sector and the plans and paths set forth to achieve these goals. However, the information they provided was relevant for identifying and mapping digital health stakeholders who potentially have vital roles in regulating the use of health apps for self-management.

The policy analysis framework (content, context, process, and actors) [ 42 ] was adapted and applied to organize the key findings. The content covered the following areas: guidance on systems quality; guidance on software and app development, acquisition, support, and maintenance; security measures for adequate protection of patients’ digital records; guidance on data exchange; interoperability as a basic requirement; minimum standards to enable integration; involvement and engagement of relevant stakeholders; and system attributes for equitable access to services. Meanwhile, the context was to address unsafe, isolated, and inconsistent implementation; to build mutual trust and maximize the benefits of eHealth information exchange; to address poor coordination, duplication of efforts, and inefficient use of resources; to promote the integration of ICT systems; and to promote inclusion and equitable access to services. The process involved the development process (which covers participatory and consultative processes and multisectoral approach, with reference to international standards and best practices) and the implementation process (which covers legal authority, coordination, capacity building, and monitoring and evaluation). The targeted actors were those who provided digital health services and those who used the ICT infrastructure of the MOH.

Furthermore, key stakeholders with potential roles in regulating health apps for self-management were identified. They include the government, regulatory bodies, funders, intergovernmental and nongovernmental organizations, academia, and the health care community.

Implications of the Study Findings for Practice

Regulatory standards and guidance act as a bridge between technological innovation and its safe and effective use in health care. They ensure that while technology continues to advance, the safety and trust of patients are never compromised. Among the plethora of health apps on the market, the over-the-counter, nonregulated apps such as wellness and fitness apps are the most mainstream [ 93 - 95 ]. On the other side of the spectrum, there are regulated health apps that are classified under medical devices or software as medical device products [ 94 , 95 ]. Some of these are prescription-only apps, such as digital therapeutics (DTx) apps for managing substance dependence [ 95 , 96 ].

Although some high-income countries have made significant strides in ensuring the safety, effectiveness, and accessibility of health apps, the journey has indeed not been without challenges and hurdles. Sub-Saharan Africa, although dealing with its own unique set of challenges, has the opportunity to learn from the experiences of these high-income countries. This could potentially allow the region to bypass some of the hurdles encountered by high-income countries in their journeys.

Technical and clinical safety are essential requirements that health apps must meet before they can be considered for use for self-management to minimize the risk of harm to patients. It is well documented that health apps that function poorly pose a serious threat to the safety of patients. An example illustrating how health apps used for self-management can threaten patient safety is evident in a study [ 12 ]. This study [ 12 ] revealed that widely used health apps designed to calculate and estimate insulin doses could endanger patients by providing incorrect or inappropriate dose recommendations. Similarly, 2 successive studies that assessed the contents and tools of apps for asthma discovered that none of the apps in the first study offered comprehensive information or adequate tools for asthma self-management, whereas the follow-up study, which was conducted 2 years later, showed a 2-fold increase in the number of asthma apps, yet there was no improvement in the content and tools offered by the newer apps. In fact, many apps recommended self-management procedures that were not supported by evidence [ 13 , 14 ]. Accordingly, some health apps that support the self-management of long-term conditions do not adhere to evidence-based guidelines and are unresponsive to the evolving health needs of patients.

Although the context of included regulatory standards with regard to technical and clinical safety was to address unsafe, isolated, and inconsistent implementation, the guidance provided by these regulatory standards is not specific to health apps, and they do not provide appropriate guidance and standards for health organizations and other key stakeholders to establish a framework for managing the clinical risks associated with deploying and implementing self-management health apps. Considering the rapid advancements in digital health (including artificial intelligence [AI] or machine learning and big data), health apps will increasingly play a crucial role in supporting self-management through digitally enabled care pathways that will improve personalized care and health outcomes [ 97 , 98 ]. Therefore, it is imperative to ensure the technical reliability and clinical safety of health apps for self-management through robust regulatory standards and guidance. For instance, a guide on the criteria for health app assessment, developed by the UK government, includes technical stability and clinical safety as criteria for deciding whether health apps should be considered for use in the National Health Service (NHS) [ 99 ]. In addition, medical device apps are required to conform to the NHS clinical risk management standards as part of the clinical safety requirements [ 99 , 100 ]. In the event of any concerns regarding the safety of a medical device app, the Yellow Card reporting system can be used by a responsible clinical safety officer or any other individual to notify the Medicines and Healthcare products Regulatory Agency (MHRA) [ 101 , 102 ].

To adequately manage patient information when health apps are used for self-management, data protection and security standards and guidance are required. They guarantee that data are kept and handled safely and responsibly within the provisions of the law and that patients’ rights and interests are respected.

There have been ongoing concerns about compliance with ethical standards, the principles of confidentiality of information, and data privacy. For example, an assessment of apps that had previously been endorsed by the former UK NHS Apps Library revealed substantial gaps in compliance with data protection principles regarding the collection, storage, and transmission of personal information. This has raised a fundamental concern about the credibility of developer disclosures and whether these disclosures can be trusted by certification programs [ 15 ]. A study assessed the privacy practices of the 36 most popular apps for depression and smoking cessation for Android and iOS in the United States and Australia [ 16 ]. The findings revealed that although only 69% (25/36) of the apps included a privacy policy, 92% (33/36) of the apps shared data with a third party, and only 92% (23/25 with privacy policy) of the apps disclosed sharing data with a third party in their policy. Although 81% (29/36) of the apps shared data with Google and Facebook for the purposes of advertising, marketing, or analytics, only 43% (12/28) of the apps that shared data with Google and 50% (6/12) of the apps that shared data with Facebook disclosed this in their policy [ 16 ].

In this regard, health app developers and providers in the United Kingdom are required to conduct a data protection risk assessment before they launch or update their apps to ensure compliance with the United Kingdom General Data Protection Regulation (GDPR) and other relevant regulations, including the Data Protection Act 2018 [ 103 ]. By conducting a data protection risk assessment, health app developers and providers can demonstrate that they are accountable; they respect the privacy and dignity of their users; and that they deliver safe, effective, and ethical solutions [ 104 ].

Health apps are expected to play an increasingly important role in supporting self-management. However, this ambition can only be achieved if citizens trust that these apps are collecting and analyzing data safely and in accordance with robust regulatory standards and guidance. It is also crucial that these apps provide reliable information that clinicians can act on [ 98 ]. The context of the standards included in this study regarding data protection and security was to build mutual trust and maximize the benefits of eHealth information exchange. Trust is a key factor in the successful adoption and use of health apps, and transparency in data handling and clinical decision-making is essential to build and maintain that trust. This is also paramount for the widespread acceptance and impact of health apps on health care outcomes in sub-Saharan Africa.

We acknowledge the existence of numerous national laws related to data protection and security outside the health sector. Hence, guidelines that link these legislations together must be provided to ensure compliance with all relevant laws and guidance when using patient data. An example of how to achieve this is the United Kingdome’s guide to good practice for digital and data-driven health technologies that provides guidelines on how to abide by the laws and principles that govern data security and protection in the United Kingdom, including the GDPR, Data Protection Act 2018, and Caldicott Principles [ 105 ].

Standards and interoperability are essential for effectively developing, deploying, and implementing health apps to support self-management in sub-Saharan Africa. Interoperability is the ability of different systems, devices, or applications to communicate and exchange data with each other in a coordinated manner, thus providing timely and seamless portable information across organizational, regional, and national boundaries and optimizing both individual and population health [ 106 ]. In the same vein, standards enable interoperability between systems or devices through a common language and a common set of expectations [ 106 ].

Interoperability is crucial in improving the quality, safety, and efficiency of care delivery as well as empowering patients and providers with access to relevant and timely information [ 99 ]. One of the most widely used and accepted interoperability standards for health care data exchange is FHIR [ 106 , 107 ]. FHIR is a global industry standard developed by HL7 International. FHIR is designed to be quick to learn and implement and to support a variety of use cases, including self-management [ 108 ]. By using apps that are based on an FHIR standard, patients can benefit from data analytics that show how their health data relate to their chronic conditions or wellness goals [ 109 ]. They could also access all their health information from one place, even if they visit different health professionals who use different electronic medical records or EHR, thus promoting integrated care [ 28 , 31 , 33 , 109 - 115 ]. As a result, patient care can easily be coordinated.

The context of the included regulatory standards with regard to standards and interoperability was to address poor coordination, duplication of efforts, and inefficient use of resources and to promote the integration of ICT systems. However, in sub-Saharan Africa, there are many challenges and barriers to the adoption and implementation of interoperability standards, such as the lack of awareness or knowledge of the benefits and requirements of interoperability standards among stakeholders; lack of incentives or regulations to encourage or enforce the adoption of interoperability standards by app developers and vendors; lack of resources or capacity to implement interoperability standards, including technical expertise, infrastructure, funding, or governance; and lack of alignment or coordination among the different actors and initiatives involved in developing, deploying, and implementing the digital health interventions [ 30 , 116 - 119 ]. To address these challenges, some possible solutions may include raising awareness and education on the importance and value of interoperability standards for health apps among all relevant actors; developing and implementing policies and guidelines that promote or mandate the use of interoperability standards by app developers and vendors; providing technical assistance and support for app developers and vendors to adopt and implement interoperability standards, such as tools, frameworks, testing, certification, or accreditation; and establishing and strengthening collaboration and coordination among the different stakeholders and initiatives involved in health app development, deployment, and implementation in sub-Saharan Africa. In addition, the Digital Health Platform Handbook, a toolkit developed by the collaborative efforts of the WHO and ITU [ 120 ], can help countries in sub-Saharan Africa to develop and implement digital health platforms as the underlying infrastructure for interoperable and integrated national digital health systems. The digital health platform is a system-wide approach to developing digital health solutions with the aim to overcome the problems of siloed, vertical, and isolated applications and systems that hamper data management, innovation, efficiency, and impact in the health sector.

Inclusion and equitable access are crucial to ensuring that health apps and related services are culturally appropriate and relevant as well as accessible to all who need them, regardless of gender, ethnicity, geographical location, ability, or financial status [ 24 - 29 ]. This is the key to promoting a “sense of belonging” and “ownership” and thus underscoring the importance of stakeholder mapping and involvement or engagement through the development and implementation process [ 22 ].

In this study, the included regulatory standards demonstrate the importance of inclusion by adopting both a participatory and consultative approach involving multiple stakeholders from different sectors. However, the standards do not provide clear guidance to ensure the adequate participation and sustained engagement of all relevant stakeholders. The lack of concise guidance to ensure the adequate participation and engagement of all relevant stakeholders, especially the susceptible and disadvantaged groups, can increase the risk of tokenistic tendencies, which can undermine the cultural appropriateness of health apps [ 25 , 121 ]. Some susceptible groups, such as women and people with low socioeconomic status, may face additional barriers to accessing and using health apps, such as lack of digital literacy, privacy concerns, cultural norms, or stigma [ 25 ]. Similarly, the cost of developing, maintaining, and updating health apps may not be covered by public or private health insurance schemes, which could limit their affordability and availability for low-income or uninsured populations [ 95 ]. However, there is no specific guidance or model for an effective funding mechanism for health apps in the included regulatory standards.

To address these challenges and ensure equitable access to health apps for self-management in sub-Saharan Africa, possible measures may include developing policies and regulations that support integrating health app interventions into existing health systems and financing mechanisms and engaging with stakeholders from different sectors and backgrounds (including health professionals, patients, communities, governments, civil society, academia, and industry) to co-develop and co-implement frameworks or models that promote the use of health apps for self-management in ways that are responsive to the local context and needs. Moreover, establishing regulations that provide appropriate financing or reimbursement options will reduce the risk of developers of good quality health apps turning to data mining for revenue, thus increasing privacy concerns [ 95 ]. For instance, in Germany, the reimbursement of health apps classified as medical devices (Digitale Gesundheitsanwendungen) was introduced in 2021 under the statutory health insurance [ 122 , 123 ]. When a medical device is prescribed by a physician or a physiotherapist, the manufacturer must submit an application to the German Federal Institute for Drugs and Medical Devices (Bundesinstitut für Arzneimittel und Medizinprodukte) for approval [ 123 ]. The Federal Association of the Statutory Health Insurance Funds (Spitzenverband Bund der Krankenkassen) determines and negotiates the reimbursement thresholds following approval. However, the manufacturer must demonstrate that the app is safe, functional, and of good quality; complies with data protection requirements; and benefits patient care [ 123 ].

The process of regulating health apps essentially involves the development and implementation of regulatory standards and guidance. According to our study, the development process comprises a participatory and consultative process, a multisectoral approach, and a reference to international standards and best practices. In contrast, the implementation process is ongoing and requires appropriate legal authority, coordination, capacity building, and monitoring and evaluation.

We recognize that health apps can be accessed and used by patients from different parts of the world, and this means that countries need to carefully consider whether health apps that are accessed and used by their citizens meet the national or regional legal and ethical requirements, including their cultural and linguistic needs [ 23 ]. For countries in sub-Saharan Africa, a cross-border or regional collaboration between national legal authorities through the coordination of agencies such as the African Medicines Regulatory Harmonization (AMRH) may help to ensure that health apps built for the region are safe, effective, and user-friendly for everyone, considering the contextual differences of the countries [ 23 ]. For instance, all medical device companies that want to sell their products in the European market must obtain a Conformité Européenne (CE) mark for their devices, which indicates that they meet the legal requirements and can be freely circulated within the European Union [ 124 ]. Although the European Union member states regulate medical devices, the European Medicines Agency is involved in the regulatory process.

The regulation of health apps is extremely complex and involves a wide range of stakeholders. Therefore, a robust coordination mechanism is essential to reduce the risk of fragmentation and duplication of efforts and to promote the efficient use of resources. Most countries in sub-Saharan Africa have units in health ministries that coordinate and oversee the regulation of medical products. These units should be autonomous, full-fledged departments with legal authority (boards or commissions) to ensure independent, transparent, and accountable decision-making, but this is often not the case [ 125 ]. These units are recognized by the national authorities as regulators (eg, the National Medicines Regulatory Authority [NMRA]) [ 126 ]. Such organizational structures hinder the effectiveness of the national regulatory authorities in fulfilling their mandate and prevent the establishment of quality management systems to ensure transparent and accountable decision-making [ 125 ].

Furthermore, Essén et al [ 23 ] analyzed health app policy or regulation in 9 high-income countries (Sweden, Norway, Denmark, Netherlands, Belgium, Germany, England, the United States, and Singapore) and found that most of these countries adopted centralized approaches to app evaluation. Although centralized approaches might have advantages over self-evaluation, they may create bottlenecks and limit the availability of high-quality health apps for users. As suggested by Essén et al [ 23 ], a decentralized approach, such as the accreditation of evaluation agencies, maybe a worthwhile solution. However, this will require adequate coordination to ensure the consistency and reliability of the evaluation criteria and methods across different agencies as well as the transparency and accountability of the accreditation process. A possible way to achieve this is to adopt a common framework that can guide the evaluation and accreditation of health apps.

Similarly, the postmarket surveillance (PMS) system, which is a new regulation for medical devices in Europe, is a process of collecting and analyzing data on medical devices after they have been launched into the market to ensure their safety and performance and to identify any problems or need for improvements [ 127 , 128 ]. The PMS system is important because premarket data, which are obtained from testing a medical device before it is launched, have limitations in capturing the long-term performance and risks of the device [ 128 ]. Currently, the PMS system does not cover fitness and wellness apps, which are commonly used in self-management. Hence, Yu [ 93 ] proposed that the PMS system should also be applied to DHTs, such as fitness and wellness apps. They argue that the postmarket data would help regulators periodically review and adjust the regulatory standards for these groups of health apps based on their risks and benefits.

Drawing on the experience of the United Kingdom, it can be clearly demonstrated that the regulation of health apps is a complex, a multifaceted, and an evolving process that involves different regulators and criteria depending on the nature and function of the app. For instance, a centralized NHS Apps Library was launched as a beta site in April 2017 to provide patients with a collection of trusted and easy-to-use digital health tools [ 129 ]. The library provided access to a range of health apps that were reviewed and approved by the NHS, including apps that could help patients manage conditions such as diabetes, mental health, and chronic obstructive pulmonary disease [ 130 ]. However, the library was closed in December 2021 [ 131 ]. Although no reason for the closure was provided on the website, it is likely because of persistent concerns regarding the safety of patients and data privacy involving multiple apps including those listed in the library [ 12 , 14 - 16 , 131 , 132 ]. The NHS App was introduced in January 2019 before the closure of the NHS Apps Library to serve as the gateway for accessing NHS services including ordering repeat prescriptions and booking or managing appointments [ 133 ].

Furthermore, the United Kingdom Health Security Agency, formerly known as Public Health England, issued a guidance on criteria for health app assessment in October 2017 [ 99 ]. The purpose of this guidance was to ensure that all health apps built for the UK population work well and provide clear information about their functions, benefits, and intended outcomes for patients and health care professionals. On the basis of this guidance, those intending to build an app are required to conform to certain regulations before being considered for the app assessment process. The 2 main regulations are the medical device regulation and the Care Quality Commission (CQC) registration. Apps that are considered as medical devices must register with the MHRA and have a CE mark. Apps providing health or social care that fit into 1 of 14 regulated activities are required to register with the CQC before they can be assessed [ 134 ]. CQC is an independent regulator of health and social care services in England.

Similarly, the Organisation for the Review of Care and Health Apps (ORCHA) is a UK-based organization that independently evaluates and distributes health apps. It provides services such as app review, accreditation, curation, and recommendation within the United Kingdom and across the world [ 135 ]. ORCHA also enables organizations (including the NHS) to build a decentralized web-based digital health library of consumer-friendly over-the-counter apps [ 135 - 137 ]. These apps are continuously assessed by ORCHA against the standards and regulations in clinical and professional assurance, data quality and privacy, and usability and accessibility [ 137 ].

In addition, the Digital Technology Assessment Criteria (DTAC) were introduced in beta in October 2020, and its first official version was subsequently launched in February 2021 [ 138 ]. The DTAC plays a crucial role in ensuring that digital health tools meet the necessary standards in areas such as clinical safety, data protection, technical security, interoperability, usability, and accessibility. By serving as the national baseline criteria for DHTs in the NHS and social care, it provides a valuable framework for health care organizations during procurement. It also offers guidance for developers on the expectations for their digital technologies within the NHS and social care. This is an example of how a harmonized framework can help ensure the quality and safety of DHTs, including health apps.

In addition, the National Institute for Health and Care Excellence Evidence Standards Framework is a set of evidence standards for a wide range of DHTs designed to help evaluators and decision makers in the health care system to consistently identify DHTs that are likely to offer benefits to the users and the health care system [ 139 ]. The Evidence Standards Framework was first published in March 2019 and is ideally used before DHTs (including health apps) are considered for commissioning or procurement by the NHS [ 140 ]. It is a crucial tool for ensuring that DHTs are clinically effective and offer value to the health and care system in the United Kingdom. In August 2022, the framework was updated to include AI and data-driven technologies with adaptive algorithms [ 140 ].

Furthermore, DTx apps, which are a type of medical device, are not allowed into the UK market unless they comply with the UK GDPR and meet the requirements of DTAC. In addition, they must bear the CE or UK Conformity Assessed marks [ 141 ]. This means that DTx apps must demonstrate their safety and efficacy through clinical trials and comply with the relevant regulations for data protection and quality standards as regulated by the MHRA. DTx products are also recognized as DHTs under the National Institute for Health and Care Excellence Evidence Standards Framework [ 142 ]. DTx incorporates software to treat, prevent, or manage specific diseases or conditions [ 143 , 144 ]. The fact that DTx products typically focus on a narrow clinical indication and generate evidence of clinical efficacy underscores their potential to make a substantial contribution to self-management and health care delivery in general. The increasing recognition of the role of DTx in patient care by regulators is also noteworthy, and the creation of regulatory and reimbursement pathways for approved apps further enables DTx products to continue to play an important role in impacting health care delivery [ 1 , 143 ]. This is a testament to the potential of regulated health apps to revolutionize health care and improve patient outcomes.

Among the many lessons to learn from the experience of the United Kingdom is that the regulation of health apps must evolve to keep pace with advances in DHTs and adapt to the changing needs and demands of digital health. Moreover, efforts are being made to streamline the multifaceted approaches to simplify app regulation and access in the United Kingdom [ 23 ]. Therefore, a robust and dynamic coordination mechanism, along with political will, skilled personnel, reliable funding, and a robust framework for monitoring and evaluating progress and aligning key performance indicators, is essential for countries in sub-Saharan Africa to keep pace with the advancement in the regulation of health apps. There is also a need to strengthen collaboration and ensure regulatory harmonization among national regulatory authorities and continental bodies such as the regional economic communities, AMRH, and the WHO AFRO [ 126 ].

Capacity building and monitoring and evaluation are important factors for ensuring effective regulation of health apps given the complex nature of the process. The regulation of medical products (including health apps) in sub-Saharan Africa generally includes licensing and accreditation, evaluation, inspection, quality control, information dissemination and promotion, and monitoring of adverse events [ 125 ]. Therefore, high-level skills as well as effective monitoring and evaluation will be required to ensure the success of the process. For most countries in sub-Saharan Africa, the NMRA is responsible for coordinating and overseeing the regulatory system of medical products [ 125 , 126 ]. However, in most cases, NMRAs are unable to perform the core regulatory functions expected of them [ 145 ]. More than 90% of African countries have limited or no capacity to regulate medical products, with only 7% having moderately developed capabilities [ 145 ]. The lack of effective NMRAs in Africa exposes the citizens to potential harm by allowing unsafe, low-quality, and fake medical products to circulate and be used [ 145 ].

Although it is the responsibility of governments to establish functional regulatory systems and ensure effective monitoring and evaluation of the regulatory process, the involvement of international and continental organizations to support sub-Saharan African countries improve the regulatory capacity of their national regulatory agencies would be extremely beneficial. For instance, the African Medicines Agency (AMA) was established in November 2019 as a treaty adopted by the African Union Member States to help address the concerns arising from weak regulatory systems on the continent. At present, 37 countries have signed the AMA treaty, including 26 countries that have ratified it [ 146 ]. The main objective of the AMA is to enhance the capacity of States Parties and regional economic communities to regulate medical products to improve the quality, safety, and efficacy of medical products on the continent [ 147 ]. The AMA, in collaboration with other existing capacity building initiatives or organizations, such as the WHO Global Initiative on Digital Health, ITU, AMRH, WHO AFRO, and United Nations Children’s Fund, can assist sub-Saharan African countries in aligning their regulatory requirements with available resources and support them to acquire the necessary tools and skills to build effective and sustainable regulatory systems for health apps. This can be achieved by adopting a decentralized approach to engage a network of technical experts across the African Union similar to the model of the European Medicines Agency [ 148 ].

Actors or Stakeholders

The regulation of health apps often requires working with a wide range of actors or stakeholders. However, in this review, we identified only 2 main actor groups (those who provide digital health services and those who use the ICT infrastructure of the health ministry). These are the groups that are targeted by the included regulatory standards.

From a broader perspective, 12 categories of stakeholders according to their potential role in regulating health apps for the self-management were mapped in this study. The potential contribution of these stakeholders to the regulation of health apps for self-management in sub-Saharan Africa not only depends on their roles and responsibilities but also on their interests, needs, expectations, and influence [ 41 , 149 - 151 ]. Thus, a robust stakeholder analysis is paramount as it can help define the scope of the regulatory process, prioritize the requirements, manage the expectations, and ensure the engagement and participation of stakeholders throughout the regulatory process [ 41 , 152 - 156 ]. Our stakeholder mapping, as presented in Table 2 (refer to Multimedia Appendix 4 for more details), lays the foundation for national governments to conduct a robust stakeholder analysis and to adopt an all-inclusive stakeholder engagement strategy to manage and sustain the engagement and participation of all relevant stakeholders [ 157 , 158 ].

Recommendations

Our review found that the regulation of health apps in sub-Saharan Africa is especially poor and almost nonexistent, as only Kenya has national standards that could address some of the regulatory issues related to health apps. Therefore, we recommend the following actions to help sub-Saharan African countries improve the regulation of health apps to support self-management:

• Establish a clear and consistent definition of what constitutes a health app (considering AI or machine learning) and what level of regulation is required for different types of apps.
• Develop and implement criteria and guidelines that ensure the quality, safety, and usability of health apps.
• Engage with independent app evaluators, such as ORCHA, to adopt a common framework that can guide the evaluation and accreditation of health apps and use the framework to create and maintain decentralized and transparent platforms that showcase and evaluate health apps for users and health care professionals.
• Develop and implement policies and regulations that enable sustainable funding for health apps such as integrating the use of health apps for self-management into existing health systems and financing pathways or mechanisms.
• Support and facilitate innovation and collaboration across the sub-Saharan Africa region, especially in areas including but not limited to data security and privacy, interoperability standards, usability, accessibility, funding, capacity building, and monitoring and evaluation of the regulatory process.
• Manage and sustain the engagement, involvement, and participation of all relevant stakeholders in the regulatory process by conducting a robust stakeholder analysis and adopting an all-inclusive stakeholder engagement strategy.

Strengths and Limitations of the Study

This study has several strengths, which include an extensive search of gray literature and repositories, contact with key individuals, and the use of a systematic approach. Given that regulatory standards and guidance are unavailable in scientific databases, a wide range of gray literature and repositories were searched. In addition, contact was made with key staff members to obtain relevant documents, including those at the MOHs, the WHO country offices, and the WHO AFRO. Second, to enhance the strength of the study, a policy analysis framework was adapted and used to systematically organize the key study findings, whereas a deductive descriptive qualitative content analysis approach was used to identify and analyze texts that contained relevant concepts and other related information based on the 4 predefined themes. Third, the RISA tool was used to guide the mapping of key stakeholders. This has further increased the robustness of the study findings.

The limitations of this study include the fact that our literature search was conducted in English. Although the literature search was conducted in English, it yielded documents written in French and Portuguese from the ICTworks repository. Second, regulatory standards and guidance are not readily available on scientific databases; hence, it is possible that some relevant documents might have been missed. However, efforts were made to obtain these documents by contacting key stakeholders including key contact persons at the WHO AFRO, WHO country offices, and MOHs. In addition, contacting key individuals only for the purposes of requesting documents rather than conducting direct interviews was one of the limitations of this study. Interviewing key contact persons and stakeholders to obtain additional information could have strengthened the review; however, we did not interview any key individuals or stakeholders because it was beyond the scope of this review. Nonetheless, we recommend that future studies consider incorporating interviews to explore the perspectives of key stakeholders.

Conclusions

Health apps are increasingly being used by patients to manage their health, and sub-Saharan African countries can leverage these apps to advance their progress toward achieving SDG 3 (good health and well-being) and UHC, especially given the rapid advancement of AI and big data. However, our study has established that the regulation of health apps in sub-Saharan Africa is inadequate to ensure that health apps are technically reliable and clinically safe; interoperable across systems; compliant with the principles of confidentiality of information and data privacy; culturally appropriate and relevant; and accessible to everyone regardless of gender, ethnicity, location, or income. Therefore, the region can learn from the experiences of some high-income countries such as the United Kingdom and Germany to develop and implement a robust and responsive regulatory system that supports the widespread adoption of safe, effective, and beneficial health apps for its population.

Following the publication of this review, a summary of the findings will be disseminated to the relevant organizations. In addition, the key findings will be summarized and presented at national, regional, and international conferences.

Acknowledgments

The authors would like to thank Rebecca Jones, the Library Manager and Liaison Librarian at Charing Cross Library, who advised and assisted with the search strategy for this study. This work is part of the PhD research of BAB, which is sponsored by the government of Nigeria. AM and JC were supported by the National Institute for Health and Care Research (NIHR) Applied Research Collaboration Northwest London (NIHR200180). The views expressed in this publication are those of the authors and not necessarily those of the government of Nigeria or the NIHR or the Department of Health and Social Care. In the Results and Discussion sections, Microsoft Copilot in Bing [ 159 ] was used to help summarize and modify a few texts as well as suggest some citations.

Data Availability

The search strategy for PubMed, Scopus, and the World Health Organization AIM is presented in Multimedia Appendix 1 . All data generated or analyzed during this study are included in this published article (and its supplementary information files). The documents analyzed are available directly from the relevant institutional websites, ICTworks repository [ 44 ] or upon request from the relevant government departments in each country. Additionally, documents in the list of references that are not accessible on the web can be solicited from the corresponding author on reasonable request.

Authors' Contributions

BAB and JC conceived the study. BAB designed the study with contributions from JC and NM. BAB drafted the manuscript, and JC, NM, AM, SI, KPF, BIH, and NU read and contributed to it. AM was the clinical lead, and JC acted as a guarantor for this study. The final manuscript was read and approved by all the authors.

Conflicts of Interest

None declared.

PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

Database search strategy.

Details of included documents.

Mapping of the stakeholders according to their potential role in regulating health apps for self-management.

• Aitken M, Nass D. Digital health trends 2021: innovation, evidence, regulation, and adoption. IQVIA Institute for Human Data Science. 2021. URL: https:/​/www.​iqvia.com/​-/​media/​iqvia/​pdfs/​institute-reports/​digital-health-trends-2021/​iqvia-institute-digital-health-trends-2021.​pdf?&_=1669449368070 [accessed 2022-11-26]
• Mobile app threat landscape report. RiskIQ. 2020. URL: https://www.riskiq.com/2020-mobile-threat-landscape-report-thank-you/ [accessed 2021-07-19]
• El-Sappagh S, Ali F, Hendawi A, Jang JH, Kwak KS. A mobile health monitoring-and-treatment system based on integration of the SSN sensor ontology and the HL7 FHIR standard. BMC Med Inform Decis Mak. May 10, 2019;19(1):97. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Labrique AB, Vasudevan L, Kochi E, Fabricant R, Mehl G. mHealth innovations as health system strengthening tools: 12 common applications and a visual framework. Glob Health Sci Pract. Aug 06, 2013;1(2):160-171. [ FREE Full text ] [ CrossRef ]
• Adepoju IOO, Albersen BJA, De Brouwere V, van Roosmalen J, Zweekhorst M. mHealth for clinical decision-making in sub-Saharan Africa: a scoping review. JMIR Mhealth Uhealth. Mar 23, 2017;5(3):e38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vegesna A, Tran M, Angelaccio M, Arcona S. Remote patient monitoring via non-invasive digital technologies: a systematic review. Telemed J E Health. Jan 2017;23(1):3-17. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Use of appropriate digital technologies for public health: Report by the Director-General. World Health Organization. 2016. URL: https://iris.who.int/handle/10665/274134 [accessed 2023-05-06]
• El-Osta A, Rowe C, Majeed A. Developing a shared definition of self-driven healthcare to enhance the current healthcare delivery paradigm. J R Soc Med. Nov 2022;115(11):424-428. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hussein R. A review of realizing the universal health coverage (UHC) goals by 2030: part 2- what is the role of eHealth and technology? J Med Syst. Jul 2015;39(7):72. [ CrossRef ] [ Medline ]
• Sustainable development goal 3: Ensure healthy lives and promote well-being for all at all ages. United Nations. URL: https://sdgs.un.org/goals/goal3 [accessed 2023-05-07]
• Coronavirus: apps to help the elderly. Organisation for the Review of Care and Health Apps. 2020. URL: https://orchahealth.com/coronavirus-apps-to-help-the-elderly/ [accessed 2021-07-19]
• Huckvale K, Adomaviciute S, Prieto JT, Leow MKS, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med. May 06, 2015;13:106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. Nov 22, 2012;10:144. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Morrison C, Ouyang J, Ghaghda A, Car J. The evolution of mobile apps for asthma: an updated systematic assessment of content and tools. BMC Med. Mar 23, 2015;13:58. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Prieto JT, Tilney M, Benghozi PJ, Car J. Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment. BMC Med. Sep 07, 2015;13:214. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. Apr 05, 2019;2(4):e192542. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ndlovu K, Mars M, Scott RE. Interoperability frameworks linking mHealth applications to electronic record systems. BMC Health Serv Res. May 13, 2021;21(1):459. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kenya standards and guidelines for mHealth systems. Kenya Ministry of Health. 2017. URL: https:/​/www.​health.go.ke/​wp-content/​uploads/​2020/​02/​Revised-Guidelines-For-Mhealth-Systems-May-Version.​pdf [accessed 2023-03-21]
• Standard for electronic health record system (EHRs) in Ethiopia. Ethiopia Minister of Health. 2021. URL: https:/​/registry.​betterehealth.eu/​ehealth-policies/​standard-electronic-health-record-system-ehrs-ethiopia [accessed 2023-04-21]
• National health normative standards framework for digital health interoperability in South Africa. South Africa Department of Health. 2021. URL: https://www.health.gov.za/wp-content/uploads/2022/10/HNSF_Gazette_21_October_2022.pdf [accessed 2023-05-15]
• Ferretti A, Ronchi E, Vayena E. From principles to practice: benchmarking government guidance on health apps. Lancet Digit Health. Jun 2019;1(2):e55-e57. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Diao JA, Venkatesh KP, Raza MM, Kvedar JC. Multinational landscape of health app policy: toward regulatory consensus on digital health. NPJ Digit Med. May 11, 2022;5(1):61. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Essén A, Stern AD, Haase CB, Car J, Greaves F, Paparova D, et al. Health app policy: international comparison of nine countries' approaches. NPJ Digit Med. Mar 18, 2022;5(1):31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brown SA, Garcia AA, Kouzekanani K, Hanis CL. Culturally competent diabetes self-management education for Mexican Americans: the Starr County border health initiative. Diabetes Care. Feb 2002;25(2):259-268. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chaney SC, Mechael P. Self-Care Trailblazer Group. 2020. URL: https://media.psi.org/wp-content/uploads/2020/09/31000510/Digital-Self-Care-Final.pdf [accessed 2021-05-20]
• Kanzaveli T. Healthcare: shiftingfrom “one size fits all” to “one size fits one”. Medium. 2017. URL: https:/​/tkanzaveli.​medium.com/​healthcare-shifting-from-one-size-fits-all-to-one-size-fits-one-d56136ded705 [accessed 2022-03-04]
• Myth 1 – one app will fit all!. Organisation for the Review of Care and Health Apps. URL: https://orchahealth.com/myth-1-one-app-will-fit-all/ [accessed 2022-03-04]
• Aitken M, Lyle J. Patient adoption of mHealth: use, evidence and remaining barriers to mainstream acceptance. IQVIA Institute for Human Data Science. Sep 2015. URL: https://www.iqvia.com/-/media/iqvia/pdfs/institute-reports/patient-adoption-of-mhealth.pdf [accessed 2021-05-21]
• Mechael P, Batavia H, Kaonga N. Barriers and gaps affecting mhealth in low and middle income countries: policy white paper. Center for Global Health and Economic Development Earth Institute, Columbia University. 2010. URL: http://www.globalproblems-globalsolutions-files.org/pdfs/mHealth_Barriers_White_Paper.pdf [accessed 2021-03-24]
• Bene BA, Ibeneme S, Fadahunsi KP, Harri BI, Ukor N, Mastellos N, et al. Regulatory standards and guidance for the use of health applications for self-management in Africa: scoping review protocol. BMJ Open. Feb 11, 2022;12(2):e058067. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Aitken M, Gauntlett C. Patient apps for improved healthcare: from novelty to mainstream. IMS Institute for Healthcare Informatics. 2013. URL: https://ignacioriesgo.es/wp-content/uploads/2014/03/iihi_patient_apps_report_editora_39_2_1.pdf [accessed 2024-03-10]
• National eHealth Strategy Toolkit. World Health Organization, International Telecommunication Union. 2012. URL: https://www.itu.int/pub/D-STR-E_HEALTH.05-2012 [accessed 2021-06-28]
• Global strategy on digital health 2020-2025. World Health Organization. 2021. URL: https://www.who.int/docs/default-source/documents/gs4dhdaa2a9f352b0445bafbc79ca799dce4d.pdf [accessed 2021-06-23]
• Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. Feb 2005;8(1):19-32. [ FREE Full text ] [ CrossRef ]
• Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Syst. Jul 09, 2008;6:7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci. Sep 20, 2010;5:69. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. Oct 2020;18(10):2119-2126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leitner C, Potenziani D. Mendeley reference manager. Mendeley. 2022. URL: https://www.mendeley.com/reference-management/reference-manager [accessed 2022-08-03]
• Better systematic review management. Covidence. URL: https://www.covidence.org/ [accessed 2023-02-13]
• Franco-Trigo L, Fernandez-Llimos F, Martínez-Martínez F, Benrimoj SI, Sabater-Hernández D. Stakeholder analysis in health innovation planning processes: A systematic scoping review. Health Policy. Oct 2020;124(10):1083-1099. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Walt G, Gilson L. Reforming the health sector in developing countries: the central role of policy analysis. Health Policy Plan. Dec 1994;9(4):353-370. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Digital health: a call for government leadership and cooperation between ICT and health. Broadband Commission. 2017. URL: https://broadbandcommission.org/wp-content/uploads/2021/09/WGHealth_Report2017-.pdf [accessed 2021-06-28]
• Vota W. Every African country’s national eHealth strategy or digital health policy. ICT works. 2019. URL: https://www.ictworks.org/african-national-ehealth-strategy-policy/ [accessed 2023-12-10]
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. Jul 21, 2009;6(7):e1000097. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Standards and guidelines for electronic medical record systems in Kenya. Kenya Ministry of Medical Services, Kenya Ministry of Public Health and Sanitation. 2010. URL: http://guidelines.health.go.ke:8000/media/Standards_and_Guidelines_for_EMR_Systems.pdf [accessed 2023-04-21]
• Kenya standards and guidelines for E-health systems interoperability. Kenya Ministry of Health, AfyaInfo Project. 2014. URL: https://pdf.usaid.gov/pdf_docs/PA00TB2K.pdf [accessed 2023-03-21]
• Health sector ICT standards and guidelines. Kenya Ministry of Health. 2013. URL: https://www.medbox.org/pdf/5e148832db60a2044c2d2895 [accessed 2023-03-21]
• Health information exchange standard operating procedure (SOP) and guideline. Nigeria Federal Ministry of Health. Jul 2020.
• National eHealth strategy 2018-2022. Benin Ministry of Health. 2017.
• The eHealth strategy of botswana 2020-2024. Botswana Ministry of Health. URL: https://ehealth.ub.bw/bhdc/Docs/MOH%20ehealth%20Strategy%20Book%20A4.pdf [accessed 2023-04-22]
• Health sector digital strategy 2016-2020. Burkina Faso Ministry of Health.
• National health informatics development plan of Burundi. Burundi Ministry of Public Health. 2015.
• The 2020-2024 national digital health strategic plan. Cameroon Ministry of Public Health. 2020.
• National eHealth strategy 2017-2021. Comoros Ministry of Health. 2016.
• eHealth strategic plan. Cote d’Ivoire Minister of Health and Public Hygiene. 2011.
• National development plan for health informatics. Democratic Republic of Congo Ministry of Public Health. 2014.
• Kingdom of Swaziland eHealth strategy 2016 - 2020. Kingdom of Swaziland Ministry Of Health. 2016.
• Information revolution strategic plan (2018-2025). Ethiopia Ministry of Health. 2018.
• Strategic master plan of the health information system of the Gabon. Gabon Ministry of Public Health and Population. 2017.
• National e-Health strategy. Ghana Ministry of Health. 2010.
• Kenya national e-Health strategy. Kenya Ministry of Medical Services, Kenya Ministry of Public Health & Sanitation. 2011.
• Kenya national eHealth policy 2016-2030. Kenya Ministry of Health. 2016.
• National strategy - Liberia - 2016-2021. Liberia Ministry of Health. 2016.
• Strategic plan for strengthening the health information system of Madagascar 2018–2022. Madagascar Ministry of Public Health. 2017.
• National digital health strategy 2020-2025. Malawi Ministry of Health. 2020.
• National eHealth policy in Mali. Mali Ministry of Health and Public Hygiene. 2013.
• Health 2015: seamless continuity of care. Mauritius Ministry of Health and Quality of Life. 2015.
• Strategic plan of information system for health 2009-2014. Mozambique Ministry of Health. 2009.
• National eHealth strategy 2019-2023. Niger Ministry of Public Health. 2018.
• National digital health strategy 2021-2025. Nigeria Federal Ministry of Health. 2021.
• National digital health policy. Nigeria Federal Ministry of Health. 2021.
• National digital health strategic plan 2018-2023. Rwanda Ministry of Health. 2018. URL: https://elearning.helinanet.org/mod/resource/view.php?id=890 [accessed 2023-05-09]
• Strategic plan for digital health 2018-2023. Senegal Ministry of Health and Social Action. 2018.
• National digital health strategy 2018-2023. Sierra Leone Ministry of Health and Sanitation, Sierra Leone Ministry of Information and Communication. 2018.
• The national digital health strategy 2019 – 2024. Tanzania Ministry of Health, Community Development, Gender, Elderly and Children. 2019.
• National digital health strategy for South Africa 2019 - 2024. South Africa Department of Health. 2019.
• Strategic plan for the development of eHealth in Togo 2013-2015. Togo Ministry of Health. 2012.
• Uganda national eHealth policy. Uganda Ministry of Health. 2016.
• Uganda national eHealth strategy 2017 - 2021. Uganda Ministry of Health. URL: https://health.go.ug/sites/default/files/National%20e_Health%20Strategy_0.pdf [accessed 2023-05-16]
• National eHealth strategy 2017-2021. Zambia Ministry of Health. 2017.
• Zimbabwe’s E-Health strategy 2012-2017. Ministry of Health and Child Welfare. 2012.
• National eHealth strategy 2021-2025. Namibia Ministry of Health & Social Services. 2021. URL: https://www.scribd.com/document/639371316/eHealth-Strategy-Namibia-2021# [accessed 2023-05-13]
• Health sector ICT policy and strategy. Ghana Ministry of Health. 2005. URL: https://www.moh.gov.gh/wp-content/uploads/2016/02/Health-Sector-ICT-Policy-and-Strategy.pdf [accessed 2023-05-08]
• Adebola OJ. Beyond national digital health strategy: final report of end term evaluation for the National Health ICT Strategic Framework 2015-2020. Nigeria Federal Ministry of Health. May 2021.
• National Health ICT Strategic Framework 2015 - 2020. Nigeria Federal Ministry of Health. 2016. URL: https://www.health.gov.ng/doc/HealthICTStrategicFramework.pdf [accessed 2023-05-16]
• Digital health blueprint. Ethiopia Ministry of Health. 2021. URL: http:/​/repository.​iifphc.org/​bitstream/​handle/​123456789/​1658/​Ethiopian-Digital-Health-Blueprint.​pdf?sequence=1&isAllowed=y [accessed 2023-05-16]
• Kenya health information systems interoperability framework. Kenya Ministry of Health. 2020. URL: https:/​/www.​data4sdgs.org/​sites/​default/​files/​services_files/​Kenya%20Health%20Information%20Systems%20Interoperability%20Framework.​pdf [accessed 2023-05-16]
• National community health digitization strategy 2020-2025. Kenya Ministry of Health, Division of Community Health Services. 2021. URL: https:/​/www.​eahealth.org/​sites/​www.eahealth.org/​files/​content/​attachments/​2021-08-02/​eCHIS-Strategy-2020-2025.​pdf [accessed 2023-05-16]
• Leitner C, Potenziani D. Health information systems interoperability in Liberia. IntraHealth International. 2016. URL: https://elearning.helinanet.org/mod/resource/view.php?id=938 [accessed 2023-05-16]
• Narrative for 2022 national digital health annual operational plan (AOP). Nigeria Federal Ministry of Health. 2022.
• Tanzania digital health investment road map 2017-2023. Tanzania Ministry of Health, Community Development, Gender, Elderly and Children, President’s Office Regional Administration and Local Government. 2017.
• Yu H. Regulation of digital health technologies in the European Union: intended versus actual use. In: Cohen GI, Minssen T, Price II NW, Robertson C, Shachar C, editors. The Future of Medical Device Regulation: Innovation and Protection. Cambridge. Cambridge University Press; Mar 31, 2022;103-114.
• Policy for device software functions and mobile medical applications: guidance for industry and Food and Drug Administration staff. U.S. Food and Drug Administration. 2022. URL: https://www.fda.gov/media/80958/download [accessed 2023-10-10]
• Gordon WJ, Landman A, Zhang H, Bates DW. Beyond validation: getting health apps into clinical practice. NPJ Digit Med. 2020;3:14. [ FREE Full text ] [ CrossRef ] [ Medline ]
• FDA clears mobile medical app to help those with opioid use disorder stay in recovery programs. U.S. Food and Drug Administration. 2018. URL: https:/​/www.​fda.gov/​news-events/​press-announcements/​fda-clears-mobile-medical-app-help-those-opioid-use-disorder-stay-recovery-programs [accessed 2021-01-27]
• Digital maturity model: achieving digital maturity to drive growth. Deloitte. 2018. URL: https:/​/www2.​deloitte.com/​content/​dam/​Deloitte/​global/​Documents/​Technology-Media-Telecommunications/​deloitte-digital-maturity-model.​pdf [accessed 2021-10-20]
• May E. How digital health apps are empowering patients. Deloitte. 2021. URL: https:/​/www2.​deloitte.com/​us/​en/​blog/​health-care-blog/​2021/​how-digital-health-apps-are-empowering-patients.​html [accessed 2023-10-06]
• Guidance: criteria for health app assessment. Public Health England. 2017. URL: https:/​/www.​gov.uk/​government/​publications/​health-app-assessment-criteria/​criteria-for-health-app-assessment [accessed 2023-10-16]
• Clinical risk management standards. National Health Service Digital. 2020. URL: https://digital.nhs.uk/services/clinical-safety/clinical-risk-management-standards [accessed 2023-10-28]
• Report a problem with a medicine or medical device. Gov.uk. URL: https://www.gov.uk/report-problem-medicine-medical-device [accessed 2023-11-07]
• Digital technology assessment criteria for health and social care (DTAC) - Version 1.0. National Health Service X. 2021. URL: https:/​/view.​officeapps.live.com/​op/​view.​aspx?src=https%3A%2F%2Ftransform.​england.​nhs.​uk%2Fmedia%2Fdocuments%2FDTAC_version_1.​0_FINAL_updated_16.​04.​odt&wdOrigin=BROWSELINK [accessed 2023-11-07]
• Data protection impact assessment: NHS login - formerly Citizen Identity. National Health Service Digital. 2022. URL: https://digital.nhs.uk/services/nhs-login/data-protection-impact-assessment [accessed 2023-11-07]
• Risks and data protection impact assessments (DPIAs). Information Commissioner’s Office. URL: https:/​/ico.​org.uk/​for-organisations/​uk-gdpr-guidance-and-resources/​accountability-and-governance/​accountability-framework/​risks-and-data-protection-impact-assessments-dpias/​ [accessed 2023-11-07]
• A guide to good practice for digital and data-driven health technologies. Department of Health and Social Care. 2021. URL: https:/​/www.​gov.uk/​government/​publications/​code-of-conduct-for-data-driven-health-and-care-technology/​initial-code-of-conduct-for-data-driven-health-and-care-technology [accessed 2023-10-30]
• Interoperability in healthcare. Healthcare Information and Management Systems Society (HIMSS). 2023. URL: https://www.himss.org/resources/interoperability-healthcare [accessed 2023-10-17]
• DAPB4020: UK core Fast Healthcare Interoperability Resources (FHIR) release 4 (R4) governance. National Health Service Digital. 2022. URL: https:/​/digital.​nhs.uk/​data-and-information/​information-standards/​information-standards-and-data-collections-including-extractions/​publications-and-notifications/​standards-and-collections/​dapb4020-uk-core-fhir-r4-governance [accessed 2023-10-17]
• Fast Healthcare Interoperability Resources (FHIR). National Health Service Digital. 2022. URL: https://digital.nhs.uk/services/fhir-apis [accessed 2023-10-17]
• FHIR Interoperability Basics: 4 things to know. Health IT Analytics. 2022. URL: https://healthitanalytics.com/news/4-basics-to-know-about-the-role-of-fhir-in-interoperability [accessed 2023-11-07]
• Giordanengo A, Bradway M, Pedersen R, Grøttland A, Hartvigsen G, Årsand E. Integrating data from apps, wearables and personal electronic health record (pEHR) systems with clinicians’ electronic health records (EHR) systems. Int J Integr Care. Nov 09, 2016;16(5):16. [ FREE Full text ] [ CrossRef ]
• A plan for digital health and social care. Department of Health & Social Care. 2022. URL: https:/​/www.​gov.uk/​government/​publications/​a-plan-for-digital-health-and-social-care/​a-plan-for-digital-health-and-social-care [accessed 2022-12-01]
• Ryu B, Kim N, Heo E, Yoo S, Lee K, Hwang H, et al. Impact of an electronic health record-integrated personal health record on patient participation in health care: development and randomized controlled trial of MyHealthKeeper. J Med Internet Res. Dec 07, 2017;19(12):e401. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Winter A, Takabayashi K, Jahn F, Kimura E, Engelbrecht R, Haux R, et al. Quality requirements for electronic health record systems*. A Japanese-German information management perspective. Methods Inf Med. Aug 07, 2017;56(7):e92-e104. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wachter RM. Making IT work: harnessing the power of health information technology to improve care in England. National Advisory Group on Health Information Technology. 2016. URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​550866/​Wachter_Review_Accessible.​pdf [accessed 2021-07-22]
• Framework on integrated people-centred health services (IPCHS). World Health Organisation. 2023. URL: https:/​/www.​who.int/​teams/​integrated-health-services/​clinical-services-and-systems/​service-organizations-and-integration [accessed 2023-06-05]
• Ibeneme S, Karamagi H, Muneene D, Goswami K, Chisaka N, Okeibunor J. Strengthening health systems using innovative digital health technologies in Africa. Front Digit Health. 2022;4:854339. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ibeneme S, Ukor N, Ongom M, Dasa T, Muneene D, Okeibunor J. Strengthening capacities among digital health leaders for the development and implementation of national digital health programs in Nigeria. BMC Proc. 2020;14(Suppl 10):9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Delivering safe digital health. Organisation for the Review of Care and Health Apps. URL: https://orchahealth.com/ [accessed 2023-10-22]
• Mamuye AL, Yilma TM, Abdulwahab A, Broomhead S, Zondo P, Kyeng M, et al. Health information exchange policy and standards for digital health systems in Africa: a systematic review. PLOS Digit Health. Oct 2022;1(10):e0000118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Digital health platform handbook: Building a digital information infrastructure (infostructure) for health. World Health Organization, International Telecommunication Union. 2022. URL: https://www.itu.int/dms_pub/itu-d/opb/str/D-STR-E_HEALTH.10-2020-PDF-E.pdf [accessed 2021-05-22]
• Framework for involving patients in patient safety 2021. National Health Service England. 2021. URL: https://www.england.nhs.uk/patient-safety/framework-for-involving-patients-in-patient-safety/ [accessed 2023-03-23]
• Olesch A. Towards harmonised EU landscape for digital health: summary of the roundtable discussions in selected EIT Health InnoStars countries. EIT Health InnoStars. Jan 2023. URL: https://eithealth.eu/wp-content/uploads/2023/02/EIT_Health_DiGA_report_Jan2023.pdf [accessed 2023-10-10]
• Grieb J, Tschammler D, Färber C, Woitz S. Digital health laws and regulations germany. Global Legal Group. 2023. URL: https://iclg.com/practice-areas/digital-health-laws-and-regulations/germany [accessed 2023-11-03]
• Human regulatory: medical devices. European Medicines Agency. URL: https://www.ema.europa.eu/en/human-regulatory/overview/medical-devices [accessed 2023-10-12]
• Strengthening the capacity for regulation of medical products in the African region. World Health Organization Regional Office for Africa. 2013. URL: https://iris.who.int/bitstream/handle/10665/94308/AFR_RC63_7.pdf?sequence=1 [accessed 2023-10-17]
• Ncube BM, Dube A, Ward K. Establishment of the African Medicines Agency: progress, challenges and regulatory readiness. J Pharm Policy Pract. Mar 08, 2021;14(1):29. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Post market surveillance system. European Union Medical Device Regulation. 2023. URL: https://eumdr.com/post-market-surveillance-system/ [accessed 2023-10-31]
• Dayal R. Effective post-market surveillance for medical devices: An essential part of medical devices regulation (MDR). Capgemini. 2020. URL: https:/​/www.​capgemini.com/​insights/​expert-perspectives/​effective-post-market-surveillance-for-medical-devices-an-essential-part-of-mdr/​ [accessed 2023-10-31]
• NHS app library reaches 70 apps in honour of the NHS birthday. Northampton General Hospital NHS Trust. 2018. URL: https:/​/www.​northamptongeneral.nhs.uk/​News/​News-Archive/​2018/​NHS-App-Library-reaches-70-apps-in-honour-of-the-NHS-birthday.​aspx [accessed 2023-09-21]
• Developers invited to add to NHS apps library. National Health Service Digital. 2018. URL: https://digital.nhs.uk/news/2018/developers-invited-to-add-to-nhs-apps-library [accessed 2023-09-22]
• The NHS apps library has closed. National Health Service Digital. 2021. URL: https:/​/digital.​nhs.uk/​services/​nhs-apps-library#:~:text=The%20NHS%20Apps%20Library%20was%20decommissioned%20in%20December%202021.​&text=Further%20information%20can%20be%20found%20on%20the%20NHS.​UK%20website [accessed 2023-09-22]
• Larsen ME, Huckvale K, Nicholas J, Torous J, Birrell L, Li E, et al. Using science to sell apps: evaluation of mental health app store quality claims. NPJ Digit Med. 2019;2:18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• About the NHS app. National Health Service. Dec 4, 2023. URL: https://www.nhs.uk/nhs-app/about-the-nhs-app/ [accessed 2023-09-22]
• Scope of registration: regulated activities. Care Quality Commission. 2022. URL: https://www.cqc.org.uk/guidance-providers/scope-registration-regulated-activities [accessed 2023-11-05]
• Distributing great apps into health and care services across the world. Organisation for the Review of Care and Health Apps. 2020. URL: https://orchahealth.com/wp-content/uploads/2020/12/Health-and-Care-1.pdf [accessed 2023-10-09]
• Our founder, our story and our values: we exist to make digital health healthy. Organisation for the Review of Care and Health Apps. URL: https://orchahealth.com/about-us/ [accessed 2023-10-09]
• Health app library: empower your community with safe access to health apps and digital health products. Organisation for the Review of Care and Health Apps. URL: https:/​/orchahealth.​com/​our-products/​health-app-library/​#:~:text=A%20Health%20App%20Library%20is,on%20the%20Health%20App%20Library [accessed 2023-10-09]
• Digital technology assessment criteria (DTAC). National Health Service X. URL: https://www.nhsx.nhs.uk/key-tools-and-info/digital-technology-assessment-criteria-dtac/ [accessed 2023-10-09]
• Evidence standards framework (ESF) for digital health technologies. National Institute for Health and Care Excellence. 2023. URL: https:/​/www.​nice.org.uk/​about/​what-we-do/​our-programmes/​evidence-standards-framework-for-digital-health-technologies [accessed 2023-10-08]
• Tsang L, Kerr-Peterson H. UK NICE updates its evidence standards framework for data-driven digital health technologies. Ropes & Gray. 2022. URL: https:/​/www.​ropesgray.com/​en/​insights/​alerts/​2022/​10/​uk-nice-updates-its-evidence-standards-framework-for-data-driven-digital-health-technologies [accessed 2023-10-09]
• Guidance: medical device stand-alone software including apps (including IVDMDs). Medicines and healthcare products regulatory agency. 2023. URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​1168485/​Medical_device_stand-alone_software_including_apps__including_IVDMDs_.​pdf [accessed 2023-10-09]
• Digital therapeutics in the United Kingdom. Digital Therapeutics Alliance. 2021. URL: https://dtxalliance.org/wp-content/uploads/2021/06/DTA_DTx-Overview_UK.pdf [accessed 2023-10-09]
• Transforming global healthcare by advancing digital therapeutics. Digital Therapeutics Alliance. 2023. URL: https://dtxalliance.org/ [accessed 2023-10-10]
• International Organization for Standardization (ISO) digital therapeutic definition. Digital Therapeutic Alliance. Jun 2023. URL: https://dtxalliance.org/wp-content/uploads/2023/06/DTA_FS_ISO-Definition.pdf [accessed 2023-10-09]
• Ndomondo-Sigonda M, Miot J, Naidoo S, Dodoo A, Kaale E. Medicines regulation in Africa: current state and opportunities. Pharmaceut Med. 2017;31(6):383-397. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chinele J. East Africa shows solid support for African Medicines Agency treaty. Health Policy Watch. Aug 16, 2023. URL: https://healthpolicy-watch.news/east-africa-shows-solid-support-for-african-medicines-agency-treaty/ [accessed 2023-10-09]
• Treaty for the establishment of the African Medicines Agency 2019. African Union. 2019. URL: https://au.int/sites/default/files/treaties/36892-treaty-0069_-_ama_treaty_e.pdf [accessed 2023-10-17]
• European Medicines Agency: about us. European Medicines Agency. Mar 1, 2023. URL: https://www.ema.europa.eu/en/documents/other/about-us-european-medicines-agency-ema_en.pdf [accessed 2023-10-18]
• Bryson JM. What to do when stakeholders matter. Public Adm Rev. Mar 2004;6(1):21-53. [ FREE Full text ] [ CrossRef ]
• Iyawa G, Herselman M, Botha A. Potential stakeholders and perceived benefits of a digital health innovation ecosystem for the Namibian context. Procedia Computer Science. 2017;121:431-438. [ CrossRef ]
• Ferretti V. From stakeholders analysis to cognitive mapping and multi-attribute value theory: an integrated approach for policy support. European Journal of Operational Research. Sep 2016;253(2):524-541. [ CrossRef ]
• Brugha R, Varvasovszky Z. Stakeholder analysis: a review. Health Policy Plan. Sep 2000;15(3):239-246. [ CrossRef ] [ Medline ]
• Schmeer K. Guidelines for conducting a stakeholder analysis 1999. Partnerships for Health Reform, Abt Associates. 1999. URL: https://www.ktecop.ca/wordpress/wp-content/uploads/guidelines-stakeholder-analysis-PHR-1999.pdf [accessed 2023-10-17]
• Gilmour J, Beilin R. Stakeholder mapping for effective risk assessment and communication. Australian Centre of Excellence for Risk Analysis, University of Melbourne. Apr 2007. URL: https://cebra.unimelb.edu.au/__data/assets/pdf_file/0006/2220990/gilmour0609.pdf [accessed 2023-10-17]
• Quality, service improvement and redesign tools: stakeholder analysis. National Health Service England, National Health Service Improvement. 2022. URL: https://www.england.nhs.uk/wp-content/uploads/2022/02/qsir-stakeholder-analysis.pdf [accessed 2023-10-20]
• Craven MP, Lang AR, Martin JL. Developing mHealth apps with researchers: multi-stakeholder design considerations. Springer; 2014. Presented at: Third International Conference, DUXU 2014, held as a part of HCI International; June 22-27, 2014;15-24; Heraklion, Greece. URL: https://doi.org/10.1007/978-3-319-07635-5_2 [ CrossRef ]
• How to encourage stakeholder participation. SustaiNet Software International. URL: https://sustainet.com/how-to-encourage-stakeholder-participation/ [accessed 2023-10-20]
• Stakeholder engagement. Organisation for Economic Cooperation and Development. URL: https:/​/www.​oecd.org/​governance/​better-international-rulemaking/​compendium/​keyprinciples/​stakeholderengagement.​htm [accessed 2023-10-20]
• Microsoft Copilot in Bing. Microsoft. URL: https://www.bing.com/chat?form=NTPCHB [accessed 2023-03-15]

Abbreviations

Edited by A Mavragani; submitted 19.05.23; peer-reviewed by N O'Brien, A Essén; comments to author 07.09.23; revised version received 08.12.23; accepted 23.02.24; published 11.04.24.

©Benard Ayaka Bene, Sunny Ibeneme, Kayode Philip Fadahunsi, Bala Isa Harri, Nkiruka Ukor, Nikolaos Mastellos, Azeem Majeed, Josip Car. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Microsoft to launch AI hub in London

• Medium Text

WHY IT'S IMPORTANT

By the numbers.

The Technology Roundup newsletter brings the latest news and trends straight to your inbox. Sign up here.

Reporting by Martin Coulter Editing by Tomasz Janowski

Our Standards: The Thomson Reuters Trust Principles. New Tab , opens new tab

Technology Chevron

Apple plans Mac line overhaul with AI-focused M4 chips, Bloomberg News reports

Apple is nearing the production of M4 computer processors which would have AI processing capabilities and plans to update every Mac model with it, Bloomberg News reported on Thursday, citing people with knowledge of the matter.

• Work & Careers
• Life & Arts

Become an FT subscriber

Try unlimited access Only $1 for 4 weeks

Then $75 per month. Complete digital access to quality FT journalism on any device. Cancel anytime during your trial.

• Global news & analysis
• Expert opinion
• Special features
• FirstFT newsletter
• Videos & Podcasts
• Android & iOS app
• FT Edit app
• 10 gift articles per month

Explore more offers.

Standard digital.

• FT Digital Edition

Premium Digital

Print + premium digital, digital standard + weekend, digital premium + weekend.

Today's FT newspaper for easy reading on any device. This does not include ft.com or FT App access.

• 10 additional gift articles per month
• Global news & analysis
• Exclusive FT analysis
• Videos & Podcasts
• FT App on Android & iOS
• Everything in Standard Digital
• Premium newsletters
• Weekday Print Edition
• FT Weekend newspaper delivered Saturday plus standard digital access
• FT Weekend Print edition
• FT Weekend Digital edition
• FT Weekend newspaper delivered Saturday plus complete digital access
• Everything in Preimum Digital

Essential digital access to quality FT journalism on any device. Pay a year upfront and save 20%.

• Everything in Print
• Everything in Premium Digital

Complete digital access to quality FT journalism with expert analysis from industry leaders. Pay a year upfront and save 20%.

Terms & Conditions apply

Explore our full range of subscriptions.

Why the ft.

See why over a million readers pay to read the Financial Times.

International Edition

Watch CBS News

Why is looking at a solar eclipse dangerous without special glasses? Eye doctors explain.

By Sara Moniuszko

Edited By Allison Elyse Gualtieri

Updated on: April 8, 2024 / 8:54 AM EDT / CBS News

The solar eclipse will be visible for millions of Americans on April 8, 2024, making many excited to see it — but how you watch it matters, since it can be dangerous for your eyes. 

A  solar eclipse occurs when the moon passes between the sun and Earth, blocking the sun's light . When the moon blocks some of the sun, it's a partial solar eclipse, but when moon lines up with the sun, blocking all of its light, a total solar eclipse occurs,  NASA explains . Either way, you need eye protection when viewing.

"The solar eclipse will be beautiful, so I hope that everyone experiences it — but they need to experience it in the right way," said Dr. Jason P. Brinton, an ophthalmologist and medical director at Brinton Vision in St. Louis.

Here's what to know to stay safe.

Why is looking at a solar eclipse dangerous?

Looking at the sun — even when it's partially covered like during an eclipse — can cause eye damage.

There is no safe dose of solar ultraviolet rays or infrared radiation, said  Dr. Yehia Hashad , an ophthalmologist, retinal specialist and the chief medical officer at eye health company Bausch + Lomb.

"A very small dose could cause harm to some people," he said. "That's why we say the partial eclipse could also be damaging. And that's why we protect our eyes with the partial as well as with the full sun."

Some say that during a total eclipse, it's safe to view the brief period time when the moon completely blocks the sun without eye protection. But experts warn against it. 

"Totality of the eclipse lasts only about 1 to 3 minutes based on geographic location, and bright sunlight suddenly can appear as the moon continues to move," notes an eclipse viewing guide published in JAMA , adding, "even a few seconds of viewing the sun during an eclipse" can temporarily or permanently damage your vision. 

Do I need special glasses for eclipse viewing?

Yes.  Eclipse glasses are needed to protect your eyes if you want to look at the eclipse.

Regular sunglasses aren't protective enough for eclipse viewing — even if you stack more than one. 

"There's no amount of sunglasses that people can put on that will make up for the filtering that the ISO standard filters and the eclipse glasses provide," Brinton said.

You also shouldn't look at the eclipse through a camera lens, phone, binoculars or telescope, according to NASA, even while wearing eclipse glasses. The solar rays can burn through the lens and cause serious eye injury.

Eclipse glasses must comply with the  ISO 12312-2 international safety standard , according to NASA, and should have an "ISO" label printed on them to show they comply. The American Astronomical Society  has a list  of approved solar viewers.

Can't find these, or they're sold out near you? You can also  make homemade viewers ,   which allow you to observe the eclipse indirectly — just don't accidentally look at the sun while using one.

How to keep kids safe during the solar eclipse

Since this eclipse is expected to occur around the time of dismissal for many schools across the country, it may be tempting for students to view it without the proper safety precautions while getting to and from their buses. That's why some school districts are  canceling classes early so kids can enjoy the event safely with their families.

Dr. Avnish Deobhakta, vitreoretinal surgeon at New York Eye and Ear Infirmary at Mount Sinai, said parents should also be careful because it can be difficult for children to listen or keep solar eclipse glasses on. 

"You want to actually, in my opinion, kind of avoid them even looking at the eclipse, if possible," he said. "Never look directly at the sun, always wear the right eclipse sunglasses if you are going to look at the sun and make sure that those are coming from a reliable source."

Brinton recommends everyone starts their eclipse "viewing" early, by looking at professional photos and videos of an eclipse online or visiting a local planetarium. 

That way, you "have an idea of what to expect," he said. 

He also recommends the foundation  Prevent Blindness , which has resources for families about eclipse safety.

What happens if you look at a solar eclipse without eclipse glasses?

While your eyes likely won't hurt in the moment if you look at the eclipse without protection, due to lowered brightness and where damage occurs in the eye, beware: The rays can still cause damage .

The harm may not be apparent immediately. Sometimes trouble starts to appear one to a few days following the event. It could affect just one or both eyes.

And while some will regain normal visual function, sometimes the damage is permanent. 

"Often there will be some recovery of the vision in the first few months after it, but sometimes there is no recovery and sometimes there's a degree to which it is permanent," Brinton said. 

How long do you have to look at the eclipse to damage your eyes?

Any amount of time looking at the eclipse without protection is too long, experts say. 

"If someone briefly looks at the eclipse, if it's extremely brief, in some cases there won't be damage. But damage can happen even within a fraction of a second in some cases," Brinton said. He said he's had patients who have suffered from solar retinopathy, the official name for the condition.

Deobhakta treated a patient who watched the 2017 solar eclipse for 20 seconds without proper eye protection. She now has permanent damage in the shape of a crescent that interferes with her vision. 

"The crescent that is burned into the retina, the patient sees as black in her visual field," he said. "The visual deficit that she has will never go away."

How to know if you've damaged your eyes from looking at the eclipse

Signs and symptoms of eye damage following an eclipse viewing include headaches, blurred vision, dark spots, changes to how you see color, lines and shapes. 

Unfortunately, there isn't a treatment for solar retinopathy.

"Seeing an eye care professional to solidify the diagnosis and for education I think is reasonable," Brinton said, but added, "right now there is nothing that we do for this. Just wait and give it time and the body does tend to heal up a measure of it."

Sara Moniuszko is a health and lifestyle reporter at CBSNews.com. Previously, she wrote for USA Today, where she was selected to help launch the newspaper's wellness vertical. She now covers breaking and trending news for CBS News' HealthWatch.

More from CBS News

Couple gets engaged on flight to see total solar eclipse

How to find the best tax relief company

Arizona's abortion ban likely to cause people to travel to states it's still legal

Inflation's rising. Here's how debt relief can help.

• International edition
• Australia edition
• Europe edition

Google considering charge for internet searches with AI, reports say

Cost of artificial intelligence service could mean leaders in sector turning to subscription models

Google is reportedly drawing up plans to charge for AI-enhanced search features, in what would be the biggest shake-up to the company’s revenue model in its history.

The radical shift is a natural consequence of the vast expense required to provide the service, experts say, and would leave every leading player in the sector offering some variety of subscription model to cover its costs.

Google’s proposals, first reported by the Financial Times , would entail the company exclusively offering its new search feature to users of its premium subscription services, which customers already have to sign up to if they want to use artificial intelligence assistants in other Google tools such as Gmail and its office suite.

With that search experience, being trialled in beta for selected users, Google’s generative AI is used to respond to queries directly with a single answer, in a similar style to the conversational approach of ChatGPT and competitors.

“AI search is more expensive to compute than Google’s traditional search processes. So in charging for AI search Google will be seeking to at least recoup these costs,” said Heather Dawe, chief data scientist at the digital transformation consultancy UST.

Much of the focus within AI is on the huge expense of the computing power used to train cutting-edge generative models. In the last year Amazon ran a single training run that cost $65m (£51m), according to James Hamilton , an engineer , who expects, in the near future, the company to break the $1bn mark.

Last week, OpenAI and Microsoft announced plans to build a $100bn datacentre for AI training, while in January Mark Zuckerberg said his goal was to spend at least $9bn just on Nvidia GPUs alone.

But the cost of training AI is just a tenth of the total cost of the sector, according to the analyst Brent Thill at the investment firm Jefferies. Thill wrote in a briefing note: “The majority of AI compute spend today is directed to the running, not training, of models, and 90%+ of AI compute spend today is being directed towards inferencing [the process by which an AI model is queried], as inferencing spend has been growing much faster than training as more models and tools get put into production.”

He added: “Some have priced new Gen AI features at a monthly rate, betting that higher charges will cover usage expenses, while others have priced on a per-usage basis to protect themselves on the cost-side. Some have also incorporated into existing plans, hoping to drive [user] growth.”

Competitors in AI search offer similar subscription plans. Perplexity , an AI-powered search engine, runs no adverts but offers a $20 monthly “pro” tier that provides access to more powerful AI models and unlimited use.

Others, though, continue to offer their products at a loss. The AI features in Microsoft’s Bing are free to use but tied to the company’s Edge browser. The browsing and search startup Arc offers its products free to users and says it intends to raise revenue in future by charging companies for business features.

• Artificial intelligence (AI)
• Search engines

Most viewed