research and development healthcare

Psychiatry's new frontiers

This new issue of Stanford Medicine magazine reports on emerging research and innovative treatments to improve mental health.

Follow Stanford Medicine

Health Care

Healing through compassion and world-class science

Fueling discovery and innovation to advance human health

Empowering tomorrow's leading physicians and scientists

School Leadership

Academic Affairs

Continuing Medical Education

Faculty Development and Diversity

Faculty Profiles

MD Admissions

PhD Program

Dual-Degree Programs

Masters Programs

Community Engagement

Clinical Trials

High School & Undergraduate Programs

News, Events and Resources

Stanford Medicine leaders' message regarding Supreme Court ruling on race-conscious university admissions

About Stanford Medicine

A leader in the biomedical revolution, Stanford Medicine has a long tradition of leadership in pioneering research, creative teaching protocols and effective clinical therapies.

Night owl behavior could hurt mental health, sleep study finds

In a new, large-scale study of sleep behavior, Stanford Medicine scientists found that night owls don’t really thrive late at night.

Speech impairment in Parkinson’s

 New research by Stanford Medicine scientists uncovers the brain connections that could be essential to preserving speech.

In the age of fentanyl, factual drug education can save teen lives

Toolkits designed by Stanford Medicine researchers are helping teens think critically about the choices they make around substance use.

Could the avian flu be our next pandemic threat?

What does it mean that H5N1 bird flu, also known as highly pathogenic avian influenza A, is spreading among dairy cows?

Symposium tackles AI’s role in medicine

Trust, human-centered AI and collaboration the focus of inaugural RAISE Health symposium.

• Become a Member

An Introduction to U.S. Federal Funding for Healthcare Innovation

September 8, 2021 – By Amrika Ramjewan, Principal Strategist – Mayo Clinic Innovation Exchange

Each year, U.S. government agencies with extramural research and development (R&D) budgets invest in federal funding programs designed to stimulate technological innovation and foster entrepreneurial activity. Coordinated by the U.S. Small Business Administration (SBA), these federal funding programs are accessible via a competitive award process through major research agencies such as the National Institutes of Health (NIH), Department of Defense (DoD), and National Science Foundation (NSF).

Entrepreneurs seeking to advance research and bring their innovations to market have the opportunity to compete for these funds and access a wealth of support from these agencies. Through 2019, over 179,000 awards have been granted totaling over $54.3 billion in investment by the federal government.

Jon Zurn, director of the Strategic Funding Office for Research at Mayo Clinic , spoke with the Exchange’s members about the variety of federal funding options available, how the non-dilutive granting process works, and the resources available to entrepreneurs.

Q: Can you share an overview of the types of federal funding programs available to entrepreneurs?

JZ: The U.S. Small Business Administration (SBA) coordinates non-dilutive, research innovation funding programs to assist entrepreneurs and small businesses with planning and conducting R&D activities and advancing their products and technologies through validation and commercialization. Two types of grants are available including the Small Business Innovation Research (SBIR) award and the Small Business Technology Transfer (STTR) award . These awards focus on R&D, stimulating technological innovation, and increasing private-sector commercialization of innovation derived from federal R&D funding.

SBIR funds are offered by 11 federal agencies and divisions within these agencies including the National Institutes of Health (NIH), Department of Defense (DoD), National Science Foundation (NSF) and the Environmental Protection Agency (EPA), among others. The funds are intended to assist small businesses with conducting principal investigator-led R&D on their own or with subcontractors, with the expectation that a majority of the work will be completed by the small business.

STTR funds are offered by five federal agencies, including the Department of Health and Human Services (HHS, and its constituents NIH, FDA, CDC, and ACL). The funds are intended to facilitate collaboration and foster technology transfer between small businesses and non-profit research institutions. The government recognizes that small firms often don’t have the research resources and infrastructure to complete early-stage R&D, and that there is tremendous value in partnering with large, research-intensive institutions to bring innovations to market.

Both awards are non-dilutive sources of funding, meaning that the government takes no interest or equity stake in your business. Recipients of these funds are expected to fulfill the reporting requirements laid out by the awarding federal agency, and all intellectual property is owned by the small business (except in special circumstances).

Q: What can the funds from these programs be used for?

JZ: SBIR and STTR grants are intended for performing R&D. Purchasing equipment, commercializing a technology that has already been developed, or pursuing a low-risk idea that requires capital will typically not be funded by these programs. Before applying, it’s best to consult with an agency’s program officer to make sure your idea meets the R&D criteria.

Award solicitations for these grants, called an Omnibus or Parent Announcement, are published by the NIH three times each year — in January, April, and September. Each participating NIH institute and center (I/C) has its own research priorities. It is important to understand how these priorities align with your projects. Additionally, targeted solicitations for specific needs may also be published, but these are not released on a regular cycle. Other federal agencies tend to post opportunities throughout the year.

Funding for each award is focused on distinct phases. In Phase I, the objective is to establish technical merit, feasibility and commercial potential prior to seeking Phase II funding. Phase I SBIR/STTR awards normally do not exceed $150,000 in total over six months (for SBIR), or over one year (for STTR) — with some exceptions.

In Phase II, the funding is based on the results achieved in Phase I, with the possibility of funding through Phase IIB. Phase II awards normally provide up to $1,000,000 in total over two years. However, exemptions may even be granted for approved research areas that may extend the award ceiling up to $1.7 million for Phase II.

Phase III research on the path of commercialization is not funded by SBIR/STTR. However additional, late stage development federal dollars are available through programs such as the NIH’s Commercialization Readiness Pilot (CRP).

Q: Who is eligible for these funds?

JZ: Small, for-profit business organizations that are U.S. concerns, and operating primarily in the United States with a U.S.-based location are eligible to apply for SBIR and STTR dollars. The small business, including its affiliates, must have no more than 500 employees, and must be more than 50% directly owned and operated by one or more individuals who are citizens or legal permanent residents of the United States. Small businesses that are subsidiaries of larger companies are not eligible.

However, if a small business is majority-owned by multiple venture capital operating companies (VCOCs), hedge funds, or private equity firms that each meet small business size criteria, it is eligible to apply for an NIH SBIR funding opportunity. These grants are also not designated for large institutions, universities, or non-profit organizations.

Foreign (non-U.S.-based) firms may access SBIR and STTR dollars through two avenues — either as a subcontractor to a U.S.-based firm, or by having a U.S. location where the work for which the funds being sought will be completed. That is, all grant dollars must be spent in the U.S. For example, if a foreign firm owned by a U.S. legal permanent resident were to receive a three-year grant for $100,000, all of the funds must be spent in the U.S. to complete the research.

For SBIR grants, subcontracting is limited to 33% of the total effort in Phase I of the project, and 50% of the total effort in Phase II. Also, the principal investigator (PI) leading the research must be employed by the small business seeking the funds. This means that the PI will be unable to work elsewhere during the project period, as more than 50% of their time must be spent in service to the small business.

For STTR grants, 40% of the work must be completed by the small business, and 30% by the collaborating research institution (RI). The remaining 30% may be completed by the small business, or outsourced to either the RI or another subcontractor. The PI may be primarily employed by either the small business or the RI. Co-investigators may be affiliated with either the small business, or the RI, or they may serve as consultants — however, this is dependent on any restrictions that may be set by the funder.

Any organization located in the U.S. that is a university, non-profit institution, or contractor-operated federally funded research and development center (FFRDC) is eligible to collaborate with small firms on STTR projects.

Q: What resources are available to entrepreneurs and small businesses interested in seeking funding through these programs?

JZ: There are a tremendous number of online resources , tutorials , as well as local SBA affiliates in every state. Many agencies provide applicant assistance programs for businesses interested in applying for SBIR and STTR grants, and technical support staff provide good, free advice — remember, they are there to help.

The National Institute of Allergy and Infectious Diseases (NIAID) publishes sample applications , as well as many useful templates for preparing proposals. Agencies such as the NIH and NSF offer Innovation Corps (I-Corps™) programs, which offer more in-depth support, including funding, mentoring, and networking opportunities on a team’s journey towards commercialization.

The realm of finding funding and development opportunities can seem complicated, but once you’re in it, the ecosystem is quite exciting, with many resources available to help you navigate.

Q: What breakthrough innovations in healthcare delivery or technology excite you most?

JZ: Artificial intelligence (AI) is already a burgeoning field and seems to be growing daily. It’s being applied in nearly every corner of biomedical research and healthcare, from mechanistic studies to improving staffing workflows. It’s also ripe for multidisciplinary collaboration, including small businesses with specialized skills or AI technologies.

Federal agencies are increasing their AI investments, including a plan to stand up an entirely new $6.5 billion office at NIH, the Advanced Research Projects Agency for Health (ARPA-H) . The goal is to build high-risk, high-reward capabilities (or platforms) to drive biomedical breakthroughs. This includes achieving viable products and market feasibility. Undoubtedly, small businesses will be welcome in this new arena.

I’m excited, too, about microbiome research, which is an interest area to not just the NIH, but 15 other federal agencies as well. The biome is another new frontier in medicine that we’re now learning plays a role in numerous diseases. Like AI, we’re finding application of microbiome research in a wide span of applications, including unlocking molecular secrets, developing biomarkers, creating therapeutics, and improving lifestyles.

This is such an exciting time to be involved in medical research.

Call for Healthcare Innovations

Are you developing an innovative healthcare technology? Contact the Mayo Clinic Innovation Exchange to learn how membership can help bring your company or your idea closer to patients.

An official website of the United States government

Here's how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. A lock ( ) or https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Keyboard Navigation

• Agriculture and Food Security
• Anti-Corruption
• Conflict Prevention and Stabilization
• Democracy, Human Rights, and Governance
• Economic Growth and Trade
• Environment, Energy, and Infrastructure
• Gender Equality and Women's Empowerment
• Global Health
• Humanitarian Assistance
• Innovation, Technology, and Research
• Water and Sanitation
• Burkina Faso
• Central Africa Regional
• Central African Republic
• Côte d’Ivoire
• Democratic Republic of the Congo
• East Africa Regional
• Power Africa
• Republic of the Congo
• Sahel Regional
• Sierra Leone
• South Africa
• South Sudan
• Southern Africa Regional
• West Africa Regional
• Afghanistan
• Central Asia Regional
• Indo-Pacific
• Kyrgyz Republic
• Pacific Islands
• Philippines
• Regional Development Mission for Asia
• Timor-Leste
• Turkmenistan
• Bosnia and Herzegovina
• North Macedonia
• Central America and Mexico Regional Program
• Dominican Republic
• Eastern and Southern Caribbean
• El Salvador
• Middle East Regional Platform
• West Bank and Gaza
• Dollars to Results
• Data Resources
• Strategy & Planning
• Budget & Spending
• Performance and Financial Reporting
• FY 2023 Agency Financial Report
• Records and Reports
• Budget Justification
• Our Commitment to Transparency
• Policy and Strategy
• How to Work with USAID
• Find a Funding Opportunity
• Organizations That Work With USAID
• Resources for Partners
• Get involved
• Business Forecast
• Safeguarding and Compliance
• Diversity, Equity, Inclusion, and Accessibility
• Mission, Vision and Values
• News & Information
• Operational Policy (ADS)
• Organization
• Stay Connected
• USAID History
• Video Library
• Coordinators
• Nondiscrimination Notice and Civil Rights
• Collective Bargaining Agreements
• Disabilities Employment Program
• Federal Employee Viewpoint Survey
• Reasonable Accommodations
• Urgent Hiring Needs
• Vacancy Announcements
• Search Search Search

Health Research and Development

USAID has played a critical role in promoting U.S. interests abroad by investing in research and development that has led to essential breakthroughs in prevention, diagnosis and treatment of global diseases.

USAID is committed to addressing some of the world’s most challenging health and development issues through research, introduction and scale-up of proven solutions. The Bureau for Global Health's investments in research and development have led to critical breakthroughs in prevention, diagnosis and treatment of deadly global diseases.

As a leader in the application of science and technology to achieve development objectives, USAID recognizes that a foundation in research is vital in creating game-changing innovations. Whether it's through product development or field implementation trials, we seek to broaden scientific progress and drive innovation that will empower host countries and improve lives. Through partnerships with our field missions, stakeholder countries and the private sector, the Bureau for Global Health's research and development agenda includes implementing high-impact solutions to prevent child and maternal deaths , control the HIV/AIDS epidemic and combat infectious diseases.

Learn how USAID's Health Research Program supports research and development, introduction, adoption and scale-up of products, technologies and systems to improve maternal, newborn, and child health and nutrition in low and middle income countries.

Numbers at a Glance

Through Grand Challenges for Development, USAID and its partners have cultivated a pipeline of more than 150 innovations and supported their testing, development, and scale-up on their path to deliver health impact.

Reports to Congress

• USAID Global Health Research and Development Strategy 2017–2022 [PDF, 1.0MB]
• USAID Report to Congress on Health-Related Research and Development for FY 2017 [PDF, 313KB]
• Past Health-Related Research and Development Progress Reports

HEALTH RESEARCH 2018 WEBINAR SERIES

Webinar 1: USAID’s Support to Global Health R&D – Presentation [PDF, 4MB] and Adobe Connect

Webinar 3: USAID Global Health R&D and HIV/AIDS – Presentation [PDF, 5MB] and Adobe Connect .

Webinar 4: USAID Global Health Grand Health Challenges – Presentation [PDF, 7MB] and Adobe Connect .

Webinar 5: Health Systems and Maternal and Child Health – Presentation [PDF, 3MB] and Adobe Connect .

Health Research and Implementation Process

Research & Development (R&D)

Research and development (R&D) surrounds ventures that companies undertake to innovate and introduce new products, ideas, and services to various markets. R&D is typically the initial stage in the development process of a startup product or company. Research and development allows for companies to keep ahead of their business competitors always. The R& D process is employed in many industries, such as tech, healthcare, pharmaceutical, and more.

Types of Research & Development (R&D)

There are three primary types of research and development models. Each model is used in different situations, depending on the reasons that the research and development process is being used. One form of R&D comprises a department staffed essentially with engineers. Their role is to undertake a significant amount of research with the aim of inventing and designing new products. Typically with this approach, no specific goal or application is outlined prior. Instead, the research is done to find something meaningful or needed that can be created.

A department made up of researchers, or industrial scientists is another model used quite frequently in research and development. These individuals are all employed to efficiently apply existing research in technical, scientific, or industrial fields. Such a model is used when there is a need for the enhancement of already available products and services or for the development of future products and services.

The third most basic model consists of business incubators and accelerators. This is the most commonly used of the three. With this type of R&D, organisations invest in startups and grant funding assistance and guidance to entrepreneurs with the goal that these new and upcoming innovations will be successful so that they can be used in a beneficial way. Companies also use mergers, acquisitions, and partnerships as forms of R&D. By coming together, they are able to benefit from the talent and knowledge from each other.

Research & Development In Healthcare

Research and development in medical technology is responding with new products and services to the changing landscape of healthcare, especially due to the COVID-19 pandemic. R&D is also finding new approaches to innovation by searching broadly for ideas that can be implemented in the healthcare industry as new solutions. Collaboration among governments, companies, universities, clinicians, and investors is a necessary feature for these new approaches to be successful. Forbes conducted a survey, which found that 80% of executives in MedTech expect innovation to happen through partnerships, rather than in-house. They also anticipate an increasing emphasis on open innovation, that facilitates cooperation in research and development among companies from a wide range of industries.

Research & Development In Australia’s Healthcare Sector

It is a well-established fact that Australia excels in health and medical research (HMR). The country has pioneered the treatment of spinal cord injuries, which is a jaw-dropping accomplishment and also the development of artificial heart valves. Both of these successes are indicators of the talent, importance, enthusiasm, and, dedication of our HMR workforce.

Related Companies

Aged Care , Allied Health , Hospital , Telehealth , Communications

Australia's leading telehealth provider with in-built teleassessments and purpose-built features for clinicians.

Medical Apps , Medical Devices , Systems Integration , Wearables

TRUSTED & CERTIFIED MED-TECH DEVELOPMENT EXPERTS We have years of experience helping Med-Tech clients develop new tech, specialising in class I, II medical devices.

Related Episodes

Predictive Powers Transforming Clinical Trials In this episode of Talking HealthTech , we dive into the predictive... View More

In Japan, the intersection of healthcare and technology is a pivotal area, particularly highlighted by insights... View More

Healthcare and technology are becoming increasingly intertwined, transforming the way we approach patient care.... View More

In this second of a two part series of Talking HealthTech , we discuss the integration of technology and collaboration... View More

In this special two part series of Talking HealthTech , we focus on healthcare research and systemic change in... View More

In a world where health research and clinical trials are crucial for advancing medical innovations, the process... View More

Spotlight on Trailblazing Women in Digital Health Telstra Health recently shone a spotlight on the remarkable... View More

The Landscape The ever-changing realm of healthcare and technology offers a little-explored dimension: adaptive... View More

In this episode of Talking HealthTech , Peter Birch is joined by guest Oliver Canfell, a Research Fellow in ... View More

Clinical trial recruitment is a complex field, presenting both challenges and opportunities for innovation. Ben... View More

This episode is part of a short series of Talking HealthTech episodes, produced in partnership with the Digital Health... View More

In this episode of Talking HealthTech, host Peter Birch interviews  Alan Robertson , the founder... View More

In this episode of Talking HealthTech , host Peter Birch takes us on an exciting journey through the dynamic landscape... View More

During the recent school holidays in Australia, Peter Birch from Talking HealthTech visited Wollongong University... View More

        The use of paper for collecting patient  data  across... View More

This is a feature episode recorded at the  MTAA MedTech 22 Annual Conference , an industry event... View More

Coming up with an idea for a cutting-edge  medical device  is, one might say,... View More

Did you know that one in twenty children face a birth defect or genetic disease?  Did you know... View More

The word innovation gets used a lot when we talk about healthcare, which is good because on one hand it... View More

How do you achieve credibility and momentum in international markets as an Australian healthcare technology... View More

The amount of  data   that healthcare churns out is almost incomprehensible. Somewhere... View More

Autism   is one of the fastest-growing developmental disabilities and the families and caregivers... View More

In this episode of Talking HealthTech, we have a chat with Dr Navid Toosi Saidy and Nicholas Therkelsen-Terry... View More

In this episode of Talking HealthTech, we speak with Sam Lanyon from  Planet Innovation (PI)... View More

In this episode of Talking HealthTech, we spoke to Sally-Ann Williams, the CEO of  Cicada Innovations... View More

    At the time of this recording, Karen Borg was the CEO of   Healthdirect Australia... View More

Related Videos

Australia has a great track record when it comes to research and new discoveries in health and science.  How... View More

How does Australia fare compared to other countries around the world when it comes to research and innovation in... View More

Subscribe to Podcast

Search website, you might also like..., join our newsletter.

• Talking HealthTech features content and community for those passionate about technology in healthcare.
• Get in touch
• Join the community
• Guest suggestion
• Sponsorship
• Search providers
• How it works
• List your company
• Make healthtech happen. Join THT+

• Talking HealthTech Pty Ltd
• All Rights Reserved
• Terms of Use
• Privacy Policy
• Subscribe to Newsletter
• Become a THT+ member

You are using an outdated browser. Please upgrade your browser to improve your experience.

You have accessed  https://www.emdgroup.com , but for users from your part of the world, we originally designed the following web presence  https://www.merckgroup.com .

Internet Explorer is not supported

You are using an outdated browser. If you would like to use our website, please use Chrome, Firefox, Safari or other browser

Share Disclaimer

By sharing this content, you are consenting to share your data to this social media provider. More information are available in our  Privacy Statement . 

Cookie Disclaimer

We use cookies so that we can offer you the best possible website experience. This includes cookies which are necessary for the operation of the app and the website, as well as other cookies which are used solely for anonymous statistical purposes, for more comfortable website settings, or for the display of personalized content. You are free to decide in the Cookie Settings which categories you would like to permit. Please note that depending on what you select, the full functionality of the website may no longer be available. You may review and change your choices at any time. Further information can be found in our Privacy Statement .

These cookies are necessary for the website to operate. Our website cannot function without these cookies and they can only be disabled by changing your browser preferences.

These cookies enable the provision of advanced functionalities and are used for personalization. The cookies are set in particular in response to your actions and depend on your specific service requests (e.g., pop-up notification choices).

These cookies may be set to learn more about your interests and show you relevant ads on other websites. These cookies work by uniquely identifying your browser and device. By integrating these cookies, we aim to learn more about your interests and your surfing behavior and to be able to place our advertising in a targeted manner.

Publication of Merck KGaA, Darmstadt, Germany. In the United States and Canada the subsidiaries of Merck KGaA, Darmstadt, Germany operate under the umbrella brand EMD.

Select your Market

• Our R&D Approach

R&D in Healthcare

A pioneering approach to research and development enables our Healthcare business to discover new treatments and help patients around the world to live a better life.

Explore More

• Our R&D Areas of Focus
• Our R&D Hubs
• Positions & Policies
• Clinical Trials

Vibrant science across the globe

Patients are at the center of all our research and development efforts. We are committed to innovation in science that will ultimately make a meaningful difference in people’s lives all over the world. Ever since Merck was founded 350 years ago, we have gained unmatched experience in the discovery, development and manufacturing of medicines.

On top of leveraging our experience and expertise, we also believe that a healthier future deserves a sizable investment. In 2018 our Healthcare business devoted roughly 20% of total sales to R&D activities aimed at discovering and developing new therapies. All of these are integrated in a global organization with six strategic locations and highly connected hubs : Darmstadt, Boston, Beijing ,  Tokyo, Bangalore, and Yavne.

Our pipeline

View our robust R&D pipeline across Immunology, Oncology, and Immuno-Oncology to see what we’re currently working on.

Latest R&D News

Press Releases

Positive CHMP Opinion by EMA for Arpraziquantel to Treat Schistosomiasis in ...

Committee for Medicinal Products for Human Use (CHMP) of the European Medicines Agency (EMA) has adopted a positive opinion for arpraziquantel.

Merck Provides Update on Phase III Results for Evobrutinib in Relapsing Multiple ...

Merck announced that its two Phase III EVOLUTION clinical trials of evobrutinib did not meet their primary endpoints.

Merck and p-Chip close the gap between physical and digital trust layers

We successfully integrated p-Chip's microtransponder technology as a crypto-anchor into a cyber-physical trust framework solution developed by Merck.

Work With Us

Join our r&d team.

Are you curious to see how you can contribute to a healthier future? Join our team of scientists and development professionals! You can find all open positions within our Healthcare business in the Careers section.

Partner with us

Productive partnerships.

Partner with our R&D experts to unlock your project’s potential. We believe in mutually beneficial alliances built on transparency and commitment to ambitious, well-defined goals. These can be many and varied. For example, to help bring innovations to market, or to better market commercial products. Whatever your project, we are excited to hear your proposal!

Grants & Awards

Innovators wanted! At Merck, we highly value creativity, initiative and innovation. So, it’s very important to us that we recognize scientists, students and entrepreneurs who exhibit these qualities in outstanding work.

We're studying the links between human health and environmental change.

Funding Opportunities for Faculty, Post-docs and Students  

We're advancing research and innovation in disease ecology, health, and the environment through:.

Keystone projects in basic and applied science aimed at ecological solutions for health

Workshops, working groups, and community-building events

Cross-disciplinary curriculum development and student involvement

Deep engagement with global health, science, and environmental leaders

Current Keystone Projects

Partners in Schistosomiasis Reduction

The Upstream Alliance

Water projects like dams and irrigation schemes support agricultural expansion to feed the growing human population. But these water projects come with costs in the form of degradation of ecosystems and in some cases, of compromised human health.

Interventions to End Cycles of Poverty Driven By Neurocysticerosis

Pigs & Parasites

Pigs, which thrive on cheap and abundant nutritional sources and grow quickly, are one of the most important sources of food and income for the world’s poorest populations. But pigs also serve as the reservoir for the pork tapeworm, Taenia solium.

Nature-based water solutions for urban revitalization and health

Revitalizing Informal Settlements and their Environments (RISE)

Informal settlements ('slums') are home to more than a billion people, mostly in rapidly growing urban areas of low- to middle-income countries.

Community-Led Solutions

Health in Harmony

Health in Harmony embraces the links between human health, ecosystem health, and planetary health.

A Community-based intervention pilot in Kwale County, Kenya

Trash to Treasure

Trash, specifically unused containers like bottles, buckets, and tires, can be re-purposed for a variety of profitable items.

Meet the people who make it all happen...

We support health and environmental research and innovative solutions.

• Introduction
• Article Information

eAppendix. Search Terms Used for Data Extraction, Based on Janiesch et al. and MeSH-Terms for Artificial Intelligence

Data Sharing Statement

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Druedahl LC , Price WN , Minssen T , Sarpatwari A. Use of Artificial Intelligence in Drug Development. JAMA Netw Open. 2024;7(5):e2414139. doi:10.1001/jamanetworkopen.2024.14139

Manage citations:

© 2024

• Permissions

Use of Artificial Intelligence in Drug Development

• 1 Centre for Advanced Studies in Bioscience Innovation Law (CeBIL), Faculty of Law, University of Copenhagen, Copenhagen, Denmark
• 2 University of Michigan Law School, Ann Arbor
• 3 Program On Regulation, Therapeutics, And Law (PORTAL), Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts

Considerable focus has been placed on the health care applications of artificial intelligence (AI). Already, machine learning, a subset of AI that involves “the use of data and algorithms to imitate the way that humans learn” 1 has been used to predict diseases, 2 while AI-powered smartphone apps have been developed to promote mental health and weight loss. 3 Owing in part to such successes, the market for AI in health care has been forecasted to increase more than 1000% between 2022 and 2029, from $13.8 billion to $164.1 billion. 4

One area of substantial promise is drug development, which is poised to benefit from advances in the use of AI to predict protein folding, molecular interactions, and cellular disease processes. 5 Successful application of AI to drug development, however, faces several obstacles, including poor model performance caused by nondiverse training data and shortcut learning. Additionally, the often opaque ways that AI systems reach their predictions conflict with regulatory approval frameworks that require a rationale for decision-making. Given these obstacles, we sought to identify the scope and breadth of AI use in drug development.

We conducted a cross-sectional study of investigational and approved drugs (n = 102 454) listed in the global research and drug development database Pharmaprojects (Informa, Citeline) on February 11, 2024. Institutional review board approval was not required because this study did not involve human participants, in accordance with 45 CFR §46. We followed the STROBE reporting guideline.

To identify AI-developed drugs, we used AI search terms from Janiesch et al 6 and the National Library of Medicine’s Medical Subject Headings database (eAppendix in Supplement 1 ). Automated scans of information for each drug were evaluated. If a drug was described as developed with AI, the type and purpose of AI use were noted. When the type or purpose of AI use could not be determined, additional information was obtained from internet sources. The most specific term identified was used to categorize the AI application ( Figure 1 ). Among AI-developed drugs, descriptive statistics were compiled of AI type, AI purpose, therapeutic area, and development status. Statistical analysis was performed using Excel version 16 (Microsoft) from February to March 2024.

The database search yielded 406 drugs, of which 241 were excluded upon review for no reported use of AI in drug development. AI use was reported in the development of 164 investigational drugs and 1 approved drug. The most frequent types of AI use were machine learning (n = 46 [28%]) and deep learning (n = 28 [17%]). AI was used for 12 purposes, most commonly drug molecule discovery (n = 125 [76%]) ( Figure 2 ). Examples of such use ranged from platform screening of drugs, in which AI was used to analyze molecular images of the effects of drugs on a cell, to deep generative modeling to design virtual novel molecules. Modest AI use was observed for drug target discovery (n = 37 [22%]), including machine learning to find previously unknown connections between genomic, chemical, and clinical data. AI use for clinical outcomes analysis, such as the use of AI-based pattern recognition algorithms to identify correlations between immune responses and patient survival, was more limited (n = 5 [3%]). Regarding therapeutic area, AI use was most common for intended anticancer (n = 52 [27%]) and neurological (n = 24%) treatments. For the 1 approved drug, which was the stem cell therapy remestemcel-L, a bayesian method was used to estimate the likelihood of obtaining significant results on the primary end point at study completion.

This study found modest use of AI for drug development focused primarily on early-stage applications and on anticancer and neurological therapies. Possible explanations include a lack of high-quality data available in the subsequent stages of drug discovery and uncertain regulatory expectations concerning late-stage AI applications. Study limitations included relying upon public disclosures by drug manufacturers. Ultimately, this study’s results suggest that greater clarity from medicines regulators is needed to guide sponsors over acceptable AI standards and applications to satisfy marketing authorization requirements.

Accepted for Publication: March 29, 2024.

Published: May 31, 2024. doi:10.1001/jamanetworkopen.2024.14139

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2024 Druedahl LC et al. JAMA Network Open .

Corresponding Author: Ameet Sarpatwari, PhD, JD, Brigham and Women’s Hospital, 1620 Tremont St, Boston, MA 02120 ( [email protected] ).

Author Contributions: Dr Druedahl had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: Druedahl, Price, Sarpatwari.

Drafting of the manuscript: Druedahl, Minssen, Sarpatwari.

Critical review of the manuscript for important intellectual content: All authors.

Statistical analysis: Sarpatwari.

Administrative, technical, or material support: Sarpatwari.

Supervision: Price, Minssen, Sarpatwari.

Conflict of Interest Disclosures: Dr Druedahl reported working for and having shares in Novo Nordisk starting June 2023 and November 2023, respectively. However, Dr Druedahl’s work on this study was carried out under the umbrella of the Centre for Advanced Studies in Biomedical Innovation Law, the current Centre for Advanced Studies in Bioscience Innovation Law, and independently from Novo Nordisk. Dr Minssen reported personal fees from serving as an advisor to a company engaged in artificial intelligence compliance and governance and to a law firm, both outside the submitted work.

Funding/Support: The study was supported, in part, by the Collaborative Research Program for Biomedical Innovation Law, a scientifically independent research program supported by the Novo Nordisk Foundation (grant NNF17SA0027784). Dr Price’s and Dr Minssen’s work was also supported, in part, by a subsequent Novo Nordisk Foundation Grant for a scientifically independent International Collaborative Bioscience Innovation & Law Programme (Inter-CeBIL programme- grant No. NNF23SA0087056). Dr Sarpatwari’s work was also supported by a grant from Arnold Ventures.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2 .

Additional Contributions: The authors would like to thank Dr Ole Lund for his insightful comments on an earlier draft.

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

UConn Today

• School and College News
• Arts & Culture
• Community Impact
• Entrepreneurship
• Health & Well-Being
• Research & Discovery
• UConn Health
• University Life
• UConn Voices
• University News

June 4, 2024 | Mikala Kane - Neag School of Education

UConn’s Renzulli Center Hosts Wallace Research Symposium on Talent Development

The national symposium came to Storrs for the first time, bringing together experts on gifted education and talent development

A group photo of attendees at the Wallace Research Symposium on Talent Development at McHugh Hall on May 20 in Storrs. (Peter Morenus/UConn Photo)

On May 19, the preeminent research conference on gifted education and talent development came to Storrs for the first time. For three days, around 180 researchers, scholars, and educators filled UConn’s McHugh Hall and the third floor of the Student Union for the 2024 Wallace Research Symposium on Talent Development.

Previously hosted solely by the University of Iowa’s Belin-Blank Center for Gifted Education and Talent Development , the symposium was also co-hosted this year by UConn’s Renzulli Center for Creativity, Gifted Education, and Talent Development and the National Center for Research on Gifted Education (NCRGE).

“The Wallace Research Symposium has traditionally taken place every two years but had not been held since the COVID-19 lockdown,” says Del Siegle, UConn Neag School of Education faculty member and director of the Renzulli Center and the NCRGE. “Two years ago, at the National Association for Gifted Children’s conference, I approached Megan Foley Nicpon, the new director at the Belin-Blank Center, about collaborating to hold the symposium at UConn. She embraced the idea, and we began holding monthly planning meetings online.”

When my colleague Dr. Del Siegle mentioned that our Renzulli Center was looking to collaborate with Iowa’s Belin-Blank Center to host this illustrious gathering, I couldn’t think of a more natural partnership. — Dean Jason G. Irizarry

The Renzulli Center, part of the Neag School, is one of the leading centers in the world in gifted education and talent development. Since 1996, the center has promoted enjoyment, engagement, and enthusiasm for learning in educators and students at all levels through research and professional learning activities. Its affiliated faculty are renowned experts in the field and the center supports graduate students in master’s/sixth-year, Ph.D., and online certificate programs.

It was a natural partnership, then, for the Renzulli Center to join with Iowa’s Belin-Blank Center for this year’s Wallace Research Symposium. The Belin-Blank Center aims to create opportunities for equitable talent development through five strategic priorities: outreach, professional learning, psychological services, research, and student programming. Founded in 1988, the center launched the Wallace Research Symposium after a generous endowment from the Wallace Research Foundation.

“Henry B. (H.B.) Wallace was an exceptionally talented individual who used his abilities to enhance society,” says Foley Nicpon, director of the Belin-Blank Center. “H.B. and his wife, Jocelyn, deeply cared about students and the future of American education and demonstrated this caring by their outstanding support of the Belin-Blank Center at the University of Iowa. H.B. and Jocelyn strongly believed that the future of America rested with its young people and that students of exceptional educational promise should have every opportunity to develop their talents and then use these talents to better society.”

The 2024 symposium began on May 19 with a keynote by Malik Henfield, professor in the School of Education and founding dean of the Institute for Racial Justice at Loyola University Chicago, titled “Illuminating Equitable Pathways: Unveiling Racial Dynamics in Gifted Education.” A panel discussion on the future of gifted education and talent development followed later in the evening.

On Monday, May 20, Neag School Dean Jason G. Irizarry welcomed the symposium attendees to Storrs before a full day of activities.

“When my colleague Dr. Del Siegle mentioned that our Renzulli Center was looking to collaborate with Iowa’s Belin-Blank Center to host this illustrious gathering, I couldn’t think of a more natural partnership,” Irizarry said. “Both centers have led the way in creativity, gifted education, and talent development for decades. I knew their collaboration would only further enhance a symposium that has always offered innovative programming and rich opportunities for experts like yourselves.”

He also took a moment to thank the planning team behind the 2024 symposium: Brian Douglas, Foley Nicpon, Stephanie Huntington, Emily Ladendorf, Catherine Little, Ann Lupkowski-Shoplik, D. Betsy McCoach, Lisa Muller, Siegle, and Siamak Vahidi.

Another Monday highlight was the Julian C. Stanley Distinguished Lecture, given in the morning by Susan Assouline, emerita director of the Belin-Blank Center. Her lecture was titled, “Current Mindsets in Gifted Education and Talent Development: What Happened to Talent Discovery?”

Most of the rest of Monday was then dedicated to paper sessions, with three invited speakers presenting around midday: Rena Subotnik, Sally Reis, and Frank Worrell. Before an evening of poster presentations, UConn’s Provost and Executive Vice President for Academic Affairs Anne D’Alleva briefly welcomed attendees, and Laura Giuliano of the University of California-Santa Cruz gave the afternoon keynote on “Gifted Education Research from the Economists’ Perspective: What Have We Learned?”

The final day of the symposium featured additional paper sessions and three more invited speakers – Paula Olszewski-Kubilius, Jacob Michaelson, and Franzis Preckel – before Po-Shen Loh of Carnegie Mellon University gave the closing keynote: “Uniting Game Theory, Math Stars, and Actors to Build Human Intelligence in the AI Age.”

“It was a pleasure to host the nation’s and the world’s top researchers in gifted education and talent development here at UConn,” Siegle says. “We were proud to showcase our beautiful campus and share the cutting-edge research being conducted by the Renzulli Center’s faculty and staff.”

To learn more about the 2024 Wallace Research Symposium on Talent Development, visit gifted.uconn.edu/wallace .

Recent Articles

June 4, 2024

Dr. George Kuchel to Serve on Centers for Disease Control and Prevention Advisory Committee

Read the article

Favarh’s Project SEARCH Graduates 3 More at UConn Health

Game On: UConn 4-H Introduces Youth to Biotechnology

Language selection

• Français fr

Government of Canada supports leading research infrastructure across Canada

From: Employment and Social Development Canada

News release

Funding will advance the next generation of cutting-edge Canadian research and innovation infrastructure

Funding will advance the next generation of cutting-edge Canadian research and innovation infrastructure May 31, 2024                Ottawa, Ontario              Employment and Social Development Canada Modern, high-quality research facilities and equipment are essential for breakthroughs in Canadian research and science. These laboratories and research centres are where medical and other scientific breakthroughs are born, helping to solve real-world problems and create the economic opportunities of the future. Today, the Honourable Jenna Sudds, Minister of Families, Children and Social Development on behalf of the Honourable François-Philippe Champagne, Minister of Innovation, Science and Industry, and the Honourable Steven MacKinnon, Member of Parliament for Gatineau, highlighted $176 million over five years, through Budget 2024 , to support CANARIE, a national not-for-profit organization that connects Canada's researchers, educators, and innovators, to each other and to scientific data and instruments through an ultra high-speed network. CANARIE is also the Canadian operator for eduroam, the secure, global Wi-Fi network for students and researchers. With network speeds hundreds of times faster than conventional home and office networks, this investment will ensure this critical infrastructure can securely connect researchers across Canada's world-leading post-secondary institutions. CANARIE and its 13 provincial and territorial partners form Canada’s National Research and Education Network (NREN). The NREN connects Canada’s researchers, educators, and innovators to each other and to global data and technology. CANARIE collaborates with partners in the NREN, government, academia, and the private sector to strengthen cybersecurity at over 220 Canadian post-secondary institutions and research facilities. Canada’s world-class research facilities play a critical role in finding solutions to major challenges and advancing a resilient and sustainable future. Investments in infrastructure drive innovation and help attract and train the next generation of scientific talent, creating a better future for all Canadians and people around the world.

“Canadian research has helped improve our society, economy and healthcare, time and time again. These strategic investments underscore the government dedication to fostering innovation, addressing global challenges, and nurturing the next generation of scientific leaders. Through enhancing cutting-edge facilities and equipment, these initiatives will propel Canadian research to new heights of excellence." – The Honourable François-Philippe Champagne, Minister of Innovation, Science and Industry

“CANARIE plays a crucial role in advancing Canada's digital economy by providing high-speed networks, data management tools, and cybersecurity solutions for research institutions across the country. This funding underscores our government’s commitment to fostering innovation and ensuring that Canadian researchers have the world-class tools and resources they need to drive groundbreaking discoveries and bolster our nation's competitiveness on the global stage. Together, we’re creating opportunities, boosting innovation, and accelerating economic growth for generations to come.” – The Honourable Jenna Sudds, Minister of Families, Children and Social Development

“Advancing the next generation of cutting-edge Canadian research and innovation infrastructure is essential to find solutions to major challenges and to advance an innovative and sustainable future. Investments like this one demonstrate our government's ongoing commitment to supporting Canada's science and research ecosystem.” – The Honourable Steven MacKinnon, Leader of the Government in the House of Commons and Member of Parliament for Gatineau

“Congratulations to CANARIE on the renewal of their five-year mandate with our government, allowing them to continue their cutting-edge support of Canadian researchers, educators and innovators. Since 1993, CANARIE has grown to be a global leader, creating a powerful digital platform connecting our world class researchers and educational institutions to one another and the world. Our renewed investment in CANARIE over the next five years will further enhance scientific collaboration and accelerate Canadian innovation, opening new frontiers leading to Canadian economic growth and the well-being of our people.” – Yasir Naqvi, Member of Parliament for Ottawa Centre

Quick facts

Since 2016, the government has provided more than $16 billion to support science and research.

This new investment builds on existing federal research support:

• The Strategic Science Fund, which announced the results of its first competition in December 2023, providing $800 million to support 24 third-party science and research organizations starting in 2024-25;
• Canada recently concluded negotiations to be an associate member of Horizon Europe, which will enable Canadians to access a broader range of research opportunities under the European program starting this year;
• In addition, Budget 2024 provides $825 million to increase support for master’s, doctoral and post-doctoral students, as well as $1.8 billion to the federal granting councils to increase core research grant funding and support Canadian researchers.
• The steady increase in federal funding for extramural and intramural science and technology by the government which was 44 per cent higher in 2023 relative to 2015.

Associated links

• Budget 2024

Geneviève Lemaire Press Secretary Office of Minister of Families, Children and Social Development [email protected] Media Relations Office Employment and Social Development Canada 819-994-5559 [email protected] Audrey Milette Press Secretary Office of the Minister of Innovation, Science and Industry [email protected] Media Relations Innovation, Science and Economic Development Canada [email protected] Stay connected Follow @CDNScience on social media for Canadian science news:  Twitter ,  Instagram ,  Facebook

Page details

Along with Stanford news and stories, show me:

• Student information
• Faculty/Staff information

We want to provide announcements, events, leadership messages and resources that are relevant to you. Your selection is stored in a browser cookie which you can remove at any time using “Clear all personalization” below.

Image credit: Claire Scully

Gen Z is growing up: In 2024, the generation born between 1996 to 2010 is expected to overtake Baby Boomers in the full-time workforce, according to a recent analysis by Glassdoor .

They are bringing to the office a different set of values, behaviors, and expectations than prior generations, according to research by Roberta Katz , a former senior research scholar at Stanford’s Center for Advanced Study in the Behavioral Sciences (CASBS) . Katz collaborated with a team of researchers to conduct a large, multi-year study to find out what matters to Gen Z and why – findings that culminated in a book and website .

Stanford Report sat down with Katz to talk about this research and what to expect from Gen Z in the workplace.

1. Gen Z expects change

The world Gen Zers came of age in was fundamentally different from that of their parents and even millennials, people who were born in the early 1980s to 1996.

The world of Gen Z has been defined by technological changes happening at rapid speeds that also reshaped social experiences. Disruption and impermanence have always been part of the world Gen Z experienced – for them, it’s a norm, not an exception.

“There is an expectation of constant change,” said Katz.

Growing up amid uncertainty has given Gen Z a unique set of characteristics, including being flexible and resilient. It has opened them up to new ways of thinking about the future and doing things – and questioning the ways things are done, which leads to the next trait Gen Zers will bring with them to work.

2. Gen Z is pragmatic

Gen Z has a strong sense of self-agency.

Gen Z lives in a world that has always been one search engine result away. If they want to know more about something, they readily seek the answer out for themselves ( even if it’s not always the correct one ).

They question everything and everyone – from their peers, parents, or people at work. “They don’t necessarily see elders as experts,” Katz said. “They want to understand why something is done in a certain way. They’re very pragmatic.”

They are also not afraid to challenge why things are done the way they are.

“When an older person says to them, ‘This is how you should do it,’ they want to check that out for themselves. It doesn’t mean they’re always right; it’s a different way of understanding,” Katz explained.

3. Gen Z wants to make a difference

Gen Zers not only expect change – they demand it.

They are inheriting a set of complex problems – from climate change to inequality to racial injustice, to name but a few – and want to fix it. They want to work for a place that they believe is doing good in the world.

Some Gen Zers will hold their employers accountable on the causes and issues that matter to them.

Katz warns that for some employers, it can be challenging – if not untenable – to take a position on politically charged or sensitive topics. “It is impossible for most institutions that represent lots of people and lots of identities to satisfy everybody,” Katz said.

4. Gen Z values collaboration and teamwork

For some Gen Zers, the digital world helped shape their identity: Through social media and in online groups, they found subcultures to connect and interact with.

They grew up with wikis – websites collaboratively built and edited by its users – and fandoms – enthusiastic and energetic communities centered around a shared, common interest. For example, K-pop sensation BTS has its Army , Beyonce has her Beyhive, and Taylor Swift has her Swifties.

“They’re in a posse – even with their headphones on,” Katz said.

To get things done, they value collaboration.

“There is a hope that everybody who is contributing is in it for the good of the whole,” Katz describes. “They want to have a team spirit.”

5. Gen Z wants leaders who guide by consensus

Gen Z is also less hierarchical than previous generations.

“They don’t believe in hierarchy for hierarchy’s sake,” Katz said. “They do believe in hierarchy where it is useful.”

Instead, Gen Zers prefer leadership that is dependent on expertise that is task or time specific. That could mean they favor management where team members take turns leading the group (known as a “rotating leadership” model). Another style they may prefer is “collaborative leadership,” in which people from across the organization participate in decision-making and problem-solving.

Transparency is also important.

Gen Zers value consensus and they look for leaders who are in service of the group (also called “service leadership”).

6. Gen Z cares about mental health and work-life balance

Gen Z grew up in a period that saw the blurring of the 9-to-5 work schedule and the rise of flexible work models – a mode of working that led to older generations feeling a pressure to always be “on.”

“Work and home life are all so integrated that if you don’t pay attention, you could be working all the time,” said Katz. “I think Gen Z is sensitive to that.”

Having a work-life balance and maintaining mental and physical health is also important to Gen Z.

“They’re placing a value on the human experience and recognizing that life is more than work,” Katz said.

7. Gen Z thinks differently about loyalty

Because Gen Z grew up amid so much change, Gen Z has a different perspective on loyalty.

But as Katz pointed out, “they also grew up with workplaces not being very loyal to their employees.”

Gen Zers were raised in the shadows of the global financial crisis of 2008, an event that has had long-lasting impacts on employment and the nature of work. “It used to be that people went to work for big companies thinking they’d be there for their entire career and that the company would watch out for them: providing health insurance, and so on,” Katz said.

But after the 2008 recession, and even more recently following the COVID-19 pandemic, companies have cut back labor costs and implemented other cost-saving measures, like reducing perks and benefits. Meanwhile, mass layoffs have also been rampant.

“There’s a reason that employees don’t feel the same degree of loyalty, too,” Katz said.

Meanwhile, the gig economy has also been present throughout Gen Zers’ lives, as has the rise of contract work. They are entrepreneurial, which is part of their pragmatic tendencies.

8. Gen Z looks for trust and authenticity

Gen Z also values authenticity.

“Authenticity is about trust,” Katz said. “Words and actions need to match.”

Honesty and openness are important.

For Katz, it’s all about mutually respectful communication. “My bottom line always to employers is stay open to hearing about different ways to get things done, because Gen Z has one foot in the future.”

Katz is associate vice president for strategic planning, emerita, and is currently involved in a strategic role with the Stanford Doerr School of Sustainability and the Stanford Institute for Human-Centered Artificial Intelligence . She also serves as vice chair of the board of the Center for Advanced Study in the Behavioral Sciences (CASBS).

Katz studied Gen Z as part of a multi-year CASBS research project with Sarah Ogilvie, a linguist at the University of Oxford and formerly at Stanford; Jane Shaw, a historian who is the principal of Harris Manchester College at Oxford and was previously dean for religious life at Stanford; and Linda Woodhead, a sociologist at King’s College London. The research was funded by the Knight Foundation.

From 2004 to 2017, Katz served under Stanford University Presidents John Hennessy and Marc Tessier-Lavigne as associate vice president for strategic planning, and in 2017 as interim chief of staff.

Press Release Details

Nvidia enables real-time healthcare, industrial and scientific ai applications at the edge with enterprise software support for nvidia igx with holoscan.

TAIPEI, Taiwan, June 02, 2024 (GLOBE NEWSWIRE) -- COMPUTEX  -- To address the increasing need for real-time AI computing at the industrial edge, NVIDIA today announced the general software availability of NVIDIA AI Enterprise-IGX with NVIDIA Holoscan on the NVIDIA IGX™ platform . Together, they empower solution providers within the medical, industrial and scientific computing sectors to develop and deploy edge AI solutions faster, with enterprise-grade software and support.

NVIDIA AI Enterprise-IGX is a new offering providing enterprises with unprecedented performance, security and support for the edge computing software stack, streamlining AI-powered operations and the deployment of AI applications at scale. NVIDIA Holoscan is a sensor-processing platform for streamlining the development and deployment of AI and high-performance computing applications to deliver real-time insights.

By combining NVIDIA AI Enterprise-IGX and Holoscan on IGX, NVIDIA offers an enterprise-grade platform that delivers powerful AI compute, flexible sensor integration, real-time performance and functional safety for the industrial edge — cutting the time and costs required to build advanced AI solutions across industries.

“As software-defined functionality continues to transform businesses across industries, enterprises are seeking powerful edge AI solutions that can meet their unique performance and regulatory requirements,” said Deepu Talla, vice president of robotics and edge computing at NVIDIA. “The NVIDIA IGX platform’s new capabilities deliver powerful enterprise-grade software from the cloud to the industrial edge, giving customers increased performance, safety and scalability.”

NVIDIA IGX Platform Expands In addition to the introduction of NVIDIA AI Enterprise-IGX with NVIDIA Holoscan, the NVIDIA IGX platform has undergone a major refresh .

• The NVIDIA IGX Orin™ 700, previously called the IGX Boardkit, now supports the NVIDIA RTX™ 6000 Ada GPU as a new configuration option, delivering up to 1,705 trillion operations per second — a 7x increase in AI performance compared with using an onboard iGPU. This provides even more computing power at the edge for generative AI and high-performance computing workloads.
• NVIDIA IGX now supports a new product, the IGX Orin 500 system-on-module, which enables the creation of flexible carrier-board designs and custom configurations without sacrificing enterprise software support.
• The NVIDIA-Certified Systems™ program has expanded to include the IGX platform, ensuring systems built with IGX are validated to run accelerated AI workloads with optimized performance. Companies including Advantech, ADLINK, Aetina, Ahead, Cosmo Intelligent Medical Devices (a division of Cosmo Pharmaceuticals), Dedicated Computing, Leadtek, Onyx and YUAN are building NVIDIA-Certified IGX systems.

Bringing AI to the Medical Edge Leading medical technology companies Barco, Karl Storz, Medtronic and Moon Surgical are adopting NVIDIA IGX with Holoscan to accelerate the development of AI-powered solutions for medical diagnostics, surgical copilots, surgical robots, patient care agents and more. Additionally, Johnson & Johnson MedTech is working on how NVIDIA IGX with Holoscan could accelerate the development of AI for Polyphonic, Johnson & Johnson MedTech’s digital ecosystem for surgery.

Healthcare technology provider Medtronic is leveraging NVIDIA IGX with NVIDIA Holoscan for its GI Genius™ intelligent endoscopy module designed, developed and manufactured by Cosmo Intelligent Medical Devices. It’s the first FDA-cleared, AI-assisted colonoscopy tool to help physicians detect polyps that can lead to colorectal cancer.

“The NVIDIA IGX with Holoscan platform has significantly accelerated our AI innovation in endoscopy,” said Raj Thomas, president of endoscopy at Medtronic. “By leveraging NVIDIA's advanced technology, we can focus on developing groundbreaking software applications that ultimately enhance patient outcomes and provide greater support to physicians. This collaboration underscores our commitment to pioneering advancements in medical technology for the benefit of all.”

Robotic surgery company Moon Surgical is using IGX with Holoscan to power its Maestro System, a state-of-the-art surgical robotics system designed to assist surgeons with precision and control during minimally invasive procedures.

“NVIDIA’s IGX platform with Holoscan accelerated the development of our Maestro System as well as enhance Maestro’s capabilities,” said Anne Osdoit, CEO of Moon Surgical. “Our collaboration with NVIDIA has allowed us to get our intelligent robotic assistant into the hands of surgeons sooner and with more features, enabling us to get valuable feedback on our path to commercialization.”

Industrial AI at the Edge The NVIDIA IGX platform significantly improves functional safety and high-bandwidth sensor processing, transforming factory automation and robotic collaboration with AI.

ADLINK is using NVIDIA IGX to build industrial-grade edge AI solutions for its manufacturing processes.

“ADLINK leverages NVIDIA IGX and Holoscan to deliver proactive safety capabilities that ensure more efficient, seamless human and robot collaboration,” said Stephen Huang, president and chief operating officer at ADLINK. “Working with NVIDIA, we continue to drive precision, safety and latency improvements to optimize manufacturing operations like machine movement routing, robotic arm operation and charging-station monitoring all at once.”

Edge AI for Exploring Extraterrestrial Worlds NVIDIA IGX is helping scientific researchers venture into new worlds with AI, transforming radar processing and radio astronomy with real-time edge computing capabilities.

Nonprofit research organization SETI Institute is leveraging NVIDIA IGX Orin to power radio astronomy capabilities for its Hat Creek Radio Observatory, which aims to detect technologically capable extraterrestrial life.

“The SETI Institute is using the IGX Orin platform’s advanced capabilities with the Holoscan sensor-processing platform to enable transformational capabilities in radio astronomy,” said Andrew Siemion, Bernard M. Oliver Chair at SETI Institute. “We can now stream multiple terabits per second of radio telescope data directly into AI classifiers with minimal overhead and exceptional computational performance, allowing us to process more bandwidth from more antennas to detect weaker and rarer astrophysical phenomena.”

Watch NVIDIA founder and CEO Jensen Huang’s COMPUTEX keynote to learn the latest on AI computing at the industrial edge.

About NVIDIA NVIDIA (NASDAQ: NVDA) is the world leader in accelerated computing.

For further information, contact: Olivia Bass NVIDIA Corporation +1-203-313-6771 [email protected]

Certain statements in this press release including, but not limited to, statements as to: the benefits, impact, and performance of NVIDIA’s products, services, and technologies, including NVIDIA AI Enterprise-IGX, NVIDIA Holoscan, NVIDIA IGX platform, NVIDIA IGX Orin 700, NVIDIA RTX 6000 Ada GPU, IGX Orin 500 system-on-module, and NVIDIA-Certified System program; the benefits and impact of NVIDIA’s collaboration with third parties, and the features of their services and offerings; third parties using or adopting our products or technologies; enterprises seeking powerful edge AI solutions that can meet their unique performance and regulatory requirements; and NVIDIA IGX platform’s new capabilities delivering powerful enterprise-grade software from the cloud to the industrial edge, giving customers increased performance, safety and scalability are forward-looking statements that are subject to risks and uncertainties that could cause results to be materially different than expectations. Important factors that could cause actual results to differ materially include: global economic conditions; our reliance on third parties to manufacture, assemble, package and test our products; the impact of technological development and competition; development of new products and technologies or enhancements to our existing product and technologies; market acceptance of our products or our partners' products; design, manufacturing or software defects; changes in consumer preferences or demands; changes in industry standards and interfaces; unexpected loss of performance of our products or technologies when integrated into systems; as well as other factors detailed from time to time in the most recent reports NVIDIA files with the Securities and Exchange Commission, or SEC, including, but not limited to, its annual report on Form 10-K and quarterly reports on Form 10-Q. Copies of reports filed with the SEC are posted on the company's website and are available from NVIDIA without charge. These forward-looking statements are not guarantees of future performance and speak only as of the date hereof, and, except as required by law, NVIDIA disclaims any obligation to update these forward-looking statements to reflect future events or circumstances.

© 2024 NVIDIA Corporation. All rights reserved. NVIDIA, the NVIDIA logo, NVIDIA Certified-Systems, NVIDIA IGX, NVIDIA IGX Orin and NVIDIA RTX are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the respective companies with which they are associated. Features, pricing, availability and specifications are subject to change without notice.

A photo accompanying this announcement is available at: https://www.globenewswire.com/NewsRoom/AttachmentNg/1287fd10-7aa3-48f9-a727-4a166005f0ad

NVIDIA IGX with Holoscan

NVIDIA IGX with Holoscan used in medical, industrial robotics and scientific computing applications.

Quick links.

• Email Alerts
• Request Printed Materials
• Download Library

To receive notifications via email, enter your email address and select at least one subscription below. After submitting your information, you will receive an email. You must click the link in the email to activate your subscription. You can sign up for additional subscriptions at any time.

Email Alert Sign Up Confirmation

Investor contact.

2788 San Tomas Expressway Santa Clara, CA 95051

• Contact Investor Relations

Investor Resources

• Request Information
• Stock Quote & Chart
• Historical Price Lookup
• Investment Calculator
• Fundamentals
• Analyst Coverage
• Management Team
• Board of Directors
• Governance Documents
• Committee Composition
• Contact the Board
• Corporate Sustainability
• Events & Presentations

Financial Info

• Financial Reports
• SEC Filings
• Quarterly Results
• Annual Reports and Proxies

Investors and others should note that we announce material financial information to our investors using our investor relations website, press releases, SEC filings and public conference calls and webcasts. We intend to use our  @NVIDIA  Twitter account,  NVIDIA Facebook  page,  NVIDIA LinkedIn  page and company  blog  as a means of disclosing information about our company, our services and other matters and for complying with our disclosure obligations under Regulation FD. The information we post through these social media channels may be deemed material. Accordingly, investors should monitor these accounts and the blog, in addition to following our press releases, SEC filings and public conference calls and webcasts. This list may be updated from time to time.

• Privacy Policy
• Manage My Privacy
• Do Not Sell or Share My Data
• Terms of Service
• Accessibility
• Corporate Policies

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Institute of Medicine (US) Roundtable on Value & Science-Driven Health Care; Olsen LA, McGinnis JM, editors. Redesigning the Clinical Effectiveness Research Paradigm: Innovation and Practice-Based Approaches: Workshop Summary. Washington (DC): National Academies Press (US); 2010.

Redesigning the Clinical Effectiveness Research Paradigm: Innovation and Practice-Based Approaches: Workshop Summary.

• Hardcopy Version at National Academies Press

1 Evidence Development for Healthcare Decisions: Improving Timeliness, Reliability, and Efficiency

• INTRODUCTION

The rapid growth of medical research and technology development has vastly improved the health of Americans. Nonetheless, a significant knowledge gap affects their care, and it continues to expand: the gap in knowledge about what approaches work best, under what circumstances, and for whom. The dynamic nature of product innovation and the increased emphasis on treatments tailored to the individual—whether tailored for genetics, circumstances, or patient preferences—present significant challenges to our capability to develop clinical effectiveness information that helps health professionals provide the right care at the right time for each individual patient.

Developments in health information technology, study methods, and statistical analysis, and the development of research infrastructure offer opportunities to meet these challenges. Information systems are capturing much larger quantities of data at the point of care; new techniques are being tested and used to analyze these rich datasets and to develop insights on what works for whom; and research networks are being used to streamline clinical trials and conduct studies previously not feasible. An examination of how these innovations might be used to improve understanding of clinical effectiveness of healthcare interventions is central to the Roundtable on Value & Science-Driven Health Care’s aim to help transform how evidence is developed and used to improve health and health care.

• EBM AND CLINICAL EFFECTIVENESS RESEARCH

The Roundtable has defined evidence-based medicine (EBM) broadly to mean that, “to the greatest extent possible, the decisions that shape the health and health care of Americans—by patients, providers, payers, and policy makers alike—will be grounded on a reliable evidence base, will account appropriately for individual variation in patient needs, and will support the generation of new insights on clinical effectiveness.” This definition embraces and emphasizes the dynamic nature of the evidence base and the research process, noting not only the importance of ensuring that clinical decisions are based on the best evidence for a given patient, but that the care experience be reliably captured to generate new evidence.

The need to find new approaches to accelerate the development of clinical evidence and to improve its applicability drove discussion at the Roundtable’s workshop on December 12–13, 2007, Redesigning the Clinical Effectiveness Research Paradigm. The issues motivating the meeting’s discussions are noted in Box 1-1 , the first of which is the need for a deeper and broader evidence base for improved clinical decision making. But also important are the needs to improve the efficiency and applicability of the process. Underscoring the timeliness of the discussion is recognition of the challenges presented by the expense, time, and limited generalizability of current approaches, as well as of the opportunities presented by innovative research approaches and broader use of electronic health records that make clinical data more accessible. The overall goal of the meeting was to explore these issues, identify potential approaches, and discuss possible strategies for their engagement. Key contextual issues covered in the presentations and open workshop discussions are reviewed in this chapter.

Issues Motivating the Discussion.

Background: Current Research Context

Starting points for the workshop’s discussion reside in the presentation of what has come to be viewed as the traditional clinical research model, depicted as a pyramid in Figure 1-1 . In this model, the strongest level of evidence is displayed at the peak of the pyramid: the randomized controlled double blind study. This is often referred to as the “gold standard” of clinical research, and is followed, in a descending sequence of strength or quality, by randomized controlled studies, cohort studies, case control studies, case series, and case reports. The base of the pyramid, the weakest evidence, is reserved for undocumented experience, ideas and opinions. Noted at the workshop was the fact that, as currently practiced the randomized controlled and blinded trial is not the gold standard for every circumstance.

The classic evidence hierarchy. SOURCE: DeVoto, E., and B. S. Kramer. 2005. Evidence-Based Approach to Oncology. In Oncology an Evidence-Based Approach. Edited by A. Chang. New York: Springer. Modified and reprinted with permission of Springer SBM.

The development in recent years of a broad range of clinical research approaches, along with the identification of problems in generalizing research results to populations broader than those enrolled in tightly controlled trials, as well as the impressive advances in the potential availability of data through expanded use of electronic health records, have all prompted re-consideration of research strategies and opportunities ( Kravitz, 2004 ; Schneeweiss, 2004; Liang, 2005; Lohr, 2007 ; Rush, 2008 ).

Table 1-1 provides brief descriptions of the many approaches to clinical effectiveness research discussed during the workshop—and these methods can be generally characterized as either experimental or non-experimental. Experimental studies are those in which the choice and assignment of the intervention is under control of the investigator; and the results of a test intervention are compared to the results of an alternative approach by actively monitoring the respective experience of either individuals or groups receiving the intervention or not. Non-experimental studies are those in which either manipulation or randomization is absent, the choice of an intervention is made in the course of clinical care, and existing data, that was collected in the course of the care process, is used to draw conclusions about the relative impact of different circumstances or interventions that vary between and among identified groups, or to construct mathematical models that seek to predict the likelihood of events in the future based on variables identified in previous studies. The data used to reach study conclusions, can be characterized as primary (generated during the conduct of the study); or secondary (originally generated for other purposes, e.g., administrative or claims data).

Selected Examples of Clinical Research Study Designs for Clinical Effectiveness Research.

While not an exhaustive catalog of methods, Table 1-1 provides a sense of the range of clinical research approaches that can be used to improve understanding of clinical effectiveness. Noted at the workshop was the fact that each method has the potential to advance understanding on different aspects of the many questions that emerge throughout a product or intervention’s lifecycle in clinical practice. The issue is therefore not one of whether internal or external validity should be the overarching priority for research, but rather which approach is most appropriate to the particular need. In each case, careful attention to design and execution studies are vital.

Bridging the Research–Practice Divide

A key theme of the meeting was that it is important to draw clinical research closer to practice. Without this capacity, the need to personalize clinical care will be limited. For example, information on possible heterogeneity of treatment effects in patient populations—due to individual genetics, circumstance, or co-morbidities—is rarely available in a form that is timely, readily accessible, and applicable. To address this issue, the assessment of a healthcare intervention must go beyond determinations of efficacy (whether an intervention can work under ideal circumstances) to an understanding of effectiveness (how an intervention works in practice), which compels grounding of the assessment effort in practice records. To understand effectiveness, feedback is crucial on how well new products and interventions work in broad patient populations, including who those populations are and under what circumstances they are treated.

Redesigning the Clinical Effectiveness Research Paradigm

Growing opportunities for practice-based clinical research are presented by work to develop information systems and data repositories that enable greater learning from practice. Moreover, there is a need to develop a research approach that can address the questions that arise in the course of practice. As noted in Table 1-1 , many research methods can be used to improve understanding of clinical effectiveness, but their use must be carefully tailored to the circumstances. For example, despite the increased external validity offered by observational approaches, the uncertainty inherent in such studies due to bias and confounding often undermine confidence in these approaches. Likewise, the limitations of the randomized controlled trial (RCT) often mute its considerable research value. Those limitations may be a sample size that is too small; a drug dose that is too low to fully assess the drug’s safety; follow-up that is too short to show long-term benefits; underrepresentation or exclusion of vulnerable patient groups, including elderly patients with multiple co-morbidities, children, and young women; conduct of the trial in a highly controlled environment; and/or high cost and time investments. The issue is not one of RCTs versus non-experimental studies but one of which is most appropriate to the particular need.

Retrospective population-level cohorts using administrative data, clinical registries, and longitudinal prospective cohorts have, for example, been valuable in assessing effectiveness and useful in helping payers to make coverage decisions, assessing quality improvement opportunities, and providing more realistic assessments of interventions. Population-based registries—appropriately funded and constructed with clinician engagement—offer a compromise to the strengths and limitations of, for example, cohort studies, and can assess “real-world” health and economic outcomes to help guide decision making for patient care and policy setting. Furthermore, they are a valuable tool for assessing and driving improvements in the performance of physicians and institutions.

When trials, quasi-experimental studies, and even epidemiologic studies are not possible, researchers may also be able to use simulation methods, if current prototypes prove broadly applicable. Physiology-based models, for example, have the potential to augment knowledge gained from trials and can be used to fill in “gaps” that are difficult or impractical to answer using clinical trial methods. In particular, they will be increasingly useful to provide estimates of key biomarkers and clinical findings. When properly constructed, they replicate the results of the studies used to build them, not only at an outcome level but also at the level of change in biomarkers and clinical findings. Physiology-based modeling has been used to enhance and extend existing clinical trials, to validate RCT results, and to conduct virtual comparative effectiveness trials.

In part, this is a taxonomy and classification challenge. To strengthen these various methods, participants suggested work to define the “state of the art” for their design, conduct, reporting, and validation; improve the quality of data used; and identify strategies to take better advantage of the complementary nature of results obtained. As participants observed, these methods can enhance understanding of an intervention’s value in many dimensions—exploring effects of variation (e.g., practice setting, providers, patients) and extending assessment to long-term outcomes related to benefits, rare events, or safety risks—collectively providing a more comprehensive assessment of the trade-offs between potential risks and benefits for individual patients.

It is also an infrastructure challenge. The efficiency, quality, and reliability of research requires infrastructure improvements that allow greater data linkage and collaboration by researchers. Research networks offer a unique opportunity to begin to build an integrated, learning healthcare system. As the research community hones its capacity to collect, store, and study data, enormous untapped capacity for data analysis is emerging. Thus, the mining of large databases has become the focus of considerable interest and enthusiasm in the research community. Researchers can approach such data using clinical epidemiologic methods—potentially using data collected over many years, on millions of patients, to generate insights on real-world intervention use and health outcomes. It was this potential that set the stage for the discussion.

• PERSPECTIVES ON CLINICAL EFFECTIVENESS RESEARCH

Keynote addresses opened discussions during the 2-day workshop. Together the addresses and discussions provide a conceptual framework for many of the meeting’s complex themes. IOM President Harvey V. Fineberg provides an insightful briefing on how clinical effectiveness research has evolved over the past 2.5 centuries and offers compelling questions for the workshop to consider. Urging participants to stay focused on better understanding patient needs and to keep the fundamental values of health care in perspective, Fineberg proposes a meta-experimental strategy, advocating for experiments with experiments to better understand their respective utilities, power, and applicability as well as some key elements of a system to support patient care and research. Carolyn M. Clancy, director of the Agency for Healthcare Research and Quality, offers a vision for 21st-century health care in which actionable information is available to clinicians and patients and evidence is continually refined as care is delivered. She provides a thoughtful overview of how emerging methods will expand the research arsenal and can address many key challenges in clinical effectiveness research. Emphasis is also given to the potential gains in quality and effectiveness of care, with greater focus on how to translate research findings into practice.

• CLINICAL EFFECTIVENESS RESEARCH: PAST, PRESENT, AND FUTURE

Harvey V. Fineberg, M.D., Ph.D.

President, Institute of Medicine

An increasingly important focus of the clinical effectiveness research paradigm is the efficient development of relevant and reliable information on what works best for individual patients. A brief look at the past, present, and future of clinical effectiveness research establishes some informative touchstones on the development and evolution of the current research paradigm, as well as on how new approaches and directions might dramatically improve our ability to generate insights into what works in a clinical context.

Evolution of Clinical Effectiveness Research

Among the milestones in evidence-based medicine, one of the earliest examples of the use of comparison groups in a clinical experiment is laid out in a summary written in 1747 by James Lind detailing what works and what does not work in the treatment of scurvy. With 12 subjects, Lind tried to make a systematic comparison to discern what agents might be helpful to prevent and treat the disease. Through experimentation, he learned that the intervention that seemed to work best to help sailors recover most quickly from scurvy was the consumption of oranges, limes, and other citrus fruits. Many other interventions, including vinegar and sea water, were also tested, but only the citrus fruits demonstrated benefit. What is interesting about that experiment and relevant for our discussions of evidence-based medicine today is that it took the Royal Navy more than a century to adopt a policy to issue citrus to its sailors. When we talk about the delay between new knowledge and its application in practice in clinical medicine, we therefore have ample precedent, going back to the very beginning of systematic comparisons.

Another milestone comes in the middle of the 19th century, with the first systematic use of statistics in medicine. During the Crimean War (1853–1856), Florence Nightingale collected mortality statistics in hospitals and used those data to help discern where the problems were and what might be done to improve performance and outcomes. Nightingale’s tables were the first systematic collection in a clinical setting of extensive data on outcomes in patients that were recorded and then used for the purpose of evaluation.

It was not until the early part of the 20th century that statistics in its modern form began to take hold. The 1920s and 1930s saw the development of statistical methods and accounting for the role of chance in scientific studies. R. A. Fisher ( Fisher, 1953 ) is widely credited as one of the seminal figures in the development of statistical science. His classic work, The Design of Experiments (1935), focused on agricultural comparisons but articulated many of the critical principles in the design of controlled trials that are a hallmark of current clinical trials. It would not be until after World War II, that the first clinical trial on a medical intervention would be recorded. A 1948 study by Bradford Hill on the use of streptomycin in the treatment of tuberculosis was the original randomized controlled trial. Interestingly, the contemporary use of penicillin to treat pneumonia was never subjected to similar, rigorous testing—perhaps owing to the therapy’s dramatic benefit to patients.

Over the ensuing decades, trials began to appear in the literature with increased frequency. Along the way they also became codified and almost deified as the standard for care. In 1962, after the thalidomide scandals, efficacy was introduced as a requirement for new drug applications in the Kefauver amendments to the Federal Food, Drug, and Cosmetic Act. But it was not until the early 1970s, that a decision of the Sixth Circuit Court certified RCTs as the standard of the Food and Drug Administration (FDA) in its judgments about efficacy. Subsequently, organizations like the Cochrane Collaboration have developed ways of integrating RCT information with data from other types of methods and from multiple sources, codifying and putting these results forward for use in clinical decision making.

The classic evidence hierarchy that starts at the bottom with somebody’s opinion—one might call that “eminence-based medicine”—and rises through different methodologies to the pinnacle, randomized controlled double blind trials. If double masked trials were universal, if they were easy, if they were inexpensive, and if their results were applicable to all patient groups, we probably would not have a need for discussion on redesigning the clinical effectiveness paradigm. Unfortunately, despite the huge number of randomized trials being conducted a number of needs are not being met by the current “randomized trial only” strategy.

Effectiveness Research to Inform Clinical Decision Making

Archie Cochrane, the inspiration for the Cochrane Collaboration, posed three deceptively simple yet critical questions for assessing clinical evidence. First, Can it work? Implied in this question is an assumption of ideal conditions. More recently, the Office of Technology Assessment popularized the terms efficacy and effectiveness to distinguish between the effects of an intervention in ideal and real-world conditions, respectively. The question “Can it work?” is a question of an intervention’s efficacy. Cochrane’s second question, “Will it work?,” is one of effectiveness. That is, how and for whom does an intervention work in practice—under usual circumstances, deployed in the field, and utilized under actual clinical care conditions with real patients and providers.

The third of Cochrane’s questions asks, “Is it worth it?” This can not only be applied to the balance of safety, benefit, and risk to an individual patient, but also can be applied to the society as a whole, for which the balance of costs and effectiveness also come into play. This final question introduces important considerations with respect to assessing different approaches and strategies in clinical effectiveness—the purpose of and vantage point from which these questions are being asked. Because of the significant range and number of perspectives involved in healthcare decision making, addressing these issues introduces many additional questions to consider upstream of those posed by Cochrane.

When assessing the vast array of strategies and approaches to evaluation, these prior questions are particularly helpful to put into perspective the roles and contributions of each. Many dimensions of health care deserve to be considered in evaluation, particularly if we start with the idea of prevention and the predictive elements prior to an actual clinical experience. The scope of an evaluation might be at the level of the intervention (e.g., diagnostic, therapeutic, rehabilitative, etc.), the clinician (specialty, training, profession, etc.) or organization of service, institutional performance, and the patient’s role. Thinking critically about what is being evaluated and mapping what is appropriate, effective, and efficient, for the various types of questions, is an ongoing and important challenge in clinical effectiveness research.

Certain clinical questions drive very different design challenges. Evaluation of a diagnostic intervention, for example, involves the consideration of a panoply of factors: The performance of the diagnostic technology in terms of the usual measures of sensitivity and specificity, and the way in which one can reach judgments, make trade-offs, and deal with both false-positive and false-negative results. One also has to be thinking of the whole cascade of events, through clinical intervention and outcomes that may follow. For example, thought should be given to whether results enhance subsequent decisions for therapy, or whether the focus is on patient reassurance or other measurable outcomes, and how these decisions might affect judgments or influence the ultimate outcome of the patient. Considerations such as these are important for evaluating an intervention but are very different from those aimed at assessing system performance (e.g., clinician and health professionals’ performance, organizational approaches to service). These differences apply not only to methodologies and strategies but also to information needs. The same kinds of data used to evaluate a targeted, individual, time-specified intervention are not the same as those needed if the goal is to compare various strategies of organization or different classes of providers.

A related set of considerations revolves around this question: For whom—or for what patient group, are we attempting to make an evaluation? The limitations of randomized controlled trials, in terms of external validity, often reduce the relevance of findings for clinical decisions faced by physicians for their patients. For example, the attributes of real-world patients may differ significantly from that of the trial population (e.g., age, sex, other diagnoses, risk factors), with implications for the appropriateness of a given therapy for patients. Early consideration of “for whom” will help to identify those specific methods that can produce needed determinations, or add value by complementing information from clinical trials.

Finally, we must also consider point of view: For any given purpose, from whose point of view are we attempting to carry out this evaluation? For example, is the purpose motivated by one’s interest in a clinical benefit, safety, cost, acceptability, convenience, implementability, or some other factor? Is it representing the patient or the pool of patients, the payers of the services, the clinicians who provide the services, the manufacturers? Does the motivation come from a regulatory need for decisions about permissions or restrictions, and if so, under what circumstances? These perspectives all extend differences to what kind of method will be suitable for what kind of question in what kind of circumstance. The questions of what it is we are evaluating, for whom, and with whose perspective and purpose in mind are decidedly nontrivial, and in fact they can be important guides as we reflect on the strengths and weaknesses of a variety of approaches and formulate better approaches to clinical effectiveness research.

Several key challenges drive the need for better methods and strategies for developing clinical effectiveness information. First, it is evident to anyone who has spent an hour in a clinic or has attempted to keep up-to-date with current medical practices that the amount of information that may be relevant to a particular patient and the number of potential interventions and technologies can be simply overwhelming. In fact, it is estimated that the number of published clinical trials now exceeds 10,000 per year. A conscientious clinician trying to stay current with RCTs could read three scientific papers a day and at the end of a year would be approximately 2 years behind in reading. This constant flow of information is one side of a central paradox in heath care today: Despite the overwhelming abundance of information available, there is an acute shortage of information that is relevant, timely, appropriate, and useful to clinical decision making. This is true for caregivers, payers, regulators, providers, and patients. Reconciling those two seemingly differently vectored phenomena is one of the leading challenges today.

Additional barriers and issues include contending with the dizzying array of new and complex technologies, the high cost of trials as a primary means of information acquisition, as well as a complex matrix of ethical, legal, practical, and economic issues associated with fast-moving developments in genetic information and the rise of personalized medicine. These issues are compounded by the seemingly increased divergence of purpose between manufacturers, regulators, payers, clinicians, and patients. Finally, there is the challenge to improve how knowledge is applied in practice. In part this is related to the culture of medicine, the demands of practice, and information overload, but improvements to the system and incentives will be of critical importance moving forward.

Innovative Strategies and Approaches

Fortunately, researchers are developing many innovative strategies in response to these challenges. Some of these build on, some displace, and some complement the traditional array of strategies in the classic evidence hierarchy. Across the research community, for example, investigators are developing improved data sources—larger and potentially interoperable clinical and administrative databases, clinical registries, and electronic health records and other systems that capture relevant data at the point of care (sometimes called “the point of service”). We are seeing also the evolution of new statistical tools and techniques, such as adaptive designs, simulation and modeling, large database analyses, and data mining. Some of these are adaptations of tools used in other areas; others are truly novel and driven by the peculiar requirements of the human clinical trial. The papers that follow offer insights on adaptive designs, simulation and modeling approaches, and the various analytic approaches that can take adequate account of very large databases.

In particular, discerning meaningfully relevant information in the health context begs for closer attention, and strategies for innovative approaches to data mining are emerging—strategies analogous, if you will, to the way that Internet search engines apply some order to the vast disarray of undifferentiated information spread across the terabyte-laden database of the World Wide Web.

We are also seeing the development of innovative trial and study methodologies that aim to compensate for some of the weaknesses of clinical trials with respect to external validity. Such methods also hold promise for addressing some of the cost-related and logistical challenges in the classic formulation of trials. New approaches to accommodate physiologic and genetic information speak to the emergence of personalized medicine. At the same time, there are emerging networks of research that can amplify and accelerate, in efficiency and time, the gathering and accumulation of relevant information and a variety of models that mix and match these strategies.

Regardless of the approaches taken and the variety of perspectives considered, we must ultimately return to confront the critically central question of clinical care for the individual patient in need, at a moment in time, and determine what works best for this patient. In this regard, we might ideally move toward a system that would combine a point-of-care focus with an electronic record data system that is coupled with systems for assembling evidence in a variety of ways. Such a system would accomplish a quadrafecta, if you will—a four-part achievement of goals important to improving the system as a whole. Component goals would include enabling the evaluation and learning of what works and what does not work for individual patients that includes weighing benefits and costs; providing decision support for the specific patient in front of a clinician—identifying relevant pieces of evidence in the available information, while also contributing to the pool of potentially useable information for future patients; providing meaningful continuing education for health professionals—moving beyond the traditional lecture approach to one that enables learning in place, occurs in real-time, and is practical and applied in the course of clinical care; and finally, collectively providing a foundation for quality improvement systems . If we can achieve this kind of integration—point of service, patient-centered understanding, robust data systems on the clinical side, coupled with relevant analytic elements—we can, potentially, simultaneously, advance on the four critical goals of evaluation, decision support, continuing education, and quality improvement.

As we move in this direction, it is worth considering whether the ultimate goal is to understand what works and what doesn’t work in an “N of 1.” After all, this reflects the ultimate in individualized care. Granted, for those steeped in thinking about probability, Bayesian analysis, and decision theory, this seems a rather extreme notion. But if we consider for a moment that probability is an expression of uncertainty and ignorance, a key consideration becomes: What parts of uncertainty in health care are reducible?

Consider the classic example of the coin toss—flip a coin, the likelihood of it landing heads or tails is approximately 50/50. In such an experiment, what forces are constant and what forces are variable in determining what happens to that coin? Gravity, obviously, is a fairly reliable constant, and the coin’s size, weight, and balance can be standardized. What varies is the exact place where the coin is tossed, the exact height of the toss, the exact force with which the coin is flipped, the exact surface on which it falls, and the force that the impact imparts to the coin. Imagine, therefore, that instead of just taking a coin out of a pocket, flipping it, and letting it fall on the floor, we instead had a vacuum chamber that was precisely a given height, with a perfectly absorptive surface, and that the coin was placed in a special lever in the vacuum chamber with a special device to flip it with exactly the same force in exactly the same location every time it strikes that perfectly balanced coin. Instead of falling 50/50 heads or tails, how the coin falls will be determined almost entirely by how it is placed in the vacuum chamber and tossed by the lever.

If we apply that analogy to individual patients, the expression of uncertainty about what happens to patients and groupings of patients should be resolvable to the individual attribute at the genetic, physiologic, functional, and historical level. If resolution to the level of individual patients is not possible, an appropriate objective might be to understand an intervention’s impact in increasingly refined subgroups of patients. In point of fact, we are already seeing signs of such movement, for example, in the form of different predictive ability for patients who have particular genetic endowments.

As we consider the array of methods and develop strategies for their use in clinical effectiveness research, a guiding notion might be “a meta-experimental strategy” that aids the determination of which new methods and approaches to learning what works, and for what specific purposes, enables the assessment of several strategies, separately and in concert, and develops information on how the various methods and strategies can be deployed successfully. In other words, rather than focusing on how well a particular strategy evaluates a particular kind of problem, in a particular class of patient, from this particular point of view, with these particular endpoints in mind, we might ask what is the array of experimental methods that collectively perform in a manner that enables us to make better decisions for the individual and better decisions for society. What is the experiment of experiments? And how could we structure our future learning opportunity so that as we are learning what works for a particular kind of patient, we are also learning the way in which that strategy of evaluation can be employed to achieve a health system that is driven by evidence and based on value.

• THE PATH TO RESEARCH THAT MEETS EVIDENCE NEEDS

Carolyn M. Clancy, M.D.

Agency for Healthcare Research and Quality

All of us share a common passion for developing better evidence so we can improve health care, improve the value of health care, and provide clinicians, patients, and other relevant parties with better information to make health decisions. In that context, this paper explores a central question: How can our approach to clinical effectiveness research take better advantage of emerging tools and study designs to address such challenges as generalizability, heterogeneity of treatment effects, multiple co-morbidities, and translating evidence into practice? This paper focuses on emerging methods in effectiveness research and approaches to turning evidence into action and concludes with some thoughts about health care in the 21st century.

Emerging Methods in Effectiveness Research

Early in the development of evidence-based medicine, discussions of methods were rarely linked to translating evidence into everyday clinical practice. Today, however, that principle is front and center, in part because of the urgency that so many of us feel about bringing evidence more squarely into healthcare delivery. Evidence is a tool for making better decisions, and health care is a process of ongoing decision making. The National Business Group on Health recently issued a survey suggesting that a growing number of consumers—primarily people on the healthier end of the spectrum, but also including some 11 percent of individuals with a chronic illness—say that they turn to sources other than their doctors for information. Some, for example, are even going to YouTube—not yet an outlet for our work, but who knows what the future might bring? If there is a lesson there, it is that evidence that we are developing has to be valid, broadly available, and relevant.

Traditional Hierarchies of Evidence: Randomized Controlled Trials

Traditional hierarchies of evidence have by definition placed the randomized controlled trial (RCT) at the top of the pyramid of evidence, regardless of the skill and precision with which an RCT is conducted. Such hierarchies, however, are inadequate for the variety of today’s decisions in health care and are increasingly being challenged. RCTs obviously remain a very strong tool in our research armamentarium. Yet, we need a much more rigorous look at their appropriate role—as well as more scrutiny of the role of nonrandomized or quasi-experimental evidence. As we talk about the production of better evidence, we must keep the demand side squarely in our sights. Clearly a path to the future lies ultimately in the production of evidence that can be readily embedded in the delivery of care itself—a “Learning Healthcare Organization”—a Holy Grail that is not yet a reality.

The ultimate questions are fairly straightforward for a particular intervention, product, tool, or test: Can it work? Will it work—for this patient, in this setting? Is it worth it? Do the benefits outweigh any harms? Do the benefits justify the costs? Does it offer important advantages over existing alternatives? The last question in particular can be deceptively tricky. In some respects, the discussion is really a discussion of value. Given the increases in health expenditures over the past few years alone, the issue of value cannot be dismissed.

Clinicians know, of course, that clearly what is right for one person does not necessarily work for the next person. The balance of benefits versus harms is influenced by baseline risk, patient preferences, and a variety of other factors. The reality of medicine today is that for many treatment decisions, two or more options are available. What is not so clear is determining with the patient what the right choice is, in a given case, for that particular person. (As an aside, we probably do not give individual patients as much support as we should when they choose to take a path that differs from what we recommend.)

Even the most rigorously designed randomized trial has limitations, as we are all acutely aware. Knowing that an intervention can work is necessary but not sufficient for making a treatment decision for an individual patient or to promote it for a broad population. Additional types of research can clearly shed light on such critical questions as who is most likely to benefit from a treatment and what the important trade-offs are.

Nonrandomized Studies: An Important Complement to RCTs

Nonrandomized studies will never entirely supplant the need for rigorously conducted trials, but they can be a very important complement to RCTs. They can help us examine whether trial results are replicable in community settings; explore sources of differences in safety or effectiveness arising from variation among patients, clinicians, and settings; and produce a more complete picture of the potential benefits and harms of a clinical decision for individual patients or health systems. In short, nonrandomized studies can enrich our understanding of how patient treatments in practice differ from those in trials. A good case in point are two studies published in the 1980s by the Lipid Research Clinics showing that treatment to lower cholesterol can reduce the risk of coronary heart disease in middle-aged men ( The Lipid Research Clinics Coronary Primary Prevention Trial Results. I. Reduction in Incidence of Coronary Heart Disease, 1984 ; The Lipid Research Clinics Coronary Primary Prevention Trial Results. II. The Relationship of Reduction in Incidence of Coronary Heart Disease to Cholesterol Lowering, 1984 ).

The researchers gave us an invaluable look at middle-aged, mostly white men who had one risk factor for coronary artery disease and were stunningly compliant with very unpleasant medicines, such as cholestyramine. The field had never had such specific data before, and, although the study was informative, patients rarely reflect the study population, at least in my experience as most come with additional risk factors and infrequently adhere to unpleasant medicines.

My colleague David Atkins recently alluded to an important nuance of RCTs that bears mention here. He observed that “trials often provide the strong strands that create the central structure, but the strength of the completed web relies on a variety of supporting cross strands made up of evidence from a more diverse array of studies”( Atkins, 2007 ). In other words, if clinical trials are the main strands in a web of evidence, it is important to remember that they are not the entire web.

For example, recent Agency for Healthcare Research and Quality (AHRQ) reports that have relied exclusively on nonrandomized evidence include one on total knee replacement and one on the value of islet cell transplantation. The latter study found that 50–90 percent of those who had the procedure achieved insulin independence, but it raised questions about the duration of effect. Another study on bariatric surgery found the surgery resulted in a 20–30 kilogram weight loss, versus a 2–3 kilogram loss via medicine, but raised questions about safety. Available nonrandomized studies are adequate to demonstrate “it can work,” but may not be able to answer the question, “Is it worth it?”

Nonrandomized studies complement clinical trials in several ways. Because most trials are fairly expensive, requiring development and implementation of a fairly elaborate infrastructure, their duration is more likely to be short term rather than long term. One finding from the initial Evidence Report on Treatment of Depression—New Pharmacotherapies , which compared older with newer antidepressants, was that at that time the vast majority of studies followed patients for no longer than 3 months ( Agency for Healthcare Policy and Research, 2008d ). This situation has changed since then thanks to investments made by the National Institute of Mental Health.

Nonrandomized trials also help researchers to pursue the similarities and differences between the trial population and the typical target population, and between trial intervention and typical interventions. Nonrandomized trials enable researchers to examine the heterogeneity of treatment effects in a patient population that in some ways or for some components may not look very much like the trial population. This, in turn, may create a capacity to modify the recommendations as they are implemented. Finally, nonrandomized trials enable researchers to study harm and safety issues in less selective patient populations and subgroups.

One example is the National Emphysema Treatment Trial, the first multicenter clinical trial designed to determine the role, safety, and effectiveness of bilateral lung volume reduction surgery (LVRS) in the treatment of emphysema. An AHRQ technology assessment of LVRS concluded that the data on the risks and benefits of LVRS were too inconclusive to justify unrestricted Medicare reimbursement for the surgery. However, the study also found that some patients benefited from the procedure. This prompted the recommendation that a trial evaluating the effectiveness of the surgery be conducted. The National Emphysema Treatment Trial followed to evaluate the safety and effectiveness of the best available medical treatment alone and in conjunction with LVRS. A number of interesting occurrences transpired within this study. First, people were not randomized to medicine versus surgery—all patients went through a course of maximum medical therapy or state-of-the-art pulmonary rehabilitation, after which they were randomized to continue rehab or to be enrolled in the surgical part of the trial.

With many patients, this was their first experience with very aggressive pulmonary rehab. Because many felt very good after the rehab, at the end of the first course of treatment many patients pulled out of the study. That extended the time it took for study enrollment. Thereafter, some of the study’s basic findings were unexpected. Two or 3 years into the study, a paper in the New England Journal of Medicine effectively identified a high-risk subgroup whose mortality was higher after surgery. The talk of increased mortality further impeded study enrollment rates, which of course further delayed the results. Today, the number of LVRS procedures is very low. The reason for the decline is an open question—perhaps it is a matter of patients’ perspectives or perhaps the trial took so long that one could argue that it was almost anti-innovation. Regardless, the example illustrates both that trials can take a long time and that they may not provide the magic bullet for making specific decisions.

The Challenge of Heterogeneity of Treatment Effects

In terms of the approval of products, clinical trials may not always represent the relevant patient population, setting, intervention, or comparison. Efficacy trials may exaggerate typical benefits, minimize typical harms, or overestimate net benefits. Clearly, external validity becomes a problem or a challenge, and that dilemma has been the subject of many lively debates. As noted by Nicholas Longford, “clinical trials are good experiments but poor surveys.” In a paper a few years ago in the Milbank Quarterly , Richard Kravitz and colleagues suggested that the distribution of specific aspects and treatment effects in any particular trial for approval could result in a very different sense of the expected treatment effect in broader populations ( Kravitz et al., 2004 ).

Discussing the difficulties of applying global evidence to individual patients or groups that might depart from the population average, the paper argues that clinical trials “can be misleading and fail to reveal the potentially complex mixture of substantial benefits for some, little benefit for many, and harm for a few. Heterogeneity of treatment effects reflects patient diversity in risk of disease, responsiveness to treatment, vulnerability to adverse effects, and utility for different outcomes.” By recognizing these factors, the paper suggests, researchers can design studies that better characterize who will benefit from medical treatments, and clinicians and policy makers can make better use of the results.

A relevant area of study that has received a great deal of attention in this regard has been the use of carotid artery surgery and endarterectomy. Figure 1-2 shows 30-day mortality in older patients who had undergone an endarterectomy. The vertical values show what happened for Medicare patients in trial hospitals as well as hospitals with high, low, and medium patient volume. The lower horizontal line shows, by contrast, mortality in the Asymptomatic Carotid Artery Surgery (ACAS) trial; the line above it represents mortality in the North American Symptomatic Carotid Endarterectomy Trial (NASCET). Clearly, generalizing the results observed in the trials to the community would have been mistaken in terms of mortality rates.

Thirty-day mortality in older patients undergoing endarterectomy (versus trials and by annual volume). SOURCE: Derived from McGrath et al., 2000.

The limitations of approval trials for individual decision making are well known, such as the previously mentioned LRC trial. In point of fact, a trial may represent neither the specific setting and intervention nor the individual patient. Issues of applicability and external validity really come into focus, because essentially what we are reporting from trials is the average effect, and for an individual that may or may not be specifically relevant. In addition, of course, the heterogeneity of treatment effects seen in a trial becomes very important. One can come to very different conclusions depending on the distribution of the net treatment benefit and how narrow that distribution is (see Figure 1-3 , for example), and yet we don’t often know that with the first trial that has been done for approval.

Treatment benefit distribution of different sample population subgroups for a clinical trial. NOTE: Reprinted, with permission, from John Wiley & Sons, Inc., 2008. SOURCE: Kravitz et al., 2004.

An ongoing challenge and debate is the extent to which we can count on subgroup analysis to gain a more complete picture and information that is more relevant to a heterogeneous treatment population. In terms of such analyses, we need to be cautious in individual trials. Subgroup analyses are not reported regularly enough for individual patient meta-analyses. Moreover, we need to look beyond RCTs to inform judgments about the applicability and heterogeneity of treatment effects. Several years ago, AHRQ was asked to produce a systematic review of what was known about the effectiveness of diagnosis and treatment for women with coronary artery disease. A critical body of evidence existed, because it was a requirement of federal trials that women and minorities be enrolled in all clinical studies. We found, however, that it was extremely difficult to combine results across those studies, which underscores how difficult it is to combine data across trials as a basis for meta-analysis or any other quantitative technique.

Practical Clinical Trials: Embedding Research into Care Delivery

Another interesting set of considerations revolves around the differences between practical (or pragmatic) clinical trials (PCTs) versus explanatory clinical trials and how PCTs might move the field closer to the notion of embedding research into care delivery and contending directly with issues confronting clinicians. In PCTs, hypotheses and study design are formulated based on information needed to make a clinical decision; explanatory trials are designed to better understand how and why an intervention works. Also, while practical trials address risks, benefits, and costs of an intervention as they would occur in routine clinical practice, explanatory trials maximize the chance that the biological effect of a new treatment will be revealed by the study ( Tunis et al., 2003 ).

Another example comes from a systematic review of treatments for allergic rhinitis and sinusitis. For patients with either diagnosis there is a considerable body of information. The real challenge, though, is that in primary care settings most patients are not really interested in the kinds of procedures that one needs to make a definitive diagnosis. Thus, clinicians and patients typically do not have a lot of practical information to work with, even with data from the systematic study reviews. Practical clinical trials would address risks, benefits, and causes of interventions as they occur in routine clinical practice rather than trying to determine whether, for example, a particular mechanism actually provides a definitive diagnosis. Comparisons of practical versus explanatory trials date back to the 1960s, perhaps not surprisingly, in the context of the value of various types of psychotherapy. Since then, however, the literature seems to have gone silent on such issues. It would be a useful activity to revisit these questions and actually develop an inventory of what we know about practical clinical trials and how difficult they can be.

Different study designs contribute different effects. Looking at the variety of study designs, we can move down a continuum from efficacy trials, the least biased estimate of effect under ideal conditions, where we have maximum internal validity; to effectiveness trials, which provide a more representative estimate of the benefits and harms in the real world and presumably have increased external validity; to systematic review of trials that have used the same end-points, outcomes meta-analysis, or other quantitative techniques. The latter types of trials provide the best estimate of overall effect, investigate questions of heterogeneity of treatment effect, and explore uncommon outcomes.

Collaborative Registries

We can also use cohort studies registries, which use risk prediction to target treatment and are effective in reaching underrepresented populations, and case control studies, which can be particularly helpful in detecting relatively rare harms or adverse events that were not known or not expected. There is a substantial, growing interest among a number of physician specialties in creating registries. Registries provide a way to explore longer term outcomes, adverse effects, and practice variations, and they provide an avenue for the investigation of the heterogeneity of a treatment effect. The Society for Thoracic Surgeons is probably the best (and most familiar) model, but other surgical societies also are currently developing registries of their own. It will be a very interesting challenge to determine how, when, and where registries fit into the overall context of study designs, and how information from registries fits within the broader web of evidence-based medicine. AHRQ is using patient registry data as one approach to turning evidence into action, which is discussed in the next section.

Indeed, in the world of evidence-based medicine, two of the key challenges are the translation of evidence into everyday clinical practice, and how to manage situations when existing evidence is insufficient. Virtually every day, AHRQ receives telephone calls that focus on the fact that lack of evidence of effect is not the same thing as saying that a treatment is ineffective. Similarly, what is equivalent for the group is not necessarily equivalent for the individual. Measuring and weighing outcomes such as quality of life and convenience is obviously not a feature of most standardized clinical trials. Moreover, in considerations of such issues, we need to keep in mind the downstream effects of policy applications, such as diffusion of technology, effects on innovation, and unintended consequences. For example, one of the reasons that RCTs are such a poor fit for evaluating devices or any evolving technology is that the devices change all the time. One of the challenges that AHRQ hears regularly is that we invested all of this money and it took years to complete this trial, and we answered a question that is no longer relevant.

Observational Studies

This leads to the question of whether observational studies can reduce the need for randomized trials. Clearly observational studies are a preferred alternative when clinical trials are impractical, not possible, or unethical (we seem to debate that question a lot even though we are not particularly clear about what it means). Last year, a paper defended the value of observational studies. The authors asserted that observational studies can be a very useful tool when one examines something with a stable or predictable background course that is associated with a large, consistent, temporal effect, with rate ratios over 10, and where one can demonstrate dose response, specific effects, and biological plausibility ( Glasziou et al., 2007 ). Observational studies are also of value when the potential for bias is low.

The advantages of observational studies are worth considering. First, most clinical trials are not sufficiently powered to detect adverse drug effects. AHRQ felt the effect of this design weakness when a report it sponsored on the benefits and harms of a variety of antidepressants was published. The report concluded that there was insufficient useful information to say anything definitive about the comparative risk profiles of newer and older antidepressants. The fact that the report had nothing to say about side effects drew a flood of protests from passionate constituents. Among other drawbacks, clinical trials clearly are limited by poor external validity in many situations; they may not be generalized to many subgroups that are of great interest to clinicians. On the other hand, longer term follow-up, not the rule for clinical trials, can be a strong advantage of observational studies. Moreover, observational studies clearly facilitate risk management and risk minimization and may indeed facilitate translation of evidence into practice. There are those who argue that all clinical trials should have an observational component, and in fact the field is starting to see some of that in trials. Comparing surgery to medical treatments for low back pain, for example, the Spine Patient Outcomes Research Trial (SPORT) randomized people if their preferences were neutral. Both patients and clinicians had to be neutral about the value of medicine versus surgery, and those people who were not neutral were followed in a registry.

Grading Evidence and Recommendations

A notable and exciting development is the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) collaborative, whose goal is to promote a more consistent and transparent approach to grading evidence and recommendations. 1 This approach considers that how well a study is done is at least as important as the type of study it is. GRADE evidence levels, as summarized in Figure 1-4 , suggest that randomized trials that are flawed in their execution should not be at the top of the pyramid in any hierarchy of evidence. Similarly, observational studies that meet the criteria shown in the figure (and perhaps others as well), and which are done very well, might in some instances be considered better evidence than a randomized trial, if a randomized trial is poorly done. These standards are being adopted by the American Colleges of Physicians, American College of Chest Physicians, National Institute for Clinical Excellence, and World Health Organization, among others.

Evidence levels—Grades of Recommendation, Assessment, Development and Evaluation (GRADE).

All of us imagine a near-term future where there is going to be much greater access to high-quality data. However, in order to take full advantage of that, we need to continue to advance work in improving methodological research. Why is this necessary? We need more comprehensive data to guide Medicare coverage decisions and to understand the wider range of outcomes. We need to address the gap when data from results of well-designed RCTs are either not available or incomplete. Finally, there are significant quality, eligibility, and cost implications of coverage decisions (e.g., consider implantable cardioverter defibrillators).

To help advance the agenda for improving methodology, a series of 23 articles on emerging methods in comparative effectiveness and safety were published in October 2007 in a special supplement to the journal Medical Care . These papers are a valuable new resource for scientists who are committed to advancing the comparative effectiveness and safety research, and this is an area in which AHRQ intends to continue to push. 2

Approaches to Turning Evidence Into Action

The Agency for Healthcare Research and Quality has several programs directed at turning evidence into action. AHRQ’s program on comparative effectiveness was authorized by Congress as part of the Medicare Modernization Act and funded through an appropriation starting in 2005. This Effective Health Care Program (EHCP) is essentially trying to produce evidence for a variety of audiences, based on unbiased information, so that people can make head-to-head comparisons as they endeavor to understand which interventions add value, which offer minimal benefit above current choices, which fail to reached their potential, and which work for some patients but not for others. The overarching goal is to develop and disseminate better evidence about benefits and risks of alternative treatments, which is also important for policy discussions. The statute is silent on cost effectiveness, although it does say that the Medicare program may not use the information to deny coverage. Less clear is whether prescription drug plans can use EHCP information in such a way; again, the statute is silent.

The AHRQ EHCP has three core components. One is synthesizing existing evidence through Evidence-Based Practice Centers (EPCs), which AHRQ has supported since 1997. The purpose is to systematically review, synthesize, and compare existing evidence on treatment effectiveness, and to identify relevant knowledge gaps. (Anyone who has ever conducted a systematic or even casual review knows that if you are searching through a pile of studies, inevitably you will have unanswered questions—questions that are related to but not quite the main focus of the particular search that you are doing.)

The second component is to generate evidence—to develop new scientific knowledge to address knowledge gaps—and to accelerate practical studies. To address critical unanswered questions or to close particular research gaps, AHRQ relies on the DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) network, a group of research partners who work under task-order contracts and who have access to large electronic clinical databases of patient information. The Centers for Education & Research on Therapeutics (CERTs) is a peer-reviewed program that conducts state-of-the-art research to increase awareness of new uses of drugs, biological products, and devices; to improve the effective use of drugs, biological products, and devices; to identify risks of new uses; and to identify risks of combinations of drugs and biological products.

Finally, AHRQ also works to advance the communication of evidence and its translation into care improvements. Many researchers will recall that our colleague John Eisenberg always talked about telling the story of health services research. Named in his honor, the John M. Eisenberg Clinical Decisions and Communications Science Center, based at Oregon Health Sciences University, is devoted to developing tools to help consumers, clinicians, and policy makers make decisions about health care. The Eisenberg Center translates knowledge about effective health care into summaries that use plain, easy-to-understand, and actionable language, which can be used to assess treatments, medications, and technologies. The guides are designed to help people to use scientific information to maximize the benefits of health care, minimize harm, and optimize the use of healthcare resources. Center activities also focus on decision support and other approaches to getting information to the point of care for clinicians, as well as on making information relevant and useful to patients and consumers.

The Eisenberg Center is developing two new translational guides, the Guide to Comparative Effectiveness Reviews and Effectiveness and Off-Label Use of Recombinant Factor VIIa . In April 2007, AHRQ also published Registries for Evaluating Patient Outcomes: A User’s Guid e, co-funded by AHRQ and the Centers for Medicare & Medicaid Services (CMS), the first government-supported handbook for establishing, managing, and analyzing patient registries. This resource is designed so that patient registry data can be used to evaluate the real-life impact of healthcare treatments and can truly be considered a milestone in growing efforts to better understand what treatments actually work best and for whom ( Agency for Healthcare Research and Quality, 2008c ).

Clearly, there are a variety of problems that no healthcare system is large enough or has sufficient data to address on its own. Many researchers envision creation of a common research infrastructure, a federated network prototype that would support the secure analyses of electronic information across multiple organizations to study risks, effects, and outcomes of various medical therapies. This would not be a centralized database—data would stay with individual organizations. However, through the use of common research definitions and terms, the collaborative would create a large network that would expand capabilities far beyond the capacity of any one individual system.

The long-term goal is a coordinated partnership of multiple research networks that provide information that can be quickly queried and analyzed for conducting comparative effectiveness research. There are enormous opportunities here, but to come to fruition the effort will take considerable difficult work upfront. In that regard, AHRQ has funded contracts to support two important models of distributed research networks. One model being evaluated leverages partnerships of a practice-based research network to study utilization and outcomes of diabetes treatment in ambulatory care. This project is led by investigators from the University of Colorado DEcIDE center and the American Academy of Family Physicians to develop the Distributed Ambulatory Research in Therapeutics Network (DARTNet), using electronic health record data from 8 organizations representing more than 200 clinicians and over 350,000 patients ( Agency for Healthcare Research and Quality, 2008a ). The second model is established within a consortium of managed care organizations to study therapies for hypertension. This project is led by the HMO Research Network (HMORN) and the University of Pennsylvania DEcIDE centers ( Agency for Healthcare Research and Quality, 2008a ). It will develop a “Virtual Data Warehouse” to assess the effectiveness and safety of different anti-hypertensive medications used by 5.5 to 6 million individuals cared for by six health plans.

Both projects will be conducted in four phases over a period of approximately 18 months, with quarterly reports posted on AHRQ’s website. These reports will describe the design specifications for each network prototype; the evaluation of the prototype; research findings from the hypertension and diabetes studies; and the major features of each prototype in the format of a prospectus or blueprint so that the model may be replicated and publicly evaluated.

In addition to the AHRQ efforts, others are also supporting activities in this arena. Under the leadership of Mark McClellan, the Quality Alliance Steering Committee at the Engelberg Center for Health Care Reform at the Brookings Institution is engaged in work to effectively aggregate data across multiple health insurance plans for the purposes of reporting on physician performance. Effectively the plans will each be producing information on a particular physician, and its weighted average will be computed and added to the same information derived from using Medicare data. The strategy is that data would stay with individual plans, but would be accessed using a common algorithm. As recent efforts to aggregate data for the purposes of quality measurement across plans have found, this is truly difficult but important work.

Among other efforts, the nonprofit eHealth Initiative Foundation has started a research program designed to improve drug safety for patients. The eHI Connecting Communities for Drug Safety Collaboration is a public- and private-sector effort designed to test new approaches and to develop replicable tools for assessing both the risks and the benefits of new drug treatments through the use of health information technology. Results will be placed in the public domain to accelerate the timeliness and effectiveness of drug safety efforts. Another important ongoing effort is the Food and Drug Administration’s work to link private- and public-sector postmarket safety efforts to create a virtual, integrated, electronic “Sentinel Network.” Such a network would integrate existing and planned efforts to collect, analyze, and disseminate medical product safety information to healthcare practitioners and patients at the point of care. These efforts underscore the commitment by many in the research community to creating better data and linking those data with better methods to translate them into more effective health care.

Health Care in the 21st Century

We must make sure that we do not lose sight of the importance of translating evidence into practice. For all of our excitement about current and anticipated breakthroughs leading to a world of personalized health care in the next decade, probably larger gain in terms of saving lives and reducing morbidity is likely to come from more effective translation. Researcher Steven Woolf and colleagues published interesting observations on this topic in 2005 ( Figure 1-5 ) ( Woolf and Johnson, 2005 ). They showed that if 100,000 patients are destined to die from a disease, a drug that reduces death rates by 20 percent will save 16,000 lives if delivered to 80 percent of the patients; increase the drug delivery to 100 percent of patients and you save an additional 4,000 lives. To compensate for that in improved efficacy you would have to have something that is 25 percent more efficacious. Thus, in the next decade, translation of the scientific evidence we already have is likely to have a much bigger impact on health outcomes than breakthroughs coming on the horizon.

Potential lives saved through quality improvement—The “break-even point” for a drug that reduces mortality by 20 percent. SOURCE: Woolf, S. H., and R. E. Johnson. 2005. The break-even point: When medical advances are less important (more...)

The clinical research enterprise has talked a lot about phase 1 and 2 translation research (T1 and T2). Yet, we need to think about T3: the “how” of high-quality care. We need to transcend thinking about translation as an example of efficacy and think instead about translation as encompassing measurements and accountability, system redesign, scaling and spread, learning networks, and implementation and research beyond the academic center ( Dougherty and Conway, 2008 ). Figure 1-6 outlines the three translational steps that form the 3T’s road map for transforming the healthcare system. Figure 1-7 suggests a progression for the evolution of translational research. This area is clearly still under development and in need of more focused attention from researchers

The 3T’s roadmap. NOTE: T indicates translation. T1, T2, and T3 represent the three major translational steps in the proposed framework to trans-form the healthcare system. The activities in each translational step test the discoveries of prior (more...)

Evolution of translational research.

In closing, we can no doubt all agree that the kind of healthcare system we would want to provide our own care would be information rich but patient focused, in which information and evidence transform interactions from the reactive to the proactive (benefits and harms). Figure 1-8 summarizes a vision for 21st-century health care. In this ideal system, actionable information would be available—to clinicians and patients—”just in time,” and evidence would be continually refined as a by-product of health-care delivery. The goal is not producing better evidence for its own sake, although the challenges and debates about how to do that are sufficiently invigorating on their own that we can almost forget what the real goals are. Achieving an information-rich, patient-focused system is the challenge that is at the core of our work together in the Value & Science-Driven Health Care Roundtable. Where we are ultimately headed, of course, is to establish the notion, discussed widely over the past several years, of a learning healthcare system. This is a system in which evidence is generated as a byproduct of providing care and actually fed back to those who are providing care, so that we become more skilled and smarter over time.

Model for 21st-century health care.

• AcademyHealth. Health Services Research (HSR) Methods. 2008. [accessed June 21, 2010]. [cited June 15, 2008]. http://www ​.hsrmethods.org/
• Agency for Healthcare Research and Quality. Developing a Distributed Research Network to Conduct Population-based Studies and Safety Surveillance 2009. 2008a. [accessed June 21, 2010]. [cited June 15, 2008]. http: ​//effectivehealthcare ​.ahrq.gov/index ​.cfm/search-for-guides-reviews-and-reports ​/?pageaction=displayproduct&productID=150 .
• Agency for Healthcare Research and Quality. Distributed Network for Ambulatory Research in Therapeutics. 2008b. [accessed June 21, 2010]. [cited June 15, 2008]. http: ​//effectivehealthcare ​.ahrq.gov/index ​.cfm/search-for-guides-reviews-and-reports ​/?pageaction=displayproduct&productID=317 .
• Agency for Healthcare Research and Quality. Effective Health Care Home. 2008c. [accessed June 21, 2010]. [cited June 15, 2008] http: ​//effectivehealthcare.ahrq.gov .
• Agency for Healthcare Research and Quality. Evidence Report on Treatment of Depression-New Pharmacotherapies: Summary. 2008d. [accessed June 21, 2010]. (Pub. No. 99-E013) [cited June 15, 2008]. http://archive ​.ahrq.gov ​/clinic/epcsums/deprsumm.htm .
• Atkins D. Creating and synthesizing evidence with decision makers in mind: Integrating evidence from clinical trials and other study designs. Medical Care. 2007; 45 (10 Supl 2):S16–S22. [ PubMed : 17909376 ]
• DeVoto E, Kramer BS. Evidence-based approach to oncology; Oncology: An Evidence-Based Approach. Chang AE, Ganz PA, Hayes DF, Kinsella T, Pass HI, Schiller JH, Stone RM, Strecher V, editors. New York: Springer; 2006.
• Dougherty D, Conway PH. The “3T’s” road map to transform US health care: the “how” of high-quality care. Journal of the American Medical Association. 2008; 299 (19):2319–2321. [ PubMed : 18492974 ]
• Fisher RA. The Design of Experiments. London: Oliver and Boyd; 1953.
• Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. British Medical Journal. 2007; 334 (7589):349–351. [ PMC free article : PMC1800999 ] [ PubMed : 17303884 ]
• Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Quarterly. 2004; 82 (4):661–687. [ PMC free article : PMC2690188 ] [ PubMed : 15595946 ]
• Liang L. The gap between evidence and practice. Health Affairs. 2007 March/April; 26 (2):w119–w121. [ PubMed : 17259192 ]
• The Lipid Research Clinics Coronary Primary Prevention Trial Results. I. Reduction in incidence of coronary heart disease. Journal of the American Medical Association. 1984; 251 (3):351–364. [ PubMed : 6361299 ]
• The Lipid Research Clinics Coronary Primary Prevention Trial Results. II. The relationship of reduction in incidence of coronary heart disease to cholesterol lowering. Journal of the American Medical Association. 1984; 251 (3):365–374. [ PubMed : 6361300 ]
• Lohr KN. Emerging methods in comparative effectiveness and safety: Symposium overview and summary. Medical Care. 2007; 45 (10):55–58. [ PubMed : 17909383 ]
• McGrath PD, Wennberg DE, Dickens JD Jr, Siewers AE, Lucas FL, Malenka DJ, Kellett MA Jr, Ryan TJ Jr. Relation between operator and hospital volume and outcomes following percutaneous coronary interventions in the era of the coronary stent. Journal of the American Medical Association. 2000; 284 (24):3139–3144. [ PubMed : 11135777 ]
• Rush AJ. Developing the evidence for evidence-based practice. Canadian Medical Association Journal. 2008; 178 :1313–1315. [ PMC free article : PMC2335183 ] [ PubMed : 18458263 ]
• Sackett DL, Brian Haynes R, Guyatt GH, Tugwell P. Dealing with the media. Journal of Clinical Epidemiology. 2006; 59 (9):907–913. [ PubMed : 16895812 ]
• Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clinical Pharmacology and Therapeutics. 2007; 82 (2):143–156. [ PMC free article : PMC2905665 ] [ PubMed : 17554243 ]
• Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. Journal of the American Medical Association. 2003; 290 (12):1624–1632. [ PubMed : 14506122 ]
• Woolf SH, Johnson RE. The break-even point: When medical advances are less important than improving the fidelity with which they are delivered. Annals of Family Medicine. 2005; 3 (6):545–552. [ PMC free article : PMC1466946 ] [ PubMed : 16338919 ]

See www ​.gradeworkinggroup.org .

All of the articles are available for free download at the website www ​.effectivehealthcare ​.ahrq.gov/reports/med-care-report.cfm or can be ordered as Pub. No. OM07-0085 from AHRQ’s clearinghouse.

• Cite this Page Institute of Medicine (US) Roundtable on Value & Science-Driven Health Care; Olsen LA, McGinnis JM, editors. Redesigning the Clinical Effectiveness Research Paradigm: Innovation and Practice-Based Approaches: Workshop Summary. Washington (DC): National Academies Press (US); 2010. 1, Evidence Development for Healthcare Decisions: Improving Timeliness, Reliability, and Efficiency.
• PDF version of this title (6.2M)

In this Page

Related information.

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Evidence Development for Healthcare Decisions: Improving Timeliness, Reliability... Evidence Development for Healthcare Decisions: Improving Timeliness, Reliability, and Efficiency - Redesigning the Clinical Effectiveness Research Paradigm

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Pregnancy and Early Childhood

• Drug use during pregnancy can affect the health of a pregnant person and their child. For example, a pregnant person’s use or misuse of opioids can cause a newborn infant to experience withdrawal symptoms, a condition known as neonatal opioid withdrawal syndrome. Overdose deaths are also rising among women during and after pregnancy.
• Treatment for a substance use disorder during pregnancy such as behavioral interventions and medication for opioid use disorder reduces health risks, including preterm delivery and low birth weight. Treatment also helps people with substance use disorders stay employed, take care of their children, and engage with their families and communities. However, pregnant people with substance use disorders often face challenges when seeking treatment, including fear, stigma and access to care.
• NIDA plays a leading role in the HEALthy Brain and Child Development (HBCD) Study , which seeks to better understand how drug use during pregnancy interacts with genetics and other biological influences to affect a child’s mental and physical health over time.

The HEALthy Brain and Child Development Study

The study explores how parental use of opioids and other environmental factors affect a child’s brain and development

Latest from NIDA

More than 321,000 U.S. children lost a parent to drug overdose from 2011 to 2021

Overdose deaths increased in pregnant and postpartum women from early 2018 to late 2021

Innovative projects answer NIDA’s challenge to implement substance use prevention in primary care

Find more information about pregnancy, early childhood and substance use.

• Learn more about medications during pregnancy at the Centers for Disease Control and Prevention website.
• Read about alcohol use during pregnancy from the National Institute on Alcohol Abuse and Alcoholism.
• Read about the NIH’s Helping to End Addiction Long-term (HEAL) Initiative .
• For information on exposure to drugs and chemicals while breastfeeding, see the National Library of Medicine’s Drugs and Lactation Database (LactMed) .