research centre case study

Sustainable Buildings Research Centre by Cox Architecture

Cox Architecture was established in Sydney in 1962. From the beginning, it can be seen that their priority is the environment, creating sustainable buildings, and spreading the ideas. Their understanding of sustainable buildings showed itself with these approaches; depth knowledge about the site and its surroundings, passive design , and use of long-lasting locally sourced materials . Sustainable Buildings Research Centre (SBRC) is one of the many projects they have completed with this mission. The Sustainable Buildings Research Centre is an evidence-based prototype of a sustainable building that also serves as a research centre for itself.  

University of Wollongong’s Sustainable Buildings Research Centre (SBRC)

Architect: Cox Architecture Year: 2014 Location: Wollongong, Australia Area: 2600 m2

The Sustainable Buildings Research Centre (SBRC) is a multidisciplinary organisation that brings together a wide range of researchers to address the challenges of making our buildings sustainable and effective places where we live and work. The mission of the SBRC is to support the rapid decarbonization of our built environment.

It was decided that the SBRC building, as the home of a sustainable buildings research centre, would go far beyond current Australian sustainability standards for educational and other facilities. The vision was that in a few years or decades, the occupants, visitors, and the wider community would be able to look back and know that the design of the SBRC had pushed the boundaries of what was possible at the time of its design and that this project represented a significant step forward in the delivery of highly sustainable buildings. With these aims, the Cox Architecture team collaborated with UoW and Living Building Challenge to create a structure worthy of these mottos. 

Knowledge About Site 

Cox Architecture’s primary approach to sustainable building starts with getting deep knowledge about the site. The selected site for the SBRC building was at the University of Wollongong’s Innovation Campus, which consisted of low-grade fill from surrounding mining operations in the region and was generally unproductive, with landscape cover primarily composed of native and exotic weeds. Even though this background can set a limit for growth, Cox Architecture’s lead set urban agricultural areas for production and student experience. The plant palette for the urban agriculture areas is decided from the local sources, what indigenous people grow in their gardens for food, medicine, etc. 

The SBRC gardens consist of 5 raised beds made from recycled bricks, five raised beds made from corrugated metal barrels, a vegetable bed on the green roof , and basic planting of fruit trees/vines and herbs in the garden. 

Passive Design

One of the approaches for sustainability that Cox Architecture has carried out is designing passive systems to minimise active use of energy and other sources. Passive design is a key element for sustainable buildings. For SBRC, Cox Architecture has taught about water management and net zero energy consumption and production.  

Water Management

The first decision for water management in SBRC is minimising water use. Most of the facility’s water needs to be provided with the captured rainwater. The grey water captured from the rooftops of the buildings is being collected in properly sized tanks. Collected rainwater goes through filtration without chemicals and is distributed to the showers, PV washings, hand basins, laboratories, and toilet flushes. 

The second approach of Cox Architecture’s water management is the general water flow. One of the essential things about circular, sustainable building is the remainder of what is used. Wastewater management is not what we usually see in a so-called sustainable building. To reach net zero water management, Cox Architecture has thought about this issue too. The approach is going with the natural process. All water sent to drains goes through the blackwater treating plant, and after the operation, the clean water is used to irrigate the garden. Design decisions about landscape also came from providing the stormwater runoff. Hardscapes have been minimised around the building to reach the natural water absorption process.

Energy Management

Energy concern was one of the main challenges of the SBRC. From the construction to the life of the user has been thought to reach net zero energy. The building is created to sustain itself with the gained energy. A total of 600 PV panels have been used to gain enough power. A unique PVT system has been designed with the BlueScope team, and this system also contributes to the SBRC as an archive for a sustainable building approach.

Material Selection

Selecting long-lasting, locally sourced material is one one the key elements of sustainable buildings for Cox Architecture. In SBRC, which can be taught as an archive and research itself, the material selection was essential to show how to do more with less. For the material selection , the team went to appropriate sourcing for reusable materials, and all sources for the process were local. The idea of dematerialization was a key that every product could do more than one job, and locally reused materials were selected wherever possible. Bridge timbers, steel railway tracks, abandoned telegraph poles, reused bricks, and local timbers can be given as examples of the used materials. The richness of textures from the outside can make the building a material library where it proves the power of what can be done with reused materials.

The building itself became a research topic for sustainability, and it is a prototype for such an issue. Cox Architecture defines the structure as dynamic , adaptable, and retrofittable . All design decisions came from the idea of being site-specific and getting the maximum benefit from the surroundings rather than depending on wasteful systems. The orientation also influences the planning of the building. The thin floor plates and suitable orientation provide adequate natural ventilation and sunlight. The form of the building lets sea breezes in, and the eaves provide sun shading during the noon time. The dynamic approach of the building invites people into the building and semi-open social areas. The site is publicly open at all times and lets every curious one experience the natural aura of this education centre.  

References List:

• COX. (n.d.). Sustainable Buildings Research Centre. [online] Available at: https://www.coxarchitecture.com.au/project/sustainable-buildings-research-centre/ .
• COX. (n.d.). Practice. [online] Available at: https://www.coxarchitecture.com.au/practice.
• International Living Future Institute. (n.d.). Sustainable Buildings Research Centre. [online] Available at: https://living-future.org/case-studies/sustainable-buildings-research-centre/ [Accessed 28 Sep. 2022].
• Editor, J.G. – (n.d.). Sustainable Buildings Centre. [online] Available at: https://www.builtworks.com.au/sustainable-buildings-research-centre/ [Accessed 28 Sep. 2022].
• www.uow.edu.au. (n.d.). Research – University of Wollongong – UOW. [online] Available at: https://www.uow.edu.au/sbrc/research/ [Accessed 28 Sep. 2022].
• archiroots.net. (n.d.). University of Wollongong’s Sustainable Buildings Research Centre (SBRC) | archiroots. [online] Available at: https://archiroots.net/project/university-of-wollongong-s-sustainable-buildings-research-centre-sbrc/b

A graduate student who sees architecture as a way to think critically. Using her architectural background, she aims to draw attention to the ways of existing with the earth, not against earth with her writings. She believes that critical thinking will open different doors to both people and the world.

Sustainable Land use Strategies in Urban Planning

15 Places to Visit in Udaipur for Travelling Architect

Related posts.

Kopeshwar temple, Khidrapur

Shipping Container Skyscraper for Mumbai Slum by Ganti + Asociates (GA) Design

Gyan Museum, Jaipur

Community Architecture: Stoneleigh Terrace

Kariba Dam: Engineering Marvel and Environmental Impact

School for Blind and Visually Impaired Children by SEAlab

• Architectural Community
• Architectural Facts
• RTF Architectural Reviews
• Architectural styles
• City and Architecture
• Fun & Architecture
• History of Architecture
• Design Studio Portfolios
• Designing for typologies
• RTF Design Inspiration
• Architecture News
• Career Advice
• Case Studies
• Construction & Materials
• Covid and Architecture
• Interior Design
• Know Your Architects
• Landscape Architecture
• Materials & Construction
• Product Design
• RTF Fresh Perspectives
• Sustainable Architecture
• Top Architects
• Travel and Architecture
• Rethinking The Future Awards 2022
• RTF Awards 2021 | Results
• GADA 2021 | Results
• RTF Awards 2020 | Results
• ACD Awards 2020 | Results
• GADA 2019 | Results
• ACD Awards 2018 | Results
• GADA 2018 | Results
• RTF Awards 2017 | Results
• RTF Sustainability Awards 2017 | Results
• RTF Sustainability Awards 2016 | Results
• RTF Sustainability Awards 2015 | Results
• RTF Awards 2014 | Results
• RTF Architectural Visualization Competition 2020 – Results
• Architectural Photography Competition 2020 – Results
• Designer’s Days of Quarantine Contest – Results
• Urban Sketching Competition May 2020 – Results
• RTF Essay Writing Competition April 2020 – Results
• Architectural Photography Competition 2019 – Finalists
• The Ultimate Thesis Guide
• Introduction to Landscape Architecture
• Perfect Guide to Architecting Your Career
• How to Design Architecture Portfolio
• How to Design Streets
• Introduction to Urban Design
• Introduction to Product Design
• Complete Guide to Dissertation Writing
• Introduction to Skyscraper Design
• Educational
• Hospitality
• Institutional
• Office Buildings
• Public Building
• Residential
• Sports & Recreation
• Temporary Structure
• Commercial Interior Design
• Corporate Interior Design
• Healthcare Interior Design
• Hospitality Interior Design
• Residential Interior Design
• Sustainability
• Transportation
• Urban Design
• Host your Course with RTF
• Architectural Writing Training Programme | WFH
• Editorial Internship | In-office
• Graphic Design Internship
• Research Internship | WFH
• Research Internship | New Delhi
• RTF | About RTF
• Submit Your Story

Looking for Job/ Internship?

Rtf will connect you with right design studios.

Microbiological Laboratory | Penelas Architects

AstraZeneca’s The Discovery Centre | Herzog & de Meuron

PANNAR Sufficiency Economic & Agriculture Learning Center | Vin Varavarn Architects

Ford Calumet Environmental Center | Valerio Dewalt Train

Texoversum Innovation Center | allmannwappner

The Vietnam Institute for Advanced Study in Mathematics | 1+1>2 Architects

Google Bay View | BIG + Heatherwick Studio

NanFang University Technology Park and B1 Tower Building | Saltans Architects

Learning Center At Quest | KSM Architecture

Johns Hopkins University, Applied Physics Laboratory, Building 201 | CannonDesign

Youngmeyer Field Station | Hutton

Dobra 55 Modern Languages and Applied Linguistics Building | Kuryłowicz & Associates

• Next Page »

Symbiosis University Hospital and Research Centre | IMK Architects

Information

• Project Name: Symbiosis University Hospital and Research Centre
• Practice: IMK Architects
• Products: Wipro , Legero Lights , Alstone , Saint Globin , Nyati Engineers & Consultants Pvt Ltd. , Parryware , Cera , Weathercool
• Completion year: 2019
• Gross Built up Area: 41800 sqm
• Project Location: Pune
• Country: India
• Lead Architects/Designer: Rahul Kadri
• Design Team: Nithin Hosabettu (Design Director), Sahil Bipin Deshpande (Architect), Viraj Naralkar (Architect), Aakash Kumar Srivastav (Architect), Oshmi Sengupta (Architect),Pallavi Rai (Architect)
• Clients: Symbiosis Society
• Engineering: The Axis Structural Consultants, Pune
• Structural Consultants: The Axis Structural Consultants, Pune
• MEP Consultants: Radiant Consulting Engineers, Navi Mumbai
• Landscape Consultants: IMK Architects
• Photo Credits: Rajesh Vora
• Others: HVAC: Radiant Consulting Engineers, Navi Mumbai Lighting: IMK Architects & Radiant Consulting Engineers Façade: IMK Architects

Excerpt: Symbiosis University Hospital and Research Centre is an architectural project designed by IMK Architects in Pune, India. Sitting along a slope, the building is strategically positioned to minimize the cut-and-fill of the hill site. Planned as a robust curve along the contours of the land, it forms the project’s façade. Imbibing the client brief of ‘grandeur’ being a key element, two significant and symbolic entrances have been designed, distinct in approach to cater to the client brief of unique identities for the hospital and the academic block.

Project Description

[Text as submitted by architect] Established on the lower slopes of a hill, amidst the sprawling 40 acres land of Pune’s renowned educational institute Symbiosis, SUHRC has been developed on a forested hill, in a discrete and quiet location. Envisaged as a Multi-Specialty Hospital to provide excellent health care facilities and a State of the art – Centre that would enhance Skill development in the Field of Medicine, the development envisions to educate and empower medical students. Conceptualized to cater to the needs of all the stakeholders, the hospital caters to the nearby population of Pune and its neighboring areas, while providing Tele-Medicine services to ensure outreach services to peripheral, far-flung and access-compromised settlements. Amidst today’s context, currently, the hospital is being used for Government welfare as COVID 19 hospital.

Sitting along a slope, the building is strategically positioned to minimize the cut-and-fill of the hill site. Planned as a robust curve along the contours of the land, it forms the façade of the project. Imbibing the client brief of ‘grandeur’ being a key element, two significant and symbolic entrances have been designed, distinct in approach to cater to the client brief of unique identities for the hospital and the academic block. While the entry to the hospital is welcoming, peaceful, it also provides a sense of grandeur along with a structure that expresses Solidarity, resonating care, and shelter for the patients in distress. A large open-to-sky courtyard separates the Centre from the hospital. The entrances for both these blocks lie on either side of the building, making them seem like two completely different entities, providing the students with different access as well as a space for them for relaxation and academic purposes.

The entrance for the Skill Centre draws inspiration from the stainless steel surgical instruments used in hospitals. A mammoth silver steel bird, with wings wide open, welcomes the visitor into the building. Supported by steel pipes, a futuristic roof under the sun beaming brightly, is symbolic of a contemporary architectural response. The roof sits above the large semi-circular staircase that leads the visitor to the entrance. In contrast, the response to the other block is humbler. Split into two entries, where one is for casualties and the other for regular populous, the base of the entrance is lifted to be in line with the interiors. Above this large entrance, a slightly curved roof is designed, shading the entrance and making it possible for people to wait outside too. The tip of the entrance is stretched upwards, thus letting in more light to create a deeper canopy so that at least two lanes of vehicles could be parked along the length of the building under it. The canopy was stretched to the complete width of the building and gave it a smooth curvilinear shape in the front, which mellows down the impact of the sheer size of this structure. A balance between light and shade has been achieved with smaller skylights within the roof, and a larger opening towards the upward bending tip. The upper surface of this canopy was converted to a terrace garden, such that even the single bed patients would be treated to a biophilic space, which would promote healing. Two large courtyards in the building create buffer zones that help in healing patients, bring in ample light, and are overlooked by wardrooms and the Out-Patient Department. The OPD has no air-conditioning but allows for fresh, natural air while ensuring sufficient ventilation.

All departments and spaces of the hospital are designed such to bring in daylight and natural ventilation. Even areas like OPD, waiting have courtyards on both sides and are naturally ventilated. Similarly, at all levels, there is a 3m wide corridor that abuts the central courtyard, which lets in natural light and ventilation, thereby reducing the AC load and power consumption for these areas. The courtyards act as spaces for healing, for the congregation while reducing cross-infections. The terrace gardens are landscaped with flowering trees and plants to create a soothing effect for the patients as well as the accompanying caretakers. The main central courtyard transforms the space around it, unlike any regular hospitals, which are much mundane, completely air-conditioned spaces without any daylight and natural ventilation. Keeping sterility in mind, the courtyard is non-accessible, full of plants and small trees, etc. to avoid the risk of contamination and maintenance. The accessible spaces in the courtyard are paved with tiles and stones that are maintenance-free. The main central courtyard not only brings in the above features but also makes the space aesthetically pleasing, soothing to the eye creating a calm oasis. The space is completely transformed, unlike any regular hospitals, which are much mundane, completely air-conditioned spaces without any daylight & natural ventilation in significant areas. This also helped in simplifying the way-finding in the building, which otherwise in other hospital projects is a complicated affair for the users adding to their trauma in searching spaces/ departments within the structure.

This 216 Bedded Hospital provides state of the art 5 Operation Theatres, equally well equipped 22 ICU units. The Radiology department offers MRI, CT, X-ray, Fluoroscopy, BMD, and other scans. A dedicated Cardiology department with Cath Lab facility, Kidney dialysis with nine beds, separate Gynaecology Department with an independent OT Facility. Functionally, the building comprises of four sections; three of them belong to the hospital and the last one being the Skill Centre. Every section has been planned and conceptualized for its functionality- driven design and a distinct formal response as a result. The three blocks of the hospital, namely, the general hospital block, the procedure block, and the multi-specialty block, all have been planned with keeping in mind the ease of functionality and avoiding criss-cross movement. Amongst the 900 beds in the general hospital block, 600 beds are for free patients and procedures, which are all taken care of and treated by the medical college students. This block has been connected to the procedure block, with services running along one side of the corridor. At the same time, the departments are situated on the other, allowing for ease of access without causing any disturbances. Further, the procedure block is well segregated from the ward block to ensure privacy to every patient, while being shared as a standard procedure block by paid and free patients alike. This block is equipped with the latest medical technology being robotic Operation Theatres and surgical operations.

The hospital is planned across five levels; departments such as the OPD, casualty, radiology, MHC, etc. which require easy ground access and are frequented by more patients every day have been planned at the ground level. The first-floor compromises of General, Twin, and Single bed wards, most of which overlook the garden space. The critical areas such as OT’s, Pre-Op and Post Op, ICU’s Cath, etc. are planned on the 2nd floor, making it the sterile zone and thus are also segregated from other areas of the hospital. The 3rd floor, which was earlier planned for future expansion, was also converted to a ward floor as the client decided to get a DNB Accreditation, which required 200 + beds. All services are planned in the basement and lower ground level, including parking, stores, and other necessary hospital facilities such as morgue, medical gases, workshop engineering offices, garbage disposal, etc. The partly enclosed basement and LG levels are designed such that they use natural ventilation and daylight as much as possible. A 2.5 m wide ramp is also provided in the central courtyard, which will help patients to evacuate during a fire without being dependent on any mechanical systems.

Attention has been paid to construction details, where post-tensioned slabs are used to achieve flexibility, minimum beams, and larger spans that facilitate different size room arrangements and to allow for easy routing of ducts. The flexible grid is designed to synchronize the structural system at all levels. To enable a natural, original, and permanent finish on the building, which would be maintenance-free, brick was adopted as the material of choice for the double skin on the façade with deeper shading projections that would reduce heat gain. The resultant boxing forms were skewed, twisted, or tapered to achieve variations in shape to form a multi-faceted façade that reflects light in different tones in any part of the day. Creating dynamism through its texture, capturing the play of light and shadow each day, the façade looks different, complimenting every mood of the day. What is typically done in concrete, has been made possible in the brick and looks exclusive. Naturally compressed, sundried earthen bricks produced on-site, have been used for façade and masonry work and methods such as brick-boxing were incorporated to achieve efficiency, while reducing pollution. Together with exposed concrete, the skin and the façade flatter the green hills beyond.

Smaller details have been taken into account to ensure a calm and serene atmosphere that promotes healing while encouraging research and creating spaces that are easy to use and maintenance-free. Colour coding has been enabled for easy identification of the assortment of spaces and critical areas such as ICU’s are endowed with light colors of soothing shades to reduce anxiety. To maintain a warm and tranquil environment for patients, soft home-like colors cover the inner walls of the hospital. The ward rooms are designed with warmer and subtle hues of colors that are complemented with teak laminates. Varied temperatures and light with correct lux levels have been worked out for different areas, and the furniture has been customized for special and diverse needs. Acoustic materials in the ceilings of corridors and rooms reduce noise pollution and provide easy access to services. Encouraging way-finding, the nurse stations are highlighted with shades of warm yellow/orange to be identifiable from any side of the long corridor.

The Skill Centre is planned on the fourth and fifth floor with a separate entrance at the ground level. Being a part of the Symbiosis institution, the client brief called for a grand entrance that would be symbolic to the medical field, reflect Symbiosis legacy, and depict a futuristic approach. A grand entrance with steps ascending 8 m moves into the entrance porch. Shaped like the wings of a bird stretched to its extreme ends, the entrance is made of steel with an aluminium skin. Resting on a combination of steel and concrete Columns that give it a solid base and balance, the double-height entrance leads into a curvilinear café, which in turn connects to the Skill Centre designated vertical cores connecting to the 4th and 5th floor. The café gains ample daylight and natural ventilation as it has an enclosed courtyard on one side and stepped landscape on the other, thus creating a pleasant view for the users of the Café. The Skill Centre has four departments; Centre for health skills, School of Nursing, Institute of health science, and a School of Open and Distance learning. They share 9 classrooms with approximately 60- 90 students per class, a 300-seater auditorium, a library, meeting rooms, labs, changing areas, where all professionals such as Surgeons, Doctors, Nurses, and other allied fields.  All the departments are designed, planned, and implemented with one cohesive design vocabulary for medical professionals, with sufficient natural ventilation and daylight.

As multi-facetted angular walls create an orthogonal earthen tone on the façade, the landscape is designed as a free-flowing organic form, using plants of different colors, flower shrubs, and small trees. This creates a healing effect on the patients while acting as pleasant distractions for the patients’ relatives and hospital staff. The vast open space has been planned with approximately ____ number of trees of different species, shrubs, and plants with a vast retention pond at the lowermost level to facilitate zero discharge. The large retaining wall of exposed concrete also gets covered with climbers and plants, converting a blank mass of concrete to an aesthetically pleasing feature that adorns the approach to the hospital. Carefully and strategically planned, the building attempts to make gestures that are grand, yet local and responsive with attention to details such as the brick-art and the exposed concrete. Sitting comfortably on the fringe of the hill, the inner courtyards seem like a continuation of the hill, where the built form amalgamates with the site. Allowing nature to be a part of the hospital and integrating it as a comforting element for the patients, the hospital creates a space for recovery and rejuvenation.

Leave a Reply Cancel reply

Login to post a comment.

Privacy Overview

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

• Submit News
• Purchase Webinar Listing

ArchShowcase

• Submit New Showcase
• Submit New Event
• Submit New Video
• Submit New YouTube Video
• AECCafe Media Kit
• Banner Ad Specifications
• eMail Blast Specifications

Back to 'AECCafe Blogs'

Subscribe to aeccafe.

Submit Your Project

Recent posts.

• AS Barcelona Home Renovation Project, Spain by C & Arquitectura
• Huanggang Skyscraper Redefines the Shenzhen Urban Landscape by Aedas-designed
• Hamdan Bin Rashid Cancer Hospital, Dubai at UAE by Stantec
• Two-family house in Yawata Japan by Hirokazu Taniguchi
• 40 Spring Gardens, Manchester, UK by SpaceInvader
• Entries RSS

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Case study: A Not-For-Profit Medical Research Centre

Related Papers

Dominique Deplanque

PROJECT PROPOSAL FOR ESTABLISHMENT OF COMMUNITY HEALTH CARE SERVICES

Gerald Onyekachi Onuoha

Establishment of a community based health clinic is very essential in every community to provide health care delivery to the grass root. Hence the need to establish such in the community is imperative. It has been observed that before residents of the community assess any health care service in the urban city that they spend not less than one thousand Naira in terms of transportation. Besides, there has been an increase in the rate of maternal, neonatal and infant mortality rate in this area.

Rosh Pilapil

Vikram Gowda

Nursing Children and Young People

Theresa Pengelly

Hussain Varawalla

karthi keyan

ikeokwu anderson

Texila International Journal , M.S. Seahloli

A case study on the process of applying for ethical and provincial clearance to conduct a medical informatics research for a PhD programme in South African Hospitals. The programme was looking for current status of medical informatics and implementing electronic healthcare record, challenges, and future direction in South Africa. Nine provinces were contacted and all approved the study, however, the approval was obtained after averaged of 9 months which was longer than expected. The public hospital CEOs and medical managers were also contacted to acknowledge participation and give clearance for the study to be conducted at hospitals. After an average of 3 months to give clearance and out of 40 hospitals contacted only 70% acknowledged. Overall the process took longer than expected to approve a non-clinical trial study for academic purpose. This has delayed the start of the PhD research program and the challenges encountered in the provinces applications are due to autonomy and lack of standardisation of procedure between the provinces, lack of understanding the procedure of the study by the province personnel and hospitals, lack of expertise in handling electronic submissions and non-clinical trial submissions. IT is still a challenge to most of government employees and the infrastructure that can accommodate amount of information.

RELATED PAPERS

International Journal of Hydrogen Energy

Sepideh Madadkhani

Retno Puspita

JOURNAL OF ADVANCES IN AGRICULTURE

Virginia Sarropoulou

Jacob Hudis

Ronald Smith Tairo Medina

Molecular Pharmacology

Bing shuo Chen

Journal of Pediatric Surgery

Stefano Avanzini

English Language, Literature & Culture

Dr. Stephania Albert Jonglai

Clinical Child and Family Psychology Review

Dennis Embry

Trakya university journal of natural sciences

Erdem Seven

Tidskrift för genusvetenskap

Carin Holmberg

Korean Journal for Food Science of Animal Resources

Yujin (Heidi) Ahn

Cahiers De Topologie Et Geometrie Differentielle Categoriques

René Guitart

E-Jurnal Akuntansi

在线制作乔治华盛顿大学毕业证 gwu毕业证文凭证书实拍图学历学位认证原版一模一样

Jacqueline Teixeira

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Booking open for the Members' Case Forum 2024

View the full programme and book your place on the members' case forum..

Meet our case teaching scholars

Our 2024 case teaching scholarship recipients represent organisations from all over the world..

The impact of case writing

We explore the role case writing plays across many spheres of a business school's activity..

5 minutes with... Narentheren Kaliappen

Hear from narentheren about his passion for writing and teaching with cases..

The Case Centre Awards and Competitions 2024

Discover more about our winners and their winning cases..

Come on a case workshop

Develop your case teaching and writing skills at our in-person workshops at porto business school in june/july 2024..

Search, preview and buy from our diverse range of business and management teaching materials.

Submit your cases or collection to us for distribution to a global audience.

Find out about the case method through our videos, features, resources and newsletter.

Discover how we recognise the work of case teachers and writers worldwide.

Support for case teachers, writers and students through workshops, scholarships and much more.

The case community is at the heart of everything we do. Find out how you can get involved.

Discover our case collections

Cases in the spotlight

Read about more cases in the spotlight

Learning with cases can be a challenging experience.

Our interactive study guide takes students through the process, providing practical tips, tricks and tools.

Latest news

Joachim Sandt received his trophy at Frankfurt University of Applied Sciences on 22 May 2024.

• Hispanoamérica
• Work at ArchDaily
• Terms of Use
• Privacy Policy
• Cookie Policy
• Higher Education

Research and Training Centre in the Construction Trades / ACDF*

• Curated by ArchDaily
• Architects: ACDF Architecture : ACDF* Architectes
• Area Area of this architecture project Area:  57000 ft²
• Year Completion year of this architecture project Year:  2010
• Photographs Photographs: Marc Cramer

Text description provided by the architects. La Cité collégiale is a French-language college of applied arts and technology located in Ottawa , Ontario, Canada. To ensure that enough trained skilled trades workers are available to keep our economy moving forward, La Cité collégiale decided to organize an architectural competition for the realization the New Research and Training Centre in Construction Trades as the first phase of the new Campus of Orléans. The centre will allow La Cité collégiale to almost triple its apprenticeship program, to include training in 18 high-demand construction trades. This new building will include workshop, laboratories, studios, and classrooms.

The Trades Centre project proposed by the architects pays special attention to archi- tecture issues, site organization and potential for future development. It ensures consistency of buildings throughout the territory, construction quality, respect of rural identity and recognition of the regional territory icons and landscape memory. The new facility will take advantage of the topography and of the immediate site by deploying in a unique landscape and ecologic set. The proposal aims to define a concept of unique development that perfectly integrates its environment. It reflects a distinctive architecture while clearing the atmosphere suitable for student life. The objectives considered in the architectural approach are:

Composition of a quality project that stands out positively in terms of architecture, urbanism, landscape and environment; Ensure a high visibility from the regional road during the day and night; Ensure a strong dialogue with the natural components of the site and landscape; Enhance the existing streams and revitalize the banks; Provide a comprehensive site plan management that will, over the time, generate a consistent link with the regional landscape memories; Provide a flexible concept that will ensure the sustainability of the project and architectural design.

Context and/or Urban Design Components: The richness of the different agricultural landscape strata and the richness of the topography of the site are the starting points of our conceptual approach. The analysis of the features of the site can see the richness of the dialogue between the topography of the area and the new building and the contrast between the peaceful and contemplative character of the landscape and the dynamic movement at the cars.

Like a vegetal plate that rises above the ground, the concept proposes the development of a sculptural planted roof, a kind of reconstituted topography who engaged a dialogue with the horizontal natural landscape while generating a strong sculptural presence on Route 174. The Trades Centre project is orientated according to the frame of rows. The position of the buil- ding highlights topographical depressions and streams. The parking lots match with the path and are not visible from Highway 174, in order to magnify the project and its development into the natural landscape. The project’s implementation allows further development to the North and the East while maintaining the logical hierarchy of the main entrance first. To highlight the horizontal nature and topography of the project, a land form is proposed along Route 174. The land form, planted with reflectors, will identify the project on Route 174 and mark the specific topography, landscape and ecological aspects of the project.

Integration of Sustainable Design: The landscape project revolves around the existing drainage ditches on the site. Through decades of development in the agricultural sector, streams have remained intact and functional for collec- ting surface water from field to river. This drainage network becomes the generating element of the design. It is the central point around which all the formal language of the site articulates and where all the principles of ecological landscaping emerge. These gaps’ prints, such as an open hand, extend its fingers in every direction, providing texture and atmosphere within the agricul- tural matrix. This matrix is magnified and integrated into pedestrian circulation throughout the campus, defining areas of building settlement, landscaped green spaces and functional spaces. The pedestrian corridors link the future buildings crossing the green areas and stepping over the streams through bridges.

New plantings are organized into two formal languages. The first one is composed of indigenous pioneer plants that are settled in the ditches following the contour lines and interacting with the existing woodlands to create, with years, rich ecological and spatially well defined corridors. The second one is composed of large-scale trees, planted along the agricultural frame and cutting in a formal way the collective spaces, the functional public site, such as parking lots and grassy areas.

Innovation in Addressing Program and/or the Clients' Requirements: The project generates a strong sculptural presence on Highway 174 and sets down the Trades Centre as a benchmark in the community and regional landscape. The proposed building location allows future development to the North and East sides while maintaining the logical hierarchy of the main entrance at the start of the visit. Pending further development, this is where a platform for external outdoor courses and exhibi- tion of student’s projects will be installed. The presence of the belvedere, a headland reached by a wide staircase / agora, offers a view on the river and the landscape of the region, but also on the platform for educational purpose (green roof, mechanical appliance).

The sculptural and aerial character of the concept can generate a multitude of programmatic arrangements that give the project a perfect flexibility in the process of establishing a school through an architectural competition. This will make it possible and easy to adapt the planning of program functions, to project future expansions while retaining the basic concept. The Trades Centre is mainly organized on one level and, programmatically, is divided into two sectors: the construction sector and the education sector . Spatially, the Centre consists of a heart a space of construction, discussion / dissemination and materiality. This core is surrounded to the South by shops, and to the North by administrative and teaching areas.

The East-West section shows the education sector, the main entrances, the spatial fluidity and the great curtain wall facing the site and the Ottawa River. The green roof and the access to the belvedere are also visible on it. The North-South section demonstrates the relationship between the Trades Centre, Route 174 and the topography of the site. This view also explains the visual and acoustic protection that can offer the bank while showing a breathtaking view of the building. The natural sunlight is recom- mended for workshops. The second level enjoys abundant natural light and offers spectacular views of the site and the Ottawa River. Simple glass partitions are placed below the class timber volume and represent the administrative area of the Trades Centre. This sector is composed by the director’s office, his assis- tant (e) and teachers’ offices, a working and resource room, the staff lounge and a multipurpose conference room. The construction details of the building will exhibit and showcase construction techniques so that students can appreciate and understand the different components required in building construction.

Project gallery

Project location

Address: orléans, ottawa, on, canada.

• Sustainability

想阅读文章的中文版本吗?

建筑学培训中心 / ACDF*

You've started following your first account, did you know.

You'll now receive updates based on what you follow! Personalize your stream and start following your favorite authors, offices and users.

Check the latest Desks

Check the latest Desk Accessories

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

India Case Research Centre

CASE CATALOGUE

CASEPEDIA the case journal

INDUSTRY best practice case

The AIMA case research centre is established after a lot of discussion, deliberation and research. The purpose of setting up of India case research centre at AIMA is primarily to focus on developing and publishing Industry based India-focused research cases.

Submit your Case

• Case Competition 2021
• Case Competition 2022
• Case Competition 2023
• Case Competition 2024

New Arrival

Wheels of Change ...

NAVIGATING CULTURAL COMMERCE AND FI...

‘IS MY ATTITUDE GETTING ON MY WAY...

CHAT-GPT: PLAGIARIZING THE FUTURE...

A CASE STUDY ON THE ROLE OF AI/ML I...

IT Management Simulation: Cyber Attack!

15FEN: Finding The Blue Ocean in China’s Fresh Food

The Toshiba Accounting Scandal: How Corporate Governance Failed

Zandu Pharmaceutical Works: The Tak...

Ketan Logistics—Charting the Next...

SHOESTRING ARBITRAGE WITH STOCK BUY...

NEW DELHI TELEVISION ...

How To Grow On Chinese Soil

Events & Workshop

Affiliate organisations.

Due to the outbreak of the Novel Coronavirus pandemic, we at AIMA shall strive to do our bit to avoid unnecessary risk and spread of the disease.

In keeping with the Government orders, we are postponing all major events scheduled in the next few weeks. However, for the tests like MAT and UGAT, please refer to respective section of the website for their modified schedules and keep visiting those sections for periodic updates time to time. The AIMA Library shall also remain closed. We will review the situation week by week.

Adopting all modern means of technology, all staff members will "Work from Home". However, there will be no slowdown in our response to you.

All staff members will be available on phone, email, WhatsApp etc. and will respond to you as efficiently as possible.

• Anniversary
• Publications

Case Study: The Democratic Republic of the Congo

Navigating human rights and peacebuilding amidst transition.

• Share on Twitter
• Share on LinkedIn
• Share on Facebook

In the Democratic Republic of the Congo (DRC), a recent escalation in conflict, past issues in human rights enforcement and accountability, and the ongoing withdrawal of United Nations Organization Stabilization Mission in Democratic Republic of the Congo (MONUSCO) present challenges for advancing human rights and peacebuilding. This case study considers how programming that combined human rights and peacebuilding tools and strategies sought to overcome those challenges, exploring four projects that supported transitional justice, reconciliation, reintegration of ex-combatants, women’s empowerment, countering gender-based violence, and youth empowerment within the DRC.  This case study is an excerpt from a larger 2024 Peacebuilding Fund (PBF) Thematic Review examining synergies between human rights and peacebuilding. It examined a select sample of PBF programming – 92 projects implemented in 45 countries and territories – that were supported between 2017 and 2022, with a view to collecting best practices and lessons learned, and contributing to better understanding of how human rights and peacebuilding tools and strategies may complement each other in advancing peace and preventing conflict. This case study appears on pages 36 to 46 of the full report. Access the DRC case study infographic here .

Related content

2024 PBF Thematic Review: Synergies between Human Rights and Peacebuilding in PBF-supported Programming

31 May 2024

Case Study: Colombia

Case Study: Hate Speech

Cybersecurity Threats, Vulnerabilities and Resilience Among Women Human Rights Defenders and Civil Society in South-East Asia

Home / Guides / Citation Guides / How to Cite Sources

How to Cite Sources

Here is a complete list for how to cite sources. Most of these guides present citation guidance and examples in MLA, APA, and Chicago.

If you’re looking for general information on MLA or APA citations , the EasyBib Writing Center was designed for you! It has articles on what’s needed in an MLA in-text citation , how to format an APA paper, what an MLA annotated bibliography is, making an MLA works cited page, and much more!

MLA Format Citation Examples

The Modern Language Association created the MLA Style, currently in its 9th edition, to provide researchers with guidelines for writing and documenting scholarly borrowings.  Most often used in the humanities, MLA style (or MLA format ) has been adopted and used by numerous other disciplines, in multiple parts of the world.

MLA provides standard rules to follow so that most research papers are formatted in a similar manner. This makes it easier for readers to comprehend the information. The MLA in-text citation guidelines, MLA works cited standards, and MLA annotated bibliography instructions provide scholars with the information they need to properly cite sources in their research papers, articles, and assignments.

• Book Chapter
• Conference Paper
• Documentary
• Encyclopedia
• Google Images
• Kindle Book
• Memorial Inscription
• Museum Exhibit
• Painting or Artwork
• PowerPoint Presentation
• Sheet Music
• Thesis or Dissertation
• YouTube Video

APA Format Citation Examples

The American Psychological Association created the APA citation style in 1929 as a way to help psychologists, anthropologists, and even business managers establish one common way to cite sources and present content.

APA is used when citing sources for academic articles such as journals, and is intended to help readers better comprehend content, and to avoid language bias wherever possible. The APA style (or APA format ) is now in its 7th edition, and provides citation style guides for virtually any type of resource.

Chicago Style Citation Examples

The Chicago/Turabian style of citing sources is generally used when citing sources for humanities papers, and is best known for its requirement that writers place bibliographic citations at the bottom of a page (in Chicago-format footnotes ) or at the end of a paper (endnotes).

The Turabian and Chicago citation styles are almost identical, but the Turabian style is geared towards student published papers such as theses and dissertations, while the Chicago style provides guidelines for all types of publications. This is why you’ll commonly see Chicago style and Turabian style presented together. The Chicago Manual of Style is currently in its 17th edition, and Turabian’s A Manual for Writers of Research Papers, Theses, and Dissertations is in its 8th edition.

Citing Specific Sources or Events

• Declaration of Independence
• Gettysburg Address
• Martin Luther King Jr. Speech
• President Obama’s Farewell Address
• President Trump’s Inauguration Speech
• White House Press Briefing

Additional FAQs

• Citing Archived Contributors
• Citing a Blog
• Citing a Book Chapter
• Citing a Source in a Foreign Language
• Citing an Image
• Citing a Song
• Citing Special Contributors
• Citing a Translated Article
• Citing a Tweet

6 Interesting Citation Facts

The world of citations may seem cut and dry, but there’s more to them than just specific capitalization rules, MLA in-text citations , and other formatting specifications. Citations have been helping researches document their sources for hundreds of years, and are a great way to learn more about a particular subject area.

Ever wonder what sets all the different styles apart, or how they came to be in the first place? Read on for some interesting facts about citations!

1. There are Over 7,000 Different Citation Styles

You may be familiar with MLA and APA citation styles, but there are actually thousands of citation styles used for all different academic disciplines all across the world. Deciding which one to use can be difficult, so be sure to ask you instructor which one you should be using for your next paper.

2. Some Citation Styles are Named After People

While a majority of citation styles are named for the specific organizations that publish them (i.e. APA is published by the American Psychological Association, and MLA format is named for the Modern Language Association), some are actually named after individuals. The most well-known example of this is perhaps Turabian style, named for Kate L. Turabian, an American educator and writer. She developed this style as a condensed version of the Chicago Manual of Style in order to present a more concise set of rules to students.

3. There are Some Really Specific and Uniquely Named Citation Styles

How specific can citation styles get? The answer is very. For example, the “Flavour and Fragrance Journal” style is based on a bimonthly, peer-reviewed scientific journal published since 1985 by John Wiley & Sons. It publishes original research articles, reviews and special reports on all aspects of flavor and fragrance. Another example is “Nordic Pulp and Paper Research,” a style used by an international scientific magazine covering science and technology for the areas of wood or bio-mass constituents.

4. More citations were created on  EasyBib.com  in the first quarter of 2018 than there are people in California.

The US Census Bureau estimates that approximately 39.5 million people live in the state of California. Meanwhile, about 43 million citations were made on EasyBib from January to March of 2018. That’s a lot of citations.

5. “Citations” is a Word With a Long History

The word “citations” can be traced back literally thousands of years to the Latin word “citare” meaning “to summon, urge, call; put in sudden motion, call forward; rouse, excite.” The word then took on its more modern meaning and relevance to writing papers in the 1600s, where it became known as the “act of citing or quoting a passage from a book, etc.”

6. Citation Styles are Always Changing

The concept of citations always stays the same. It is a means of preventing plagiarism and demonstrating where you relied on outside sources. The specific style rules, however, can and do change regularly. For example, in 2018 alone, 46 new citation styles were introduced , and 106 updates were made to exiting styles. At EasyBib, we are always on the lookout for ways to improve our styles and opportunities to add new ones to our list.

Why Citations Matter

Here are the ways accurate citations can help your students achieve academic success, and how you can answer the dreaded question, “why should I cite my sources?”

They Give Credit to the Right People

Citing their sources makes sure that the reader can differentiate the student’s original thoughts from those of other researchers. Not only does this make sure that the sources they use receive proper credit for their work, it ensures that the student receives deserved recognition for their unique contributions to the topic. Whether the student is citing in MLA format , APA format , or any other style, citations serve as a natural way to place a student’s work in the broader context of the subject area, and serve as an easy way to gauge their commitment to the project.

They Provide Hard Evidence of Ideas

Having many citations from a wide variety of sources related to their idea means that the student is working on a well-researched and respected subject. Citing sources that back up their claim creates room for fact-checking and further research . And, if they can cite a few sources that have the converse opinion or idea, and then demonstrate to the reader why they believe that that viewpoint is wrong by again citing credible sources, the student is well on their way to winning over the reader and cementing their point of view.

They Promote Originality and Prevent Plagiarism

The point of research projects is not to regurgitate information that can already be found elsewhere. We have Google for that! What the student’s project should aim to do is promote an original idea or a spin on an existing idea, and use reliable sources to promote that idea. Copying or directly referencing a source without proper citation can lead to not only a poor grade, but accusations of academic dishonesty. By citing their sources regularly and accurately, students can easily avoid the trap of plagiarism , and promote further research on their topic.

They Create Better Researchers

By researching sources to back up and promote their ideas, students are becoming better researchers without even knowing it! Each time a new source is read or researched, the student is becoming more engaged with the project and is developing a deeper understanding of the subject area. Proper citations demonstrate a breadth of the student’s reading and dedication to the project itself. By creating citations, students are compelled to make connections between their sources and discern research patterns. Each time they complete this process, they are helping themselves become better researchers and writers overall.

When is the Right Time to Start Making Citations?

Make in-text/parenthetical citations as you need them.

As you are writing your paper, be sure to include references within the text that correspond with references in a works cited or bibliography. These are usually called in-text citations or parenthetical citations in MLA and APA formats. The most effective time to complete these is directly after you have made your reference to another source. For instance, after writing the line from Charles Dickens’ A Tale of Two Cities : “It was the best of times, it was the worst of times…,” you would include a citation like this (depending on your chosen citation style):

(Dickens 11).

This signals to the reader that you have referenced an outside source. What’s great about this system is that the in-text citations serve as a natural list for all of the citations you have made in your paper, which will make completing the works cited page a whole lot easier. After you are done writing, all that will be left for you to do is scan your paper for these references, and then build a works cited page that includes a citation for each one.

Need help creating an MLA works cited page ? Try the MLA format generator on EasyBib.com! We also have a guide on how to format an APA reference page .

2. Understand the General Formatting Rules of Your Citation Style Before You Start Writing

While reading up on paper formatting may not sound exciting, being aware of how your paper should look early on in the paper writing process is super important. Citation styles can dictate more than just the appearance of the citations themselves, but rather can impact the layout of your paper as a whole, with specific guidelines concerning margin width, title treatment, and even font size and spacing. Knowing how to organize your paper before you start writing will ensure that you do not receive a low grade for something as trivial as forgetting a hanging indent.

Don’t know where to start? Here’s a formatting guide on APA format .

3. Double-check All of Your Outside Sources for Relevance and Trustworthiness First

Collecting outside sources that support your research and specific topic is a critical step in writing an effective paper. But before you run to the library and grab the first 20 books you can lay your hands on, keep in mind that selecting a source to include in your paper should not be taken lightly. Before you proceed with using it to backup your ideas, run a quick Internet search for it and see if other scholars in your field have written about it as well. Check to see if there are book reviews about it or peer accolades. If you spot something that seems off to you, you may want to consider leaving it out of your work. Doing this before your start making citations can save you a ton of time in the long run.

Finished with your paper? It may be time to run it through a grammar and plagiarism checker , like the one offered by EasyBib Plus. If you’re just looking to brush up on the basics, our grammar guides  are ready anytime you are.

How useful was this post?

Click on a star to rate it!

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Citation Basics

Harvard Referencing

Plagiarism Basics

Plagiarism Checker

Upload a paper to check for plagiarism against billions of sources and get advanced writing suggestions for clarity and style.

Get Started

This paper is in the following e-collection/theme issue:

Published on 28.5.2024 in Vol 26 (2024)

An Extensible Evaluation Framework Applied to Clinical Text Deidentification Natural Language Processing Tools: Multisystem and Multicorpus Study

Authors of this article:

Original Paper

• Paul M Heider 1 , PhD   ; 
• Stéphane M Meystre 2 , MD, PhD  

1 Biomedical Informatics Center, Medical University of South Carolina, Charleston, SC, United States

2 Institute of Digital Technologies for Personalised Healthcare (MeDiTech), University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland

Corresponding Author:

Paul M Heider, PhD

Biomedical Informatics Center

Medical University of South Carolina

22 WestEdge Street, Suite 200

Charleston, SC, 29403

United States

Phone: 1 843 792 3385

Email: [email protected]

Background: Clinical natural language processing (NLP) researchers need access to directly comparable evaluation results for applications such as text deidentification across a range of corpus types and the means to easily test new systems or corpora within the same framework. Current systems, reported metrics, and the personally identifiable information (PII) categories evaluated are not easily comparable.

Objective: This study presents an open-source and extensible end-to-end framework for comparing clinical NLP system performance across corpora even when the annotation categories do not align.

Methods: As a use case for this framework, we use 6 off-the-shelf text deidentification systems (ie, CliniDeID, deid from PhysioNet, MITRE Identity Scrubber Toolkit [MIST], NeuroNER, National Library of Medicine [NLM] Scrubber, and Philter) across 3 standard clinical text corpora for the task (2 of which are publicly available) and 1 private corpus (all in English), with annotation categories that are not directly analogous. The framework is built on shell scripts that can be extended to include new systems, corpora, and performance metrics. We present this open tool, multiple means for aligning PII categories during evaluation, and our initial timing and performance metric findings. Code for running this framework with all settings needed to run all pairs are available via Codeberg and GitHub.

Results: From this case study, we found large differences in processing speed between systems. The fastest system (ie, MIST) processed an average of 24.57 (SD 26.23) notes per second, while the slowest (ie, CliniDeID) processed an average of 1.00 notes per second. No system uniformly outperformed the others at identifying PII across corpora and categories. Instead, a rich tapestry of performance trade-offs emerged for PII categories. CliniDeID and Philter prioritize recall over precision (with an average recall 6.9 and 11.2 points higher, respectively, for partially matching spans of text matching any PII category), while the other 4 systems consistently have higher precision (with MIST’s precision scoring 20.2 points higher, NLM Scrubber scoring 4.4 points higher, NeuroNER scoring 7.2 points higher, and deid scoring 17.1 points higher). The macroaverage recall across corpora for identifying names, one of the more sensitive PII categories, included deid (48.8%) and MIST (66.9%) at the low end and NeuroNER (84.1%), NLM Scrubber (88.1%), and CliniDeID (95.9%) at the high end. A variety of metrics across categories and corpora are reported with a wider variety (eg, F 2 -score) available via the tool.

Conclusions: NLP systems in general and deidentification systems and corpora in our use case tend to be evaluated in stand-alone research articles that only include a limited set of comparators. We hold that a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons. Our open pipeline should reduce barriers to evaluation and system advancement.

Introduction

An ironclad pillar of clinical data reuse is the proper protection of protected health information (PHI). Deidentification is the process of tagging and removing personally identifiable information (PII) to prevent incidental privacy breaches. Unfortunately, manual deidentification is an expensive and error-prone process [ 1 , 2 ], and automated deidentification remains an unsolved challenge [ 3 , 4 ]. Since the first published automated deidentification system [ 5 ], a variety of systems using a range of technologies have been released. In tandem, a series of competitions have been organized around shared corpora annotated with PII to further encourage the development of deidentification systems [ 6 - 9 ]. Researchers publishing about a new system tend to release comparative performance metrics against 1 or 2 publicly available systems using 1 or 2 corpora [ 3 , 10 - 12 ]. These stand-alone research articles that only include a limited set of comparator systems and corpora cannot always even be directly compared to create a single meta-analysis because of the differences in annotation categories evaluated and reported on or how exactly matches are aligned and scored. Replication of the process to confirm results and extension of the process to evaluate new systems or corpora have been stymied by a combination of lack of documented evaluation code, closed evaluation tools, and incompatible PII categories, among other reasons.

As such, a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons between systems. A single pipeline allows researchers (and staff scientists responsible for evaluating systems before deployment) to consistently and reproducibly generate scores for a range of systems across a range of corpora using the exact same methods. Researchers with preferences to evaluate deidentification systems at the character level versus token level versus PII mention can run each evaluation in turn. Similarly, researchers can differentiate or collapse PII categories as desired across all systems and corpora at once to get a clear picture of how each system and corpus, respectively, treats different categories. Not all categories of PII are equally sensitive [ 3 ]. Patient names are more revealing than provider names, which are, in turn, more revealing than hospital names.

To that end, we evaluated PII extraction performance at multiple levels of granularity because these systems should not be judged on a single summary performance metric. For instance, this pipeline provides an easy means for surfacing the false negative rate or recall (also called sensitivity) for patient names as distinct from provider names (or other names). We can also easily compare performance when differences between PII categories are important to maintain, as opposed to when all PII categories are treated interchangeably.

As a case study, to help us understand the performance trade-offs of available deidentification systems that are critical to clinical data reuse and natural language processing (NLP) and to foster building larger repositories of directly comparable evaluation results, we developed a reusable and extensible pipeline for evaluating 6 off-the-shelf deidentification systems across 2 freely available corpora, 1 previously available corpus, and 1 private corpus. The systems and corpora all use English, although the pipeline is language agnostic. As none of the systems or corpora use exactly the same annotation schema, we provided mappings to allow approximately equitable performance metrics across all components. Any deidentification system programmatically runnable from the command line can be added to the set. Similarly, new corpora can be added to the evaluation process with minimal constraints on their format or annotation categories. Furthermore, the evaluation tool used in the pipeline allows for analysis at configurable levels of annotation category granularity and with multiple text annotation matching styles [ 13 , 14 ]. We focused on off-the-shelf systems for 3 primary reasons. First, off-the-shelf systems have the lowest barrier to entry. Not all potential users have the skills, capacity, or annotated corpora available to retrain a model. If a deidentification system developer considers their tool to be unusable without retraining, then that limitation should be made explicit, which brings us to the second reason. Undertaking the controlled experiments required to determine when a retrained system has met a reliable and safe performance threshold requires a test harness exactly like the one we propose here. Third, the potential variants for retraining any given system to optimize its performance for a local site constitutes its own large undertaking and falls outside the scope of this research. Furthermore, comparing the retrained variants to determine optimal performance is best organized using a test harness as described in this study.

Thus, we hold that a single evaluation pipeline across multiple systems and corpora allows for the nuanced comparisons required for safe deployment of NLP systems, in general, and deidentification systems, in particular. We document the use and extensibility of such a tool with deidentification as a use case. In summary, we found an order of magnitude difference between the fastest and slowest systems in terms of processing speed. We found that none of the 6 systems consistently outperformed the others across corpora and PII tag categories. A nuanced comparison of the top performers under slightly different conditions would be much more difficult without a cohesive framework like the one we describe.

Ethical Considerations

This study was assessed by the Medical University of South Carolina (MUSC) Institutional Review Board for Human Research (IRB) and officially considered Not Human Research. It was therefore not subject to oversight by the MUSC IRB, since it met the criteria set forth by the Code of Federal Regulations (45CFR46): (1) the data were not collected specifically for the currently proposed research project through an interaction or intervention with living individuals and (2) investigators including collaborators on the proposed research cannot readily ascertain the identity of the individuals to whom the coded private information of specimens pertains.

The other three corpora used in this study (ie, 2006 i2b2 shared task, 2014 i2b2 and University of Texas Health Science Center at Houston (i2b2/UTHealth), and 2016 Centers of Excellence in Genomic Science Neuropsychiatric Genome-Scale and RDOC Individualized Domains [CEGS N-GRID] shared task) were publicly available and already deidentified, and therefore not subject to IRB approval requirements.

PII Categories

In the United States, HIPAA (Health Insurance Portability and Accountability Act) is the primary legal mandate guiding and governing data privacy and security provisions within the health care domain [ 15 ]. Other regions of the world have enacted similar privacy laws such as the General Data Protection Regulation in the European Union [ 16 ], although they are not all specific to the health care industry, as evidenced by the General Data Protection Regulation. The HIPAA privacy rule specifies sensitive classes of PII that should be removed for the data to be considered deidentified. In this study, we focus on the subset of HIPAA’s 18 categories of PII relevant to unstructured clinical notes, as all our data sets were created in the United States under the jurisdiction of HIPAA. We have listed a curated and organized set of these categories in Figure 1 in the 4 left-most columns. The first column indicates the original HIPAA Safe Harbor category names. The following 3 columns are curated classes to help group or simplify the categories. We call these the tier 0 category (which represents the general class of PII), the tier 1 categories (which represent 7 high-level sets of categories grouped by semantic domain and common textual realizations), and tier 2 categories (which represent the finest-grained division of categories approximating the original HIPAA categories). The tier 2 categories include several practical and functional extensions of the strict HIPAA categories that have been treated as PII by deidentification researchers in the clinical domain. For instance, HIPAA considers ages >89 years to be PII but not the not younger ages, while some researchers consider any age to be PII. Additional columns in this figure represent categories annotated in deidentification corpora (columns 5-7) and categories flagged by deidentification systems (columns 8-13). Rows across the figure indicate rough equivalency of categories between tiers, corpora, or systems. Specific details of corpus and system categories are covered in the next two subsections: Deidentification Shared Tasks and Corpora and Deidentification Systems . Perfect category synchrony between corpora and annotations is impossible. We have included a sample sentence in Figure 2 , cross-annotated according to the specifics of each category schema to help highlight the local variation between corpora and deidentification systems.

Deidentification Shared Tasks and Corpora

Several shared tasks organized around the goal of evaluating deidentification systems have been organized within the clinical domain. The 2006 Informatics for Integrating Biology and the Bedside (i2b2) shared task was the first such task and focused on a small set of PII categories present in unstructured clinical notes written in English [ 6 ]. A similar shared task was organized as part of the 2012 NII Test Beds and Community for Information Access Research Medical Natural Language Processing task for fabricated but realistic medical reports written in Japanese [ 7 ]. Two more English language tasks were organized with an array of PII categories that were more representative of the full list of HIPAA categories as part of the 2014 i2b2 and UTHealth shared task [ 8 ] and 2016 CEGS N-GRID shared task [ 9 ].

We used each of the 3 English language deidentification shared task corpora for our study as they have been used as the standard reference corpora in the domain. The 2006 and 2014 corpora are publicly available with an appropriate data use agreement. The 2016 corpus was publicly available but has since been removed from circulation due to privacy concerns. The fourth corpus, also in English, is called the “Medical University of South Carolina (MUSC)” corpus in this study as it was developed at MUSC by the authors. It is not publicly available as its PII has only been annotated and not redacted or resynthesized to prevent the release of PHI.

We refer to the oldest corpus in our study as “2006” as it was used for the 2006 i2b2 shared task [ 6 ]. It consists of 889 discharge summaries from Partners HealthCare. This corpus has the most reduced set of annotation categories with 8 distinct categories: location, hospital, phone, doctor, patient, medical record, age (>89 years), and date, as shown in Figure 1 . Of note, the tier 1 category that we call “Address” maps to the 2006 category “Location.” “Medical Records” are the only recorded tier 1 “Identifiers.” “Age” is confined by the strict HIPAA notion to those >89 years, in contrast to the other 3 corpora that treat all ages as PII.

Internally, we aligned the annotation categories for the 2014 i2b2 and UTHealth [ 8 ] and the 2016 CEGS N-GRID shared tasks [ 9 ] on deidentification, which we refer to as “2014” and “2016,” respectively. In contrast with 2006, the tier 1 Address and Identifiers categories for 2014 and 2016 are split into fine-grained categories. Overall, there are 28 distinct categories, as shown in Figure 1 . The 2014 corpus consists of 1304 discharge summaries and correspondences between providers from Partners HealthCare. The 2016 corpus consists of 1000 psychiatric notes from Partners HealthCare.

The MUSC corpus consists of 728 notes split across 8 note types: consults, discharge summaries, history and physicals, nursing, pathology and cytology, patient instructions, plan of care, and progress notes. Similar to the 2014 and 2016 corpora, the MUSC corpus includes a wide range of tier 1 “Identifiers” but groups all of them except social security numbers into 1 class: “Other ID.” Due to a change in the annotation guidelines over the course of the corpus annotation, some “Street” annotations and “City” annotations have been merged into a single “StreetCity” annotation. Similarly, some “State” annotations and “Country” annotations have been merged into “StateCountry” annotations.

All 4 corpora have been divided into train and test splits. None of the systems have used the official test splits for training. Therefore, we report all performance metrics with respect to the official test split in the main body of this paper. Multimedia Appendix 1 contains results for both train and test splits and Multimedia Appendix 2 [ 2 , 11 , 17 - 32 ] contains de-identification system details. All reported timing results include both train and test splits.

Deidentification Systems

Our initial set of systems was constrained by an implementation science framing of the problem. As researchers invested in reproducible science, we need to understand the best tools for facilitating ethical data sharing, but our primary research area may not be related to deidentification. We need a resource for evaluating scalable, off-the-shelf systems that do not require additional model training or fine-tuning. Concretely, this framing restricts us to freely available systems that can be run programmatically on a local server without any requirements to annotate a local data set. A total of 6 systems meet these requirements: CliniDeID (version 1.6.1) [ 17 , 18 ], deid by Massachusetts Institute of Technology (version 1.1, now available via PhysioNet) [ 2 , 19 ], MITRE Identity Scrubber Toolkit (MIST; version 2.0.4) [ 20 - 22 ], NeuroNER (commit 3817fea on GitHub) [ 23 , 24 ], National Library of Medicine Scrubber (version 19.0403L Linux ×86 64) [ 25 - 27 ], and Philter (commit 780da99 on GitHub) [ 11 , 28 ]. All 6 of these systems run on English clinical notes, read in files from disc, write annotated files indicating identified PII mentions to disc, can be run from the command line on a Linux server, and have been run using the latest available version. System reporting will be in alphabetical order. Refer to Multimedia Appendix 2 for more details on each system’s configuration and use.

We evaluated systems in terms of both timing and binary classification performance metrics. For timing purposes (reported in the Timing Results subsection), each system processed 1 corpus at a time on the same Red Hat Linux server. This evaluation pipeline was the only program beyond routine background processes running on the machine. We used the real (ie, wall clock) time generated by the command line tool time. For performance metrics (ie, encompassing the 3 Performance Results subsections in the Results section), we used the Evaluation Tool for Unstructured Data and Extractions (ETUDE) to score each system by corpus pairing. ETUDE uses definitions from a configuration file for each of the reference and system outputs to determine how each “native” annotation category is represented in the corpus. These native categories are then mapped in the same configuration file to “scoring value” categories. Drawing from an example depicted in Figure 1 , the 2016 native categories of “Phone” and “Fax” are mapped in the 2016 configuration file to the “PhoneFax” scoring value category for tier 1 entries. The configuration files allow us to separate the logic for annotation extraction from annotation alignment matching.

ETUDE also provides several alignment matching algorithms for determining which set of annotations are considered a match between the reference and system outputs. We focused on 3 of these matching algorithms for this study: “exact,” “partial,” and “fully contained.” Exact matching requires that the character offsets of 2 annotations be the same to count as a match. Partial matching only requires that any part of the 2 annotations overlap to be considered a match. Finally, fully contained matching requires the system output annotation to at least cover the entirety of the reference annotation to count as a match. The system output annotation can include more text before or after the extent of the reference annotation but not less. The intuition relevant to deidentification is that if the system PII output annotation fully contains the reference PII annotation, then we know that no PII is leaking, as is potentially possible for a partial match of annotations.

The final evaluation feature we used in ETUDE was the ability to collapse all patterns into a single category for evaluation purposes. Thus, for some of our evaluations, we tracked both annotation and category. For other evaluations, we only tracked the annotations and ignored the differences in annotation category.

In the end, ETUDE generated counts for true positives, false positives, and false negatives. From these counts, we calculated precision, recall, and F 1 -score values (ie, the harmonic mean of precision and recall). Given the sensitive nature of deidentification, we give primacy to recall in our reporting in the main body of this paper but include all 3 values in Multimedia Appendix 1 . The F 2 -score, which gives more weight to recall than precision, can also be generated by ETUDE.

Processing Pipeline

Expanding on the work of our previous comparison of 3 deidentification systems [ 33 ], we developed a larger and more flexible pipeline for using a set of off-the-shelf deidentification systems to process a set of corpora and then scoring all system output at a range of PII category granularities, as shown in the block diagram in Figure 3 . The core of this pipeline rests on command line shell scripts with configurable custom functions for processing any given corpus with a given deidentification system. Specifically, the core shell script uses environment variables to set input and output folders for each corpus, running folders and Python environments for each deidentification system, and flags for which set of corpora and systems should run on any given instantiation. The processed output for a system and corpus combination is written to disc in a given folder, allowing for repeated evaluation loops by ETUDE [ 13 , 14 ], a freely available open-source tool developed by the first author. Each evaluation loop uses different configuration settings to highlight different tiers of categories, different alignments of categories between reference and system, and different annotation alignment algorithms. Adding a new deidentification system requires adding a few simple shell commands to preprocess files (as expected by the system), to run the system from the command line with all parameters fully specified, and to postprocess files (if they are not in a format already supported by ETUDE). A new corpus or deidentification system may require creating a new annotation schema mapping file, if the schema is not already covered by those shown in Figure 1 . As ETUDE, all shared task corpora, and all deidentification systems are already publicly available, we also released our shell scripts and R-based evaluation scripts on Codeberg and GitHub to allow for near-complete reproducibility of this study [ 34 , 35 ]. System-specific settings, in terms of explicit configuration files or command line flags and settings, are also included in these repositories.

Timing Results

The 8 timed trials for each system (ie, 4 corpora with 1 train and 1 test split) are summarized in Table 1 . We report both the seconds per note (for which a lower value is better) and notes per second (for which a higher value is better). MIST is the fastest with an average of 24.57 (SD 26.23) notes per second and is an order of magnitude faster than all systems except Scrubber, which averages 8.56 (SD 3.54) notes per second. CliniDeID is the slowest with an average of 1 (SD 0.38) note per second. Deid, NeuroNER, and Philter are all marginally faster at an average of 1.41 (SD 0.60), 1.28 (SD 0.48), and 1.36 (SD 0.64) notes per second, respectively.

a Each data point reflects a single corpus split (eg, the 2006 train split vs the 2006 test split).

Performance Results Across All Categories Based on Partial Matching

For our initial performance metric evaluation, we focused on each system’s overall ability to identify PII, regardless of the category. In the terminology of ETUDE, we collapsed evaluation across all categories. Figure 4 presents the recall, precision, and F 1 -scores for each system against each corpus using the partial match alignment (this is the most generous evaluation possible).

The u-shaped curves for CliniDeID and Philter indicate that both these systems prioritize recall over precision. In contrast, the other 4 systems consistently have higher precision scores than recall scores. CliniDeID and NeuroNER show the highest scores across all corpora. Deid and MIST have the largest variance between corpora.

Performance Results for Specific Tier 1 Categories Based on Partial Matching

As different categories of PII are differentially sensitive, we wanted to delve deeper into the specific performance of systems with respect to the 7 tier 1 categories shown in Figure 1 . For this analysis, we report the recall for partial annotation matching in Figure 5 . Precision and F 1 -score values are available in Multimedia Appendix 1 . However, the nature of the limited categories generated by deid and Philter means that precision values for these systems are not truly meaningful.

CliniDeID shows more between-corpus variance than between-category variance and overall performs most consistently well. NeuroNER performs the next best across all categories, although “Contact Information,” “Names,” and “Time” are clearly better identified than the other 4 categories. Philter shows the same facility with “Contact Information” and “Names.” Deid shows a large between-corpus variance but is generally consistent across categories. “Occupations” is not included as deid, MIST, and Scrubber do not extract that category. MIST identifies “Address” and “Contact Information” categories at a lower rate than other categories. Similarly, Scrubber identifies “Address” categories at a lower rate and does not extract any “Contact Information” annotations.

Performance Results for Names Based on Fully Contained Matching

As names are perhaps the most sensitive and unique PII category, we further analyzed the tier 1 “Names” category and the tier 2 categories of “Patient” and “Provider” for those systems that distinguish between these categories. For this analysis, we used the stricter fully contained annotation matching algorithm. Figure 6 summarizes the recall, precision, and F 1 -score values across systems and corpora.

No strong corpus-specific trends show up in these results. MIST and NeuroNER have worse recall for identifying patient names than provider names, while CliniDeID does not appear to treat them differently. MIST and NeuroNER also have worse recall than precision for patient names but, similar to CliniDeID, no obvious difference between the 2 metrics for provider names.

The gap between recall and precision is reduced for MIST and NeuroNER at the tier 1 “Names” level, which implies that some (but not all) of the tier 2 performance issues are due to patient names being incorrectly flagged as provider names. CliniDeID is consistently high for both recall and precision at tier 1. The deid system performs the worst for this evaluation, although its scores for “Names” are not so low for the partial matching evaluation in the Performance Results for Specific Tier 1 Categories subsection. This performance discrepancy indicates that deid does extract names reliably but does not extract large enough annotations to fully identify the relevant PII. Scrubber and Philter show much higher recall than precision for tier 1 “Names.” As noted previously, Philter’s precision for this evaluation is expected to be very low as the native system category relevant to “Names” includes all PII categories except dates. It is not obvious why Scrubber’s precision is low.

As expected, given the complexity of the deidentification domain and feature variability in deidentification tools, we found that that none of the 6 systems uniformly outperformed the others across corpora and PII tag categories. CliniDeID, NeuroNER, and Philter generally outperformed the older (and not actively maintained) systems, but they also each have their performance drawbacks or limitations. Our evaluation framework was helpful in identifying these strong and weak facets, which, in turn, helped us identify additional aspects to include in future releases of the framework, as discussed in the next subsection.

Principal Findings

The principal findings of this study fall into 3 major classes: the evaluation pipeline implementation, the category mapping for bridging analyses across corpora or deidentification systems, and the evaluation results for the 6 off-the-shelf systems. The evaluation pipeline implementation includes many reusable components, including the shell scripts for passing corpus notes through systems, the ETUDE configuration scripts for scoring system outputs against reference annotations, and R scripts for plotting results of these analyses. The code for replication of the workflow is available via Codeberg and GitHub public code repositories [ 34 , 35 ]. This code base can be used to increase reproducibility between studies on deidentification systems, lower the barrier to fine-tuning or retraining systems, and increase portability of evaluation across data sets and systems. The ETUDE configuration scripts provide a means for encoding and leveraging the PII category cross-mapping displayed in Figure 1 . The cross-mapping can also be used to help compare the coverage of different deidentification systems with an eye to identifying the validity of different deidentification systems for different use cases (eg, when some categories of PII are known a priori to be either absent from or very frequent in a data set). Finally, our analysis of the text deidentification systems themselves yielded mixed results. CliniDeID and Philter had the highest recall for select categories, while CliniDeID and NeuroNER had the highest recall across all categories. We also found large differences in performance when comparing partial matches for PII categories to fully contained matches. The tabular data used to generate all tables and figures in this study have been included in Multimedia Appendix 1 to allow for finer-grained evaluation of the results.

Limitations

As discussed in the Introduction section, we focused our analysis on off-the-shelf models. CliniDeID, MIST, and NeuroNER can be retrained, while National Library of Medicine Scrubber and Philter have dictionaries that can be customized to a local environment. Thus, while our framework provides a means to easily measure the average baseline performance (across multiple corpora), more work is required to ascertain the average potential performance for each system (after retraining). Fortunately, each retrained model would only require a limited pass through the evaluation framework but would not require reprocessing and reevaluating all systems against all corpora. To further facilitate this use case, future iterations of this tool will make it even easier to add new corpora, systems, and evaluation configurations. Specifically, a more robust implementation of the pipeline would leverage reproducible workflow tools such as the targets R package [ 36 ]. We have released a related implementation using the targets R package for evaluating algorithmic bias in deidentification systems [ 37 ]. Xiao et al [ 38 ] provide an alternate approach using 100 synthetic templates imitating realistic contexts for PII as it occurs in unstructured clinical notes. For researchers who do not have the infrastructure or privacy needs of data sets with PII, other reproducible pipelines and shared evaluation frameworks, such as NLP Sandbox, exist [ 39 ].

In addition, we have not included all the essential evaluations for deciding between deidentification systems and will continue to expand our result set. For instance, we did not account for system start-up time. A system that takes 1 second per note for 1 or 10 million notes has different usability than a system that takes 0.1 second per note after loading models for 10 minutes. Another evaluation complication is how to equitably compare systems with significantly different input or output features. For instance, CliniDeID resynthesizes annotations by replacing the original PII with similar surrogate values, which likely slows down the overall pipeline compared to a system such as deid that does not track PII category in a manner accessible to a standard user. Similarly, TiDE (Text DEidentification) [ 40 ] meets all the aforementioned system requirements but uses prefilled PII details to boost performance. That is, when running a note through TiDE, one can provide the system with the patient’s name, known providers, and other relevant PII that are likely to show up in a note. While this seems like a good plan at the system level, we need to develop a more complex evaluation pipeline to accommodate for the given information and to disaggregate results based on whether TiDE has correctly annotated known PII versus unknown PII (eg, a patient’s nickname or misspelled name and a relative’s name). TiDE was not evaluated and compared in this study for this reason, among others.

Finally, we relied on the mapping in Figure 1 to perform our evaluations, which, in turn, relies on a manual mapping built using experts’ knowledge. Kim et al [ 41 ] evaluated a successful automated category mapping algorithm on deidentification concepts, which could be used as an alternate means for cross-mapping categories between corpus annotations and system outputs.

Comparison With Prior Work

Previous studies have included many of the same individual components that we present as novel in this study but, to the best of our knowledge, none have included all the components together. For instance, a category mapping between the 2014 and 2016 data sets was released at the time of the canonical paper on the 2016 shared task [ 9 ]. That same year, the researchers who developed Philter [ 42 ] released a mapping between the HIPAA Safe Harbor categories, the i2b2 2014 corpus annotations, and the categories redacted in the public release of the Multiparameter Intelligent Monitoring in Intensive Care II corpus [ 43 , 44 ]. The mapping we present in Figure 1 includes more corpora and extends the mapping to the unique set of categories used by each deidentification system, which greatly extends the utility of this new mapping for the community as a whole. Furthermore, our evaluation tool (ie, ETUDE), along with the published configuration files, provides the means to easily evaluate different category groupings without reannotating the data sets.

Most prior studies do not explicitly mention details of the evaluation (eg, whether they use token counts and exact match character offsets) unless they are using the standard evaluation script released with the 2016 corpus [ 9 ]. While the widespread use and public availability of this script are a boon to reproducible science, the script’s limited options, hardcoded categories, and fixed file format are a bane. Most prior studies report the precision, recall, and F 1 -score, while a select few report the F 2 -score [ 11 , 45 ] or both [ 46 ]. We reported the F 1 -score, but ETUDE can easily be configured to report the F measure for any arbitrary β value. The relative processing speeds ( Table 1 ) for deidentification systems is also rarely reported [ 33 ]. A few prior studies have included specific annotation guidelines, including examples and edge cases, to help clarify the underlying ground truth intended by category labels [ 8 , 25 , 26 ]. Finally, our own work and many of the other researchers and developers cited in this study have relied on the same standard deidentification data sets released via the i2b2 or CEGS N-GRID shared tasks. For less common but still publicly available data sets, the developers of NeuroNER evaluated their system against the CoNLL 2003 shared task on named entity recognition [ 23 ], and Xiao et al [ 38 ] recently released a new smaller set of notes based on MIMIC-IV. We are not aware of any other study with a publicly released evaluation framework that includes all steps from initial corpus processing through plotting evaluation results.

Conclusions

We present a reusable and extensible evaluation framework applied to deidentification systems for clinical unstructured notes. We release the tool for other researchers to reduce the overhead in testing new systems and new corpora, a critical and common task within the clinical NLP community, especially for researchers who wish to ethically share data without leaking PHI.

From this initial case study, we found an order of magnitude difference in terms of processing speed between the fastest and slowest systems and that no single system out of the 6 uniformly outperformed the others across corpora and PII categories. Instead, a richer tapestry of recall and precision trade-offs emerged for different PII categories and groups of PII categories. We hold that a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons between systems and serves as a boon to the clinical NLP community.

Acknowledgments

This publication was supported, in part, by the National Center for Advancing Translational Sciences of the National Institutes of Health (UL1 TR001450) and the SmartState Endowment for Translational Biomedical Informatics. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Data Availability

The 2006 and 2014 corpora analyzed during this study are available in the National NLP Clinical Challenges website repository [ 47 ]. The 2016 corpus analyzed during this study is not publicly available due to privacy concerns. The Medical University of South Carolina corpus is not publicly available due to privacy concerns, as the data set has not been fully redacted or resynthesized to remove personally identifiable information. Code for replication of the workflow is available via Codeberg and GitHub public code repositories [ 34 , 35 ].

Authors' Contributions

PMH and SMM made substantial contributions to the design and implementation of the research and to the analysis of the experimental results. All authors drafted the work or revised it critically. PMH drafted the initial manuscript. SMM performed critical revision of the manuscript. All authors provided final approval for the version to be published. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflicts of Interest

CliniDeID, one of the systems evaluated in this manuscript, was initially developed and marketed as a commercial solution but is not commercialized anymore and has been released as free and open-source software since 2023.

CSV data files used to generate figures.

Details on each of the 6 deidentification system’s release information (eg, version number), configuration, and use.

• Douglass MM, Cliffford GD, Reisner A, Long WJ, Moody GB, Mark RG. De-identification algorithm for free-text nursing notes. In: Proceedings of the Computers in Cardiology. 2005. Presented at: CIC 2005; September 25-28, 2005; Lyon, France. [ CrossRef ]
• Neamatullah I, Douglass MM, Lehman LW, Reisner A, Villarroel M, Long WJ, et al. Automated de-identification of free-text medical records. BMC Med Inform Decis Mak. Jul 24, 2008;8:32. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Steinkamp JM, Pomeranz T, Adleberg J, Kahn CEJ, Cook TS. Evaluation of automated public de-identification tools on a corpus of radiology reports. Radiol Artif Intell. Oct 14, 2020;2(6):e190137. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chen F, Bokhari SM, Cato K, Gürsoy G, Rossetti SC. Examining the generalizability of pretrained de-identification transformer models on narrative nursing notes. Appl Clin Inform. Mar 06, 2024. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sweeney L. Replacing personally-identifying information in medical records, the Scrub system. Proc AMIA Annu Fall Symp. 1996:333-337. [ FREE Full text ] [ Medline ]
• Uzuner O, Luo Y, Szolovits P. Evaluating the state-of-the-art in automatic de-identification. J Am Med Inform Assoc. 2007;14(5):550-563. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morita M, Kano Y, Ohkuma T, Miyabe M, Aramaki E. Overview of the NTCIR-10 MedNLP task. In: Proceedings of the 10th NTCIR Conference. 2013. Presented at: NTCIR 2013; June 18-21, 2013; Tokyo, Japan.
• Stubbs A, Kotfila C, Uzuner Ö. Automated systems for the de-identification of longitudinal clinical narratives: overview of 2014 i2b2/UTHealth shared task Track 1. J Biomed Inform. Dec 2015;58 Suppl(Suppl):S11-S19. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stubbs A, Filannino M, Uzuner Ö. De-identification of psychiatric intake records: overview of 2016 CEGS N-GRID shared tasks Track 1. J Biomed Inform. Nov 2017;75S:S4-18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murugadoss K, Rajasekharan A, Malin B, Agarwal V, Bade S, Anderson JR, et al. Building a best-in-class automated de-identification tool for electronic health records through ensemble learning. Patterns (N Y). May 12, 2021;2(6):100255. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Norgeot B, Muenzen K, Peterson TA, Fan X, Glicksberg BS, Schenk G, et al. Protected Health Information filter (Philter): accurately and securely de-identifying free-text clinical notes. NPJ Digit Med. Apr 14, 2020;3:57. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu L, Perez-Concha O, Nguyen A, Bennett V, Jorm L. De-identifying Australian hospital discharge summaries: an end-to-end framework using ensemble of deep learning models. J Biomed Inform. Nov 2022;135:104215. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Heider PM, Accetta JK, Meystre SM. ETUDE for easy and efficient NLP application evaluation. In: Proceedings of the AMIA NLP-WG Pre-Symposium. 2018. Presented at: AMIA NLP-WG Pre-Symposium 2018; November 3, 2018; San Francisco, CA.
• Heider PM, Accetta JK, Meystre SM. ETUDE: demonstrating multiple matching styles and offset calculators within a natural language processing evaluation tool. In: Proceedings of the 17th World Congress on Medical and Health Informatics. 2019. Presented at: MEDINFO 2019; August 25-30, 2019; Lyon, France.
• 45 CFR parts 160 and 164. U.S. Department of Health and Human Services. URL: https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/introduction/index.html [accessed 2024-05-01]
• General data protection regulation compliance guidelines. EU General Data Protection Regulation. URL: https://gdpr.eu/ [accessed 2024-05-01]
• Meystre SM, Heider PM, Kim Y, Trice A, Underwood G. Clinical text automatic de-identification to support large scale data reuse and sharing: pilot results. In: Proceedings of the American Medical Informatics Association Annual Symposium. 2018. Presented at: AMIA 2018; November 3-7, 2018; San Francisco, CA.
• Clinacuity / CliniDeID. GitHub. URL: https://github.com/Clinacuity/CliniDeID [accessed 2024-04-08]
• De-identification software package. PhysioNet. Dec 18, 2007. URL: https://physionet.org/content/deid/1.1/ [accessed 2024-04-09]
• Aberdeen J, Bayer S, Yeniterzi R, Wellner B, Clark C, Hanauer D, et al. The MITRE identification scrubber toolkit: design, training, and assessment. Int J Med Inform. Dec 2010;79(12):849-859. [ CrossRef ] [ Medline ]
• MITRE identification scrubber toolkit files. SourceForge. URL: https://sourceforge.net/projects/mist-deid/files/ [accessed 2024-04-08]
• MIST: the MITRE identification scrubber toolkit (documentation). SourceForge. URL: https://mist-deid.sourceforge.net/docs_2_0_4/html/index.html [accessed 2024-04-08]
• Dernoncourt F, Lee JY, Szolovits P. NeuroNER: an easy-to-use program for named-entity recognition based on neural networks. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2017. Presented at: EMNLP 2017; September 9-11, 2017; Copenhagen, Denmark. [ CrossRef ]
• Franck-Dernoncourt / NeuroNER. GitHub. URL: https://github.com/Franck-Dernoncourt/NeuroNER [accessed 2024-04-08]
• Kayaalp M, Browne AC, Sagan P, McGee T, McDonald CJ. Challenges and insights in using HIPAA privacy rule for clinical text annotation. AMIA Annu Symp Proc. Nov 05, 2015;2015:707-716. [ FREE Full text ] [ Medline ]
• Kayaalp M, Sagan P, Browne AC, McDonald CJ. Guidelines for annotating personal identifiers in the clinical text repository of the National Institutes of Health (version 6/28/2016). Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, National Institutes of Health. 2016. URL: https://lhncbc.nlm.nih.gov/scrubber/annotation.html [accessed 2022-12-23]
• NLM-scrubber downloads. National Institutes of Health National Library of Medicine. URL: https://lhncbc.nlm.nih.gov/scrubber/download.html [accessed 2024-04-08]
• BCHSI/philter-ucsf. GitHub. URL: https://github.com/BCHSI/philter-ucsf [accessed 2024-04-08]
• Carrell D, Malin B, Aberdeen J, Bayer S, Clark C, Wellner B, et al. Hiding in plain sight: use of realistic surrogates to reduce exposure of protected health information in clinical text. J Am Med Inform Assoc. 2013;20(2):342-348. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pennington J, Socher R, Manning C. GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014. Presented at: EMNLP 2014; October 25-29, 2014; Doha, Qatar. [ CrossRef ]
• Stenetorp P, Pyysalo S, Topić G, Ohta T, Ananiadou S, Tsujii J. brat: a web-based tool for NLP-assisted text annotation. In: Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. 2012. Presented at: EACL 2012; April 23-27, 2012; Avignon, France.
• MUSC-TBIC/corpus-utils. GitHub. URL: https://github.com/MUSC-TBIC/corpus-utils [accessed 2020-07-27]
• Heider PM, Obeid JS, Meystre SM. A comparative analysis of speed and accuracy for three off-the-shelf de-identification tools. AMIA Jt Summits Transl Sci Proc. May 30, 2020;2020:241-250. [ FREE Full text ] [ Medline ]
• Heider PM. HeiderLab/article-addenda. Codeberg. URL: https://codeberg.org/HeiderLab/article-addenda [accessed 2023-10-04]
• MUSC-TBIC/article-addenda. GitHub. URL: https://github.com/MUSC-TBIC/article-addenda [accessed 2023-10-04]
• Landau WM. The targets R package: a dynamic Make-like function-oriented pipeline toolkit for reproducibility and high-performance computing. J Open Source Softw. Jan 15, 2021;6(57):2959. [ CrossRef ]
• Heider PM. Algorithmic bias in de-identification tools. In: Proceedings of the IEEE 11th International Conference on Healthcare Informatics. 2023. Presented at: ICHI 2023; June 26-29, 2023; Houston, TX. [ CrossRef ]
• Xiao Y, Lim S, Pollard TJ, Ghassemi M. In the name of fairness: assessing the bias in clinical record de-identification. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 2023. Presented at: FAccT '23; June 12-15, 2023; Chicago, IL. [ CrossRef ]
• NLP sandbox. GitHub. URL: https://github.com/nlpsandbox [accessed 2020-07-27]
• TiDE text deid. STAnford Medicine Research data Repository. URL: https://starr.stanford.edu/methods/tide-text-deid [accessed 2023-10-02]
• Kim Y, Heider PM, Meystre SM. Automated category alignment applied to different de-identification annotation schemata. In: Proceedings of the AMIA 2021 Virtual Informatics Summit. 2021. Presented at: AMIA 2021; March 22-25, 2021; Online.
• Dernoncourt F, Lee JY, Uzuner O, Szolovits P. De-identification of patient notes with recurrent neural networks. J Am Med Inform Assoc. May 01, 2017;24(3):596-606. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation. Jun 13, 2000;101(23):E215-E220. [ CrossRef ] [ Medline ]
• Saeed M, Villarroel M, Reisner AT, Clifford G, Lehman LW, Moody G, et al. Multiparameter intelligent monitoring in intensive care II: a public-access intensive care unit database. Crit Care Med. May 2011;39(5):952-960. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferrández O, South BR, Shen S, Friedlin FJ, Samore MH, Meystre SM. Evaluating current automatic de-identification methods with Veteran's health administration clinical documents. BMC Med Res Methodol. Jul 27, 2012;12:109. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferrández O, South BR, Shen S, Friedlin FJ, Samore MH, Meystre SM. BoB, a best-of-breed automated text de-identification system for VHA clinical documents. J Am Med Inform Assoc. Jan 01, 2013;20(1):77-83. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Data sets. National NLP Clinical Challenges (n2c2). URL: https://n2c2.dbmi.hms.harvard.edu/data-sets [accessed 2022-03-08]

Abbreviations

Edited by G Tsafnat; submitted 20.12.23; peer-reviewed by C Taridzo, L Radhakrishnan; comments to author 07.03.24; revised version received 11.04.24; accepted 13.04.24; published 28.05.24.

©Paul M Heider, Stéphane M Meystre. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.