Harnessing policy to promote inclusive medical product evidence: development of a reference standard and structured audit of clinical trial diversity policies

Defining a reference standard for commitments to diversity in clinical trials

We established a reference standard for policies on diversity in clinical trials by conducting a content analysis of policy guidance from the Pharmaceutical Research and Manufacturers of America (PhRMA), International Federation of Pharmaceutical Manufacturers and Associations (IFPMA), Biotechnology Industry Organization, International Committee of Medical Journal Editors, FDA, European Medicines Agency, and the World Health Organization, up to 15 May 2023.16–22 Guidance documents were reviewed for recurring themes on recommendations for sponsors' policies, procedures, and practices. A data structure was then developed to reflect a best practices standard of distinct thematic commitments to diversity in clinical trials.

We used a dual review process. Two researchers (JM and WP) independently analyzed each guidance document by conducting several readings of the texts to become familiar with their scope and identify key themes. In agreement with standard content analysis methods,23 we extracted phrases or sentences with separate meaning units into an Excel file, and then condensed and categorized them into distinct themes describing recommended commitments to diversity in clinical trials (online supplemental appendix box 1). A codebook was created with all of the identified themes. Guidance documents were then reanalyzed applying the codebook to extract language from the documents responsive to each theme. Researchers met to agree on the final identification and categorization of themes for the reference standard. Unanimous agreement was reached.

Company policy review

Next, the same two researchers independently used the codebook to analyze 50 corporate clinical trial policies, extracting sentences or fragments from policies into Excel that were relevant to each theme in the codebook. Researchers met to agree, through consensus, on the presence or absence of specific themes in each policy.

The 50 pharmaceutical companies were selected from the top 500 pharmaceutical companies by market capitalization in 2021 from https://companiesmarketcap.com. From this sample, we included the 25 largest companies by market capitalizations. We also included an exploratory sample of 25 randomly selected companies from the remaining 475 companies, referred to here as non-large companies, using Google's random number generator.

For each company, we manually searched their website and annual reports, with the search function, for the presence of a publicly available policy on diversity in clinical trials with the key terms “diversity” or “inclusion” or “representation” or “equity” AND “clinical trial” or “clinical research,” or “research.” We also conducted a Google search with the same key words to identify company policies. Copies of all identified policy pages were archived as PDFs on 6 July 2023. Membership of PhRMA by sponsors was determined on 1 March 2023.

Data analysis

We used descriptive statistics to summarize the proportion of guidance documents and corporate policies with distinct thematic commitments to diversity in clinical trials established in our reference standard. We used a Fisher exact test to assess whether company size, categorized as large versus non-large, was associated with a company having a public diversity policy. We used a χ² test to assess whether the location of the headquarters of the company was associated with having a public policy on diversity in clinical trials. Analyses were conducted in Microsoft Excel, version 15.11 (Redmond, WA).

Patient and public involvement

Patients and the public were not directly engaged in the conduct of this research study. Our thematic focus of diversity and fair inclusion in clinical research, however, has been identified by patients and the public as essential for developing a just and equitable research ecosystem.24 We aim to engage patients, patient groups, and the public in the next phase of this project, using modified Delphi methods to gather perceptions on which of the leading indicators identified in this study are most likely to help improve diversity and representation in clinical trial enrollment.

Defining a reference standard for commitments to diversity in clinical trials

We identified 14 themes in FDA and stakeholder guidance for improving diversity in clinical trials, which we built into a reference standard. The most common themes, appearing in five guidance documents, were recommendations for sponsors to: use _targets for trial enrollment that reflected the incidence, prevalence, or severity of the condition or disease in various populations _targeted by the trial; use broad eligibility criteria in trial protocols when scientifically appropriate; and identify and remedy barriers to trial recruitment and retention to diversify participation in trials. Facilitators for recruiting and retaining diverse patient populations recommended in the guidance included providing language access for participants with limited English proficiency, consideration of paying participants, reducing the frequency of required study visits, adoption of electronic communications and digital health technology tools to replace site visits, use of decentralized trials, and partnerships with community organizations in trial recruitment (online supplemental appendix box 2).

Four guidance documents recommended that sponsors should commit to increasing diversity in the workforce to improve participation in clinical trials. Four documents also recommended incorporating patient input and experiences into the design of the trial and product development, and conducting community and patient outreach and engagement to increase awareness of trial opportunities and health literacy. Also, four documents suggested that sponsors should adopt a multidimensional approach to diversity beyond sex, age, race, and ethnic group, to include other variables, particularly ancestry, gender, disability, pregnancy and lactation status, comorbidities, geography, socioeconomic status, and access to healthcare.

Two guidance documents recommended that sponsors should identify and use site locations for the trials with diverse providers who treat underserved or under-represented populations, collect data after product approval to enhance drug safety and efficacy data for diverse populations, when needed, and adopt a global perspective to diverse enrollment. Sponsors were recommended to implement policies and practices within their own organizations to support diverse enrollment into studies in every country where they conduct clinical trials or where the _targeted disease burden is high. Two documents also recommended that diversity efforts should include transparent reporting of the personal characteristics of participants in the trial in publications and use new trial designs to facilitate broader population enrollment.

Of the reviewed guidance documents, the FDA guidance published in 2020 had the most commitments (10/14, 71%), which uniquely recommended that sponsors should offer expanded access programs for patients who cannot participate in trials because of trial eligibility criteria and other reasons. IFPMA's guidance published in 2022 had nine of the 14 commitments (64%) and uniquely recommended that diversity efforts should be a focus for all phases of clinical trials, not just for later stage trials or pivotal trials. PhRMA's guidance published in 2020 had eight of the 14 commitments (57%) and uniquely suggested that sponsors should make their policies on diversity in clinical trials, or information about such practices, publicly available on their corporate websites. WHO guidance also had eight of the 14 commitments (57%). Guidance from the International Committee of Medical Journal Editors, updated in 2023, had three commitments (3/14, 21%). Guidance from the European Medicines Agency did not directly deal with any of the 14 commitments. We could not find public guidance on diversity in clinical trials from Biotechnology Industry Organization (figure 1).

Request permission

Characteristics of sample sponsors

Of the 50 companies (25 large and 25 non-large) in our sample, we found that 56% (28/50) had their company headquarters in North America, 24% (12/50) in Europe and Central Asia, 14% (7/50) in East Asia Pacific, 4% (2/50) in South Asia, and 2% (1/50) in the Middle East and North Africa. No company had their headquarters in the Latin America or Caribbean region or in sub-Saharan Africa (online supplemental appendix table 1). Twenty (40%) of these 50 companies were members of PhRMA, comprising 67% of PhRMA's total membership (20/30). The median market capitalization in 2021 for large companies was about $111.3 (£87.8; €103.9) billion (interquartile range $60.3-240.6 billion) compared with $1.4 billion ($244 million to $8.5 billion) for non-large companies in our sample.

Policies on diversity in clinical trials

Overall, 52% (26/50) of companies had a publicly available policy on diversity in clinical trials. The average policy had five of the 14 thematic commitments (36%, range 0-8%) recommended in FDA and stakeholder guidance. Publicly available policies were more common for large companies (21/25, 84%) than non-large companies (5/25, 20%; P<0.001). The location of the company's headquarters, by region, was not associated with having a clinical trial diversity policy (North America: 13 of 28 (46%) companies had a policy; Europe and Central Asia: nine of 12 (75%) had a policy; and East Asia Pacific, Middle East, and North Africa, and South Asia: four of 10 (40%) had a policy; P=0.17).

Large companies

Most large companies (21/25, 84%) had a publicly available policy stating a commitment to diversity in clinical trials. Although no large company's policy had all 14 themes in the FDA and stakeholder guidance, 13 of the 14 themes were covered by at least one large company's policy. No large company publicly committed to using varying trial designs and methodological approaches to facilitate broader population enrollment.

More than half of the large companies with a publicly available policy committed to: using _targets for enrollment in trials that represented the incidence, prevalence, or severity of the condition or disease in various populations _targeted by a trial (15/21, 71%); remedying barriers to trial recruitment and retention (15/21, 71%); conducting patient and community outreach to increase awareness of the opportunities of clinical trials and health literacy (14/21, 67%); and identifying and using trial sites with diverse populations and providers treating underserved or under-represented populations (12/21, 57%). Several large companies also committed to increasing diversity in the workforce (9/21, 43%) and transparently disseminating the personal characteristics of participants in the trial (9/21, 43%).

Fewer companies among those with publicly available policies committed to: incorporating patient input and experiences into the development of medical products (7/21, 33%); improving diverse enrollment for every country where a trial is conducted (5/21, 24%); offering expanded access to a product for individuals with life threatening conditions outside of a clinical trial (5/21, 24%); use of broad eligibility criteria in clinical trial protocols (4/21, 19%); using a multidimensional approach to diversity beyond sex, age, race, and ethnic group (4/21, 19%); collecting data after product approval to enhance drug safety and efficacy information for diverse populations (3/21, 14%); or adopting a focus on diversity efforts in all trials and not just for later stage trials or pivotal trials (3/21; 14%) (figure 2).

Request permission

Figure 2

Proportion of companies making public policy commitments to themes in US Food and Drug Administration and stakeholder guidance on diversity in clinical trials

Agreement between industry guidance and company commitments

Nearly all company members of PhRMA had a public policy on diversity in clinical trials (19/20, 95%). Examining whether PhRMA member policies matched their trade association's Industry-wide principles to enhance diversity in clinical trial participation, we found that no members committed to all PhRMA principles in their public policies. The most common commitments missing from policies included commitments to conduct additional studies after product approval, if needed, to enhance information on drug safety and efficacy for diverse populations, and the use of broad eligibility criteria for trials, when appropriate (figure 3).

Request permission

Principal findings

In this study, we developed a reference standard for evaluating the comprehensiveness of policies on diversity in clinical trials of pharmaceutical companies in incorporating key stakeholder guidance. We also characterized the public policies on diversity in clinical trials of 50 pharmaceutical companies, 25 large and an exploratory sample of 25 non-large companies, assessing their similarity to FDA and stakeholder guidance, based on the reference standard that we developed.

Overall, we found that only about half of the companies had a publicly available policy on diversity in clinical trials, with large companies more likely to have a public policy than our exploratory sample of non-large companies. Also, corporate policies varied widely and often lacked important commitments recommended in guidance from FDA, PhRMA, WHO, and other stakeholders. For example, few companies with publicly available policies committed to incorporating patient input or experiences into product development or to using broad eligibility criteria for enrollment to increase representation, and even fewer publicly considered diversity in clinical trials beyond adequate representation by sex, age, race, and ethnic group.

Policy implications

These findings suggest that pharmaceutical corporate policies can be better leveraged to promote diversity in clinical research. Corporate policies are considered important elements of effective governance systems, by helping to identify and communicate long term corporate goals and align behaviors and corporate culture with defined goals.25 Corporate policies enable shareholders to hold directors, and directors to hold management, accountable for implementation of polices.12 26 27 To achieve these benefits for diversity in clinical trials, our findings suggest two actions might be needed by pharmaceutical companies on the policy level. First, more companies should publicly communicate their commitments to diversity in clinical trials on their websites to increase public awareness and accountability. Second, companies should improve the comprehensiveness of their public policies.

Our reference standard established a series of 14 commitments recommended in stakeholder guidance that should be prioritized for inclusion in policies on diversity in clinical trials by pharmaceutical companies and other research funders. These commitments ranged from using _targets for trial enrollment that reflect the incidence, prevalence, or severity of conditions or diseases in various populations _targeted by a trial, to broad eligibility criteria for trials, and trial sites with diverse providers and patient populations. The reference standard also recommended a multidimensional approach to increasing diversity in trials by consideration of other factors, such as gender, disability, pregnancy and lactation status, comorbidities, geography, socioeconomic status, and access to healthcare. Beyond enrollment in trials, the reference standard looked at diversity in the workforce and patient engagement in the design of trials, among other commitments.

These 14 commitments generally align with and are responsive to barriers to participating in clinical trials and to facilitators identified in studies and surveys engaging patients and under-represented groups in research.28–30 Some barriers potentially not adequately considered in the analyzed stakeholder guidance and our developed reference standard, which have been identified in studies engaging patients, might include that: oncologists and patients are “more likely to consider clinical trials in advanced or refractory disease”; some patients might fear being allocated randomly to a placebo arm in research; patients have concerns about the side effects of experimental interventions; or “trial involvement would have a negative effect on the relationship with their physician.”31 The roles of religion, religious leaders, and patient access to health insurance also might not be fully considered in stakeholder guidance.31

Currently, few repercussions exist for research sponsors and pharmaceutical companies who fail to commit to recommendations in leading stakeholder guidance for improving diversity in clinical trials. In theory, trade associations, such as PhRMA and IFPMA, could remove member companies who violate their established principles and codes of conduct. PhRMA has precedence for this action. In 2017, PhRMA expelled 22 members who failed to meet their investment requirements for research and development because their business models were based on buying undervalued drugs and marking up their prices, rather than investing in researching and developing new products.32 Also, little to no monitoring exists of companies' commitments to diversity in clinical trials or incorporation of stakeholder guidance into corporate public policies. To strengthen monitoring as well as encourage adoption of select recommended commitments, we may build portions of this reference standard into the Good Pharma Scorecard, after engagement and validation with stakeholders. The Good Pharma Scorecard is an index that annually evaluates, rates, and ranks the performance of pharmaceutical companies on their bioethics and social responsibility. Currently, the Good Pharma Scorecard evaluates companies on whether they enroll representative patient populations in their pivotal trials supporting FDA approval of new oncology therapeutics.11

Limitations of this study

Our study had some limitations. We focused on publicly available policies on diversity in clinical trials. Companies could have internal policy commitments not reflected in our findings. Also, our analyses focused on pharmaceutical companies and did not evaluate other major research sponsors, including companies who exclusively manufacture medical devices, or government agencies, such as the National Institutes of Health in the US. Our reference standard evaluated implementation of select stakeholder guidance; future work should validate its use across settings and assess buy-in from diverse stakeholders. Lastly, having a public commitment to diversity in clinical trial is important but does not guarantee successful implementation, and therefore an evaluation of outcome performance is critical, which we have previously done in other work.11

A range of stakeholders must collaborate and be supported to deal with the lack of transparency and diversity in enrollment in clinical trials. The National Academies of Science, Engineering, and Medicine (NASEM) describes the role academic medical centers, community hospitals, institutional review boards, non-industry research funders, and medical journals should have in achieving a more equitable research ecosystem.33 For example, NASEM's 2022 report states that, “the federal government has a notably prominent role and responsibility in achieving the goal of more inclusive research, as a primary funder of the research enterprise with taxpayer dollars, regulator of the processes of scientific research, gatekeeper to approvals for monetizing scientific discovery, and purchaser of new drugs and devices.” In this regard, the FDA is improving the reporting of the personal characteristics of participants in trials through the publication of its Drug Trials Snapshots, among other initiatives. NASEM suggests further governmental action, specifically that the Department of Health and Human Services form an interdepartmental taskforce to perform a variety of functions, including developing guidance on equitable compensation for research participation. NASEM also recommends that the National Institutes of Health should “standardize the submission of demographic characteristics for trials to ClinicalTrials.gov… so trial characteristics are labeled uniformly across the database and can be easily disaggregated, exported, and analyzed by the public.”

Currently, many variables are required to be reported and can be exported in a csv file from ClinicalTrials.gov entries, but the personal data of trial participants are generally not required for posting of trial registrations and results reporting or exportable when reported. Institutional review boards should evaluate planned enrollment goals in trial protocols for adequate diversity and representation, and consider requiring amendments before approval of unjustified goals. NASEM suggests that the Centers for Medicare and Medicaid Services should “amend its guidance for coverage with evidence development to require that study protocols include… a plan for recruiting… representative participants” and a remediation process if coverage with evidence development studies fail to meet defined goals. Notwithstanding the role of these and other stakeholders in improving inclusive clinical research, pharmaceutical companies are key players because they sponsor most clinical trials supporting FDA product approvals.

Conclusions

In this study, we found that many pharmaceutical companies did not have public policies on diversity in clinical trials, and those that were publicly available varied widely and lacked important commitments. Large companies were more likely than non-large companies to have public policies on diversity in clinical trials. Our findings suggest that biopharmaceutical company policies can be better leveraged to improve diversity in clinical research and implementation of FDA and other stakeholder guidance.