Appendix B: Research Methodology

Overview 

This is the second longitudinal study of workplace equity and culture in US art museums of a series of data collection surveys planned every two years between 2022 and 2030. The data collected in 2024–25 reflected a 70% increase in Partner Museums and utilized a more streamlined approach to data security and analysis, which enabled deeper insights. 

The survey structure remained the same, with participating museums engaging in three distinct investigations: a director survey, an HR survey, and a survey of all paid staff members. In total, 3,018 museum staff participated in the 2024–25 MMF surveys through 91 art museums, which MMF calls “Partner Museums.” In addition, 83 staff from nonparticipating institutions completed the survey. The response rate for the staff survey among Partner Museums was 48% (the average response rate across all institutions), and the response rates for the director and HR surveys were 74% and 78%, respectively.

MMF partnered with Dr. Jennifer Benoit-Bryan and Daniel Fonner at SMU DataArts to lead the survey design, programming, weighting, and analysis. The MMF team also assembled a research advisory group in 2024 with a range of experts to help shape research design, analysis, and meaning making. The advisors (listed below) provided invaluable feedback, and their contributions are deeply appreciated. 

• Charlotte Burns, founder of Studio Burns

• Deirdre Harkins, Senior Analyst, Ithaka S+R

• Jenni Kim, Senior Vice President, Operations, David Lynch Foundation

• Carys Kunze, Research and Data Manager, American Alliance of Museums

• Dr. Amanda Tobin Ripley, independent researcher 

Design Process 

The research team reviewed all items from the pilot study in the survey instruments and identified a few questions to cut to make room for new lines of inquiry. Ultimately, of the 2023 survey questions, the team modified (by adding or removing options or slightly altering questions) 67% and removed 19% of the original questions. The team also added 19 new questions. Questions cut from the staff survey included if a worker’s position is endowed, perceptions of pay relative to peers, and role in museum DEAI efforts. New questions were added to the staff survey on preference for another employment level, unions, income, active job search in the past year, source of discrimination or harassment, life ladder assessment, sources of support in museums, household composition, caregiving responsibilities, and a detail regarding disability. The team also updated response options for items that were retained to streamline categories or add commonly written-in options from the pilot study. The survey was revised based on feedback from the research advisory group.

The MMF team continued to hold a set of three core criteria to recruit a range of Partner Museums by budget size, geography, and type (collecting/noncollecting, modern/contemporary, encyclopedic, culturally specific, college/university based, and city/county/state/government affiliated). All 54 institutions who participated in the pilot study were invited to participate again (74% continued their participation). The MMF research team and advisors reached out through their networks to share the open call for any additional interested institutions to participate in the research. One-on-one meetings were held with many art museum leadership teams to share the opportunity for participation. Partner Museums were promised their own institution’s results (including longitudinal data if they participated in the pilot) to be benchmarked against that of relevant institutions (by geography, budget size, and type). Staff at Partner Museums were given access to the same results as leadership received for the staff survey only. 

For the HR and director surveys, only the officers who completed them for their institutions received access (alongside a survey administrator, if assigned) to the findings and benchmarks against comparable institutions. There was no monetary cost for institutions to participate. However, institutions were required to receive at least ten responses to see their organization’s individual staff survey data in order to protect the privacy of participating workers. Interested institutions had the option of distributing the staff survey links themselves or providing their staff mailing list to the MMF team to distribute the survey. (30% were ultimately distributed by MMF). The surveys were open to all respondents between November 2024 and February 2025. Any institutions with fewer than ten responses were still provided access to fieldwide aggregate data and relevant benchmark comparisons based on budget, size, and institution type. 

Analysis Process

A longitudinal comparison across the years of the survey was a core motivation for this research design. The comparisons across data collection years reflect the full sample of Partner Museums from each year. To ensure that changes over time are not artifacts of additional museums new to the sample possibly skewing the results, we ran the comparative analysis for the consistent cohort only and compared it to the full cohort to identify any discrepancies. The full cohort aligned with the consistent cohort within 2% across all responses.

To assess how representative the respondents were of all staff within participating museums, the SMU DataArts team compared aggregate staff responses with data on the full staff provided by HR representatives. We found that staff survey respondents skewed toward higher incomes than the full staff population, so we developed weights based on income distributions reported by HR, which are applied in all salary analyses. 

A set of analysis queries or splits for the data was determined in three main categories: organizational characteristics, individual demographics, and individual job characteristics. Five splits were run on organizational characteristics: region, budget size, collecting/noncollecting, institution type, and churn rate. An additional four individual demographic splits were run, including race/ethnicity (single-select), race/ethnicity (multi-select), gender, and generation. Four splits also were run on individual job characteristics: position level, union membership, full-time or part-time status, and if the worker had experiences of discrimination or harassment in their current workplace. 

Report Process

The MMF team—led by Emiliano Burgos, Genevieve Hoffman, and Michael Guidetti—built a password-protected portal system from which Partner Museums could manage the full data collection and reporting process. The portal was designed for leaders to access the HR and director surveys, as well as share unique survey links for the staff survey with their employees. Once the data collection period ended, MMF gave access to individual museums’ organizational reports through the same portal. 

Data from all three surveys was provided to the leadership teams of participating museums on April 30, 2025, and the staff survey results were available to all staff on May 30, 2025. A series of convenings to walk through the portal, how to use the data, and preview field-wide trends were held for Partner Museum directors, HR officers, and staff. 

Data Management

MMF and SMU DataArts store and maintain data records in the cloud at Amazon Web Services (AWS). All data hosted at AWS is secured and backed up at regular intervals. Survey data is collected via the survey management platform Alchemer and is protected by AWS global infrastructure as well as Open Web Application Security Project (OWASP) standards. Data from Alchemer is then fed back into the MMF portal to provide aggregate response insights to partner museums. Data and metadata are preserved indefinitely on secure servers as well as via Alchemer. This preservation allows for future research that might include comparative analysis of multiple studies or probing the data if new research questions arise. A museum can request that their data be deleted after the 2025 Data Study report is developed, but doing so means the institution will not have the ability to compare their 2025 data to future waves of data collection.

Profile of Participating Art Museums

The MMF team aligned geographic categories for participating museums with those used by the Association of Art Museum Directors: Mid-Atlantic, Midwest, Mountain Plains, New England, Southeast, and Western. The largest geographic representation was from the Midwest (26 museums) and the smallest from New England (11). In terms of budget size, there were quite a few museums in the budget range of less than $2.5 million (18), and at least six museums in each of the seven budget ranges that were used for analysis. The majority of institutions (73) were collecting, with 18 noncollecting institutions participating. Five or more museums of each type participated, with city/county/state/government affiliated (5) the least common, followed by culturally specific (10), encyclopedic (16), college/university affiliated (26), and contemporary and/or modern (61).