Catalyzing Organizational Change for Equity in Graduate Education: A Case Study of Adopting Collective Impact in a College of Engineering

Abstract

Graduate education in engineering is an extremely challenging, complex entity that is difficult to change. The purpose of this exploratory research paper was to investigate the applicability of the Collective Impact framework, which has been used within community organizing contexts, to organize the change efforts of a center focused on advancing equitable graduate education within engineering. We sought to understand how the conditions of Collective Impact (i.e., common agenda, backbone organization, mutually reinforcing activities, shared measurement system, and continuous communication) could facilitate the organization of equity-focused change efforts across a college of engineering at a single institution. To achieve this, we took an action research approach. We found the Collective Impact framework to be a useful tool for organizing cross-sectional partnerships to facilitate equity-focused change in graduate education; we also found the five conditions of Collective Impact to be applicable to the higher education context, with some intentional considerations and modifications. Through coordinated efforts, the Collective Impact framework can support the goal of reorienting existing decentralized structures, resource flows, and decision processes to foster bottom-up and top-down change processes to advance equitable support for graduate students.

1. Introduction

Variations in graduate student experiences persist across social identities (e.g., race/ethnicity, gender, social class, disability/able-bodied, and sexual identity) despite decades of national and local efforts [1,2,3,4,5,6,7]. Racism, sexism, ableism, and other interlocking systems of oppression create higher education environments and shape graduate education experiences that are not conducive to the success of students from marginalized groups. The literature notes that Black women graduate students, for example, deal with invisibility, social and academic exclusion, tokenism, and a lack of support for scholarly research dedicated to women and communities of color to name a few issues [5]. As another example, despite the passage of the Americans with Disabilities Act in 1990, ableism remains normalized in higher education institutions, so much so that scholars have needed to fight for disability accommodations to be viewed as shared accountability as opposed to a burden that undermines productivity [2]. In essence, inequity in graduate education is omnipresent.

STEM (science, technology, engineering, and mathematics) fields are particularly notable for being challenging environments for several groups. The national government has explicitly expressed the importance of diversifying these fields so that the workforce better reflects the country’s demographic profile [8]. Unfortunately, marginalized groups continue to have a significant disparity in degree attainment in these areas. For example, African American and Latinx doctoral students in STEM take longer to complete their doctoral programs or leave them between the first two years [9,10]. Moreover, extensive research has focused on exploring the experiences of STEM graduate students from historically marginalized groups and the challenges they encounter, such as isolation [11,12,13], tokenism and exclusion [14], inequality [15,16], lack of personal support [10], and racial stereotyping [17,18,19]. Scholars have explored these challenges from ecological and socialization perspectives, highlighting issues related to program environments, advisor–advisee relationships, program expectations, and social interactions leading to disruption in marginalized students’ persistence, time to degree, and attrition [9,16,20,21,22,23,24].

Creating change that improves the experiences of STEM graduate students from historically marginalized groups is challenging and complex. As a National Academies Working Group [8] articulated in its analysis of graduate student mentoring, university leaders do not know how to effectively change graduate education or develop integrated networks across organizational layers that include institutions, departments, programs, and individual advisors. As explained by Fleming and colleagues [25], because graduate education tends to be controlled at the individual discipline or departmental level, as opposed to higher college or university levels of the organization, graduate student socialization happens at the discipline or departmental level [26,27,28,29]. Processes tied to students’ time in programs tend to occur at this level, including managing admissions, funding, and degree requirements, all of which are influenced by disciplinary norms and practices [30].

To add another layer of complexity, a significant proportion of STEM graduate students in some disciplines, such as engineering, are funded via research assistantships, which tend to be managed by individual faculty members. Relative to life and physical sciences, graduate education in engineering is less coupled to the undergraduate enterprise from a funding perspective (i.e., via teaching assistantships), leaving colleges of engineering with even fewer internal resource mechanisms to incentivize or demand changes in graduate education [31]. In short, the highly decentralized nature of graduate education in engineering makes integrated reform strategies extremely challenging [25].

Critically, efforts to diversify student demographics, including different racial, ethnic, and gender groups, are not enough to promote degree completion, reduce attrition rates, or improve overall student experiences. Dr. Julie Posselt, the author of Equity in Science, states that equity work is most effective via organizational change, which entails changing policies, practices, and mindsets [32]. Instead of trying to make graduate students from marginalized groups change to fit within an inequitable system, we join scholars like Posselt in arguing that structural, political, and social transformation is needed to promote student success and well-being. Systems of oppression (e.g., racism, ableism, sexism, heterosexism, classism) and other disadvantages across the interconnected systems and processes that shape graduate education need to be disrupted. Furthermore, the common use of student-focused interventions (e.g., mentoring, tutoring, and bridge programs) cannot solely be responsible for combatting the effects of these oppressive systems. Thus, we believe transformative organizational change must occur to realize equity in engineering graduate education.

1.1. Purpose

The purpose of this exploratory research paper is to investigate the applicability of the Collective Impact framework [33,34,35,36] to organizing and establishing change efforts focused on promoting equity in engineering graduate education at the college level. Because Collective Impact is traditionally used to address social problems beyond the purview and authority of a single organization, our project focused on adapting this approach to the context of engineering graduate education. For example, Collective Impact led us to emphasize cross-unit partnerships and collaboration instead of working through a single organization, such as an individual department or the dean’s office. Ennis and Tofa [37] note the need for contextual adaptation or translating to advance understanding of where and how the Collective Impact framework has been used and to what extent it has been useful. Accordingly, the remainder of this paper further describes our process for translation using action research. We focus on the development of a research- and practice-based center focused on organizational change across a college of engineering at a large, predominantly white, research-intensive institution.

1.2. Case Context

The center discussed herein came to fruition following a call from the National Science Foundation (NSF). In 2021, NSF requested proposals focused on developing centers focused on equity in engineering that are intended to catalyze systemic cultural change. Building on the practical and scholarly backgrounds of the faculty and administrators on our team, we successfully proposed a project focused on developing a center focused on transforming graduate education throughout the College of Engineering (COE) at a single institution. The center is called Partnerships and Research on the Equity of Graduate Education in Engineering (PROTEGE). The NSF specified that the development of the center take place over a two-year period. At the time of writing this paper, we were in the middle of Year 2.

PROTEGE is located at Virginia Tech, a large, public, predominantly white, research-intensive institution. Our project team includes COE leadership, education researchers, engineering faculty, postdoctoral researchers, and graduate students. The COE dean is the principal investigator for the grant, with the Associate Dean for Graduate and Professional Studies, the Associate Dean for Equity and Engagement, and two faculty members serving as co-principal investigators. We approached this work from the perspective that engineering graduate education is a multilayered system and that, if we are to address inequities across engineering graduate education, we must take a systematic approach. These approaches must consider the entirety of graduate education and not depend on student-focused interventions alone. We believe it is the system that must sustainably transform, not the students navigating this system.

2. Adopting a Theory of Change: Collective Impact

We situated our center’s organizational change strategy in the Collective Impact approach (and language), popularized by John Kania and Mark Kramer [33,34,35,36]. Although Collective Impact is most commonly used within community organizing contexts, we chose this approach because of its emphasis on cross-sectional partnerships [34], an approach we presumed to be vital given the decentralized nature of graduate education in engineering. Though the terminology of Collective Impact more recently gained traction and popularity in the United States, several scholars and organizers have noted that the ideas themselves are not new. “Against [a] backdrop of decades of work on coalitions and other forms of organizational partnerships, collective impact can best be understood as a synthesis of practice-based principles for those seeking to build alliances and coalitions to tackle complex problems in local communities” ([38], p. 426). Collective Impact also centers on systemic approaches focused on the dynamics between contributing organizations [34].

In contrast to adopting an isolated impact perspective, where organizations work independently on isolated interventions, Collective Impact emphasizes the need for cross-sector collaboration and partnership, where many organizations commit to a common agenda for lasting, effective social change [33,34,39]. Such partnerships have proven successful in scaling up initiatives and supporting large-scale change [40], and prior work demonstrates the usefulness of this lens within the broadening participation in the STEM space [41,42,43,44]. For example, Edwards and colleagues [41] used the Collective Impact framework to analyze the success of the National Society of Black Engineers’ (NSBE) Summer Engineering Experience for Kids (SEEK) program, a large, national scale STEM outreach program. They used Collective Impact as a lens for understanding how NSBE scaled up SEEK nationally, given the need for collaboration across schools, companies, and other partners in various cities across the United States. Fletcher and colleagues [42] explored Collective Impact as a framework for coordinating systemic changes necessary to remove structural barriers for marginalized students accessing computer science education across multiple states. The Expanding Computing Education Pathways Alliance leveraged Collective Impact along with a five-stage model of change to serve as a guide for state leaders to develop broadening participation solutions relevant to their state-specific needs while utilizing common language and a common approach across all projects. Uddin [43] discussed the early application of Collective Impact as a solution to increase Latinx students’ access to STEM education in the northeast Tennessee region. Local non-profit organizations, educational institutions, industries, and the State Education Department were identified as necessary partners that collectively created an initiative to increase access to STEM education for a growing Latinx population.

Scholars assert that Collective Impact initiatives typically rely on the presence of five conditions: (1) Common agenda, (2) shared measurement systems, (3) mutually reinforced activities, (4) continuous communication, and (5) backbone support. Each condition is summarized in

Table 1. Five Conditions of Collective Impact [33].

Condition	Description
Common Agenda	Partners share a consistent vision and understanding of the problem and approach, addressing it through agreed upon actions. Although consensus is not a requirement, it is important that differences be discussed and resolved.
Shared Measurement Systems	Partners share an understanding of how success will be measured and reported, resulting in data being measured and reported consistently across organizations/stakeholder groups.
Mutually Reinforced Activities	Partners undertake a specific set of activities, informed by the shared measurement system, in a way that supports and is coordinated with the actions of others. The roles and activities of participants should be differentiated yet aligned.
Continuous Communication	Partners regularly meet and exchange correspondence to ensure sustained communication is maintained across the organization.
Backbone Support	Partners are coordinated by a separate organization that serves as the backbone for the entire initiative, led by staff with the skills and time needed to manage logistical and administrative logistics while supporting partner initiatives.

. In addition to pursuing these conditions, we also reviewed literature proposing additional conditions, such as assessing community readiness (e.g., [45,46]); highlighting dilemmas that emerge during the early stages of Collective Impact, such as deciding when to combine existing efforts as opposed to taking up new initiatives (e.g., [47]); and forefronting its deficiencies, such as not addressing the need to meaningfully engage those most affected by the issues (e.g., [48,49]). The insights gained from these efforts will be discussed in the Results and Discussion section.

3. Materials and Methods

To establish a system that can sustain change efforts and work through the process of contextual adaptation systematically, we took an action research approach. Our view of action research is the same as [50]: it is a phenomenological methodology for researching organizational processes and practices. Action research aligns with the principles of Collective Impact, whereby, “the involvement with practitioners over things that actually matter to them provides a richness of insight that could not be gained in other ways … and likely to be of practical values” ([50], p. 388). According to Susman and Evered [51], action research typically involves five cyclical phases: (1) Diagnosing, (2) action planning, (3) action taking, (4) evaluating, and (5) specifying learning. Throughout this two-year project, we have engaged with each of the five phases, albeit non-linearly, with the purpose of establishing the infrastructure needed to sustain equity-focused changes to graduate education across a college of engineering.

In this section, we discuss the activities (

Table 2. Research Activities Aligned with Phases of Action Research.

Center Activity	Action Research Phase
	Diagnosing	Action Planning	Action Taking	Evaluating	Specifying Learning
Team Meetings	X	X	X		X
Reflection Activities	X	X			X
Mini-Projects			X
Stakeholder Meetings				X	X
External Evaluation				X
Writing/Synthesizing					X

) we engaged in as first attempts at implementing the Collective Impact framework. The team engaged in this work included the College of Engineering leadership (e.g., COE Dean, Associate Dean of Graduate and Professional Studies, Associate Dean of Equity and Engagement), education researchers, the engineering faculty, postdoctoral researchers, and graduate students. Although our methods are not traditional (e.g., surveys, focus groups), we believe that sharing our approach can establish a blueprint for others who may similarly want to adopt the Collective Impact approach for their own context.

3.1. Team Meetings

Team meetings were regularly held, primarily focusing on overseeing change efforts led by team members and ensuring the team was cognizant of efforts ongoing elsewhere in the COE. These meetings also included discussions about translating the five conditions of Collective Impact, using formal and informal reflection activities that we discuss in subsequent sections. Because it was logistically infeasible to identify a single meeting that worked for everyone on a team of more than 10 people, we held multiple weekly or biweekly meetings that targeted different stakeholder groups. For example, during the first semester of the project, we held three different meetings. The first meeting included the non-administrative members of the team (i.e., regular tenure-track faculty members); the second meeting included team members most actively involved in the project, often referred to as members of the Backbone Organization; and the third meeting included administrative members of the team (i.e., the dean and associate deans). The center director attended each of these meetings to ensure communication channels were open across these different groups. During all meetings, detailed notes were documented and later used as a data source for synthesizing our findings for this paper.

3.2. Reflection Activities

We conducted reflection activities periodically to ensure team members could share their honest opinions and perspectives with each other without the potential power dynamics that can be present in a team meeting. These activities focused on the conditions of Collective Impact. For example, two reflection activities occurred during the first phase of the project: (1) a common agenda reflection activity and (2) a communications reflection activity. Project team members were prompted to reflect on the essential elements of a common agenda, including (a) guiding principles, (b) common problem definition, (c) goals, (d) framework for change, and (e) a plan for learning and evaluation [52]. For the communications reflection activity, the purpose was to help the teamwork toward developing a communications plan, another key element of Collective Impact [33]. For each activity, individual responses were synthesized, summarized, shared with team members, and discussed in subsequent team meetings. Given that Continuous Communication and the Common Agenda are two of the five conditions for Collective Impact, the individual responses from each activity were also used as data sources for this paper. The artifacts produced from these activities will be further discussed in the upcoming sections.

3.3. Mini-Projects

Mini-projects were engaged to provide us with real-time feedback on what it is like to pursue change in our local context. These projects were initially conceptualized and led by faculty team members. To ensure that the activities aligned with the common agenda and focused on the appropriate systems component within graduate education, each researcher led a mini-project focused on one of four focus areas—Expectations and Accountability, Access and Resources, Culture and Skill Development, and Community and Advocacy. Postdocs and graduate students were also encouraged to propose ideas for projects relevant to the focus areas. These projects carefully considered the timeline/cycle for their respective graduate processes (e.g., graduate admission occurs in the early spring). Each group was responsible for leading efforts that facilitated a connection between research and practice in that particular area, leveraging existing information whenever possible. Examples include providing a series of one-pagers to the system changers that synthesize research on particular topics and analyzing existing documents to highlight areas for improvement. Attempting to enact change while establishing the center ensured that our discussions did not simply remain theoretical and grappled with the organizational realities and constraints of the local context.

3.4. Stakeholder Meetings

Stakeholder meetings were organized and attended to ensure that our team considered the perspectives of those beyond its core members. For example, we assembled an advisory board of experts outside our organization to provide formative feedback to the center. Advisory board members were college/university-level administrators from a diverse set of institutions and early-to-mid-career scholars with strong records in graduate education, diversity, equity, and inclusion. It was our hope that, with this mixture, we could facilitate brainstorming and feedback that pushes our initiatives to new spaces while learning from prior implementation experiences of advisory board members. Beyond the advisory board, we similarly engaged stakeholders local to our context, such as graduate program directors and coordinators, graduate students, faculty/staff, and department heads.

3.5. External Evaluation

We leveraged external evaluation to obtain an outside perspective on the process we were adopting. During the first year of the project, we used an external evaluator to assess the center’s ability to provide equitable and inclusive graduate education. The overarching evaluation question used to guide the external evaluation was as follows: “To what extent did the Center for Equity in Engineering (CEE) create an infrastructure that provides a more equitable and inclusive graduate engineering education where every graduate student is provided with opportunities to develop their technical and professional skills, establish their identities as professional engineers, and be included and engaged in the community?” We underwent a process evaluation approach to evaluate the progress toward this question. The external evaluator collected data via semi-structured interviews with members of the project team and a document analysis of documents in the project team’s cloud storage and sharing system. The final report was shared with the entire project team and was used to inform the approach to Year 2 of the grant.

3.6. Writing/Synthesizing

Lastly, we used writing/synthesizing to more explicitly specify learning. As activities were completed, multiple team members worked on writing and synthesizing the products of the mini-projects and activities implemented. These outputs are both internal to the center and the COE graduate education environment and external via conference and journal publications. Internal to the center, we conducted audit trails to reflect on decisions being made in real time and noted insights that emerged as we engaged in center activities. Audit trails are oftentimes used to establish trustworthiness and validity in the research process [53]. While it is not a widely adopted practice, it was established by Halpern [54] as a method to control quality issues that may arise in the qualitative research process. It is defined as a record of how the study was conducted and concluded by researchers [53]. We used the audit trail to practice reflexivity by documenting our thoughts, feelings, and reactions. Because we also took meeting notes, in the context of this project, audit trails were used primarily to document our reflections on decisions that were being made in relation to the activities described in this section. Similar to the meeting notes, the audit trails were later used as a data source for synthesizing our findings for this paper.

4. Results and Discussion: Contextual Adaptation of Collective Impact

After exploring and/or adopting all five conditions of Collective Impact, we strengthened our approach to each and gained insights about the application of Collective Impact in the engineering graduate education context. In the following sections, we discuss the possibilities ofdrawing on this framework to advance equitable support for graduate students.

4.1. Common Agenda

Creating a common agenda proved to be a useful activity, establishing a shared understanding of the problem and approach among our team members. Creating a common agenda also helped us to recognize and discuss our different perspectives. To do so, a reflection activity was drafted and completed by all team members individually. The activity asked members to reflect on various aspects of the center, including what principles and values should guide our work, what the biggest issues are in relation to equity in the COE, and how we might prioritize improving various levers of the graduate education system. The center director conducted a thematic analysis to synthesize responses into one document. The results of this activity follow below.

Because our team was initiated through the development of a grant, it was relatively easy to agree on a vision: Our vision is to catalyze more equitable and inclusive graduate engineering education, where student experiences and outcomes are not predicted by demographic variables and every graduate student is provided with opportunities to develop their technical and professional skills, establish their identities as professional engineers, and be included and engaged in the community. However, through reflection and discussion, we realized that team members had varied views on the extent to which equity was an issue locally, often noting that it depended on student level, departmental context, and the faculty involved. Although we were not able to reach a consensus on the exact nature of the problem, we were able to identify patterns in relation to how our team describes the problem and how it could be divided into four aspects.

First, graduate education policies and practices are seldom built with equity in mind from the onset. Second, inequities are not often addressed in graduate education, leaving many with the perception that there is no accountability (i.e., tolerance of poor behavior and incivility). Third, promoting equity is not an existing skill amongst most administrators, faculty, and staff. Lastly, equity is one of many values and can often conflict with other values held by the college. Although we had varying opinions on the lived experiences of graduate students, our team was able to rally around the need to address each of these four issues. A collective understanding of our vision and how we define the problem of equity in engineering graduate education helps ensure that all team members can contribute in ways that best fit their skills and expertise while knowing that all efforts work towards a shared goal.

In addition to agreeing on the problem, we also found it productive to establish a shared approach via guiding principles that represented how we would go about doing this work. We agreed on five principles. First, we would pursue equity through organizational change. Second, we would pursue changes that would be likely to have a lasting impact. Third, we would ensure the work remained the college’s responsibility. Fourth, we would pursue change in a manner that empowered graduate students while leveraging existing resources wherever possible. These principles (summarized in

Table 3. Overview of Guiding Principles for the Common Agenda.

Principle	Description
Equity through Organizational Change	We will explicitly address issues of social and economic injustice and structural oppression (e.g., racism, sexism). We will strive to change the overall system by focusing on policies, systems, people in positions of power, and structures. We will build on the existing scholarship related to organizational change. For example, we asked ourselves how we should prioritize improving the many parts of the graduate education system. In summary, for every process in graduate education, we need to ask ourselves where we see differences across student subpopulations and work to address them.
Lasting Impact	We strive for changes that are grounded in reality and applicable across departments and disciplines. We strive for changes that will be resourced beyond the lifecycle of the grant. We will leverage existing projects, positions, infrastructure, and (strategic) plans. We will balance the trade offs associated with resource allocation. For example, when partnering with departments across the COE, we first consider what practices are currently in place and recommend ways to improve and enhance as opposed to demanding the implementation of new methods that may not be sustainable or relevant to specific department needs.
The College’s Responsibility	We will determine how to best distribute responsibilities and the most effective organizational structure. We will construct core functions for the center that facilitate collaboration and build department ownership and leadership. We will strive to ensure equity becomes and remains a priority and shared responsibility. For example, all projects are in alignment with the college’s strategic plan and discussed with administrators to illuminate opportunities to improve existing COE efforts that align with the center’s objectives.
Empowering Graduate Students	We commit to ensuring graduate students have an appropriate role in shaping the center’s agenda and resource allocation. We will value the opinions and perspectives of graduate students. We will empower and uplift graduate students from marginalized groups. We will not burden graduate students with the responsibility of promoting equity in the college. We will be thoughtful about how we select and compensate those who do contribute. For example, we developed a Graduate Advisory Board where graduate students are paid to provide feedback on center projects and aid in the development of a sustainable and equitable plan for graduate student engagement with the center.
Leveraging Existing Resources	We will leverage existing resources (i.e., assets, literature, and data sets) that support and/or capture student voices, experiences, and educational outcomes. We will not create unnecessary programs, collect unnecessary data, or burden stakeholders with avoidable requests. For example, the development of our shared measurement system prioritized the use of existing graduate-level data collected in the college and at the university-level, as opposed to developing any new data collection instruments.

) have guided our team’s work and provided us with easy reference points for ensuring that our approach to change aligns with our shared vision.

4.2. Backbone Organization

Because the primary goal of the first year was to develop an organizational structure and a clear description of what the center does and does not do, it was critical that we establish a clear understanding of what role a backbone organization would play in this context. We identified three different roles, as displayed in

Table 4. Backbone Organization Roles Alignment with Shared Understanding of the Problem.

Problem	Backbone Roles
	System Changers	Leadership Developers	Direction Providers
Equity is missing	X
Inequity is not addressed	X
Promoting equity is not an existing skill		X
Equity must Compete with Other Values			X

, that needed to be fulfilled. First, the backbone organization needs to enable changing the system, focusing both on changing processes and changing attitudes. Second, the backbone organization needs to provide direction, ensuring that equity is more often a guiding principle in work across the organization. Lastly, the backbone organization needs to support leadership development, cultivating ownership and leadership among the departments while supporting the training of administrators (e.g., department heads, graduate program directors) and any other department-level bureaucrats (e.g., coordinators) to ensure the main issues are not perpetuated through their practices and enactments of policy. Establishing clear roles for a backbone organization allowed our team to identify what contribution the center could make in the local context.

4.3. Mutually Reinforcing Activities

“Advocacy can point out problems and recommend solutions, while managerialism has a role in implementing change”—Julie R. Posselt ([32], p. 141)

Ensuring that activities across graduate education in a college mutually reinforce each other is a daunting task, primarily because graduate education is a complex and multifaceted system. As Posselt notes in the quote above, it is also important to ensure that activities leverage both advocacy and managerialism. To enact change, we identified 13 levers within this system (i.e., system components) that need to be addressed to realize the change we want to see. The identification of levers came from the experience and expertise of team members and was facilitated through the development of the common agenda and our shared understanding of the problem (discussed in Section 4.1), where each team member was instructed to note how they would prioritize each lever on a scale of 1–5, with 1 being “Not a Priority” and 5 being “Essential”. The levers were further refined in our team meetings and through our attempts to summarize how all the components of our project fit together. These levers represented the components of graduate education that we identified as (a) being vital to achieving an equitable graduate education and (b) within our project team’s sphere of influence. We also considered necessity and feasibility. Focusing on one area or lever at a time will not bring the transformative change that we seek; thus, following the conditions of Collective Impact, our goal is to push on more than one lever at a time.

To support a systemic approach to using these levers, additional organization was needed to help the team conceptualize the work we needed to do. We decided to group these levers into four focus areas that align with the dimensions of organizational justice, as shown in

Table 5. Summary of the Expectations and Accountability Levers.

Lever	Issue or Opportunity	Goal (Informational Justice)
Faculty/Staff (F/S) Performance Expectations	No explicit performance criteria around advising quality or establishing an inclusive culture.	Ensuring documentation that minimizes ambiguity and clearly communicates expectations for faculty regarding the promotion of equitable working and learning conditions.
F/S Accountability Mechanisms	Insufficient accountability mechanisms that only address extreme cases of misconduct, and not the more common cases of mistreatment; power dynamics lead to fear of retribution.	Establish processes that demonstrate a commitment to college/departmental expectations of funding and advising, as well as a willingness to hold the faculty accountable for those expectations.
F/S Incentive Structures	There is a strong emphasis (perhaps overemphasis) on activities that raise the profile of the COE (research, publications) and less on people management; faculty have competing demands.	Establish processes that demonstrate a commitment to college/departmental expectations of funding and advising and a willingness to hold faculty accountable for those expectations.
Graduate Student (GS) Performance Expectations	Graduate education expectations are often ambiguous to the point of being toxic.	Ensuring documentation minimizes ambiguity and ensures that all students are aware of their graduate program’s expectations for environmental conditions as an advisee and employee while communicating degree requirements and employment expectations.

,

Table 6. Summary of the Access and Resources Levers.

Lever	Issue or Opportunity	Goal (Distributive Justice)
GS Recruitment Practices	There is bias in the recruitment processes and insufficient engagement with diverse recruiting sources and existing institutional equity-focused recruitment resources; missed opportunities for integration with existing equity-focused recruitment resources.	Building diverse applicant pools through quality and intentional engagement with underrepresented groups to increase the likelihood of graduating diverse cohorts of students.
GS Admission Practices	There is bias and inconsistency in considering and evaluating admission applications.	Promoting holistic admissions (process) to reduce the impact of bias (explicit and implicit) in the admissions process.
GS Funding Practices	Graduate students are often underpaid and/or overworked; there is unequal access to opportunities for people within the same categories (e.g., PhD students, MS students).	Ensuring students have equitable access to different kinds of funding mechanisms (i.e., research assistantships, teaching assistantships, and fellowships) and that decision-makers fully consider the tradeoffs associated with each of those mechanisms with respect to student experience and outcomes.

,

Table 7. Summary of the Culture and Skill Development Levers.

Lever	Issue or Opportunity	Goal (Interpersonal Justice)
GS Advising Practices	There are equity issues related to interpersonal treatment.	Increasing the use of student-centered and/or culturally relevant advising and supervising practices.
GS Supervising Practices	Faculty have minimal to no training in leadership, managing others, and cultural competencies; graduate students are often underpaid and/or overworked.	Increasing the use of student-centered and/or culturally relevant advising and supervising practices.
COE Socialization	Our community is composed of people that have very different life experiences (e.g., many international students and faculty) and ways of communicating and working with each other.	Cultivating a welcoming culture and climate supportive of graduate student success and wellbeing.

and

Table 8. Summary of the Community and Advocacy Levers.

Lever	Issue or Opportunity	Goal (Procedural Justice)
Department. Lead Involvement	Department leadership often lacks the time, resources, skills, and basic understanding of graduate education equity issues.	Ensuring leadership engagement with developmental opportunities, with focus on advancing skills in this area.
F/S Involvement	There is often a lack of resources and basic understanding among individual faculty and staff.	Ensuring faculty/staff engagement with developmental opportunities, with focus on advancing skills in this area
	There is often a lack of commitment to diversity and inclusion among individual faculty and staff.	Ensuring appropriate mechanisms for shared ownership for change projects whereby faculty and staff can become involved or express needs.
GS Involvement	Graduate students are often underpaid and/or overworked; there is a lack of basic understanding about university operations and mechanisms for change.	Ensuring appropriate mechanisms for shared ownership for change projects whereby students can become involved or express needs.

. Organizational justice is a framework for conceptualizing equity within this context and refers to an employed individual’s perceived fairness of their treatment in an organization [55]. There are four dimensions of organizational justice, including (1) distributive, (2) procedural, (3) interpersonal, and (4) informational [55,56]. Distributive justice refers to the fairness of the resource distribution and outcomes, whereas procedural justice refers to the processes that lead to decisions being made [56,57]. Informational and interpersonal justice, grouped under interactional justice, refers to how people treat one another through sharing information (informational justice) and respect (interpersonal justice). When considering which activities to pursue, we consider each focus area, its associated levers, issues or opportunities for change, and its goal for change.

4.4. Shared Measurement System

Members of the center identified the following goals and constraints for the targeted set of metrics: (1) each metric must be directly related to a process or outcome of at least one change lever (refer to Table 5, Table 6, Table 7 and Table 8); (2) metrics should align with measures identified in strategic plans of the COE or the university, when possible; (3) metrics should rely on data that are currently being collected by the COE or the university, or that will be naturally produced as part of center activities. It was important to the center that our efforts focus on enacting change, not on collecting data, and our plan for a shared measurement system reflects this emphasis. With these constraints and goals in mind, the center explored existing datasets to identify which constructs and outcomes are currently being measured and what data can be disaggregated by demographic groups.

During a collaborative and reflective process, we developed a targeted set of metrics to measure overall center progress, as well as progress related to each of the change levers. Student success and student satisfaction are both crucial components in regard to evaluating the center’s performance; we will monitor demographic-specific data on graduate degrees awarded, withdrawal rate, and graduate students’ overall satisfaction with their experience. However, equity-focused changes resulting from center activities will not be immediately apparent in these measures, as multiple years must pass between recruitment and degree completion. Therefore, we will also monitor progress related to each change lever. These metrics take multiple forms, such as logging changes to departmental manuals and policies and tracking involvement in equity-focused change efforts.

As new data points are available, the metrics in the shared measurement system will be used to evaluate the overall COE performance, as well as the progress related to individual change levers. Furthermore, members of the center will discuss whether any changes are needed in the shared measurement system and whether data-collection and data-analysis efforts can be improved in any way. Updated trends for each of the metrics will be made available to stakeholders through center communication channels. With all stakeholders using a common set of measurements, the center will be facilitating communication among these individuals and groups while also keeping the center accountable.

4.5. Continuous Communication

As a collective, we found the following to be the most representative of the center’s purpose when communicating with constituents: educating the COE about the issues our organization addresses; rallying supporters or the COE to action for our cause; countering the arguments, misunderstandings, or, occasionally, the lies or misrepresentations of those opposed to our work; connecting center activities with existing and future COE activities; and becoming known, or better known, in the COE. As a team, we also all agreed on a set of communication principles (

Table 9. Guiding Principles for Communication Plan.

Principles	Explanation
Data and Stories	Leveraging and capturing the realities of graduate education in the COE quantitatively and qualitatively
Attention and Style	Intentionally tailoring language to be familiar to the audience we are engaging with (i.e., using the language of the discipline)
Effective	Having a clear purpose that is in alignment with the message and strategy in order to achieve said purpose
Comprehensive	Offering adequate and complete messaging
Clarity	Clearly communicating the purpose of the message in a way that the receiver can understand
Coherency	Alignment between all channels of communication, as well as with the center’s mission, goals, and principles and with COE’s policies, plans, and objectives
Timeliness and Urgency	Ensuring that communication happens at the proper time with the proper level of urgency to ensure messages are conveyed and met with the appropriate response
Importance of Feedback	Having open lines of communication between the center and the community
Acknowledgement of Power	Acknowledging power differentials that exist in graduate education
Ethical	Transparency in how we collect data, address confidentiality and anonymity, how information and project outcomes will be used, shared, and published

Note: Principles were crowdsourced from Microsoft 365 Community Content and inspired by Posselt [32].

) and strategies (

Table 10. Reasons for Prioritizing Communication Strategies.

Strategy	Reason for Prioritizing
One-on-one meetings with key stakeholders	Create buy-in and top-down support; following up on other meetings; enable co-leading
Website content	Central location; but not proactive
In-person event	Build community and support; ability to answer questions (similar to forums)
Social media post	Regular/frequent engagement to wide audience
Fact sheet/one-pager/Infographic	Reference document; useful for other strategies
Presentation/briefing	Networking; opportunity to share information directly with stakeholders
Community forums or conference	Build community and support; ability to answer questions (similar to forums)

) that would support us reaching target stakeholders.

5. Implications

We found that, in our context, Collect Impact appears to have the potential to be a useful framework for organizing change efforts. To make this a reality, we had to be mindful of a few considerations along the way. First, the challenge of organizing large change efforts requires patience and intentionality. It is very easy to fall into the trap of implementing changes, programs, or initiatives without ensuring sustainability. We wanted to be mindful of this, as our goal is sustainable systematic transformation. Additionally, we found significant value in the establishment of our guiding principles. Given the size of our project team and the significance of the center, it was key that everyone had some set of guidelines to gauge their work, as we were still in the process of determining what the center was and how it would operate within the COE. Next, we found a great utility in understanding the priorities and operations of our COE. This understanding helped inform what activities we even considered as options for us to engage in. We knew there would be no use in taking up projects that would not garner the support of college-level administrators.

Additionally, we reasoned that prioritizing what the dean prioritized would make it much easier to entice departments to support changes that they were already expected to make, even if they may have been a little resistant. We also found ourselves contending with the centrality of the student-employee tension. The landscape of graduate education places graduate students in a space where they are both students seeking a degree and employees, oftentimes even working for the person who is supposed to support their degree attainment. Thus, there are at least two roles that graduate students have to play and, therefore, two sets of expectations that they must be aware of while navigating academia. This dual role can pose great challenges when engaging in community-centered work where graduate students are temporary occupiers within the community and already have enough to balance. The center had to intentionally consider the totality of the graduate student experience as we navigated our work and our approach to evolving graduate students.

Finally, we found that, despite its utility, Collective Impact alone was not enough to guide our thinking for what equitable change would look like in our context. While a helpful approach for surfacing what, where, and how to devote our attention, we found that using an additional lens (in our case, organizational justice) helped us further refine and organize our actions into meaningful and transferable outcomes.

6. Limitations and Future Work

The purpose of this grant was to ‘stand up’ a center focused on equitable transformation of graduate education in the COE at a single institution. Therefore, we found ourselves trying to build the plane while we were flying it; in other words, we were attempting to build an infrastructure for sustainable change while trying to engage in various change efforts. Although we were intentional about our engagement with this work, it is not without limitations.

The primary limitation comes from research design limitations. Because our research team is early in the process of adapting Collecting Impact, we do not yet have empirical data to support the efficacy of our application, leaving us unable to definitively suggest that this theory of change is efficacious in promoting equity in graduate education. For instance, we cannot yet address how well the Collective Impact approach withstands the transiency inherent in graduate education (i.e., people come and go quite frequently, including graduate students, some faculty, and administrators). The Collective Impact framework is typically utilized in contexts where the community members have a larger buy-in and attachment to the work being conducted and its outcomes. We are operating in a context of an academic and work environment where community members, namely graduate students, are not fixed, and we are committed to not asking students to take on the burden of fixing environments and systems that they did not break. Additionally, community members may not feel beholden to fix these environments because they only serve as a place of work or learning. To date, we can only speculate as to whether the Collective Impact approach will support overcoming such challenges. This limitation does not negate the value of this work or the utility of this framework. For example, there is plentiful evidence of graduate students and faculty serving in various roles aiming to improve the experiences of marginalized communities in graduate education. There is also a strategic commitment by the dean of our COE to prioritize the transformation of graduate education. Nonetheless, our lack of longitudinal data should be noted for those wishing to adapt this approach for their own context.

Additionally, this effort sits alongside several other change initiatives within the institution (which is the case of higher education everywhere) that were not described here. Tensions arise regarding the allocation of resources, both financial and time, that have an effect on these objectives, and members of our team are responsible for advancing those sometimes-competing initiatives simultaneously. For example, leading campaigns to bolster undergraduate scholarships could be perceived as taking time and philanthropic potential away from graduate education. Managing these competing demands presents an additional layer of complexity for applying the Collective Impact framework to the graduate education context.

Future work for this project must include longitudinal studies whereby we monitor and evaluate the impact of our strategic efforts. Although we are confident in our claim that Collective Impact offers a useful framework for organizing change efforts, future work is needed to examine the impact of this organizing by using empirical data collected against the efforts’ success metrics.

Future work should also explore the applicability of other theories of change. There are a plethora of other change theories and frameworks that change agents could consider, such as resource dependency theory [58], shared governance [59], institutional isomorphism [60], communities of practice [61], Kotter’s eight step model of change [62], Torres’ transformational resistant leadership theory [63], and the Competing Values Framework [64]. Although we chose Collective Impact for our context, other theories may be more appropriate in different situations.

7. Conclusions

The purpose of this paper was to explore the applicability of Collective Impact to organize change efforts focused on promising equity in engineering graduate education at the college level. To achieve this, we established a center, assembled a team of college-level administrators, faculty, postdocs, and graduate students, and began adapting the five conditions of Collective Impact to fit the engineering graduate education context. We engaged in action research as a translation method. By studying our translation process, we join other scholars in reflectively and empirically investigating the efficacy of the collective impact approach to organizational change in the engineering graduate education context (e.g., [65,66,67,68,69,70,71]).

While we are focused on creating change within our institution, we hope that our efforts and learnings can help inform others on similar journeys. The high-level ideas in this manuscript (e.g., the tables in the Results and Discussion section) could be used as a starting point by other organizations seeking to improve equity in graduate education.