Student Voice is Often Highlighted as an Approach to Better Understand Student Problems, but How Do You Capture It?
In fall 2023, Hollins University, a historical women’s liberal arts university, was awarded funding through the S-STEM program for The Artemis Scholarship for Women in STEM. The award was received, in large part, because of the proposal’s inclusiveness around student voice. In this post, we discuss the strategy the co-PI team used to tailor a successfully funded proposal to meet the needs of our unique student population. We outline key processes that were impactful in the successful launch of the grant, share an approach that allowed us to improve the programming throughout the first year of implementation, and conclude with recommendations for those interested in a similar approach.
In January 2023, the interdisciplinary co-PI team conducted a needs assessment of our STEM student body (n=195) that provided data to tailor our proposal to the needs of Hollins’ students. This needs assessment was concise and asked students to highlight key strengths and areas for improvement in their STEM student experience. This included what was working related to social, academic, financial, and mental health, as well as what wasn’t working. The responses (n=65) identified existing successful student support systems to be amplified and provided clarity on areas where additional supports were needed. Following the survey, we conducted a focus group consisting of students who would have been eligible for a scholarship under the S-STEM program. The results were clear and actionable. They included immediate data to support the funding of additional subject-matter tutors and highlighted the utility of an evidence-based approach to program design for our grant.
We Asked Students What Was and Wasn’t Working. We Then Used Student Voice to Drive Design Decisions About Student Programming Supports That are Central to Our S-STEM Program.
In fall 2023, our proposal was funded. Reviewers consistently identified our internal assessment as a strength of the proposal. In addition, they noted their excitement about the proposed participatory evaluation process, inspired by the needs assessment, that framed the generation of knowledge component of the proposal. We believed it was important to empower student voice by ensuring that these voices led the discussion on what the success of the program looked like. With this in mind, the co-PIs explored systematic approaches to ensure the scholars were able to share their version of success.
We Conducted Research with Scholars, Not on Scholars. By Establishing a Participatory Evaluation Process with the Scholars, We Prioritized Developing an Understanding of Scholars’ Perspectives on Success.
In this context, participatory evaluation is a systematic approach centered on student voice that places students as co-evaluators (Cousins & Chouinard, 2012), informing programming efforts throughout the duration of the grant. This approach allows the students to co-identify what their experience of success looks like.
Key components of participatory evaluation in our context include:
Recognition of Power Dynamics:
- Scholars are seen as co-evaluators. The goal of this evaluation approach is to “level the playing field,” as scholar and evaluator feedback are equally valuable (Chevalier & Buckles, 2019).
- A faculty member outside of the co-PI team, familiar with STEM majors and evaluations, became a co-evaluator with the scholars. This minimized the effect of power dynamics on the outcome of the participatory evaluation process.
- This external faculty member regularly engages students outside of the evaluation process, intentionally building up positive rapport and trust through authentic and transparent interactions.
Listening to Scholars:
- Evaluation sessions are held once a semester towards the end of the academic term and are designed to cultivate a welcoming space, as the timing and location are chosen by considering student input.
- Scholars provide constructive feedback on their experiences in the Artemis program, which is shared anonymously with the co-PIs. The leadership team considers the scholars’ input and adapts programming as appropriate.
- All feedback is centered around scholars’ perception of their success, often branching into what supports their success and what are their perceived barriers to success.
Clear and Transparent Communication:
- There is strong communication and transparency between all stakeholders. Clear communication and transparency is essential to making real-time adjustments that support scholars’ needs and improve student success outcomes.
These Efforts Helped to Build Trust, Camaraderie, and Support Among the Scholars and Between the Co-PI Team and the Scholars
The participatory evaluation conducted in year one yielded other tangible results. The co-PIs used scholars’ feedback from the fall to guide interactions in the spring. For example, during the first fall evaluation session, the conversations revolved around students’ classes, being a scholar in those classes, and where they felt successful and where they did not. Scholars shared that they often felt “I feel like I shouldn’t be here” and “my background didn’t prepare me for a STEM major.” That following January, the cohort participated in an Artemis-supported research methods class. In this class, first-year STEM students (both Artemis scholars and non-scholars) engaged in reflective conversations with the co-PIs about their first-year experience, which helped boost scholars’ confidence and sense of belonging in STEM.
What Developed Was a Clear Message to Students, Your Voice Matters, and When We Can, We’ll Adapt Our System to Meet Your Needs
The trust that was fostered within scholars and between scholars and the co-PI team was the result of intentional practices. Scholars recognized they are a part of this process and see that the co-PI team is responsive to shifting needs. The students also recognized that while not all of their concerns could be addressed, the ones that could be controlled by the co-PI team were acknowledged, with adaptations made when applicable. This relationship has enabled transparency and is fostered by the sharing of honest, constructive feedback that continuously improves the program and scholar experience. Although our program reflects the context of Hollins University, the underlying principles are broadly applicable and may be scaled or adapted to fit other institutional settings.
Student-Centered Programming in Your Settings
At this stage in the grant, the resources required to support student-centered programming have demonstrated efficacy. Student-centered programs need opportunities for direct student feedback. Current and future S-STEM programs should consider the evidence-based practices of a needs assessment and participatory evaluations (Odera, 2021; Odera, 2018).
- When preparing student-centered programming, be sure to ask students what they need and what’s currently working for them;
- Consider a participatory evaluation approach to incorporate student-voice as co-evaluators of their experience;
- Through appropriate transparency and authenticity, foster trust between co-PIs, between students, and between co-PIs and students.
If you’re interested in learning more about participatory evaluation’s impact on groups, consider exploring Better Evaluation Knowledge and find practical tips from the University of Kansas’ Community Toolbox.
Acknowledgements
The authors acknowledge the partnership of our external evaluators, Brockport Research Institute, who partner with our internal participatory evaluator to provide wrap-around support for the evaluation of our program and its outcomes.
References
Chevalier, J., & Buckles, D., (2019). Participatory Action Research: Theory and Methods for Engaged Inquiry (2nd edition). Routledge Taylor & Francis Group.
Cousins, B., & Chouinard, J.A. (2012). Participatory evaluation up close: An integration of research-based knowledge. Charlotte, North Carolina, Information Age Publishing.
Odera, E. (2018). Examining the impacts of youth participatory evaluation methods on a youth participatory action research setting (Doctoral dissertation, The Pennsylvania State University). https://etda.libraries.psu.edu/catalog/15692exo5072
Odera, E. L. (2021). Capturing the added value of participatory evaluation. American Journal of Evaluation, 42(2), 201–220. https://doi.org/10.1177/1098214020910265
