Evaluation & Data Plan
| Research Questions | Performance Measures | Operationalize | Instruments |
|
1. What key variables are associated with Hispanic and low-income student success? (PM 1.1, 1.2) Summative |
PM 1.1: Number of Hispanic & low-income undergraduate students enrolled increase by 10% | Fall Enrollment: headcount of students enrolled at each institution (CSUN, COC, PC). |
Institutional Data |
| PM 1.2: Number of Hispanic and low-income undergraduate students retained and who completed a program increase by 10% and 20% respectively |
Fall to Fall Retention: percentage of students who are retained fall to fall (CSUN, COC, PC) Graduation: students who complete their degree (CSUN, COC, PC). For CSUN: FTF: 4-year & 6-year & FTT 2-year & 4-year. |
Institutional Data | |
| Other key variables for QED to explore student success | Term & Cumulative GPA: student GPA (CSUN) |
Institutional Data | |
|
2. How does a culturally-enhancing model associate with changes in Hispanic and low-income student sense of cultural validation, academic and research skills development, and academic career preparation? (PM 1.3, 2.1, 2.2, 2.3, 2.4, 3.1, 3.2, 3.3, 4.2) Formative/Summative |
PM 1.3: Number of Hispanic and low-income undergraduate students and faculty participating in grant-funded activities will increase by 500 | Participation in grant funded activities: headcount of Hispanic and/or low-income undergraduate students and headcount of faculty who participate (CSUN, COC, PC) |
Program Data: participation in grant-funded activities (all students & faculty) |
| PM 2.1: Number of Hispanic and low-income undergraduate students who participated in experiential learning opportunities will increase by 500 |
Participation in experiential learning opportunities: headcount of Hispanic and low-income undergraduate students who participate in summer research, research symposiums, mentoring programs, conferences, & industry panels (all CSUN; COC & PC: mentoring programs & conferences) [RQ: This supports academic skill development and academic career preparation] |
Program Data: participation in experiential learning opportunities (mentoring, summer research, research symposium, industry panels) | |
| PM 2.2: Number of undergraduate-level courses that are hybrid or virtual will increase by 10 per year | Count of hybrid or virtual courses: online modality | Institutional Data: count of classes that are fully online, hybrid, or hyflex | |
| PM 2.3: Number of partnering institutions that support work-based learning experiences increase by 10 |
Count of partnering institutions that support work-based learning experiences [RQ: This supports academic and research skill development and academic career preparation] |
Program Data: number of partnering institutions | |
| PM 2.4: Gains in reports of ability to work independently, develop research interests, enhance research experiences, and early career roles for Hispanic and low-income undergraduate students increase by 5 % annually |
Early Career Roles & Ability to Work Independently Using Undergraduate Research Student Self-Assessment (URSSA) & using participatory photography, structured journals, & interviews. [RQ: This supports academic career preparation]Develop Research Interests & Enhance Research Experiences: Using URSSA, & using participatory photography, structured journals, & interviews. [RQ: This supports academic & research skill development] [RQ: This supports cultural validation. Cultural validation is when students have experiences that affirm their identities and experiences (Rendón, 1994). Using URSSA & using participatory photography, structured journals, & interviews.] |
URSSA Questions: Early Career Roles: Questions 4.2, 7.5, 16.7 [RQ: cultural validation (4.2, 16.7) & academic career preparation (7.5)] Ability to Work Independently: Questions 2.5, 3.13 Develop Research Interests: Questions 7.1, 23.1 Enhance Research Experiences: Questions 2.1, 2.8, 4.1, 7.4, 11.6, 11.7, 11.8 Participatory photography, structured journals, & interviews: Conducted in summer by evaluation leads with support of UR Fellow lead |
|
| PM 3.1: The number of professional development opportunities for faculty to enhance understanding and use of culturally responsive pedagogies will increase by 50 and the number of faculty participating in trainings will increase by 100 |
Count of professional development opportunities: PD use culturally-enhancing model (goal to increase to 50 opportunities over 5 years) Faculty Participation in professional development opportunities: Headcount of faculty who participate in trainings and FCOP showcase (goal of 100 over 5 years) |
Program Data: Count of professional development opportunities (workshops, FCOP, showcase)Headcount of who participated in professional development opportunities (faculty participant name, employment status (PT/FT), tenure-track or tenured, rank, ethnic and gender identity) |
|
| PM 3.2: The products of faculty who participate in professional development that enhance understanding and use of culturally responsive pedagogies in re-designed courses will increase by 100, serving 6,000 students |
Count of re-designed courses: will need amount and which course numbers are redesigned Count of students in redesigned courses: IR will be able to give us enrollment with course numbers [RQ: This supports a culturally-enhancing model] |
Program Data: number of re-designed courses Institutional Data: number of students enrolled in redesigned courses |
|
| PM 3.3: Gains in reports of understanding and use of culturally responsive and equity-focused pedagogy by faculty will increase by 5% |
Understanding & use of culturally responsive & equity-focused pedagogy: increases by 5%. Will use to questionnaires and personal interviews to measure (updated syllabus, updated policies, using transparent design assignments, affirms student identities) [RQ: This supports using a culturally-enhancing model] |
Workshop Questionnaires: understanding and use of pedagogy FCOP Showcase Questionnaire: understanding and use of pedagogy Personal Interviews: understanding and use of pedagogy |
|
| PM 4.2: Gains in reports of cultural validation and campus belonging for students who participated in grant-supported services will increase by 5 % annually |
Reports of cultural validation: Will use Mentor Collective assessment tools (proprietary instruments), CECE survey, relevant student workshop questionnaires, and student qualitative data (participatory photography, structured journals, & personal interviews) to measure increase.Cultural validation is when students have experiences that affirm their identities and experiences (Rendón, 1994). Reports of campus belonging: Strayhorn (2012) defines belonging as not only a basic human need, but that in higher education, belonging can mean a feeling of connectedness to the campus community, faculty, or peers, and that belonging is when students feel valued and welcome on campus. |
CECE (Culturally Engaging Campus Environments) Survey: administered by CSUN Eval Leads in Y2 & Y4 – items to be identified Mentor Collective Assessment Tools: for students who participate in mentoring Workshop Questionnaires: for after servingness workshops & orientations URSSA Questions: Participatory photography, structured journals, & interviews: Conducted in summer by evaluation leads to students participating in summer research and workshops |
|
|
How does a servingness model shape Latinx and low-income student mental and physical health, emotional wellbeing, and academic outcomes? (PM 4.1, 4.3, & 4.4) Formative/Summative |
PM 4.1: Gains in reports of peer-peer interaction, peer-mentoring, and positive peer environment, including shared identity and experiences increase by 5 % annually |
Peer-peer interaction: count of messages sent (students who are interacting on Mentor Collective via dashboards) Peer-mentoring: headcount of students participating in peer-mentoring (Mentor Collective data) & local data collection (COC & PC) Positive peer environment (shared identity & experiences): Data will be collected via CECE survey administered Y2 & 4, Mentor Collective assessment tools, possibly URSSA questions, and using participatory photography, structured journals, & interviews. [RQ: This supports servingness model and emotional wellbeing] |
Program Data: students who participated in Mentor Collective & summer research; students who participated in peer-mentoring activities (COC & PC) Mentor Collective Dashboards (proprietary software): examines student interactions & students who participate CECE (Culturally Engaging Campus Environments) Survey: administered in Y2 & Y4 Mentor Collective Assessment Tool: for students who participate in mentoring URSSA Questions: Participatory photography, structured journals, & interviews: Conducted in summer by evaluation leads to students participating in summer research, mentoring, and workshops |
| PM 4.3: Number of students participating in grant-funded workshops, activities, and experiences related to mental and physical health and self-care will increase by 500 | Participation in workshops, experiences, & activities related to health and self-care: headcount of students who participate (goal 500 by year 5) (CSUN, COC & PC) |
Program Data: participation in grant-funded activities related to health & self care | |
| PM 4.4: Gains in reports of mental and physical health for undergraduate students who participated in grant-supported services will increase by 5 % annually |
Reports of mental & physical health: Will use workshop questionnaires, CECE surveys, and participatory photography, structured journals, & interviews. Using 8-dimensions of wellness as guide. Physical wellness includes sleep, nutrition, physical activity, & health. Emotional wellness includes coping with stress, feeling supported, and finding time for self-care. |
Program Data: all student participants is workshops related to health and self-care Workshop Questionnaires: for after-health & self-care workshops CECE Survey: administered in Y2 & Y4 Participatory photography, structured journals, & interviews: Conducted in summer by evaluation leads to students participating in summer research, mentoring, and workshops |