Fostering Pathways: 30 Years of Inspiring High School Students to Pursue Science Careers through Biomedical Research Experiences

Since 1987, nearly 450 high school students have participated in the High School Intern Program (HIP) led by the Science and Health Education Partnership (SEP) at the University of California San Francisco (UCSF). Acceptance is not contingent upon grades, being solidly on a path to college, or commitment to a science career. Rather, HIP seeks students for whom this program could make a critical difference through engagement with research projects, peer learning, mentorship, and college counseling. The majority of interns come from backgrounds underrepresented in the sciences (minority, low-income, from families without a history of college going, and/or students with disabilities). Interns become comfortable working in a laboratory, begin to conduct experiments independently, learn to present scientific research in oral and written formats, and, critically, begin to see the opportunities available to them should they pursue post-secondary education. Longitudinal studies have documented significant outcomes for HIP alumni. College matriculation rates exceed 95% annually, while student demographics predict only approximately 60% of these students would continue their education after high school. HIP alumni, many of whom enter the program unsure of their plans after high school, pursue post-baccalaureate education and are now serving as science and health professionals in significant numbers. INTRODUCTION Despite widespread recognition of the importance of increasing diversity in the biomedical sciences workforce, significant disparities persist in academic, clinical, and industry settings. Recent investigations into National Institutes of Health (NIH) grant funding found that only 5% of NIH-funded principal investigators are from underrepresented groups (Working Group on Diversity in the Biomedical Workforce, 2012). Similarly, African-Americans, Latinos, and Native Americans are significantly underrepresented in the physician population with only 9% of US physicians identifying as members of these groups, while they comprise 29% of the US Population over age 18 (AAMC, 2016; Annie E. Casey Foundation, 2020). Further, the number of African-American males entering the medical profession is declining (National Academies, 2018). There is an ethical imperative to broaden participation in the biomedical workforce as this underrepresentation holds profound implications for our nation, impacting our ability to advance health equity, while also limiting access to fulfilling and secure careers to growing segments of the US population. Beyond the ethical importance of broadening participation in science, there is evidence that doing so benefits the scientific enterprise itself. Diverse teams are better at solving complex problems more quickly and effectively than homogeneous teams (Valantine and Collins, 2015; Hong and Page, 2004). Further, Freeman and Huang (2014) demonstrated that diverse teams produce higher quality research, defined as being published in journals with a higher impact factor and generating more citations than work completed by more homogeneous teams. While manifest in the workforce, these disparities originate much earlier. Differences in educational access and attainment can be found between students of different races/ethnicities, socioeconomic status, and based on students’ parents’ educational attainment. Three key points that contribute to these differences are: a) matriculation to college: nationwide a much smaller proportion of underrepresented minority (URM) students than their Caucasian and Asian peers matriculate to college – while 41% of K-12 students in public schools are from Hispanic/Latino or African-American backgrounds these students make up only 33% of undergraduates (NCES, 2019); b) choosing science majors: talResearch Internships as Career Exploration Witzel Vol. 3, Issue 2, August 2020 Journal of STEM Outreach 2 ented URM students with an initial interest in science leave science majors and college at a higher rate than their non-minority counterparts due to poor preparation in high school, poor teaching in college science courses, lack of encouragement from professors, and a self-perception of inability to be successful in science careers (Seymour and Hewitt 1997; NRC, 2003; George et al., 2001); and c) matriculation to graduate school: although approximately 24% of bachelor’s degrees in the sciences are awarded to URM, these students receive only 14% of doctorates in the sciences (NSF, 2020). Clearly, there is much that can be done at the undergraduate level to help increase the numbers of students who choose science majors and persist to degrees in those fields. Summer and course-based research experiences have positive benefits for undergraduates (National Academies, 2017). These experiences build students’ understanding of how scientific progress is made, while also increasing students’ self-confidence in science (Healy and Rathbun, 2013; Hunter et al., 2007; Laursen et al., 2010), sense of belonging and inclusion in science (Lee and Davis, 2000; Mendoza-Denton et al., 2002), and persistence to STEM degrees (Nagda et al., 1998; Rodenbusch et al., 2016; Chemers et al., 2011; Jones et al., 2010, Schultz et al., 2011). Moreover, as research experience is a critical component of a graduate school application, the opportunity to conduct research and develop relationships with mentors can help make candidates more competitive for doctoral programs (Weiner, 2014). As a result, these types of research programs are a promising way to help reduce the disparities in persistence to STEM baccalaureate and graduate degrees and ultimately contribute to making the STEM workforce more diverse. Yet, a tremendous amount of potential STEM talent is lost before students even get to college. In a 2005 study, Cleaves found that one reason high school students were deterred from further STEM studies was a lack of knowledge of career options in science as well as the perception that the science they were learning in schools was not relevant to their lives or future work. Maltese and Tai (2010) found that support and encouragement from a teacher can not only help students maintain an interest in science, but critically, can actually stimulate the initiation of interest in STEM subjects. The National Academies report, Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads (2011), specifically discusses summer programs as an effective mechanism to encourage underrepresented minority students to aspire to majors in STEM as these programs often provide a first introduction to scientific research, exposure to STEM role models, and critically help students become familiar with the path to college and STEM careers. In addition, scientific mentors and role models have been shown to play an important role in promoting students’ development of a science identity (Carlone and Johnson, 2007; Ashbacher et al., 2010; Hill et al., 2018; Packard and Nguyen, 2003; Wenger, 1998). Much like undergraduate research opportunities, research internships for high school students have the potential to help increase diversity in the biomedical workforce. There are a wide variety of program models at institutions around the country and evidence of the benefits of these programs is emerging. Studies have found that these programs promote gains in students’ self-reported research skills and research self-efficacy (Salto et al., 2014), and that program alumni pursue college degrees in the sciences (Salto et al., 2014; Rohrbaugh and Corces, 2011; Kabacoff et al., 2017). Most high school level programs currently described in the literature present their student population as “collegebound,” require a GPA minimum (typically 3.0) for acceptance, and/or require that students have a prior demonstrated interest in science. While these programs play an important role in increasing the numbers of students who ultimately pursue STEM majors, they overlook additional STEM talent. A 2001 report from the National Center of Education Statistics describes how students’ lack of knowledge of the path to college limits the educational attainment of talented students from families without a history of college attendance. Increasing the numbers of students from backgrounds underrepresented in the biomedical sciences who matriculate to college is critical if our goal is to make the scientific workforce reflective of the American population. Moreover, many talented students from backgrounds the NIH defines as underrepresented in the biomedical sciences are navigating systemic oppression resulting in experiences that often impact students’ GPAs rendering the GPA a poor proxy for students’ potential. Finally, there is significant evidence that these students have less access to science opportunities both in and outside of school (Aschbacher et al., 2010). Students from high socioeconomic backgrounds were more than three times as likely to take at least one advanced science course in comparison with their low socioeconomic status peers (National Science Board, 2018). In addition, high schools with high URM enrollment offer fewer high-level math and science courses when compared to schools with low URM enrollment (National Science Board, 2018); even in schools where high level math and science courses are offered, URMs are often overlooked or tracked into classes that are not as academically challenging (Jones, 2018; Zuniga et al., 2005). Thus, they may not have had an opportunity to develop an expressed science interest. This paper describes a longstanding high school summer research program based at a premier biomedical research university. For more than thirty years, the program has placed rising seniors from San Francisco’s public high schools in UCSF research laboratories to conduct biomedical research under the direction of a day-to-day mentor (most typically a graduate student or post-doctoral fellow). The program works closely with science teachers to identify talented students who may not yet be aware of their own Research Internships as Career Exploration Witzel Vol. 3, Issue 2, August 2020 Journal of STEM Outreach 3 potential. Over the course of eight weeks, students conduct a novel, independent research project, while engaging in a co-curriculum that in parallel builds their awareness of the path to college and how to pay for it, introduces students to early career scientists to learn about the many possible paths to a career in science, develops students’ scientific writing and speaking skills, and builds a supportive community among students, thereby reinforcing one anothers’ discovery of their potential in science. Longitudinal results of the program suggest alumni attend college, major in STEM fields, and pursue postgraduate study at rates higher than would be predicted by their parents’ educational attainment. CONTEXT The High School Intern Program (HIP) is led by the Science and Health Education Partnership (SEP) at the University of California San Francisco (UCSF). SEP fosters interest in science, builds scientific literacy, and advances diversity and inclusivity in science by connecting the scientific community and the public. Its work falls into four program areas: 1) classroom-based partnerships that bring together teachers and UCSF scientists and health professionals to support innovative learning experiences for K-12 students, 2) professional development workshops for teachers, 3) summer science programs for high school students, and 4) public science events. SEP’s partner school district, the San Francisco Unified School District (SFUSD), is a highly diverse urban school district enrolling over 60,000 students in which the largest ethnic group (Hispanic/Latino) comprises 32% of the student population. The other largest racial/ethnic subgroups include Asian students at 30%, African American students at 8% and white students at 14% of the student body (CDE, 2019). More than half of SFUSD students are from economically disadvantaged families and 29% of the students are English language learners (CDE, 2019). Moreover, the majority (60%) of SFUSD students come from families where neither parent has a bachelor’s degree (NCES, 2017). Thus, the majority of SFUSD students come from backgrounds defined as underrepresented in the biomedical sciences by the NIH. PROGRAMMATIC STRUCTURE Overview. Through SEP’s High School Intern Program, rising high school seniors from backgrounds defined as underrepresented in the biomedical sciences according to the NIH are introduced to scientific research through a paid 8-week internship at UCSF. Each intern works in a laboratory with a scientist mentor who helps them learn the content and laboratory skills necessary to carry out their unique research project. Additionally, interns participate in weekly meetings designed to: introduce them to other early career scientists and faculty members, develop their science communication skills, support their learning through reflection and peer collaboration, and offer guidance for the college selection and application process. The program averages 20 interns each summer, with the actual number varying slightly year-toyear due to funding fluctuations. While the research experience is limited to the summer, the program cycle is nearly nine months long. The cycle begins each fall with a rigorous student recruitment process that seeks to identify students for whom participation will make a critical difference in their lives. Program staff begin recruiting mentors in the winter, and mentor preparation and student matching takes place in the late spring. This phase is followed by intensive work with interns and mentors during the summer laboratory experience. The cycle concludes with reflection: program staff support interns and mentors to reflect on their experiences, and the staff analyze evaluation data and reflect on successes and challenges to improve implementation and outcomes in subsequent years. Details about each step in the process and the program structures that support success are provided below. Student Recruitment. Recruiting begins in November when SEP staff invite San Francisco Unified School District high school science teachers to nominate up to three students. Program staff update teacher lists annually and work closely with the SFUSD Science Department in their efforts to reach all science teachers. Additionally, prior years’ nominations are analyzed by school and additional efforts are made to reach teachers at schools with low nomination numbers; these efforts include providing program information to teachers at department meetings at their school sites, one-to-one phone calls with science department heads, and mailings to high school principals. While a small number of nominating teachers are highly proactive about the annual nomination, nominator diversity varies annually. All communications emphasize that HIP seeks students who may not yet be on the path to college and/or who may face significant hurdles to college matriculation. Teachers are encouraged to identify students who are curious about science but not necessarily those who know they are interested in pursuing it as a career. Each year, teachers nominate more than 100 juniors for the approximately 20 available positions. The nomination process plays a critical role in identifying students who might not feel confident to put themselves forward; and evaluation has found that simply being identified by their teacher as a student who has the potential to conduct independent scientific research and is thought to be capable of being part of this program provides nominees an enormous boost to their confidence. Every year, students report that they would not have applied to a research internship if they had not been nominated by a teacher. In a 2019 application, Research Internships as Career Exploration Witzel Vol. 3, Issue 2, August 2020 Journal of STEM Outreach 4 one intern wrote: In reality, I didn’t really expect my teachers to recognize me and present me with the opportunity because I never thought that they were able to notice how interested I was in a lot of topics. I never saw myself having the potential to reach for bigger things, but my teachers made me realize that there might be a career path to focus on in the future. Nominated students are required to attend an informational session where program staff communicate selection criteria, and introduce students to the goals and expectations of the program, as well as the application procedures. To apply, students submit a written application including a cover letter, resume, and a letter of recommendation from their science teacher. Out of nearly 100 applicants, approximately 40 are selected for interviews to assess science interest, ability to participate fully during the eight weeks of the internship, and interest in the program. The final cohort of 20 students is selected after the interviews are completed. Through the applications and interviews, the selection committee is looking for students for whom this opportunity could make a critical difference in their lives. “Critical difference” is a dynamic criterion and program staff critically review their evaluation protocols annually. In addition to spending significant time in conversation about each applicant, program staff regularly consider the following in their assessments: • What are systemic hurdles students may have that would prevent them from going to college, studying STEM and pursuing STEM fields? • Parental educational attainment of the applicant • Government assistance programs qualification (added in 2019) • Race/ethnicity • Does the applicant have access to other collegesupport programs? • Has the applicant been able to access other science research opportunities? The critical difference criterion was added in 2001 in an effort to ensure the cohort of accepted students better reflected the diversity of SFUSD. The majority of program participants come from backgrounds underrepresented in the sciences (minority, low-income, from families without a history of college going, and/or students with disabilities). Many HIP students have struggled either academically or with personal challenges, and have minimal, if any, prior participation in extracurricular science programming. Staff look for students who are curious about science, interested in learning and motivated by project-based experiences. Grades are not considered in the application process; rather staff look for students who convey their interest in the internship and enthusiasm for science through their written application and in an interview and rely heavily on teacher recommendation letters to assess potential for success in the program. Spring Intern Preparation. Prior to the summer, students who have been accepted attend a collaborative and discussion-driven workshop about communication and workplace norms. This workshop seeks to make transparent the often unwritten rules and expectations in a scientific workplace. For most HIP interns this experience is their first in a professional work environment and even for those who have held prior jobs, research laboratories have a unique structure and culture. Interns are entering into a new cultural and professional landscape and this workshop provides them with resources and strategies before they meet with prospective mentors. In addition, this meeting serves as a time for program staff to collect the paperwork required to register high school students as interns at UCSF. Critical to this program’s sustainability is SEP’s ability to maintain compliance with UCSF policies as they evolve to ensure the safety of minors in labs. During the spring, interns and prospective mentors meet to help finalize lab placements. Experience has demonstrated that the intern-mentor relationship plays an important role in the success of the program. In an effort to make positive and mutually agreeable matches, each intern meets two prospective mentors, and each mentor meets two interns. During the meeting, the mentor tells the intern about their research, shows them their lab, and the intern and mentor engage in conversation to get to know one another. After these meetings take place, interns and mentors give feedback to program staff and that feedback is used to make final matches. Intern Orientation. HIP begins with a two-day orientation in early June. The first day of orientation includes community building activities, a general lab safety training, an introduction to the UC Learning Center (a portal for interns to complete laboratory specific safety trainings), a panel conversation with former interns, a campus tour, and a review of program structures and expectations. The second day of orientation is an interactive lab skills training led by UCSF doctoral students. These hands-on activities help interns develop basic lab skills and deepen their understanding of key biological concepts. Topics include pipetting, making solutions and dilutions, and data analysis and interpretation. In the data analysis and interpretation lesson, interns are exposed to a variety of scientific data including results from cell biology, biochemistry and molecular biology research, as well as from clinical, population health and public health studies. This lesson module seeks to increase students’ exposure to mathematical analysis of data, to introduce them to a variety of research disciplines, and to show them how Research Internships as Career Exploration Witzel Vol. 3, Issue 2, August 2020 Journal of STEM Outreach 5 tions with peers, group work, and evaluation of timesheets. Each of these various elements enables program staff to assess student success and engagement while also providing a proactive approach to addressing challenges that might influence an intern’s interest in finishing the program. Many interns’ in-lab learning experiences are influenced by their confidence and familiarity with professional workplaces. Through these weekly meetings, program staff support interns’ learning in an environment where they feel comfortable asking questions and sharing challenges and successes. In addition to offering oversight as the interns progress through the program, these weekly meetings position staff to build strong relationships with interns that allow for the further development of interns’ personal and professional skills. Weekly meetings are designed to help interns develop confidence in their abilities to do science and become stronger contributors and collaborators in their labs. Interns also attend a series of talks given by UCSF researchers. Program staff provide support and coaching to presenters as they craft their talks, helping ensure that each session is relevant to high school students. Talks include the researcher’s personal story about how and when they became interested in science, their unique pathway to their career in science, and a rich description of their research. In addition, the series includes similarly structured panel discussions: scientists share their path to science and their graduate school experiences and answer questions from students. Prior panelists have included members of the UCSF chapter of the Society for the Advancement of Chicanos and Native Americans in Science (SACNAS) and UCSF’s Medical Scientist Training Program. The differences in speakers’ backgrounds, experiences, and interests underline the wide variety of opportunities and pathways that are open to students. A few 2019 interns reflected on the talks: I enjoyed hearing about careers that I didn’t even know existed but seemed interesting. It was also reassuring to see that these people didn’t always have the straightest path to get to where they ended up. Hearing the experiences of other people in our Tuesday talks made me realize what I should do in college if I want to pursue a career in science. I was unaware of all the opportunities because no one in my family has a science background. It was very useful because it painted a picture of what college is gonna be like. College Support. HIP includes a number of college-going activities. All interns have one-on-one meetings with a college counselor and attend group workshops, in which the college counselor addresses the assets and concerns of HIP’s interns. Interns map their personal goals, and converse with the counselor and each other about overcoming obstacles data analysis in these disciplines has parallels and distinct differences. The orientation brings the cohort of interns together as a group, where they get to know and learn from each other, building the foundation for a supportive learning community. One 2019 intern reflected about the lab skills


INTRODUCTION
Despite widespread recognition of the importance of increasing diversity in the biomedical sciences workforce, significant disparities persist in academic, clinical, and industry settings. Recent investigations into National Institutes of Health (NIH) grant funding found that only 5% of NIH-funded principal investigators are from underrepresented groups (Working Group on Diversity in the Biomedical Workforce, 2012). Similarly, African-Americans, Latinos, and Native Americans are significantly underrepresented in the physician population with only 9% of US physicians identifying as members of these groups, while they comprise 29% of the US Population over age 18 (AAMC, 2016;Annie E. Casey Foundation, 2020). Further, the number of African-American males entering the medical profession is declining (National Academies, 2018). There is an ethical imperative to broaden participation in the biomedical workforce as this underrepresentation holds profound implications for our nation, impacting our ability to advance health equity, while also limiting access to fulfilling and secure careers to growing segments of the US population.
Beyond the ethical importance of broadening participa-tion in science, there is evidence that doing so benefits the scientific enterprise itself. Diverse teams are better at solving complex problems more quickly and effectively than homogeneous teams (Valantine and Collins, 2015;Hong and Page, 2004). Further, Freeman and Huang (2014) demonstrated that diverse teams produce higher quality research, defined as being published in journals with a higher impact factor and generating more citations than work completed by more homogeneous teams. While manifest in the workforce, these disparities originate much earlier. Differences in educational access and attainment can be found between students of different races/ethnicities, socioeconomic status, and based on students' parents' educational attainment. Three key points that contribute to these differences are: a) matriculation to college: nationwide a much smaller proportion of underrepresented minority (URM) students than their Caucasian and Asian peers matriculate to college -while 41% of K-12 students in public schools are from Hispanic/Latino or African-American backgrounds these students make up only 33% of undergraduates (NCES, 2019); b) choosing science majors: tal-ented URM students with an initial interest in science leave science majors and college at a higher rate than their non-minority counterparts due to poor preparation in high school, poor teaching in college science courses, lack of encouragement from professors, and a self-perception of inability to be successful in science careers (Seymour and Hewitt 1997;NRC, 2003;George et al., 2001); and c) matriculation to graduate school: although approximately 24% of bachelor's degrees in the sciences are awarded to URM, these students receive only 14% of doctorates in the sciences (NSF, 2020).
Clearly, there is much that can be done at the undergraduate level to help increase the numbers of students who choose science majors and persist to degrees in those fields. Summer and course-based research experiences have positive benefits for undergraduates (National Academies, 2017). These experiences build students' understanding of how scientific progress is made, while also increasing students' self-confidence in science (Healy and Rathbun, 2013;Hunter et al., 2007;Laursen et al., 2010), sense of belonging and inclusion in science (Lee and Davis, 2000;Mendoza-Denton et al., 2002), and persistence to STEM degrees (Nagda et al., 1998;Rodenbusch et al., 2016;Chemers et al., 2011;Jones et al., 2010, Schultz et al., 2011. Moreover, as research experience is a critical component of a graduate school application, the opportunity to conduct research and develop relationships with mentors can help make candidates more competitive for doctoral programs (Weiner, 2014). As a result, these types of research programs are a promising way to help reduce the disparities in persistence to STEM baccalaureate and graduate degrees and ultimately contribute to making the STEM workforce more diverse.
Yet, a tremendous amount of potential STEM talent is lost before students even get to college. In a 2005 study, Cleaves found that one reason high school students were deterred from further STEM studies was a lack of knowledge of career options in science as well as the perception that the science they were learning in schools was not relevant to their lives or future work. Maltese and Tai (2010) found that support and encouragement from a teacher can not only help students maintain an interest in science, but critically, can actually stimulate the initiation of interest in STEM subjects. The National Academies report, Expanding Underrepresented Minority Participation: America's Science and Technology Talent at the Crossroads (2011), specifically discusses summer programs as an effective mechanism to encourage underrepresented minority students to aspire to majors in STEM as these programs often provide a first introduction to scientific research, exposure to STEM role models, and critically help students become familiar with the path to college and STEM careers. In addition, scientific mentors and role models have been shown to play an important role in promoting students' development of a science identity (Carlone and Johnson, 2007;Ashbacher et al., 2010;Hill et al., 2018;Packard and Nguyen, 2003;Wenger, 1998).
Much like undergraduate research opportunities, research internships for high school students have the potential to help increase diversity in the biomedical workforce. There are a wide variety of program models at institutions around the country and evidence of the benefits of these programs is emerging. Studies have found that these programs promote gains in students' self-reported research skills and research self-efficacy (Salto et al., 2014), and that program alumni pursue college degrees in the sciences (Salto et al., 2014;Rohrbaugh and Corces, 2011;Kabacoff et al., 2017).
Most high school level programs currently described in the literature present their student population as "collegebound," require a GPA minimum (typically 3.0) for acceptance, and/or require that students have a prior demonstrated interest in science. While these programs play an important role in increasing the numbers of students who ultimately pursue STEM majors, they overlook additional STEM talent. A 2001 report from the National Center of Education Statistics describes how students' lack of knowledge of the path to college limits the educational attainment of talented students from families without a history of college attendance. Increasing the numbers of students from backgrounds underrepresented in the biomedical sciences who matriculate to college is critical if our goal is to make the scientific workforce reflective of the American population. Moreover, many talented students from backgrounds the NIH defines as underrepresented in the biomedical sciences are navigating systemic oppression -resulting in experiences that often impact students' GPAs -rendering the GPA a poor proxy for students' potential. Finally, there is significant evidence that these students have less access to science opportunities both in and outside of school (Aschbacher et al., 2010). Students from high socioeconomic backgrounds were more than three times as likely to take at least one advanced science course in comparison with their low socioeconomic status peers (National Science Board, 2018). In addition, high schools with high URM enrollment offer fewer high-level math and science courses when compared to schools with low URM enrollment (National Science Board, 2018); even in schools where high level math and science courses are offered, URMs are often overlooked or tracked into classes that are not as academically challenging (Jones, 2018;Zuniga et al., 2005). Thus, they may not have had an opportunity to develop an expressed science interest. This paper describes a longstanding high school summer research program based at a premier biomedical research university. For more than thirty years, the program has placed rising seniors from San Francisco's public high schools in UCSF research laboratories to conduct biomedical research under the direction of a day-to-day mentor (most typically a graduate student or post-doctoral fellow). The program works closely with science teachers to identify talented students who may not yet be aware of their own potential. Over the course of eight weeks, students conduct a novel, independent research project, while engaging in a co-curriculum that in parallel builds their awareness of the path to college and how to pay for it, introduces students to early career scientists to learn about the many possible paths to a career in science, develops students' scientific writing and speaking skills, and builds a supportive community among students, thereby reinforcing one anothers' discovery of their potential in science. Longitudinal results of the program suggest alumni attend college, major in STEM fields, and pursue postgraduate study at rates higher than would be predicted by their parents' educational attainment.

CONTEXT
The High School Intern Program (HIP) is led by the Science and Health Education Partnership (SEP) at the University of California San Francisco (UCSF). SEP fosters interest in science, builds scientific literacy, and advances diversity and inclusivity in science by connecting the scientific community and the public. Its work falls into four program areas: 1) classroom-based partnerships that bring together teachers and UCSF scientists and health professionals to support innovative learning experiences for K-12 students, 2) professional development workshops for teachers, 3) summer science programs for high school students, and 4) public science events.
SEP's partner school district, the San Francisco Unified School District (SFUSD), is a highly diverse urban school district enrolling over 60,000 students in which the largest ethnic group (Hispanic/Latino) comprises 32% of the student population. The other largest racial/ethnic subgroups include Asian students at 30%, African American students at 8% and white students at 14% of the student body (CDE, 2019). More than half of SFUSD students are from economically disadvantaged families and 29% of the students are English language learners (CDE, 2019). Moreover, the majority (60%) of SFUSD students come from families where neither parent has a bachelor's degree (NCES, 2017). Thus, the majority of SFUSD students come from backgrounds defined as underrepresented in the biomedical sciences by the NIH.

PROGRAMMATIC STRUCTURE
Overview. Through SEP's High School Intern Program, rising high school seniors from backgrounds defined as underrepresented in the biomedical sciences according to the NIH are introduced to scientific research through a paid 8-week internship at UCSF. Each intern works in a laboratory with a scientist mentor who helps them learn the content and laboratory skills necessary to carry out their unique research project. Additionally, interns participate in weekly meetings designed to: introduce them to other early career scientists and faculty members, develop their science communication skills, support their learning through reflection and peer collaboration, and offer guidance for the college selection and application process. The program averages 20 interns each summer, with the actual number varying slightly year-toyear due to funding fluctuations.
While the research experience is limited to the summer, the program cycle is nearly nine months long. The cycle begins each fall with a rigorous student recruitment process that seeks to identify students for whom participation will make a critical difference in their lives. Program staff begin recruiting mentors in the winter, and mentor preparation and student matching takes place in the late spring. This phase is followed by intensive work with interns and mentors during the summer laboratory experience. The cycle concludes with reflection: program staff support interns and mentors to reflect on their experiences, and the staff analyze evaluation data and reflect on successes and challenges to improve implementation and outcomes in subsequent years. Details about each step in the process and the program structures that support success are provided below. Student Recruitment. Recruiting begins in November when SEP staff invite San Francisco Unified School District high school science teachers to nominate up to three students. Program staff update teacher lists annually and work closely with the SFUSD Science Department in their efforts to reach all science teachers. Additionally, prior years' nominations are analyzed by school and additional efforts are made to reach teachers at schools with low nomination numbers; these efforts include providing program information to teachers at department meetings at their school sites, one-to-one phone calls with science department heads, and mailings to high school principals. While a small number of nominating teachers are highly proactive about the annual nomination, nominator diversity varies annually. All communications emphasize that HIP seeks students who may not yet be on the path to college and/or who may face significant hurdles to college matriculation. Teachers are encouraged to identify students who are curious about science -but not necessarily those who know they are interested in pursuing it as a career. Each year, teachers nominate more than 100 juniors for the approximately 20 available positions. The nomination process plays a critical role in identifying students who might not feel confident to put themselves forward; and evaluation has found that simply being identified by their teacher as a student who has the potential to conduct independent scientific research and is thought to be capable of being part of this program provides nominees an enormous boost to their confidence. Every year, students report that they would not have applied to a research internship if they had not been nominated by a teacher. In a 2019 application, one intern wrote: In reality, I didn't really expect my teachers to recognize me and present me with the opportunity because I never thought that they were able to notice how interested I was in a lot of topics. I never saw myself having the potential to reach for bigger things, but my teachers made me realize that there might be a career path to focus on in the future.
Nominated students are required to attend an informational session where program staff communicate selection criteria, and introduce students to the goals and expectations of the program, as well as the application procedures. To apply, students submit a written application including a cover letter, resume, and a letter of recommendation from their science teacher. Out of nearly 100 applicants, approximately 40 are selected for interviews to assess science interest, ability to participate fully during the eight weeks of the internship, and interest in the program. The final cohort of 20 students is selected after the interviews are completed. Through the applications and interviews, the selection committee is looking for students for whom this opportunity could make a critical difference in their lives. "Critical difference" is a dynamic criterion and program staff critically review their evaluation protocols annually. In addition to spending significant time in conversation about each applicant, program staff regularly consider the following in their assessments: • What are systemic hurdles students may have that would prevent them from going to college, studying STEM and pursuing STEM fields? The critical difference criterion was added in 2001 in an effort to ensure the cohort of accepted students better reflected the diversity of SFUSD. The majority of program participants come from backgrounds underrepresented in the sciences (minority, low-income, from families without a history of college going, and/or students with disabilities). Many HIP students have struggled either academically or with personal challenges, and have minimal, if any, prior participation in extracurricular science programming. Staff look for students who are curious about science, interested in learning and motivated by project-based experiences. Grades are not considered in the application process; rather staff look for students who convey their interest in the internship and enthusiasm for science through their written application and in an interview and rely heavily on teacher recommendation letters to assess potential for success in the program.

Spring Intern Preparation.
Prior to the summer, students who have been accepted attend a collaborative and discussion-driven workshop about communication and workplace norms. This workshop seeks to make transparent the often unwritten rules and expectations in a scientific workplace. For most HIP interns this experience is their first in a professional work environment and even for those who have held prior jobs, research laboratories have a unique structure and culture. Interns are entering into a new cultural and professional landscape and this workshop provides them with resources and strategies before they meet with prospective mentors. In addition, this meeting serves as a time for program staff to collect the paperwork required to register high school students as interns at UCSF. Critical to this program's sustainability is SEP's ability to maintain compliance with UCSF policies as they evolve to ensure the safety of minors in labs.
During the spring, interns and prospective mentors meet to help finalize lab placements. Experience has demonstrated that the intern-mentor relationship plays an important role in the success of the program. In an effort to make positive and mutually agreeable matches, each intern meets two prospective mentors, and each mentor meets two interns. During the meeting, the mentor tells the intern about their research, shows them their lab, and the intern and mentor engage in conversation to get to know one another. After these meetings take place, interns and mentors give feedback to program staff and that feedback is used to make final matches.
Intern Orientation. HIP begins with a two-day orientation in early June. The first day of orientation includes community building activities, a general lab safety training, an introduction to the UC Learning Center (a portal for interns to complete laboratory specific safety trainings), a panel conversation with former interns, a campus tour, and a review of program structures and expectations. The second day of orientation is an interactive lab skills training led by UCSF doctoral students. These hands-on activities help interns develop basic lab skills and deepen their understanding of key biological concepts. Topics include pipetting, making solutions and dilutions, and data analysis and interpretation. In the data analysis and interpretation lesson, interns are exposed to a variety of scientific data including results from cell biology, biochemistry and molecular biology research, as well as from clinical, population health and public health studies. This lesson module seeks to increase students' exposure to mathematical analysis of data, to introduce them to a variety of research disciplines, and to show them how tions with peers, group work, and evaluation of timesheets. Each of these various elements enables program staff to assess student success and engagement while also providing a proactive approach to addressing challenges that might influence an intern's interest in finishing the program. Many interns' in-lab learning experiences are influenced by their confidence and familiarity with professional workplaces. Through these weekly meetings, program staff support interns' learning in an environment where they feel comfortable asking questions and sharing challenges and successes. In addition to offering oversight as the interns progress through the program, these weekly meetings position staff to build strong relationships with interns that allow for the further development of interns' personal and professional skills. Weekly meetings are designed to help interns develop confidence in their abilities to do science and become stronger contributors and collaborators in their labs.
Interns also attend a series of talks given by UCSF researchers. Program staff provide support and coaching to presenters as they craft their talks, helping ensure that each session is relevant to high school students. Talks include the researcher's personal story about how and when they became interested in science, their unique pathway to their career in science, and a rich description of their research. In addition, the series includes similarly structured panel discussions: scientists share their path to science and their graduate school experiences and answer questions from students. Prior panelists have included members of the UCSF chapter of the Society for the Advancement of Chicanos and Native Americans in Science (SACNAS) and UCSF's Medical Scientist Training Program. The differences in speakers' backgrounds, experiences, and interests underline the wide variety of opportunities and pathways that are open to students. A few 2019 interns reflected on the talks: I enjoyed hearing about careers that I didn't even know existed but seemed interesting. It was also reassuring to see that these people didn't always have the straightest path to get to where they ended up. College Support. HIP includes a number of college-going activities. All interns have one-on-one meetings with a college counselor and attend group workshops, in which the college counselor addresses the assets and concerns of HIP's interns. Interns map their personal goals, and converse with the counselor and each other about overcoming obstacles data analysis in these disciplines has parallels and distinct differences. The orientation brings the cohort of interns together as a group, where they get to know and learn from each other, building the foundation for a supportive learning community. One 2019 intern reflected about the lab skills training saying: I did take biotech at City College, but back then I was afraid to ask questions so I just struggled on my own. I think that all the activities we did really helped me get a better understanding of what I expected in lab. Everything I learned in the meetings helped me the next day at my lab. I still have the work from those days and I get to review them and use them as reference when I was in lab.
Internship Experience. Interns begin work in their assigned lab placements following the orientation. Each mentor and intern work together to determine the schedule that is best suited to the lab and other commitments, and together they ensure that the intern is completing the required number of work hours. It is important to note that the total program required hours are intentionally designed to be less than full time over the eight weeks. This schedule allows interns flexibility to attend to personal responsibilities, and/or take summer courses either for enrichment or credit recovery. While most students work an average of 23 hours per week, some significantly exceed the minimum required hours. Each student receives a stipend that is roughly based on San Francisco's minimum wage. Without this financial support, many interns would not be able to participate as most need to earn an income during the summer.
Interns each complete any laboratory or project-specific required training and begin their lab-based instruction. They learn about the overarching research questions of the lab and how their project fits into its overall efforts. Interns are provided with extensive background material on their project. They also interact with other lab members to understand the range and variety of projects that take place in the lab, the roles researchers play, and the backgrounds and career pathways of their lab mates. Interns also commonly attend lab meetings with their scientist mentor.
Weekly Meetings and Talks. The entire cohort of interns attend regular meetings with program staff. These workshops are designed to build community, develop interns' ability to communicate in science including scientific writing, and teach them how to give a poster presentation and a scientific talk. Additionally, program staff play a critical support role for interns, and these three-hour long weekly meetings offer an invaluable opportunity for staff to gather information about the experiences interns are having in their labs through activities such as written reflections, conversa-and the resiliency and focus required for attending college. Additionally, students and their families attend an evening session focused on applying to college and financial aid. The session is designed to be inclusive for parents who have no prior experience with the American higher education system and importantly, information from the evening is presented in multiple languages to ensure access for all students' families. Negotiating the college application process is a known hurdle that reduces the number of first-generation students who make it to college (NCES, 2001); therefore, this support is critical to help ensure these students' matriculation to college and potential for future careers in the sciences. Intern evaluations demonstrate that the support provided for the college application process is a highly valued component of the program.
Culminating Projects. Interns are required to complete two final projects: a 10-minute talk and a science poster. During the 10-minute talk, each intern presents their research project background, method and results to an audience of their peers and HIP mentors. Interns often report that this is the most daunting portion of the program, and that they have fears of public speaking. In order to support the development of their public speaking skills and presentation, program staff coordinate two practice sessions prior to the formal talk. Each intern prepares a PowerPoint presentation, practices giving the presentation, then receives feedback, questions, and suggestions from SEP staff and two to three other interns. Repeating this process twice allows time for interns to make changes and discuss with their mentor ways to make their presentation clear and concise, and also allows them to understand their own research at a deeper level. Interns from the 2019 year commented that the practice session helped them in the following ways: I got a sense of what it will be like when I go up to actually present in the end. I was very nervous when I had to present but the practice sessions allowed me to get feedback. The practice talk helped me to understand my project from other people's point of view.
I have a giant fear of public speaking, so the practice talks helped me prepare for the real talk.
The program culminates with the Poster Session and Celebration, attended by about 200 family members, friends, and UCSF and SFUSD community members. During the event, interns present their research in a poster session, hear from invited speakers, and are presented with a certificate of completion. Notably, interns often will present their posters in multiple languages -English to members of the UCSF community and their teachers, but in their home language to their own and other students' families. Interns' reflections on their experience of presenting their summer's work indicate their new-found confidence in their own learning, their pleasure in sharing with others, and their pride in having reached a level of achievement beyond their own expectations. Mentor Recruitment and Support. The role of the mentor is vital to HIP. Mentors serve as the day-to-day guide and teacher for the intern, and support them in all aspects of the lab experience, as well as with the preparation for their talk and final poster presentation. Mentor recruitment begins in January. UCSF staff, faculty, and students are invited to attend monthly information sessions via listserv postings, flyers, and emails from departmental and UCSF leadership. Including a past mentor as a panelist at these information sessions is very helpful for recruiting new mentors. A past mentor is able to credibly describe the time commitment and its impact on their research, dispel stereotypes and biases against working with high school students, and share the rewards of involvement in the program with their peers. Laboratories from across the different UCSF schools (the Schools of Medicine, Nursing, and Pharmacy) and departments (including Biochemistry and Biophysics, Neurology, Physiology, Internal Medicine, Pathology, Cellular and Molecular Pharmacology, Cell and Tissue Biology, Microbiology and Immunology, Hematology/Oncology) host HIP students each summer. While faculty are actively involved and are required to assent to a member of their lab hosting an intern, the majority of day-to-day mentors are graduate students and postdoctoral fellows. The motivation to participate as a mentor varies, and prospective mentors are typically very expressive in describing their interest: This program is the perfect opportunity to mentor a promising student who might not have the experience of working in a laboratory setting otherwise.

Mentoring has always been an integral component of my academic career. As the first in my family to graduate from college -even before ever thinking about pursuing graduate school -I attribute my academic success to the mentoring I received from professors and peers. As a result, I have always had a strong sense of responsibility when it comes to my role mentoring the students coming behind me on the academic pipeline. From a teaching standpoint,
I would like to learn strategies for effectively mentoring high school students (which I'm sure will be different than those required to mentor undergraduate or rotation students). Additionally, having to take a step back to teach the fundamentals of our work will provide me with the opportunity to strengthen my own theoretical background.
Mentors are required to attend four workshops from May to August. Mentors derive benefit from working not in isolation, but rather as part of a cohort in which they can share challenges and successful strategies, discuss appropriate teaching and mentoring methods, address the needs of this specific age group and population of students, and improve their mentoring skills. Each cohort typically includes scientists new to mentoring and those with previous experience; the variation in experiences catalyzes rich conversations related to mentoring best practices. During the first workshop, mentors develop or refine a research project for the 8-week internship and identify lab skills and science content that their intern will need to know in order to complete the project. The timing of the first workshop is strategic in that mentors then have one month to revise their plan and prepare for their intern's arrival. Subsequent meetings address learning styles, communication strategies, and how to help students prepare their research talks and posters. The last workshop focuses on lessons learned and evaluation; this evaluation shows that mentors' attendance has a direct, positive impact on interns' educational experiences.

SUSTAINABILITY
The High School Intern Program began in 1987, the year SEP was founded. Over the past 32 years, the number of interns each summer has varied, and there were at least two hiatus years during a transition between funding sources. Yet, in contrast to many of SEP's other programs, HIP has weathered shifting priorities in science education and an ever changing funding landscape.
Over its history, HIP has received funding from many different sources. In the 1990s, HIP supported between eleven and 16 students each summer with funding from the UC Office of the President (UCOP), UCSF Department of Physiology, NIH Minority High School Student Research Apprenticeship Program, NIH Science Education Partnership Award (R25 RR010194), American Chemical Society, Noyce Foundation, Genentech Foundation, French Foundation, and PG&E Foundation. In the early 2000s, HIP was supported by UCOP with funds that sought to increase the numbers of students prepared to enter UC and CSU campuses. That funding ended suddenly, when the California state budget was drastically cut as the dot-com bubble burst. As a result, SEP did not host interns in 2004. From 2006 to 2011, HIP was funded by the Howard Hughes Medical Institute and an anonymous private donor. Additionally, in 2009, the Baskin Family Foundation began supporting HIP. Since 2011, HIP has been funded by the California Institute for Regenerative Medicine (TC1-05961; EDUC3-08422), the Baskin Family Foundation, and continued support from the anonymous donor. During that time, HIP also received intermittent support from the Gordon and Betty Moore Foundation and the Simons Foundation. SEP's budget as a whole is stabilized by ongoing funding commitments from the UCSF Chancellor, Executive Vice Provost and Chancellor, and School of Medicine. This institutional support has proved invaluable as it provides demonstrated evidence to prospective funders that UCSF views the work of SEP as integral to its public mission. SEP also benefits tremendously from this funding as it helps to ensure continuity in program staffing, preserving critical relationships between SFUSD teachers, administrators, UCSF faculty, and scientist mentors that are so important for HIP.
How has HIP weathered a constantly changing educational environment? HIP's longevity is due in part to the program's focus on critical difference as a selection criterion. This criterion was implemented in 2001 to enable the program to focus on broadening participation in the biomedical sciences. The narrative arc of HIP is compelling and generally easy to grasp: this 8-week summer research internship, at a relatively low per student cost, leads to significant educational outcomes for students from backgrounds underrepresented in the sciences. The critical difference criterion allows staff to review the applicants in a holistic manner and consider the potential impact of HIP on each student's future educational and career opportunities.
Documenting the long-term outcomes of the program is critical to telling HIP's story. Response rates are directly proportional to effort put into following up with students: staff rely on several strategies and make efforts to utilize generationally relevant modes of communication. In the past this has included connection via MySpace and Facebook. In more recent years the focus has shifted to maintaining a more professional network through SEP's LinkedIn group dedicated to alumni of HIP. Initial follow up with students happens towards the end of their senior year to learn their post high school plans, including which colleges they were admitted to and which they plan to attend. They are asked to provide an email address and mobile phone number they expect to be permanent since personal emails and texts from program staff increase response rates. Finally, staff routinely use the National Student Clearinghouse to determine the educational outcomes of alumni. The Clearinghouse tracks the nearly 20 million people attending collegiate institutions in the United States each year. To make the best use of the database, one needs student birthdates and social security numbers for use as unique identifiers. All interns and their parents (if the intern is a minor) complete an IRB-approved consent form giving the program permission to use this information to track educational outcomes.

METHODOLOGY
The data reported in this paper was collected as part of the ongoing evaluation efforts of HIP. All instruments were reviewed and approved by the UCSF Institutional Review Board (IRB) who approved this study (Study number: 10-04297). Participating students who were minors at the time of their participation provided assent, and their parents gave informed consent for their student to be a part of the study. Any student who was over the age of 18 gave informed consent to participate. Student Demographics. All students complete a demographic form when registering for one of the informational sessions that are a part of the application process. This data, while not formally used in the application review, provides program staff descriptive data of who is nominated for the program, who applies, and who is ultimately selected. Demographic data includes descriptors of students' racial/ethnic identity, languages spoken at home, parents' educational attainment (and whether that education was obtained in the United States). Beginning in 2019, questions related to family income level were added to the demographic form.
End-of-Program Retrospective Survey. At the close of the program, students complete a retrospective pre/post survey. Retrospective pre/post (RPP) surveys are a validated methodology that ask respondents to respond to prompts in a post-intervention setting. They rate their response to the item in the present, and also reflect on how they felt prior to the intervention. All data is collected in the same sitting, allowing students to reflect on their perceptions of how much their knowledge, skills, and attitudes have changed since the beginning of the program. For example, respondents would be presented with these two parallel prompts: 1. Before your participation in the High School Intern Program, how much would you have agreed with the statement, "I am good at science."

Now, because you have participated in the High
School Intern Program, how much do you agree with the statement, "I am good at science." While RPP surveys are underutilized compared to the traditional pre/post format, there is evidence of their validity, and moreover, RPPs avoid some of the challenges inherent in a traditional pre/post study design (Little et al, 2019). The end-of-program survey also contains several open-ended responses in order to collect additional descriptive detail about the impact of the program on students.
While HIP has been using a RPP survey design since 2006, data presented in this paper represent responses from the five most recent years (2015-2019, 107 total respondents). The survey contains both Likert items (5-point scale, Strongly Disagree (1) to Strongly Agree (5)) as well as free response questions asking participants to reflect on their experiences and personal growth as a result of participation in the program. Completion of the survey is required, though not all questions were required on the survey. Note that if a respondent chose to answer either a pre or post item, but not both for a particular item, both were coded as a missing response. Responses on this type of survey are paired by design and pre-post answers were analyzed for statistical significance using a paired samples t-test.
Longitudinal Follow-Up. HIP uses multiple strategies to understand the long-term impact of HIP on its alumni, and their educational and career outcomes. At the conclusion of students' senior year of high school, program alumni are sent a follow-up survey. This survey asks students to reflect on the impact HIP had on their senior year, college applications and decisions, and future goals, and also provides an opportunity for program staff to collect permanent contact information from students before their school provided email accounts are inactivated. In addition, program staff maintain a LinkedIn group of HIP alumni. This LinkedIn group has allowed tracking, with minimal effort, the educational and professional milestones of our alumni. Finally, HIP uses the National Student Clearinghouse (https://www.studentclearinghouse.org/) to systematically verify the educational attainment of HIP alumni.
Demographic Profile of HIP Students. HIP's selection criterion of "Critical Difference" results in an accepted student population in which nearly every student meets the NIH criteria for underrepresented in the biomedical sciences or from disadvantaged backgrounds which includes not only students from particular racial or ethnic groups but also students with disabilities, those who have experienced homelessness or spent time in the foster care system, come from families where neither parent has completed a Bachelor's degree, or whose families qualify for certain federal benefit programs (free or reduced price lunch, SNAP benefits, and/ or Pell Grants) (NIH, 2020). Table 1 presents a demographic description of the students from 2015-2019. These are the same students whose end-of-program survey responses are included in the sections that follow.
The overwhelming majority of HIP students also have the potential to be first generation to college. Of the 167 interns who provided information about their parents' educational attainment from 2012-2019, only 5% had a parent who had earned a 4-year college degree in the United States. Similarly, only 5% of students had a parent whose highest degree was a 2-year college degree earned in the United States. Twenty nine percent of interns (49 students total) had parents whose highest degree was a high school diploma, many earned outside of the US; and 23% (38) of students' parents did not complete high school. Thus, the vast majority of Interns are from families that do not have experience with the American higher education system or knowledge of the path to college. (Note, 23 students (14%) reported not knowing their parents' highest level of education). Figure 1 presents a profile of the 2012-2019 interns' parents' educational attainment, disaggregated by race/ethnicity.

OUTCOMES
While the end-of-program survey and longitudinal follow-up surveys examine a wide range of outcomes for students, the analysis below focuses on those that relate directly to students' future college and career trajectories. Interns provided their answers on a five-point Likert scale (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree); for the purposes of analysis, the ratings were scored from 1-5, respectively.

HIP Helps Students See A Place for Themselves in
Science. Students' perceptions of their ability in science changes dramatically as a result of their participation in HIP (t(106) = -6.95, p<.001; see Table 2 for means and standard deviations). Only 43% of students agreed/strongly agreed with the statement Prior to your participation in HIP, how much would you have agreed with the statement "I am good at science," whereas, after participation, 83% of students perceive that they are good at science.
Similarly, students' ability to imagine themselves as scientists changes significantly as a result of HIP (t(106) = -8.58, p<.001). When reflecting on the statement Prior to your participation in HIP, how much would you have agreed with the statement "I can imagine myself as a scientist," only 35% of students agreed or strongly agreed. However, agreement rose to 81% of students at the end of the program.  While fairly high numbers of HIP students agreed or strongly agreed that "going to a 4-year college seems possible to me" at the start of the program (82%), this number increased to 94% at the end of the program (t(106) = -5.66, p<.001) . In contrast, at the beginning of the program, only 41% of students reported knowing the selection criteria at colleges they might want to attend, whereas, at the end of the program, 75% did so (t(106) = -8.91, P<.001).  (2018) found that while 78% of students from families whose parents have a bachelor's degree enroll in college (at either a 2-or 4-year institution), that rate drops to 63% of students whose parents have some college, and only 58% of students who come from families where their parents did not attend any college. Based on these national averages, we would predict that 60% (170 of the 283 intern alumni from program years 2001-2018) would have matriculated to college. HIP alumni greatly exceed this rate, with 99% matriculating to either a 2-or 4-year institution in the fall following their high school graduation (Figure 2). Based on the national sample from this report, just over 100 of the 170 (59%) of HIP alumni would be expected to remain enrolled in college or have earned a degree or other credential within 6-years of matriculation (NCES, 2018). Data from students and confirmed by National Student Clearinghouse provides evidence that HIP alumni are also exceeding this benchmark. As of December 2019, 70.6% alumni, who were at least 6-years post-matriculation, had earned degrees or were currently enrolled in post-secondary education.
Finally, significant numbers of HIP alumni earn degrees in STEM fields. Nationwide, 35% of Bachelor's Degrees are earned in STEM fields (NSF, 2020). Of HIP alumni who earn degrees, 79.5% earn STEM degrees. In addition, since 2001, 14 alumni have earned Masters degrees and 11 alumni have earned PhDs, MDs, PharmDs, or other terminal degrees in STEM-intensive fields.

CONCLUSION
As one of SEP's longest running programs, the High School Intern Program has offered support to hundreds of youth at a critically formative time in their lives and serves as a launching point for many careers in science. HIP alumni matriculate to college at rates that greatly exceed national averages. Moreover, they earn degrees in STEM fields, and many are now working as professionals in the biomedical workforce. HIP alumni are university faculty, physicians, pharmacists, nurses, and clinical research coordinators. They work in public health, mental health, and dentistry. More recent alumni are graduate and/or professional students and postdoctoral fellows. From a longitudinal follow-up survey, alumni offer insight into the impact the program has had on their lives: wanted to be a doctor. However, during the program, I realized how much I loved working in a lab. I felt like I was doing something really cool. I also loved the atmosphere. I decided after the program I wanted to pursue research and a PhD in the field of Biology.
The UCSF SEP Program was the first opportunity I received to envision myself studying and working in a professional field. Looking back 6 years later on my internship experience I recognize that moment as very influential in providing me with that first ounce of professional self-confidence. SEP provided me with a beneficial real world exposure to the realm of scientific inquiry and research, from which I learned about my own untapped potential.
This program was my first internship experience and I am very thankful. It was one of my best experiences because it exposed me to careers in research, it allowed me to get hands-on and work on my own project. Most importantly, this program allowed me to be surrounded by professionals from so many different backgrounds, many of whom were Hispanic which helped me to believe in myself and believe that I too could be successful in this field.
In 2001, HIP staff developed new selection criteria, with a goal of identifying talented students who might not have, using traditional criteria, been considered competitive for admission to a research internship at a premier Research I university. Subsequent review found that the "critical difference" criterion described in this paper resulted in an accepted student cohort in which the numbers of students from backgrounds underrepresented in the sciences (underrepresented minority, low-income, students from families without a history of college going, and students with disabilities) was greatly enriched compared with both prior cohorts, and the pool of nominated students. Notably, this criterion helps program staff identify talented students who would be overlooked by the selection criteria more traditionally used by summer research programs, most commonly a minimum GPA of at least 3.0 and/or being identified as "college-bound." Critically, HIP students matriculate to college at rates similar to or exceeding many of these other programs with published student outcomes (Kabacoff et al, 2017;Rohrbaugh and Corces, 2011;Salto et. al, 2014). This demonstrates that the HIP model is robust and suggests that expanding selection criteria for summer science opportunities could be a powerful tool to increase the numbers of first generation students who successfully matriculate to college and ultimately earn degrees and pursue careers in the biomedical sciences. Moreover, these documented outcomes have helped to ensure ongoing funding for the program.

Author Contributions
The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.

ACKNOWLEDGMENTS
We would like to thank all of the high school interns who have been a part of HIP since the program's inception, as well as their families and their teachers -you are a daily inspiration to us. We would also like to thank the many graduate students, postdoctoral fellows, researchers and faculty at UCSF who have welcomed high school interns into their labs and provided guidance and support. Past SEP staff, including Tracy Stevens, Liesl Chatman, Elisa Stone, Andrew Grillo-Hill, and Ben Koo, played critical roles in the program's development. We are thankful to Michelle Channel for her college counseling expertise and her commitment to supporting HIP students and their families. We appreciate Karen Peterman for providing statistical expertise and advice. Finally, we are grateful to UCSF leadership for its support, as well as to HIP's many past funders.

FUNDING SOURCES
HIP currently receives funding from the California Institute for Regenerative Medicine (EDUC3-08422), the Baskin Family Foundation, and an anonymous private donor.