Best Practices to Support Early-Stage Career URM Students with Virtual Enhancements to In-Person Experiential Learning

: STEM training of college-bound and college students has reliably employed hands-on experiential learning by placing students in on-campus research settings. Dana-Farber/Harvard Cancer Center’s Young Empowered Scientists for ContinUed Research Engagement (DF/HCC’s YES for CURE) program introduces Massachusetts high school and college students from underrepresented populations to cancer research by immersing them in scientific and nursing research environments. Amidst the COVID-19 pandemic, the 2020 summer program was re-designed and delivered virtually for 45 students. Because the program spans three years, we could evaluate the experiences of 18 students (cohort 2) who completed the 2019 (pre-pandemic) and 2020 (pandemic) summer programs. Analysis of cohort 2 data revealed three areas where students felt their competence improved with virtual programming (i.e., effective communication of ideas, access to high caliber speakers, engagement with program leaders) and two areas where it declined significantly (i.e., engaging other stu dents, learning lab material). Additionally, student-reported competence to perform 21 scientific research and seven critical thinking processes were not negatively impacted by the virtual transition. Herein, we describe the adaptation of DF/HCC’s YES for CURE program to a virtual format and the impact on students as a resource for institutions interested in enhancing their STEM training programs with virtual programming.


INTRODUCTION
There are significant barriers to diversity in the STEM workforce that can be partially addressed by early intervention programs for underrepresented minority (URM) students as early as high school (Maltese and Tai, 2011;Svoboda et al., 2016;Tyson et al., 2007). For URM high school and undergraduate students, an abundance of literature has shown that early intervention programs that include research preparation, professional development, and mentorship programming increases retention of URM talent in the STEM workforce (Bradford et al., 2021;Doerschuk et al., 2016;Dyer-Barr et al., 2014;Kitchen et al., 2018;Salto et al., 2014;van den Hurk et al., 2019). The Dana-Farber/Harvard Cancer Center (DF/HCC) administers three student training programs under the Continuing Umbrella of Research Experiences (CURE) umbrella with the vision of eliminating cancer disparities and diversifying the STEM workforce ( Figure  1). These student training programs provide URM and underserved high school and college students with a paid com-prehensive research internship that encourages scientific curiosity, promotes academic success, and broadens students' interest in basic, clinical, computational, and/or population science research and nursing.
For nearly two decades, DF/HCC's flagship CURE Summer Only program has provided rich and meaningful hands-on science research experiences to support the success of 534 trainees to date. This program is an 8 to 12-week full-time research training initiative for high school and undergraduate students interested in a career in science. In 2018, two additional programs were added under the CURE umbrella -Summer Program to Advance Research Careers (SPARC) and Youth Empowered Scientists for ContinUed Research Engagement (YES for CURE). SPARC is a 12week intensive research experience for students at the University of Massachusetts Boston and three partnering community colleges (MassBay, Roxbury, Bunker Hill) in the Boston, Massachusetts area; it focuses on engaging students with emerging technologies in cancer research and cancer disparities. DF/HCC's YES for CURE program is a threeyear training program that features two summers of in-lab scientific research accompanied by scientific training and professional development activities throughout the academic year. Each of the CURE umbrella programs operate under the same research education framework (Figure 2) that focuses on enhancing self-efficacy and scientific identity, two critical precursors to persistence in STEM (Estrada et al., 2018;Hernandez and Schultz, 2018;Maton et al., 2017). The research education framework has three pillars-scientific knowledge, communication skills, and career preparationthat guide all research programming to achieve the outcomes of having students build belief in their capabilities for a scientific career and to expect a productive and rewarding career in STEM.
A key feature of the CURE umbrella programs is their reliance on highly immersive, hands-on experiences where students conduct research in laboratory and clinical settings across seven Harvard institutions. In March 2020, the ability to continue these programs in their traditional format was halted by the onset of the COVID-19 pandemic in the United States and ensuing quarantine and social distancing guidelines. While the COVID-19 pandemic brought about sudden and unprecedented challenges, it also created opportunities for innovation to the longstanding programmatic content and structure of the CURE umbrella of programs. In a deliberate and thoughtful decision, the DF/HCC YES for CURE was re-designed to be conducted virtually for the summer of 2020 with the goal of delivering a comprehensive experience grounded in scientific research and skill-building in a way that would be sufficient to meet the requirements of the federal funding for the program. The program was re-designed with a particular emphasis on addressing three anticipated challenges with a new virtual format: (1) a lack of access to on-site mentored research training; (2) the absence of in-person exposure to the DF/HCC cancer research community via formal events and informal interactions; and (3) a dearth of informal check-ins between program staff and students at program events.
This article focuses on the student outcomes and lessons learned from converting the DF/HCC YES for CURE program to a virtual-only format during the summer of 2020. Because the program includes two consecutive summers of student internships, the program had the unique opportunity to analyze the outcomes of 18 students (cohort 2) who experienced both an in-person summer pre-COVID-19 (Summer 2019) and a virtual-only summer amidst the COVID-19 pandemic (Summer 2020). The goals in transitioning DF/ HCC's YES for CURE program to a virtual platform were to (1) continue to build scientific knowledge and skills in our trainees, (2) provide engagement and career development opportunities, and (3) enhance student self-efficacy to  support their success in scientific research careers. With the exception of in-lab research, all other existing programming (e.g., educational seminars, student-created abstracts and presentations) was converted to virtual delivery (Table 1). In addition, four additional program elements -literature reviews, experimental design series, informational interviews, and weekly reflection groups comprised of a mix of high school and undergraduate students -were put in place to address the aforementioned anticipated challenges of a virtual program (Table 1). These new elements are described in more detail below.

BACKGROUND AND METHODS
Experimental Design Series. A series was developed to ensure training in experimental design given that students did not engage directly in traditional in-person research projects in summer of 2020. This series consisted of three, two-week modules covering basic science, clinical science, and population science fields. In week one of each module, graduate students and post-doctoral fellows participated in a session that introduced students to fundamental concepts pertaining to the module. At the end of the session, students were presented with a hypothetical research problem and worked in small groups of approximately five students to devise an experimental plan, predict the potential outcomes, and interpret the significance of potential outcomes. During the second week of the module, students presented their study designs to another student group; both groups provided and received feedback from peers and their facilitators.
Informational Interviews. Students were trained and supported in conducting informational interviews with STEM professionals as a means to practice engaging with the scientific community, establishing relationships, and practicing networking skills. Students completed a total of four informational interviews over the course of the summer, starting with their faculty mentor and branching out to near-peer mentors and professionals. These experiences were scaffolded with a set of reflection questions students completed after each interview. Additionally, students provided updates during their weekly reflection groups with their peers and program leadership. Finally, a summative reflection assignment where students had to articulate their learnings and future plans was due at the end of the summer.
Weekly reflection groups. Program leadership conducted weekly reflection groups with small groups of students (approximately five students per group combining high school students and undergraduates) to check in on student satisfaction, assess progress toward program deliverables, and address any concerns or questions. These mandatory meetings lasted for 30 minutes. Students had the opportunity to ask questions, clarify assignments or expectations, and provide advice to each other for near-peer mentoring as well. The topics of the reflection group were driven by student interest and varied across groups. Program leaders facilitated organic discussion among the students to address top-of-mind issues raised by the students. Literature Reviews. Students were assigned literature reviews in lieu of hands-on experiences in research environments given that research institutions were closed in Summer 2020 due to the pandemic. The intention behind these assignments was to ensure students were connected to their research laboratories and able to focus their learning of their mentor's area of research. The literature reviews were approximately 500 words that included relevant background information, key discoveries in the field, and the importance and significance of the work of their mentor. This deliverable gave students an opportunity to hone their scientific writing skills while delving deeply into the topics of their research labs.

Informational Interviews
Four times Students conduct four informational interviews with lab members, students in graduate, medical, or professional school and career professionals of their choice.

Weekly Reflection Groups Weekly
Students meet in small groups of five with a program leader to discuss how the program is going.

EVALUATION
The effectiveness of DF/HCC's YES for CURE program is regularly assessed through the use of mixed methods, including quantitative questionnaires of student participants, content assessments, qualitative student interviews, and program record analysis. DF/HCC's CURE program leadership partners with an external evaluation firm, Strategic Evaluations, Inc. (SEI) to conduct a multifold external evaluation that aims to: (1) address if, and to what extent, DF/HCC's YES for CURE program is meeting its stated goals and outcomes, (2) identify mid-course challenges and recommend approaches to address these challenges, (3) investigate statistical correlations between program components and outcomes, and (4) identify associations between successful individuals and certain personal, academic, or profession-al attributes. Ultimately, the evaluation goal is to tease out promising practices and facilitate the contribution of DF/ HCC's YES for CURE program to national studies. Herein, the research analyses primarily focus on the investigation of statistical correlations between program components and outcomes.

Study Sample and Methodology.
Study Sample. The focus of this study is a group of 18 YES for CURE students (called cohort 2) who had the unique experience of participating in person for their first year of the program (2019, pre-pandemic) and virtually for their second year of the program (2020, during the pandemic). The program evaluators collect data on students' experiences in the program and its impacts throughout their tenure in the program. As a result, the program was able to examine how the transition from the in-person experience, as originally planned and executed, to the virtual experience, adjusted to due to the pandemic, impacted these students. Table 2 shows the demographics of cohort 2; there was a disproportionately high number of female students and 89% of the students indicated race/ethnicities that are considered underrepresented in the sciences. Two-thirds of students identify as first generation.
Questionnaires. SEI collaborated with the program leaders to design and administer questionnaires at three time points during the students' participation in the DF/HCC YES for CURE program. The questionnaires were completed via SurveyMonkey during the program's orientation in February 2019 (baseline), at the end of the first summer of the program in August 2019 (in-person summer experience), and at the end of the second summer of the program in August 2020 (virtual summer experience) ( Figure 3). Questionnaire   fell short of their expectations. Lastly, given that cohort 2 students had the unique experience of having spent one year working in-person and one year working virtually, they were asked on the last survey in August 2020 to compare their online experience from the spring and summer of 2020 with their 2019 in-person experience. Comparison ratings across nine areas were based on a 5-point Likert scale from 1 (far worse) to 3 (about the same) to 5 (far better ). An open-ended item asked students to describe how the shift to an online format during 2020 impacted their academic performance and/or ability to learn material and/or engage with the program. The study included responses from the 15 out of 18 student (83%) who provided data on the latter two data sets.
Statistical Analysis. IBM SPSS Statistics Version 21 (Spss 2012) was used to compute descriptive statistics, as well as test for statistical significance. Given the sample size, non-parametric Friedman tests were used to look for significant differences across the data students provided regarding their competencies of scientific research processes and critical thinking processes. These data were matched by the same participant across the three survey time points. Therefore, the Friedman tests were used to determine whether overall competence levels were independent or related among our matched subjects at different points in time. P-values for these items were set at .05, and also analyzed at the p-values of .01 and .001.

Effectiveness of Virtual Experience Compared to In-Per-
son Experience. Since cohort 2 students had the unique experience of having spent one year working in-person with DF/HCC's YES for CURE program and another year working virtually, these students were asked to compare their online experience in the spring and summer of 2020 with their in-person experience back in 2019. When asked to compare their virtual and in-person experiences in the program, a majority (60% or more) of students rated the virtual data were collected, processed, and analyzed by the external evaluation team. Only the external evaluation team had the ability to link student identifiers to survey responses. Those completing the questionnaire within the first two weeks of the launch were entered into a one-time raffle. No other incentives were offered for students to complete the questionnaire, and completion of all questionnaires was optional. Both quantitative and qualitative survey items were included to document students' self-reported competence levels across 21 scientific research processes and 7 critical thinking processes (Table 3). All ratings for competence-related items were collected based on a 5-point Likert scale from 1 (not at all competent) to 5 (very competent). "Competent" is an aggregate of "4-adequately competent" and "5-very competent". Students for whom all three data points were available (14 out of 18 [78%] of cohort 2) were included in the study analysis. Additionally, students were surveyed on the extent to which DF/HCC's YES for CURE in-person and virtual programs met their expectations. These items were rated on a 3-point scale where 1 equaled "The program has fallen short of my expectations," 2 equaled "The program met my expectations", and 3 equaled "The program has exceeded my expectations". Students were also asked in open-ended questions to explain why the program had exceeded, met, or  experience as better or far better than the in-person experience in the three areas of communication skills, access to high caliber speakers, and engagement with program leaders (Figure 4). Approximately three out of four students cited that their ability to learn how to effectively communicate ideas was better or far better via virtual programming than in-person programming. These data are consistent with the increase in the number and types of sessions where students were able to discuss scientific literature, experimental design strategies, scientific techniques, and scientific terminology and concepts. In fact, students spent 20-25 hours per week preparing for and participating in a variety of structured virtual scientific discussion forums in the summer of 2020 compared to only 10 hours per week in-person during the summer of 2019; they spent another 10-15 hours doing independent work under the guidance of their mentor. Regarding access to high caliber speakers, 73% of students also felt this was better or far better in the virtual programming compared to in-person programming. It is unclear why so many students felt this way; this may be due to the variety of speakers from across the country that spoke with students virtually during 2020 whereas the in-person speakers were limited to those local to the Boston area in 2019.
The transition to virtual programming and the impact of the absence of in-person touchpoints between students and program staff spurred a considerable amount of concern among the program's leadership who wanted to ensure that connections to students remained despite physical separation. As a result, virtual weekly reflection groups were added to the program during the summer of 2020 and will continue in the 2021 YES for CURE programming. These small group meetings provided a regular, dedicated time for program staff to check-in with students in contrast to the informal and spontaneous touchpoints that happened organically during the summer of 2019 when all programming was conducted face-to-face. Feedback from the students indicate that this particular change in the programming was a positive one. Indeed, 60% of students shared that their engagement with program leaders was better or far better during the virtual summer program compared to in-person programming. These data are further supported by students' own words explaining their perceptions of the weekly reflection groups.

Quotes Related to Engagement with Program Leaders.
"The YFC program leadership is always very involved and hands on. It was very helpful to have weekly check-ins. I was able to know the leaders better, feel comfortable talking about struggles in the program. Remote weekly check-ins would be something to keep in the program regardless of it being in the lab or remote." (Undergraduate Student) "I enjoyed participating in reflection groups, it is the perfect space to just talk and get to know the leadership team more." (High School Student) "I didn't think that the YFC programming was going to continue with COVID serving as an obstacle, but the staff did a terrific job at making a seamless transition for us." (Undergraduate Student) "The program has exceeded my expectations because I was very impressed with how quick and organized everything transitioned to virtual. We had no idea coming in how things were going to work out and it was the first time really working with [the program leadership] as well. The fact that everything worked so well and immediately was beyond impressive." (Undergraduate Student) Not surprisingly, students indicated that the virtual experience was worse or far worse than the in-person experience in two areas -the students' engagement with other students in the program and enabling students to learn laboratory material. The vast majority (80%) of students found that their ability to engage with their peers in the program was limited during the virtual summer compared to the summer where students were conducting research and participating in events together on campus.

Quotes Regarding the "WORSE and FAR WORSE" Experiences.
"As I previously stated, this year's virtual programming allowed [me] to connect with people we would

"I would say that being in-person is much more rewarding because I was able to be in the lab and make mistakes, learn from them and grow. But one very annoying thing about virtual format was that I was often times having poor internet connections and it was difficult to focus having my sister roaming around bothering me. Whereas I would have the time to focus and study when I was in Boston in my dorm. Reading articles at home compared to reading articles in my own room in school makes a huge difference." (Undergraduate Student)
"Overall, my online experience was very strong, it was just more difficult to work with other students since we were not in person to work together on projects and bounce off ideas." (High School Student)

"…The ability to learn and develop lab skills along with mentor accessibility was worse than last year. Without an in lab experience it is impossible to develop the skills needed for wet lab/bench work. With labs starting to reopen my mentor was bombarded with work and had difficulties meeting with me. Last
year when we were in the lab together it was far easier to contact him/her…" (Undergraduate Student)

Student Satisfaction with DF/HCC's YES for CURE Pro-
gram. The switch from an in-person experience to a virtual one did not appear to have a negative impact on the program's ability to measure up to students' expectations. When thinking about their overall experience in the program from when they started in February 2019 through August 2020, more than half (60%) of students indicated that participating in DF/HCC's YES for CURE program exceeded their expectations; an additional 33% indicated that the program met their expectations ( Figure 5). When asked to compare the virtual format of the program to their initial expectations, all students (100%) indicated it met or exceeded their expectations ( Figure 5). The high ratings of the YES for CURE program are further supported by students' qualitative comments. In particular, students commented on how the switch to a virtual format did not diminish their experience.
"Especially this year, with everything up in the air, I am grateful for the leadership for their working making this summer so successful. There was a lot of uncertainty about whether we would be able to even conduct a summer session effectively, but I think their leadership and programming allowed us to work on a new side of research with literature reviews. My expectations were that I would be able to continue to work in a lab, learning new things, and having a meaningful responsibility in the lab, contributing in a significant way to my mentor's project. This summer program allowed me to do all of these things, therefore the program has met my expectations." (High School Student) "Even though this summer program went virtually, I was still able to learn as much scientific content as I could. The producers and facilitators of the program tried their best and that allowed me to have a great mentor who taught me everything that I needed to know this summer." (High School Student)

Gains in Level of Competence with Scientific Research
Processes. One of the three pillars of DF/HCC's CURE program research education framework (Figure 1) is improving students' scientific knowledge and skills through integrating scientific communication (e.g. writing, presenting, engaging in lab meetings) and science literacy with foundational skills to execute the scientific research process (e.g., critical thinking, experimental design, hypothesis formation and testing, data analysis). Traditionally, this was accomplished by having students work on campus on a project alongside researchers combined with a variety of seminars, events, and small group activities. In the absence of the hands-on research component of the program in 2020, especially in the summer when students would traditionally spend 30 hours per week for 8-12 weeks in a research environment, the leadership of DF/HCC's YES for CURE program developed several new program components to support students in building scientific knowledge and skills. As discussed previously, the experimental design curriculum and literature reviews (Table 1) were specifically designed to meet this need and complemented the already established journal clubs, scientific seminars, abstract writing, and student presentations that have been a part of the program for many years. Moreover, the switch in programming from in-person to virtual did not appear to have negatively impacted student outcomes as it relates to competence around aspects of the scientific research process. Data collected across three time points for cohort 2 students show their level of competence to perform various scientific research processes were not statistically different across 17 of the 21 progress areas included in this study (Figures 6-9). Most notably, there were statistically significant gains in four process areas related to the scientific research process. A comparison of their baseline ratings from Feb 2019 (pre-pandemic) to their August 2020 ratings (during the pandemic) revealed that students experienced a significant increase in their self-reported competence to prepare a science research paper/talk (p=0.017), communicate research findings orally with professionals in the science research community (p=0.004), write a science research abstract (p=0.002), and interpret data (p=0.028) throughout their time in the program (Figures 7 and 8). While it appears that students' ratings of their competence   Asterisk(s) indicates differences were statistically significant (*p < .05; **p < .01; ***p < .001) using matched Friedman significance test (n=14).

Figure 9. Student-Reported Competence with Scientific Re-
search Processes: Project Management and Organization. Asterisk(s) indicates differences were statistically significant (*p < .05; **p < .01; ***p < .001) using matched Friedman significance test (n=14). levels may have showed improvement in six other academic-related areas (Figure 10), there was no statistical difference across time points. These six areas spanned the continuum of the scientific research process including the early steps of reviewing the literature, identifying a problem and formulating a solution to latter stages of communicating the output of their research and applying their knowledge to critically review the work of others. Indeed, from the February 2019 baseline assessment to August 2020 assessment there was a 20 to 36 percentage point change in the percentage of students who felt competent in each of the six areas.

Critical Thinking Gains.
A key element of the research education framework is to support the development of critical thinking skills among the students. Specifically, the program aims to help students gain skills beyond the mechanics of doing technical laboratory experiments and memorizing information; instead, students are encouraged and challenged to engage in sense-making of, processing of, making inferences about, and applying scientific knowledge. The change of DF/HCC's YES for CURE program to a virtual format did not negatively impact students' perceptions of their critical thinking skills, as data show that a majority of students continued to rate themselves as fairly competent across seven critical thinking items included on the survey. In fact, these students showed a statistically significant increase in their competence in their ability to relate results to the "bigger picture" in their field across the three timepoints of the survey (p=0.007) (Figure 11).
Although not statistically significant, students' perceptions of their competence trended lower after the second summer in the program in the areas of being able to infer plausible reasons for failed experiments, formulating a research hypothesis based on a specific question, identifying flaws in the interpretation of data, and figuring out the next steps and reformulating an original research hypothesis. This trend may be due to students moving to an online research environment in their second summer compared to a hands-on, in-person lab experience in their first summer. The online research activities included attending lab meetings, meeting to discuss ongoing research being conducted in the lab, and secondary research for literature related to their laboratory's research areas.

DISCUSSION
Lessons Learned. DF/HCC's 2020 virtual YES for CURE program builds on 19 years of successful implementation of immersive STEM research experiences for over 500 URM and underserved high school and college students. These programs have reliably leveraged local talent (e.g., laboratory mentors, journal club facilitators, event speakers, program leaders) to deliver in-person, hands-on learning experiences that promote self-efficacy and increase student interest in pursuing a scientific career. The sudden onset of the COVID-19 pandemic threatened to derail the DF/HCC's 2020 YES For CURE summer research program given strict social distancing guidelines, and laboratory and office closures that made it impossible to continue to offer in-person experiences. Instead of canceling the summer program, its leadership endeavored to create a virtual experience that would meet established guideposts for success based on the CURE umbrella research education framework (Figure 2). While the re-designed virtual program presented novel programming and logistical challenges and required resetting student expectations, it imparted powerful lessons that will enhance and enlarge the impact of CURE. In fact, the program is applying these lessons to the other CURE umbrella programs, SPARC and CURE Summer Only. We believe that the following lessons have universal appeal to other STEM programs.

Virtual Programming Cannot Replace In-Person Experiences But Can Meaningfully Enhance
Programming. DF/ HCC's YES for CURE speaker programs (e.g., individual speakers, speaker panels) have historically focused on leveraging talent in the local Boston, Massachusetts area given the city's numerous colleges and universities and its vibrant healthcare and life sciences industries. While the program has been successful at engaging local talent, it faces perennial hurdles related to recruitment and diversity of speakers. The onset of the COVID-19 pandemic eliminated in-person speaking activities and necessitated the use of a virtual forum to continue our speaker series. While the program could have continued to focus on recruiting of local professionals, this new format presented an opportunity to reach beyond Boston to engage professionals from across the country and across the globe. This shift brought numerous challenges and learnings. Despite the disruptions to the normal program curriculum, the virtual delivery format, and a lack of in-lab experiences, many of the competency gains that students made between February 2019 and August 2019 were sustained through August 2020; in some areas, students made additional gains in their self-reported proficiency. Part of the students' success can be attributed to continued engagement in learning and applying scientific concepts to virtual sessions (e.g., journal clubs, experimental design sessions) and tangible deliverables (e.g., abstracts, oral presentations, literature review summary). One area that undoubtedly enhanced the program was the increased frequency and regular cadence of student engagement with program leadership, particularly the virtual reflection groups. Indeed, 60% of students reported that their engagement with program leadership was better or far better in the virtual program compared to in-person programming. These data highlight a program element that is easy to implement and requires little additional time from the program staff but has a big impact. Even with the return of in-person activities and laboratory research, the use of virtual check-ins between program leadership and students is highly recommend with an emphasis of meeting in small groups and tailoring the discussions to the unique needs of the group of students. Importantly, we recognize that even though students were spending more time "together" in virtual sessions, this likely did not compensate for the quality of the in-person interactions students had become accustomed to before the onset of the COVID-19 pandemic. By their very nature and design, the virtual meetings of the program did not offer the opportunity for casual conversation and social engagement that was abundant during in-person events and activities. Learning their lab material is also facilitated by in-person interactions where students can ask ad hoc questions of nearby members of the lab or engage in impromptu scientific conversations. The inability of students to have these in-person lab experiences hindered their ability to learn their lab material. In fact, nearly half of students reported that their ability learn their lab material was worse or far worse in the virtual programming compared to in-person programming. Hence, it is important to acknowledge the limitations of virtual programming on student social interactions and engagement while working to develop virtual programming that can address numerous other facets of student learning during STEM research programs.

Expectations Must Be Reset for All Stakeholders Around Logistics, Participation, and Goals of Virtual Programming.
The transition from in-lab and in-person to virtual learning required that program leadership set clear expectations starting with program managers to encourage them to think through the learning and behavior goals they want to achieve in a virtual STEM training program. We found that this foundational work set the tone for the virtual learning experience and facilitated the creation of training and onboarding programs for facilitators, speakers, and students. This was particularly important to help students understand how to learn in a remote-working environment and ensure that engagement in the learning process takes place. While we set many of the traditional behavioral expectations for students (e.g., required attendance, participation in program activities, timely communication), we encountered a new set of challenges including assuming that our facilitators and speakers could easily transition to a virtual format; this was not always the case. In fact, solicited and unsolicited feedback from session facilitators and program managers who observed virtual sessions highlighted the need for more training. We uncovered that facilitators could benefit from more upfront training on strategies to fuel student engagement which extend their academic abilities and require opportunities to interact with peers. Student Self-Reliance is High. While is it imperative to measure how students perform in a virtual learning session for STEM training, it is just as important to support students in areas that go beyond student competencies and proficiencies for scientific knowledge and skill building. Virtual programming introduces a host of new challenges for students who need to be highly self-reliant in order to be successful in a virtual learning environment. To be prepared for virtual learning, we found that the onus is on the student to create a working space, have the tools to engage virtually, find ways to stay motivated with remote learning, and be proactive and responsive in their communication with program staff. In fact, we learned that there is significant value in taking the time to develop a plan to support students in each of the four aforementioned areas. Strategies we used were to deliberately and repeatedly emphasize the use of responsive two-way communication between students and program staff, weekly reflection groups to check-in with program staff, and checking in on the emotional well-being of students. The pandemic shined a spotlight on this latter area -student emotional well-being -in the context of how social distancing and isolation affects students of all ages. In several instances, we encountered students who shared that they need help to address mental health issues. Our approach included a communication strategy which delivered current and reliable messaging, and the availability of services and resources through the employee assistance program.

Evaluate Student Ability to Participate in Virtual
Programming. The pandemic required students to scramble to ready their home environment to be able to participate in hours of virtual learning and meetings per day. For the duration of the summer portion of the YES for CURE program we encountered students who simply were illprepared for virtual learning. As a result of these experiences, we have learned that it is imperative to help students assess and prepare their environment for a virtual meeting including finding a place to avoid environmental distractions (e.g., visual, sound), avoiding multitasking (e.g., cell phone use, working on another project), setting aside time for meetings and alerting those around them of their need to focus. Additionally, students were challenged with limited internet bandwidth; some were participating via a cell phone instead of a computer. One recommendation we suggest to those program managers considering virtual programming is to screen students to learn if they can accommodate virtual learning and understand where they need support. For our upcoming 2021 summer program we designed and deployed a survey to evaluate student readiness to participate in virtual programming. However, the challenge for program managers is what to do with this information if students are not prepared for virtual learning. Do students screen out of the program? Does the program have additional funding to support students' at-home virtual learning needs? Do programs try to secure incremental funds to accommodate students so they can participate in virtual programming? In other words, it is important for programs to determine the extent to which they can support students in virtual learning.
Concluding Remarks. The necessary and rapid adoption of virtual learning for DF/HCC's traditionally in-person student research training program due to the COVID-19 pandemic opened the door to new ways to add value to its YES for CURE program; it highlighted how virtual programming can enhance STEM training for high school and undergraduate students. The lessons learned call attention to the importance of sound logistical planning, training of program leadership, mentors, and facilitators, and program execution. Additionally, this experience highlights the power of using virtual touch points to build rapport and relationships with students, facilitate peer-to-peer interactions, expand student access to global speakers, and supplement the invaluable hands-on experiential learning that students gain from immersion in research. Of all the lessons we have learned, perhaps the most significant is that virtual programming is more than a temporary back up to in-person learning but is a vibrant and value-added element to in-person STEM training for high school and undergraduate students.

Author Contributions
The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.