Continuing to Meet the Needs of Middle School Students in Southwest Florida Despite COVID-19 Restrictions

: We describe two established STEM camps that went virtual following the outbreak of COVID-19 during the summer of 2020. Our Summer Research Opportunity (SRO) camp engages middle school science fair winners in a forensic research opportunity, while The Immokalee Foundation (TIF) camp introduces middle school students from historically underrepresented groups to STEM topics. Home kits were carefully prepared and delivered to all participants to enhance their engagement with our faculty facilitators. Modifications to accommodate the virtual experience are discussed. Our program assessment used a retrospective pretest-posttest design delivering a modified STEM Semantics survey to measure interest in STEM topics and STEM careers. While the programming and the population was different for each camp, both programs achieved positive outcomes in the virtual environment enhancing participant interest in, excitement for, and impor tance of STEM topics and STEM careers. The local TV news even broadcasted a short video of one of the camps.


INTRODUCTION
Students need exciting experiences that capture and speak to their interests both in school and beyond the classroom, sparking a lifelong passion (PCAST, 2010). During the past 10-15 years, much has been written about the positive impacts resulting when K-12 students engage in out-of-school time (OST) science, technology, engineering, and mathematics (STEM) activities and summer camps (Clark et al., 2016;Essig et al., 2021;Mohr-Schroeder et al., 2014;Robles, 2018; for a review see Young et al., 2017). Although student interest in STEM can wane during the middle school years, extracurricular educational environments can positively impact the motivational beliefs and long-term persistence of middle school students (Fortus and Vedder-Weiss, 2014). Middle school students who attend STEM summer camps are at least two times more likely to report a career interest in science and engineering (Kong, et al., 2014;Binns et al., 2016;Dabney et al., 2011;Knox et al., 2003). Students who express an interest in STEM in eighth grade are up to three times more likely to ultimately pursue STEM degrees later in life than students who do not express interest (Tai et al., 2006;Maltese and Tai, 2010).
Additionally, during the summer break, there may be learning loss for students who are not involved in summer classes or hands-on activities. This learning loss is more evident for those students in low-income communities. Therefore, programs offering summer experiences, handson activities, and innovative new ways to keep young minds stimulated and engaged in education are imperative (Gibson and Chase, 2002).
Regarding the evaluation of such OST STEM programming, Wilkerson and Haden (2014) suggest that the evaluation match the duration of the program. For example, programs with short durations (a single day) should focus on short-term outcomes such as interest and attitudes while academic outcomes are more appropriate for programs with longer durations such as a two-week summer camp. They further suggest that the program duration informs the data collection methods. Shorter programs should have shorter surveys and longer programs can use more data collection tools.
Here, we describe two established OST STEM summer camps for middle school students that were shifted to a virtual format in the summer of 2020 due to COVID-19 restrictions. The student populations participating in the two camps could not be more different. Summer Research Opportunity (SRO) is a research camp for students who participate in a regional science fair during the previous school year. The Immokalee Foundation (TIF) camp is offered to students enrolling in summer camps in the migrant farming community of Immokalee, Florida. Although the camps cater to different populations of students and have differing durations, both programs were assessed with a common short survey. In both camps, we measured participants' interest in the STEM topics that were part of the camp and STEM career interest. TIF camp also measured changes in the participants' math readiness and environmental awareness.
While the literature indicates exposure to STEM fields in a camp setting has beneficial developmental outcomes for youth, it is unclear whether engagement in a virtual camp experience will have similar effects. We found evidence of STEM camps pivoting to virtual during the pandemic, (Rugh, 2021, teachHouston, 2021 however, learning outcomes associated with these programs is currently limited. We note that Braswell et al. reported on a 2020 virtual summer camp for Black and Latina girls focused on computer science and did find an increased interest in computing confidence (Braswell et al, 2021). Due to the lack of literature, we do not hypothesize a directional outcome for changes in perceptions; rather, we look to the data collected to determine and describe the extent to which change occurred.

METHODS
Camp Descriptions. Facilitators develop single-year, selfcontained activities. While some camp participants may attend the camp for more than one year, the majority of the students attend the camps for a single year, so a focus on scientific development over multiple years is not a focus. Both camps followed a similar daily schedule with morning, midday, and afternoon sessions with one of these sessions being "off platform" for self-paced work. The schedules for three two-week camp sessions of SRO, TIF camp (Math and Chemistry), and TIF camp (Environmental Science) are shown in Figure 1. An undergraduate STEM intern was also designated to be a part of all of the virtual camp meetings to monitor and record all meetings for compliance. Both camps were designed for middle school participation and contained supply kits for each STEM topic that was part of the different camps, developed and allocated by the STEM faculty groups, and delivered to all participants. The SRO participants picked up and returned their kits through a parent pick up line located on the FGCU campus. The kits for TIF camp participants were delivered by TIF personnel prior to the start of camp. Both camps participated in adapted versions of the short, online STEM Semantics survey so we were able to measure STEM interest in the topics of the camp and STEM career interest in both populations (See Instruments below).
Summer Research Opportunity (SRO). SRO has been running through the Whitaker Center for STEM Education at FGCU as a two-week face-to-face day camp since 2006. Students who have participated the previous school year in the local regional science fair for Lee and Charlotte counties in Southwest Florida receive an email from the regional science fair coordinator in January announcing the summer camp with a link to the application. No demographic information is solicited as part of the application. Participants are selected from these applications based on their essays and references. SRO campers are announced at the regional science fair awards ceremony. The goal of this camp is to provide an enrichment experience for students with high interest in STEM areas. Each January an FGCU faculty member is identified to bring a research experience in their STEM discipline to these students. For the summer of 2020, the camp was developed in a virtual format with a Forensic Science theme. Thirty-four middle school students participated.
Forensic Science during SRO. The 2020 Insects, Bones and Crime theme was a two-week virtual experience designed to bring hands-on learning into participants' homes by combining synchronous virtual meetings with specially designed "beetle hotel" kits that parents retrieved prior to the camp start date through a socially distant and masked pick-up line. At the conclusion of SRO, the beetles and supplies were similarly returned to the Forensic Anthropology group.
Each participant received one beetle hotel (a one-gallon sealed glass jar) containing 15 grams of beetles and a substrate (feathers or cedar chips). Additional materials included a heating pad, spray bottle, roll of paper towels, thermometer, data logger, latex gloves, 1/4 in mesh wire tray, plastic tweezers, tally counter, cardboard box to minimize light, and notebook. The beetles consumed cooked chicken bones without skin. A photo of a beetle hotel kit is provided in Figure 2. Additionally, we confirmed that each student had a stable internet connection and a device to participate in the daily sessions.
Over the two weeks, participants evaluated insect abun-dance in their individual beetle (Dermestes maculatus) hotels as part of a larger study to better understand the impact that temperature, humidity, moisture, light, substrate, and bone rendering processes have on the health of the colony. The participants also recorded changes to the sterile bones that were a food source for the beetles. These dermestid beetles are ubiquitous in the United States where they eat dry carrion and are often found in large food processing plants.
They do not pose a health risk and do not bite. Participants were organized into groups of five or six. Each group was defined by the unique beetle hotel variables each member of the group was specifically observing. Each group of participants was assigned to a member of the Forensic Anthropology research group studying with the STEM faculty facilitator, Dr. Heather Walsh-Haney. The groups met virtually via Microsoft Teams. The groups met twice a day: individual groups met once in the morning for a brief introduction to the day's activities and observations for approximately 15 minutes each, and the full group met for an afternoon session of about two hours to discuss the activities, share the results of their observations, and participate in a discussion on critical thinking questions, as well as allow time for questions.
The daily afternoon sessions included PowerPoint presentations that began with a guiding question linked to video clips which gave way to inquiry questions that had participants operationalize for their daily data collection. The daily presentations followed a 5E model (Bybee, 2009) of instruction and included Florida science standards related to • The use and understanding of previous research to underscore that scientific results are not always conclusive; • Discussions of the function of scientific models; • The knowledge that science is open to change and that scientists' varied backgrounds, talents, and goals influence their interpretations of their findings and that competing interpretations are the strength of science.
The science standards were tied to the overarching theme of SRO that was largely based upon research from the University of Tennessee-Knoxville's Anthropological Research Facility (ARF) and why their findings may lead to inaccurate time since death estimates when applied to Florida forensic cases (Megyesi et al., 2005). Using the ARF model to compare participants' beetle hotel micro environments opened the door for the presentation of other scientific models such as natural selection, geologic processes, and principles of inheritance that informed the creation of their group hypotheses and subsequent analyses using the beetle colonies. On the last day of the two-week SRO, the participant groups provided an online presentation highlighting their group results. their "beetle hotel". Each hotel contained either a feather substrate (left) or a cedar chip substrate (right) along with a heating pad, spray bottle, roll of paper towels, thermometer, data logger, latex gloves, 1/4 in mesh wire tray, plastic tweezers, tally counter, cardboard box to minimize light, and notebook.
The Immokalee Foundation (TIF) Camp. A partnership with The Immokalee Foundation began in 2018 with the first camp launching the summer of 2019. The Immokalee Foundation was interested first and foremost in raising the math skill levels of their middle school students (rising 7th and 8th graders) to better prepare them for higher level STEM courses in high school, ultimately allowing them greater access to STEM career paths. Additionally, the Whitaker Center facilitators wanted to expose the participants to other STEM topics in an OST setting. The 2020 camp ran four days a week for four weeks and included two weeks of math sessions and chemistry sessions and two weeks of environmental science activities during the last two weeks of June and the first two weeks of July. The camp facilitators chose the Google Classroom platform to host lessons, videos, and activities to engage participants during the camp based on their familiarity with the platform the previous spring. Our Immokalee Foundation partners confirmed that all participants had a stable internet connection and a laptop to participate in the camp. In 2020, forty-nine rising 7th and 8th graders were recruited by The Immokalee Foundation to participate in the camp. All students were underrepresented in STEM with over 95% of the students identifying as Hispanic. Approximately one-quarter of the rising 8th graders in the 2020 camp had participated in the 2019 camp as 7th graders. We vary the activities annually to account for students returning to the camp. Refer to Figure 1 for the sample daily schedule.

Mathematics during TIF Camp.
In 2020, participants met virtually via Zoom from 10:00-11:00AM each morning of the camp and were then assigned two pre-recorded and uploaded videos with an accompanying guided activity each afternoon. A third "challenge" video lesson was also provided each day, and participants who excelled in that day's lesson were encouraged to also attempt the challenge. The two required videos were each kept to a 25-minute maximum, and the challenge videos were typically under 15 minutes. This is in contrast to 2019 when participants met for over two hours each afternoon in a face-to-face setting on FGCU's campus.
Students were also assessed daily with various activities. The day began with a rapid-fire five-question assessment of the previous day's lessons. These were then discussed and explained immediately after their completion. TIF camp participants received a kit of supplies that aligned with the daily math activities. Several daily activities included implementation of games that the participants could play again later at home with siblings such as a dice game called SKUNK to demonstrate probability, battleship to demonstrate Cartesian coordinate graphing, and Mancala to sharpen skills in basic linear inequalities and logic. Students also finished each day with a Kahoot! quiz that assessed their learning from that day as well as previous days. Each day students also completed a worksheet or group activity to practice the skills learned that day to reinforce them. These informal assessments allowed students multiple opportunities to determine if they were properly completing the standards we had outlined without any weight attached to them. There was no repercussion for doing poorly, and throughout the camp the goal was repeatedly shared with students that we just wanted more confidence with the material and a chance to practice with some common areas of weakness.
Chemistry during TIF Camp. During our planning of the chemistry portion of the 2020 TIF camp, we considered two options, pre-recorded lectures or live Zoom, to deliver the curriculum. Ultimately, to better serve and involve the participants, we decided not only to perform all experiments live via Zoom but designed and delivered chemistry kits to all the participants. The chemistry sets included all chemicals and materials necessary for the students to perform the experiments themselves, following the instructions from a booklet developed for the camp. The instructors performed the experiments live and the participants followed the protocol as outlined in their booklets and asked questions during the experiments. The chemistry portion of TIF camp was even aired by a local TV station (YouTube, 2020). The sessions were delivered in 60 minutes daily over two weeks, from Monday to Thursday. Because of its relevancy, the first experiment was the production of hand sanitizer following the World Health Organization's (WHO) recommendations (WHO, 2010). At the same time, all safety protocols were covered to ensure the safety of the remote learners. The remaining experiments were developed by the instructors or slightly modified from Flinn Scientific (flinnsci.com/stem/). The experiments included: 1. Make hand sanitizer and safety 2. Make elephant toothpaste 3. Make invisible ink and reveal it 4. Study UV light and fluorescence 5. Make soap 6. Examine density 7. Build circuits 8. Observe chromatography of plant pigments The 2020 camp was in contrast to a "normal" face-to-face setting when middle school participants would be transported to the FGCU campus for up to two hours of hands-on and fun chemistry experiments.

Environmental Science during TIF Camp.
In 2020, COVID-19 necessitated virtual programming with facilitators structuring the Environmental Science during the TIF measuring changes in mathematics content knowledge and comparing that to the previous year's face-to-face results. This was in an effort to increase the mathematics competency of the participants to better prepare students for high school science and math courses. Students who earn higher scores on eighth-grade science and mathematics achievements are more likely to complete degrees in STEM (Maltese and Tai, 2011). The number of science and math courses taken in high school also has a positive effect on the probability that a student chooses a STEM major in college (Federman, 2007) and increases interest in a STEM career (Sadler et al., 2014). In both years, the math skills of the participants were measured pre-camp and post-camp through direct examination using assessment items from the Florida Standards Assessment (FSA) for 6th grade.
We were also interested in measuring whether TIF camp participants' environmental worldview was affected through their virtual field trip experiences compared to last year's measurements where the participants engaged in outdoor field trip experiences.
Instruments. We utilized a modified STEM Semantics Survey (Tyler-Wood et al., 2010) to measure interest, excitement, and perceptions of importance about the STEM topics that participants were exposed to in both the camps (i.e., chemistry, forensic science, math, and environmental science). This survey asks students their perception of certain STEM topics and STEM careers as part of their camp experience. The topics for our virtual camps included Mathematics, Chemistry, and Environmental Science for TIF camp and Forensic Science and General Science for SRO. The questions in the STEM Semantics survey have participants rate perceptions of STEM topics based on excitement, importance, and interest on a 7-point Likert-type scale. Each survey concludes with the same sets of perceptions being asked about STEM careers. A sample question set is shown below for the STEM Semantics survey in the retrospective pre-post design.
To determine the impact of the Environmental Science programming on TIF camp participants' environmental and ecological worldviews, we used the environmental awareness survey of Manoli et al. (2007). This assessment tool is designed to determine the environmental worldviews of children and is an adaptation of the original New Ecolog-camp so that participants explored a different virtual field trip of a Southwest Florida ecosystem in the first half of the daily program. During the second half, participants were asked to conduct a backyard ecology experiment in the afternoon. All materials for these experiments were provided in kits assembled by the Environmental Science facilitators. The virtual field trips were created by developing 3-D spaces with embedded ecological information through videos, graphs, and photos for the students to explore. These spaces were generated in the cloud on Theasys's website (https:// www.theasys.io/) from images taken using a GoPro Max 360-degree camera. The backyard ecology experiments included building and installing rain gauges, tracking rainfall over the course of the program, seed predation, backyard bird diversity, and bug bingo. All of these activities required some level of data collection that was ultimately analyzed by the end of the camp, and participants developed and presented this information to their peers in small groups on the last day of the camp.
Previously in 2019, the Environmental Science TIF camp took place on an FGCU satellite campus adjacent to the Naples Botanical Garden. The camp centered around water quality issues from inland regions to the coast. Students were instructed on the scientific method at the beginning of the camp and then applied that knowledge to water quality issues in Southwest Florida in a series of experiments the students designed and implemented with camp instructor supervision. Data collection took place in a series of field trips that included sample collections from the participants' own city, Immokalee, as well as coastal environments in Marco Island and Naples, Florida. Participants were also introduced to geographic information systems (GIS) using Google Earth. At the conclusion of the camp, participants were placed into small groups where they analyzed their data and developed and presented a PowerPoint presentation to TIF and Naples Botanical Garden staff.

Major Research Question and Hypothesis.
We set out to answer the following research question during the summer of 2020 as part of our virtual programming evaluation: How does participation in a virtual summer STEM camp experience change student perceptions of chemistry, math, forensic science, and environmental science?
To measure changes in STEM topic interest, we employed online surveys formatted using the retrospective pre-post design model (Little et al., 2020). This survey design helps to mitigate any measurement error that may arise from social desirability or confirmation bias. In this approach, each survey is administered only once at the conclusion of the camp.
During TIF camp, we were also interested in directly ical Paradigm (NEP) survey tool for adults (Dunlap and Van Lierre, 1978;Dunlap et al., 2000). Participants rate their agreement with statements on a 5-point Likert scale.
We examined three factors that the statements fall into from this survey: Rights of Nature, Human Exemptionalism, and Eco-Crisis. This allowed us to determine whether the overall experience of TIF Environmental Science camp was positively influencing students' environmental and ecological world perspective.

Summer Research Opportunity.
For several years prior to 2020, middle school student participants from this same demographic (rising 7-9th graders each year) were provided a survey to formatively assess the SRO program. The main purpose of this survey was to provide feedback regarding the program to plan for future events. Students were asked questions about the program, future programs they would like to see, and the helpfulness of the staff executing the programming. In 2016, a longitudinal study was also conducted for SRO surveying students who attended from 2006-2015 through a paper survey mailed to the parent's home address. With a 25% response rate, most (97%) of the surveys returned on this camp reported a positive experience. In almost all surveys participants indicated they planned to or were attending college (98%). However, only about 50% of the students surveyed self-reported that they would major in STEM in college after high school or chose a STEM major now that they were in college (Whitaker Center, unpublished results).

The Immokalee Foundation Camp.
Mathematics. In 2019, the middle school students were given a pre-and post-test on days one and eight of the camp and then the difference in scores was studied. This test was made up of three sections that mimicked the Florida Standards Assessment (FSA) and was given in three different timed segments with each section having different components and a total of 66 questions. Both times the test was delivered in a face-to-face modality with students completing the test on paper.
Students were instructed to simply try their best in both trials and not to guess if they had absolutely no idea how to begin a question in both trials. This was done as to not skew the results in either instance with the intention of informally testing their genuine knowledge during both trials. Throughout the camp, we chose to focus on main topics covered primarily in the first two sections of the test. In doing so, we could do more activities and facilitate a more abstract level of understanding, even though not all topics were covered. That decision was made simply due to the limited time and necessity to increase student performance in the skills that were most commonly used on a daily basis.
Environmental Science. Changes in students' environmental world views were evaluated using the survey of Manoli et al. (2007) specifically designed to assess middle school aged children's environmental attitudes. In 2019, the New Ecological Paradigm Scale for Children (NEP) survey (Manoli et al., 2007) was administered to students in an online electronic version on the first day of the Environmental Science Program through the CheckBox survey tool. The survey was given again at the conclusion of the program to assess the impact of the program on the students' environmental perspectives. A total of 22 participants from this program completed the survey.

Current Evaluation for Virtual Camps.
Both Camps. The modified STEM Semantics survey was delivered in a retrospective pre-post design model once (instead of pre-and post-camp) as it has been shown that students as young as middle school students can reliably answer surveys in this format (Little et al., 2020). We administered the STEM Semantics survey for forensic science, general science, and STEM career interest the next to last day of SRO. We administered the STEM Semantics survey for math, chemistry, and STEM career interest after the first two weeks of TIF camp. This survey was also administered at the end of the two weeks of the environmental science portion of TIF camp for interest in environmental science and careers in STEM.

Additional Evaluations for TIF Camp.
Environmental Science. In 2020, TIF camp participants were given an electronic version of the New Ecological Paradigm Scale for Children (NEP) survey modified to the retrospective pre-post format on the next to last day of the camp. A total of 42 participants were present the next to last day and completed this survey.

Mathematics.
In 2020, students completed a pre-and post-test similar to the 2019 test using the website Socrative. They were again asked to simply try their best and leave unanswered any question that they did not have any idea how to solve. This time the pre-test was completed prior to the beginning of the camp on day one and the post-test was completed live during the final day's Zoom session. Also, the pre-and post-test was narrowed down to cover only the seven math topics discussed during the camp sessions. The 40 questions included both basic and advanced examples from each of those seven topics. These were different areas of study from the previous summer, because some participants repeated the camp from the year prior and we did not want to simply repeat the same material.

STEM Topics.
To compare participant perceptions of the STEM camp topic areas before vs. after the camp experiences, we performed paired-samples t-tests of the data collected to test the hypotheses that the mean responses to the items before and after were equal. The survey-paired adjectives fell into three categories that were examined separately: importance (a measure of value), excitement (a measure of engagement), and interest (a measure of intrinsic motivation). Prior to conducting the t-test analysis, we examined the assumption of normally distributed difference scores and found that all the data followed normal distribution patterns (Kline, 2011) with the exception of the after-camp chemistry data. For this reason, the chemistry data was treated differently, and a Wilcoxon Signed Rank t-Test was conducted with effect size r as noted in the table notes.

Importance of STEM Topics.
For all STEM topics for each camp, participants reported statistically significant increases in their perceptions of the STEM topic's importance after attending camp. For all these constructs, the effect size, Cohen's d, was estimated using Cohen's guidelines (1992), and most effects were found to be of medium size (see Table  1).

Excitement for STEM Topics.
With the exception of general science, participants reported statistically significant increases in their excitement about STEM topics after attending camp (see Table 2). Among the significant results, effect sizes ranged from medium for chemistry (r = .61), math (d = .58), and forensic science (d = .61) to large for environmental science (d = 1.06).

Interest in STEM Topics.
With the exception of general science, participants reported statistically significant increases in their interest in STEM topics after attending camp (see Table 3). Among the significant results, effect sizes ranged from small for forensic science (d = .34) to medium for math (d = .67) and chemistry (r = .62) and environmental science (d = 1.05).

STEM Careers.
To examine how perceptions of STEM careers changed as a result of the camps, we performed paired-samples t-tests on the pre-vs. post-means. Prior to conducting the analyses, the distribution of scores was again inspected. Only TIF camp perceptions of STEM careers as exciting was found to be unevenly distributed and was further determined to not significantly affect subsequent analyses nor the interpretation of results. The null hypotheses of equal means for pre-post measures of excitement for, importance of, and interest in STEM careers were rejected for each   of the three pairs of measures, with each post-camp perception score being significantly higher than the pre-camp perception score (see Table 4).

Mathematics -TIF Camp.
FSA Testing Results. Because we gave a math assessment both in 2019 and 2020 and the student demographic was similar we can compare pre-and post-camp test scores and performance between the face-to-face and virtual formats of the math portions of the camp. In 2019, the test was set up to mimic the sixth grade FSA test. This test contained a total of 66 questions with a total time of 105 minutes utilizing two quick breaks after 35 and 70 minutes. It contained multiple-choice, free response, and bubble response questions. While the variety of questions is a useful tool for assessment, it made quick results and feedback for the camp directors difficult to accomplish. Thus, in 2020 when the switch was made to a virtual camp, we also implemented a change to the number of questions and format. For the virtual camp there were 40 multiple-choice questions completed in one 60-minute online session via Socrative. This allowed us to immediately begin the data analysis to identify areas of weakness that needed more focus and topics where the student scores supported a better understanding of the material.
For both 2019 and 2020 TIF camps we ran a related-samples t-test with pre-post-design to examine if there was any statistically significant difference in the math test scores for each individual student. As outlined in Figure 3, during both the summer of 2019 and the summer of 2020, we saw a statistically significant difference in test scores. In 2019, participants increased from an average pre-test score of 24.49% to an average post-test score of 44.22%. This represents a 98.83% improvement. In 2020, participants increased from an average pre-test score of 24.73% to an average post-test score of 41.14%, which represents a 66.37% improvement. Even though students did excel more in 2019 with a face-toface modality, we still measured a significant improvement in 2020 using the virtual format. We also note that in 2020 less than half of the time was spent live with the participants than in the previous year.
For both 2019 and 2020, Figure 3 shows that there was a significant improvement from pre-to post-test. Running a paired samples t-test on each set of data saw an extremely significant difference with p-values less than 0.0001 in both years. Thus, there is sufficient evidence to support the claim that participants showed significant improvement from Day 1 to Day 8 in both years of the camp.

Environmental Awareness -TIF Camp.
We analyzed the data for TIF Environmental Science NEP surveys using a paired samples t-test on the pre-and post-test survey data. The participants in both years of this camp were similar. Our participants from both 2019 and 2020 began the programs with a slightly pro-environmental worldview (Mean Total NEP scores for both years = 2.4 on a scale of 1-5 with numbers closer to 1 being more pro-environmental worldviews, see Figure 4 and Figure 5). By the conclusion of the program, participant total scores were enhanced by approximately 0.2 points in 2019 and by 0.3 points in 2020, towards a more pro-environmental worldview.
We further analyzed our data to investigate which factors of participants' views were more or less impacted by the program. The average score on the "Rights of Nature" at the beginning of the program was 2.8 in 2019. At the completion of the program there was a very significant shift in participants' views on the "Rights of Nature" to a more pro-environmental perspective with 1.1-point shift in overall scores (Figure 4). In 2020, perceptions of the rights of nature at the start of the program averaged 2.1 and shifted 0.5 points to a more pro-environmental worldview ( Figure 5). Compared  to 2019, the participants in the program in 2020 both started and ended the program with more pro-environmental perspectives.
Regarding the "Ecological Crisis" factor, the 2019 participants scored an average of 2.4 on the survey at the beginning of the program. Our data indicated a strong trend for a shift towards a more pro-environmental view by the end of the program (post-test scores averaged 2.1). In 2020, participants started the program with an average score of 2.2 and showed a significant shift towards a more pro-environmental view (by the end of the program scores averaged 1.9).
Interestingly, for questions on the factor of "Human Exemptionalism" (the belief that the relationship between humans and the natural environment is unimportant, and humans can adapt to environmental issues), the average score was 2.5 on the pre-test and the scores actually became less pro-environmental by the end of the program with an average post-test score of 3.1. For the 2020, participants showed no change in their views for this factor averaging scores of 3.3 both before and after the program.

Student Presentation Hypotheses -SRO.
The student research groups in SRO were asked to present on their two weeks of findings on the last day of the camp. As an example of the student hypothesis and results, one SRO group compared their beetle hotel temperature (that was kept between 82 and 84F with a heating pad) and beetle abundance to an SRO group that was unable to maintain a consistent high tank temperature (the temperature range was 74 to 78F which was the ambient temperature of the home). Their hypothesis was as follows: Temperature differences between the two SRO beetle hotels would not affect insect abundance. Indeed, these participants did find there was a statistically significant difference between the temperature and beetle abundance of the two colonies. The group surmised that the beetles prefer warmer temperatures as measured by the increase in insect abundance. This provided an excellent learning experience on the nature of science for the participants.
Impact on Other Stakeholders. While we did not formally measure the impact that these virtual camps had on our instructors, FGCU student assistants, or our community, we offer the following insights on how the virtual programming affected these groups.
Instructors. While many activities were cancelled during the summer of 2020 due to the pandemic, the core group of faculty instructing these two particular camps felt an obligation to offer STEM programming to the student groups served through these programs. We stretched our thinking to include activities that could be done at home and managed through virtual check-ins and created new ways of engaging with the environment through virtual field trips that before this summer were not required of us. In hindsight, while these activities were a lot of work, the satisfaction that the Figure 5. Assessment of students' environmental worldviews at the conclusion of the 2-week environmental portion of TIF camp in 2020 that met virtually. Students were asked to assess their perceptions before (blue) the program and after (orange) the completion of the program. Comparison of mean and standard error of the pre-and post-test scores from the NEP Scale for Children assessment (Manoli et al., 2007). Survey responses are scored on a scale of 1-5 with scores below 3 representing more pro-environment worldviews. Data from the assessment are grouped to identify patterns in different aspects of students' views along with the total score. Double asterisks (**) indicate a significant shift in average perceptions towards a more pro-environment perspective at p < 0.05. (nd) indicates no significant shift in the average perception (p > 0.10).

Figure 4. Assessment of students' environmental worldviews
before and at the conclusion of the 2-week environmental science portion of TIF camp in 2019 that met in person. Comparison of mean and standard error of the pre-and post-test scores from the NEP Scale for Children assessment (Manoli et al., 2007). Survey responses are scored on a scale of 1-5 with scores below 3 representing more pro-environment worldviews. Data from the assessment were grouped to identify patterns in different aspects of students' views along with the total score. Single asterisks (*) indicate a trend for a shift in mean perceptions towards a more pro-environment perspective at p < 0.10. Double asterisks (**) indicate a significant shift in average perceptions towards a more pro-environment perspective at p < 0.05. authors of this paper received from executing the programming for these participants was worth the effort. FGCU Student Assistants. Student assistants were overwhelmingly positive about the virtual experience and were inspired by how well participants followed instructions, collected data, and understood the virtual platforms. The graduate student assistants working with the SRO program noted that the virtual platform challenged them to "rethink how to best communicate and engage with students to develop the best learning outcomes possible". They also commented that they spent less time cleaning and sanitizing the laboratory space (which would occur after in-person classes or office hours).
Community. During the COVID-19 pandemic, we were forced to cancel many Whitaker Center programs including other STEM camps and OST activities. The facilitators, organizers and partners of the two camps discussed here were motivated to try a virtual camp during 2020. As noted previously, TIF camp was featured on the local news (You-Tube 2020). This helped show our community of Southwest Florida that FGCU was rethinking our STEM outreach programming and innovating to provide services. The foundations that supported these two summer camp programs (see Acknowledgements) were both supportive of our efforts and we feel our ties to those community foundations has strengthened as a result of continuing these camps during the challenging summer of 2020. One of the TIF personnel commented: "The professors did an amazing job of creating projects and activities to keep the kids engaged. Overall, I truly enjoyed the virtual summer program, but I feel as though our students respond better to in-person programming. I still believe that the virtual camp was worth it under the circumstances we were in."

DISCUSSION
While the FGCU STEM faculty participating in the camps described in this paper had previous face-to-face experience with STEM middle school camps, the onset of the COVID-19 pandemic necessitated creative thinking to implement a similar experience for this programming in the 2020. Our data suggest that our faculty along with their student assistants we were able to provide a positive experience for both distinct middle school populations using remote platforms. We saw statistically significant increases in the participant levels of interest, importance, and excitement in the STEM topics of mathematics, chemistry, and environmental science in TIF camp participants and forensic science in SRO participants. No difference in interest, importance, or excitement was noted in the topic of general science for SRO. Because SRO participants have previously achieved advancement to regional science fair, we consider the high scores recorded "before" attending SRO as a plausible reason for this result. We were equally pleased to see that we were able to affect participant interest in STEM careers after two weeks of camp (both TIF camp and SRO) and that this interest was maintained after the fourth week of TIF camp. Even though the SRO gains were not as significant, we note again that this population of participants was previously primed to be interested in a STEM career prior to participation as the student participants were regional science fair participants.
While we were pleased to see gains using the modified STEM Semantics survey, we recognize several limitations to our study. First, it was clear from the data collected that some students entered their assigned identifier codes incorrectly, making it difficult to match pre-post data at the individual-level. For that reason, growth had to be calculated at the group-level only. Although group estimates can offer important findings for assessing the overall efficacy of the summer camps, they often mask the variability found at the individual-level. Without the ability to match on the ID variable, an intraindividual analyses could not be conducted.
Another methodological limitation of the study was the short duration of the summer camps. Repeated measures research designs are commonly associated with time constraint limitations; for example, if participants were unable to attend all camp sessions, perceptions reported may not be based on a full experience. Attendance can be difficult to ensure, especially among high-risk groups (Pratt et al., 2000). To mitigate problems associated with traditional pre-post measures (e.g., response shift bias), we adopted the retrospective pretest design (Allen and Nimon, 2007); however, there are limitations to the retrospective pretest model, too. Of these, the most salient for this study is that this approach relies on recall, which can be impacted by social desirability bias (O'Leary and Israel, 2013). For example, participants may have felt motivated to make the program look good or wanted to please their instructors. Finally, some of our data showed a slight ceiling effect in the high degree to which our participants endorsed positive initial perspectives about STEM careers and subject matter on the survey items.
While the virtual TIF camp presented logistical limitations on the amount of time that could be spent with participants, the results of the mathematics portion were still conclusively positive and correlate with participant perceptions of increased interest, importance, and excitement regarding mathematics. For future mathematics sessions, a hybrid model may be effective and could achieve profound results as participants could still get the full experience of an in-person camp, with a short online component to complete after hours for even further application of the day's math lessons. One of the limitations to the mathematics data that we acknowledge is that participants were asked to perform the math test online individually. This could have contributed to the lack of improvement for nine participants during the camp as we did not regulate their time limit for the pre-test although they were asked to set a timer for 60 minutes. The Socrative software does not have a built-in timer.
Even in the virtual TIF camp, the results of the environmental awareness survey indicated similar trends to our faceto-face camp assessment. We attribute some of this to the realistic interactive field trip experiences participants were asked to explore on their own with follow-up discussions along with the backyard "ecology" experiments and observations they made on their own. While both environmental programs demonstrated shifts towards more pro-environmental perspectives (more ecocentric and less anthropocentric attitudes and values) of the participants, the larger gains were for the face-to-face camp in 2019. This pattern could potentially be the result of multiple factors we are unable to tease apart from these data. First, even though this pattern existed between the two delivery formats, the 2020 participants started with a more pro-environmental view making it mathematically more challenging to "move the needle" much further (i.e., the closer the group is to either end of the survey spectrum the less room for shifting perspectives). Second, the 2019 Environmental Science TIF camp program was the first for TIF students, and some participated in both the 2019 and 2020 programs. As a result, there may have been some residual effect of the 2019 program on the returning students in 2020, which may have led to the more pro-environmental view at the start of the 2020 program. Lastly, the content of the two camps was different. The 2019 program focused on water quality issues, while the 2020 virtual theme focused on local ecosystem issues. It is quite possible and likely the two themes influenced participants in different ways.
Interestingly for the 2019 participants, for questions relating to "Human Exemptionalism" (the belief that the relationship between humans and the natural environment is unimportant, and humans can adapt to environmental issues), our data suggest participants actually became less pro-environmental by the end of the program with an average. This pattern is on face-value surprising. However, most environmental issues and programs have traditionally been taught to illustrate the negative impacts of environmental issues. In our program, we focused on many aspects of water quality and the impacts of human development. However, we spent a great deal of time discussing ways humans can solve these issues by altering their behavior. It may be that the participants are responding to these questions in a way that suggests our program is giving them hope we can overcome the environmental issues that society is creating. This hypothesis appears to be supported by data from the 2020 program which focused largely on natural ecosystems and processes, on environmental problems and solutions, and showed no change in the participants' perceptions of human exemptionalism.
Future Directions. Some of the practices used during the virtual camps will be incorporated into future offerings of the camps. The use of a retrospective pre-post survey in 2020 allows us to survey students only once instead of twice for each measure. In 2021, SRO was offered on a virtual platform and participants followed a similar schedule conducting experimentation during the midday with an afternoon follow-up. TIF camp followed a hybrid model for the first two weeks with the first week being fully virtual (math and chemistry) and the second week being face-to-face (math and chemistry) as the Southwest region of Florida gradually stabilizes from the pandemic. Facilitators used some of their successful virtual activities from 2020 in the first week and be able to expand the hands-on portion of their camp in the second week. As a best practice used in 2020, the use of after-camp-time activities online will be available for the mathematics programming. The Environmental Science TIF camp will be able to provide a wider range of habitat explorations through the addition of virtual field trips outside of Southwest Florida. The facilitator's development of the 3-D explorations has led us to consider many different locations of interest to TIF students in Central and South America.

CONCLUSION
We note a striking difference between our two populations of participants, one group (SRO) being high achievers in the areas of science and the other group (TIF) being introduced to the STEM topics during the camp. Regardless of population, our results suggest that both virtual camps were able to demonstrate positive gains in student interest, excitement, and importance of STEM topics and STEM careers even in a virtual environment.
We attribute our positive outcomes to the following factors: • Faculty with previous middle school STEM camp experience in the face-to-face environment.
• The creative development of kits by these same faculty (math, chemistry, environmental science, and forensic science) that specifically targeted their activities of the camp.
• Oversight through the Whitaker Center for STEM Education ensuring evidence-based practices are included in the camp activities.
Our results align with the findings rife in the literature about the positive outcomes of K-12 students' engagement in OST STEM experiences (Clark et al., 2016;Mohr-Schroeder, 2014;Robles, 2018; and for a review see Young et al., 2017). Because strong correlations exist in the literature between interest in STEM, future coursework, and careers in STEM (Tai, Liu, Maltese, and Fan, 2006;Maltese and Tai, 2010;Maltese and Tai, 2011;Federman, 2007) we are encouraged by our results and look forward to incorporating some of the best practices necessarily implemented in our virtual camps into future camp offerings.

Author Contributions
The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.