STEM Lab on a Kitchen Table: An Investigation of Remote Student-Driven Problem-Based Research

: In 2020, the COVID-19 pandemic affected formal and informal education programs in the USA. The pan demic had a devastating impact on programs that required a dedicated physical space and in-person laboratory research. The distinguishing feature of New Hampshire Academy of Science (NHAS) programs is the participation of secondary school students in STEM research projects that emulate university-level research. Moving to a remote format presented various challenges. In this case study, we describe and discuss our experiences transforming a summer STEM research program for secondary school students from on-site and in-person to a remote platform, providing details of the planning phase, the logistics of maintaining the quality of the students’ research, and the results of internal and external evaluations. Of the 33 students who participated, 32 completed all central elements of the program, and 25 went further and submitted summary pa pers and presented their research at the remote annual meeting of the American Association for the Advancement of Science. External evaluation found that students saw their work as similar to that of professional scientists, and perceived themselves to have gained proficiency in the use of scientific techniques and instrumentation. Students expressed they missed elements of in-person lab work including social interactions.


INTRODUCTION
The New Hampshire Academy of Science (NHAS) is engaged in STEM Outreach throughout Northern New England. The NHAS provides research opportunities for secondary school students in a well-equipped STEM laboratory designed solely for the use of students and teachers. Our flagship Applied Scientific Research 3-week summer program allows students to experience all components of university-level research and the scientific process from formulating a hypothesis to presenting their research. The students are also offered an optional two weeks at the end of the summer to complete or extend their research.
The COVID-19 pandemic forced a shutdown of all in-person activities in early spring 2020. Motivated by the support and requests of families in our local community, we decided to offer the summer programs in a remote format rather than canceling them, despite their fundamental reliance on lab access and intensive mentor-student interaction. The challenges that were associated with the remote format ranged from the selection of meaningful research projects suitable for the kitchen table to the logistics of preparing for and running the student-driven research. Here, we offer an account of this inadvertent experiment as a case study. The results were both encouraging and sobering with implications for remote STEM outreach beyond the conditions imposed by the public health restrictions of the pandemic.
In what follows, we provide the details of NHAS STEM programming as a baseline against which the remotely administered programs are compared. This allows us to highlight the inherent challenges that were addressed to prepare for and run the summer programs remotely. We measured the success of the effort through internal observations and record-keeping, and externally administered surveys. Finally, we discuss the implications of our experience in the context of the continued use of the remote format to improve STEM outreach to rural areas of New England with limited access to STEM opportunities. Program Description. The importance of informal STEM education in the format offered by the NHAS is supported by research indicating that this model can play an important role in attracting and retaining underrepresented secondary school students in STEM careers, especially girls (Edelson et al., 1999;Hill et al., 2018;Lowell B Lindsay et al., 2009;Miyake, 2017;Riedinger and Taylor, 2016;Stevenson et al., 2021;Tan et al., 2013). Both our institutional experience and the literature suggest that informal STEM programs make durable impacts via the following approaches: a) providing opportunities to conduct scientific research, b) giving multiple chances to interact with a variety of STEM profession-als, c) supplying opportunities to work and correspond with a diverse group to break stereotypes early in the pipeline, and d) performing research that is both culturally relevant to the students and developmentally appropriate, where the complexity of the project matches the knowledge base and comfort level of the student (Cheryan et al., 2015;Hill et al., 2018;National Research Council, 2009;National Science Foundation, 2020;Pluth et al., 2015;Riedinger and Taylor, 2016;Robinson et al., 2016).
The NHAS STEM laboratory is designed to allow middle and high school students from New Hampshire and Vermont to experience authentic scientific investigations using research-grade instruments in a problem-based learning format. In these PBL programs, students are presented with a variety of culturally relevant problems to investigate under the close mentorship of STEM professionals. Culturally relevant projects excite and engage students more than those that do not seem applicable to their everyday lives. All research projects in the NHAS STEM lab tie back in some way to the environment, which is of general interest to our rural students. An example of one of our culturally relevant projects within our molecular biology research platform is the assessment of the genetic diversity of locally endangered lady's slipper orchid populations. This project not only teaches students molecular biology skills but also introduces the students to a concrete example of genetic diversity and its importance in the prevention of potential extinction. As the students learn more about the local environmental chal-  lenges these species face, such as habitat loss, and warmer winters, they can extrapolate these local phenomena to the greater global challenges many species are presented by climate change.
Research programs at the NHAS start with mentors guiding each student through the development of a working hypothesis or the identification of a problem to solve within a research platform. There is no pre-requisite coursework for these programs and, typically, most research platforms are open to all students. Mentors guide individual students based on the student's scientific knowledge and interests to design a project suited to the student's interests and abilities. This often means that middle school students conduct more observation-based experiments or a simpler engineering task. High school students tend to have more complex projects, which are often built on the simpler projects they completed previously.
The skills to conduct the proposed research are developed under close and continued supervision in the STEM lab environment. Students then conduct their proposed research with one-on-one near-peer mentor assistance and expert guidance from adult mentors. The expectation is for students to gradually take control of their experiments, contribute original ideas, and eventually direct their own research.
Often as students start becoming more comfortable with the techniques, they begin teaching and training the younger students in the lab. During our summer programs, instructors take note of these skills in rising high school seniors to identify students to recruit as near-peer mentors the following summer.
The NHAS Applied Scientific Research three-week summer program either serves as an intensive entry point or as the culminating experience for students who have developed their skills and investigations in other NHAS programs. Applications to the Applied Scientific Research program are open to rising 6th -12th grade students and we offer financial aid to ensure students are never turned away due to financial need. Students present their findings at the end of the summer program to their fellow students, families, and STEM professionals in a formal lecture hall. Students can elect to write a summary paper of their work and results. This writing often occurs at the end of the summer during an open period when students can return to the lab if desired. Students whose written summary papers pass peer review by graduate students are given the opportunity to present their research at the annual AAAS meeting. Following this professional conference presentation, the students have experienced all major components of university-level research and the scientific process.
Our challenge was to re-create all aspects of this personalized lab-based process in a remote format.

Program Adaptation.
In the transition to a remote format, considerable thought went into offering research projects that could be safely and effectively carried out in students' homes. It was important to ensure the research projects continued to offer immersion in the real world of STEM professionals and remained culturally relevant to the students. Since the NHAS is based primarily on a PBL approach that seeks to engage students' personal interests while scientifically addressing a real-world issue with uncertain research findings, nearly all of our research platforms are modeled to have some relationship to an environmental issue that is relevant to our students' lives. Considerable planning went into ensuring that the social and mentoring components that make the in-lab experience personally rewarding were present even in the remote format.
Our student body consisted of 33 students entering grades 6-12 from nine schools and two homeschools. Nine were participating in NHAS programs for the first time. Twenty-six students were living within a 30-minute drive to the lab, three were about an hour away, two were about two hours away, and two were in Texas.
A major concern in moving our Applied Scientific Research program to the students' homes was safety. We also considered the challenges of loaning and transporting research equipment. Research platforms that needed close one-on-one guidance for developing technical skills, such as micropipetting or the delicate techniques required for counting and manipulating C. elegans under a microscope, were largely restricted to those students who had sufficient previous technical training.
Based on these parameters, we narrowed our offerings of research problems for student research to six platforms: 1) molecular biology, 2) machine learning, 3) C. elegans behavioral responses, 4) optical engineering, 5) protein modeling, and 6) surface water quality analysis. Protein modeling and water quality analysis were new additions crafted to fit the remote research environment. Some popular established platforms that we did not offer to the students because they could not effectively operate remotely for technical reasons were: sterile axenic seed culture of lady's slipper orchids and analytical chemistry projects requiring HPLC.
Research Equipment. We adapted and expanded our Equipment Loan Library to loan instruments directly to students rather than to schools and teachers. With scant opportunities for students to share instruments, we purchased additional microscopes, incubators, and laptop computers. Students were loaned the equipment and supplies appropriate to their chosen research. In some cases, the loaned material consisted of a computer or a box containing small equipment; in others, multiple boxes were needed to carry all necessary equipment and supplies ( Figure 1). Equipment and supplies were left outside the lab for contactless pickup during designated time blocks (Figure 1c). This same method was used to exchange faulty equipment or refresh supplies throughout the program. An added challenge was dealing with not only backorders but also delays in orders of equipment and consumable supplies in a pandemic when even paper towels were difficult to purchase.
Online Engagement. Our consultations with educators and students, who were wrapping up a spring of unanticipated remote teaching and learning, highlighted the importance of establishing and following a regular daily schedule and limiting the number of participants in video calls to promote participation. We scheduled twice-daily group meetings arranged by research platform. Each group meeting had fewer than 6 students to ensure opportunities for communication by all students. We planned for a daily schedule of a morning meeting for each research platform, a mid-day talk by students or guests for all students, and an end-of-day wrap-up for each research platform ( Figure 2). Each research group would have a primary mentor scientist and one or two nearpeer mentors. On a very limited basis, students could ask to schedule time in our STEM lab to work with a mentor scientist in troubleshooting a technique or to use equipment that could not be loaned. At the lab, the student was screened for symptoms of COVID-19 before entry and was required to wear a proper mask and follow CDC-mandated social distancing guidelines. The contents of the toolboxes varied between research platforms and between students, depending on factors such as if they had a computer available at home. b) Part of our STEM Lab that we sectioned off to act as a staging area for the packed toolboxes along with any other equipment that did not fit in the toolboxes before being place outside for contactless pick up. c) Equipment loaned out for a single water quality project, ready for contactless pick up. Water quality was the most equipment-intensive research platform. The timing of contactless equipment pick up was arranged to stagger families and ensure sensitive equipment or reagents requiring refrigeration were not left outdoors for an extended period.

Figure 2.
A typical weekly schedule in the remote Applied Scientific Research Program. The morning meetings were organized by research platform and involved all associated students, near-peer mentors, and the primary scientific mentor. These were sometimes supplemented by additional meetings for instruction, such as the Thermodynamics and Intermolecular Forces meetings seen here. The mid-day meetings were lunch talks by guest STEM professionals or by students introducing techniques used in their research. In the wrap-up meeting, students and near-peer mentors were placed in research platform breakout rooms and the scientific mentors moved between rooms. then applied, refined, or added codes when reviewing the rest of the sample. Finally, evaluators condensed some related codes and calculated code frequencies.
To analyze the quantitative outcome data, evaluators followed recommended practices for ordinal data (Carifio and Perla, 2007;Lovelace and Brickman, 2013). For individual items about the program, they reported percentages of responses for each category. For the ratings of proficiency with instruments and glassware, evaluators created scaled scores and analyzed them with parametric statistics for continuous data. Evaluators first examined the reliability of the sets of the pre and post proficiency items. All had a Cronbach's alpha of at least .80, justifying their grouping into scales (J.C. Nunnally and Bernstein, n.d.;Lance et al., 2006). Evaluators then ran repeated measures analyses of variance (ANOVAs) for all students with pre and post surveys.

RESULTS
Logistics. The contactless pickup system for equipment and supplies resulted in nearly all participants being able to pick up their equipment in a timely fashion. One logistical challenge was that we only allowed ourselves one weekend to receive, unpack, and inventory the equipment that had been loaned out for the first 3-week session and then repackage it for distribution to the next 3-week session. There were no losses or major damage to loaned equipment such as microscopes, thermocyclers, or incubators. In a small number of cases, students were late in returning equipment.
Some of the research platforms, such as molecular biology and water quality, required students to be involved in the logistical management of their consumable items as resupplies were needed as the students progressed through their research. Some families made many resupply trips as students learned how to forward plan their experiments. Near constant need for resupplies required mentors to consistently refresh consumable item inventories and keep up with preparing and packaging them. One on-site near-peer mentor spent most of her days preparing media for water quality and C. elegans projects.
Students were creative in finding locations in their homes to set up their laboratories (Figure 3), but this often meant that family members were treated to the experience of living in a home laboratory with all of the accompanying sounds and smells. There were some minor and somewhat humorous remarks from parents about the odor of incubating bacteriological plates in their basements.
Scheduling was an additional logistical challenge. As shown in Figure 2, the schedule for each day was quite busy for students as well as mentors. The virtual group meeting format also meant that students had to voice questions or concerns about their research in front of their peers instead of directly to an instructor.
Research Presentations. At the end of their three weeks of research, students normally present in a formal lecture hall at a day-long symposium where we invite local STEM professionals to provide direct feedback. Instead, we planned for a broader geographic group of STEM professionals to attend three mini-symposia to be held via video conference. These video conferences were over three days to diminish screen fatigue.
In an effort to have an in-person community event that paralleled an end-of-summer symposium with a semblance of social interaction, students would be asked to prepare 3-to-4-minute videos of their research. These videos would be stitched together into a film that we would play for a socially distanced audience of students, families, and STEM professionals at a local drive-in theater. The event would be free and open to the general public.
Program Evaluation. Evaluators created pre and post surveys to collect data from students in the NHAS programs. The NHAS staff, in consultation with Rockman et al. evaluators, developed identical online pre and post surveys to gather feedback about the program and assess outcomes.
Students were asked to evaluate the remote environment and programmatic components, with questions such as, "Online lessons provided by instructors to help understand the research were appropriate." and, "The preparation meetings for planning my project were effective." Students answered both sets of questions with a 5-point scale (1 = Strongly disagree, 3 = Neither agree nor disagree, 5 = Strongly agree).
Students were asked about the authenticity of their work, such as, "Think about the labs you did in this program. How similar were they to the work that scientists do in their own labs?" Students answered these on a 6-point scale (1 = very similar, 6 = very different) and were asked to explain their answers.
Students rated their understanding of 17 types of instruments and glassware, and 6 techniques. Evaluators used a 6-point response scale for this in order to elicit precision and variation in student responses (1 = I've never heard of this; 2 = I've heard of this but have never used it; 3 = I can use this with a lot of help; 4 = I can use this with a little help; 5 = I can use this on my own; 6 = I can teach others how to use this).
The Applied Scientific Research program had a total of 31 individuals in the post survey sample that could be matched to pre survey responses. Evaluators used descriptive statistics to analyze students' reactions to the programs. Evaluators also prepared a descriptive summary of students' open-ended responses. They created codebooks for three open-ended questions: 1) "Please explain your answer [regarding lab similarity]", 2) "What have you liked about the program", and 3) "How could the program have been better?" They generated initial codes for five responses, and Research Platforms. The 33 students enrolled in Applied Scientific Research chose from the six offered research platforms to complete their investigations in their home labs. The distribution of students among the six research platforms can be seen in Figure 4. Below, we give a brief overview of each platform and its challenges.

Molecular Biology.
Because of the extensive use of micropipettes in PCR and gel electrophoresis, this research platform was limited to students who had experience with micropipettes. Two PBL projects fell into this category: one investigated the genetic diversity of regionally endangered lady's slipper orchids and the other tested food products for the presence of DNA markers indicative of genetically modified organisms. The molecular biology projects were difficult to run remotely. Students confused buffers used in DNA extraction and had difficulty in choosing the correct micropipette and pipette tip for a given task.

C. elegans Behavioral
Response. This subject area was also limited to students who had experience with C. elegans because of the manual training needed to maintain the worms and to identify the different developmental stages. Students in this group were continuing research done at our lab in previous years, ranging from measuring the effect of CBD oil exposure on C. elegans fertility to testing genotoxicity assays for use in the worms. Since C. elegans behaviors and development are affected by temperature, controlling/monitoring temperature in students' home laboratories became a common challenge since many Northern New England homes do not have air conditioning.
Machine Learning. There was no need to limit this platform based on student experience, but the only student who opted for it was continuing from a previous year. This project was the development of an object detection model capable of identifying and categorizing the various life stages of C. elegans from high-quality images taken through a microscope. Adapting this project to work remotely meant shifting from a desktop-based to a cloud-based computing platform to allow for computer code to be shared easily among the student, mentor, and near-peer mentor. In addition to requiring a shift in computational resources, operating this platform remotely required supplying the student with everything needed to maintain and photograph a population of C. elegans at home. This enabled him to obtain the high-quality images required to construct both training and testing data sets for the machine learning algorithm.
Optical Engineering. Two of the five projects related to optical engineering were aimed at using 3D printing to develop low-cost optical systems (spectrometer and interferometer) Figure 3. Examples of the student researchers' home labs. a) The home lab of a molecular biology researcher, including a MiniPCR and Bluegel electrophoresis system. b) The home lab of a water quality researcher, including a small incubator for growing bacterial samples. c) The mobile lab of a water quality researcher, used as she traveled from site to site. d) Part of a home engineering lab set up in a basement "darkroom." e) A protein modeling student at her home work station. f) Part of a C. elegans home lab with a microscope and plates of C. elegans.
for education and STEM research. While the Computer-Aided Design (CAD) design of 3D printed parts turned out to be well suited as a home activity, the reliance on the 3D printer located at the STEM lab turned out to be a major bottleneck. Directing the alignment and measurements related to optical breadboard experiments posed an additional problem, which in some instances was not resolved during the program, but postponed until an in-person encounter could be arranged.
Three projects were offered to develop Raspberry Pibased multispectral camera systems for environmental monitoring. These projects allowed students to move through the entire setup of the system, including both hardware and software. To address the potential need to troubleshoot the student's system remotely, all materials and components provided to students were purchased in duplicate, with one exact copy available to the instructor. This strategy allowed us to quickly resolve technical issues. Students were allowed to experience obstacles, but not to be frustrated. The duplicated experimental environment also facilitated remote demonstrations since students were able to replicate instructions instantly and directly compare their setup with the instructor's setup.
Protein Modeling. One of the newly introduced platforms to accommodate remote student research was protein modeling. Participation in this research platform was restricted to high school students as an in-depth understanding of both biology and chemistry is required to assess the computer models. Research projects in this platform varied, but all shared the thread of investigating the binding of the spike glycoproteins of the current and previous epidemic coronaviruses (SARS-CoV-1, MERS-CoV, and SARS-CoV-2) to vertebrate cell receptors. A challenge in this platform was developing questions that were not already being asked and quickly answered by the fast-moving SARS-CoV-2 research field. This platform also presented logistical and mentoring challenges because it was new to us, required the students to quickly become proficient in biochemistry, and was the most popular probably due to its topical relevance (Figure 4). The near-peer mentor played a substantial role in ensuring technical challenges faced by students in using the various modeling programs were addressed quickly.

Surface Water Quality Analysis.
Fieldwork has not historically been a component of the NHAS summer program beyond harvesting tissue samples of lady's slipper orchids from their natural habitats. Since the NHAS is located in rural Northern New England, many of the students we serve live near recreational waterways. This research platform sought to leverage their natural curiosity regarding the health of these waterways. Students' research questions ranged from comparing the health of two local swimming holes to measuring the success of remediation at an EPA Superfund site. This research platform was the most equipment intense re- forms. Thirty-three students participated in the program, 26 were high school students, and 7 were middle school students. Values are given as the percent of total students in both sessions of the Applied Scientific Research program. Protein modeling was the most popular option, with 28% of students. Surface water quality, molecular biology, and C. elegans behavioral response each had 18% of students. Fifteen percent of students participated in optical engineering research, and 3% of students conducted machine learning work. chart shows the total number of students who completed each aspect of the scientific process associated with our program. Thirty-three students enrolled in the program, completed a research proposal, and performed three weeks of investigations. Thirty-two presented their research to their peers and invited STEM professionals at the end of their session. Twenty-five wrote a summary paper, had their summary paper pass peer review, and participated in the remote AAAS meeting by presenting a virtual poster and publishing an abstract. quiring students to be supplied with everything needed for chemical, ecological, and bacteriological analyses ( Figure  1c). Water quality became a whole-family endeavor with parents enlisted into setting up mobile labs in the back of the family minivan and driving students to their selected testing locations.
Evaluation Outcomes. The NHAS performs internal tracking and evaluation for all of our programs in addition to the added external evaluation prompted by our SEPA. As shown in Figure 5, we internally tracked students as they moved through the scientific process in our program. Thirty-three students started the Applied Scientific Research Program and performed investigations over the course of the threeweek program. Thirty-two of these students presented their work to their peers and STEM professionals via our videoconference research symposia. Twenty-five students wrote summary papers of their research. All 25 summary papers passed our peer review, qualifying their authors to submit abstracts for posters to be presented at the 2021 annual meeting of the AAAS. The 25 students then prepared and presented online posters as part of the remote AAAS meeting (https://projectboard.world/ajas/viewAll?tag=NewHamp-shireAcademyofScience).
Our external evaluators included questions about the remote environment in their evaluation surveys. On a 5-point Likert response scale, all students generally agreed that "The group sizes for remote meetings were small enough for me to participate when I wished to do so" (n = 26, Agree = 26.9%, Strongly Agree = 73.1%), and "I was able to contact my instructors when I needed their guidance" (n = 31, Agree = 35.5%, Strongly Agree = 64.5%). Students reported that they would have preferred to be in person at the lab, although they understood why the programs were remote. They also reported a desire for more opportunities for social interaction with their fellow students. About half of the students agreed or strongly agreed with the statement "I got to know other people in my program" (n = 31, Agree = 41.9%, Strongly Agree = 6.5%), which was the lowest rating of all 14 remote environment items. Example comments included: "I really don't think it could have been improved. For a virtual intensive summer science program, I think it was pretty good. Maybe an ice cream social virtually?" "I think it would have been great if more could have been done to introduce the other students to each other, so we could have gotten to know each other better; although it is challenging virtually." "i wish i could have made more friends" The students continued to identify their research as similar to research conducted by professionals in their respective fields ( Figure 6). This identification with performing research that paralleled a professional was especially illustrated in responses to questions regarding the research-grade equipment students were using. The notable exception to identifying with professional scientists was the facet of working from home, which was seen as "not similar" to what scientists do ( Figure 6). One student remarked, "My environment was not sterile because I was at home, which made it challenging to search. Most respondents agreed that their research was similar to that performed by professional scientists. Notably, a few students did state that their homes were not like real science labs for reasons like cleanliness and not having a distinct space for their work.  (Ferguson, 2009).)

NHAS Research Presentations.
Thirty-two of the 33 enrolled students presented to peers and scientists at the end of their session ( Figure 5). The students normally present in a lecture hall in a one-day symposium hosted by a regional engineering corporation on the last day of each program. This provides a professional venue and a visible audience of their peers and invited STEM professionals. These research presentations had to be adapted to the remote world, with mini-symposia taking place over videoconference. Spreading the presentations out over three days added to mentor and student stress and reduced research time, with some students presenting two days earlier than others. Although we were able to invite STEM professionals from outside our region, the virtual venue did not have the same personal rapport. Since giving a research presentation is a culminating step in the scientific process, we considered it important to maintain it as a component of the authentic research experience.
To add a community-based component, we rented a drive-in movie theater. Twenty-three students submitted short 3-to-4-minute informal videos explaining their work that we stitched together into a single 72-minute film. We had a one-night premiere that was free and open to the public. We invited the students, their families, friends, and local STEM professionals. This allowed for a socially distanced event that had over 150 people in attendance (Figure 8).

DISCUSSION
Although there were many hurdles and challenges for both students and mentors in performing STEM research remotely, both the internal and external evaluation results suggest the transition to a remote platform for our Applied Research Program retained the majority of the most important qualities of in-person research. Students were still able to design and conduct authentic research projects with research-grade equipment. The large majority presented their results to their peers and to the broader scientific community ( Figure 5). They still viewed their research as relevant and valuable and their loaned equipment as reflective of what STEM professionals use in their careers. This supports the positive impacts of such culturally relative and extensive PBL experiences noted in previous studies Cheryan et al., 2015;LaForce et al., 2017;Otieno and Wilder, 2010;Riedinger and Taylor, 2016;Tan et al., 2013).
Working from home was the major difference students perceived in comparing their activities to STEM professionals. Although this remote model is no substitute for the onsite lab experience, it is a way of adapting scientific research to ensure that students are still able to get a rewarding experience performing authentic STEM research. It may also inform institutions that wish to operate remote research programs for geographical or economic reasons.
Of the 33 students in the Applied Scientific Research program, all completed three weeks of investigations addressing their research problem. Thirty-two presented to their peers and scientists at the end of the program ( Figure 5). We consider a formal research presentation an important part of the scientific process, so we sought to maintain it, even in a virtual format that did not carry the same prestige or have the same level of feedback from the audience as our in-person symposium.
The drive-in movie event was very successful. It allowed a wide audience to view many different research projects from 23 students in a relaxed and COVID-19-safe environment. Students had to record their films on their own time after the program ended and some students reported that they could not make a film because of other time commitments, such as overnight camps, school sports, family vacation, and the start of the academic year. The drive-in movie allowed for students and families to finally meet or see each other again in-person. They were able to socialize, masked and outside, prior to the screening. We luckily had a clear and beautiful summer night for all to enjoy (Figure 8).
Twenty-five of the 33 students in the programs submitted summary papers ( Figure 5). All 25 papers passed peer was rented for the premiere of the film compiled from the students' 3-to-4-minute informal research presentations. This communal event allowed the students and their families to gather in a socially distant manner to celebrate the students' achievements. review. We typically have at least 90% of the summary papers pass our peer review that qualifies them to attend the annual AAAS meeting. We attribute this success to a mixture of mentors' encouragement of high-performing students to write a summary paper and the self-selection of students who are willing to take time to write a paper after the program has ended. All of the students who passed peer review presented their research at the remote annual meeting of the AAAS. The number in attendance was also closely in line with previous years.

KEY TAKEAWAYS
There were a variety of positive and negative consequences of going remote. With students at home, we saw more parent/family engagement in the projects. Some students in the water quality platform needed to be driven to sites, some students turned kitchen tables into labs, and some had to explain just what it was they were doing with all those text files and 3D models on the computer. Students were able to make their own schedules, which allowed for more flexibility in designing and conducting experiments since timeframes were not limited by the in-person hours of the program. Some students were able to focus better on their projects at home without being in a bustling lab. We feared that students may not have enough unscheduled time to do their research, but did not want them to feel like they were stuck by themselves when they ran into problems. With responses from students commenting both that there were not enough meetings and that there were too many meetings, we perhaps struck a balance.
The use of duplicate experimental setups for students and mentors turned out to be instrumental for the success of some of the engineering projects and will be replicated for future remote programming. Direct access to the experimental setup substantially reduced student frustration by allowing the mentor to assist with the identification of electronic components and troubleshooting software packages.
Both protein modeling and the field research component of water quality analysis were quite conducive to a remote format. The addition of protein modeling projects related to SARS-CoV-2 allowed students to investigate the reason they were all stuck at home and possibly played a role in it being the most popular research platform. With the remote approach, we were also able to reach a wider geographic audience with two students participating from as far away as Texas. Lastly, the drive-in movie was a social hit and we plan to continue this as a community event even after the pandemic.
The negative consequences were largely related to the lack of personal interaction. Without looking over students' shoulders, it was difficult to catch errors, such as those in pipetting techniques, until later in the research process. It was also more difficult to diagnose and troubleshoot problems as they arose without being able to directly observe students. We anticipated technical challenges, but we were not fully prepared for how inventive students could be at incorrectly using instruments. Minor issues related to experimental setup are almost always instantly corrected in a lab environment as part of the continued education, but this instantaneous oversight was not possible with students at home. The remote format hides minor and less obvious mistakes from the mentor's eyes until data are already collected and a repeat of the experiment is bound to cause significant frustration.
Students did not socially engage as much with each other as they do in-person. In normal times, students have lunch together, talk to each other about problems, and chit-chat while working side-by-side. We set up a dedicated hangout space online, but it was not used and students did not interact much outside of group meetings, except with other students with whom they were already friends. This also severely curtailed the facility of students to help each other, limiting students' opportunity to practice teaching and increasing the routing of questions to the mentors and near-peer mentors.
Lastly, there was a personnel challenge in the remote platform. Our summer programs are normally a busy time for our mentors, but the remote version put more of a demand on their time with many hours per day spent in video calls with different groups in addition to prep and cleanup around the lab, answering emails, and coordinating resupply pickups. Some urgent research problems got a delayed response because the appropriate mentor scientist was in another research group meeting, working in the lab, or otherwise not on email to respond.
To the Future. As we plan for the summer programs in 2021, we are implementing lessons learned from last summer. To address the time needed to turn around equipment and supplies from one Applied Scientific Research session to the next, we have scheduled a one-week program in between the two three-week sessions. This will give us more than one weekend to unpack and re-pack boxes for students. We are also planning a hybrid approach for the program in 2021. Students will be in the lab with a small pod 1-2 days a week and work remotely with loaned equipment for the other days. Although we anticipate this having its own challenges as mentors balance students in the lab and at home, we hope the lab time will mitigate many of the downsides we saw with the fully remote model. Mentors will be better able to train students and spot mistakes early. The students will also have in-person social engagement with others in their pod and get the experience of working in a research lab.
The lessons learned from these remote and hybrid experiences will advance the mission of the NHAS to provide students in rural, underserved areas of Northern New En-gland with exposure to STEM education and research, even without access to a nearby lab. The constraints imposed by the pandemic have clearly shown us both the potential of internet technology for STEM outreach as well as its inherent limitations. It has made us all reconsider what can be done at home and fully appreciate the value of the communal experience of in-person laboratory research.

AUTHOR INFORMATION Corresponding Author
Peter Faletra. New Hampshire Academy of Science, PO Box 378, Lyme, NH 03768. nhacadsci@gmail.com

Author Contributions
The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.