Meeting the COVID Challenge to a Research-Intensive Pre-college Science Education Program

: The objective of our program is to foster and facilitate authentic research experiences in middle and high school science courses. We do so by providing students with a complete experience in scientific experimentation and com -munication. The centerpiece is a set of experiment modules in which students investigate the effects of toxic chemicals on living organisms through the use of model organisms such as the earthworm, fathead minnow, and the zebrafish, and chem ical contaminants commonly found in the environment. In parallel, we partner with the University of Wisconsin-Milwaukee science teacher certification program to prepare pre-service teachers to offer real research experiences in their future class rooms. With the COVID-19 virus restricting or eliminating in-person learning, the program’s challenges were (i) to create new ways to conduct experiments virtually that retain elements of the authentic research experience and (ii) to move all of the accompanying facets of the program to online formats. This paper will describe the new online materials and activities that were introduced this past year as well as the challenges they presented and opportunities for the future that they offer.


INTRODUCTION
The Wisconsin Inquiry-based Scientist Teacher Education Partnership Program (WInSTEP) at the University of Wisconsin-Milwaukee (UWM) has been funded by the Science Education Partnership Award Program (SEPA) through the National Institute of General Medical Science (NIGMS), part of the National Institutes of Health, to provide pre-service teachers with the perspective, knowledge, and tools to introduce authentic research experiences into their future middle or high school science courses. This project was inspired by the goal articulated in the 2015 Federal CoSTEM Strategic Plan to prepare 100,000 excellent new K-12 STEM teachers by 2020 and to greatly increase the number of U.S. youth who have an authentic STEM experience each year prior to completing high school, particularly among disadvantaged students and girls who are underrepresented in STEM fields (Executive Office, 2014). It was recognized that meeting these aims would enhance the pipeline of pre-college students who pursue careers in STEM fields and, thereby, pro-vide human capital to meet the nation's scientific research needs. Moreover, these objectives would broadly support the development of a STEM-literate citizenry that understands the roles of STEM in society and actively supports STEM in public discourse and decision-making.
WInSTEP-SEPA partners with the UWM science teacher certification program to prepare pre-service teachers to offer robust scientific and communication opportunities to their high school and middle school students. This program depends on our long-standing collaborative partnership with regional in-service teachers. Its content is schematically shown in Figure 1, which illustrates the WInSTEP-SEPA program elements as well as the various facets of the program that were modified during the COVID-19 pandemic (dashed boxes). These facets are described in greater detail later below. WInSTEP-SEPA includes seven staff members -2 science educators and 5 scientists -who together support various aspects of the program.

PROGRAM DESCRIPTION
Program for Students. Students in middle or high schools engage in conducting one or more in-depth experiment modules that have been designed by WInSTEP-SEPA's scientific staff as their introduction to authentic scientific research. The historical demographics of our program are shown in Table 1.
Each of the modules involves the use of live organisms, including earthworms, fathead minnows, adult zebrafish, and developing zebrafish embryos and probes the impact of environmental chemicals on various life processes and organ systems (Weber et al., 2016;Weber et al., 2013;Tomasiewicz et al., 2014;Weber et al., 2019). Table 2 provides summaries of the modules. Students prepare for experimentation by choosing a chemical to study from an approved list and reading relevant scientific literature about its toxicity supplied by the LibGuide on the program's website (https://sites.uwm. edu/winstep/). Next, they design experiments with teacher guidance and begin their research. Besides physical manipulations in the laboratory, their experiments focus on making extensive observations that provide the basis for data analysis and interpretation.
Research that has an open character to its design and execution often results in unexpected issues as the process unfolds. Thus, the program provides full support for teachers as they undertake these experiment modules with their students. At times, either online consultation or in-person trou-ble-shooting may be needed. A key asset is the LibGuide, which is part of our website and contains a wealth of supporting documentation for each module that is continually updated, including video recordings of experimental set-ups and methods as well as animal husbandry.
The immediate goal at the personal level is for middle or high school students to make their own discoveries about the interaction of environmental agents with organisms. Such information gathering stems from careful planning of experiments, data acquisition, and analysis. To strengthen the rigor of this work flow, students write papers and construct posters about their results, just as members of the professional scientific community do. Then, they submit them for inclusion in the spring WInSTEP-SEPA Student Research Conference.
The Conference, itself, brings together about 600 students and their teachers for a 3.5-hour fest of research communication. Participants listen to paper presentations and then engage in a cacophonous hour of discussing results around posters. During lunch, students glimpse possible futures in science as a near-peer plenary speaker, commonly a college or university student who previously participated in the pro-  gram, reflects on her/his experiences as a budding scientist. The Conference concludes with a variety of awards for research excellence and participation.
Program for Pre-service Teachers. The WInSTEP-SEPA program partners with the UWM science teacher certification program to introduce pre-service teachers to a coordinated pedagogical approach that fulfills many of the goals of the Next Generation Science Standards (NGSS) related to Cross-cutting Concepts and Integrated Science and Engineering Practices (National Research Council, 2013). Teachers-in-training learn how to implement the experiment modules, and they contribute to the preparation for the Student Research Conference. In so doing, they gain a perspective on how to infuse their course work with authentic inquiry. Initially, the pre-service teachers attend workshops in which they learn the content and practice the methods of two experiment modules ( Figure 1). Next, as part of their student-teaching, they work with in-service mentor teachers who are undertaking these modules with their students. At a winter retreat, a number of important, ancillary topics are discussed, including animal care and safe handling and use of chemicals. A key activity brings together the pre-service teacher group and several mentor teachers for a panel discussion about how the modules impact teachers, students, and many facets of their courses. Later, working side by side with staff scientists, the pre-service teachers evaluate the scientific quality of student papers and posters. In the process, their functional scientific acuity continues to develop. Later in the year, when they receive their certification, WInSTEP-SEPA provides them with the equipment to implement the modules and the promise to continue to support them as they embark on their careers as new science teachers.

THE COVID-19 YEAR -OVERVIEW
Our program was implemented successfully and uneventfully for several years until the spring of 2020 when the arrival of the COVID-19 virus in the United States abruptly closed down all schools at the beginning of April. Fortunately, most participating schools had completed their work with the modules, but the in-person Student Research Conference had to be converted quickly into an abbreviated online experience. Moving into the 2020-2021 school year, it was evident that some schools would remain open; others planned a hybrid of in-class and on-line learning; and the Milwaukee Public Schools would embrace a completely virtual model for the school year. Considering that our program centers on students doing real, hands-on experiments, this situation necessitated a variety of changes.
The University of Wisconsin-Milwaukee declared that almost all courses for 2020-21 must be given online. This required that much of the science teacher certification program be moved to a virtual format. WInSTEP-SEPA's work with the cadre of pre-service teachers was constrained in part to this setting as well. Figure 1 shows the components that had to be shifted to online modes of communication. Fortunately, enough in-school mentoring was still available so that pre-service students were able to observe how experienced teachers utilized the modules in their courses either in-person or online.
The entire WInSTEP-SEPA program is fully evaluated by an external firm. During the pandemic year, details of various instruments used to assess students and pre-service teachers were altered to recognize the online setting. Also, modes of conducting evaluations were modified as necessary.

Rationale for Program Modifications During COVID-19.
Doing biological research is fundamentally a hands-on process. Our program is designed to model authentic experimentation in real classrooms. Thus, the thought of resorting to simulated experiments in place of the usual student activities or the work of pre-service teachers with the experiment modules caused the staff to reflect on what of significance might be retained in the virtual setting. The stimulation and empowerment of doing real, personally motivated experimentation would be lost as would learning basic laboratory technique and the challenges of doing repeatable experiments. Still, much of the learning about doing science comes in the form of developing observational skills during experiments that result in data acquisition. Therefore, we decided that it was possible, in principle, to develop and produce online recordings of our modules that provide the student with (i) options to think about experimental design and (ii) opportunities to acquire real-time data as in the classroom. Once in hand, the latter would offer the possibility of significant data analysis and testing of hypotheses.
Our ability to adapt experiment modules to an online medium depended on two assets. First, the staff members who developed the modules are research scientists. They were able to propose plausible online substitutes for in-person experimentation and immediately took responsibility for producing them. Second, these ideas depended on access to expertise in video-making and data science. Working within a major research university, it was possible to engage advanced undergraduate majors in the UWM Department of Film, Video, Animation, and New Genres as collaborators with our scientists to produce videos related to experiments with earthworms and fathead minnows (Table 2). At the same time, a faculty member from the College of Health Sciences specializing in the handling and analysis of large data sets was brought into the group to participate in developing an innovative repository of images of developing zebrafish exposed to several chemicals. A second staff member of the School of Freshwater Sciences took the lead in finding the means to make the repository available to the large group of students in our program, utilizing a university on-line tool called Canvas Learning Management System™.
Another feature of the program supported the transition to virtual learning. Over time, we have assembled numerous on-line resources on the WInSTEP-SEPA website (https:// sites.uwm.edu/winstep/) that contribute to each facet of the student experience (Table 3). During this year of virtual learning, they provided a preexisting foundation of internet-based materials that facilitated the construction of new videos as described below.
Schools that retained in-person student classes or utilized a mixed, hybrid approach presented another issue for the WInSTEP-SEPA staff. Organisms had to be delivered to these classrooms. Additional communication between the program and teachers was necessary to assure the safety of the staff during this transfer process.

Students Doing Authentic Research -Adaptation to
Pandemic Restrictions. The aim of the WInSTEP-SEPA program is to introduce students to authentic experimental science -wondering and hypothesizing about a natural phenomenon, designing and executing experiments and acquiring data to seek answers, deciding how the results address the hypothesis, and then doing more experiments. We do this by offering students opportunities to study the toxicity of substances that interest them in animal models that connect with the content of their course work. Experiment modules 1-3 introduced in Table 2 served as the starting points for the development of online versions that retain important elements of the scientific process. In each adaptation, the central objective was to present detailed visual information that provides students with the ability to gather a variety of data for testing hypotheses and reaching conclusions.
In the first two modules (earthworms and minnows), students obtain data in real time from video records of experiments. Videos were generated by a team of two student interns from the UWM Department of Film, Video, Animation, and New Genres. One student did the videography and the other managed the editing. Their creative insights into how to set up the scenes and assemble the final product were critical to the success of this effort. The staff scientist overseeing these videos was responsible for the final edits. In the zebrafish module, an extensive library of images of developing zebrafish serves as the source of data. Summaries of the virtual forms of these experiments is provided here. Full accounts may be found in the Supplementary Information.

Goals for Online Transition.
Understanding the neurobehavioral consequences of exposure to natural and manufactured chemicals is the central focus of this module (Weber et al., 2016). The following objectives were integrated into the module so that students can discover the impact of potentially toxic environmental agents on specific, easily monitored and analyzed behaviors ( Figure 2).
• Provide a comprehensive series of training videos that develop students' scientific foundation for understanding the methods used in the module.
• Design a virtual experiment that allows each student to (i) conduct the experiment at a pace amenable to that student's ability, (ii) observe contaminant-induced behavioral changes under actual experimental conditions, and (iii) collect and analyze scientifically meaningful data.

NERVE AND MUSCULAR BASIS OF EARTHWORM MOVEMENTS: Effects of Physical and Chemical Environmental Agents
This module investigates the use of earthworms as a model organism for studying neurotoxic effects on the human nervous system. Students conduct various behavioral experiments exploring the concentration-dependent sensorimotor reactions of earthworms to pH and metal solutions, including how they are modified by the worm's physical environment. Finally, students examine how these responses relate to human health and the consequences of chemical exposure.

THE EFFECTS OF LEAD EXPOSURE ON FATHEAD MINNOW: Connecting Behavior and Physiology
This module is a hands-on investigation of the effects of lead on fathead minnow reproductive behaviors. In this module, students observe normal and abnormal breeding behaviors of fathead minnows, compare the effects of lead to mercury using a video-format experiment, learn how changes in these behaviors are related to changes in fish physiology due to exposure to lead, learn the efficacy of different methods of reducing lead exposure, and see the effects of embryonic exposure to lead on embryo growth and larval behavior. The results are then compared to what happens to humans exposed to lead or mercury, i.e., how are fish models of human environmental health.

ZEBRAFISH AS MODELS: STUDYING THE EFFECTS OF ENVIRONMENTAL AGENTS ON HUMAN HEALTH: Effects of Ethanol, Nicotine, and Caffeine Exposure on Embryonic Development
Using zebrafish as models, students examine the general development of zebrafish embryos and malformations that occur due to exposure to various environmental toxicants. By applying the results of the zebrafish embryo to human embryo development, students draw conclusions regarding personal health, environmental hazards, and the risks and benefits of personal and social decisions in relation to these hazards.

EFFECTS OF TOXIC CHEMICALS ON LEARNING AND MEMORY: Using Fish as a Model for Human Environmental Health
This hands-on module uses zebrafish or fathead minnows as models for the effects of lead by using an experimental protocol that is flexible so that a variety of student-directed questions can be answered with fish in a T-maze (e.g., learning abilities of sexes, species, or age groups with or without chemical exposure). Students then examine how fish behavior relates to human health and the biological consequences of chemical exposure.  Goals for Online Transition. Lead (Pb) poisoning remains a critical environmental health issue for many communities, specifically the children in those communities (Hauptman et al, 2017). In this module, based upon research conducted by one of the UWM WInSTEP staff scientists, fathead minnows (Pimephales promelas) are utilized as model organisms in the investigation of the impact of lead/Pb on reproductive behaviors and outcomes (Weber, 1993). As with the earthworm module, educational goals that apply to classroom experiences may be frustrated when transferred to a virtual experience. Thus, additional objectives were included to maintain the experimental character of this module in its online version: • Provide a training video that builds upon those already in our WInSTEP-SEPA LibGuide and develops the scientific foundation for the methods used in the module.
• Use digitized tapes of the actual experiment that the module is based upon to create a set of virtual experiment videos (Weber, 1993). The recordings (i) make it possible for students to observe Pb-induced behavioral and anatomical changes over a 2-week period under actual experimental conditions; (ii) offer students the opportunity to collect and analyze data in a scientifically rigorous manner; and (iii) permit students to start and stop the video whenever needed or return to a previous segment to review the information.
• Encourage students to compare findings. Because all are observing the same video and identical behaviors, this activity helps students improve their observational skills and may reveal visual biases that affect data collection.
• Combine data to see the value of big data sets.
• Compare findings with other students who are observing the same videos, thereby gaining insights into biases in behavioral observation, as well as methods to overcome those biases.

Immersing Students in the Research Experience.
Each topic in the earthworm module was covered in two videos -a teaching video and a virtual experiment. Producing both sets of videos enhances the modular nature of this unit and provides teachers greater flexibility in implementing the module. The teaching video was developed as a compact version of a longer workshop video that was currently in our LibGuide. The virtual experiment videos are of actual experiments conducted by a WInSTEP-SEPA program staff scientist. While the teaching videos demonstrate how to set up these experiments, the virtual experiments focus solely on data collection. Use of close-up shots assist students in seeing the behaviors more clearly and, therefore, facilitate data collection.
Using close-up scenes of earthworm behavior, students observe and collect data under varying environmental conditions. Through this process, the student experience approaches "hands-on", as much as is possible in a virtual experiment. An additional educational outcome of these videos is that students can easily visualize how even simple experimental designs can yield important scientific insights. The videos allow students to consider how they can conduct their own, at-home experiments using these methods with earthworms to test hypotheses of how chemicals in their own environment may affect living systems, including themselves.
Teacher feedback was enthusiastic about the use of the videos to assist students in better understanding the concepts and methods of the module. One teacher noted that use of the videos was a saving grace during a year when in-class experimentation in general and our modules specifically were disrupted. Immersing Students in the Research Experience. Using live animals, especially for behavior studies, is always preferable. Yet, behavioral observation in a classroom setting presents several challenges. Students may have difficulty accurately identifying the various behaviors; data sets are too small for adequate statistical analyses (numbers of aquaria are limited because of animal care and space issues in the classroom); and, due to the need to keep lead/Pb out of the classroom, all fish are pre-exposed. The use of the videos removes these issues. These new videos (i) help students accurately identify various reproductive behaviors and secondary sex characteristics, and (ii) provide the statistical benefits of a large data set (24 pairs of control male-female pairs and 24 Pb-exposed pairs) to track the daily behaviors ( Figure 3).
The minnow experiment videos were field tested in a high school setting. One teacher wrote, "This year has definitely been a challenge for everybody and having you compile all the videos was a lifesaver. Obviously, having real fish in the classroom is the best, students are constantly looking at screens now and (it) would have been nice to have the real deal."

Goals for Online Transition.
Zebrafish embryo development occurs rapidly over the course of three days (Kimmel et al., 1995). Because embryos are transparent, the details of this process can be observed in great detail. Such features are highly attractive to scientists interested in the fundamentals of early development and concerned with abnormal development caused, for example, by exposure to environmental chemicals. Taking advantage of expertise within the WIn-STEP-SEPA staff, students are provided opportunities for experimentation with zebrafish embryos that examine the impact of chemicals on embryo development (Tomasiewicz, H. et al., 2014).
There are two objectives of the online portion of the zebrafish module: • Provide access to key experiences in doing research to those without the means to observe live zebrafish embryos in the context of an experiment.
• Furnish teachers and students with a much larger pool of animals and experimental conditions from which to collect and analyze data.
In the classroom, student observations are generally limited to one chemical at four concentrations (zero, low, medium, high) with three replicates based on classroom logistics and the supply of zebrafish embryos. To enrich the observational experience, we took advantage of an ongoing research project which is studying the impact of a range of chemical exposures on a large number of developmental indices. Several WInSTEP-SEPA chemicals were added to the experimental protocol, and their impact on the progress of development visually recorded. These images together with detailed information on experimental conditions have been gathered into an online repository that is available to students.

Immersing Students in the Research Experience.
Short videos were produced outlining the experimental process so students can understand how these images and videos came to be. Next, students decide how to use the repository to follow the development of chemically exposed embryos. The variables include the number of organisms and the identity and concentration of the chemical. Perhaps, they will want to compare the effects of different chemicals. With the large size of the repository, there are many options for experimental design.
Students create databases of their personal observations  of the images. They have the opportunity to evaluate more than 20 developmental and anatomical characteristics of each embryo (Figure 4). These data can be analyzed alone or in combination with data from others in the class, at the teacher's discretion, to increase the complexity of the data set and thus to create the challenge of analyzing "big data". The student experience is greatly enhanced by using the University of Wisconsin-Milwaukee course management system, Canvas TM , as the interface for the students to navigate through more than 3500 images and videos.

Process of Constructing the Repository.
COVID-19 restrictions imposed by the University (mandated distancing and room occupancy limits) dramatically slowed the process of image and video collection.The biggest challenge arose in modifying our existing database program to run within the Canvas™ environment. This is a new use of the Canvas™ platform and it took time to communicate our special needs to the Canvas™ team.

Students Doing Authentic Research Communication -Impact of COVID-19.
Experimental results must be made known to others in the scientific community, so that the strength of results can be tested by other scientists. The Student Research Conference makes this key connection between individual research and communication with the larger group of participants from the schools who have embedded WInSTEP-SEPA within their curriculum (Figure 1). To prepare for this year's online Conference, students wrote papers and created posters based on their research, whether done in-person in the classroom or virtually, and submitted their work for joint review by the WInSTEP-SEPA team and pre-service teachers. All submitted papers and posters were distributed among WInSTEP program scientists and the pre-service teachers for evaluation using the program's paper and poster rubrics. In total, 36 papers and 107 posters were submitted from 11 schools ( Figure 5). In a normal, non-COVID-19 year, students submit about 100 papers and 150 posters. Thus, the numbers of both kinds of submissions were strikingly smaller than in previous years. These reductions were attributed to the stress of schooling during the pandemic, even when teachers and students were meeting in-person. All of the papers and posters were reviewed online by the WInSTEP-SEPA staff together with the cadre of pre-service teachers (Figure 1). The authors of the best papers were invited to present their research at the Conference via video recording. Students who submitted papers and posters that were rated excellent received book and plaque awards.
Due to COVID-19, the conference was adapted to a fully online format utilizing Microsoft Sway, which is an engaging and interactive presentation app. To create a virtual poster gallery walk, permissions from students were obtained and 71 posters were uploaded to this program (https://sway. office.com/wrmiVhhGWJdcRo0h?ref=Link). Visitors to the site are able to easily navigate and view any or all posters. A second Sway presentation was created to highlight all the components of our in-person event but in a virtual format (https://sway.office.com/xdEj53cQE1epQC3B?ref=Link). Videos were produced and uploaded of (i) the Welcome talk by a distinguished scientist, (ii) the top 4 student paper presentations, and (iii) the keynote speaker talk, which features a former WInSTEP-SEPA student who shares how her early research experiences in high school stimulated her to pursue a biochemistry major in college together with intense involvement in an undergraduate research project. Finally, all submitted posters were printed at UWM and delivered to each school along with individual student awards. Since all conference materials are available on demand, schools have the flexibility to participate in the conference experience when it best suits their schedule.

THE COVID-19 YEAR FOR PRE-SERVICE TEACHERS
Students becoming certified to teach high school and middle school science move through the program at the University of Wisconsin-Milwaukee in cohorts. WInSTEP-SE-PA partners with the program led by Professor Craig Berg to provide pre-service teachers with opportunities to familiarize themselves with the implementation of research experiences as the cornerstone of their approach to teaching.
Moving Workshops to Virtual Settings. Two workshops designed to introduce pre-service teachers to a selection of our experiment modules had to be rethought because of COVID-19 restrictions in place at UWM. Most of the revised activities were offered online. Twenty-two students attended this past year in comparison with an average of 17 during the three previous years.

Zebrafish Module.
The zebrafish module is designed to allow students to examine the effects of environmental agents on zebrafish embryo development. The WInSTEP-SEPA program provides workshops for pre-service teachers to learn how to run these modules in a classroom setting (Figure 1). These are led by the staff scientists who developed the modules. Prior to the COVID-19 pandemic, the six-hour workshops were in-person events, where pre-service teachers learned the background and experimental details of the module, including but not limited to zebrafish biology, raising and spawning fish, raising embryos, and safe handling of chemicals used in the exposure of embryos.
This past year, the first day of the workshop was done remotely and covered information needed to undertake experiments with zebrafish embryos. Setting up the physical experiment and directly observing freshly fertilized embryos was the only part of the module that workshop participants could not do online. In preparation, during day 1, embryos were shown to the group via digital capture video microscopy and streamed live via Zoom. Visual inspection methods were introduced remotely to distinguish between healthy, malformed, and dead embryos. During the second day, following the COVID-19 safety rules, students were able to work with embryos on their own dissecting microscopes and observe the effects of chemicals on different stages of zebrafish development.
Earthworm Module. The earthworm module takes advantage of an easy-to-obtain, inexpensive organism, red worms (Eisenia fetida). The goal of the workshop is to familiarize pre-service teachers with the use of this animal for in-class behavioral toxicology studies as a means to generate hypothesis-driven experiences for their students. With the onset of the pandemic and the advisability of moving meetings online, it became necessary to refashion the workshop so that it continued to prepare pre-service teachers for implementing the earthworm module.
Because of the simplicity of the experimental set-ups in the earthworm module, it was decided that the workshop could be undertaken with the pre-service teachers at home, replicating the hands-on activities shown to them synchronously online by a staff member. After overcoming the logistical difficulties of supplying each attendee with the needed materials and worms, the six-hour workshop proceeded as usual.
Observing Mentor Teachers Doing Modules. In a normal year, pre-service teachers visit mentor teacher classrooms and observe both the zebrafish and earthworm modules in action to gain a better understanding of module delivery. The COVID-19 year dampened pre-service teacher's opportunity to visit schools. Many schools were virtual-only, and those that had in-person students were not eager to have visitors in the classroom. As such, observations of on-going modules were sometimes done online.
Retreat. Pre-service, early career, and mentor teachers are invited to attend an annual retreat that provides important ancillary training needed or useful to undertake the WIn-STEP-SEPA modules in classrooms (Figure 1). Because the modules utilize live organisms and involve working with chemicals, it is critical for teachers to understand how to implement ethical use and handling of animals in research and the basics of chemical hygiene and safety. Content related to these subjects is presented by experts from the university. The retreat is also the event in which pre-service teachers are introduced to scientific communication and supporting tools available through the program website (Hesselbach et al., 2012). The early career/mentor teacher reflection panel offers perspectives on topics ranging from integrating the WInSTEP-SEPA program into the course curriculum to stimulation of student interest in science through research activities.
Utilizing Zoom, we held the retreat virtually in February, 2021. This resulted in a modestly larger number of participants (24 vs. an average of 20 for the Retreats during 2017-19) because we have several teachers located throughout Wisconsin, and the virtual setting eliminated the barrier of traveling. All sessions were recorded and uploaded to the program LibGuide and are readily available.
There were also challenges to moving the retreat online. The virtual medium rendered presenters and attendees largely invisible to one another as PowerPoint slides dominated computer screens. As a result, the retreat tended to morph into a webinar of lectures without the camaraderie of attendees sharing the same space.

Student Research Conference.
In preparation for the Student Research Conference, papers and posters are fully reviewed for content and presentation by the WInSTEP-SEPA staff, augmented by the cadre of pre-service teachers ( Figure  1). For the Conference, the aim is two-fold. First, as part of their participation in the WInSTEP-SEPA program, students know that their work will be professionally evaluated as occurs in scientific peer review. The reviews are returned to teachers who have the option to share them with their students. Second, this appraisal serves as the basis for making a large number of awards at the conference.
Another objective of this activity is to provide the pre-service teachers with the opportunity to learn from research scientists how to thoughtfully evaluate experimental results and interpretation. On two separate dates in early April, 2021, the team convened through Zoom meetings to discuss stu-dents' submitted work. Each scientist and a small group of pre-service teachers were assigned to separate Zoom breakout rooms to discuss the strengths and weaknesses of a subset of papers and posters that were 'shared' within the group. Together, the groups provided individual feedback for all papers and posters and decided on commendation awards. Although the virtual conference setting made it possible to discuss and reach conclusions about submitted papers in an efficient manner, the lack of a common, physical space in which to inspect and compare posters made their review more difficult and time consuming.

EVALUATION
The COVID-19 pandemic necessitated changes to many components of the evaluation plan (Figures 1 and 6). Observations of the module training workshops, the retreat, and classroom implementation of the modules were conducted via online video conferencing instead of in-person. The retreat included a brief explanation of the evaluation plan, after which teachers were asked to complete a survey focused on the WInSTEP training. During pre-pandemic years, the in-person, spring Student Research Conference included surveys of the event delivered to participating teachers and their students. In contrast, student conference surveys were eliminated during the past two years. Instead, students were questioned about the process of writing scientific papers and posters for the conference in post-test surveys following the use of the module. Questions for teachers about this virtual event were included in surveys that will be fielded at the end of the academic year. Student pre-and post-test tests, analyzed at the conclusion of the academic year, may also reveal whether and how the content of the program was affected by virtual or in-person modes in various school districts.
Findings to Date. Modifying the workshops to function online resulted in different models for engaging pre-service teachers with the two modules. For the earthworm module, WInSTEP created kits containing the necessary supplies for pre-service teachers to participate actively in the workshops from home. For the zebrafish module, staff conducted the workshop remotely and in a limited capacity, in-person laboratory setting.
Pre-service teachers were asked to assess the two module workshops. The earthworm module training received a grade-point average (GPA) M=2.78 (on a standard grading scale of "A" to "F"), standard deviation (SD) of 1.15; the zebrafish module training received a GPA of M= 2.93, SD=1.33. These scores depart from past years when the sessions were conducted in person at the School of Freshwater Sciences. During the first four program years, the earthworm module had received GPA scores from between 3.3 and 3.8 (M=3.60, SD=0.23). Similarly, the zebrafish module had received higher GPA scores between 3.5 and 3.8 (M=3.62, SD=0.12). Asked to provide reasons for their grades, teachers focused on the difficulty of clearly observing and understanding in a virtual environment while appreciating the quality of the module information.
The annual winter retreat was conducted fully online. It included a half-day set of presentations on husbandry and ethics in animal research, chemical hygiene and safety, scientific communication, and a mentor teacher panel discussion about using the modules. Overall, pre-service teachers ranked the retreat as "excellent" (30%), "very good" (20%), "average" (20%), "below average" (20%), or "poor" (10%).
Pre-service teachers were asked what aspects of the workshops and retreat they thought would most likely affect their teaching. In comments about the retreat, they noted appreciation for the guidance from experienced teachers during the mentor panel discussion, resources about writing scientific papers, and other specific information that was conveyed.
Over one-third (36%) rated the amount of time available for sharing teaching ideas in the online setting as "appropriate"; 36% felt it was "not enough time"; and 29% rated it as "too much time." Pre-service teachers were asked, "What do you think is the most valuable thing you learned from your participation in the workshops and retreat?" Respondents were able to provide examples, primarily focused on specific information and the value of understanding how other teachers imple- Figure 6. WInSTEP-SEPA Evaluation Instruments. ment the modules. Asked to identify challenges, seven respondents offered comments most about the online environment, including issues with connectivity, distractions in the home environment, and difficulties in student engagement. Pre-service teachers anticipated challenges in the environment at school that could make the implementation of the modules difficult. They commented on the virtual nature of instruction, student lack of familiarity with lab work, and possible barriers from administration or other teachers.

OPPORTUNITIES AND LESSONS OFFERED BY COVID-19 YEAR ADAPTATIONS
The WInSTEP-SEPA program model provides students with an authentic introduction to research centered on hands-on experimentation in the laboratory. Nevertheless, we recognize that a student experience patterned after how scientists do their work also needs to involve accessing relevant literature and other online resources. The COVID-19 emergency necessitated rethinking our program in terms of how we could offer elements of authenticity in the midst of the virtual or hybrid environment facing many teachers and students. Still grappling with this situation, several lessons are emerging.
• Efforts to adapt to the pandemic were dependent on having a well-established program and website available with many online materials ready for adaptation.
• Having access to content experts within our staff and at UWM was critical to imagining and bringing to fruition online versions of experiment modules.
• Creating and disseminating the online versions of the modules required substantial and diverse technological expertise as well the availability of the UWM course management system, Canvas™.
• Offering virtual forms of experiments makes it possible for students to collect and analyze data much as they would in-person.
• The richness of the data collection experience can be greatly increased because experienced staff did the hands-on part of the experiments, eliminating problems resulting from poor technique and mistakes that are expected during student research.
• Online versions of experiments are not constrained by the length of class periods or their intermittent time frame.
• The lack of in-person experimentation, in which students learn the mental and physical habits of active scientific study, also creates a large deficit in the online program.
• Depressed evaluation scores from pre-service teachers and their concerns about synchronous virtual learning underscore the importance of in-person activities as the foundation for our program to prepare effective science teachers.
• Early information from classrooms doing in-person courses indicates that the new module formats are being used to complement, not replace, actual student experimentation.
The next phase of the WInSTEP-SEPA program includes expanding pre-service teacher preparation at UWM to programs at other colleges and universities and also extending the reach of our experiment modules to teachers and students outside of our normal geographical range of support. Delivering this program online this past year has fortuitously spurred our interest and capacity to develop online tools that will help us access each of these groups. Stay tuned to find out whether and how we can harmonize the new videos and other resources, created to support conducting authentic experimentation online, with our foundational position that students need to undertake real, hands-on research in order to mature as young scientists.

ASSOCIATED CONTENT
Supplemental material mentioned in this manuscript can be found uploaded to the same webpage as this the manuscript.

Author Contributions
The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.

FUNDING SOURCES
Research reported in this publication was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R25GM129191. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.