Impacts of Hands-On Science Curriculum for Elementary School Students and Families Delivered on a Mobile Laboratory

High-quality, hands-on science education is lacking in many low-resource schools. Seattle Children’s Research Institute launched a mobile science lab program in 2009 to support the efforts of schools in Washington state by providing access to engaging hands-on science activities and opportunities to interact with scientists. For this study, we provided intensive science enrichment to students in grade four and their families with the goals of improving knowledge of key topics in the life sciences, increasing engagement in learning science, and encouraging student interest in a career in STEM. Four new activities were created for this project, two of which included family participation. Students at ten Title I-eligible schools completed two new curriculum modules on the mobile lab, and students and family members attended a science night at the school and a field trip to Seattle Children’s Research Institute. Students who participated in the project demonstrated statistically significant gains in knowledge of the topics addressed by the activities and in the ability to recognize typical STEM careers and tasks associated with those careers, which persisted into grade five. The family events were well attended with family members reporting increased confidence in talking with their child about STEM. INTRODUCTION There is a pressing need to provide youth with the type of high-quality science education that establishes a strong foundation and stimulates interest in pursuing careers in science and other related fields. Of the 30 occupations projected to grow the most over the next decade, more than twothirds are in the STEM or healthcare fields (U.S. Bureau of Labor Statistics, 2018). According to the 2015 National Assessment of Educational Progress (NAEP) – known as The Nation’s Report Card – only one-third of students in U.S. schools have the skills they will need to be adequately prepared for college-level science classes and for a career in STEM (U.S. Department of Education, 2016). The report also revealed that only 38%, 34%, and 22% of students in grades four, eight, and twelve, respectively, scored at or above the level of Proficient in the science subjects (U.S. Department of Education, 2016). Many underserved schools lack the resources needed to provide high-quality, hands-on science education. Mobile labs are being increasingly used as a valuable tool to supplement traditional classroom science lessons and increase equity of access to authentic laboratory experiences and equipment. Mobile labs offer a number of benefits by providing outreach across a large geographic area, limiting the burden on school resources, allowing students to use authentic laboratory and medical equipment, and giving students the opportunity to interact with scientists (Bonetta, 2007; Franzblau et al., 2011; Jones and Stapleton, 2017; Reid, 2001; Roden et al., 2018). Seattle Children’s Research Institute (SCRI) launched its mobile science lab, the Science Adventure Lab (SAL), in 2009. The SAL is a custom-built, 45-foot state-of-theart mobile science lab equipped with an evolving array of cutting-edge medical and scientific equipment. It can accommodate classes of up to 32 students and is staffed by a team of research scientists and STEM educators from the Science Education Department (SED). To date, the SAL has visited more than 62,500 students at 183 schools in 76 cities throughout Washington state. The program provides innovative, inquiry-based science laboratory experiences to underserved rural and urban schools at no cost. The science education literature suggests that elementary school students generally enjoy science, but that interest wanes in middle school (reviewed in (Potvin and Hasni, 2014)). Thus, when initially developing the model for the SAL program, we chose to focus on students in grades four through eight in the hopes of providing powerful experiences that could further add to students’ enjoyment in science in elementary school, and encourage and support their interest in middle school. Mobile Lab Impacts On Students and Families Jones Vol. 2, January 2019 Journal of STEM Outreach 2 The hands-on activities in the curriculum modules are based on real-world laboratory and medical protocols and were designed initially in 2012 to support the Washington state science standards and the Next Generation Science Standards (NGSS; (National Research Council, 2013)) when they were adopted in 2013. The curriculum modules introduce concepts and content drawn from multiple disciplines including physiology, anatomy, neuroscience, chemistry, and biology. All modules require the use of authentic research-grade scientific and medical equipment, computers and other technology, and include opportunities for students to analyze data and perform mathematical calculations. Here we describe a five-year project that provided intensive science enrichment to students in grade four at ten Title I-eligible elementary schools and their families. We sought to include families since previous research indicates that families play an integral role in encouraging and supporting student learning and interest in careers (Archer et al., 2012; Aschbacher et al., 2010; Swift and Watkins, 2004; Tai et al., 2006). The project had four components: two for students only and two for students and their families. The two components for students were curriculum modules completed on the SAL. Two new modules were created for this project: Sense, Think, Move, which focused on neuroscience, and Vital Signs, which introduced students to body systems and medical equipment used to monitor health. Students completed these modules onboard the SAL during the school day. In the former, students learned about the basic senses, brain anatomy and physiology, and measured the electricity produced by their muscular movements. In the latter, students measured their heart rate, blood pressure, respiratory rate, and temperature using authentic medical equipment in the same manner as healthcare professionals. The components of the project that included families were a family science night, which took place in the evening at the school following one of the SAL visits, and a field trip to SCRI in downtown Seattle. These two events expanded upon the foundation built when students completed the SAL curriculum modules and included additional content and activities designed to deepen students’ understanding of career opportunities in STEM and healthcare. Collectively, the four components of the project were intended to produce gains in scientific knowledge, to build interest and engagement in learning science, and to stimulate interest in a career in STEM. Here we describe the impacts of participating in the project for students and family members. METHODS Project Overview. Figure 1 shows an overview of the project which included two curriculum modules on the SAL, a family science night at the school, and a field trip to SCRI, all of which were completed in grade four. Families were invited to participate in the science night and field trip. An additional SAL visit was offered to schools when the students reached grade five. For this visit teachers could select from any of the other available curriculum modules. Students were given assessments to measure knowledge of the topics, interest and engagement in learning science, and Figure 1. Project overview. Mobile Lab Impacts On Students and Families Jones Vol. 2, January 2019 Journal of STEM Outreach 3 awareness and interest in STEM careers. Students in grade six completed a follow-up assessment but did not complete any additional curriculum modules on the SAL. Student participation in each of the school-based project components was tracked using State Student Identification (SSID) numbers. Not all students participated in all components of the project due to absences and high levels of mobility at some schools. Teachers recorded SSID numbers on all assessments so that responses on the assessments could be linked. Teachers identified which students had a family member attend the field trip. All students who participated in the family science night at the school participated with a family member. Teachers decided which module they did first, which allowed for flexibility in scheduling the family science night. The field trip was always the last component completed. The project team met with all of the grade four teachers in person prior to starting in order to explain the project components, logistics, expectations, and to address any questions or concerns. Teachers were compensated for their time and effort in participating in the project. There was no cost to the school or families for participating in the project. Prior to the initiation of the project at each school, parents or guardians of all grade four students received detailed information via the teacher about the project including the objectives and benefits of participation. The materials included an invitation to attend the family science night and field trip with dates and logistics. A permission slip signed by a parent or guardian was required for students to participate in the curriculum modules onboard the mobile lab. The permission slip described the curriculum modules, the specific laboratory activities that the students would do, and the expectations for appropriate behavior and attire. Parent information sheets and permission slips were routinely provided in English and Spanish, as well as any other language requested by the school. Project Components. The curriculum modules were created using the 5E Instructional Model (Bybee et al., 2006) by a team of subject matter experts and professional educators. The modules were aligned with the Washington state science standards and the NGSS. The laboratory activities were part of a comprehensive curriculum module that included two pre-lab videos: a ten-minute animated video that introduced the activities the students would do onboard the mobile lab and the key vocabulary terms and a three-minute video on laboratory safety. The SAL curriculum modules were optimized for delivery on mobile labs (Roden et al., 2018) and were designed to be taught in 70 minutes. Careers in science, healthcare, and other STEM fields were highlighted throughout both modules. Experienced Science Education Department team members provided the instruction onboard the SAL. A standardized script was used by all instructors to ensure fidelity of delivery of the modules. Students used the Qwizdom remote audience response system (clickers) to answer multiple-choice questions that were embedded in the PowerPoint presentations used for each module. Vital Signs: Monitoring Our Body’s Systems. This module introduced students to the concept of body systems and how medical professionals use various devices to measure vital signs as a means to monitor health. Students performed three activities using authentic medical equipment. First, students monitored their blood pressure and heart rate using a standard wrist blood pressure cuff. Next, students calculated their respiratory rate using a Vernier spirometer. Finally, students measured their temperature using an Exergen temporal artery thermometer. Sense, Think, Move: Exploring Brain Functions. This module introduced students to the general anatomy and physiology of the nervous system using three hands-on activities that demonstrate how the brain controls sensing, thinking, and moving. In the first activity, students learned about their senses and identified five common odors as a demonstration of olfactory processing. In the second activity, students learned about comparative brain anatomy while identifying five animal brains embedded in Lucite (Lucite Treasures) from clues written by the project team. In the final activity, students used a Vernier electromyograph to measure the electrical activity of their forearm muscles to illustrate the brain’s control of muscle movement. Family Science Night. The science night took place at the school in the evening following the SAL visit where the students completed the Vital Signs module. Teachers invited family members to attend with their child. The project team developed three activities that were designed to deepen students’ understanding of vital signs, body systems, as well as general science knowledge. The activities were also designed to build interest in and awareness of a wide range of careers in STEM and healthcare. In the first activity, students taught their families how to use the lab equipment onboard the SAL in a reversal of roles. The second activity utilized a Laerdal SimJunior® patient simulator and monitor to help students deepen their understanding of vital signs and their importance in assessing health. Students and families participated in a scenario that simulated an anaphylactic reaction to peanuts, assisted in monitoring vital signs, and observed a demonstration of administering epinephrine using an EpiPen® trainer. Finally, they participated in a Jeopardy-style science quiz using the Qwizdom clickers that included engaging questions and fun facts about a broad range of sci-


INTRODUCTION
There is a pressing need to provide youth with the type of high-quality science education that establishes a strong foundation and stimulates interest in pursuing careers in science and other related fields. Of the 30 occupations projected to grow the most over the next decade, more than twothirds are in the STEM or healthcare fields (U.S. Bureau of Labor Statistics, 2018). According to the 2015 National Assessment of Educational Progress (NAEP) -known as The Nation's Report Card -only one-third of students in U.S. schools have the skills they will need to be adequately prepared for college-level science classes and for a career in STEM (U.S. Department of Education, 2016). The report also revealed that only 38%, 34%, and 22% of students in grades four, eight, and twelve, respectively, scored at or above the level of Proficient in the science subjects (U.S. Department of Education, 2016).
Many underserved schools lack the resources needed to provide high-quality, hands-on science education. Mobile labs are being increasingly used as a valuable tool to supplement traditional classroom science lessons and increase equity of access to authentic laboratory experiences and equipment. Mobile labs offer a number of benefits by providing outreach across a large geographic area, limiting the burden on school resources, allowing students to use authen-tic laboratory and medical equipment, and giving students the opportunity to interact with scientists (Bonetta, 2007;Franzblau et al., 2011;Jones and Stapleton, 2017;Reid, 2001;Roden et al., 2018).
Seattle Children's Research Institute (SCRI) launched its mobile science lab, the Science Adventure Lab (SAL), in 2009. The SAL is a custom-built, 45-foot state-of-theart mobile science lab equipped with an evolving array of cutting-edge medical and scientific equipment. It can accommodate classes of up to 32 students and is staffed by a team of research scientists and STEM educators from the Science Education Department (SED). To date, the SAL has visited more than 62,500 students at 183 schools in 76 cities throughout Washington state. The program provides innovative, inquiry-based science laboratory experiences to underserved rural and urban schools at no cost. The science education literature suggests that elementary school students generally enjoy science, but that interest wanes in middle school (reviewed in (Potvin and Hasni, 2014)). Thus, when initially developing the model for the SAL program, we chose to focus on students in grades four through eight in the hopes of providing powerful experiences that could further add to students' enjoyment in science in elementary school, and encourage and support their interest in middle school.
The hands-on activities in the curriculum modules are based on real-world laboratory and medical protocols and were designed initially in 2012 to support the Washington state science standards and the Next Generation Science Standards (NGSS; (National Research Council, 2013)) when they were adopted in 2013. The curriculum modules introduce concepts and content drawn from multiple disciplines including physiology, anatomy, neuroscience, chemistry, and biology. All modules require the use of authentic research-grade scientific and medical equipment, computers and other technology, and include opportunities for students to analyze data and perform mathematical calculations.
Here we describe a five-year project that provided intensive science enrichment to students in grade four at ten Title I-eligible elementary schools and their families. We sought to include families since previous research indicates that families play an integral role in encouraging and supporting student learning and interest in careers (Archer et al., 2012;Aschbacher et al., 2010;Swift and Watkins, 2004;Tai et al., 2006). The project had four components: two for students only and two for students and their families. The two components for students were curriculum modules completed on the SAL. Two new modules were created for this project: Sense, Think, Move, which focused on neuroscience, and Vital Signs, which introduced students to body systems and medical equipment used to monitor health. Students completed these modules onboard the SAL during the school day. In the former, students learned about the basic senses, brain anatomy and physiology, and measured the electricity produced by their muscular movements. In the latter, stu-dents measured their heart rate, blood pressure, respiratory rate, and temperature using authentic medical equipment in the same manner as healthcare professionals. The components of the project that included families were a family science night, which took place in the evening at the school following one of the SAL visits, and a field trip to SCRI in downtown Seattle. These two events expanded upon the foundation built when students completed the SAL curriculum modules and included additional content and activities designed to deepen students' understanding of career opportunities in STEM and healthcare.
Collectively, the four components of the project were intended to produce gains in scientific knowledge, to build interest and engagement in learning science, and to stimulate interest in a career in STEM. Here we describe the impacts of participating in the project for students and family members.

METHODS
Project Overview. Figure 1 shows an overview of the project which included two curriculum modules on the SAL, a family science night at the school, and a field trip to SCRI, all of which were completed in grade four. Families were invited to participate in the science night and field trip. An additional SAL visit was offered to schools when the students reached grade five. For this visit teachers could select from any of the other available curriculum modules. Students were given assessments to measure knowledge of the topics, interest and engagement in learning science, and awareness and interest in STEM careers. Students in grade six completed a follow-up assessment but did not complete any additional curriculum modules on the SAL. Student participation in each of the school-based project components was tracked using State Student Identification (SSID) numbers. Not all students participated in all components of the project due to absences and high levels of mobility at some schools. Teachers recorded SSID numbers on all assessments so that responses on the assessments could be linked. Teachers identified which students had a family member attend the field trip. All students who participated in the family science night at the school participated with a family member. Teachers decided which module they did first, which allowed for flexibility in scheduling the family science night. The field trip was always the last component completed. The project team met with all of the grade four teachers in person prior to starting in order to explain the project components, logistics, expectations, and to address any questions or concerns. Teachers were compensated for their time and effort in participating in the project. There was no cost to the school or families for participating in the project.
Prior to the initiation of the project at each school, parents or guardians of all grade four students received detailed information via the teacher about the project including the objectives and benefits of participation. The materials included an invitation to attend the family science night and field trip with dates and logistics. A permission slip signed by a parent or guardian was required for students to participate in the curriculum modules onboard the mobile lab. The permission slip described the curriculum modules, the specific laboratory activities that the students would do, and the expectations for appropriate behavior and attire. Parent information sheets and permission slips were routinely provided in English and Spanish, as well as any other language requested by the school. Project Components. The curriculum modules were created using the 5E Instructional Model (Bybee et al., 2006) by a team of subject matter experts and professional educators. The modules were aligned with the Washington state science standards and the NGSS. The laboratory activities were part of a comprehensive curriculum module that included two pre-lab videos: a ten-minute animated video that introduced the activities the students would do onboard the mobile lab and the key vocabulary terms and a three-minute video on laboratory safety. The SAL curriculum modules were optimized for delivery on mobile labs (Roden et al., 2018) and were designed to be taught in 70 minutes. Careers in science, healthcare, and other STEM fields were highlighted throughout both modules. Experienced Science Education Department team members provided the instruction onboard the SAL. A standardized script was used by all instructors to ensure fidelity of delivery of the modules. Students used the Qwizdom remote audience response system (clickers) to answer multiple-choice questions that were embedded in the PowerPoint presentations used for each module.
Vital Signs: Monitoring Our Body's Systems. This module introduced students to the concept of body systems and how medical professionals use various devices to measure vital signs as a means to monitor health. Students performed three activities using authentic medical equipment. First, students monitored their blood pressure and heart rate using a standard wrist blood pressure cuff. Next, students calculated their respiratory rate using a Vernier spirometer. Finally, students measured their temperature using an Exergen temporal artery thermometer.
Sense, Think, Move: Exploring Brain Functions. This module introduced students to the general anatomy and physiology of the nervous system using three hands-on activities that demonstrate how the brain controls sensing, thinking, and moving. In the first activity, students learned about their senses and identified five common odors as a demonstration of olfactory processing. In the second activity, students learned about comparative brain anatomy while identifying five animal brains embedded in Lucite (Lucite Treasures) from clues written by the project team. In the final activity, students used a Vernier electromyograph to measure the electrical activity of their forearm muscles to illustrate the brain's control of muscle movement.
Family Science Night. The science night took place at the school in the evening following the SAL visit where the students completed the Vital Signs module. Teachers invited family members to attend with their child. The project team developed three activities that were designed to deepen students' understanding of vital signs, body systems, as well as general science knowledge. The activities were also designed to build interest in and awareness of a wide range of careers in STEM and healthcare. In the first activity, students taught their families how to use the lab equipment onboard the SAL in a reversal of roles. The second activity utilized a Laerdal SimJunior® patient simulator and monitor to help students deepen their understanding of vital signs and their importance in assessing health. Students and families participated in a scenario that simulated an anaphylactic reaction to peanuts, assisted in monitoring vital signs, and observed a demonstration of administering epinephrine using an Epi-Pen® trainer. Finally, they participated in a Jeopardy-style science quiz using the Qwizdom clickers that included engaging questions and fun facts about a broad range of science topics. Field Trip. Students and families were invited to attend a three-hour field trip to SCRI, which occurred during regular school hours. All transportation costs, including school bus fees and personal vehicle use, were paid from the grant that supported the project to avoid placing undue burden on the schools or families. Lunch was also provided. Four interactive mini-lessons were created by the project team that further expanded on the concepts addressed by the modules and family science night. Students learned more about brain anatomy by examining plastic models of human brains and preserved bovine brains. Students also reinforced their knowledge of the respiratory system by interacting with healthy and simulated-smoker porcine lungs. They also toured the laboratory spaces, performed a pipetting experiment, and participated in an interactive presentation on STEM careers.
Assessments for Students. Grade four students completed the assessment online (Supplemental Table 1). The survey required a response for each question before allowing the students to proceed. Paper assessments were used for students in grades five and six. The initial Pre-assessment was administered prior to the first SAL visit in grade four. Questions focused on their knowledge of the subject matter addressed by the project components, their ability to identify typical STEM careers and tasks associated with these careers, and their interest and engagement in learning science and in a career in STEM.
For questions relating to knowledge of science topics and STEM careers and tasks associated with the careers, answers were evaluated as either correct or incorrect. Students were asked to rate a series of statements designed to measure their perception of the importance of STEM, their interest and engagement in learning science, and their interest in a career in STEM. A three-point Likert scale with answer choices agree, not sure, or disagree was used. The assessment was repeated following the completion of the project components in grade four. In the first two years of the project (2012-2013), the project team took tablet computers and a wireless access point to the schools and administered the assessments. As the project progressed, more of the schools had access to computers and a stable Wi-Fi connection, and, by year four (2016), nine of the ten schools were able to administer the assessments. Project team members and teachers were given detailed instructions to clearly read all of the questions aloud and to allow sufficient time for the students to read and answer each question on their computer. Follow-up assessments in grades five and six were performed by teachers. Teachers were provided with printed copies of the assessments, instructions on how to administer the assessment, and pre-paid envelopes to return them once completed. Teachers wrote the SSID number for each student on the assessment so that we could verify participation and link the responses to those from earlier assessments.
Two different approaches were used to obtain assessment data from grade six students who participated in the project while in grade four. For those students who were at the six K-5 schools in grade four, we identified which middle school they were likely to attend and asked grade six teachers at those schools to give an assessment to their classes. Nine middle schools agreed to participate, all of which were eligible for Title I funding. We also asked grade six teachers at the four K-6 schools participating in the project to give the assessment to their classes. The questions and statements on the assessments given to grade six students were modified to be developmentally appropriate by adjusting the language and adding matching questions to measure retention of topic knowledge. Additional statements about their interest and engagement in science were added to the rating section, and the Likert scale was expanded to five points (Supplemental Table 2). The grade six classes at the nine middle schools that did the follow-up assessments were a mix of students who had participated in our project and students who came from elementary schools that had not participated. The grade six classes at the K-6 schools were made up almost entirely of students who had participated in the project in grade four.
Teacher and Family Surveys. Grade four teachers at the participating schools were sent online surveys that allowed them to rate the effectiveness of the communication, each component of the project, and the support they received (Supplemental Table 4). The survey also included open-ended questions in order to collect more detailed feedback. Teachers sent online and/or paper surveys to the parents or guardians of students in their class (Supplemental Table 3). Both options were provided so that teachers could select the format they were comfortable using. The purpose of the survey was to get information about their perception of the benefits they and their child received by attending. The survey also asked respondents to identify barriers to attending. Both the teacher and parent surveys contained skip logic so that respondents could answer only the questions relevant to the project components they participated in. Open-ended responses were sorted into general categories and enumerated. Table 1 shows the characteristics of the participating schools: six were K-5 schools and four were K-6 schools. Title I eligibility and free and reduced-price meal (FRPM) data were obtained from the Office of the Superintendent of Public Instruction for Washington state and the geographical classification obtained from the National Center for Education Statistics (NCES) website. Eight of the schools were in urban locations, and two were in rural locations. The two rural schools had the fewest number of grade four classes (1.5; one class was a split grade class). The largest school that participated had 4.5 classes of grade four students. An average of 75.0% of students received FRPM at the participating schools in 2013 when the same school, and 318 students who had gone to a participating K-5 school in grade four and were now at one of middle schools that agreed to provide us with assessment data. All grade six teachers we received assessments from had at least one student who participated in grade four in their classes. The 1,704 non-participants included 65 students from the four K-6 schools who were not at the school in grade four or did not participate in the project for another reason, and 1,639 students at the middle schools who attended an elementary school that did not participate in project. No information other than gender and SSID was collected from the grade six students.

Schools and Participants.
Family Participation. The project included two events where families were also invited to participate. Table 2 shows family attendance at the field trip to SCRI and science night held at the school. A total of 1,013 family members participated in the family science night and/or the field trip over the course of the project. The field trips were held during regular school hours and were well attended with 90% of grade four students attending. Of those, 22% of students had a family member attend the field trip with them. There were three opportunities for schools to host family science nights over the course of the project. As seen in Table 2, not all schools hosted a family science night each year. Overall, 25% of grade four students who participated in the project attended a family science night with a family member. Attendance varied widely between the schools (9-77%) and from year to year at a given school. Students did not attend without a family member since the event occurred in the evenings outside of regular school hours. Ultimately, only 9% (135/1,500) of grade four students attended the family science night and had a family member attend the field trip. Family members were not required to provide their names, so we could not determine if the same individual attended both events. No other information about family members was collected.
Comparative Analyses. In grades four and five, gains were defined by comparing to the baseline established for each student on the Pre-assessment. The questions and statements on the assessments used in grades four and five were not developmentally appropriate for grade six students, so responses on the assessment used in grade six could not be compared to the baseline established in grade four. To determine if any impact of participating in the project had persisted into grade six, assessment data from the students who had participated in the project were pooled and compared to the data from the pool of non-participant peers from the nine middle schools and four K-6 schools.
Questions on knowledge of scientific topics addressed by the modules and recognition of STEM jobs and tasks typically done by someone in a STEM career had binary answers (correct or incorrect). Responses across a continuum project was launched; the state average was 45.9%. We attempted to minimize any impact of low reading proficiency by instructing project staff or teachers administering the assessments to read all of the questions and answers out loud and to wait until the majority of the students had answered the question before moving on.
A total of 3,195 grade four students took part in some aspect of the project. This includes students who participated during the first year of the project when some components were pilot tested. Of those, 2,071 of the students completed the components of the project and/or completed an assessment. As anticipated, not all students completed all aspects of the project. However, most students (90.4%, 1,872) attended the field trip, 18.4% participated in one SAL visit, and 81.1% participated in both. Thus, we did not exclude data from students who did not attend the family science night, and we defined a participant as a student who completed at least one SAL module and attended the field trip in grade four. This approach allowed us to include data from as many students as possible in the analysis. Using these criteria, 1,784 grade four students were defined as participants, and 86% of participants completed at least 50% of the questions on the Pre and Post assessments. Of those, 60% (1,070) also completed 50% or more of the questions on the assessment given to students in grade five.
In an attempt to determine if any positive impact of participating in the project persisted into grade six, we collected assessment data from 519 students who had participated in the project in grade four. We also collected assessment data from 1,704 grade six students who had not participated in the project. The 519 participants included 201 grade six students at one of the four K-6 schools and were still at the  (Likert scale) were used to evaluate students' interest and engagement in STEM and their interest in a career in STEM. Ratings for the one negatively worded statement were reverse-coded prior to analysis. Only data from assessments where students completed 50% or more of the questions were included in analysis. Statistical significance of the gains for individual students in grades four and five compared to the baseline was determined using a Student's t-test for paired samples (threshold of p<0.001). An unpaired Student's t-test (threshold of p<0.001) was used to calculate the statistical significance of the differences in responses between grade six students who had participated in the project and those who had not. Effect size (Cohen's d; (Cohen, 1988)) was calculated to quantify the magnitude of the difference and add context to the differences that were observed (Sullivan and Feinn, 2012).
Ethics Statement. All aspects of the project were reviewed by the Institutional Review Board at SCRI, and the study was determined to qualify as exempt. Table  3, after completing the components of the project in grade four, students demonstrated a 23.05% increase in overall topic knowledge compared to Pre-assessment, which was statistically significant with a large effect size (p<0.001, Cohen's d = 1.26). The gains persisted into grade five, where students maintained a 20.55% increase in topic knowledge with a similarly large effect size (Cohen's d = 1.09). Students demonstrated less baseline knowledge and greater gains for the content in the neuroscience module than the Vital Signs module ( Table 3).

Identification of STEM Jobs and Tasks.
In addition to stimulating interest in a career in STEM, this project was also intended to broaden students' awareness of career options in the STEM and healthcare fields. As such, a variety of career options were introduced in all four of the project components. We measured students' ability to recognize jobs typically thought of as STEM jobs, as well as tasks associated with doing STEM jobs. The word "jobs" instead of "careers" was used on assessments given to students in grades four and five since that terminology is more familiar to younger students. There was a 5.67% increase in the correct identification of STEM jobs (e.g. computer programmer, dentist, engineer) and the tasks associated with those jobs (e.g. conducting experiments, inventing things) after completing the components of the project in grade four (Table 3). The increase was statistically significant with a small to moderate effect size (Cohen's d = 0.40). These gains compared to Pre also persisted into grade five (7.46% increase, Cohen's d = 0.48).

Interest and Engagement.
Interest and engagement in learning science has been linked to both achievement and the intention to pursue coursework or careers in STEM (Potvin and Hasni, 2014). Therefore, students were asked to rate a series of statements designed to measure their perception of the importance of STEM, their interest and engagement in science, and their interest in a career in STEM. As seen in Table 3, there was a statistically significant increase in the ratings given by the grade four students from Pre to Post for the group of statements (p<0.001, Cohen's d = 0.16) but it did not persist into grade five. Students were asked to rate a statement about their interest in a job related to STEM. As seen in Table 3, there was no increase in ratings compared to Pre for students in grades four who had completed the project or for students in grade five.   (2013)(2014)(2015)(2016). Impacts in Grades Six. As noted above, responses on the assessment used in grade six could not be compared to the baseline established in grade four. A modified, developmentally-appropriate assessment was used for students in grade six (Supplemental Table 2) and responses from grade six students who participated in the project in grade four were compared to those who did not. As shown in Table 4, students in grades six who had participated in the project had greater knowledge of the scientific topics addressed by the project than their peers, suggesting they had retained at least some of what they had learned. Students in grade six scored 8.28% higher than their peers who had not participated (p<0.001, Cohen's d = 0.40). Grade six students who had participated in the project two years prior rated the group of statements addressing their interest and engagement higher than their peers who had not participated (p<0.001, Cohen's d = 0.24). The four statements with the greatest difference in ratings between participants and their peers are shown in Table 4.   field trip was valuable for them. Similarly, 98.9% (88/89) of family members who attended a family science night reported that attending was valuable for their child, and 97.8% (87/89) reported that attending was valuable for them. The most commonly reported benefits are shown in Table 5. Survey respondents reported the following benefits for students: making connections between the various activities they participated in and prior knowledge, learning new knowledge about the scientific topics, and increased interest in science and/or STEM. The most commonly identified benefit to family members was increased confidence in talking with their child about science and/or STEM. Two primary barriers to participating in the events were reported. The most commonly reported barrier was a conflict with the day or time of the event. Survey respondents also reported that an inability to find care for other children or family members prevented their participation.
Teacher Survey Results. Each year, teachers were sent an online survey once the components of the project had been completed. A total of 48 different grade four teachers participated in the project. Some teachers participated in multiple years and were asked to complete a survey each year to ensure that we collected as much feedback as possible. A total of 90 survey responses were received though not all teachers answered all questions on the survey. Feedback from teachers was overwhelmingly positive for all aspects of the project. Foremost, 90.9% of teachers (30/33) said they felt more confident talking with their students about science and/ or STEM careers as a result of participating in the project. Also, 98.4% of teachers (62/63) reported that the curriculum modules on the SAL were relevant and age appropriate. All of the teachers (54/54) said the field trip was valuable for students, for families, and for themselves. Ultimately, 97.5% of teachers who attended the family science night reported that it was valuable for families (39/40) and for themselves (39/40).
The teacher survey also included open-ended questions so that more detailed feedback about the benefits and challenges of participating in the project could be collected. The most frequently reported feedback is shown in Table  6. Teachers reported that benefits for students included increased awareness of STEM careers and the importance of STEM, the ability to have experiences that are not available in the classroom, and seeing real-world applications of science. The primary benefit to teachers was increased confidence in talking to their students about science and STEM careers. Teachers at schools with low levels of participation in the family events provided the following reasons: scheduling conflicts, lack of childcare for younger siblings, and lack of transportation, which mirrors the results of the family surveys.

DISCUSSION
The Science Adventure Lab began visiting under-resourced Title I schools across Washington state in 2009. This new project was initiated in response to the need we observed for enriching, and in some cases, introducing life sciences education to students at these schools. In Washington state, students take the standardized state science test in grade five, so by focusing the project on students in grade four, we also saw an opportunity to help teachers prepare

Making connections to prior knowledge 283
Learning new content knowledge 101 Stronger interest in science and/or STEM 55 Commonly identified benefits for family members who attended

No. times reported
Increased confidence in talking with child about science and/or STEM 243

Barriers to attendance No. times reported
Work conflict with the day/time 220 Difficulty finding childcare/family care 59 Table 5. Family survey responses (n=414 surveys).

Commonly identified benefits for students No. times reported
Increased awareness of STEM careers and importance of STEM 27 Unique experiences not available in classroom 19 Seeing real world applications for science and making connections to prior learnings and experiences 16

Commonly identified benefits for teachers No. times reported
Increased confidence talking to students about science and STEM careers 37

Commonly identified challenges for teachers No. times reported
Technical issues associated with computer functionality or access 14 Scheduling project activities and assessments around state testing, student absences, and special classes 38 Difficulty recruiting sufficient chaperones/family members and coordinating chaperone logistics for the field trip 15 Difficulty planning/coordinating an evening event for family science night 18 Low interest or motivation of families to attend evening event at the school 12 Table 6. Teacher survey responses (n=90 surveys).
students for the state test in the following year. Since there was no existing analogous curriculum that could serve as a comparison, our objective was to determine if providing intensive science enrichment resulted in measurable gains in knowledge of key topics in the life sciences, engagement in learning science, and interest in a career in STEM, and whether any gains were sustained beyond grade four. We also sought to create opportunities to engage families in science learning. The data we collected showed that participating in the project in grade four had a number of positive impacts, some of which persisted beyond grade four. The most significant gains were in the acquisition of knowledge about the topics addressed by the project activities. Scores on the content knowledge questions remained elevated for many students compared to the baseline as they progressed into grade five. Knowledge of the topics addressed by the project activities was also higher among the pool of grade six students who participated in the project compared to a pool of their peers who had not participated. This suggests that the experiences students had in grade four were significant and memorable enough to provide a lasting impact on their scientific knowledge. The difference in knowledge between grade six participants and non-participant peers also suggests that the content included in this project is not addressed by standard classroom science education, at least through grade six at the schools that were involved in this project.
As the SAL team began visiting underserved elementary schools, they routinely noted that students had limited knowledge of the wide range of career options in science and healthcare. Thus, we included information about the diverse range of careers in STEM and healthcare and what a person in each role might do in all four components of this project. There were significant gains in students' ability to correctly identify STEM jobs and tasks from the list we provided in grade four indicating that participating in the project did increase awareness of career options in STEM, at least in the short term. Increased awareness of STEM careers was the most commonly reported benefit for students as reported by teachers, which further supports this conclusion. The increase in ability to correctly identify STEM jobs and tasks persisted into grade five.
Participating in the project did not increase students' interest in a career in STEM, at least as measured by our assessment. Anecdotally, however, many students did express newfound interest in careers in STEM after participating in the project either verbally or in the letters that many classes wrote to thank the project team. There are a number of possible explanations for the lack of a measurable increase in interest in a STEM career, including the fact that career interests may be difficult to influence at the elementary school level. It is also possible that the activities created for the project were not well suited for building interest in a STEM career. Limitations in the study design may also have played a role in our inability to detect an increase. Baseline ratings for the statements about interest in STEM careers and engagement in learning science on the Pre-assessment appeared to be inflated by the fact the students and their teachers were aware that they would be participating in an interesting new science project over the course of the school year, suggesting response-shift bias (Howard, 1980). This effect may have been compounded by the fact that we used a compressed three-point Likert scale. While it would have been preferable to enroll completely naïve schools to get a true baseline, given the widespread awareness of the SAL program, this would have been challenging. The SAL program has been visiting schools since 2009 and is well known in many communities for providing fun, hands-on science experiences. While the grade four students had not participated in any SAL programming prior to this study, the SAL had visited seven of the ten schools in the past, so the students would have seen the mobile lab and potentially heard older students talking about their experiences. The use of a retrospective pre-test approach may have attenuated the response-shift bias (Drennan and Hyde, 2008). Future studies will utilize a more robust survey of STEM career interest such as the Student Attitudes Toward STEM (S-STEM) survey (Friday Institute for Educational Innovation, 2012), though this will require additional classroom time in order to complete longer assessments.
Beyond grade five, we were no longer able to compare individual student responses to the baseline on the pre-assessment due to the simplicity and language of the questions and statements. In an attempt to determine if there was any detectable impact of participating in the project in grade six, we compared responses from students who participated in the project in grade four, to those that had not at the same schools. Grade six students who participated in the project gave higher ratings than their peers to several statements intended to measure interest and engagement in learning science, and the statement of their interest in a career in STEM. These data suggest that participating in the project had a sustained, positive impact. While the comparison groups of grade six students are from the same schools and presumably reside in the same communities as the participants, our enthusiasm is tempered by the fact that we could not control for factors such as parental education level, race/ethnicity, or other experiences that may have influenced their responses since this information was not available to us.
We were not able to follow students who participated in the project beyond grade six due to the funding timeline, so we cannot determine whether any increased interest in a STEM career persisted into high school or resulted in them choosing STEM coursework in high school and college. While longitudinal studies are the gold standard, it should be noted that with each successive year, it became more diffi-cult to track students who had participated in grade four and to get assessment data from sufficient numbers of participants. This was due to mobility, transition to middle school, and highly variable ratios of participants to controls depending on the number of elementary schools feeding into each middle school. Planning is underway for a longer-term study that follows students who participate in our programs in middle school as they progress into high school and beyond.
Teachers were given instructions for preparing the students for each of the project components as well as information on their role and responsibilities. However, over the course of the project, the team observed variability in how well teachers prepared their students for the SAL visits and other project components. It was noted that some teachers did not show their students the two pre-lab videos, while others reviewed them multiple times. The team also observed some teachers reviewing the vocabulary in the classroom prior to the assessments, while others did not. As such, we cannot rule out the possibility of teacher effects on the outcomes we observed. Fidelity logs for teachers may have allowed us to control for some of the variability due to possible teacher effects.
In the early stages of the project, lack of internet connectivity and access to computers at several schools was a significant barrier for completing online assessments, though this did improve over time. When the project was launched, the team took tablet computers and a wireless access point to the schools to do the Pre and Post assessments. This was a significant time commitment that was not anticipated initially. As the project progressed, more schools were able to do the assessments without support from the project team. By the final year of the project, nine of the ten schools were completing the online assessments using their own equipment. When planning a study of this type, it is important to consider this possibility and plan countermeasures, especially in rural areas where connectivity may be low, or when working with under-resourced schools that may not have adequate supplies of computers.
Family participation was an important feature of this project. A total of 1,013 family members participated in the family science night and/or the field trip over the course of the project. Almost all of the family members who attended reported that the family events were valuable for them and their child. This is encouraging since research shows that families with more positive mindsets toward STEM are more likely to promote the same attitudes in their children (Archer et al., 2012;Shumow and Schmidt, 2014). The most commonly reported benefit of attending was increased confidence in talking with their child about science and/or STEM, which suggests that continued conversations were likely to occur at home. Not all schools chose to host a family science night each year, and participation varied widely among those that did.
One factor that impacted participation levels appeared to be the school culture and whether family members were accustomed to attending evening events at the school. Most teachers were enthusiastic about hosting family science nights as part of the project. However, family science nights were more successful at schools where evening events for families were common, and thus, part of the culture. After observing that some teachers were offering students incentives to attend the family science night, in the third year (2015) we added a question to the teacher survey about incentives. Interestingly, 74% of respondents stated that they had offered incentives for attending the family science night, including modified homework expectations and/or food. There was no clear correlation between a teacher offering an incentive and attendance, suggesting that the barriers reported by families in the survey were the key drivers of attendance. There was also school-to-school variability in family participation in the field trips, and, in some cases, class-to-class variability within a single school. Some teachers reported difficulty in recruiting family members to attend the field trip, while others reported that recruiting family members was relatively easy in comparison to other field trips that required adult chaperones.
Mobile laboratories are one of many possible platforms for providing out-of-school STEM enrichment to young people. While not essential for providing hands-on science activities, the mobile lab was a key component of this project as it allowed us to provide laboratory experiences and access to scientific equipment and tools where they would not otherwise have been available. The authenticity of the setting that a mobile lab provides likely also helped to create memorable experiences for the students and families who participated.
While STEM outreach programs are common, the variability in program models and the difficulty of measuring outcomes make comparisons between programs difficult (National Research Council, 2015). According to the National Research Council, informal, out-of-school enrichment programs have been shown to contribute to interest and understanding of STEM, to allow young people to connect with role models, and to reduce the achievement gap between youth from low-income and high-income families (National Research Council, 2015).
Although not all of the objectives of this study were met, the project did result in numerous positive outcomes for the students, parents, and teachers who participated. While executing the project, we attempted to address the three criteria identified by the National Research Council's report on outof-school STEM programs that produce positive outcomes: 1) the program is engaging and provides opportunities for hands-on experiences; 2) the program is responsive to learners' interests, experiences, and cultural practices; and 3) the program connects learning experiences across settings (Na-tional Research Council, 2015). Our study adds to the growing body of research on the effectiveness of STEM enrichment programs. Since student interest and engagement in science is well known to decline in the middle school years, providing powerful, positive experiences in science in elementary school, coupled with building family engagement in science learning, could help to ameliorate the decline.