Building Community Capacity for Integrating Engineering in Rural Middle School Science Classrooms

,


INTRODUCTION
An enormous task has fallen to our teachers and school systems as national calls to improve STEM workforce development have pressured school systems to enhance student learning in STEM content areas, particularly engineering. Educators, who already work into the night, over 10 hours a day and 50 hours per week, are presumed to have the resources and capacity to meet these new and expanded demands (Scholastic and the Bill and Melinda Gates Foundation, 2012). The burden of exploring new engineering standards in particular and also applying them in the context of traditional school disciplines (e.g. math, science, social studies) is unreasonable without additional support and resourcing.
Integrating engineering into the K-12 curriculum has been motivated by the goals of developing a better understanding of different pathways to engineering careers, building students' engineering and technological literacy from a young age, and exploring the possibilities of blending engineering with traditional subjects to enhance learning (Cunningham and Lachapelle, 2014). While these goals may have merit, they may seem secondary given the increased accountability to align, and potentially limit, all classroom time more directly with content assessed on state standardized tests (Saeki et al., 2015). Integrating new engineering content often requires teachers to extend their subject matter expertise, identify ways engineering intersects with grade-level specific subject matter standards, and develop the pedagogical skills necessary for successful delivery of an integrated curriculum. However, many teachers are under-resourced, under-supported, and, research suggests, underprepared to meet these challenges. Most recently, Antink-Meyer and Meyer (2016) investigated elementary and high school science teachers' understanding of science and engineering practices. They found teacher reflections to contain fundamental misconceptions about the nature of engineering and its relationship to science. Similar struggles in decoding engineering content for quality lessons occurred in the middle school classroom investigation conducted by Judson and colleagues (2016). Of the four categories of lessons developed from Next Generation Science Standard (NGSS), about half of the middle school lessons developed in their study fell into the "Vague and/or Overly Broad" category (p. 7). Online teacher support communities such as TeachEngineering.org have begun to align their lessons with the NGSS in hopes of alleviating these issues, but research suggests that coverage of the standards is limited in these instances (Samson et al., 2015), compromising the usefulness of the online support communities.
Issues related to integrating engineering content and career development throughout K-12 core subjects intersect with other critical challenges in the education system. For example, economic disparities across regions may affect the ability of school systems to address engineering content directly and indirectly. Reporting on findings from a project aimed at improving college readiness for secondary students in three rural school districts, Sepanik et al. (2018) note: Rural communities face particular challenges in ensuring college readiness and success for students. Given the distance from urban centers, geographic size, and low population density in these communities, colleges there often have trouble attracting and retaining skilled teachers and have less opportunity for collaboration across institutions (p. ix) Past research in rural areas of Virginia has identified that young rural Virginians face higher rates of poverty, lower levels of educational attainment, higher drop-out rates, and lower potential salaries (Alleman and Holly, 2012). Regarding engineering specifically, Matusovich et al. (manuscript submitted) found that even within a single rural school district engineering-related resources and access to higher education can vary by high school. Other research has highlighted the need to situate any engineering content or interest developing activities within the unique rural community context. The Promoting and Supporting Engineering Career Choices (PSECC) model suggests the importance of support for students from valued mentors or family in the community and pathways for steady and stable local work (Gillen et al., 2018).
Considering these challenges together, the purpose of our project is building community capacity for integrating engineering into middle school science classrooms by cultivating partnerships among key community and regional stakeholders. Instead of expecting science or math teachers alone to bootstrap the skills required to address engineering and career skill goals, we present a possible solution focused on collaboration among school systems, teachers, practicing engineers, and university affiliates. This cross-sector approach values the unique expertise of each partner in meeting integrative engineering student learning goals. Further, because a spectrum of employee educational backgrounds, from high school equivalency to PhD, are represented in many engineering firms and manufacturing facilities, industry-community partnerships afford students the opportunity to interact with role models representing a variety of career pathways in engineering and technology. In the literature, industry-community partnerships have documented potential for shared benefits (Googins and Rochlin, 2000) and in past collaborations involving public school systems and industry, teachers have reported favorable outcomes for student learning (Rogers and Cejka, 2006).
In this program description paper, we discuss the research-based design and evolution of Virginia Tech's Partnering with Educators and Engineers in Rural Schools (PEERS), a National Science Foundation (NSF) funded project from the Innovative Technology Experiences for Students and Teachers (ITEST) program.

PROJECT DESIGN
The PEERS project focuses on the collaborative design, implementation, and study of recurrent hands-on engineering activities with middle school youth in three rural communities in or near Appalachia. To achieve this aim, the team partners with school educators and industry experts in students' local communities to collectively develop curriculum to meet teacher-identified science standards and to facilitate regular in-class interventions throughout the academic year. These in-class interventions are led by the collaborative team so that teachers, industry volunteers, and university affiliates are all together working with youth.
The multi-faceted approach is strongly informed by the ITEST STEM Workforce Education Helix model focused on deepening the impact of the ITEST programs (Reider et al., 2016). This model describes an "iterative relationship between STEM content development and STEM career development activities… within the cultural context of schools, with teachers supported by professional development, and through programs supported by effective partnerships." We believe that successful programs must holistically integrate these elements. In accordance with the helix model and drawing upon research-informed specific needs of our target population, PEERS focuses on building partnerships with school educators and industry experts local to each of the three target rural communities (local cultural context) to regularly integrate hands-on engineering activities with core science curriculum. To intentionally connect with engineering career development activities, industry partners periodically discuss needs in their own companies for employees from varied educational backgrounds and how intervention activities align with content or skills needed to be successful in their industry. These priorities are primarily funneled through university affiliates in curriculum planning but also occur ad hoc in the classroom when industry shares their insights with the students during activity introductions or wrap-ups. While collaborative planning, curriculum development, and professional skill-building is ongoing throughout the academic year, PEERS hosts an annual summit each summer which brings together teachers, industry representatives, and university affiliates to build trust, reflect on prior work, set goals for the upcoming year, and create opportunities for focused design and development. The summit was established in part to compensate for the limited opportunity for outside interaction between teachers and industry given the constraints of time and distance. To clarify further the roles of each of the collaborative partners, Table 1 identifies specific tasks undertaken both inside and outside of the classroom.
Partnering communities for PEERS are located in rural areas, with two of three in Appalachia, where students may have limited exposure to engineering careers. Rural and Appalachian youth are underrepresented in higher education, and statistics from the partnering communities show that these regions have lower than average rates of bachelor's degree holders, per capita income, and employment opportunities in professional, scientific, and management industries. Considering these challenges, the project engages regional companies who hire workers from across the technical and engineering career spectrum (e.g. high-school graduates, trades, university degrees). Through regular engagement in classrooms, the industry representatives serve as examples of the types of careers in students' home communities and the diversity of skill sets associated with the field of engineering. Further, by collaborating with teachers to deliver hands-on activities, PEERS infuses active learning pedagogies which have been shown to better engage diverse learners who may not excel in traditional education models (Jenkins et al., 2003). Importantly, working with every student via every 6th and 7th grade science teacher during the school day ensures socio-economic barriers (e.g. after-school transportation) or academic-tracking (e.g. innovative projects deployed in "gifted programs" only) does not limit participation.
From 2017-2019 in seven schools across three rural counties, PEERS has worked with six 6th grade science teachers, eight 7th grade science teachers, and two teachers who teach both 6th and 7th grade science. The team works with every 6th and 7th grade class in these schools during their normal science class period, totaling between three and six contact days with each student (i.e. three days in semester classes, six days in year-long classes). To date, the project has engaged 26 graduate students from a local college and 27 professionals from industry partners in classroom activities with 1527 middle school students. Curricular Priorities. This project has three curricular priorities: 1) alignment with standards of learning, 2) introducing engineering in culturally relevant ways, and 3) potential for sustainability. Literature on multi-stakeholder partnerships highlights the need to "assume joint risks and responsibilities" and "find trade-offs and create value for all" (Gray and Purdy, 2018, p. 8). Given the pressure put on teachers and schools to perform on state standardized tests, curriculum development aligns engineering activities with teacher-iden- tified state standards of learning and integrates these lessons into the normal science curriculum for all students. While science learning outcomes take priority, a key innovation from the project is in grounding curriculum efforts in research on effective engineering education, culturally-relevant pedagogy, and the career choices of rural and Appalachian youth (PSECC). Research-informed best practices for engineering pedagogy have informed the program. For example, suggestions from Cunningham and Lachappelle (2014) such as: "Demonstrate how engineers help people, animals, the environment, or society" and "Ensure that design challenges are truly open-ended with more than one correct answer" (p. 135) have informed the design of hands-on activities and their associated learning goals. By incorporating elements of culturally-relevant pedagogy, activities seek to affirm the lived experiences of students and use the culture and community context of youth to make activities more authentic and relevant (e.g. Gillen et al., 2018)). A final consideration is the sustainability of the activities beyond the life of the grant which has led the project to prioritize "using low-cost readily available materials" (Cunningham and Lachappelle, 2014, p. 133) and consider the overall people-power and time required to reset materials between back-to-back class sessions.
While curriculum guides from PEERS can be found elsewhere, it is important to note that the range of covered science standards is broad, including topics such as genetics, water quality, energy, space, and ecosystems. Despite this breadth, the guiding curricular priorities are consistent across the program. For example, in an activity called Mountain Roads, students use the engineering design process (engineering) to construct a road or path around a mountain (locally-relevant) within given constraints. The activity provides an opportunity to design and revise a solution (open-ended) while getting first-hand experience with potential and kinetic energy (state science standards). Materials include buckets, trashbags, masking tape, marbles, and foam pipe insulation (low-cost, accessible materials). Industry partners are encouraged to discuss how constraints, design processes, failure, and teamwork are relevant to their own work (career pathways). To encourage continued engagement between classroom visits, the project has developed a poster that is featured in the classroom. Figure 1 demonstrates how the project attends to engineering content, cultural relevance, and career pathways via the permanent classroom poster. Specific content of the poster is rotated monthly to highlight key aspects of the curriculum outside of the core science outcomes. Elements of the poster draw from the best practices from Cunningham and curricula required a semi-structured protocol directed at answering some overarching questions (e.g. "What part of the lesson did students seem to enjoy most? What evidence do you have to support this?") as well as frequency reports and associated evidence for student engagement (e.g. "How often did you see students engage with each other in the following activities? -Activities: Compared answers; Asked each other questions; Encouraged each other; Competed with each other).
Capacity-building metrics (Goal 2). Our framework for capacity-building and sustainability of strategic collaborations is grounded in two areas: 1) curriculum reform and adoption of innovative pedagogy and 2) cross-sector partnerships. The sustainability of curriculum reform efforts is generally understood as a function of stakeholder support (e.g. teacher perceptions of innovation value) (Owston, 2007); social and cultural relevance of the intervention (Johnson and Atwater, 2014); and, teacher agency (Datnow et al., 2002). The structure and dynamics of the school-industry partnership itself also influences sustainability. The collaboration is further shaped by processes such as governance, administration, mutuality, and norms of trust and reciprocity (Thomson and Perry, 2006). The primary data for exploring these elements are beginning-and end-of-year interviews with key stakeholders (i.e. teachers, school administrators, industry partners, university affiliates). Insights from these interviews are supplemented by aspects of the periodic classroom observations focused on engagement among adult stakeholders and between adults and students.
The core interview protocol with all stakeholders includes questions targeting constructs from the collaboration literature (e.g. "What role do you and your organization play in the collaboration?"; "Who do you think is benefiting from this collaboration?"). Teacher interviews additionally focus on teacher conceptions of engineering (e.g. "How do you define engineering?"; What do you think of when you think of engineering careers?") as well as aspects of self-efficacy and motivation related to teaching engineering (e.g. "What kinds of professional development, if any, have you experienced regarding engineering or teaching engineering?"; "Are you motivated to teach engineering to middle school students?").

EVALUATION DATA
At this stage in the project, we are able to explore some preliminary findings from the evaluation data which may be useful to other practitioners seeking to learn from or adapt projects like this one. At a high-level, our student-focused metrics revealed a positive impact of our intervention and a need to revise our measures. Our capacity-building measures showed an increase in teacher's level of confidence in incorporating engineering activities as well as strengths and weaknesses of our collaboration and Lachapelle (2014) and the Engineering Design Cycle used by Engineering is Elementary (2019).
Overarching Goals and Evaluation Metrics. While the day-to-day operations of the project focus on individual school engagements and activities, the overarching goals are more far-reaching, align with broader aims of the NSF IT-EST program, and include a variety of evaluation metrics to assess progress towards those goals. Specifically: In tandem, these goals leverage the collaboratively designed and facilitated set of monthly interventions in order to impact student learning and growth while also prioritizing a focus on teachers and the broader community as influential change makers in the lives of students.
Student-focused metrics (Goal 1). Given that engineering content is not a formal focus of the middle school curriculum, PEERS focuses primarily on understanding how students conceptualize engineering careers and how those beliefs change over time throughout the project. We have adapted several extant tools for this purpose which are administered in pre/post fashion at the beginning and end of the students' science course (i.e. semester or year). One of the primary tools we use is the Draw an Engineer Test, which has been regularly used in engineering outreach settings (e.g. Gillen et al., 2017); Hammack and High, 2014;Knight and Cunningham, 2004), solicits an illustrated artifact from students of "an engineer at work" while also asking open-ended prompts (e.g. "What does an engineer do?"). The DAET is complimented by a quantitative survey, the Engineering Identity Development Survey (EIDS) (Capobianco et al., 2012) which includes questions about academic identity (e.g. "I am good at solving problems in mathematics.") as well as engineering career identity (e.g. "Engineers solve problems that help people", "When I grow up I want to be an engineer").
In addition to the pre/post measures, we periodically perform classroom observations to get a deeper sense of what aspects of activities seem to resonate most with students and incorporate that feedback in future curriculum development. Early in the project, these observations were very structured and naturalistic, with observers taking specific notes at defined intervals. This initial protocol loosely followed the structure, but not content, of an existing instrument for observing K-12 classrooms (Patrick et al., 1997). As the project evolved, we determined that the variation across classrooms Student-focused Metrics (Goal 1). To evaluate how well we are providing affirming experiences engaging with engineering concepts and broadening views of who can be an engineer and what engineers do, we examined pre/post changes in DAET and EIDS measures. Of the students engaged in the PEERS program, approximately 235 students assented, had parental consent in Year 1 for their data to be included in project sharing and research as governed by the Institutional Review Board for the Protection of Human Subjects in Research, and completed pre and post surveys or DAETs. These students represent approximately 31 percent of total students from across the three counties. It is important to note that because each implementation of classroom activities is a regional collaborative effort with teacher and industry stakeholders, specific implementation in each of the three partnering regions is intentionally different. Although we targeted common elements across the three project regions so that the curricula could be shared at a high level, implementations were tailored to meet the needs and expertise of the teachers and industry partners in each context. Therefore, at this stage in the project, we cannot offer causal explanations for each individual teacher/context and we focus on outcomes associated with high-level commonalities and project goals.
DAET data and analysis. Deeper analysis of the DAET is on-going, but we offer some preliminary findings herein related to the deeper analysis (Matusovich et al., submitted). There were 232 matched pre-and post-intervention responses to the question asking if students know any engineers. The majority of students did not change their response after the program: 28% indicating that they do know an engineer and 43% that they do not know an engineer. A smaller group changed their responses with 18% moving from not knowing an engineer to knowing an engineer and 11% from knowing to not knowing an engineer. Based on changes in who they listed as engineers as well as changes in actual drawings, we believe these numbers offer preliminary evidence that we are helping students develop concrete ideas of who engineers are and what they do. For example, some students listed members of the PEERS team (e.g. university and industry partners). Moreover, drawings and descriptions of engineers further included ideas directly related to the interventions. For example, although the drawings generally reflected the kinds of actions and artifacts found in other studies (e.g. Hammack and High, 2014), in our study more students represented cars, buildings, and the ideas of fixing and repair post compared to pre intervention which is different than prior works (e.g. Thompson and Lyons, 2008). The shift is consistent with the PEERS modules such as building mountain roads where marbles represented cars traveling on said roads, a different activity of fixing mountain roads, and the biome module that talked about engineering with regard to impacts of buildings on water run-off.
EIDS survey data and analysis. We chose the EIDS because among the limited instruments for measuring youth conceptions of engineering, the EIDS has had a robust developmental process and it widely used in the field. Nonetheless, past research on the EIDS has shown some discrepancies in how the overall instrument breaks into component scales. Early work (Capobianco et al., 2009) split into four constructs (i.e. academic identity, school identity, occupational identity, engineering aspirations) while later work (Capobianco et al., 2012) indicated just two constructs (i.e. academic, engineering career). While one might hope that psychometric scales are robust enough to behave similarly across different populations and studies, scale development and validation is always anchored on fits of the original data. Therefore, when using instruments in new contexts, it is not altogether uncommon to get different results which is why it is always important to begin with the raw results when working in a new context.
Investigating the internal consistency of these constructs in our student data revealed questionably low values of Cronbach's alpha (i.e. Academic ranging from 0.42-0.58 Pre-Post, Career ranging from 0.52-0.65 Pre-Post). Inspection of the questions themselves lends insight into why this might occur. For example, the "engineering career" construct contains items which seem to be very conceptually different as opposed to targeting the same implicit latent construct (i.e. "Engineers solve problems that help people" versus "When I grow up I want to be an engineer"). Such results could also be impacted by the scale (1:3. No, Not Sure, Yes) used which stretches assumptions about such data being at least interval level for parametric statistical tests.
Considering these concerns about the reliability of composite academic and career scales and with questionable assumptions about the scale and distribution of the items, the results that follow target specific questions most aligned with the aims of the project. Wilcoxon Ranked Sign Tests were conducted (non-parametric equivalent to the paired t-test) and a Holm correction was applied to control for family-wise error due to multiple comparisons. Item prompts, baseline score frequencies, and change score frequencies are all reported in Table 2.
Often, a majority of students are reporting "Yes" to the prompts even in the pre-test and so the instrument is limited in its ability to measure growth throughout the year. We were surprised by the high numbers of affirmative responses in the pre-test because literature continues to report that people of all ages struggle to understand what engineering is and what engineering work entails (e.g. see Montfort et al., 2013 for a study of secondary students). The classroom observations also noted instances of teachers and peers encouraging affirmative answers, perhaps out of a desire to represent the students, school and PEERS project well.
Positive statistically significant shifts occurred in the prompts around solving problems that help people, working in teams, using mathematics, and using science. Because of the structure of our project activities (e.g. open-ended problems, a focus on cultural relevance, integrated with science content) the positive statistically significant pre-post shifts in these items are notable but expected.

Capacity-building Metrics (Goal 2).
We evaluate our capacity-building work through an in-depth investigation of teacher engagement and self-efficacy for integrating engineering content into their teaching practices and the partnership-wide collaborative processes. Regular semi-structured interviews serve as the primary data source for both of these efforts. Three focus groups held at the 2019 summer summit for the project provide supplemental data. In these focus groups, participants reflected on the project to date (including changes experienced over time) and identified important factors looking forward that might influence project success. Preliminary in-depth analysis (unpublished data) shows that when teachers were asked pre-project for a definition of engineering and confidence in teaching engineering, several teachers acknowledged they knew little or nothing about engineering and had low confidence in teaching engineering. Post-interview, teachers indicated an increased level of confidence in incorporating engineering into their curriculum and attributed that to the project. Some teachers also noted "see(ing) engineering everywhere" in daily life or interest in adapting/leading new activities in future settings. One teacher independently sought/received a community grant for engineering supplies and led new activities in an after-school engineering club.
Analysis of stakeholder interviews (i.e. teachers, school administrators, industry partners, and university affiliates) has led to key findings about the collaborative processes throughout the first year of the partnership (Gillen et al., submitted). Collaboration among these stakeholder groups was observed as a dynamic and emergent process as roles, people, and even shared goals shifted over time in response to other factors. Some participants articulate an understanding or even an expectation for such flux such as a teacher who said "being the first year, you're going to have lots of wrinkles and lots of bumps that you got to smooth out. And that's expected." Other participants however expressed situations in which they felt let down by one of their partners underscoring the importance of establishing clear and regular lines of communication early so feedback can improve the partnership. Another significant finding in the cross-stakeholder analysis was in exploring factors that mitigate or exacerbate the inherent tension between individual or organizational demands and demands of the collaboration. In the context of a school, industry, university partnership, separated both geographically and conceptually in terms of day-to-day priorities and demands, the tension can be quite difficult to manage. Teachers' comfort with open-ended activities, engineering concepts, curriculum outside of state standards, and their relationship with peers or administrators can all impact the ability to successfully engage in the collaboration. In the literature on science content knowledge, researchers found a positive link between teaching science self-efficacy and content knowledge (Menon and Sadler, 2016), but the same cannot be said about the engineering education literature (Sun and Strobel, 2014). There is something about engineering in particular, that seems to have decreased the perceived capacity to contribute to the collaboration for teachers and some did explicitly state they were afraid of looking unintelligent in front of their engineering partners or students. Similarly, for industry partners, the work culture, espoused company or management values, comfort talking to middle schoolers, and individual motivations to volunteer all influence how professionals manage their commitment to both the project and their regular responsibilities. There is considerably less literature on industry partnerships with middle schoolers to which we can compare our findings. Although not individually supported by quotes, all of the factors listed here, for teachers and industry partners, were evident in our interviews and in the focus groups where stakeholders from different groups talked together.

Pre-Test Frequencies
Pre-Post Changes Across Categories

DISCUSSION
Looking across these data, reflecting upon our experiences building partnerships with stakeholders from these three regions, and in consideration of the broader literature, we note two overarching themes which warrant expanded discussion and could guide future work. The first theme discusses a need to distill goals for engineering outreach by wrestling with what success might really look like for middle-school youth engagement with engineering. The second theme focuses on cultivating community capacity to better support education systems and the simultaneous potential for and challenges of collaborating to build such infrastructure.

Distilling Engineering Learning Goals for Youth and
Evaluating Success. Formally stated, our student-focused project goal is to increase youth awareness of, interest in, and readiness for diverse engineering related careers and educational pathways. Government agencies, school personnel, and researchers have identified many key engineering learning outcomes for K-12 students (e.g. Cunningham and Lachapelle, 2014;Moore et al., 2014;NGSS Lead States, 2013). Further, there are research instruments which focus on constructs the field believes are important in predicting pursuit of engineering careers. Aware of this literature, and with project-wide experience in teaching and researching in these arenas, our team still has repeatedly found ourselves having to pause and question our goals and approach. We iteratively ask ourselves: for a project aligning engineering activities with science standards and regularly working with all students in the school as part of their normal science instruction, what is the distilled essence of our engineering goals? What would our middle schoolers know or say which would most demonstrate project success?
While we had literature-rooted answers to these questions early in the project, several factors that are not unique to our project have caused us to reconsider. Practitioners new to teaching about engineering and engineering design find themselves in a position where "the 'answer is in the book' system breaks down" (Brophy et al., 2008, p. 381). Moreover, constraints of the school system such as district versus school level support for engineering education (Douglas et al., 2016) and access through implementation of new pedagogy or knowledge (Gulamhussein, 2013) impact teachers' ability to invest time and energy into bootstrapping new content or pedagogies. As teachers have stepped up to highlight engineering elements of the lessons, we have become acutely aware of the need to prioritize a short list of accessible tidbits. Our first iteration, like many other engineering outreach efforts, focused on introducing the engineering design cycle. However, in the context of our middle school students and teachers, with such heavy focus on standardized testing in the science discipline, building class-wide knowledge about the engineering design cycle might interfere with student knowledge of the scientific method. Further, teachers reported discomfort with fielding student questions about specific engineering content knowledge as a significant barrier to integrating engineering into their classroom. As a pragmatic team seeking to recognize the constraints of the system we are working within, we realized that our top priorities were to strive (1) to provide affirming experiences for all students in "doing engineering" as part of learning science and (2) to broaden conceptions of what engineering work is and who can do engineering. From these top priorities, and iteratively adapting and refining elements from work of Cunningham and Lachappelle (2014), we translated our aspirational goals into the following concrete items we want our middle school youth to know about engineering and engineers: • Engineering is in every community and makes a difference in people's lives.
• Everyone can learn to do engineering.
• Engineers are creative, curious, and imaginative.
• Engineers work with many types of people to understand problems and create solutions.
• Engineers rely on knowledge from multiple subjects to understand as much as they can.
• Solving engineering design problems requires compromise and trade-offs.
• Engineers view mistakes as normal and important and try to learn from them.
As we have distilled our goals, so too have we revisited benefits and limitations of our student-focused research instruments. For example, in the EIDS, one of the prompts for response is "When I grow up, I want to be an engineer." While students responding affirmatively to this after engagement with our program would be exciting, we know that career aspirations, especially in youth, change often in response to many other pressures, opportunities, and barriers. Questions on the EIDS could be very effective at revealing potential historical misconceptions (e.g. student responses to if engineers are creative, if engineers solve problems that help people) but given the skewed responses we consider that either these misconceptions do not operate in our context or there is acquiescence bias impacting results. Further, while increases in reports of "knowing an engineer" and drawings of more diverse engineers at work in the DAET are also encouraging, they alone do not capture the quality of data we really desire to understand if and how we might be broadening students conceptions of what engineering work is and who can do engineering. Additionally, the anal-ysis of DAET drawings is sufficiently time-consuming that the drawing itself is impractical when implemented as only a part of a suite of evaluative tools such as in our project. While one might have anticipated some of the challenges associated with the instruments (e.g. the DAET requires extensive analysis time), few studies report when instruments do not work as anticipated. Moreover, these instruments are widely used and, even though flawed, alternative options are even more time-consuming and risk having less participation in at-risk communities such as the ones with whom we collaborated. For example, student interviews take time to conduct, often have small samples, take additional time to analyze and require parental permission for recording. Our future plans across these student-level instruments include expanding upon targeted open-ended questions such as on the DAET, discontinuing the drawing component, and synthesizing other survey items which can better focus on nuanced growth around our objectives. Ideally, such measures would be practical and useful to teachers as well in ways that the DAET is not since it requires extensive analysis procedures.
It is important to again note that our argument about distilling down engineering learning goals and modifying research tools are influenced by our project context which includes work with science teachers with a wide range of knowledge and experience with engineering and all of the students in specific grade levels at the school. As examples, a Project Lead the Way course or an after-school robotics club might be able to center more comprehensive engineering content and, as a result, could study deeper impacts from that work on students.
Cultivating Community Capacity to Better Support Education Systems. Multiple lines of research converge on the need for broad-based community investment in education systems, especially in rural areas. As discussed in the introduction, teachers are already over-tasked as a result of both in-school and out-of-school non-teaching responsibilities in addition to their instruction, supervision, planning, grading, and communicative tasks (Scholastic and the Bill and Melinda Gates Foundation, 2012). Incorporating important but un-assessed priorities like engineering content and workforce development just further add to that strain. While the unique context of rural areas offers many strengths for seasoned teachers tapped into strong social networks, barriers to entry into these community norms and challenges of the work environment impact recruitment and retention in STEM (Goodpaster et al., 2012). Individual student barriers are also prevalent: hour-long bus-rides each way to and from school, persistent literacy challenges, or already limited expectations about possible futures (Boynton, 2014). While partnerships are not an answer to all problems, Barbara Gray's foundational work (1989) in interorganization-al collaboration highlights particular promise when "stakeholders have a vested interest in the problems and they (the stakeholders) are interdependent" (p. 10). Our experiences corroborate that these broader issues from the literature do indeed occur in our areas and that our project stakeholders each have unique vested interest. Yet, as we have discovered throughout the project, fostering collaboration as part of building community capacity that can sustain beyond the life of a grant-funded project, even among talented and willing partners, is no easy feat. Though future research is needed to more comprehensively understand the factors that mitigate or amplify the tension inherent in collaboration discussed in the results, in our work across the project we note some specific challenges that seem to repeatedly influence our context.
Though collaborative processes are expected to be dynamic and emergent, so too are the environments of the individual partners. In particular, under-resourced schools are documented as having particular volatile environments with high rates of turnover in teachers and administrators (Quartz et al., 2005). Thus, it is vital to build a network of connections between individuals across the organizations so that the shared endeavor doesn't hinge on any single person who may be moved or leave. Our project has navigated significant shifts in school personnel in a short time. For example, in one county school system, we saw personnel changes including five principals and two administrators in the superintendent's office. We also had a principal change and a change in teacher assignments right before the start of the school year in another school system. Such changes require a heavy pull on other members of the network to onboard new individuals. While on-boarding is a collaborative process and has been successfully navigated in PEERS, there are potential negative impacts beyond the time and effort required of the team. For example, studies on student motivation and career choice among rural youth highlight the importance of not only having an interest in engineering sparked, such as by a stellar 6th grade science teacher, but also on the ability to sustain that interest . The sustained interest is impacted both by the quality of future science experiences and whether or not that stellar 6th grade science teacher remains an active support or available resource in the student's community (Boynton, 2014;Boynton et al., submitted). In a similar vein, literature on sustaining teacher-led classroom innovations highlights the importance of supportive school administration and learning communities (Nieto, 2007). High turnover rates among teachers or administrators can make building collaborations challenging and can leave teachers feeling unsupported. Data from the focus groups confirmed the need for having the support of the administration and other teachers to keep the PEERS engagement going. Resolving the high turnover rates is beyond the scope of the project but we have demon-strated that some ill effects can be navigated through partnerships with supportive infrastructure.
While persistent infrastructure created by the enacted partnership of many different individuals has benefits as described above, so too does it present a challenge because of significant variation in views of the what, why, and how of the collaboration itself. While there need not be one common answer to these questions across all parties, the questions themselves are vital when considering longer-term sustainability of efforts to address the project goals. We have found ourselves wrestling with the question of what success might look like beyond the grant. Is it teachers continuing to implement the lessons we developed together as science teachers, industry partners, and engineering educators? Or, what if the teacher doesn't have sufficient volunteer resources to assist in the hands-on lesson resets between classes but does regularly arrange for the industry partner to come talk to students about a range of engineering and technical careers that apply science they are learning? While both could be argued as great successes, the different ways individuals operationalize the goals and the role expectations associated with those notions can cause friction if not discussed. During the grant-funded life of the project, the university affiliates serve as facilitators to (re)build shared understanding about the partnerships. Theory on organizations suggests that such regular reflective and forward looking discussion is vital; for example, Day and Day (1977) note that "the negotiated order theory downplays the notion of organizations as fixed, rather rigid systems…. Instead, it emphasizes the fluid, continuously emerging qualities of the organization, the changing web of interactions woven among its members…" (p. 132). In the process of our work, and looking towards providing tools to facilitate the partnership building even after the grant has ended, we have created an evidence-based reflection tool for interorganizational collaboration which can be found in Appendix A. Further, Table 3 pragmatically  distills the broader discussion into a table summarizing the key issues evident, how we actively work to address those issues, and related resources and recommendations for those engaged in similar work.

ASSOCIATED CONTENT
The appendix mentioned in this manuscript can be found uploaded to the same webpage as this manuscript.

Author Contributions
The manuscript was written through contributions of all authors. All authors have given approval to the final version of the manuscript.

FUNDING SOURCES
This material is based upon work supported by the National Science Foundation under Grant No. 1657263. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

ABBREVIATIONS
ITEST: Innovative Technology Experiences for Students and Teachers. NGSS: Next Generation Science Standard. NSF: National Science Foundation. PEERS: Partnering with Educators and Engineers in Rural Schools. PSECC: Promoting and Supporting Engineering Career Choices. Alleman, N. F., and Holly, L. N. (2012). Doing more with less:

REFERENCES
The role of school-community partnerships in the academic success and postsecondary aspirations of low-Income students in small rural schools in Virginia. Adopt an iterative approach to research design, periodically revisiting instrumentation. Insights may be found in design-based research methods (e.g., Bakker 2018).

Sustainability challenges:
Maintaining collaborative capacity in a complex system (e.g., mitigating high turnover in schools) Distributing responsibility and buy-in through regular communication and annual meetings Establish mechanisms for monitoring and feedback such as our reflective tool in Appendix A