Methodologies Archives - The Free-Choice Learning Laboratory at HMSC

I have been absent from blog posting as of late due to the whirlwind of grad school, but that also means there is quite a bit to share related to work in the lab and research! My last post described the experience at ASTC in North Carolina – a great opportunity to see work at other science and technology centers, and to meet professionals in the field that are doing incredible things at these locations. Since then I have been ramping up on my personal research, but also balancing coursework.

I am really excited to be enrolled in Oregon State University’s Free Choice Learning (FCL) series this year. Everything I was learning “in the field” is now gaining context through courses in personal, sociocultural, and physical dimensions of learning. I have the opportunity to practice evaluation methods through assignments and read papers related to my research on family group interactions in the museum. I am thankful that I get to take these classes from Dr. John Falk and Dr. Lynn Dierking, two researchers that have studied FCL for many years!

In the visitor center, we are focused on getting our facial recognition cameras consistently working and capturing data. We have been collecting images, but getting 11 cameras to stream a lot of data at the same time is challenging as both the hardware and software have to sync. This has been a great learning opportunity in trial and error, but also learning the “language” of a field I am not familiar with. As I have to troubleshoot with engineers and software developers, I have been learning vocabulary related not only to the camera system, but also the usability of configuring the cameras and the software. Beyond the task of setting this up, it is an experience that I will reflect on with future projects that require me to learn the language of another industry, embrace trial and error, and patience in the process.

In addition to Cyberlab duties, I am busy coding video of families using the multi-touch table collected in August 2014. Over the past twenty years, research on family learning has shown us how exhibits are often used (much of this research was done by Falk, Dierking, Borun, Ellenbogen, among others). I am curious about the quality of interactions occurring at the touch table between adults and children. I developed a rubric based on three different dimensions of behaviors – responsive engagement, learning strategies and opportunities, and directive engagement, and whether they are observed at low, moderate, or high levels. These categories are modified from the types of behavior outlined by Piscitelli and Weier (2002) in relation to adult-child interactions surrounding art. From their work, they found that a distribution of behaviors from these categories support the value of the interactions (Piscitelli & Weier, 2002). Each category looks at how the adult(s) and child(ren) interact with each other while manipulating the touch table. I also modeled the rubric after what is used in the classroom to assess teacher and student interactions around tasks.

An example of a high level of responsive engagement would be that the adults and children are in close proximity to each other while using the exhibit, their hands are on the touch surface for a majority of the time, the adults are using encouraging words and acknowledging the child’s statements or questions, and there similar levels of emotional affect expressed between them. Learning strategies focuses more on the verbalization of the content of the exhibit and the integration of information, such as connecting the content to prior knowledge or experiences external to exhibit use. Finally, directive engagement looks at whether the adult is providing guidance or facilitating use of exhibit by directing a task to be performed, showing a child how to accomplish the activity. From this data, I hope to understand more at depth how the table is used and the ways adults and children interact while using it. This may give us some idea as to how to support software and content design for these forms of digital interactives, which are becoming more popular in the museum environment.

My goal is to have videos coded by the end of the month, so back to work I go!

Piscitelli, B., & Weier, K. (2002). Learning With, Through, and About Art: The Role of Social Interactions. In S. G. Paris (Ed.), Perspectives on Object-Centered Learning in Museums (pp. 121–151). Mahwah: Lawrence Erlbaum Associates.

I started “18^th” grade this past week, also known as the second year of my two year program at OSU. The beginning of a new academic year is a great time to reflect and I’ve been thinking about my evolution as a graduate student and on the work we have accomplished in the Cyberlab thus far. Since my first posts from last year, much of what I wrote about being patient in the process still rings true. Iteration and refinement help to direct the course. As I have made progress in my own research study, I still have to be patient as the project unfolds as some unique results may appear that I might otherwise miss. Looking forward to where I might be next September is exciting too. It is unknown at this time, but thinking about all of the potential opportunities…who knows!

I am proud to say that I have transitioned into the analysis phase of my Master’s research. I have some results from my interviews of the families that used the touch table, but more will be following as I start to review the videos. One challenge has been to develop a strategy for analyzing the video of families using the table. This is something I have not done before. There are some resources for analyzing video in non-school settings, so I am referencing that heavily. One book that has been particularly helpful is Video Research in the Learning Sciences (Goldman, Pea, Barron, and Derry, 2007). This is the most comprehensive source with theoretical and methodological guidance I have seen, especially with connections to filming observations in an informal science setting. As family behavior and interactions in a museum setting has been studied (Falk, Dierking, Ash, to name a few), we have a better idea of the types of behaviors that take place in this environment. I am interested in the degree to which they are occurring around the touch table. We know parents may read content on signage aloud, point, question, recall past events…but to what extent is this happening with technology that is not commonly seen (at least scaled to a table on a daily basis)? I’m going to approach this on a spectrum or scale of low to high levels of the presence of behaviors. Using a rubric as a way to score the interactions, something done to assess teacher facilitation in the classroom, I believe this is a way to put a “measure” on the adult and child interactions. From the results, we may have a better idea of what the quality of interaction with touch tables looks like in a science center, allowing us to point to specific areas to improve content that affords these behaviors on a deeper level.

This quarter I also started taking the free-choice learning series through the College of Education. It is perfect timing as I work through my research project. I am gaining knowledge and a better understanding of what learning is and the context to which it takes place, and how we do not learn in isolation. Our perspectives and experiences can be shaped by those around us, one reason for my interest in family learning behaviors. The first course is “Personal Dimensions of Learning” and I appreciate the new resources to read about motivations and identity as related to self-driven learning. As this is an Ecampus course, there are students from around the country doing incredible science education projects both in and outside of a formal classroom setting. I am looking forward to getting to know them better as the quarter progresses.

Next post will recount my first experience at the annual meeting of the Association of Science and Technology Centers (ASTC). I will be tweeting from Raleigh next week – follow me @East_JennyL.

What do erupting volcanoes, learning to code, building bridges, creating art, and cooking have in common? They are all STEAM activities! This past year I started working as an external evaluator on Lincoln County School District’s (LCSD) 21^st Century After School Program. There are seven sites throughout the county that are funded as 21^st Century Community Learning Centers. The goal of the federal program is to “provide academic enrichment opportunities during non-school hours for children…offer students a broad array of enrichment activities that can complement their regular academic programs, and offer literacy and other educational services to the families of participating children.” LCSD’s program aligns with these goals, focusing specifically on providing students and their families STEAM-based activities.

Being a new project in the 2013-2014 school year, the 21^st Century After School Program kept me on my toes as an external evaluator. All documents for data collection had to be written very quickly so they could be distributed as close to the start of the school year as possible. While the tools for collecting data did not fail they certainly could have been better. As the program moves into the second of it’s five years, I thought it best not only to redesigned some of the tools but also to communicate to the site coordinators why collecting this data is important. Site coordinators, one for each of the seven sites, are responsible for distributing, collecting, and turning in all of the instruments I use to collect data. The only part of data collection they are not responsible for are the standardized test scores, which are handled by the school district. Collecting data is no small task for the site coordinators because they already have a lot on their plate. Therefore, the program coordinator and I agreed that sharing results from the first year would showcase the importance of data collection.

Early in September I presented preliminary findings at the first site coordinator meeting of the second year of the program. You can view and listen to my presentation here. The presentation ends with me addressing the four main instruments for data collection: a monthly recording sheet to document the STEAM activities and resources, the student STEM interest survey, a family literacy and STEAM night reporting document, and the parent survey.

Working as an external evaluator on these types of projects is always exciting for me. They push me to think in new ways and certainly make me a better researcher and evaluator. If you have any questions about the 21^st Century After School Program or the documents I’ve produced, please feel free to be in touch.

“Hands-On Science Museums and Their Visitors” is the topic of a two-day conference coming up September in Rio de Janeiro, Brazil. Cyberlab will represent Hatfield Science Center/Oregon State University and will join other Science communication professionals from Argentina, Brasil, Chile, Colombia, Costa Rica, United States, Mexico and the United Kingdom to engage in dialogue about visitor meaning making, basically the kind of conversation we are very enthusiastic about engaging in and promoting, especially in such a multicultural setting.

Luisa Massarani, who was a Cyberscholar this Summer and who is the Director of the RedPop, the Network for Science Communication for Latin America and the Caribbean, organized this event to discuss strategies Museums around the world employ not only to investigate learning but also how a diverse public construct meaning from their visits. Although a bit intimidated I will admit, I am supper excited to participate in this event because it strikes me as a place where paradigmatic shifts in learning research are possible and in fact welcome, as a place where we can make room to discuss strategies to capture and analyze meaning making, to look at visitors from their perspectives, to go beyond the traditional measures of learning outcomes in research, to really give our visitors a voice we can dialogue with in the academic written world.

We talk about this need for a new culture of learning in our Free-Choice Lab meetings, Luisa talked about that in her seminar presentation as a Cyberscholar and the need to understand “provocation” and build provocative exhibits. Shawn and I talked about this in an article just published in the NAI Magazine “Legacy”, which led us to an invitation to expand this thinking through a series of articles for the InterpNews Magazine next year. As these kinds of dialogues spread and increase (as it seems to be happening in my opinion), this discussion becomes highly related to current dialogues on learning research methods and applications in the world of practice. I have been recently involved with the new “Methods” Research Interest Group of NARST (National Association for Research in Science Teaching) and the current development of a broad scope dialogue on learning research that seems to be heading in the direction of valuing these paradigmatic discussions and the need to change.

Even though we are all trying to do this kind of more inclusive, learner-based research in our work, we need to see ourselves as important voices in the larger network of discussions, and commit to speak our mind in fruitful and inclusive ways. Meetings like this really allow us to reflect on how we are trying to do that in the context not just of our own lab and cohort here, but in the larger international context as well. It also gives us a chance to make things real, to move from discussion to actual application invigorated by the good work of others and motivated by our own growth and learning as professionals in the field.

To learn more about RedPop visit the following pages:

http://www.redpop.org/redpopasp/paginas/InfoPrensaDetalle.asp?SitioID=1&InfoPrensaId=90

http://www.redpop.org/redpopasp/paginas/pagina.asp?PaginaID=3

As the school year comes to an end so too do the school-based projects I evaluate. What this means, first and foremost, is a mad rush to collect data. It’s also a time for those involved in the project to come together and share what they’ve been doing for the past 8 months. As an evaluator, I have been focused on the mad rush of data collection – writing surveys, distributing surveys, leading focus groups, and conducting site observations. All of this data is needed to prove these projects are doing great things; however, what I truly love is hearing about the activities educators are using to engage their students in STE(A)M.

As an evaluator I have to ask: how do you capture the amazing ideas these educators are coming up with and how do you evaluate the impact they’re having? And by impact I mean both the impact on students and the impact on other educators who are hearing what’s been done in other educational settings. What I’m actually asking is how do you evaluate jaw-dropping moments?

To put these questions into some context I need to clarify that I am, from here on, talking about my experience as one of the evaluators with the Oregon Coast Regional STEM Education Center . The STEM Center is a collaboration between Lincoln County School District, Tillamook County School District, and countless institutions and organizations up and down the Oregon Coast. A U.S. Department of Education Math Science Partnership grant funds the STEM Center, which offers professional development to teachers in both school districts. Over the course of the school year the teachers put Project-Based Learning (PBL) into practice.

Now to turn back to the questions I asked above. How do you capture the amazing ideas these educators are coming up with? We do collect and archive as much as we can, specifically PBL overviews, PowerPoints, assessments, and other resources teachers and students use during PBL and include them all on the website for others to use. 2013-2014 school year PBLs should be up this summer but you can peruse 2012-2013 PBLs here.

Sean Bedell shows his colleagues a core sample he and his students took while looking for evidence of Oregon’s 1700 tsunami (project further discussed below).

How do you evaluate the impact these projects have? This question is more difficult to answer. For students, we distribute a STEM interest survey at the beginning and end of the school year and we use student test scores, but to me that can’t tell the whole story. The hard pill to swallow as an evaluator is that in order to capture what I would call the true impact on students and the whole story, this project would require longitudinal study (think 5 or 10+ years of collecting data and interviewing students). We also have teachers complete pre- and post-project surveys and have them write a reflection and those sources have proven to be useful in past projects to understand impact. We talked about running a focus group with the 2013-2014 STEM Center teachers to gauge how they incorporate all of the information delivered through professional development to plan and implement their PBLs. Anecdotal evidence shows that teachers are no longer taking a kit or pre-written lessons and using it as is in the classroom; instead, they are taking ideas from multiple sources and piecing together large scale projects. Essentially, their self-efficacy to do PBL and STE(A)M in the classroom is rising.

Most of the teachers presented their 2013-2014 PBL to their colleagues last Saturday. I was in the audience with my jaw on the floor for most of the day. I really appreciated the variety of presentations, which included posters, ignite presentations (i.e. short, sweet, and fast), and student voice-over presentations. In the afternoon some students came and presented PBLs from their perspective. I can’t cover all of the projects here and encourage any readers to keep checking the website as we add the 2013-2014 PBLs. Here’s a selection of projects that caught my attention:

– Students at Newport Prep Academy studied marbled murrelets and corvids, specifically how the latter prey on the former’s eggs. Human interference (i.e. leaving trash at picnic sites) brings corvids closer to marbled murrelets. Check out the Public Service Announcements produced by the students using iPads and iMovie. QR codes the students created will soon be at picnic areas of state and national parks.

– Students at Eddyville Charter School focused on tsunamis. They designed, built, and tested their own tsunami structure at Hinsdale Wave Research Lab. Students also researched the earthquake and tsunami that hit Oregon in 1700 by taking core samples at five different locations to look for tsunami evidence. Check out their website, which contains videos and student wikis about the project.

– At another school, students had to engineer a closed forest ecosystem to gain an understanding on how we could sustain life on another planet. This was definitely a test-retest project as students had to monitor pH and water levels to keep plants alive. Many students had to re-engineer their plans and use different materials to meet the challenge.

Examples of student-designed forest ecosystems.

– In Tillamook, elementary students were given a challenge by the local utility company, which was really a fake letter written by the teacher with the company’s approval. Students had to evaluate different sources of renewable energy and where such sources could be placed within the landscape to be most efficient. At the end of the PBL, students presented their findings to an expert panel.

See? Jaw on the ground! And this is just a sample of what these amazing teachers in Lincoln and Tillamook County School Districts are working on with their students!

Last week I wrote about Bakhtin’s idea that in order to put together a real, full research account, the researcher point of view has to be put in dialogue with the point of view of the participant in research. Neither point of view is complete in and of itself. The question I raised was how do we make sure and include the voices of research subjects in our work such that they are co-researchers with us and help create those fuller research accounts of experience. One of the primary tools for engaging in shared research used in professional development of educators is video. When we video our practice as educators and (re)view it with others, we create the possibility of real dialogue among multiple points of view. My own experience working with classroom teachers and museum educators, floor staff, and volunteer interpreters using video to reflect on experience has convinced me that neither my outsider observations nor their reflective writing have been sufficient to create real dialogic relationships where we become co-researchers. In some cases, overarching cultural and social narratives about teachers and learners inevitably drown out the details of their experiences as they experienced them. In other cases, the details of those experiences defy categorization and reflection.

As one example, in one project to develop a professional learning community among veteran K-10 teachers, observations showed very little evidence of student led inquiry, but teacher narratives about their teaching reported detailed regular use of student-centered science inquiry techniques as part of their normal routines. Having teachers observe each other using a researcher-generated rubric did little to change their assertions about their teaching even though they were directly contradicted by the observational evidence. Similarly, in multiple projects with museum educators, those educators report a basic belief that visitors do not read labels. Putting these educators in the position of researchers observing visitors generated copious examples of visitors reading labels, yet educator narratives about visitors consistently fail to include that reading. The data and observations simply don’t stick and are overwhelmed by other kinds of details or by larger-scale institutional narratives about visitor behavior.

In both instances, we eventually turned to video as a way of creating what we hoped would be shared texts for analysis and reflection. Yet, the existence of video itself as a shared text is also not enough to form the grounds for researchers and participants to become co-researchers. Watching video and talking about it, even using a rubric to analyze it definitely helps educators be more reflective about their experiences and to put them in larger contexts than the overarching narratives we tend to fall back on. But there still seems to be a missing step.

For Bakhtin the missing step seems to engaging in co-authorship to create some kind of new text or new representation of or about that experience. When we watch video and reflect on it with each other, educators and researchers both come away with a stronger shared sense of what’s happening, but in the absence of creating some kind of new shared text or representation, we don’t have the opportunity for truly developing as co-researchers. Are there places and projects beyond video that we can do on the museum floor that will help visitors (re)create, write about, or otherwise represent their experiences with us as co-authors?

The Free-Choice Learning Laboratory at HMSC

Informal science education research

Category Archives: Methodologies

Research update — making progress one video at a time

Reflections as I start “18th” Grade

STEAMing Ahead at Full Speed

Cyberlab joins Latin America Discussion on Visitor Meaning Making

How to Evaluate Jaw-Dropping STE(A)M Project-Based Learning

Promoting Participants as Co-researchers

Contact Info