Towards Creating a Feedback Loop: Assessing Perceptions of Teacher Preparedness and Beliefs About Student Learning Amongst Math Teachers, University Faculty, and School District Administrators
Authors: R. Lorraine Bernotsky, Kimberlee Brown

Contents
3. Design, Data & Analysis
Print Poster
« Back to Poster Hall
3. Design, Data & Analysis
Next »

We employed multiple methodologies for this study, both quantitative and qualitative, but the research questions (above) remained the central focus of each methodology:

Methodology I:  Analysis of Existing Data
Existing data for this project includes assessment documents, previous survey results, and other forms of data collected previously within the departments preparing pre-service teacher candidates.  In addition, existing data includes lesson plans from math and science educators (in school district settings), as well as assessment, evaluation, and program documents.  The research team for this study includes a research methodologist as well as specialists in the areas of mathematics, biology, and secondary education.  This collective background provides the necessary expertise to interpret and analyze data from across the disciplines and institutions in order to frame the backdrop for the remaining methodologies detailed below. 

Methodology II:  Surveys
In our previous work, a survey of math and science teacher graduates was conducted.  Through the analysis of those data, it became clear to the research team that information from other groups, specifically university faculty and school district administrators, would be needed in order to continue to address the research objectives noted above.  Two identically worded versions of the in-service teacher surveys were produced for our previous study, one for math teachers and one for science teachers, with the use of "science" or "math" language being the only difference between the two instruments.  An additional instrument was created for the school administrators, again containing as many common core questions as possible to allow for comparison of key data from all sources. 

It is important to note that the instruments used in our current work, and those we expect to use in any future work, are based on instruments that have been refined and tested in other studies (see for instance Blank, 2002; Harwood, Hansen, and Lotter, 2006; Love and Kruger, 2005; Woolley, Benjamin, and Woolley, 2004).  This not only enhances the reliability of our findings, it provides another context for analysis that is useful for increasing the generalizability of our findings.  What is unique about our work, however, is that we are using the instruments in ways they have not been used before (for instance by surveying school administrators with instruments initially designed for in-service teachers). 

Methodology III: Focus Groups
Focus groups are an important research tool for interpreting survey data and data gathered during the observations as this methodology allows respondents to elaborate on issues in more detail and within a clearly defined context.  Focus groups also provide researchers the opportunity to clarify issues that arise during the initial data analysis of survey results and to probe issues that need more explanation than a survey can provide.  Potential participants were identified during the survey process and selected respondents were invited to participate in focus groups based on their responses to certain questions or specific characteristics, as determined during the course of analysis. 

The focus group protocol was developed by the research team and was administered to specific groups of respondents in the study (in-service teachers, university faculty, and school administrators).  Focus groups were video and audio recorded, transcribed and analyzed using qualitative analysis techniques. 

Methodology IV: Feedback Teams
The final methodology for this study involved the creation of feedback teams, comprised of participants from the previous methodologies.  After the focus groups met and the data from all of the methodologies was analyzed, the research team assembled feedback teams comprised of university faculty, science and math teacher graduates, and school administrators.  The research team served as participant observers on the feedback teams as well.  The feedback teams were asked to respond to the recommendations of the research team (based on the prior data analysis) and make suggestions for implementation of a feedback loop (or edits to the recommendations).  The feedback teams were asked to think about what mechanism would be needed to create a feedback loop in a more formalized way so that the work that this study has begun can carry on in future years.