Long Story, Short

• Developed and executed a mixed methods research evaluation of the Dual Language Immersion (DLI) program to measure its impact on student, staff, and organizational outcomes based on Key Performance Indicators (KPIs) and performance insights.
• Conducted focus groups and in-depth interviews with participants and assessed their responses using sentiment and thematic analysis through Qualtrics and NVivo software.
• Established a protocol and methodology for naturalistic observational research to gather insights on the consistency of classroom structure and DLI program practices as well as student response and behavior.
• Analyzed expansive multi-year cohort and cross-sectional student and staff demographic, performance outcomes, and survey data using R (Posit) programming software.
• Created a comprehensive and coherent evaluation report as well as concise slideshow and poster presentations to disseminate information to various stakeholders including district executives and various conferences.

Long Story, Not so Short

While multilingual education has become the standard in many countries around the world, the United States still has minimal foreign language requirements for students in K-12 education. However, some public-school districts have introduced programs such as Dual Language Immersion (DLI) in which children can begin language learning as soon as they begin their formal education. In 2014, Gwinnett County Public School (GCPS) District in Gwinnett County, Georgia introduced DLI education programs in a pair of kindergarten classrooms at two different elementary schools to offer their youngest students a 50:50 learning environment in which they receive half of their core subject education in their primary language and the other half in a language other than their primary language. A decade after the initial implementation of DLI within the district, the program had expanded to 22 schools within the district and included Spanish, French, and Korean program options.

Because the program had expanded so rapidly, the DLI program team within GCPS wanted to ensure that the program was effective and sustainable for the level of growth and interest that it was experiencing.

That’s where I came in.

I was tasked with developing a comprehensive evaluation of the effectiveness of the DLI program on student outcomes with the district’s existing student data however, “effectiveness”, “outcomes”, and even “data” we came to find out, were open to interpretation.

Gwinnett County Public Schools serves an immensely multilingual population and as such, the DLI program is considered a two-way program. A two-way DLI program, as opposed to a one-way program, is a program that serves to teach a population with different primary languages. While many students have a primary language of English, a significant proportion of the population have a primary language other than English and as such, the 50:50 language immersion allows them to become immersed in English as their English-speaking counterparts are immersed in the non-English language. This structure requires a split classroom of students with two teachers, each teaching two core subjects, one in the English language and one in a non-English language. To evaluate a program with this structure, there are many factors to consider when assessing student outcomes. For example, student attendance outcomes may be influenced by their level of comfort with their teacher, and it can be difficult to determine if that’s due to teacher behavior or language barriers when there are two teachers, each speaking in a different language.

Working with the DLI program team offered an opportunity to learn about the structure of the program including the resources available to them, their goals and progress, and personal anecdotes about the various obstacles they have faced throughout the implementation of the program. This project was just as much a collaboration between the DLI team and the evaluation team as it was an evaluation and research study. I engaged with the DLI team as one would a client by understanding their needs and communicating consistently and effectively about our approach and findings. Through ongoing feedback and conversation with the DLI team, our evaluation team gained valuable insights about the program that informed our approach to the study. Client facing work requires meeting and even surpassing expectations so that you can maintain client trust and a continuing relationship.

Expanding the scope of this evaluation was my first step in assuring the DLI team that I valued their work and wanted to offer them the highest quality information so that they could offer students the highest quality education.

Together with the DLI team and my research partner, Dr. Shannon Rodríguez, we established the student outcomes and levels of effectiveness that we wanted to measure. We decided to assess academic outcomes using state-wide assessment scores, student and teacher demographics of the program using GCPS data, student and teacher sentiment using the Educational Effectiveness Survey and qualitative data collection, and DLI program structure using classroom observations.

While the Gwinnett County Public Schools district maintains a significantly large quantity of data about its students and the district, the DLI program was still new enough to the district to not yet have a flag or marker for students in the system. Because DLI students were not explicitly marked as such in the available data, Dr. Rodríguez and I were tasked with first identifying the nearly 2,000 students in the program. Through this process, we learned that to best support the DLI program, we not only needed to conduct the evaluation, but we also needed to help them develop their data infrastructure so that they could track their progress moving forward. While this added an additional unforeseen deliverable to the original evaluation, it gave me an opportunity to show the DLI team that their program’s success was my main priority.

I developed a multifaceted evaluation with both qualitative and quantitative components and both primary and secondary data collection and analysis respectively. While GCPS did not previously retain any qualitative data about students or programs, the DLI team was more than willing to support our data collection process.

We conducted student focus groups as well as individual in-depth interviews with teachers and administrators so that we could gauge perspectives of the program from various stakeholders thus giving more comprehensive information about its impact.

The qualitative interviews involved asking the participants about areas in which the program has been successful and areas in which the program may need improvement. Among other questions, participants were asked to give specific examples about their experiences with the program. Participant responses were recorded, transcribed, and then coded for sentiment and thematic analysis using Qualtrics and NVivo software. These systems offered the opportunity to optimize our coding process, establish uniformity amongst reviewers, and offer additional validation through trained AI analysis.

Additional qualitative research was conducted through the naturalistic observation of the DLI classrooms. Each elementary grade level was observed for various factors including classroom structure, language consistency, and student behavior. These observations were then compiled and coded along with the other qualitative data. This data was then combined with the quantitative data available within GCPS.

Gwinnett County Public Schools is the largest school district in the state of Georgia and the 13th largest in the country. Because of the size of the district, GCPS has developed an elaborate system of district-wide data collection and storage. This allowed our evaluation to include secondary data analysis, coded using R (Posit) programming software, for student and teacher demographics and sentiments as well as student assessment scores. After establishing those within the DLI program, our analysis was able to utilize one-to-one matching to establish an appropriate comparison group for each of our quantitative variables.

The findings from this year-long study were then used to inform the continued practices within the DLI program. Deliverables for this evaluation were extensive and included a comprehensive report that was used to refine program policy and practices, a slideshow presentation to district executives, and a poster presentation for multiple conferences.