Field Trip: How to Use Analytics in Schools

EPISODE 66 | 11/3/2021

There’s no question that analytics play a massive role in modern life. With COVID and the pandemic, average Americans have probably seen more charts and graphs on a daily basis than ever before. But what role should analytics play in school? And how does a school district begin to use them?

DR. ADAM CIBULKA: The first step is to know what are the 2, 3, 4 key questions you’re trying to answer for your district, and then having a strategic way to communicate what those results are over time.

From Frontline Education, this is Field Trip.

Today we’re talking to Dr. Adam Cibulka. He’s the senior manager for 5Lab at Forecast5 Analytics, which is now part of Frontline Education.

Prior to joining Forecast5, Dr. Cibulka spent the better part of two decades in public education in the Chicago area, in roles ranging from teaching social studies, to coaching basketball, to serving as a high school principal, and later. as a district curriculum director.

DR. ADAM CIBULKA: When I was in my district seat. We had the challenge of monitoring, a variety of student metrics related to college and career readiness.

 

Passed along by the state of Illinois. And I got tired of using spreadsheets to monitor all that. So I reached out to the forecast five team and had just made mention, “Hey, can you help me out with this?” And we spent the better part of a year building to measure and automate.

 

Key indicators related to grades, discipline, attendance, but then also being able to combine, what is the beta version of what 5Lab looks like today: those data elements related to students with test scores like ACT, SAT, AP exams, to determine whether or not a student is college and career ready. So that’s been kind of my journey and I enjoyed the project so much that when there was an opportunity to join the team, I was excited to do just that.

 

RYAN ESTES: Well, welcome. We are thrilled to have you on the team and Adam before we get into questions about how analytics relates to school districts, let’s pull back for a second and, and stay a bit broader. Talk to me about your sense of how data and analytics impact the average person today, compared to say, 20 years ago.

 

DR. ADAM CIBULKA: Well, there’s so many data points on students, right now. Things like attendance, discipline, grades, all of those things were really kept at the local level in the teacher’s classroom. And then, with no child left behind, you saw an expansion of accountability for schools and the public reporting around not only student performance, but school and district performance. There’s this thought of comparing schools and, there’s varying degrees of what that looks like depending on the state. Because of that, there was this push to have more data related to students. Started with a state test, has expanded to tracking things like.

 

Participation in AP courses, career credentials, workplace experiences. The usual suspects of attendance, grades, and discipline are now being submitted to, in many cases, the state level to create this complete picture of, how a student’s performing and that volume of data needs to be housed somewhere.

 

But in spite of all this data, the question of, “How are students doing?” can be difficult to answer. If disparate data sets live in different places within a district, it can be hard for that district to tell its own story.

DR. ADAM CIBULKA: And there is a story to tell, especially the great work that’s going on in so many of our districts that the struggle is how do we get a speed to insight around how students are doing. To know, what are the points to celebrate? And then, where are the areas of growth for us as an organization? We officially have entered the era, I think, of big data in education.

 

And now it’s trying to synthesize that, and working with those data sets to say, “Okay, what are the critical pieces that inform what we are doing?” Not only student performance wise, but then how does that relate to expenditures, and hiring and, you know, ultimately supporting kids.

 

RYAN ESTES: A lot of talk is out there around data and analytics, but I don’t know that necessarily, we’re all talking about the same thing. From your perspective, what are analytics as they relate to K-12 school districts?

 

DR. ADAM CIBULKA: Any data point that informs decisions around a key theme and or topic. That’s really how I would define it, is to keep it at a really simple level of how are we doing on a particular topic: we’ll just take something as simple as attendance, and being able to see what that looks like at a district level, at a school level, and even at a student level, information that tells us how we are doing on a particular theme, topic, initiative, and doing it in an efficient manner.

 

RYAN ESTES: And you may have already touched on this a little bit, but maybe we can go a little deeper. And that is the idea of, why schools use analytics. Obviously we do that to support decision-making right? But what do analytics get us that we don’t have otherwise?

 

DR. ADAM CIBULKA: Fundamentally, it’s helping the child. To know what does a student need to know and be able to do by the end of the grade, by the end of their time, within the school and on to graduation.

 

But like any journey, there are always ways that students could get off track.

DR. ADAM CIBULKA: Could be a social-emotional reason, could be an academic reason. It could be a combination of both. But, districts and schools need to know are students staying on track with the goals and the initiatives we have outlined for each child, are they staying on, you know, the proper path to graduation and or success at the grade level.

 

What analytics help do is provide a story around the student and aggregate it up to the building level to say, “How are we doing on the things we deem important for our kids?” Are students coming to school? Are students successful in the class?

 

Are students staying out of trouble to really put it simply. But without analytics or without those measurements to support the decisions for each and every student, you’re kind of wandering in the dark a bit. And that’s what we hope we provide individuals, is that path to insight and that speed to insight. So you don’t have to guess but you actually know how students are doing.

 

RYAN ESTES: So analytics right now is often talked about often highlighted as something that yes, we should be doing and we should be using analytics to help support decision-making. But there’s a difference between saying “yes, we should do this,” and actually doing it. Is this common? Are many schools using analytics to help with decision support? Or is it something that is a little bit beyond where we are right now for most districts?

 

DR. ADAM CIBULKA: Well, we have definitely seen over the last few years a need for this and a request for this from districts. They know that this is an area for them to incorporate in their district to be more strategic around decision-making.

 

Surely it’s important to see how each and every student is doing, but through the use of analytics, you can see how specific student groups are doing to make sure, regardless of race, ethnicity, socioeconomic status, whether or not a student has an IEP or not, english learners, being able to dis-aggregate the information and say, “Are all students truly learning?” Which is the goal of every, every school district in the country.

 

RYAN ESTES: If you were to look at the state of education as a whole in America today, and the use of analytics, what letter grade would you give us so far?

 

DR. ADAM CIBULKA: Well, I think we’re probably at a, B. You know, we’re above average because you’re just seeing so many data sets being pulled in at the state level and at the local level. And that’s, honestly, the starting spot. The area of growth, though, to get us to that a mark is being able to visualize that, and having a way to visualize that in a manner that all levels of the organization can use the information.

 

A lot of times it’s just, “Hey, what’s on our state report card?” And that’s fine for one set of users. But at the building level, they need to know the actual student names, and the list of students to support. So having tiered analytics that start at the district level performance, but then work its way down to the individual student level is that area of growth for us. I would say we might be at a C in terms of the visualizations. You know, a lot of times the tools are a little limited. But we are to that level we need to be at.

 

RYAN ESTES: You mentioned tools just now. What are the different types of tools that schools use at the moment? I’m guessing it ranges anywhere from really hyper specialized data analytics tools, to spreadsheets and Excel and trying to cobble together charts and graphs. What do you see most in use across the board?

 

DR. ADAM CIBULKA: At the very fundamental level, it starts with some CSV files and some Excel spreadsheets. But what we are starting to see is some integrated dashboarding.

 

Giving district and building leaders, one place to go to view all of the important metrics that they need to on a daily basis. It’s that growth process that takes a district from a lot of separate file sets and a lot of separate handmade charts and graphs. As I was dealing with — I was doing that myself — to then automating that process to limit the lift, to where the district can spend 90% of their time using the information and only 10% of the time crunching the numbers, where in most cases that is flipped. To where 90% of the time is crunching the numbers, and only 10% of the time is acting.

 

RYAN ESTES: I think over the past year and a half, most Americans have probably seen more charts and graphs than we ever have before in our entire lives with COVID and the pandemic. And at the same time, there’s a difference between being able to look at data and being able to truly understand what it is telling us. What kind of training or skills are needed to do data analysis effectively, or even to set up and use these tools effectively?

 

DR. ADAM CIBULKA: The first step is to know what are the 2, 3, 4 key questions you’re trying to answer for your district, and then having a strategic way to communicate what those results are over time. In this time, what we’re seeing is a very simple question: “How are kids doing right now? How does that compare to how they’ve done in the past? And what are we going to do for students moving forward?” And to answer those three questions there needs to be some sort of measurement, and those measurements are defined differently by each and every district. What we try to coach districts on is there’s a lot of information out there, but what are the three or four key questions you are looking to answer on a day-to-day basis, on a quarterly basis, and at the end of the year. So again, when a school board member or a community member sees you at the football game and asks the question, “Hey, how are kids doing right now?” You have a high level answer because you’ve used the information, you’ve had it at your fingertips. To know how you’re doing on those, on those key questions.

 

Thinking about those key questions, I asked Dr. Cibulka, what are the kinds of questions he frequently sees districts trying to answer with data? And then, what kinds of data do they most commonly use to answer those questions and make decisions?

DR. ADAM CIBULKA: Yeah, it is especially to speak of the current year, a really big one is simply, “Are students coming to school?” Given the varied environment that students have experienced last year, they want to know first and foremost which students are chronically absent, and let’s triage and get to those students. The second question that they’re wanting to answer are “How are they doing in the classroom?” And that’s answered in a variety of ways at the secondary level, a lot of times it’s based on course grades are students successfully passing their classes and staying on track to graduate. At the elementary level, we see it a lot with benchmarking tools, NWEA map or iReady or Star 360, Renaissance — benchmark assessment here at the fall to say, “Okay, how many students are at grade level? How many students are above grade level, and which students are approaching or below grade level in terms of reading and math?” And having at their fingertips, the analytics to say, “Okay, here are the students that, in the case of a benchmark assessment, are below grade level. Let’s have a game plan of how to support those specific students.”

 

Dr. Cibulka says districts that apply analytics really well excel because they get the data to building level leaders, when that data has not historically been passed down below the district level.

DR. ADAM CIBULKA: What we see our most successful users doing is getting it down to that principal, the assistant principal level, where the building level leaders are knowing where are the areas where they are really knocking it out of the park and being super successful, and then where are the areas of pockets for growth, and then mobilizing their teams accordingly.

 

They need the information at their fingertips and what districts that are successful in the, you say analytics, are doing is saying “Okay, here’s the tool. Let’s teach you how to use the tool. And here are the two or three go-to dashboards that you can use to find out which students are not coming to school, which students are not at grade level are passing their classes, and which students potentially are on a path that could make them at risk to fall off track or below grade level. And being able to just give those three dashboards to answer those really important questions at the principal level allows the principal again, to change that ratio of action, where 90% of their time is spent, and very little time is spent on crunching the numbers. They can just use the information and act.

 

RYAN ESTES: Beyond that, are there other common challenges or barriers that districts run into in trying to use analytics effectively?

 

DR. ADAM CIBULKA: Some of it just is, as I mentioned before, making sure that the data that they are wanting to visualize is good information. There are so many input points within a school district for data points. You think of it, at the classroom level, you have a teacher putting in attendance and grades. Then at front office staff they could be editing that attendance based on a call-in by appearance that moves it from a unexcused absence to an excused absence. And then that potentially could change three days later. And then you have deans of students that put in discipline infractions and could code things differently, code in one way that has been a legacy code versus what it should be in this day and age. So really working with districts to own their information is always a barrier. Not one that can’t be overcome. I mean, we have seen many districts just through the use of analytics, see some gaps in how they are entering the information. And actually use that as a validation tool to say, wait a second, we need a pause on, on entering it this way. We should be entering it the way the state is requiring us, or, but we’re now doing it as a district.

 

So probably the biggest barrier is just making sure that there’s clean information going in and that all stakeholders and parties on that data entry front know the criteria for what they need to put into the system.

 

RYAN ESTES: You mentioned making sure the data is clean, that it’s good information. You mentioned making sure that the right people have access to the data. What else? What other tangible concrete action steps can districts take to set themselves up for a success in using analytics to make these kinds of data-informed decisions?

 

DR. ADAM CIBULKA: It’s probably going back to that those key questions. There are at times even though data entry is one access point or point of failure potentially, or barrier. Knowing the why, why are we using analytics is huge. When you use analytics, you potentially are shining a light on areas that you may not have known existed. And you’re really expressing vulnerability, and being okay with having areas of improvement is a huge step, and the organization has to know the why, “Why are we visualizing these elements? Why are we looking into dis-aggregated results of discipline infractions across all student groups? Do we potentially have a student group that is being over disciplined versus another?” Those are really hard questions to answer at times. And it really requires a lot of introspection and for the organization to know the why that it’s not just punitive or evaluative it is really just an opportunity for the district to grow, and support all students fundamentally. But on the surface, it couldn’t look like, if you don’t set that table for the why you could fall into the place of what “We’re just trying to evaluate somebody out of a position or a program. We’re trying to find an excuse to eliminate a program.” That’s really, the path, it’s a path of improvement and opportunity to say, “Yes, we are getting a return on investment in this program. And here are the results to show you that yes, we are supporting all students, not just some. And here are the results to support that, so that all students are getting an equitable education, they’re getting a rigorous education, and getting opportunities. Not just some students.” So really it is an opportunity to measure the important things for district and keep everybody accountable, in line, in lock-step the goals of the board of education, of the district, and then ultimately supporting the students.

 

RYAN ESTES: What kinds of data do you see being most helpful for districts who are trying to optimize or improve how they serve their students through their teachers and staff?

 

DR. ADAM CIBULKA: Hiring and credentialing staff, I think, is going to be a a huge future visualization and a data point for us. We know in terms of the equity conversations to diversify the workforce to make sure that students have role models from all backgrounds in the classroom is going to be huge. So we are already seeing districts. Looking at, “What are the demographics of my staff and does that mirror, that of my student population?” And I see that as a, as a trend moving forward. Again with the teacher shortage, with the substitute shortage analytics around, “are we able to fill the classrooms with qualified staff members, and are we creating a pipeline really for, for teachers is, you know, a huge way of the future, you know in terms of we’re always getting interest around salary information, of course, I mentioned demographics, credentials absence, and knowing again, “How do we support staff in their growth to make sure that their credentials aligned with what the trends are that we see in education?” You know, a lot of desire to make sure that we have bilingual staff, a lot of desire to make sure that we have students that are getting opportunities in workplace experience or CTE credentials is one interesting piece around social emotional learning is that, “Are we creating a sense of belonging for our, our students?” And it starts with the staff. And one way that we have seen this most recently is districts really making sure that they have enough coaches and clubs sponsors on staff, because we know that extracurriculars are a great way for a student to feel that sense of belonging. And when students feel like they belong and have a trusted adult in their life, they’re more likely to be successful in the classroom and continue on a path to graduation and post-secondary success.

 

RYAN ESTES: Last question here for you. I would like you to look into the crystal ball, make a prediction. What do you think the future holds around data and analytics? I mean, 10 years from now, what kinds of data will schools be collecting and using it to make decisions in ways that they aren’t at the moment?

 

DR. ADAM CIBULKA: I think the biggest thing is getting the information quickly to the, to the district leaders. Things like automated insights where you know, no longer does the district have to log in or look at a dashboard or anything like that, the dashboard is brought to them. So it’s almost that data on demand of, if you have a key question, you can type in that key question.

 

It automatically will send you the results based on all the data sets that you have. I know that that is a way that is already being worked on on our end to try to bring that data on demand to the district. Being able to predict. Some predictive analytics around student success at as early as we can do that.

 

You know, whether it’s uh, in third grade, can you predict high school success? Those types of things. That’s really, as we begin to get these robust data sets, being able to predict and give some trends on where students are going really early on, I think is where the future of analytics lies.

 

So you can kind of boil things down to a single data point to whether or not a student needs support. And from the staff, I shouldn’t say from going back to prior question of, also the staff element, I do see, trying to create a pipeline of educators as a huge need in terms of “Where do we need to hire most? And how do we get ahead of this?” The best that we can to create a diverse workforce and how the diverse teaching staff within our buildings is going to be huge. Typically districts do not have an insight into that until after the fact. And they’re always looking for new staff members still, to provide maybe more real-time information to who’s in that pipeline. I see on the staffing front is going to be huge.

 

Dr. Adam Cibulka joins us from Forecast5 Analytics, which provides advanced yet easy-to-use analytics software for the public sector, including K12 education. Forecast5 is now part of Frontline Education.

And Field Trip is a podcast from Frontline Education, the leading provider of school administrative software. For more information about Frontline’s solutions to help schools with human capital management, business operations, and student management, visit FrontlineEducation.com/FieldTripPodcast. And, of course, don’t forget to subscribe to Field Trip while you’re at it. You can find us on…well, you know where to find us: anywhere you get your podcasts.

For Frontline Education, I’m Ryan Estes. Thanks for listening, and have a great day.