#FailForward: The Pilot

An entire chapter of the book Go Blended by Liz Arney is dedicated to running successful pilots of blended learning initiatives. For those looking for an in-depth explanation of a successful pilot, order her book (seriously, it’s fantastic). This post will outline a few of the lessons I learned the hard way from our most recent pilot.

At ReNEW, we run a year-long schedule in an attempt to prevent the ‘summer slump’ for our students. When October rolls around, we take a two-week long ‘break’ called Intersession.  This 10-day period is an opportunity for a small group of our students to receive remediation – to fill a few gaps and help catch them up before the next quarter starts. Due to the personalized nature of intersession, blended learning software is used to meet students on their level. We decided to pilot an adaptive math program to compare its effectiveness with a program currently in use on many of our campuses. Specifically, we wanted to see how a truly ‘adaptive’ program would affect student achievement, and to determine if the program would be worth piloting on a larger scale later on in the year.

In attempting the pilot, I learned a number of lessons the hard way; many of them are quite obvious, and all of them are covered in depth in Liz’s book.

1. Plan Ahead
I have all sorts of excuses for this one, but ultimately it became clear that I needed to find a way to set-up everything as far in advance as possible. We didn’t know who would be on the rosters until just days before Intersession started, which put us in a holding pattern. In retrospect, I could have pushed for a full SIS integration weeks in advance, and followed-up with the unique intersession classes once we had rosters in hand. Instead, when it came time to load the students, the SIS integration took a shockingly long time – leaving teachers and students hanging without access to the program they had been promised for the first three days of Intersession.

2. Vendors Make Mistakes
Despite my egregious frustrations from our set-up this fall – I had no idea vendors could be so difficult – I trusted our new friends when they told me we could be up and running in the short timeframe we needed. I even trusted the automated email on Sunday night alerting me to a successful student data upload, only to find out it unexpectedly that rosters were not working Monday morning. This is a consistent problem with edtech companies. So what seems to be happening? For one, the sales and support departments never seem to be on the same page. As our data guys continue to remind me, I need to start talking more directly with the folks that make the software work – not the ones who are selling me on it. We could have alleviated a number of problems along the way with more streamlined and accurate communication from the vendor.

3. Communication Matters
To build on that second point, accurate, timely communication is critical to the success of a pilot (and most of what we do!). School leaders need to communicate their intentions – curriculum needs, student rosters, etc. – ahead of time. Vendors need to communicate the actual over the theoretical. Most importantly, I have come to learn how important it is for me to communicate more often and with greater clarity with everybody: leaders need deadlines and updates; teachers need to know what to expect when; vendors need to know our needs.

4. Clarify Expectations
Liz does a wonderful job of explaining the how and why of this. How we communicate the purpose, desired outcomes, and strategy behind a pilot is as important as the pilot itself (sort of). Mis-communicating can cause a good pilot to go bad – quickly. Controlling the expectations of all involved will allow your pilot to give you the information you need to make decisions moving forward. I’ve learned this lesson best from the vendors themselves, actually – nearly every single one has promised something – a timeframe, SIS integration capabilities, program features, etc. – that was unrealistic and ultimately fell short; instead of planning appropriately and communicating early with teachers, we get frustrated as our expectations aren’t met. This disregard for the experience of the teacher forces me to rethink the usefulness of the product. From this experience, I am learning to push vendors to provide more accuracy and clarity, in hopes that I can do so with our leaders and teachers.

If I take one thing away from my experience with vendors this year, it’s to expect delays and false promises. Every yes comes with an asterisk – “yes, we can integrate with Clever” could have any number of caveats, from “if you pay for it,” to “if you create extremely specific structures that mimic our system (and don’t work with yours),” to “if you wait 6 months!”
 

Our pilot will conclude in the coming days. Hopefully, we’ll learn about the impact of the product itself on student learning – and not more about how not to pilot.

Fail forward!

Hitting Reset

Photo by Flickr user Redux

When I took on my new tech role back in June, I approached it with a focus on teachers. In my years in the classroom (the only job I’ve known is teaching), I came to believe that teachers not only were the leverage points in schools but also that they deserved better. By that I mean that teachers are too often questioned, unsupported, reigned-in, and dismissed by those in leadership positions.  So when I began this work, I committed myself to the teachers – in hopes of winning them over with constant support, and providing them with the resources and guidance for successful tech implementation. If change management requires an initial focus on the innovators and early-adopters of an organization, then I must be doing something right. Right?

Well, the irony I’ve come to see (with generous advice from folks internally and externally – including Liz Arney and Shawn Rubin) is that indeed it is the ones in leadership positions who allow for the innovators to flourish or fail. Simply put, leaders are the leverage point; they have the ability to support or stifle innovation in their classrooms. A few reasons why I’m stumbling into this realization…

1. Leaders have the ultimate say – Under the charter umbrella, our schools are given a great deal of autonomy.  Therefore, leaders are given the power to make decisions – nearly all decisions – as to what is best for their school. If I’m hoping for a teacher to pilot a station-rotation model, it will require the school leaders support – not just approval. The leader needs to see the value in the program, because inevitably things will go wrong with technology (internet outages, new management requirements, furniture rearrangement or acquisition, schedule restructuring, or any number of other issues that will require building-level support). If a school leader’s vision doesn’t include blended learning, then scaling the pilot becomes practically impossible, as well; one teacher’s innovation will simply fall flat without the active support and advocacy for innovation. On the contrary, a leader who is committed is not only going to support that pilot but actively spread it to others if it’s successful.

2.  Leaders have greater reach – I met with an assistant principal recently who wanted to identify next steps in supporting his teachers use of a blended program. As he explained his system, whereby each teacher was already checking the data biweekly, and using it to intervene with students, I realized how much more powerful an engaged school leader is than myself. Had I attempted to push his teachers to do the exact same thing, I would have fallen short – I don’t have the same rapport with his staff, I don’t see their classes daily, I don’t know their students personally, and I don’t have the power to hold them accountable and make demands if necessary. Leaders are in the building all day every day; they’re able to check-in with the teachers in their building on an as-needed (at least daily) basis; they’re able to push teachers to follow through on the initiatives that they’ve outlined for their building. I can’t compete with that; But I can support it. If I can support other leaders to make similar commitments to consistent data analysis and action, my impact will be exponentially greater. Leaders are the leverage point.

All of this said, I haven’t given up on the teachers. To the contrary, I’ve doubled-down on my efforts around the teacher-leader program I’ve envisioned from the beginning. The INSPIRES Fellowship will be an avenue to leverage rockstar teachers within each school, not only to support blended learning in their own buildings but also – I hope – to spread innovation and best practices across the network. This program is, in Shawn Rubin’s words, “a building leader play.” By that, he means that the program’s success will hinge on leaders’ valuing and support of their teachers and the program; with leader support, we will be able to commit ourselves to supporting innovation across schools.

So I’ve hit reset. I’m going to restructure my time to reflect this shift, in hopes of making a greater impact with my work. This means monthly leader check-ins to evaluate progress on specific blended goals, along with constant walk-throughs with leaders to assess needs. Ultimately, my job is to promote and support personalized learning for all of our students. To do so, the work will need to support each school leader’s vision and goals for their students, meeting the instructional needs unique to their program. But who knows? Might need to hit reset again soon.