Middling Choices

Was this forwarded to you by a friend? Sign up, and get your own copy of the news that matters sent to your inbox every week. Sign up for the On EdTech newsletter. Interested in additional analysis? Try with our 30-day free trial and Upgrade to the On EdTech+ newsletter.

Among the questions I got most frequently when I worked at Gartner were best practice questions. People wanted to know what great projects other colleges and universities were doing that they could study and emulate. It is a question with which I always struggled, not because I didn’t have examples close at hand, but because of the way that best practices are used in EdTech.

Higher Ed’s love of best practices is understandable: the desire to look at what has worked and to adopt those methods is a natural human instinct, and, well, a best practice. Lots of industries do it, and that’s how good ideas spread. But I think there is a problem with EdTech’s use of best practices, and I think as a group we need to rethink how we approach them.

There are a couple of core problems with how EdTech uses best practices - the level at which they are framed and the way that institutions limit their choices when it comes to best practices.

The faulty but popular middle option

There are, roughly speaking, three levels at which you can think about EdTech best practices.

• At the lowest level you have best practice procedures. These are things you probably ought to be doing, for example creating a course shell for every course every semester in an LMS, or having students show a university ID rather than a government issued ID for identification in an online proctoring environment.

• In the middle you have specific projects or initiatives such as implementing an early alert system or starting a hyflex learning initiative. These are bigger than individual procedures but are discrete projects.

• At the highest level you have a set of practices embedded in a broader context which is connected to and supports the best practices they surround. It is difficult to give examples of this as each context is different, but an overall approach to online learning or student success could be examples of such a context and set of practices.

The problem in EdTech is that we tend to choose the middle option. We adopt best practices at the project or initiative level. They are bigger than best practice procedures, but without considering the context in which they sit, they are fraught with problems.

One example would be the ASAP student success initiative out of CUNY which I have written about previously. It is great that the CUNY pilots resulted in a increase in student completion, but the program, especially at scale, relies on many other things being in place. Funding and student advising improvements (which becomes clear if you read the CCRC account of scaling the program). If you focus too narrowly on the best practice and miss that broader context, you stand a very real risk of failing.

As another example, many in higher ed have been inspired by the work that Georgia State University has done in improving student success. Its results are indeed impressive, saving students money and improving grad rates, but again the broader context is missed. The focus is too often on using data and analytics or using this or another type of technology (for example see this story and this story). What is overlooked in this narrow focus is that it took Georgia State sustained attention over a decade and many supporting initiatives, such as short term loans, grants to students, and increasing the number of student advising sessions by a factor of 10 to move the needle the way they did. Buying a piece of technology or partnering with a company will not get the job done.

You can’t just copy and paste

The tendency to adopt best practices at this middle layer of extraction comes in part from the fact that it is bigger than a procedure so seems “worthwhile” but is easier than understanding and adapting something in order to embed it in a broader context.

It would be far more productive for higher ed when thinking about best practices either to focus just on procedures (for example the kinds discussed above with LMSs or online proctoring) or add back in broad context for meaningful learning and adapting.

In EdTech we tend to compound the problem of choosing best practices that are bigger than individual procedures but lack the necessary broader context by limiting our choices with too narrow a set of options.

The same best practices are chosen repeatedly.

Higher ed is a bandwagon driven sector, a fact which we see in many different areas such as LMS choice. Folks look to their peers and tend to follow a similar path. This tendency is pronounced when it comes to best practices, where we rely on a few well-publicized alternatives around which a lot of hype develops. The ASAP example above is a case in point, but just about everything that Arizona State or Central Florida do are similarly hyped and followed.

Which is not to say that any of these projects aren’t really good and shouldn’t be examined. I have sent many folks to learn from the University of Central Florida myself over the years. But we tend to hear a lot about the same examples over and over again. This leads to a lot of uniformity in EdTech and not a lot of innovation. One of the wonderful things about higher education, especially in the US, is its diversity in terms of different types of institutions with different missions and different ways of doing things. Shouldn’t we have a more diverse set of best practices to match this variety of institutions?

We only use best practices from higher ed

Related to the tendency to follow the pack when it comes to best practices is a tendency on the part of higher ed to pull ideas only from other higher ed institutions and not look further afield. I have been around EdTech a long time and had a lot of conversations with folks about best practices and innovation. Seldom do we ever see anyone trying to look outside of higher ed for inspiration.

When you do see higher ed looking outside the sector for best practices it is really refreshing and often very productive. For example, in looking to improve their student experience, American University sought to learn from places like Wegmans and the Cleveland Clinic about how to think about and approach the idea.

But if you adopt best practices from different industries, you can’t just apply them wholesale, you need to understand the broader context and tweak and change them in order for them to work in your own setting. This is a feature, not a bug. This broad view is often difficult for higher ed, but the payoff will be greater and will result in more innovation.

Parting thoughts

These problems in how higher ed chooses which best practices to implement help explain why we see so many failures to extrapolate successful projects even despite the explicit efforts of some organizations such as the Gates Foundation, APLU, and others to foster adoption of some best practices.

Higher ed would get better results and more real innovation if, in choosing which best practices to emulate they try to avoid the mistakes described above and either limit best practices to the procedural level or going broad enough to include context.

The main On EdTech newsletter is free to share in part or in whole. All we ask is attribution.

Thanks for being a subscriber.