Is a survey good enough?

K.Park/ January 23, 2022/ 1 comments

Question

What resources can educators/administrators use to design professional development that responds to their institution’s needs, when surveys don’t collect enough data or aren’t reliable enough to provide the necessary data? 

Introduction

One of the challenges that I’ve had in my current position has been identifying the needs of the faculty I work with to create meaningful professional development opportunities.  Surveys, while fantastic for data gathering and analysis, provide a picture, but sometimes I don’t want to use a survey because for me, in practice, doesn’t yield the information I need. I also feel like I “jump through more hoops” to run a survey as opposed the phone calls or emails that I need to respond to each day.  

There’s also a piece of the puzzle where faculty may already know what they don’t know and have found resources for what they need without talking to my office.  According to Bouwma-Gerhart in 2012, concludes that “many faculty do work to make relevant improvements and can identify their own teaching and learning problems and direct themselves towards the information they need.” They go on to write that faculty change happens at the individual level and that in turn can drive department, unit, or institution-based change with respect to teaching & learning. 

I’ve been mulling over what this means in the workplace and whether my current practices and habits recognize this and what I realized is that professional development happens both at a macro and micro level, and each interaction can become a key part of both identifying issues, but also instigating change.

The Full(er) Picture

While surveys are the go-to tool for many EdTech administrators (including myself and my office) due to their ability to collect information from a large population through a variety of methods, there are times when I think surveys don’t deliver the full picture of what’s happening on campus. Sometimes that’s due to poor question or survey design, but there are also other factors, such as low response rates, respondents not being truly reflective of the larger population, survey fatigue – too many surveys to take, so some people just click through without providing honest or accurate answers.

Below are some of the ideas I had to help expand on existing practices to gather information on what faculty need for professional development.

“Water Cooler” Conversations

While there are different beliefs and attitudes towards small talk in the workplace, overall, I personally believe they are important and beneficial.  In the Harvard Business Review article Remote Workers Need Small Talk, too, the authors write: “The tidbits we learn about our colleagues — for instance, that they play guitar or love dogs — build rapport and deepen trust. Research even suggests that chance encounters and spontaneous conversations with our coworkers can spark collaboration, improving our creativity, innovation, and performance.” Not only do these conversations take place in the office I work at, but they also happen while I’m getting lunch, when I’m go to faculty gatherings or while I’m walking across campus from one meeting to the next.  Just by being out and about, I learn more about the individual person and their needs and sometimes, that’s where inspiration hits.

Faculty Collaboration

Whether through governance or other types of university supported committees, create opportunities for faculty at all stages of their careers to participate and provide feedback on professional development needs.  I think it’s important that participation on these committees also be recognized, particularly if there is robust faculty governance so service to the university can be used for promotion or other incentives that can be used to recognize service are provided.

Ticket Tracking Systems

Often utilized by call centers or IT offices, these allow those teams to track issues, ensure responses, engage with customers, and can be used to identify systemic issues. Of course, I’ve gone and simplified what the tracking system can do, because I’m focused mostly on how it would benefit identifying faculty needs.  When utilizing a system like this, I can maintain a record of requests and I can see in my reports, why types of questions or issues come to our desk the most often.  If I combined this information with information our IT office had, I would have an even bigger picture of what faculty were doing in the classrooms, such as software they’re using, issues they’re having with the room technologies or design and more.  When the support for teaching & learning are split across multiple departments, it becomes even more important for these departments to collaborate and share information like this, so they can all work together to support faculty.

Website Analytics/System Analytics

Most systems these days maintain some level of analytics.  They may not always be easily accessed, but I can guarantee, the data is there.  Often, you can find out information about why types of devices people are using or in the case of website analytics, what pages get hit the most and where to do people drop-off?  Often, these analytics help web designers create a more engaging online experience, but I think educators can do something similar, by gleaning pieces of information that who what devices their faculty use to access systems, what web pages are accessed most frequently, etc.  You can develop a profile for what a typical faculty member might do, with the understanding that there can be a lot of variability and still a need for differentiated professional development.

Informational Interviews

You could also describe these as department meetings, but the purpose is to gather information. Typically, informational interviews are used in the workplace for jobseekers who want to explore career options, industry and/or company culture.  It’s a 15–30-minute conversation where the jobseeker comes away with a clearer picture of a career, industry, or company knowledge.  There can also be other benefits as well, such as increasing your network, identifying steps to pursue a particular career, preparation for interviews and more (indeed, 2021).  I think a similar concept can be applied to working with faculty by setting up “coffee-dates” as my supervisor calls them or showing up to department meetings or others as appropriate, to collect information and learn more about what faculty are doing and experiencing when they’re teaching.

Going Forward

Some areas that I’m not familiar with but would like to explore more in the future are both predictive analytics as well as how to better incorporate and expose faculty to new and emerging concepts and technologies.  One of the things my office has tired is a emerging technology lab, so faculty would have an opportunity to try out new technology without spending their own money. Unfortunately, the pandemic waylaid the launch, but it’s something I want to come back to and explore further.

I think surveys will always be a part of the data gathering process to find out what faculty need, but what I would say is that there should also be other methods included in the data gathering process to have the best understanding of who your faculty are and what they need.

Research

Bouwma-Gearhart, J. 2012. International Journal of Teaching and Learning in Higher Education. Science Faculty Improving Teaching Practice: Identifying needs and finding meaningful professional development. Volume 24. Number 2. pp 180-188.   https://files.eric.ed.gov/fulltext/EJ996264.pdf.  

Indeed editorial team. 2021. Career guide. How to get the most benefit from an informational interview. https://www.indeed.com/career-advice/interviewing/what-to-expect-in-informational-interview

Methot, J.R., Gabriel, A.S., Downes, P., and Rosado-Solomon, E. March, 2021. Harvard Business Review. Remote Workers Need Small Talk, Too. https://hbr.org/2021/03/remote-workers-need-small-talk-too