Coaches model and support the use of qualitative and quantitative data to inform their own instruction and professional learning.
After joining ETM, I learned just how important this is and how little awareness there is in EdTech as it relates to choosing technologies that hold student data and making sure they’re secure. I like this topic because there are a lot of directions and perspectives held no this particular standard. I for own, hold primarily, a systems administrator view.
Below you will find evidence, spread across three different subsections (indicators) of how I believe I am a Data-Driven Decision Maker where I work.
Return to Portfolio Home or move next to Digital Citizen Advocate.
Assist educators and leaders in securely collecting and analyzing student data.
When I worked in ETM, I worked closely with a colleague on the Business Systems Team, who was in the IT Project Management Role. Normally, I don’t think an Associate Director of EdTech and an IT Project Manager would talk so much, but because we had worked together previously, we shared a lot of the same perspectives and had a very candid way of talking about issues with each other. There was a tacit level of trust in each other that allowed us to work well together.
The reason I bring this up here is because he’s the person who would partner with me most when discussing issues like connecting third-party systems into our Learning Management System. He was the person I bounced ideas, questions or concerns the most, and he’d give me his “two-cents” each time. This made evaluating systems and technologies that faculty would bring to ETM to connect to the LMS easier, because I had a second pair of eyes who could read a contract phrase and help me develop a better understanding of what was happening.
As an administrator, I was responsible for choosing systems that were not only secure, but also supported teaching & learning. During my tenure in ETM, I helped with the transition from one LMS to another, evaluated and selected a new Lecture Capture and Video Management platform, and more.
While I had a duty to evaluate systems, it was also important for me to share that knowledge and experience with the faculty we worked with – so we published guidelines on how to choose technologies for teaching and learning on the ETM website. It also led to my supervisor and I having a “fireside chat” about technology on our blog.
In a post I made about Learning Analytics I talk about some of my concerns when it comes to the analyzation and framing of the data collected. When I sat in meetings with our Business Intelligence staff, I remember him saying that you have to know what story you want to tell with your data. This could sound manipulative, but it’s true and that means as educators bias can easily influence how we interact and interpret the data provided, so I think this is a honestly a critical area that needs more attention and research as more information because easily accessible in systems.
Support educators to interpret qualitative and quantitative data to inform their decisions and support individual student learning.
As I noted on the 6A indicator, I have concerns about the interpretation and bias that can come with making so much data available to instructors and administrators. See Investigating Microsoft Insides and Learning Analytics for more details.
At the beginning of the pandemic, one of my children’s school asked whether or not I consented to having them be filmed during class. Since I’m a systems administrator, naturally, I wrote back and asked a slew of questions like:
- Where are the recordings stored?
- Who will have access to the recordings?
- When will the recordings be shared or viewable by others?
- How long will the recordings be stored?
In the end, I didn’t get satisfactory answers so I didn’t provide consent. The reasons why is because I was concerned about bias across the school if say my child does something strange on Zoom, and someone who doesn’t know my child sees that video – what perception would they have and why are they evening accessing that video?
I use this story to help illustrate how important it is for all parties to know what’s happening with the data collected about them. In Empowering Instructors and Students: Learning Data, I talked about five steps that administrators should do to protect learning data:
- Raise awareness and be transparent – remind users all the time and make policies and practices published and easy to access
- Review existing tools and procedures – just because a system was selected, doesn’t mean there aren’t concerns, staff should come back and evaluate regularly.
- Be proactive and responsive – administrators should plan in advance as much as possible (documentation, FAQ’s, canned responses) and respond quickly when there are known issues
- Develop Relationships – with instructors, other staff, administrators, etc. Everyone needs to be aware and sometimes, people hear information differently when it’s from a friend as opposed to an administrator.
- Ask Questions – develop a process that evaluates and gets to the heart of some of these security issues and be consistent.
Partner with educators to empower students to use learning data to set their own goals and measure their progress.
I think one of the challenges with this indicator and some of the technologies that I’ve seen is that students are limited in what they can see. For example, Canvas Analytics for the longest time, was only available to instructors. The same is happening with Microsoft Teams Learning Analytics. Hopefully, one day this information will be more broadly available.
Now that Canvas has Analytics available for students, there’s an awareness issue – that students may not know this exists and may not know how to interpret the information.
During the pandemic, various departments partnered together to create a Student Essentials for Remote Learning course that was available to all students. A course like this would be the perfect place for information on learning analytics and how not to get discouraged or anxious when reading through the information. While there is a lot of good information, such as course grade, weekly online activity and communication, there needs to be some additional framing about what students are seeing and how they can respond. For example, the grade is only going to be based on things that have been graded so far in the course, for online activity – that likely changes based on how the course has been setup and the same could be said about communication. While you can see the data, if you’re not careful about “correlation may not equal causation” you can end up drawing incorrect conclusions and spending time on areas that aren’t productive.
As an administrator who then creates documentation, it’s not enough for me to just tell people that this information exists, but to also explain how the course is designed to work – including grading policies and practices, communication methods, and instructor recommendations for how to be successful in the course. This will then allow students to look at the data and make balanced decisions on how to interact with their course content.
Return to ISTE Standards.
