We have recently began discussing the possibility of revising our course evaluations instrument. To begin brainstorming, I was wondering if anyone has gone through this process recently and if they could share a little bit about their experience including:
Our core survey instruments is by campus. Two of the campuses, the instrument is owned by the faculty senate, the third is embedded in the faculty handbook. When the senates made changes, they did so through their process and submitted revisions to me.
However, there has been recent traction at the University level to rethink the evaluation process from the instrument to the culture surrounding evaluations, so this will be interesting to follow. As we make progress or lack there of, I can keep you informed, just send me an email. But I believe for us, it is going to be a multi-year task from buy-in, redesign and implementation.
However you redesign, I would definitely involve faculty as they are the ones that will make or break the process. The questions will depend on them. What do they need to know? What does the university need to know? Also involve your CTLE if you have one. What I have been silently pushing is to have a core university question or two, then each campus can choose one or two questions that is important to them (each of our campuses are very unique), and then allow Colleges/Departments a question or two that is important, but leave the bulk of the questions to the faculty as if they have ownership then response rates will be better and the results will actually be used. As what all this looks like, I will let you know!
At Wichita State University, we also incorporated the faculty in the process. The Faculty Senate created a course evaluation task force to be the driving force for our change over from an in-house evaluation to Watermark CES. I am part of this task force, and I guide them on what needs to be accomplished. They looked into what other universities used as questions, used a book by Raoul Arreola called Developing a Comprehensive Faculty Evaluation System, and we surveyed our faculty on what they liked/not liked about the in-house evaluation questions. From there we created a pilot evaluation. We cut down our evaluation by a lot. We went from 49 questions to 5 and without demographic questions. During the pilot, we then asked the students how they felt about the evaluation along with asking if there were questions we should have asked. Our intention is to have the Main Course Evaluation with these 5 primary questions and two open ended questions. We would then have an Item Bank that will include the questions from the feedback we received as well as allowing the instructors to request to add questions to the item bank. We are hoping to start the process of the Item Bank next semester starting with figuring out the policy and procedures as we do not want faculty to go overboard on the questions. We wanted to use the Main Course Evaluation for at least a year, before introducing an other options. I have everything come through the Faculty Senate Task Force, so that faculty have a stake in the process and that they cannot come back to me with complaints and wanting an individualized process. It is so nice to say.... This is the process/policy approved by the Faculty Senate.
After going through training to learn CES, we started the task force. In our timeline below, some of it was because of learning the product as well.
FA22 pilot - the evaluation used was with some of the questions from the in-house evaluation and with only a handful of courses. Task Force committee began this semester, and started to create our Main Course Evaluation Questions. SP23 pilot - the evaluation used was with our new main questions along with extra evaluation questions for student feedback. We only sent this pilot to the 2nd 8 week and semester courses. SU23 - we went live and included all courses for the semester. We continued to ask the student feedback questions. FA23 - FA24 - the evaluation was only our Main Course Evaluation Questions. This was the year that we worked on informing the faculty and students about the process and to work on issues that arose. SP25 - going to start on Department Target Surveys & Building the Item Bank all through Faculty Senate Task Force Committee.
Thank you both for your responses, I really appreciate it as I begin to brainstorm this project. Our current instrument is at its 10th year and it is time to revise it.
I completely agree. As I am seeing from other institutions and your feedback, creating and even just re-evaluating an instrument is quite an undertaking that takes several terms. It includes from building the taskforce, discussing, researching, to piloting. I don’t think that my team is aware of that just yet!
This information really gives a place to start, including how I may need to break down the stages, to determining who needs to be involved. We currently only have a main survey, but from what I am seeing it may be beneficial to add a question bank, as our faculty currently do not have the option to select their own questions.
Once again thank you for your time, and I will definitely be reaching out. I am happy to hear any ideas and progress you may have!
Comments
Christopher,
Our core survey instruments is by campus. Two of the campuses, the instrument is owned by the faculty senate, the third is embedded in the faculty handbook. When the senates made changes, they did so through their process and submitted revisions to me.
However, there has been recent traction at the University level to rethink the evaluation process from the instrument to the culture surrounding evaluations, so this will be interesting to follow. As we make progress or lack there of, I can keep you informed, just send me an email. But I believe for us, it is going to be a multi-year task from buy-in, redesign and implementation.
However you redesign, I would definitely involve faculty as they are the ones that will make or break the process. The questions will depend on them. What do they need to know? What does the university need to know? Also involve your CTLE if you have one. What I have been silently pushing is to have a core university question or two, then each campus can choose one or two questions that is important to them (each of our campuses are very unique), and then allow Colleges/Departments a question or two that is important, but leave the bulk of the questions to the faculty as if they have ownership then response rates will be better and the results will actually be used. As what all this looks like, I will let you know!
Kim Brantley
Embry-Riddle Aeronautical University
kimberly.brantley@erau.edu
At Wichita State University, we also incorporated the faculty in the process. The Faculty Senate created a course evaluation task force to be the driving force for our change over from an in-house evaluation to Watermark CES. I am part of this task force, and I guide them on what needs to be accomplished. They looked into what other universities used as questions, used a book by Raoul Arreola called Developing a Comprehensive Faculty Evaluation System, and we surveyed our faculty on what they liked/not liked about the in-house evaluation questions. From there we created a pilot evaluation. We cut down our evaluation by a lot. We went from 49 questions to 5 and without demographic questions. During the pilot, we then asked the students how they felt about the evaluation along with asking if there were questions we should have asked. Our intention is to have the Main Course Evaluation with these 5 primary questions and two open ended questions. We would then have an Item Bank that will include the questions from the feedback we received as well as allowing the instructors to request to add questions to the item bank. We are hoping to start the process of the Item Bank next semester starting with figuring out the policy and procedures as we do not want faculty to go overboard on the questions. We wanted to use the Main Course Evaluation for at least a year, before introducing an other options. I have everything come through the Faculty Senate Task Force, so that faculty have a stake in the process and that they cannot come back to me with complaints and wanting an individualized process. It is so nice to say.... This is the process/policy approved by the Faculty Senate.
After going through training to learn CES, we started the task force. In our timeline below, some of it was because of learning the product as well.
FA22 pilot - the evaluation used was with some of the questions from the in-house evaluation and with only a handful of courses. Task Force committee began this semester, and started to create our Main Course Evaluation Questions.
SP23 pilot - the evaluation used was with our new main questions along with extra evaluation questions for student feedback. We only sent this pilot to the 2nd 8 week and semester courses.
SU23 - we went live and included all courses for the semester. We continued to ask the student feedback questions.
FA23 - FA24 - the evaluation was only our Main Course Evaluation Questions. This was the year that we worked on informing the faculty and students about the process and to work on issues that arose.
SP25 - going to start on Department Target Surveys & Building the Item Bank all through Faculty Senate Task Force Committee.
If you have any questions, you can email me.
Sandra Ranney
sandra.ranney@wichita.edu
Thank you both for your responses, I really appreciate it as I begin to brainstorm this project. Our current instrument is at its 10th year and it is time to revise it.
I completely agree. As I am seeing from other institutions and your feedback, creating and even just re-evaluating an instrument is quite an undertaking that takes several terms. It includes from building the taskforce, discussing, researching, to piloting. I don’t think that my team is aware of that just yet!
This information really gives a place to start, including how I may need to break down the stages, to determining who needs to be involved. We currently only have a main survey, but from what I am seeing it may be beneficial to add a question bank, as our faculty currently do not have the option to select their own questions.
Once again thank you for your time, and I will definitely be reaching out. I am happy to hear any ideas and progress you may have!
Christopher Mendez
Christopher.mendez@artcenter.edu
Please sign in to leave a comment.