Talk:TRC Gimmick Rallye Evaluation Cards
I don't want to "mess up" the draft, but am thinking we could separate (into separate sections)
* scoring, and awards presentation efficient? (moved close to end) * material clear and legible? * registration and CP workers competent?
- Dean
The rallyemaster doesn't have much control over registration, so we shouldn't count that as part of the "best rallye" awards (although it may still be useful feedback). It would be good to ask about the beginners school too, although we shouldn't expect upper division rallyists to answer that question. And we shouldn't count it when someone other than the rallyemaster gives the class.
I like the idea of putting the questions in rough chronological order (i.e., moving scoring and awards presentation towards the end).
And as we mentioned last night, very few people can offer meaningful feedback on whether protests were handled fairly. Perhaps a better indicator would be how often the protest committee overrules the rallyemaster.
- Darin McGrew 10:51, 14 November 2007 (PST)
I did many of the tasks that we had been talking about. See what you think.
- Dean
I recommend leaving the existing form alone, except for the changes to the "Length". 10 Categories is sufficient for the feedback that we need.
I am not sure if many rallyemasters use the form as it exists today as a vehicle to improve or change rallies. As a consequence, making changes to the form is likely not going to do much. If we add more categories, rallyists may be less likely to fill out the form.
A better use of time might be for rallyemasters to use the existing form to create a post-mortem of their event that includes 2 topics:
a) What worked well b) What can be improved upon.
Such a post-mortem should be optional, but would be nice to include in the TRC minutes (if it is done).
What to Cut?
Okay, we've got 13 sections now. What do we cut/consolidate to get back to 10 (or fewer)?
Darin McGrew 18:20, 21 November 2007 (PST)
I think 13 sections and 15 easy-to-answer questions might not be too much, if it fits nicely on a half page.
- Dean
Using Incomplete Evaluations
I also propose that each question get scored by those who answer that question, so partial evaluations are used.
- Dean
If we include topics that not everyone will be able to answer, then we'll need to do something reasonable with "incomplete" evaluations. So, how should we handle the math for a rallye's total score while counting incomplete evaluations?
It's easy enough to average the responses we get in each category, and it's okay if some categories have more data points than others. But do we calculate the total score by adding the average scores for each category? I can't think of a better way, but this still seems clunky to me.
Darin McGrew 10:50, 28 November 2007 (PST)
Making Evaluations Useful
Personally, I don't find the evaluation forms useful, except for the comments written on them (in the margins, or on the back). The numbers are purely arbitrary, and apparently even our worst rallyes are well above average (the Lake Wobegon effect), since they score above a 5 in all categories.
Maybe it would be more useful to have a number of statements (e.g., "Traffic was not a problem.") and ask whether they agree strongly, agree somewhat, neither agree nor disagree, disagree somewhat, or disagree strongly? Or maybe we could use a 7-point scale: agree strongly, agree, agree somewhat, etc.
- Darin McGrew 10:57, 28 November 2007 (PST)
I think this might be the best approach of all. It has the benefit of simplifying the questions, too. One of the complaints I've heard about the present eval card is that the questions cover slightly disjoint subjects, so the competitor has to average out things in their head.
- Steve 12:24, 2 December 2007 (PST)
Okay, I took a stab at creating a new evaluation card that uses the agree/disagree format. What do you think?
- Darin McGrew 13:26, 3 December 2007 (PST)