Teammates: Student: view results: rank questions: show average

[damithc]

Scenario: Student views results of a numerical scale rank question

Current: Can see individual ratings received
Suggested: Show average rating as well. Can show both average with and without self rating (if there is a self-rating involved).

[nidhi98gupta]

@damithc I would like to work on this.

[damithc]

Go ahead @nidhi98gupta

[nidhi98gupta]

@damithc Sir, A little clarification is needed.

Screenshot above is of studentFeedbackResultsPage. Aren't we already showing average with and without self rating in numerical scale question?

[damithc]

@nidhi98gupta this issue was created based on a user request. Most likely the user was referring to ranking type questions instead. Can you check if ranking type questions are showing stats to students?

[nidhi98gupta]

@damithc Rank (recipients) question doesn't show stats to students.

Is this what you're referring to?

[damithc]

@damithc Rank (recipients) question doesn't show stats to students.

Thanks for checking @nidhi98gupta
I've updated the issue title/description

[nidhi98gupta]

@damithc Although the issue description says that average should be shown, but since same ranks can be given, I feel average rank is somewhat deceptive. Moreover we show normalized overall rank to instructor in results page.

Now to show overall rank we need all the responses of team members, so we will have to restrict visibility options in Rank Recipients Question to Show to recipient and team members (either anonymous or visible) like in Team Contribution Question so as to make sure we always get all responses of that question in the responseBundle.

If you feel average rank is better option, then no need to make extra changes as stated above.

[damithc]

That's true. How about this? If the visibility setting is permissive enough (i.e., if the ranks for all team members are in the results bundle), we show overall rank. Otherwise, we show nothing. @joanneong @wenmogu any inputs? I remember you worked on the overall rank feature.

[wenmogu]

Hi @nidhi98gupta thx for raising the issue and offering us ur thoughts. They are indeed valuable!
However I think we need to discuss the definition of overall ranking and what the user really wants...

Now to show overall rank we need all the responses of team members

I dont really get why we need all the responses of the team members... currently on the instructor side, even if not all students have submitted their responses, there is still overall ranks available as it is computed from the existing input data at the point of time when the instructor views the result... maybe we have different understandings of overall ranks? In my view, overall rank is the recipient's current standing at the point of viewing the result, instead of the rank computed using all results...

Based on the issue description, my understanding of the user's request is that, the student wants to know his/her standing at the point of viewing, given the existing student inputs (as u may know, the instructor can choose to publicize the responses before the session closes). I will illustrate my point with the following example:

Suppose C is the student who views the result, and at the time he views the result, A, B, C receives ranks (1, 2) (2, 3) (3, 4) respectively (Some jokers might not want to give ranks to all ppl lets assume). As such, the overall ranking for A, B, C at the point of viewing will be 1, 2, 3 respectively for A, B, C.
If C is limited to see his own ranking i.e. he cannt see what A or B receives, then C would want to know his overall ranking i.e. 3 even tho he cannt see others' overall rankings.

To summarize everything (err the explanation a bit too long i guess), the user wants to know his/her own standing which is in sync with the overall ranking shown on the instructor side, at the point of him viewing the response.

I think we might need some other opinions here. What do u think about this @damithc @joanneong

[joanneong]

Thanks @nidhi98gupta for looking into this issue :)

It seems like there are two different ideas being floated now: (i) displaying average for rank questions, and (ii) displaying overall ranking for rank questions.

Personally, I feel that showing overall ranking makes more sense than average ranking because (i) overall ranking is harder to calculate (students can just calculate their average ranks themselves if they want to but overall ranking is more complicated) and (ii) displaying overall ranking provides new information on how the student compares to other students. Despite this, I would recommend either showing just the "average ranking" or not showing anything additional at all for rank questions.

Here's why:

Currently, overall ranking is calculated by comparing the average of the ranks received by students. So, reusing @wenmogu 's example above:

Average ranking --> Overall ranking
A: (1 + 2) / 2 = 1.5 --> 1
B: (2 + 3) / 2 = 2.5 --> 2
C: (3 + 4) / 2 = 3.5 --> 3

This means that when calculating the overall ranking for a student, we have to utilise the responses for all students, and not just a single student as the students are being compared here (@wenmogu , I believe you may have misunderstood @nidhi98gupta here).

Thus, the issue at hand is whether there is a need to restrict the visibility settings for rank questions to "Show to recipient and team members" if we implement overall ranking so that each student can have access to how other students did the ranking when they view their own results.

~However, I feel that changing the visibility setting in this case will still not enable us to display an overall ranking. This is because to know the overall ranking, a student has to have knowledge of the rankings received by all other students as well - something which not even the most 'lax' option "Visible to recipients and instructor" can grant. Instead, from a student's point of view, what "Visible to recipients and instructor" does is that the student can now see who gave what ranking to him/her, but is still unable to see what rankings were given to other students in his/her team.~

~Hence, if we are to do an overall ranking, it might require a radical relook at what students can see on TEAMMATES - they will have to be offered the chance to see the responses given to other students as well, in the way instructors can. This makes me feel uneasy because it sounds like something that most instructors would not want.~

Alternatively, we can just leave the visibility setting as-is now, and do all the calculations for overall ranking behind the scenes. Then, we can just treat overall ranking as another statistic, and display it whenever responses are visible to students (anonymous or otherwise). However, this means that we need to keep track internally of all the information for students in the same team, and retrieve such team information whenever students want to view their individual results. To me, this sounds like a lot of unnecessary work to produce a single statistic.

So yes, I would recommend either showing just the "average ranking" or not showing anything additional at all for rank questions.

[nidhi98gupta]

@joanneong I think you missed one of the visibility options that can achieve this Visible to recipient and team members and instructor?
If we look at the Team Contribution Question, we need all responses to calculate perceived contribution and actual contribution which we show to the student. I believe we achieve this by setting the one of the two visibility options to Show to recipient and team members which enables responseBundle to have all the responses?
Can refer to this function in which we add team members' responses for Team Contribution question.
https://github.com/TEAMMATES/teammates/blob/3c0d42b34561108d4427ae7416207ea675c6bb3d/src/main/java/teammates/logic/core/FeedbackResponsesLogic.java#L817-L855

[joanneong]

@nidhi98gupta You're right; I did miss out that particular option :)

That said, I believe that adding the overall ranking is still not as simple as what you mentioned above. For example, for team contribution questions, there are a number of additional methods to retrieve team results in FeedbackContributionQuestionDetails.java and even additional classes like TeamEvalResult.java.

But I guess it can be done :) A few additional concerns if you are going to do this - first, it would probably be good to update how the responses are being displayed on the students' end as well. For example, comparing team contribution questions and rank (recipients) questions we see:

Team contribution questions:

Rank (recipients) questions:

Another thing to think about would be whether to extend this overall ranking for rank (options) questions as well. I'd say yes, since they are both rank questions and overall ranking make sense for both.

The last concern I have (for now) is regarding backwards compatibility - what will happen for instructors who have already made a form with visibility options "Visible to recipient, invisible to instructor" etc?

[nidhi98gupta]

@joanneong Addressing the valid concerns you have :)

1. Responses can be displayed on the students' end in the same way we do it in instructors' end.
   

1. I have few questions regarding whether to extend this overall ranking for rank (options) questions as well. This is how we show statistics to instructor.
   
   Suppose for this particular question is it really advantageous for student to know which areas does his peers want to improve upon?
   On the other hand if it is peer ranking in rank options then we can show in which area overall student should improve according to his teammates.

2. Regarding backward compatibility, as @damithc Sir said previously if the visibility setting is permissive enough (i.e., if the ranks for all team members are in the results bundle), we show overall rank. Otherwise, we show nothing.