Introduction

For high stakes tests or exams, in particular in higher education, teachers want to analyse the quality of their questions. There are established methods for this, looking at the difficulty and discrimination of each question based on the student answers. This is not something that we have had in itslearning before, and we are happy to announce that such an analysis is now available. The option is found in the Results tab, close to the summary information for the test. Simply press "Export question analasys". This triggers the calculation and subsequent download of an excel (.xsls) file. The analysis is performed on all counting attempts that are fully assessed (max one per student). If a student later submits a new attempt that becomes the counting attempt, simply click the button again to trigger a new analysis. 

The excel spreadsheet will have a header row, followed by a row with a short descriptions of what each column contains. There is one row for each question in the test.

Further column classification:

• Explanations for each column

  Column	Possible values	Explanation
  Column	Possible values	Explanation
  Question number	Integers	Question number as seen in the question list in the test by the teacher.
  Question text	Up to 100 characters of text	Question text, cut at 100 characters if long. HTML is stripped. For Fill in the blank or Select from a list, this is the introduction text, not the question.
  Question type	Text	Name of the question type (currently 7 available)
  Assessed and counting answers	Integers	Number of times the questions has been used in a fully assessed, counting attempt.
  Max possible score	Positive number, 2 decimals	Question score as set by the teacher when creating the question. Usually an integer, but decimal values are allowed.
  Average score	Positive number, 2 decimals	Average score achieved in the attempts included in the analysis.
  Standard deviation	Positive number, 2 decimals	Standard deviation of the scores achieved in the attempts included in the analysis. Calculated as standard deviation for a sample population.
  Variance	Positive number, 2 decimals	Variance of the scores achieved in the attempts included in the analysis. Calculated as variance for a sample population.
  Answers with max score (%)	Positive number, 2 decimals	Percentage of attempts with score = max score, based on the number of attempts included in the analysis.
  Difficulty	Hard, medium or easy	The difficulty is based on the percentage of attempts which achieved the max score, and categorised as follows: > 80% = Easy (meaning more than 80% of students got this correct) 30-80% = Medium < 30% = Hard (meaning less than 30% of students got this correct)
  Discrimination score	Positive number between 0 and 1, 2 decimals, or empty.	The discrimination score is calculated as Pearson's correlation coefficiant for a sample population. The coefficient cannot be calculated in cases where all attempts have the same score. The formula for this is Where
  	Good, fair, poor or cannot calculate	The discrimination is based on the discrimination score, and categorised as follows: > 0.3 = Good 0.1 - 0.3 = Fair < 0.1 = Poor
  Review recommended	Yes or no	Based on the difficulty and discrimination values, we will recommend review of the question if at least one of the following conditions is met: Difficulty = Easy Difficulty = Hard Discrimination = Poor