Creating a specification

We need to tell our previously partially-configured server about our 6-page 3-question 2-version source PDFs.

We do that through a “test specification”, which we an create and manipulate using the plom-create command:

$ plom-create
usage: plom-create [-h] {newspec,uploadspec,class,make,...} ...

A template specification

The test specification needs to be precise, so rather than making it from scratch each time, Plom will provide you with a template to edit:

$ plom-create newspec
Creating specification file from template: "testSpec.toml"
  * Please edit the template spec "testSpec.toml"
Creating "sourceVersions" directory for your test source PDFs.
  * Please copy your test in as version1.pdf, version2.pdf, etc.

Plom tells you that it has done 2 things:

  • it has created a specification file testSpec.toml for you to edit, and
  • it has created a subdirectory sourceVersions into which you need to place your source PDFs and you have to name them version1.pdf and version2.pdf.

So, after you have copied your source versions into place, take a look at testSpec.toml file. The specification is written in the toml format which is human-readable and easy to edit in your favourite text-editor. The start of the file should look something like

# Example test specification for Plom

# >>> Edit the data below <<<

# Human-readable names of the test - one long, one short.
name = "plomdemo"
longName = "Midterm Demo using Plom"

# Information about the test
# A test may have one or more versions
numberOfVersions = 2
# how many pages
numberOfPages = 6

and then the file continues with information about the number of questions, ID-pages and so on.

Names and numbers

We’ll start editing these to match our test.

  • name: this should be a short name that will be printed on the test. Something simple like mABCmt1 to indicate “Mathematics ABC Midterm 1”.
  • longName: a longer name that will be used to help build a test-return website. Hence we should write something like Midterm 1 for Mathematics ABC 2318. Including the year or term is a good idea to avoid potential confusion.
  • numberOfVersions: in our running example, we have 2 versions of the test.
  • numberOfPages: in our running example, each test is 6 pages.
  • numberToProduce: this is the total number of test PDFs we will produce. Each test we print must be unique, and cannot be reused. Hence this number should always be some more than the total number of students you expect to sit the test. The developers have typically produced something like 10% more the number of students. Remember that you don’t actually have to print all those PDFs, but you can keep them in reserve just in case.

We’re going to assume that we will have around 30 students sitting (15 in each section), so we’ll actually produce 40. One section wants named tests and the other doesn’t: more on this latter. So now the start of our specification looks like

name = "mABCmt1"
longName = "Midterm for Mathematics ABC"
numberOfVersions = 2
numberOfPages = 6
numberToProduce=40

Pages and questions

Now let us turn to the rest of specification – its job is to tell Plom the details of ID-pages, Do-no-mark pages, and our test questions. Take a look at the rest of specification:

# How many questions to mark
numberOfQuestions=3

# the id-page - always taken from Version1
idPage = 1  # must be exactly 1 page

# pages that are not marked (like instructions, formula sheets, examination rules etc) - always taken from Version1
doNotMarkPages = [2, 7] # can be an empty list or omitted

# Now the actual questions (groups of pages).  Each starts with [[question]].
# label = string such as "1", "Q2", or "Exercise 7.3".  Defaults to "Qn" if omitted.
# pages = [4, 5, 6] - contiguous ordered list of positive integers.
# mark = positive integer, the marks for this question.
# select = "shuffle" or "fix" (if omitted, defaults to "shuffle")
# --> "fix" = question will always be drawn from source-version 1.
# --> "shuffle" = question will be drawn randomly from one of the source versions.

[[question]]
...

To specify each of our questions we create subsections [[question]] for each new question. For each question we must specify the pages, the total mark, and how that page is to be selected from the source versions (either “fixed” or “shuffle”). When fixed that question will be taken from source-version 1 in every single test produced. When shuffle, the pages of that question will be taken from a randomly chosen source-version. For our example, we use shuffle for all the questions.

After making these edits, our specification now reads:

name = "mABCmt1"
longName = "Midterm for Mathematics ABC"
numberOfVersions = 2
numberOfPages = 6
numberToProduce = 40
numberOfQuestions = 3

idPage = 1
doNotMarkPages = [2]

[[question]]
pages = 3
mark = 5
select = "shuffle"

[[question]]
pages = 4
mark = 5
select = "shuffle"

[[question]]
pages = [5, 6]
mark = 10
select = "shuffle"

Checking that specification

To make sure that the specification is consistent and satisfies some simple “sanity checks” we run plom-create again, with the validatespec sub-command:

$ plom-create validatespec
Parsing and verifying the specification "testSpec.toml"
Check specification keys
        contains "name" - check
        contains "longName" - check
        contains "numberOfVersions" - check
...

and quite a few more lines of checks. Most of this is simply checking that Plom has everything it needs, that total-marks are positive integers, each page is used, and so on. You don’t really need to pay much attention to this unless something goes wrong.

Uploading the specification

To upload your spec to your server:

$ plom-create uploadspec

See the plom-create uploadspec --help for how to use command-line arguments or environment variables to specify the server address and manager password.

Because we want to print some tests with names printed on them, we have to supply Plom with a class list.