Link

Creating a specification

We need to tell our previously partially-configured server about our 6-page 3-question 2-version source PDFs. Change into our project directory:

$ cd midterm

We need to tell Plom all about the structure of our test and how many papers it needs to produce. We do that through a “test specification”, which we an create and manipulate using the plom-build command:

$ plom-build
usage: plom-build [-h] {new,parse,class,make,...} ...

A template specification

The test specification needs to be precise, so rather than making it from scratch each time, Plom will provide you with a template to edit:

$ plom-build new
Creating specification file from template: "testSpec.toml"
  * Please edit the template spec "testSpec.toml"
Creating "sourceVersions" directory for your test source PDFs.
  * Please copy your test in as version1.pdf, version2.pdf, etc.

Plom tells you that it has done 2 things:

  • it has created a specification file testSpec.toml for you to edit, and
  • it has created a subdirectory sourceVersions into which you need to place your source PDFs and you have to name them version1.pdf and version2.pdf.

So, after you have copied your source versions into place, take a look at testSpec.toml file. The specification is written in the toml format which is human-readable and easy to edit in your favourite text-editor. The start of the file should look something like

# Example test specification for Plom

# >>> Edit the data below <<<

# Human-readable names of the test - one long, one short.
#name = "m101mt1"
#longName = "Midterm 1 for Mathematics 101"
name = "plomdemo"
longName = "Midterm Demo using Plom"

# Information about the test
# A test may have one or more versions
numberOfVersions = 2
# how many pages
numberOfPages = 6

and then the file continues with information about the number of questions, ID-pages and so on.

Names and numbers

We’ll start editing these to match our test.

  • name: this should be a short name that will be printed on the test. Something simple like mABCmt1 to indicate “Mathematics ABC Midterm 1”.
  • longName: a longer name that will be used to help build a test-return website. Hence we should write something like Midterm 1 for Mathematics ABC 2318. Including the year or term is a good idea to avoid potential confusion.
  • numberOfVersions: in our running example, we have 2 versions of the test.
  • numberOfPages: in our running example, each test is 6 pages.
  • numberToProduce: this is the total number of test PDFs we will produce. Each test we print must be unique, and cannot be reused. Hence this number should always be some more than the total number of students you expect to sit the test. The developers have typically produced something like 10% more the number of students. Remember that you don’t actually have to print all those PDFs, but you can keep them in reserve just in case.
  • numberToName: Plom can produce tests with student names and IDs printed on them. This is very useful when you have assigned seating for example. We show an example of this below.

We’re going to assume that we will have around 30 students sitting (15 in each section), so we’ll actually produce 40. Then one section wants named tests and the other doesn’t, so we’ll tell Plom to name 15. So now the start of our specification looks like

name = "mABCmt1"
longName = "Midterm for Mathematics ABC"
numberOfVersions = 2
numberOfPages = 6
numberToProduce=40
numberToName=15

Pages and questions

Now let us turn to the rest of specification – its job is to tell Plom the details of ID-pages, Do-no-mark pages, and our test questions. Take a look at the rest of specification:

# How many questions to mark - these will be named [question.1], [question.2], ... [question.n]
numberOfQuestions=3

# the id-pages - always taken from Version1
[idPages]
pages=[1] # must contain at least 1 page and be contiguous

# pages that are not marked (like instructions, formula sheets, examination rules etc) - always taken from Version1
[doNotMark]
pages=[2,7] # must exist, but can be an empty list.

# now the actual questions (groups of pages) to be marked
# [question.n] with n a positive integer [1,2,3... etc] - contiguous please.
# pages = [1,2,3] - contiguous ordered list of positive integers
# mark = positive integer
# select = "fixed" or "shuffle"

[question.1]
pages=[3,4]
mark=10
select="shuffle"

[question.2]
pages=[5,6]
mark=10
select="fixed"

[question.3]
pages=[8]
mark=10
select="shuffle"

The rest of the specification tells Plom how to put pages together into questions, ID-pages, and do-not-mark pages (ie formula sheets and instructions).

  • numberOfQuestions: For our example this is 3.
  • [idPages]: This subsection of the specification tells Plom which pages belong together and constitute the ID pages for the test. In our case this is just page 1. However you have the flexibility to set any contiguous set of pages to be the ID pages. Note that these pages are shown while identifying papers, but will not be shown while marking/annotating papers.
  • [doNotMark]: These are pages that are neither ID pages nor questions. Typically these will be used for instructions or formula sheets. For our example it is just page 2.

To specify each of our questions we need to create subsections [question1], [question.2], [question.3] and so forth. Since we have 3 questions, we can just use the ones already in the template. For each question we must specify the pages, the total mark, and how that page is to be selected from the source versions. A selection is either fixed or shuffle. When fixed that question will be taken from source-version 1 in every single test produced. When shuffle, the pages of that question will be taken from a randomly chosen source-version. For our example, lets use shuffle for all the questions.

After making these edits, our specification now reads:

name = "mABCmt1"
longName = "Midterm for Mathematics ABC"
numberOfVersions = 2
numberOfPages = 6
numberToProduce=40
numberToName=15
numberOfQuestions=3

[idPages]
pages=[1]

[doNotMark]
pages=[2]

[question.1]
pages=[3]
mark=5
select="shuffle"

[question.2]
pages=[4]
mark=5
select="shuffle"

[question.3]
pages=[5,6]
mark=10
select="shuffle"

Checking that specification

To make sure that the specification is consistent and satisfies some simple “sanity checks” we run plom-build again, now with the parse sub-command:

$ plom-build parse
Parsing and verifying the specification "testSpec.toml"
Check specification keys
        contains "name" - check
        contains "longName" - check
        contains "numberOfVersions" - check

and quite a few more lines of checks. Most of this is simply checking that Plom has everything it needs, that total-marks are positive integers, each page used exactly once, and so on. You don’t really need to pay much attention to this unless something goes wrong. Since everything checks out okay, this tells us that the hard work of specifying the test is done. We can almost move onto building PDFs.

But first take a look the last few lines.

Assigning a privateSeed to the spec - check
Assigning a publicCode to the spec - check
Saving the verified spec to "verifiedSpec.toml"
Your spec indicates that you wish to print named papers.
Please process your class list using "plom-build class ".

This tells that Plom is assigning your test a random number seed that will be used for any random selection of versions. It is also fixing a code that will be stored (along with other information) in the qr-codes stamped on pages. This code helps plom to make sure that when you upload page images that they really do belong to this test. Then, rather than altering your spec file, Plom copies your spec file with these new codes to a new verified spec file, which is what the system will actually use.


Since the specification tells Plom to print some tests with names printed on them, we have to supply Plom with a class list. To do that, we first need to start the server.