Setting up TA MarkUs and Assignments

To help TAs be familiar with each assignment and ensure high-quality marking, a subset of each assignment’s questions will be assigned to TAs. The following are the steps to set up TA MarkUs and Assignments.

* Usually, we will have a fake course for the TA submission and testing purpose. We will use the fake course CS125 as an example to demonstrate this process.

Selecting Questions for the TA Assignment

The TA assignments should seek to accomplish 3 main things:

1. Introduce the basic concepts covered in each assignment.
2. Familarize TAs with the design recipe and CS135 Style Guide.
3. Familarize TAs with components of questions they will be marking on the assignment.

For example, TAs should need to write the contracts for questions which they will be marking for Contract Correctness and/or Contract Formatting. As well, it is a good idea to pick a few low-mid difficulty assignment question for the TAs to at least know what concepts students are learning. As assignment difficulty increases, the selection of questions can change so as not to take up an excessive amount of the TAs' time.

After selecting the coverage of the TA assignment, ask for feedback from the instructor in charge of the assignment as well as other ISAs on the handmarking team. Once confirming, you may set up the MarkUs and the basic tests, and notify the TAs. For more on TA Assignment MarkUs and basic/correctness test setup, see below.

Step 1 – Setting up TA Assignment

Usually, the testing folder and the basic tests for the student’s assignment have been created and tested properly before setting up the TA Assignment. Hence, the only thing we need to do is to copy the testing folder from the CS135 course account to the fake course created for the purpose of TA submission and testing (CS125 here). Then, keep the basic tests for the TA Assignment questions and delete basic tests for other questions.

There are two methods of copying the testing folder from one course account to another. Both methods need access to the fake course account. If the ISA cannot scp or access samba for the fake course account (cs125 in this case), then please contact Nick Lee from CSCF.

Method 1: Use Samba

If you choose to use samba, you only need to drag and drop the folders that you need from one course account to another. For more information about how to access Samba, please see AccessingCourseAccount page.

Method 2: Use Terminal

First, log into the course account through the terminal. To copy the testing folder from the course account to another course account (eg. CS125), simply run the command scp -r ~/marking/ASSN/ Another_Account@linux.student.cs.uwaterloo.ca:~/marking/ASSN

where ASSN is the assignment you’re copying, and Another_Account is the course account that is used for TA submission and testing. Here is an example:

scp -r ~/marking/a01/ cs125@linux.student.cs.uwaterloo.ca:~/marking/a01

This will copy the folder a01 in the marking directory to CS125’s marking directory.

Next, exit from the course account and go to the course account that is created for the TA submission and testing. Go to the basic tests folder (test.pt) of the assignment that you copied from the course account earlier (cd ~/marking/ASSN/test.pt)

For example, if we want to go to the basic tests folder for a01, simply run the command

cd ~/marking/a01/test.pt

Then, we need to change the “computeMarks” file. Since the TAs do not need to complete A00 in order to receive the marks for later assignments, we need to comment out all things related to A00 in the “computeMarks” file.

Since TAs will only do a subset of questions in each assignment, they do not need to submit all files required in the assignment. Hence, you need to change the filelist in the “computeMarks-postprocess” file so that the filelist will only include the required files for the TA Assignment.

Once you have changed both “computeMarks” file and “computeMarks-postprocess file”, you can go to the “in” directory (cd ~/marking/ASSN/test.pt/in) and delete the questions that are not part of the TA assignment.

At this point, you have the basic tests ready for the assignment. Next, go to the MarkUs for the fake course created for TA submission and testing, and set up the TA assignment. This process is the same as the the process of setting up MarkUs for the student assignments. Please see MarkUs for more information about Setting up and Creating Assignments.

Once the MarkUs has been set up, you can submit your solutions to the MarkUs to test the basic tests. After making sure that the basic tests work properly, you can send the emails to TAs about the TA Assignment and TAs can start working on the assignment.

Step 2 – TA Assignment Correctness Tests

After the due date for the TA Assignment has passed, we need to start the collection process.

Since we do not have finalized correctness tests for the TA assignment when it is due, we need to copy the correctness tests from the course account.

First, log into the course account through the terminal. Then, run the command

scp -r ~/marking/ASSN/test.0 Another_Account@linux.student.cs.uwaterloo.ca:~/marking/ASSN/

where ASSN is the assignment you’re copying, and Another_Account is the course account that is used for TA submission and testing. Here is an example:

scp -r ~/marking/a01/test.0 cs125@linux.student.cs.uwaterloo.ca:~/marking/a01/

This will copy the correctness tests for a01 from the course account to CS125 account.

Next, we need to change the “computeMarks” file, as we did for the basic tests. Since the TAs do not need to complete A00 in order to receive the marks for later assignments, we need to comment out all things related to A00 in the “computeMarks” file. You can copy the “computeMarks” file from the basic tests or previous TA assignment basic/correctness tests.

Since TAs will only do a subset of questions in each assignment, they do not need to submit all files required in the assignment. Hence, you need to change the filelist in the “computeMarks-postprocess” file so that the filelist will only include the required files for the TA Assignment.

Once you have changed both “computeMarks” file and “computeMarks-postprocess file”, you can go to the “in” directory (cd ~/marking/ASSN/test.pt/in) and delete the questions that are not part of the TA assignment.

At this point, you have the correctness tests ready for the assignment.

Step 3 – Setting up MarkUs Marking Scheme

Before starting the assignment collection process, you need to set up the marking scheme and annotations on MarkUs.

• First, go to the MarkUs for the course and download the csv files for the rubric and annotations.

• Next, go to the MarkUs for the fake course for TA submission and testing. Go to the relevant pages and upload both csv files.

• Delete any categories in the rubric and annotations that do not apply to the TA Assignment. In the rubric, you may readjust the weights proportionally so they add up to 25%. That way, the TAs can receive a more helpful grade on the assignment.

• At this point, you can start the assignment collection process.

Step 4 – Collecting TA Assignment

The assignment collection process is the same as the collection process for the student assignment. Please see collectAssignments page.

Step 5 – Start Marking

At this point you should be completely done with the assignment collection process for the TA Assignment and the TA assignments can now be handmarked.

Comments