nbgrader is a tool for Jupyter notebooks that allows instructors to create, distribute, and grade assignments through the Jupyter notebook interface (Formgrader). Instructors can easily create notebook-based assignments for both coding exercises and written responses. nbgrader also provides a streamlined interface for quickly grading completed assignments.
The following documentation is an overview of the basic concepts of Formgrader/nbgrader on Datahub/DSMLP to assist graders in creating assignments for their courses.
Note: the Formgrader screenshots on the nbgrader documentation site show it running in jupyterlab. If you are using the "classic" jupyter notebook user interface, you won't see the jupyterlab panel on the left side of the screen.
Prior to the beginning of a course, IT Services will create a shared nbgrader TA account and provide its login credentials to the instructor. Only the shared nbgrader account will have access to nbgrader/formgrader in the course.
To access nbgrader/formgrader from Datahub:
To create a new assignment in nbgrader, go to the Manage Assignments tab and click Add new assignment... at the bottom of the page.
assignment_idis the name of the assignment being created
Note: When naming assignments in nbgrader (not the notebook itself), DO NOT give it an .ipynb extension. An assignment named "Assignment_1" will create a folder of the same name in your (grader's) home directory. Within that folder, assignment notebooks will have .ipynb extensions.
Each assignment created with nbgrader has a toolbar for each cell which gives the option to decide how each cell will be graded. To create a toolbar for each cell:
There are five toolbar options to choose from:
When specifying a cell option, each cell must be given an id and when applicable, how many points the answer is worth. The following sections will go into detail about the different types of toolbar options.
Note: nbgrader has a special syntax lines for cells that begin with
###. When an assignment is released, the region between the special syntax lines are replaced with a code stub, so students are not able to see the solutions that graders have written. If the special syntax is not used, the content of the entire cell will be visible.
### BEGIN SOLUTIONand
### END SOLUTION, which is then used when performing the autograded tests, A common use of this type of cell is for functions that students are required to write in the cell (typically with the function name given to be used by the autograded tests).
assert(or similar) statements. When running the autograding, the cell will pass if no errors are raised, and fail otherwise. Tests in these cells can be hidden using the special syntax
### BEGIN HIDDEN TESTSand
### END HIDDEN TESTS. A common use of this type of cell is to provide tests, either visible and/or hidden, that will grade code from the autograded answer cells.
After the assignment has been created, graders can verify that their solutions are correct and will pass all test cases. To do so, use the Validate button located on the jupyter notebook toolbar. If the solution passes all the tests, a success pop-up will show. Otherwise, a pop-up will show which cells have failed.
After an assignment has been created, graders can generate the version that students will receive. To generate the student version of the assignment in nbgrader:
After generating the student version of the assignment, graders can preview this version to make sure it looks correct (e.g. hidden tests are indeed hidden from students). There should now be a new folder called
release, which has the same structure as
release directory contains the release version of the assignment files.
To release the assignment to students, click the Release button. If a grader decides to "un-release" an assignment (e.g. fixing a typo), they may do so by clicking the Release button again (which is now an "x").
Note: Students who already downloaded the assignment will still have access to their own copy.
To collect all assignments, click the Collect button, which will display a pop-up window that shows the number of submissions collected.
For more detailed information about managing assignments, check the documentation here.
To autograde submissions, click on the number of submissions in the Manage Assignments tab in nbgrader. This will take the grader to a Manage Submissions page with all submissions. The status of each assignment should be needs autograding.
autogradeddirectory, which has the same structure as the
Once the autograding is complete, graders can now see the status of each submission. If there were cells that needed manual grading, the status of the graded assignment will be changed to needs manual grading. Otherwise, the status will be changed to graded.
To manually grade assignments in nbgrader, go to the Manual Grading tab. This will provide an interface for manually grading assignments.
Note: An assignment must be run with the autograder in order to be available for manual grading in nbgrader. This applies to all assignments.
Note: The default assignment file size limit is 100MB. If some files aren't transferring, open a New->Terminal window as the course grader, and add the following to the /usr/local/etc/jupyter/nbgrader_config.py file:
c.CourseDirectory.max_file_size = 400000
Once assignments have been graded, graders may generate feedback on an assignment to release to the student.
To get grades into Canvas, graders can run a command within a JupyterHub terminal to generate a CSV file to upload to Canvas.
nbgrader export. This will create a CSV file named
grades.csvin the home directory
grades.csv file is imported, the interface will determine whether there are any assignments that are not in Canvas. If so, the grader will be given a dropdown menu asking what "This Assignment Is" in Canvas: