Running Check50

Martijn Stegeman, University of Amsterdam

Step 1. Getting Started

If you have a working Python installation, please run pip3 to install check50:

pip3 install check50

You should now be able to run the tool:

$ check50
usage: check50 [-h] [-d] [--offline] [-l] [-o {ansi,json,html} [{ansi,json,html} ...]]
               [--target TARGET [TARGET ...]] [--output-file FILE]
               [--log-level {debug,info,warning,error}] [--ansi-log] [--no-download-checks]
               [--no-install-dependencies] [-V] [--logout]
check50: error: the following arguments are required: slug

The reason that check50’s complaining is that it needs a slug, a path to the checks that have to be run. These checks are usually hosted in a GitHub repository, which makes them easy to find. Let us now choose an assignment for which we have an existing check.

Step 2. The Assignment

The assignment we’re going to be working on is called “Hello”. It’s supposed to be a Python program that asks for a name, and then outputs “Hello, name!”. Looks like this when run at a command prompt:

$ python3
Name: Martijn
Hello, Martijn!

For now, we’ll try to run the check without creating that program. As it happens, the slug for this program is cs50/problems/2020/fall/sentimental/hello. From that slug, you can infer that it lives in this location:

Let’s run it by typing the slug and adding the flag -l (lowercase letter). This ensures that the check is run on your own computer instead of Harvard’s cloud infrastructure.

This also means that check50 is only critically dependent on two things: the pip infrastructure and the availability of a GitHub repository with checks. So anyone can use it at any time.

$ check50 cs50/problems/2020/fall/sentimental/hello -l
You seem to be missing these required files:
You are currently in: ~/dev, did you perhaps intend another directory?

Again, check50 is complaining, this time because a file called is expected to be present. Check50 always runs checks on the files that are in the current directory! So if you, or your students, want to check a program, they have to cd into that directory.

But that’s OK! Lets’ create an empty file called Let’s see what check50 says now:

$ touch
$ check50 cs50/problems/2020/fall/sentimental/hello -l
Results for cs50/problems/2020/fall/sentimental/hello generated by check50 v3.2.0
:) exists.
:( responds to name Emma.
    expected prompt for input, found none
:( responds to name Rodrigo.
    expected prompt for input, found none

Finally it works! Of course, your empty solution in shouldn’t pass all checks. And indeed, your empty program correctly fails two of those. Only the “ exists” check passes now.

Here are the three potential results that you may encounter for each check:

Step 3. What checks look like

Have a look at this example check for

import check50

def exists():
    """ exists."""

def veronica():
    """responds to name Emma.""""python3").stdin("Emma").stdout("Emma").exit()

def brian():
    """responds to name Rodrigo.""""python3").stdin("Rodrigo").stdout("Rodrigo").exit()

As you can see, there are three checks:

Never mind those function names!

From the example you may notice a few things:

Incidentally, the third check is fundamentally the same as the second one. It is a simple measure to ensure that the program isn’t hardcoded towards the name “Emma”. Of course, student code could be hardcoded in the following way and still pass the check:

name = input("Name: ")
if name == "Emma":
    print("Hello, Emma!")
    print("Hello, Rodrigo!")

So you may want to write better checks to encourage students to write better code :-) But that’s something that can develop over the years!

Step 4. Creating your own checks repo

To get started, create a GitHub repository and add a subfolder for the check that you would like to write. Then you need two files at a minimum:

Step 5. Creating checks for Jupyter Notebooks