This challenge and many future challenges will require you to create a user test case that verifies whether a program is performaing a particular function correctly.
Creating User Tests
- User tests are created by modifying a JSON file in your work area for the level, for this level, the tests are located in
~/cse240/01-linux/13/user_tests
- In each JSON file under
user_tests
, you will be required to give
Running the User Tests
- Typically, you will find the instructions for the tests in the Level Description (like below) and inside the name and description fields of the JSON file
- After creating the user tests, execute
/challenge/tester
from the command line
- a bad model version, named modelBad.P.L.T.bin (P=Project number, L=Level, T=Test number) version, that does not implement the functionality correctly, your test must report the bad program failed.
- a good model version, named modelGood.bin, that implements the required functionality correctly
- if the level required you to write a program then tester will use the test to test your program as well and your program must pass too.
Test Requirements
- The input and output that you create in the utest json file will be passed into a running version of either model good or modle bad programs
- When it passes the input into the model bad the test needs to be specific enough to detect that the model bad did not provide the expected output and it should fail, try executing the related model bad (e.g.,
./modelbad1.13.1.bin
)
- Similarly, when it passes the input into the model good it needs to detect that it did provide the expected output. You can test the model good by running
./modelGood.bin
- Fill in the "input" and expected "output" in both tests.
- utest1.13.1 will input "hello" and verify the proper result is returned by the program being tested (if not sure try running
modelGood.bin
and modelBad.1.13.1.bin
).
- utest1.13.2 will input anything except "hello" and verify that the program returns with expected result.
- Tester will use the inputs and outputs in the
user_tests/utest1.13.1.json
and user_tests/utest1.13.2.json
- If the user test cases detect the missing functionality in the modelBad.P.L.T version while also detecting that modelGood.bin implements the required functionality then Tester will print out the flag.
HINT
You can run the modelGood and modelBad by typing in /challenge/modelGood.bin
, /challenge/modelBad1.13.1.bin
(for user test 1.13.1), and /challenge/modelBad1.13.2.bin
(for user test 1.13.2)