Remember that when it comes to testing three things should
come to mind and should be explained to the reader of the tests:
1) What is the expected output?
2) What is the observed output?
3) Is the observed output the same as the expected output?
Why and/or why not?
Here are some general rules of thumb that you should consider when designing, implementing and running your test programs. Some parts of this document apply more to the 2nd, 3rd and 4th assignments than the 1st but it should help you to understand what I'm looking for when it comes to testing. 1) Think about the person you are trying to demonstrate your program too and what type of and how much information they will want to see regarding your testing (i.e., think about the marker). For example, pretend that your are the project leader working in a large company. Once a month there is a large meeting of technical staff in which each of the project leaders has 10 mintues to give a status report on their project. This means that you have 10 minutes to demonstrate using the output of your test programs what parts of your project do and do not work correctly. 2) Be smart about your testing. If the test program is more than 3 or 4 pages long it is likely too long. If the output produced from running the program is more than 2 or 3 pages the test program is probably not designed very well. Think about the number of different things that need to be tested and how much time it will take to look at the output. 3) The source for the test programs should be clear, consise, and documented with comments about what the program does and expected return values for system calls. 4) The test program should be bug free. Take some time when writing the test program to ensure that it is correct. Too often people waste a bunch of time trying to fix their system when in fact the system is fine but there is a bug in the test program (or they misunderstand how the test works or the return values that are expected). 5) Output a simple message if the tests fail or succeed. Better yet only print output if unexpected results are obtained and when the program ends. 6) Someone should be able to spend about 1 minute looking at the output of each test program to determine whether all of the tests in the program failed or succeeded. 7) Use multiple test programs to test different aspects of your system. 8) Test special cases. Ideally a production quality Operating System should be completely bullet proof (not that many really are :-). No user program, even malicious ones, should be able to crash the system. In this course your system doesn't have to be that strong but there should not GLARING holes (that you haven't already pointed out). 9) Check the return values of function and system calls. That is check that the call has completed successfully - if it has not handle the error intelligently. You should check return values in your test programs as well as in your operating system!
Updated: September 30, 1996