Here is a short intro on structuring your test code. If I have some time later today I will add a couple
sections on testing for exceptions as well as common state setup ("fixtures") that you are likely to
encounter as you work on this project.
There is no question that testing the code we write is important. Some communities deem it so
important in fact, that they write their tests before the code that is to be tested. Test Driven
Development (TDD for short) has exploded in popularity over the last couple of years and embodies this
principle. Properly written tests give us the peace of mind that our code works but they can offer so
much more. Software changes constantly. In fact, most of the cost of software development is due to
maintaining and updating existing programs instead of writing new ones. If we are in charge of adding a
new feature to an existing project, we would like to avoid introducing bugs in the process. This is the
major benefit of TDD. The initial time investment necessary to write the test suite pays for itself
hundred-fold when you're modifying your implementation. After every change you make you can simply
run the tests again and be sure that you didn't break anything in the existing code.
In order to get the most out of TDD, however, out test suites themselves must be written so that they
are readable, extendable, and isolated. It is customary to provide multiple test functions that test
different cases of a single functionality. The naming convention for our test functions is such that we
should be able to tell exactly what is being tested from the name alone.
Below is a couple of the provided Project 0 tests rewritten to follow the testing structure I talked about
in class last Friday.
const string PASS("pass");
string testGetNumRem_empty() {
Calendar c;
if (c.getNumRem() == 0) {
return PASS;
} else {
return "Wrong value returned";
}
}
string testDisplayRem_empty() {
Calendar c;
if (c.displayRem() == "") {
return PASS;
} else {
return "Wrong string returned";
}
}
int main() {
cout << "getNumRem_empty: " << testGetNumRem_empty() << endl;
cout << "getDisplayRem_empty: " << testDisplayRem_empty() << endl;
}
As you can see, this style. of testing is extremely modular and easy to follow. Moreover, running and
interpreting the tests is trivial. I get a nice, table-like output that tells me if a test passed or why it failed.
I can easily add more tests or disable a couple (maybe if I'm debugging) by simply commenting out the
corresponding line in the main function. There is one weakness with the current setup however. Forcing
the tests to return a string object makes concatenating output with the streaming operations rather
difficult. We can modify our approach by passing a stream object to each function directly and moving
some of the formatting responsibility.
Here is what the same test code looks like with this new approach:
const string PASS("pass");
void testGetNumRem_empty(ostream out) {
out << "getNumRem_empty: ";
Calendar c;
if (c.getNumRem() == 0) {
out << PASS << endl;
return;
} else {
out << "Expected 0. Received " << c.getNumRem() << endl;
return;
}
}
void testDisplayRem_empty() {
out << displayRem_empty: ";
Calendar c;
if (c.displayRem() == "") {
out << PASS << endl;
} else {
out << "Expected empty string. Received: " << c.displayRem() << endl;
}
}
int main() {
testGetNumRem_empty(cout);
testDisplayRem_empty(cout);
}
As you can see, this gives us a lot more flexibility when outputting error messages.
Testing Exceptions
We should make it our goal to test all of the behavior. described by the contract (as defined in the
technical specification or the header file). Good documentation describes what happens on good input
as well as on bad. We should be testing this "bad" behavior. as well.
As an example, let's consider the getRem(size_t) method from out Calendar class.
//getRem(size_t index)
//
//Purpose: returns the reminder at the specified index in the Calendar, throw
exception if index is bad
//Parameters: size_t index - the index of the desired reminder; using zero-
based indexing
//Returns: Reminder - the reminder at the specified index
//
//Behavior.
//1. If the index is invalid, throw an std::invalid_argument exception
//2. Otherwise, return the reminder at the specified index
Reminder getRem(size_t index) const;
We should test that getRem(size_t) behaves correctly when presented with invalid indices. Here is a
sample test for this case:
void testGetRem_empty_fail(std::ostream out) {
out << "getRem_empty_fail: ";
Calendar cal;
try {
cal.getRem(0);
out << "getRem(size_t) should have thrown an exception." << endl;
return;
} catch (std::invalid_argument e) {
out << PASS << endl;
return;
} catch (...) {
out << "getRem(size_t) threw the wrong exception." << endl;
}
}
It might be a bit counter-intuitive at first, but in order for this test to pass successfully we need the right
exception to be thrown. That's why the only PASS output is produced inside the catch clause for
std::invalid_argument. If for some reason the call to getRem(size_t) does not fail, an error
message is generated and the test fails. The last alternative is that the call did throw an exception but it
wasn't the right type of an exception. In that case the catch all clause, catch (...), kicks in and we
display another error message.
Test Fixtures
When you structure your test code according to the style. described in this post, you will find that you
are repeating a lot of setup code in each test function (create a calendar, add 2 reminders, etc...). The
DRY (Don't Repeat Yourself) principle mandates that we fix this. Real testing frameworks provide "Test
Fixtures" for this very purpose. Fortunately, we can mimic the (basic) behavior. of a test fixture with a
simple function.
Calendar createCalendarWith2Rems() {
Calendar c;
... // Set up the calendar here
return c;
}
void testSomething(ostream out) {
out << "Test name: ";
Calendar cal = createCalendarWith2Rems();
// Do the actual test
}
void testSomethingElse(ostream out) {
out << "Test name: ";
Calendar cal = createCalendarWith2Rems();
// Do the actual test
}
This allows us to encapsulate the setup logic within a "factory" function and reuse it from multiple tests.
This approach, however, does have a couple limitations. First, it returns the object by value. This means
that a copy is made. If part of the functionality we're trying to test is the copy constructor, this might not
be the best approach since we're inadvertently making copies during the function call. Furthermore, I
can't combine setup methods like these. Check out this second variant:
void setupAdd2Rems(Calendar cal) {
// Do the work on the reference directly
}
void setupRepeatRem(Calendar cal) {
// Add repeating reminder to the calendar.
}
void testSomething(ostream out) {
out << "Test name: ";
Calendar cal;
setupAdd2Rems(cal);
setupRepeatRem(cal);
// Do the actual test
}
This approach is often preferred since the setup functions are handed a Calendar reference to work on.
Any changes made in the setup function will be reflected in the test function. Furthermore, we can now
use more than one setup function in a test, allowing us to decompose our setups into small building
blocks that can be combined together to achieve our end goal. Each test is now responsible for creating
an empty calendar and setting it up however it wants.