Testing via ctest
¶
This is totally optional, but you can run the unit / integration tests I use for
developing WODEN
to check your system / the code runs as expected. The tests
are compiled using the same cmake
script as the main code.
ALL tests are compiled in both FLOAT and DOUBLE precision, meaning the same test code is used to test the two different precision versions, with the compiler dropping in the necessary precision. Unless explicitly noted in the details in What do the tests actually do?, the tests require the same level of accuracy from the FLOAT and DOUBLE precision versions. Most of the time the FLOAT version is accurate to an absolute tolerance of 1e-7, and the DOUBLE to 1e-15. Read the tolerances for specific functions in the sections listed in What do the tests actually do?.
Dependencies¶
The tests use the following C library:
You don’t need to install Unity, just clone it somewhere:
$ git clone https://github.com/ThrowTheSwitch/Unity.git
You don’t need to install Unity anywhere, as WODEN
compiles the C
code directly. You just need to tell WODEN
where Unity lives in your system (for example, you could download a release version e.g. version 2.5.2 - see example below for how to link without installation).
You’ll also need to initiate the git submodule
that runs code coverage. Simply navigate to the WODEN
directory and run:
$ git submodule init
$ git submodule update
which will pull in the relevant CMake-codecov
dependencies. This allows us to track code coverage for the python
and C
code (no free tools exist for CUDA
at the time of writing, boooo).
To tell cmake
to build tests, you add TARGET_GROUP=test
to your command to tell CMake to build tests instead of the main code:
$ cd $WODEN_DIR
$ cmake .. -DTARGET_GROUP=test -DUNITY_ROOT=/usr/local/Unity-2.5.2
$ make -j 4
(where -DUNITY_ROOT=
is needed if you didn’t install unity
).
Warning
once you have done this, to go back to compiling the main woden
executable, you need to run:
$ cmake .. -DTARGET_GROUP=production
otherwise you’ll just keep building the tests.
Running tests¶
Once that compiles, you can run the tests by running:
$ ctest
You should see something like the following if successful:
$ ctest
Test project /home/jline/software/WODEN/build
Start 1: C_test_fill_primary_beam_settings_float
1/85 Test #1: C_test_fill_primary_beam_settings_float .................... Passed 0.00 sec
Start 2: C_test_fill_primary_beam_settings_double
2/85 Test #2: C_test_fill_primary_beam_settings_double ................... Passed 0.00 sec
Start 3: C_test_fill_timefreq_visibility_set_float
3/85 Test #3: C_test_fill_timefreq_visibility_set_float .................. Passed 0.00 sec
Start 4: C_test_malloc_and_free_float
4/85 Test #4: C_test_malloc_and_free_float ............................... Passed 0.00 sec
Start 5: C_test_write_visi_set_binary_float
5/85 Test #5: C_test_write_visi_set_binary_float ......................... Passed 0.00 sec
Start 6: C_test_write_visi_set_text_float
6/85 Test #6: C_test_write_visi_set_text_float ........................... Passed 0.00 sec
Start 7: C_test_fill_timefreq_visibility_set_double
7/85 Test #7: C_test_fill_timefreq_visibility_set_double ................. Passed 0.00 sec
Start 8: C_test_malloc_and_free_double
8/85 Test #8: C_test_malloc_and_free_double .............................. Passed 0.00 sec
...etc etc
Note
To test MWA Fully Embedded Element beam code, you must have the environment variable:
MWA_FEE_HDF5=/path/to/mwa_full_embedded_element_pattern.h5
declared. If you don’t have that, the MWA FEE beam code tests will just be skipped.
If you want more detail of what the tests are doing, run:
$ ctest --verbose
What do the tests actually do?¶
The tests are all located in WODEN/cmake_testing
, and each directory within contains tests
for a different file from WODEN/src
. Within each test directory, there are separate files for testing different functions, which include the function name. As an example, the directory WODEN/cmake_testing/CUDA_code/fundamental_coords
contains tests for the file WODEN/src/fundamental_coords.cu
, and contains test files that test the following functions:
cmake_testing/CUDA_code/fundamental_coords/test_lmn_coords.c -> src/fundamental_coords.cu::kern_calc_lmn
cmake_testing/CUDA_code/fundamental_coords/test_uvw_coords.c -> src/fundamental_coords.cu::kern_calc_uvw
The C
and CUDA
functions are tested using the Unity library, which has useful functions like:
TEST_ASSERT_FLOAT_EQUAL();
TEST_ASSERT_DOUBLE_WITHIN();
TEST_ASSERT_NULL();
allowing a simple testing of values. If a test says outputs are tested to be
equal, it refers to the TEST_ASSERT_FLOAT_EQUAL
or TEST_ASSERT_DOUBLE_EQUAL
function.
Note
For those unfamiliar with CMake
testing, even though the tests are located in WODEN/cmake_testing/
, when you run ctest
, the test files are copied and run in WODEN/build/cmake_testing
, so any output from the tests will be located there.
The sections below give an outline of the tests performed in each directory.
wodenpy
code tests:
C
code tests:
CUDA
code tests:
script
code tests:
Note
To be able to test CUDA
functions that are designed to work solely in GPU memory, it’s necessary to write wrapper functions that allocate GPU memory, pass the data into the CUDA
code to be tested, and then copy the results back into host memory. I’ve kept these ‘intermediate’ test functions inside the *.cu
files that contain the code being tested, as it’s not straight forward / performance degrading to have them in separate files. On casual inspection it looks like there are many functions in the *.cu
files I haven’t written tests for, but the extra functions are there because of testing. Sigh.