Testing Guide

This project uses GoogleTest (gTest) for unit testing and CTest as the test runner.

Testing Strategy

The project follows a "centralized mirror" strategy:

• All tests are located in the root tests/ directory.
• The structure inside tests/ strictly mirrors the structure of the project's source code.
• Tests link against the EnigmaCore static library, allowing them to test the logic without the application's main() function interfering.

Directory Structure

tests/
├── CMakeLists.txt
├── AssetProvider/
│   ├── TestFileAssetProvider.cpp
│   └── TestSpec.md
├── EnigmaMachine/
│   ├── TestEnigmaConfigLoader.cpp
│   ├── TestEnigmaMachine.cpp
│   ├── TestHistoricalVectors.cpp
│   ├── TestSpec.md
│   └── TestSpecLoader.md
├── PlugBoard/
│   ├── TestPlugBoard.cpp
│   └── TestSpec.md
└── RotorBox/
    ├── TestReflector.cpp
    ├── TestReflectorSpec.md
    ├── TestRotor.cpp
    ├── TestRotorBox.cpp
    ├── TestRotorSpec.md
    ├── TestSpec.md
    ├── TestTransformer.cpp
    └── TestTransformerSpec.md

How to Run Tests

Configuration

By default, tests are disabled to maintain fast build times for the core application. You must explicitly enable them during the CMake configuration step:

cmake -DENIGMA_BUILD_TESTS=ON -S . -B build

Command Line (CLI)

Navigate to your build directory (usually build/debug) and use ctest:

cd build/debug
ctest --output-on-failure

Visual Studio Code

1. Run Task: Press Ctrl+Shift+P -> Run Task -> Run Tests.
2. Debug: Open the Run and Debug sidebar, select **(gdb) Launch Tests**, and press F5.
3. Test Explorer: Use the beaker icon in the sidebar (requires CMake Tools extension).

Adding New Tests

1. Create the Test File

Add a new .cpp file in the appropriate subdirectory within tests/. Use the GoogleTest macros:

#include <gtest/gtest.h>
#include "YourHeader.hpp"
 
TEST(YourModuleTests, FeatureDescription) {
    // Arrange
    // Act
    // Assert
    EXPECT_EQ(1, 1);
}

2. Register the File

Open tests/CMakeLists.txt and add your new file to the add_executable(EnigmaTests ...) list:

add_executable(EnigmaTests
    ...
    YourModule/YourNewTest.cpp
)

3. Build and Run

Re-run your build command. CMake will automatically detect the new tests.

Historical Test Vectors

To ensure cryptographic accuracy, the project includes a suite of tests based on historical Enigma I (Wehrmacht/Luftwaffe) configurations. These tests use verified wiring and notch data found in assets/historical/.

Test Name	Description	Source
StandardAAAAA_BDZGO	Validates the classic "AAAAA" vector for Rotors I-II-III.	Historical Baseline
OCAMLVector	Complex setup with plugboard and custom starting positions.	Cornell CS 3110
LongStringVerification	Verifies stepping logic over a 35-character string.	EnigmaCore Baseline
HistoricalReciprocity	Ensures encryption/decryption symmetry with historical settings.	-

These tests are located in tests/EnigmaMachine/TestHistoricalVectors.cpp.

Sanitizers (Clang)

To improve code reliability and detect memory errors or undefined behavior, the project integrates LLVM/Clang Sanitizers.

Requirements

• Compiler: clang / clang++ must be used for sanitizer builds.

Enabling Sanitizers

You can enable sanitizers individually during the CMake configuration step:

Option	Sanitizer	Description
ENIGMA_USE_ASAN	AddressSanitizer (ASan)	Detects heap/stack buffer overflows, use-after-free, and memory leaks.
ENIGMA_USE_UBSAN	UndefinedBehaviorSanitizer (UBSan)	Detects signed integer overflow, null pointer dereference, and other C++ undefined behaviors.
ENIGMA_USE_MSAN	MemorySanitizer (MSan)	Detects uninitialized memory reads.

Example Command:

cmake -DCMAKE_CXX_COMPILER=clang++ -DENIGMA_USE_ASAN=ON -B build
cmake --build build

Local Verification

To run the same scenarios as the CI locally, use the provided helper script:

# Ensure you have built the CLI with sanitizers enabled
./scripts/run_sanitizers.sh build

Valgrind (Memory Leak Detection)

The project includes a custom CMake target to run the entire test suite through Valgrind.

Usage

1. Configure with Tests Enabled: bash cmake -DENIGMA_BUILD_TESTS=ON -S . -B build
2. Run Valgrind Target: bash cmake --build build --target Enigma_valgrind

This will execute EnigmaTests with --leak-check=full and will return a non-zero exit code if any leaks or memory errors are detected, making it suitable for both local development and CI gates.

Constraints

• Linux Only: Valgrind is currently only supported on Linux environments.
• Performance: Running through Valgrind is significantly slower than native execution.

Constraints

• Exclusion of Tests: By design, sanitizers are applied to the core engine and the CLI application but are excluded from the Unit Test suite targets. This is achieved by using a separate non-instrumented object library for tests when sanitizers are active, reducing noise and execution time for the test suite.
• Performance: Sanitizers introduce runtime overhead (typically 2x-4x) and should be used during development or in specific CI pipelines rather than production builds.

CI Integration

Code analysis is centralized in the Code Analysis workflow (code-analysis.yml), which runs on every push and pull request. It orchestrates parallel jobs for:

• Static Analysis: Running Clang-Tidy on the core logic.
• Runtime Sanitizers: Executing ASan, UBSan, and MSan checks via the reusable sanitizers.yml workflow.

Results are aggregated into a single GitHub Job Summary without failing the build, allowing the team to monitor and fix issues asynchronously.

GoogleTest Dependency

The project uses CMake's FetchContent to download GoogleTest automatically.

• Offline Mode: You must run the initial CMake configuration while online. After that, the library is stored in build/<build_type>/_deps and can be used offline.
• Installation: No manual installation of GoogleTest is required on your system.

Property-Based Testing

The project uses RapidCheck for property-based testing, which generates random test cases to verify fundamental cryptographic properties across thousands of configurations.

What is Property-Based Testing?

Unlike traditional unit testing with fixed inputs, property-based testing randomly generates inputs and verifies that certain mathematical properties hold. For the Enigma machine, the most important property is reciprocity: encrypting then decrypting with the same settings must return the original value.

Enabling Property Tests

Property tests are disabled by default. To enable them:

cmake -DENIGMA_BUILD_TESTS=ON -DENIGMA_BUILD_PROPERTY_TESTS=ON -S . -B build
cmake --build build

Running Property Tests

# Run all property tests
cd build && ctest -C Debug --output-on-failure -R Properties
 
# Or run directly
./build/tests/EnigmaPropertyTests

Test Coverage

The property-based test suite includes 60+ tests across 5 test modules:

Module	Tests	Key Properties Verified
EnigmaMachineProperties	10	Reciprocity, Decryption symmetry, Output range
RotorProperties	14	Forward/reverse transform inverse, Rotation cycles
PlugBoardProperties	15	Swapping symmetry, No fixed points, Determinism
ReflectorProperties	5	Bidirectional mapping, No self-mapping
RotorBoxProperties	14	Multi-rotor stepping, Full encryption cycle

Key Properties Tested

• Reciprocity: Encrypt(Encrypt(x)) == x - The fundamental Enigma property
• Determinism: Same input always produces same output
• Output Range: All transforms stay within [0, 25]
• Inverse Transform: Forward transform followed by reverse returns original input
• Edge Cases: Empty configurations, boundary values, self-loops

CI Integration

Property tests run automatically on every push and pull request via the test-and-coverage.yml workflow. The job includes:

• Test result XML generation
• Pass/fail summary in GitHub PR checks

Code Coverage

For code coverage analysis (gcov/lcov), thresholds, and CI integration, see Benchmarking Guide.