Fighting with JSON — A Cpp War Story

 

When an unknown compiler features drives you mad

I was refactoring the build environment for an internal C++ project at work. To keep things simple, we used the following JSON library:

https://github.com/nlohmann/json

The build system was based on CMake. I planned to replace our wrapper scripts for CMake with a preset.json. If you haven’t heard of this feature yet: when you want to set specific configurations for customers via CMake defines, you can predefine these in a dedicated preset file. It works surprisingly well. For my experimental render engine, I put all the Windows packages installation via vcpkg into a preset file like this:

{
    "version": 3,
    "configurePresets": [
        {
            "name": "default",
            "binaryDir": "${sourceDir}",
            "cacheVariables": {
                "CMAKE_TOOLCHAIN_FILE": "contrib/vcpkg/scripts/buildsystems/vcpkg.cmake"
            }
        }
    ]
}

You now invoke the build with

cmake .\CMakeLists.txt --preset=default

under Windows, and the user no longer needs to worry about installing vcpkg, since it’s shipped under contrib.

I applied the same logic for compiler- and target-specific settings in the company project. And suddenly, the build failed. The error message read:

error parsing version, missing ;

The file mentioned in the errorversion simply contained the current version string and wasn't even used in the C++ build process at all — until now. So what happened?

What happened?

After some fruitless searching through the source code, trying to find out whether the version file had accidentally been included somewhere, I confirmed that this wasn’t the case. Otherwise, this build error would have appeared much earlier.

Next, I searched the CMake build environment to see if there was a pre-processing step that might be turning the version file into a mis-configured header file. Another dead end.

So what now? I identified the .cpp file that triggered the error. Unfortunately, the MSVC build output didn’t show which include caused the build to fail. So I used the following compiler switch:

/showIncludes

With this switch enabled, all included files for the translation unit are shown. While digging through the generated output — which listed hundreds of includes — I noticed one header that seemed to be triggering the build error:

json/include/nlohmann/detail/macro_scope.hpp

So I started studying this header more closely. And bingo:

#ifdef __has_include
    #if __has_include(<version>)
        #include <version>
    #endif
#endif

So, if the preprocessor supports the __has_include feature, it checks whether a file named version exists. If it does, it gets included implicitly.
Why didn’t this happen in the old build setup I asked myself?

The answer is simple: in the new preset-based build, the build directory was set to the root directory of the project.
In the old build, we used a directory structure like:

build/<platform>/<config>

So I changed my CMake output directory to build. But that wasn't enough. It had to be at least build/<platform> — in other words, at least two directory levels deep to avoid the problem. That workaround fixed the issue.

What have I learned

1. If build errors don’t make sense, enable all compiler options to extract more info from the build process. Without the option: /showIncludes I would never have found the root cause.
2. If even then the result seems illogical: accept that it’s still happening. If reality doesn’t match your expectations, change your perspective.
   If I didn’t include the file, someone else must have. If it didn’t happen explicitly, it happened implicitly — in this case, via nlohmann.
3. Just because you don’t know a preprocessor directive to include headers doesn’t mean it doesn’t exist.
   Visual Studio knows the __has_include feature — I didn’t.
4. Never make your build depend on the target directory structure.
5. I hate these kinds of bugs.

Thanks a lot for reading!