OpenTTD

Tasklist

FS#6036 - CMake build system

Attached to Project: OpenTTD
Opened by Will Glynn (willglynn) - Thursday, 05 June 2014, 18:47 GMT
Last edited by Charles Pigott (LordAro) - Saturday, 19 August 2017, 09:13 GMT
Type Work in progress
Category Build system
Status Closed
Assigned To No-one
Operating System All
Severity Medium
Priority Normal
Reported Version trunk
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No

Details

I have a working prototype of a CMake-based build process. CMake has some advantages over the current system:

* CMake generates appropriate and (reasonably) idiomatic project files for many platforms from a single specification, simultaneously eliminating the projects/ directory and supporting additional tools (particularly ninja and Xcode).
http://www.cmake.org/cmake/help/v2.8.8/cmake.html#section_Generators

* CMake provides built-ins for things that would otherwise require shell scripting (implicitly depending on Unix) or solving separately (VBScript on Windows). This makes it possible to do things like search for libraries, probe for platform-specific functions/headers, and substitute variables into template files in a platform-independent way.

* CMake provides automatic dependency scanning based on the host compiler dependency output, meaning that changing a header file rebuilds only the files that #include it. No `make depend`, and it actually works in all the various environments.

* Out-of-tree builds are normal. This means no `make mrproper` since the build process never writes to the source tree. This also means it's easy to maintain multiple independent build environments (e.g. for cross-compiling) from a single checkout.

CMake has some disadvantages:

* People wishing to build OpenTTD must first use CMake to generate their desired project file. This is a one-time step -- CMake notices changes to the build definitions and will update the project file automatically when you `make`/`ninja`/click Build -- but it is nonetheless an additional dependency.

* People wishing to change the OpenTTD build process must understand CMake. This isn't necessarily bad – in fact, I think it's easily more maintainable than the current cross-platform Makefiles plus the configure script plus the Visual Studio generators – but the downside is that people understand the current process but not necessarily CMake.

OpenTTD targets some platforms that are unfamiliar to me (BeOS, Amiga, PSP, OS/2) but assuming these are built using cross-compilers from a host platform supported by CMake (Linux/UNIX/Windows/OSX), I don't see why that would be a problem. On the other hand, if OpenTTD can currently be built natively in these environments, switching to CMake would be a loss.

I propose replacing the current OpenTTD build process with CMake. I wish to discuss this topic.
This task depends upon

Closed by  Charles Pigott (LordAro)
Saturday, 19 August 2017, 09:13 GMT
Reason for closing:  Invalid
Additional comments about closing:  Prototype never pushed anywhere with apparently no progress made in 3 years.
If this offends, feel free to complain to LordAro on IRC
Comment by fonsinchen (fonsinchen) - Saturday, 07 June 2014, 14:44 GMT
Can we see the prototype? There are definitely a lot of hairy problems waiting for anyone replacing the build system. We can probably deprecate some of the old platforms but that has to be discussed on a case by case basis.
Comment by Alberth (Alberth) - Sunday, 08 June 2014, 20:58 GMT
You do want a mrproper target, to get rid of all the build files and all generated cmake files, ie a 'pristine copy', for packing it or for testing/fixing cmake setups.
In fact, for a unix environment, a developer typically expects all usual make targets, and make behavior, several of which it doesn't deliver.

In my experience, cmake is too much focused on the cmake configurator rather than the developers of the project.
Supposedly simple stuff like switching from a normal to a debug build isn't explained in the help printed by the program. Website documentation also doesn't mention it, except at the technically correct place, which is not findable for a developer that doesn't want to spend time digging through stuff he doesn't want to know.

Generated files have flattened variable values instead of doing substitution at runtime, which makes quick experiments non-feasible.
Comment by Will Glynn (willglynn) - Sunday, 08 June 2014, 22:53 GMT
> Can we see the prototype?

I'll push it somewhere shortly.

> You do want a mrproper target, to get rid of all the build files and all generated cmake files, ie a 'pristine copy', for packing it or for testing/fixing cmake setups.

Cmake-generated makefiles have no mrproper target because it's intended that you'd build out-of-tree, e.g.:

$ svn checkout ...
$ mkdir openttd-gcc-i386
$ cd openttd-gcc-i386
$ cmake ../openttd # options for gcc-i386
$ make

If you want to delete all the build files and generated cmake files:

$ rm -r openttd-gcc-i386

Out-of-source builds let you keep a single source tree in openttd/ and then do something like:

$ make -C openttd-gcc-i386-linux
$ make -C openttd-gcc-amd64-linux
$ make -C openttd-clang-i386-linux
$ make -C openttd-clang-amd64-linux

…plus whatever other cross-compiler environments you have set up.

> In fact, for a unix environment, a developer typically expects all usual make targets, and make behavior, several of which it doesn't deliver.

My build environment right now gives:

$ make help
The following are some of the valid targets for this Makefile:
... all (the default if no target is provided)
... clean
... depend
... edit_cache
... rebuild_cache
... OpenTTD
... strgen
... settingsgen
... tables

It's true that Cmake doesn't give you all the targets generated by autotools, but that should be expected -- it's different.

What targets and behavior do you miss? Cmake likely solved those problems in a different way.

> In my experience, cmake is too much focused on the cmake configurator rather than the developers of the project.

In my experience, I've found it to be ideal for coordinating teams with diverse development environments, especially split across Linux/Windows/OSX. Everyone gets to use their platform's native tools, and everyone's development environments keep working even as the codebase (and build process) evolves. This requires a layer of abstraction above Makefiles -- the "Cmake configurator" at issue here -- because Makefiles are simply not universal.

> Supposedly simple stuff like switching from a normal to a debug build isn't explained in the help printed by the program. Website documentation also doesn't mention it, except at the technically correct place, which is not findable for a developer that doesn't want to spend time digging through stuff he doesn't want to know.

This depends on your generator. Xcode gives me the option to switch between Debug and Release using the typical mechanisms (and in fact the built-in Debug/Run/Profile/Analyze/Archive targets do the right thing. Other IDEs including Visual Studio work the same way.

If you're using Makefiles, Cmake expects you to specify CMAKE_BUILD_TYPE=Debug or Release as appropriate. This is different but not intrinsically worse than ./configure --enable-debug. If you're building both configurations, it makes sense to set up two build directories: one for debug and one for release. This lets `make` reason about dependencies properly -- it'll build exactly what's necessary for each configuration in isolation -- which is a guarantee that re-./configure-ing doesn't provide. This is how those IDEs manage their builds internally:

$ find OpenTTD-Xcode -type d
...
OpenTTD-Xcode/src/OpenTTD.build
OpenTTD-Xcode/src/OpenTTD.build/Debug
OpenTTD-Xcode/src/OpenTTD.build/Debug/OpenTTD.build
OpenTTD-Xcode/src/OpenTTD.build/Debug/OpenTTD.build/Objects-normal
OpenTTD-Xcode/src/OpenTTD.build/Debug/OpenTTD.build/Objects-normal/i386
OpenTTD-Xcode/src/OpenTTD.build/Release
OpenTTD-Xcode/src/OpenTTD.build/Release/OpenTTD.build
OpenTTD-Xcode/src/OpenTTD.build/Release/OpenTTD.build/Objects-normal
OpenTTD-Xcode/src/OpenTTD.build/Release/OpenTTD.build/Objects-normal/i386

> Generated files have flattened variable values instead of doing substitution at runtime, which makes quick experiments non-feasible.

Yes. Cmake treats Makefiles as a build target. It treats build.ninja files, Visual Studio projects, Eclipse projects, KDevelop projects, and Xcode projects the same way. I can't think of a situation where it makes sense to edit the generated Makefile instead of editing the CMakeLists.txt, just like it doesn't make sense to edit a .o instead of the corresponding .c. It's almost certainly slower and harder than editing the source, and you'll need to change the source anyway to push changes upstream.

Loading...