OpenTTD

Tasklist

FS#5797 - [OS X] OpenTTD does not build on OS X Mavericks

Attached to Project: OpenTTD
Opened by Clemens Lang (neverpanic) - Saturday, 02 November 2013, 23:43 GMT
Last edited by Remko Bijker (Rubidium) - Friday, 08 November 2013, 22:26 GMT
Type Bug
Category Core
Status New
Assigned To No-one
Operating System Mac OS X
Severity Low
Priority Normal
Reported Version 1.3.2
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 0
Private No

Details

OpenTTD does not build on OS X Mavericks. This has multiple reasons:

Reason 1 is that /usr/bin/clang++ (or /usr/bin/c++) on Mavericks now default to linking against libc++ rather than libstdc++ as C++ STL. This breaks the build, because libc++ does not provide the debug/debug.h header included by src/os/macosx/osx_stdafx.h. This can be fixed by adding -stdlib=libstdc++ to the C++ compiler invocation. Note that this needs to present in detect_cputype() in config.lib, too, because otherwise the build system will assume a 32-bit build.

Reason 2 is that Apple has decided to remove CGDirectPaletteRef and a number of other functions, which are used in fullscreen.mm, which leads to:


/usr/bin/clang -O2 -fomit-frame-pointer -pipe -Os -arch x86_64 -fno-strict-aliasing -Wall -W -Wno-unused-parameter -Wno-unused-value -Wno-multichar -Wno-self-assign -Wno-parentheses-equality -Wno-unused-variable -DOSX -isystem/opt/local/include -DNO_QUICKTIME -mmacosx-version-min=10.5 -DUNIX -DWITH_COCOA -DENABLE_COCOA_QUARTZ -DWITH_ZLIB -DWITH_LZMA -I/opt/local/include -DWITH_LZO -D_SQ64 -I/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/3rdparty/squirrel/include -DWITH_PNG -I/opt/local/include/libpng15 -DWITH_FREETYPE -I/opt/local/include/freetype2 -I/opt/local/include -DWITH_ICU -I/opt/local/include -DWITH_ICONV -DENABLE_NETWORK -DNDEBUG -DWITH_PERSONAL_DIR -DPERSONAL_DIR=\"Documents/OpenTTD\" -DWITH_SHARED_DIR -DSHARED_DIR=\"/Library/Application\ Support/OpenTTD\" -DGLOBAL_DATA_DIR=\"/opt/local/share/games/openttd\" -I /opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/objs/release -I /opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/objs/lang -I /opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/objs/setting -c -o video/cocoa/fullscreen.o /opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:155:2: error: unknown type name 'CGDirectPaletteRef'
CGDirectPaletteRef palette; ///< palette of an 8-bit display
^
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:244:21: error: use of undeclared identifier 'CGDisplayBeamPosition'; did you mean 'CGDisplayRotation'?
double position = CGDisplayBeamPosition(this->display_id);
^~~~~~~~~~~~~~~~~~~~~
CGDisplayRotation
/System/Library/Frameworks/CoreGraphics.framework/Headers/CGDisplayConfiguration.h:381:18: note: 'CGDisplayRotation' declared here
CG_EXTERN double CGDisplayRotation(CGDirectDisplayID display)
^
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:316:26: warning: implicit conversion of NULL constant to 'int' [-Wnull-conversion]
this->window_pitch = NULL;
~ ^~~~
0
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:335:35: error: use of undeclared identifier 'CGDisplayCanSetPalette'
if (this->device_depth == 8 && !CGDisplayCanSetPalette(this->display_id)) {
^
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:428:19: error: use of undeclared identifier 'CGPaletteCreateDefaultColorPalette'
this->palette = CGPaletteCreateDefaultColorPalette();
^
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:592:9: error: cannot initialize return object of type 'CocoaSubdriver *' with an lvalue of type 'FullscreenSubdriver *'
return ret;
^~~
/opt/local/var/macports/build/_opt_dports_games_openttd/openttd/work/openttd-1.3.2/src/video/cocoa/fullscreen.mm:155:21: warning: private field 'palette' is not used [-Wunused-private-field]
CGDirectPaletteRef palette; ///< palette of an 8-bit display
^
2 warnings and 5 errors generated.
make[1]: *** [video/cocoa/fullscreen.o] Error 1



See https://developer.apple.com/library/mac/releasenotes/General/APIDiffsMacOSX10_9/CoreGraphics.html for a list of functions that have been removed. Unfortunately I wasn't able to find any documentation on what to use instead.
This task depends upon

Comment by Remko Bijker (Rubidium) - Sunday, 03 November 2013, 08:10 GMT
I think the first issue is already solved in trunk.

The second issue looks like is going to cause yet another load of #ifdefs in order in the OS X code in order to be able to keep support for the version that we already support (10.3.9-10.5) and get support for the versions we do not support (10.6 and later). As you can see, we only 'support' ancient versions because the newer versions had lots of issues and there's generally no one that has OS X and interest in fixing the issues.
Comment by Zydeco (Zydeco) - Sunday, 03 November 2013, 10:46 GMT
There's a fix for osx_stdafx.h in the trunk, but you also need to build with the 10.8 SDK with CFLAGS="-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk" or use ifdefs around the CGDirectPaletteRef stuff.
Comment by Michael Lutz (michi_cc) - Tuesday, 05 November 2013, 19:20 GMT
The attached patch should fix the remaining issues based on the error log. It does it by simply disabling the failing code as it is only used on systems below 10.7 anyway. I guess it is acceptable if a binary compiled with the 10.9 SDK only runs on 10.7+. It will not affect our compile farm.
Comment by Michael Lutz (michi_cc) - Friday, 08 November 2013, 20:20 GMT
Is r25951 (or later) building out-of-the-box on 10.9 (except possibly a missing lzo2) or does it still need the libstdc++ work-around? If it does need it, what exactly fails without?
Comment by Zydeco (Zydeco) - Friday, 08 November 2013, 22:19 GMT
It doesn't work out of the box, still has a few issues:
MAC_OS_X_VERSION_MIN_REQUIRED isn't defined by default, it defaults to 10.4 in AvailabilityMacros.h when compiling for ppc64/i386/x86_64, so it will still try to compile the full screen driver with 8-bit

and it also still needs the libstdc++ workaround

So, it compiles out of the box like this:
LDFLAGS="-stdlib=libstdc++" CFLAGS="-DMAC_OS_X_VERSION_MIN_REQUIRED=MAC_OS_X_VERSION_10_9" ./configure

Maybe the fullscreen subdriver should be compiled #if MAC_OS_X_VERSION_MIN_REQUIRED < MAC_OS_X_VERSION_10_7 instead, and use -DMAC_OS_X_VERSION_MIN_REQUIRED=MAC_OS_X_VERSION_10_7 when compiling on newer versions of Mac OS
Comment by Clemens Lang (neverpanic) - Friday, 08 November 2013, 23:12 GMT
I can reproduce that. Either the condition should be MAC_OS_X_VERSION_MIN_REQUIRED < MAC_OS_X_VERSION_10_7, or it should be MAC_OS_X_VERSION_MAX_ALLOWED < MAC_OS_X_VERSION_10_9 (which would, however, produce a binary that will not have the relevant code when run on systems < 10.7).

The include problem with <debug/debug.h> is fixed, but the openttd binary fails to link with missing symbols for me. The linker command line still mentions -lstdc++ (that's hardcoded in config.lib, line 1417). Changing that to -lc++ doesn't fix this either, though – I haven't yet found out why.
Comment by Michael Lutz (michi_cc) - Friday, 08 November 2013, 23:14 GMT
Damn, looks like I did the logic the wrong way around. Try with the attached patch on top, please.

And regarding the libstdc++ issue, could somebody™ please post the output of configure and make without the extra flag?

EDIT: "which would, however, produce a binary that will not have the relevant code when run on systems < 10.7" -> Indeed, but why would Random User make a binary on 10.9 with the intention to play on 10.6? Our own compile farm is explicitly using older SDKs exactly to avoid such issues for the official binaries.
Comment by Clemens Lang (neverpanic) - Friday, 08 November 2013, 23:21 GMT
On the last question: That's exactly what I'm thinking.

Trying the patch and generating the output your requested now.
Comment by Clemens Lang (neverpanic) - Friday, 08 November 2013, 23:42 GMT
I have attached the output of configure and build process of trunk using your latest patch.
Comment by Clemens Lang (neverpanic) - Saturday, 09 November 2013, 00:04 GMT
Using /usr/bin/clang++ -stdlib=libstdc++ as compiler leads to a working binary with your patch.
Comment by Clemens Lang (neverpanic) - Thursday, 19 December 2013, 23:59 GMT
1.3.3 works almost out of the box (except for the stdlib thing, but I can take care of that), but tries to use QZ_CreateFullscreenSubdriver, which isn't available and thus fails to link. The attached patch fixes this.
Comment by andythenorth (andythenorth) - Tuesday, 24 December 2013, 13:30 GMT
I have a build on OS X 10.9.1 of trunk at time of this comment.

I followed the instructions for mac compile in ottd wiki, with addition of I have some freetype issue that may not be related to Mavericks, so I have suppressed freetype.

./configure --enable-static --without-liblzo2 --without-freetype --without-osx-sysroot CFLAGS="-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk" LDFLAGS="-stdlib=libstdc++"
Comment by Michael Lutz (michi_cc) - Wednesday, 25 December 2013, 13:23 GMT
So, it seems that clang on 10.9 requires the stdlib thing. Does anybody know or can find out if that is true for all clang versions, i.e. also those included in older SDKs that still defaulted to gcc? When it is known which clang versions require the addition, config.lib could be changed accordingly.

Not sure if we're going to do another 1.3.x release, but trunk/1.4.0-beta seems to have the appropriate guards in cocoa_v.mm.
Comment by Clemens Lang (neverpanic) - Wednesday, 25 December 2013, 17:19 GMT
10.9 is the first release to default to using libc++ instead of libstdc++ and thus requires -stdlib=libstdc++, because openttd is currently not compatible with libc++. Older releases (I think back to 10.6, but you should be able to check by checking for the existence of /usr/lib/libc++.dylib) defaulted to libstdc++ and the clang versions shipped in those releases support the -stdlib switch, but libstdc++ was the default.

So you could set -stdlib=libstdc++ whenever /usr/lib/libc++.dylib exists on a system and the compiler is clang. This will do nothing on systems < 10.9, because you're re-specifying the default, and will change to libstdc++ on 10.9. When you're building on 10.9 with a minimum Mac OS X target version lower than 10.6 (I think), clang will refuse to compile, unless you also specify -stdlib=libstdc++.

Of course, the ideal solution to the problem would be that openttd would compile against both C++ runtime libraries.
Comment by Clemens Lang (neverpanic) - Wednesday, 25 December 2013, 17:37 GMT
Correction: 10.9's clang compiling against libc++ fails when -mmacosx-version-min is lower than 10.7:

"clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later)"

That's currently the only reason preventing the compilation of openttd against libc++. So you could either try to detect when libc++ is the default and switch the minimum deployment target to 10.7 in config.lib, or add -stdlib=libstdc++ in these cases.
Comment by Michael Lutz (michi_cc) - Wednesday, 25 December 2013, 22:41 GMT
But can you reliably detect if libc++ is the default? Looking for a file on disk doesn't strike me as the best idea, especially as nothing stops a user from passing a completely different C compiler or SDK to configure. Ideally there would be a way to get that info from clang --version, clang -dumpspec (or whatever other methods clang has besides the gcc-compatible ones) which would hopefully make it "just work" for all the various SDK/compiler/OS combinations.
Comment by Clemens Lang (neverpanic) - Saturday, 28 December 2013, 12:41 GMT
The C++ standard library default can only change between OS releases for compatibility reasons, so I think it's safe to check for clang && os.version >= 10.9.

Edit: Nevermind, that doesn't cover the case where a user deliberately passes -stdlib=libstdc++. I think in that case you can only compile a test binary and check for its c++ runtime using otool -L.
Comment by Michael Lutz (michi_cc) - Wednesday, 01 January 2014, 23:51 GMT
Apparently our configure script unconditionally adds a -lstdc++ to the linker command line. A (very) short test seems to suggest that this might not be necessary for OSX (compile farm compiler works, but that is a Very Special Case).

Attached is a patch for that. I'd be grateful for tests on as many different OSX setups as possible (both older and current OS). With a bit of luck this might be all that is needed (unless some of the external libs are built with mixed c++ libs, but I guess then you're screwed anyway).
Comment by xmirakulix (xmirakulix) - Saturday, 29 March 2014, 10:46 GMT
I can't get it to work with the config.lib patch.
But when I use ./configure LDFLAGS="-stdlib=libstdc++" it links successfully.

I am attaching the three different LDFLAGS that configure creates, as I said only the last creates a working binary:
- Version 1: clean trunk configure
- Version 2: patched config.lib
- Version 3: configure with LDFLAGS="-stdlib=libstdc++"

My setup: OSX 10.9.2, clang "Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)"
Comment by andythenorth (andythenorth) - Monday, 13 March 2017, 21:13 GMT
FWIW, I can build OpenTTD at r27785 on Sierra (macOS 10.12.3), with some deps from brew (didn't try macports this time for various reasons).

That doesn't help Mavericks users, but does indicate that there is a viable compile on 2017 macOS / OS X (daft name change by Apple).

Loading...