FS#1059 - Extra large maps patch

Attached to Project: OpenTTD
Opened by Bilbo (bilbo) - Friday, 20 July 2007, 00:23 GMT
Type Patch
Category Core
Status New
Assigned To No-one
Operating System All
Severity Low
Priority Normal
Reported Version trunk
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 2
Private No


I think that the extra large maps path is now enough stable to be candidate for including into trunk.

And there it is - attached is patch for r10634

This patch enable to play on maps up to 262144 high and up to 262144 wide, although not that large in both dimensions at once.

There is maximal number of tiles, changeable in map.h - theoretically it could be upped to 16384*16384 and still work on 32bit machines (possibly even more for 64bit ones), but I decided to limit it to 16.7 million tiles (4096*4096 map), which is the most that reasonably fast computers (~2ghz, 512mb ram) can reasonably handle. Changing the limit in map.h is all that is needed for the limit to change, rest of the code will accomodate to this. Performance on smaller map sizes is unaffected by this patch.

Other dimensions are also possible, like 8192x2048, 16384x1024 ... 262144x64. The last one is very nicely playable with fast airplanes, planes now actually spend more time in flight than by accelerating, landing, circling, taxiing, loading, unloading, etc ...

Savegame compatibility is maintained, except older versions won't be able to load maps with one or both of dimensions larger than 2048 tiles

Link to forum thread:
This task depends upon

Comment by Bilbo (bilbo) - Sunday, 22 July 2007, 16:58 GMT
Updated patch, with one minor bug fixed:

fixed selected map resolution clamped to 2048 when it is saved in openttd.cfg
Comment by Patric Stout (TrueBrain) - Monday, 23 July 2007, 16:25 GMT
Are there any noticeable side-effects of this patch? How good was your bug-testing?
Comment by Bilbo (bilbo) - Monday, 23 July 2007, 17:16 GMT
Technically, as the patch basically only removes some hardcoded limitations to max size and adapts GUI to that (with exception of raising the pool size to be able to load/save large sizes properly), I can almost guarantee that map sizes under 2048x2048 are not affected by this patch, so if this patch uncovers any bugs, they will manifest only with at least one of the dimensions being 4096 or more.

Like there is  FS#716  - a bug that will occur with long straight 2000 tiles of track without signals. A bug which will not happen on 1024 x 1024 maps. Not so common, but sometimes such tracks are build, mainly for testing :) It may be possible that some other internal limit will be triggered, but I am not aware of any currently.

As to the larger map sizes. I tried to generate many maps of various width and playing some games on it.

On some of the large maps I tested old AI was present and it behaved normally, as on regular size maps.

I played several hours with plane route over 260000 squares, with several times saving, exiting and loading the map again.

I have not yet tried to create a train, bus and ship route across all the 260000 squares. I think I'll have yet to test that.

I have not noticed any side effects or odd behavior.
Comment by Patric Stout (TrueBrain) - Monday, 23 July 2007, 19:41 GMT
The reason I ask, is that in the past we planned a 4kx4k option. This was removed later on (when the bigmap patch was created), as it gave problems. It is possible that by now all those problems are solved already, without really anyone knowing, but we want to make sure it is. Things to look at: lighthouses, transmitters, old-landscape generator, trains, AIs, ... Please let me know if this is tested really well (also by other users). Tnx :)
Comment by Bilbo (bilbo) - Tuesday, 24 July 2007, 20:29 GMT
Up till now I found two bugs (slow rail construction which I already reported and bug in old terrain generator which I am going to report), but I found out that openttd without my patch is also affected by them. So my patch is not cause of these bug. Still, I'll test things further and I'll let you know once I'm finished with testing.
Comment by Bilbo (bilbo) - Tuesday, 24 July 2007, 20:34 GMT
Well, so that was probably not a bug, just the original generator generate quite a lot different landscapes with same settings than with these settings in temperate (half of map is raised, half of map is relatively flat)

But still it looks pretty weird ...
Comment by Bilbo (bilbo) - Sunday, 29 July 2007, 21:42 GMT
Attached newer version. Removed workaround around bug that was fixed in 10719 (not showing 64bit ints in NUM and COMMA).
Comment by Patric Stout (TrueBrain) - Sunday, 04 November 2007, 00:03 GMT
Some comments:

Don't include other languages than english.txt

+ uint64 map_x = 1U << _patches_newgame.map_x;
+ uint64 map_y = 1U << _patches_newgame.map_y;
+ uint64 tiles = map_x * map_y;
+ if (tiles > MAX_MAP_TILES) {

This can be done much faster. _patches_newgame.map_x + _patches_newgame.map_y > MAX_POWER_TWO will do the same ;) (2^a * 2^b = 2^(a+b) ;))

+ SetDParam(0, MAX_MAP_TILES);
+ SetDParam(1, tiles);

Silly enter.

+ case GLAND_GENERATE_BUTTON: { // Generate

No need for the {} there.

Also, 262144 is just too big. Please make the max value 65536.
Make the MIN_MAP_SIZE, ... enums.
Don't use doxygen comments inside functions.
Last comment: we have doubts 4kx4k really works. Please keep the maximum allowed tiles on 4M for now. In the future we can do tests to increase this value.
Comment by Patric Stout (TrueBrain) - Sunday, 04 November 2007, 00:12 GMT
After a bit more talking, some other requests: limit the max values to 8192 for now. This avoids all GUI changes.

Second, if an invalid amount of mapx/mapy combination is selected, make the text RED. This is a clear indication to the user that the value is too large, which should help out the process of selecting a valid combination :)
Comment by Bilbo (bilbo) - Tuesday, 06 November 2007, 10:10 GMT
65536? Or even 8192? That is not much. I think I'll change the code so that game will load savegame with any number of tiles (as long as dimensions are power of two, at least 64 and total number of tiles is < MAX_MAP_TILES - to keep compatibility if that limit is changed in the future or to be able to load savegames created with other (older) version of this patch)
So the limits would be applied only to newly-created games.

Creation of new games could be restricted to 8192.

I can change the max_map_tiles to 4M, as it could be easily raised later :)

I'll do the other changes, but I wonder why not include czech translation as well?
Comment by Jean-Francois Claeys (Belugas) - Tuesday, 06 November 2007, 17:43 GMT
Because we have translators who are assigned to that task.
As a default behaviour, we never impose translations, we let them do the job.
So, in this case, it will make the patch smaller.
So only english.txt is enough
Comment by Bilbo (bilbo) - Monday, 12 November 2007, 01:40 GMT
Yes, I know dev's don't need to do the translations, but why forbid them doing that, especially if I am native czech speaker? :)
Well, I have removed the czech translation as requested. I have moved them to separate file (map-256k-11415-czech.patch). Feel free not to use the file :)

Also, I encounteres small issue:
I wondered whether it is possible to add strings to combo boxes without them having a string ID (or generating such strings with ID's dynamically)

Now when the maximum is to be raised, you still need to update string ID's in translations
Idea is, when someone changes just the constants (MAX_MAP_SIZE_BITS, ...), everything would work without need to make additional changes (like adding extra strings to translation) and things like:
STR_512 :512
STR_1024 :1024
STR_2048 :2048
STR_4096 :4096
could disappear from translations.
Is there any version of ShowDropDownMenu and DrawString that takes ordinary string (const char *) as input instead of string ID?

Also, I am attaching new patch with above mentioned corrections done (map-256k-11415.patch), although I don't get what was meant with "Make the MIN_MAP_SIZE, ... enums." and I have not yet made the text to be red in the GUI with invalid selection. I'll look into it further, I'll try also to somewhat avoid the StringID's for the combobox for the path to be better configurable...
Comment by Bilbo (bilbo) - Monday, 12 November 2007, 01:51 GMT
Attached modified version of the patch - the MAX_MAP_TILES limit is enforced only in GUI when creating or generating a new world, but not when loading a savegame. Thus savegames created with older versions of this patch (or with this patch with modified limits) with larger sizes (like 65536x64) can be loaded (but limit od max. number of tiles is enforced strictly), while still blocking creation of new maps larger than limit (currently 8192) in any dimension.

map-256k-11415-nolimit.patch = map-256k-11415.patch with the above mentioned change
Comment by Bilbo (bilbo) - Tuesday, 11 December 2007, 02:30 GMT
Attached is new versiion of patch against 11616 - map-256k-11616.patch

Difference against previous version:
* No longer need numbers in translation, so changing map size limits in future is easy, just modify map.h, no more need to add extra string for larger map sizes. (Note: str_64, str_128 and alike would need to be evicted from non-english language files. Do you have some automated way of doing that, or should I write a script to do it and send a patch that will clean the other language files?) Also rising maximal dimension over 8192 in future would need to resize the GUI a bit, otherwise the strings will end up misaligned
* When map size that is too large is selected, the text will turn red, as TrueLight suggested.

map-256k-11616-czech.patch (optional) contain translation for czech language.
Comment by Bilbo (bilbo) - Saturday, 29 December 2007, 21:30 GMT
Added new version compatible with changes in trunk (recent source code refactorings). No functionality changed from previous patch.
Comment by Bilbo (bilbo) - Tuesday, 08 January 2008, 00:17 GMT
New version adapted to recent minor changes in trunk (moving some thing around headers). Functionality remains the same.
Comment by Bilbo (bilbo) - Monday, 14 January 2008, 16:17 GMT
New version adapted to trunk changes (11848). Functionality remains the same
Comment by Bilbo (bilbo) - Thursday, 17 January 2008, 20:55 GMT
Due to changes in trunk (dynamic dropdown list, genworld_gui adapted to dynamic dropdown list) I have to update my patch again.

Good thing is that the changes simplified my patch (from 14.5kb to 9.5kb)

I hope the patch is now fine to get in trunk.
Comment by Bilbo (bilbo) - Wednesday, 23 January 2008, 13:59 GMT
Due to introduction of DrawWindowWidgets function, my code had to be modified again.
I am attaching updated patch (functionality remains exactly the same as in old one, no new features)
Comment by Bilbo (bilbo) - Friday, 21 March 2008, 21:18 GMT
Updated patch to 12388 (sync with very slight trunk change)
Comment by Bilbo (bilbo) - Saturday, 22 March 2008, 02:15 GMT
Fixed patch for 12388 (fixed bug that could cause trains which are more than 2047 tiles apart in one dimension to crash (overflow in train collision check), introduced in 11877)
Comment by Bilbo (bilbo) - Monday, 30 June 2008, 17:30 GMT
Ok, so trunk changed a lot and last version of the patch no longer merges with the trunk.

I am attaching patch updated to trunk (r13664).
Comment by Bilbo (bilbo) - Monday, 30 June 2008, 17:36 GMT
Forgot to reapply fix, that fixed bug when trains too far apart (more than approx. 4096 squares) can crash into each other.

Attaching fixed patch.
Comment by Bogdan (sensation) - Saturday, 31 January 2009, 13:49 GMT
hello all!...i m new here. i was searching for something that can permit me tu run extra large maps in OpenTTD.and i foud this can i run XL maps with this patch that u have here...what am i sopose to do with it?please, give me all the instructions. thank you very much
Comment by Bilbo (bilbo) - Friday, 19 June 2009, 03:46 GMT
Attaching patch updated to 16598, with coding style fixed up a bit
Comment by Bilbo (bilbo) - Friday, 19 June 2009, 21:00 GMT
Differences to previously posted patch:

Train collision checking speedup. No more int64 are used. Now in most cases, even no multiplication is used at all.

When selecting map size, map sizes too large for current dimension (with regard to map size selcted for other dimension) are painted in red. This can be used to quickly see how large value I can choose before the game starts complaining about too many tiles ... (thus if I select value in red, then I need to decrease the other value ... )
Comment by Bilbo (bilbo) - Saturday, 27 June 2009, 23:27 GMT
Train collision speedup and some parts (some enumifying) was added to trunk.

Attaching patch updated to current trunk
Comment by Bilbo (bilbo) - Thursday, 04 February 2010, 01:49 GMT
Attaching patch updated to current trunk (some modification were required due to adapt to changes in GUI code) - functionality is same as in last patch.