EliteMobs .29, .30, .31 & .32 - What happened?

If you have been keeping up with Tester notifications, you will certainly have noticed that 5 EliteMobs dev versions were released within one day, and that 4 versions (.29, .30, .31 and .32) contained critical fixes. This is highly unusual for EliteMobs - I try to release one dev build a day at most, though I feel most comfortable with releasing about two versions a week at most. In 24h I ended up releasing nearly as many critical bug fixes as I have released in total over the lifespan of the plugin.

I wanted to take a minute here and let you in on the technical side of making EliteMobs, partly because I think some of you will find it somewhat interesting, partly so I can get a minute to vent about a complicated aspect of EliteMobs and partly as a justification of why it is sometimes builds come out sideways.

The nature of this issue is with how Minecraft deals with mob de-spawning. There are two types of mob despawning:

- Based on chunk unloads

- Based on distance from player

These two sound similar but are completely different, and this has caused most issues related to mobs appearing too much or disappearing.

The first method is the most intuitive - chunk unloads because there are no players nearby? Remove the mobs and unload the chunk. However, 99.99% of aggressive Minecraft mobs do not follow this logic.

The second method is... complicated to deal with. It is based on a range check - are there players nearby? These checks happen arbitrarily and leave no trace that a plugin can check, at least not to the best of my knowledge (using the Spigot API that is).

Now let us see where the issue started. During EM 7.2.28, several reports were filed that Custom Bosses were often gone by the time players reached the location they were meant to be in.

I checked the code, and quickly concluded that Event-based Custom Bosses were not protected against distance-based removals in the new tracker system - nothing too hard to resolve. I also realized that essentially all Elite Mobs - except for Regional Bosses - were getting removed based on a distance check, when I thought they were getting removed by a chunk unload check.

Chunk unloads are detectable by plugins and are in general safer to use. Distance-based removals must be detected by a repeating task, meaning that CPU and memory must be spared storing, scanning, and deleting Elites which may have existed for all of 1 tick, but are only scanned on 5-minute intervals for performance reasons.

I patched things so all Elites would be onlyremoved by chunk unloads, did some cursory testing with persistent event mobs (the zombie king) and normal elites. Everything seemed to be working accurately, and I released 7.2.30 in a rush to continue working on overhauling the Minidungeons.

This is where all hell broke loose.

Immediately, regional bosses started to get removed with chunk unloads, which was reported by several people. In a fun turn of events, it is not that the regional bosses immediately disappeared - that I would have noticed while I was working on it. They spawned correctly, but if they unloaded with a chunk and then loaded back in with the chunk, the code changes made it so the copy that loaded back in was no longer able to survive a second chunk unload.

I quickly patched that issue, tested it by having a bunch of Minidungeons on and loading / unloading their chunks, released it and went back to working on overhauling the Minidungeons.

But the issue was not over yet.

If you are familiar with how Minecraft entity spawning works and have opened your third eye while reading this post, you may have already guessed that the part where I made all Elites survive distance-based despawning (even though they do not survive chunk unloads / reloads) was a bad move.

Here is how mob spawning work (simplified): your server has a set number of mobs that it knows it should have per player online, around 30 by default. The server will keep spawning mobs until it reaches this number. Since mobs despawn based on distance checks, servers tend to spawn and despawn quite a lot of mobs! A single player can cause thousands to spawn and despawn during a normal gaming session.

Now, what happens when 5% of these do not despawn? Well, the answer is simple, right? They stay alive, wandering the world. As it turns out, if you have a base, you end up moving a lot in circles, which promotes mob spawns in a tight radius around you. Since vanilla mobs despawn, and Elites do not, you slowly replace all vanilla mobs with elites, and once the mob cap is reached you have essentially filled your world in with nothing but elites.

This is the actual issue that caused what seemed like way too many Elites to spawn. It is not that they were spawning too much, and it is not that they were not despawning with chunk unloads - it is that they survived distance-based checks and their vanilla counterparts did not.

I have now (hopefully) patched this latest issue with EM 7.2.32 – again in a rush to continue working on overhauling the Minidungeons.

Now that I have exposed the issue itself, let us talk about why it went undetected for the releases. I count four main reasons:

- The Minecraft mechanics behind spawning are hard to deal with. There is not that much documentation out there on how mobs spawn and despawn, and the Spigot API is not made for managing mob spawning and despawning in such a detailed way.

- The code responsible for Elite mob entity management is complex and highly interlinked. This is to make it as optimized as possible without having to sacrifice any features. As an example, EliteMobs has its own persistent entity management for unloaded chunks, entirely independent from Minecraft. This allows the plugin to be more efficient and even to pull of some cool tricks like allowing server owners to use the free LibsDisguises version but still have disguises work on chunk unloads and reloads.

- This issue did not get my full attention and got rushed. I received about 450 messages (almost all of them support requests) while working on it, and if you check the changelogs you will notice that this was not the only thing on them. Additionally, most of my focus was on working on the Minidungeons to get them released as soon as possible.

- I am tired. I have essentially been working 16-26 hours straight at a time since the 17th, mostly on EliteMobs but also on my day job. EliteMobs has seen some explosive growth over the past couple of months, especially January, and I feel like it is at the cusp of getting to the next level. I hope it is apparent to people following the tester builds that I really am putting as much work into this as I possibly can, and then some.

I do not aim to make releasing multiple versions a day something that I do regularly. I reserve this kind of quick-fire version updating only to moments where I see a critical issue that needs immediate resolution. Hopefully this is the last such series of quick updates for a long while.

This concludes the breakdown of this issue. If you read all the way through, I hope you found it at least somewhat interesting. I will now resume releasing updated minidungeons; things should be completely done today.

See you on the next one!

- MagmaGuy