Question Are there settings I can use to increase the performance of ThreadedWorldUpgrader ?

Mogwai

New member
Jan 19, 2022
8
0
1
USA
www.strongback.us
Two worlds, semi large by my standards. Upgrading at the moment to 1.20..2-233

forceUpgrade in progress

But it runs insanely slow and isn't really using the available resources.

This is a dedicated machine with 24GB ram and running on (albeit older) Xeon processor with 8 cores.

What can I do to improve this?



2023-10-10 20_50_12-Corsair.png
 

electronicboy

Administrator
Staff member
Dec 11, 2021
227
10
38
28
The thing just determines the # of threads in use by the number of CPU threads you have, so, no control there; but, being frank, the entire process is get a chunk, load its NBT, run it through (trivial) data conversions, and save it back to the disk. There is generally little reason to bother running forceUpgrade given how trivial the process is, otherwise the thing is pretty much just strung by the amount of IO you can do.
 

Mogwai

New member
Jan 19, 2022
8
0
1
USA
www.strongback.us
The thing just determines the # of threads in use by the number of CPU threads you have, so, no control there; but, being frank, the entire process is get a chunk, load its NBT, run it through (trivial) data conversions, and save it back to the disk. There is generally little reason to bother running forceUpgrade given how trivial the process is, otherwise the thing is pretty much just strung by the amount of IO you can do.

Then I'm confused.
the forceUpgrade option enables the new features in unrendered/ungenerated parts of the world does it not?
So if I'm upgrading from say 1.20.1 to 1.20.2 wouldn't it be wise or necessary to run that?
 

electronicboy

Administrator
Staff member
Dec 11, 2021
227
10
38
28
the forceUpgrade option enables the new features in unrendered/ungenerated parts of the world does it not?
No, it just runs the serialized chunk data through the schema updater, i.e. DFU, and saves the data back to the disk. That process is basically what mojang uses to manipulate the serialised data of a chunk to upgrade it, i.e. renaming item types, converting old data, etc; for chunks, they'll also potentially add blocks to the chunk

loading a chunk (to a full "ticking status") in general is:
1) load data from disk
2) Run data, if it exists, through the data upgrader (X)
3) pull chunks up through the chunk system, i.e. running through the chunk generation system
4) chunk pops into the world, post processing is ran

forceUpgrade takes you up to step 2 and shoves the data back onto the disk, the retrogen stuff, i.e. below 0 chunks, is basically a weird special case where mojang basically converted the chunk status of chunks back to a lower generation so that they'd rerun through that system in order to trip the retro gen + blending system added.
So if I'm upgrading from say 1.20.1 to 1.20.2 wouldn't it be wise or necessary to run that?
No, it's never been necessary to run forceUpgrade, Mojang only ever added that because with the size of the data transformations inside of the flattening being large, as well as DFU having some sizable performance issues at first, the standard upgrade done during loading chunks was stupidly slow, hence that flag was added to just get that stage over and done with.

Outside of that huge hump, we do not, and have not, recommended using force upgrade, for the most part, it's a complete waste of time, especially with how small and fast the typical data upgrade path is these days.