Mirrors/Paper
3
0
Fork 1
Mirror von https://github.com/PaperMC/Paper.git synchronisiert 2024-12-15 11:00:06 +01:00
Code Issues Projekte Releases Wiki Aktivität
23ee0a8a88
Paper/Spigot-Server-Patches/0459-Optimize-PlayerChunkMap-memory-use-for-visibleChunks.patch

231 Zeilen
12 KiB
Diff
Originalformat Normale Ansicht Verlauf

Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
From cee599dd7b5c962429d797a091ab740da42179a3 Mon Sep 17 00:00:00 2001
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
From: Aikar <aikar@aikar.co>
Date: Wed, 8 Apr 2020 03:06:30 -0400
Subject: [PATCH] Optimize PlayerChunkMap memory use for visibleChunks
No longer clones visible chunks which is causing massive memory
allocation issues, likely the source of Humongous Objects on large servers.
Instead we just synchronize, clear and rebuild, reusing the same object buffers
as before with only 2 small objects created (FastIterator/MapEntry)
This should result in siginificant memory use reduction and improved GC behavior.
diff --git a/src/main/java/net/minecraft/server/ChunkProviderServer.java b/src/main/java/net/minecraft/server/ChunkProviderServer.java
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
index 1dcd0980ec..e627440c41 100644
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
--- a/src/main/java/net/minecraft/server/ChunkProviderServer.java
+++ b/src/main/java/net/minecraft/server/ChunkProviderServer.java
@@ -669,7 +669,7 @@ public class ChunkProviderServer extends IChunkProvider {
entityPlayer.playerNaturallySpawnedEvent.callEvent();
};
// Paper end
- this.playerChunkMap.f().forEach((playerchunk) -> {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
+ this.playerChunkMap.forEachVisibleChunk((playerchunk) -> { // Paper - safe iterator incase chunk loads, also no wrapping
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
Optional<Chunk> optional = ((Either) playerchunk.b().getNow(PlayerChunk.UNLOADED_CHUNK)).left();
if (optional.isPresent()) {
diff --git a/src/main/java/net/minecraft/server/MCUtil.java b/src/main/java/net/minecraft/server/MCUtil.java
index b9d5844520..9980e4c277 100644
--- a/src/main/java/net/minecraft/server/MCUtil.java
+++ b/src/main/java/net/minecraft/server/MCUtil.java
@@ -500,7 +500,7 @@ public final class MCUtil {
WorldServer world = ((org.bukkit.craftbukkit.CraftWorld)bukkitWorld).getHandle();
PlayerChunkMap chunkMap = world.getChunkProvider().playerChunkMap;
- Long2ObjectLinkedOpenHashMap<PlayerChunk> visibleChunks = chunkMap.visibleChunks;
+ Long2ObjectLinkedOpenHashMap<PlayerChunk> visibleChunks = chunkMap.getVisibleChunks();
ChunkMapDistance chunkMapDistance = chunkMap.getChunkMapDistanceManager();
List<PlayerChunk> allChunks = new ArrayList<>(visibleChunks.values());
List<EntityPlayer> players = world.players;
diff --git a/src/main/java/net/minecraft/server/PlayerChunkMap.java b/src/main/java/net/minecraft/server/PlayerChunkMap.java
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
index e1e4ea793a..ce645a69bd 100644
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
--- a/src/main/java/net/minecraft/server/PlayerChunkMap.java
+++ b/src/main/java/net/minecraft/server/PlayerChunkMap.java
@@ -56,7 +56,8 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
private static final Logger LOGGER = LogManager.getLogger();
public static final int GOLDEN_TICKET = 33 + ChunkStatus.b();
public final Long2ObjectLinkedOpenHashMap<PlayerChunk> updatingChunks = new Long2ObjectLinkedOpenHashMap();
- public volatile Long2ObjectLinkedOpenHashMap<PlayerChunk> visibleChunks;
+ public final Long2ObjectLinkedOpenHashMap<PlayerChunk> visibleChunks = new Long2ObjectLinkedOpenHashMap(); // Paper - remove copying, make mt safe
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
+ public transient Long2ObjectLinkedOpenHashMap<PlayerChunk> visibleChunksClone; // Paper - remove copying, make mt safe
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
private final Long2ObjectLinkedOpenHashMap<PlayerChunk> pendingUnload;
final LongSet loadedChunks; // Paper - private -> package
public final WorldServer world;
@@ -170,7 +171,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
public PlayerChunkMap(WorldServer worldserver, File file, DataFixer datafixer, DefinedStructureManager definedstructuremanager, Executor executor, IAsyncTaskHandler<Runnable> iasynctaskhandler, ILightAccess ilightaccess, ChunkGenerator<?> chunkgenerator, WorldLoadListener worldloadlistener, Supplier<WorldPersistentData> supplier, int i) {
super(new File(worldserver.getWorldProvider().getDimensionManager().a(file), "region"), datafixer);
- this.visibleChunks = this.updatingChunks.clone();
+ //this.visibleChunks = this.updatingChunks.clone(); // Paper - no more cloning
this.pendingUnload = new Long2ObjectLinkedOpenHashMap();
this.loadedChunks = new LongOpenHashSet();
this.unloadQueue = new LongOpenHashSet();
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -262,8 +263,49 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
return (PlayerChunk) this.updatingChunks.get(i);
}
+ // Paper start - remove cloning of visible chunks unless accessed as a collection async
+ private static final boolean DEBUG_ASYNC_VISIBLE_CHUNKS = Boolean.getBoolean("paper.debug-async-visible-chunks");
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
+ private boolean isIterating = false;
+ public void forEachVisibleChunk(java.util.function.Consumer<PlayerChunk> consumer) {
+ org.spigotmc.AsyncCatcher.catchOp("forEachVisibleChunk");
+ boolean prev = isIterating;
+ boolean wasUpdating = updatingChunksModified;
+ isIterating = true;
+ try {
+ for (PlayerChunk value : this.visibleChunks.values()) {
+ consumer.accept(value);
+ }
+ } finally {
+ this.isIterating = prev;
+ if (!this.isIterating && updatingChunksModified && !wasUpdating) {
+ this.updateVisibleChunks();
+ }
+ }
+ }
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
+ public Long2ObjectLinkedOpenHashMap<PlayerChunk> getVisibleChunks() {
+ if (Thread.currentThread() == this.world.serverThread) {
+ return this.visibleChunks;
+ } else {
+ synchronized (this.visibleChunks) {
+ if (DEBUG_ASYNC_VISIBLE_CHUNKS) new Throwable("Async getVisibleChunks").printStackTrace();
+ if (this.visibleChunksClone == null) {
+ this.visibleChunksClone = this.visibleChunks.clone();
+ }
+ return this.visibleChunksClone;
+ }
+ }
+ }
+ // Paper end
+
@Nullable
public PlayerChunk getVisibleChunk(long i) { // Paper - protected -> public
+ // Paper start - mt safe get
+ if (Thread.currentThread() != this.world.serverThread) {
+ synchronized (this.visibleChunks) {
+ return (PlayerChunk) this.visibleChunks.get(i);
+ }
+ }
+ // Paper end
return (PlayerChunk) this.visibleChunks.get(i);
}
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -444,8 +486,9 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
// Paper end
protected void save(boolean flag) {
+ Long2ObjectLinkedOpenHashMap<PlayerChunk> visibleChunks = this.getVisibleChunks(); // Paper remove clone of visible Chunks unless saving off main thread (watchdog kill)
if (flag) {
- List<PlayerChunk> list = (List) this.visibleChunks.values().stream().filter(PlayerChunk::hasBeenLoaded).peek(PlayerChunk::m).collect(Collectors.toList());
+ List<PlayerChunk> list = (List) visibleChunks.values().stream().filter(PlayerChunk::hasBeenLoaded).peek(PlayerChunk::m).collect(Collectors.toList()); // Paper - remove cloning of visible chunks
MutableBoolean mutableboolean = new MutableBoolean();
do {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -473,7 +516,7 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
// this.i(); // Paper - nuke IOWorker
PlayerChunkMap.LOGGER.info("ThreadedAnvilChunkStorage ({}): All chunks are saved", this.w.getName());
} else {
- this.visibleChunks.values().stream().filter(PlayerChunk::hasBeenLoaded).forEach((playerchunk) -> {
+ visibleChunks.values().stream().filter(PlayerChunk::hasBeenLoaded).forEach((playerchunk) -> {
IChunkAccess ichunkaccess = (IChunkAccess) playerchunk.getChunkSave().getNow(null); // CraftBukkit - decompile error
if (ichunkaccess instanceof ProtoChunkExtension || ichunkaccess instanceof Chunk) {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -639,11 +682,19 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
});
}
+ protected boolean updateVisibleChunks() { return b(); } // Paper - OBFHELPER
protected boolean b() {
- if (!this.updatingChunksModified) {
+ if (!this.updatingChunksModified || this.isIterating) { // Paper
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
return false;
} else {
- this.visibleChunks = this.updatingChunks.clone();
+ // Paper start - stop cloning visibleChunks
+ synchronized (this.visibleChunks) {
+ this.visibleChunks.clear();
+ this.visibleChunks.putAll(this.updatingChunks);
+ this.visibleChunksClone = null;
+ }
+ // Paper end
+
this.updatingChunksModified = false;
return true;
}
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -1104,12 +1155,12 @@ public class PlayerChunkMap extends IChunkLoader implements PlayerChunk.d {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
}
protected Iterable<PlayerChunk> f() {
- return Iterables.unmodifiableIterable(this.visibleChunks.values());
+ return Iterables.unmodifiableIterable(this.getVisibleChunks().values()); // Paper
}
void a(Writer writer) throws IOException {
CSVWriter csvwriter = CSVWriter.a().a("x").a("z").a("level").a("in_memory").a("status").a("full_status").a("accessible_ready").a("ticking_ready").a("entity_ticking_ready").a("ticket").a("spawning").a("entity_count").a("block_entity_count").a(writer);
- ObjectBidirectionalIterator objectbidirectionaliterator = this.visibleChunks.long2ObjectEntrySet().iterator();
+ ObjectBidirectionalIterator objectbidirectionaliterator = this.getVisibleChunks().long2ObjectEntrySet().iterator(); // Paper
while (objectbidirectionaliterator.hasNext()) {
Entry<PlayerChunk> entry = (Entry) objectbidirectionaliterator.next();
diff --git a/src/main/java/org/bukkit/craftbukkit/CraftWorld.java b/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
index 051506fce8..630d6470a4 100644
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
--- a/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
+++ b/src/main/java/org/bukkit/craftbukkit/CraftWorld.java
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -74,6 +74,7 @@ import net.minecraft.server.GameRules;
import net.minecraft.server.GroupDataEntity;
import net.minecraft.server.IBlockData;
import net.minecraft.server.IChunkAccess;
+import net.minecraft.server.MCUtil;
import net.minecraft.server.MinecraftKey;
import net.minecraft.server.MovingObjectPosition;
import net.minecraft.server.PacketPlayOutCustomSoundEffect;
@@ -290,6 +291,7 @@ public class CraftWorld implements World {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
return ret;
}
public int getTileEntityCount() {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
+ return MCUtil.ensureMain(() -> {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
// We don't use the full world tile entity list, so we must iterate chunks
Long2ObjectLinkedOpenHashMap<PlayerChunk> chunks = world.getChunkProvider().playerChunkMap.visibleChunks;
int size = 0;
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -301,11 +303,13 @@ public class CraftWorld implements World {
size += chunk.tileEntities.size();
}
return size;
+ });
}
public int getTickableTileEntityCount() {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
return world.tileEntityListTick.size();
}
public int getChunkCount() {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
+ return MCUtil.ensureMain(() -> {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
int ret = 0;
for (PlayerChunk chunkHolder : world.getChunkProvider().playerChunkMap.visibleChunks.values()) {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
@@ -314,7 +318,7 @@ public class CraftWorld implements World {
}
}
- return ret;
+ return ret; });
}
public int getPlayerCount() {
return world.players.size();
@@ -432,6 +436,14 @@ public class CraftWorld implements World {
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
@Override
public Chunk[] getLoadedChunks() {
Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2 Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-09 03:07:59 +02:00
+ // Paper start
+ if (Thread.currentThread() != world.getMinecraftWorld().serverThread) {
+ synchronized (world.getChunkProvider().playerChunkMap.visibleChunks) {
+ Long2ObjectLinkedOpenHashMap<PlayerChunk> chunks = world.getChunkProvider().playerChunkMap.visibleChunks;
+ return chunks.values().stream().map(PlayerChunk::getFullChunk).filter(Objects::nonNull).map(net.minecraft.server.Chunk::getBukkitChunk).toArray(Chunk[]::new);
+ }
+ }
+ // Paper end
Optimize PlayerChunkMap memory use for visibleChunks No longer clones visible chunks which is causing massive memory allocation issues, likely the source of Humongous Objects on large servers. Instead we just synchronize, clear and rebuild, reusing the same object buffers as before with only 2 small objects created (FastIterator/MapEntry) This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 09:09:05 +02:00
Long2ObjectLinkedOpenHashMap<PlayerChunk> chunks = world.getChunkProvider().playerChunkMap.visibleChunks;
return chunks.values().stream().map(PlayerChunk::getFullChunk).filter(Objects::nonNull).map(net.minecraft.server.Chunk::getBukkitChunk).toArray(Chunk[]::new);
}
--
2.25.1
In neuem Issue referenzieren Permalink kopieren