BTW worlds corrupting

[Utterbob]

Hi guys,

First off I'm posting this here because I'm not 100% sure it's BTW specific, seems to be though...

I have a problem with my worlds corrupting that seems to be BTW related. I only say that because I haven't had a vMC world corruption for ages now and it's happened twice with BTW inside 2 months.

I had a play with world edit (after seeing a kind-of similar problem posted by Renovo that was resolved) but I have to confess I wasn't sure what I was looking for and nothing seemed too out of place.

Info on the problem

Spoiler

Show

The link below is my BTB world (built mostly under BTW 3.72 and now running BTW 3.91... <3 to FC for making it playable until 1.3!!) but the same thing happened to a pure BTW world not so long ago. The BTW world was new-ish so I deleted it but since this has the same symptoms I don't think its BTB specific.

I can fix it (sort of) with the good-old-recover-MC-wolds-that-crash-on-saving problem that used to be pretty common - by generating a new world, copying region/DIM-1 folder to the new world and using something like invedit to copy the inventory across. Basically a short-cut way of generating a new level.dat.

What's a little odd compared to your average corruption is that this behaves like a memory leak but rather than getting progressively worse as I play it gets worse each time the world is loaded. I am pretty sure its over-realm related as loading when returning from the nether progresses the problem the same way as leaving the game and loading the world up again.

Also odd is the fact that after returning to my base from a random position on recovering the world there are 2 significant freezes that appear to be something taking a huge amount of time to load, after these 2 go the world is fine until a random number of loads. So far I have not returned to base from a direction that allows me see what chunks are loading and causing it. It is also only once per recovery (running away from my base and back does not trigger it again).

Anyway, not sure how much of that is of any use whatsoever so I might spoiler this out!

World

Spoiler

Show

http://www.mediafire.com/?pdo252v50f0o4sz

If anyone has the knowledge to help, or can help me find it I would be very grateful! I don't want to lose the world!

[Graphite]

Well, I can confirm that something is going on atleast. The first time I loaded the world it chugged along at a consistent 0 fps (if I had to guess it was around 0.2 or so). Funnily enough the second time I loaded it, it loaded and ran just fine.

One thing I noticed between the two loads was that you had 1100+ entities on the first load, yet only 357 on the second one. I seem to recall that collision-detection grows exceedingly worse as more entities occupy the same spot. Could it be there are some 700 mobs/animals/whatever stuck in a 1x1 hole somewhere?


Update: Just tried loading the world on vanilla minecraft and the lagging reproduces just fine. Let's see whether I can hook up a debugger to it so I can see what the heck it's doing. I've been meaning to look into the minecraft code anyhow :P


Update 2: Well, the first hint presented itself quickly enough. Seems you have some skeletons in your closet...

Spoiler

Show

Wrong location! net.minecraft.src.EntitySkeleton@1b1
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Unknown Source)
at net.minecraft.src.Chunk.addEntity(Chunk.java:796)
at net.minecraft.src.World.updateEntityWithOptionalForce(World.java:2240)
at net.minecraft.src.World.updateEntity(World.java:2163)
at net.minecraft.src.World.updateEntities(World.java:2037)
at net.minecraft.client.Minecraft.runTick(Minecraft.java:1897)
at net.minecraft.client.Minecraft.runGameLoop(Minecraft.java:870)
at net.minecraft.client.Minecraft.run(Minecraft.java:801)
at java.lang.Thread.run(Unknown Source)

Now the question is whether this would be cause or effect of the corruption...


Update 3: Iiiinteresting. The skeletons it is bitching about are located at -147, 17, 119 (a 1x1 square on the wall of a ravine). That's not the interesting part though. The moment I enter that chunk I get:

Spoiler

Show

Wrong location! net.minecraft.src.EntityPlayerSP@5ab
at java.lang.Thread.dumpStack(Unknown Source)
at net.minecraft.src.Chunk.addEntity(Chunk.java:796)
at net.minecraft.src.World.updateEntityWithOptionalForce(World.java:2240)
at net.minecraft.src.World.updateEntity(World.java:2163)
at net.minecraft.src.World.updateEntities(World.java:2037)
at net.minecraft.client.Minecraft.runTick(Minecraft.java:1897)
at net.minecraft.client.Minecraft.runGameLoop(Minecraft.java:870)
at net.minecraft.client.Minecraft.run(Minecraft.java:801)
at java.lang.Thread.run(Unknown Source)

The skeletons are definitely an effect, not a cause. The chunk seems to think it's at (-7,8) even though it's actually at (-10,7). Now the question is why the chunk seems corrupted and what caused that corruption (and whether it is the cause of any further corruption).


Update 4: I'm a bit out of my depth on figuring out the reason for the corrupted chunk, so if anyone else has any ideas, then I'd love to hear them. I suspect it might explain a lot though, as you'd have no issues whatsoever until a mob spawns in that particular chunk, at which point it starts flipping out.

[Graphite]

Separate post so the above wall of text doesn't drown out my question:

When you say that it corrupts your world, in what manner does it do that? Does it just get laggy and unresponsive or does it actually mess up blocks and such?

P.S. A workaround for the load-delay is to set the game to peaceful beforehand. Ofcourse, it's inadvisable to continue playing until the corrupted chunk has been resolved.

[Utterbob]

EDIT: Sorry for the wall of text, I'm not sure how much of this is useful so I'm leaning towards more information rather than less!
EDIT 2: Another interesting point... just made a backup and went to the co-oord you gave me... it's an explored cave with the light level probably low enough to spawn mobs but defiantly not 0. There are also some impossible water flows in that area, not something that I would have necessarily notice while exploring but now that I'm looking for it, it's going perpendicular to a source in one area, now falling over an edge in another, flowing through a block unimpeded and things like that.

Just sifting though your comments there are some interesting things popping up... I had suspected entities were part of the problem but couldn't rule out coincidental performance improvements.

Well, I can confirm that something is going on at least. The first time I loaded the world it chugged along at a consistent 0 fps (if I had to guess it was around 0.2 or so). Funnily enough the second time I loaded it, it loaded and ran just fine.

I would suspect that is the one off load I was referring too, on my rig (which happens to be a beast of a machine) it takes a good 60-120 seconds, twice. After that it's normal for a while, quitting and reloaded seems to skip the second wait if I do between the two.

One thing I noticed between the two loads was that you had 1100+ entities on the first load, yet only 357 on the second one. I seem to recall that collision-detection grows exceedingly worse as more entities occupy the same spot. Could it be there are some 700 mobs/animals/whatever stuck in a 1x1 hole somewhere?

Nothing other than the things you go on to mention further into your post. Having said that I have a large number of mobs in the farm that I assume you have seen (should have loaded in the general area of it) but there's defiantly no intentional 1x1 packing anywhere.

The skeletons are definitely an effect, not a cause. The chunk seems to think it's at (-7,8) even though it's actually at (-10,7). Now the question is why the chunk seems corrupted and what caused that corruption (and whether it is the cause of any further corruption).


Update 4: I'm a bit out of my depth on figuring out the reason for the corrupted chunk, so if anyone else has any ideas, then I'd love to hear them. I suspect it might explain a lot though, as you'd have no issues whatsoever until a mob spawns in that particular chunk, at which point it starts flipping out.

Is removing that chunk in world edit going to resolve it? I can replace the hole with whatever, just to close the big hole in the middle of nowhere when mining/exploring thing.

When you say that it corrupts your world, in what manner does it do that? Does it just get laggy and unresponsive or does it actually mess up blocks and such?

The more accurate statement would have been that it corrupts my level.dat. It reduces performance to the point of freezing, if i hit escape it will take time (more loads = more time) for the menu to pop up but everything will go normally (no lag) behind the menu (same goes for inv, chests, etc.). Leaving any of those UI's and going back to the game view takes it back to no frames. The more times the world has been loaded the worse it is. To the best of my knowledge no blocks have ever been affected.

Oh and as a point of clarification; when I say it reduces performance... performance will be completely consistent until the world is loaded again (return from nether or opening the world from the SP menu. Not all loads reduce performance perceptibly (though there may be a subtle drop) and at some point (a random number of loads) the game will go from ok to unplayable FPS, within 2-4 loads after that the world simply won't load.

And thank you for your efforts so far! Anything that helps me get closer to the solution is very appreciated!

[Graphite]

I had another look at your world and realised that I had gotten an error-message saying it relocated a chunk because it was in the wrong position as well, though unfortunately it sorta drowned in the entity-errors. The chunks involved were (surprise surprise): (-10,7) and (-7,8).

From the looks of it, chunk (-10,7) seems a copy of the (-7, 8) chunk. I'm guessing that removing the one at (-10, 7) (it extends from (-144, 127) to (-159, 112)) and letting it regenerate might solve the issue. If not, try removing the other one as well. Fortunately both seem to be just outside your base.

As for the performance degrading: I suspect it somehow duplicates the mobs within the errored chunk. So the first time you save, you save one skeleton. Then you reload and it duplicates it, so the next time you save, you end up saving two skeletons. You reload it again, and then there are 4, 8, 16, 32, 64, 128, 256, etc. (all in the same spot, too).

Incidentally: I don't think this is BtW related.

[Whisp]

This really sounds a lot like the problem RenEvo had with his BTB world.
After deleting the problematic chunks with MCEdit the problem was fixed and the chunks regenerated the correct terrain at that location on loading. To my knowledge it didn't appear again after that.

If you do this, you should make sure to let the analyze function run over your complete world. In RenEvo's world the problematic chunks where right next to his base, but at the border of the map were two additional chunks with similar error messages. I didn't walk out there in minecraft to check if they cause the same problem ingame, but if you don't have any player creations in the affected chunks, I would recommend deleting them with MCEdit until you have a clean analyze run on your world.

[Utterbob]

You guys are awesome, I love this community!

Thanks a lot, I will get into the chunk deletion now and let you know how it goes.

As you say it doesn't seem BTB or BTW related, I guess I just got lucky with vMC worlds and/or unlucky to get two of these so close.

Update 1: Ok interesting, the performance issue has been occurring since before the -10, 7 chuck, that one is new, the way it extends that 1 high grass into the water... well wasn't there a week or so ago, I swam that bay a couple of times as a short cut.

Update 2: The offending chunk at -10, 7 is gone, the regen worked but something is weird with the terrain, not the end of the world, will just do some landscaping :) If i get anymore problems I will focus that area though!

Thanks again guys so happy to have that world back and working properly!

[Graphite]

I think it's only one, so do the (-10,7) one first. The other was just in case I misunderstood the signs :P

[fearofshorts]

Just wondering, does anybody know what the probable cause of this was?
The server my girlfriend plays on has had corruption issues in the past, and I'm curious as to why it happens.

Is it MC screwing up and writing invalid data? Or is it more related to actual OS or data storage failure?

[Graphite]

If it was random corruption of the file, I'd expect other strange things to happen other than misplaced chunks. The odds are awfully low that the same few bytes always happen to get corrupted in some way. As such, I suspect it might be a bug in MC somehow, but I honestly have no idea where and how exactly.

[Utterbob]

I think in this case it was probably more the corruption happened, screwed with chunk, performace issue happened later.

Once I got far enough into the performace degridation the chunk was well established and 'saving' the wrold the old fashoned way was really just wiping the symptom out, not the cause. So the corruption was actually one off.

As far as I know MC has issues when anything happens to it while a chunk is saving (world corruption is VERY common if you have the out of memory issue regularly). The reverse is also true, if anything messes with your chunk on the HDD while MC has it loaded in memory, it's toast (some drives auto correcting firmware, for example, can corrupt a world).

Aslo Graphite, I was meaning to ask... what did you use to identify the problem chunk(s). World edit was not helpful in that area, I was able to ID it from the co-ords you gave and the fact that once I knew it's general location it stood out quite a bit.

EDIT: P.S. and yes with MC world corruptions my knowladge of cause is disproportinatly greater than my knowladge in resolution :P

[Graphite]

I was stepping through the minecraft code, actually. It's a pity that all the errors it was producing at that time aren't logged normally. Some of them were pretty useful in figuring out what was going wrong.