This has been fixed, but was bad enough that we’ll be resetting.
This happened because of 2 things:
-
Bad incoming data
The new database is more strict with how it handles incoming data, and our tick code had problematic updates in some specific scenarios. When we tested our upgrade on dev, we actually caught and fixed a bunch of similar errors, but we unfortunately didn’t trip this bug specifically.
-
Not failing gracefully
The scenarios that would trip the error would cause everything else to fail mid-tick, even though the initial error itself is not critical.
Both of these things have been fixed, although the tick code itself is still at general risk of more of these type of problems because it is so old and brittle.
I am therefore going to shift my focus to cleaning up the rest of the tick code so that we can catch any related issues before they happen.
Dev testers are welcome, as always.