You know that weird feeling where your tests sometimes pass and sometimes don't, and you eventually realise they're not deterministic? But it took a while to notice because you kept changing things to debug the failing tests and only slowly realised that every "whether it succeeded or not" change didn't follow changing the code?
In this case, there were some failing tests and I was trying to debug some of them, and the result was the same every time, but only when I ran a failing test by itself and it passed did I realise that the tests weren't actually independent. They weren't actually non-deterministic in that the same combination of tests always had the same result, but I hadn't realised what was going on.
And in fact, I'd not validated the initial state of some tests enough, or I would have noticed that what was going wrong was not what the test *did* but what it started with.
I was doing something like, there was some code that loaded a module which contained data for the game -- initial room layout, rules for how-objects-interact, etc. And I didn't *intend* to change that module. Because I'm used to C or C++ header files, I'd forgotten that could be possible. But when I created a room based on the initial data, I copied it without remembering to make sure I was actually *copying* all the relevant sub-objects. And then when you move stuff around the room, that (apparently) moved stuff around in the original copy in the initialisation data module.
And then some other test fails because everything has moved around.
Once I realised, I tested a workaround using deepcopy, but I need to check the one or two places where I need a real copy and implement one there instead.
Writing a game makes me think about copying objects a lot more than any other sort of programming I've done.
In this case, there were some failing tests and I was trying to debug some of them, and the result was the same every time, but only when I ran a failing test by itself and it passed did I realise that the tests weren't actually independent. They weren't actually non-deterministic in that the same combination of tests always had the same result, but I hadn't realised what was going on.
And in fact, I'd not validated the initial state of some tests enough, or I would have noticed that what was going wrong was not what the test *did* but what it started with.
I was doing something like, there was some code that loaded a module which contained data for the game -- initial room layout, rules for how-objects-interact, etc. And I didn't *intend* to change that module. Because I'm used to C or C++ header files, I'd forgotten that could be possible. But when I created a room based on the initial data, I copied it without remembering to make sure I was actually *copying* all the relevant sub-objects. And then when you move stuff around the room, that (apparently) moved stuff around in the original copy in the initialisation data module.
And then some other test fails because everything has moved around.
Once I realised, I tested a workaround using deepcopy, but I need to check the one or two places where I need a real copy and implement one there instead.
Writing a game makes me think about copying objects a lot more than any other sort of programming I've done.