Deterministic Setup

[Marc]

Hi !

I'm still having trouble getting a deterministic simulation running in Newton 2.0x. There are several threads discussing this topic as well, stating different observations - none includes example code. Can someone post a newton initialization code that is supposed to create a deterministic behavior?

Best regards
Marc

[JernejL]

Some of the info about this that i collected is on wiki under NewtonInvalidateCache page.

[Marc]

Hmm, there is no code on that page as well

I'll post mine current intialization as a start

Code: Select all

   nWorld = NewtonCreate(0, 0);
   NewtonSetThreadsCount(nWorld, 1);
   NewtonSetPlatformArchitecture(nWorld, 0);
   NewtonSetMultiThreadSolverOnSingleIsland(nWorld, 0);
   NewtonSetSolverModel(nWorld, 2);
   NewtonSetFrictionModel(nWorld, 1);


According to your last post, does that mean I have to call NewtonInvalidateCache as well like this?

Code: Select all

   nWorld = NewtonCreate(0, 0);
   NewtonSetThreadsCount(nWorld, 1);
   NewtonSetPlatformArchitecture(nWorld, 0);
   NewtonSetMultiThreadSolverOnSingleIsland(nWorld, 0);
   NewtonSetSolverModel(nWorld, 2);
   NewtonSetFrictionModel(nWorld, 1);

        NewtonInvalidateCache(nWorld);


[agi_shi]

No, you don't need to invalidate the cache in the beginning. Your setup should be fine with deterministic simulation.

[JernejL]

from how i understand it, you only call invalidatecache when you add or remove bodies.

[Marc]

I tried around and can't get newton deterministic for me. I have no idea what the problem could be.


There can't be much being done wrong because my scene consists of a heightfield, a few static spheres(newton)/capsules(bullet), and n moving spheres/capsules. There are no complicated set up meshes or anything. No vehilces, ragdolls etc.

There is one thing though: I have multiple newtonworlds in parallel. Only one of them I try to use deterministic and the others are not important to be deterministic and get created and destroyed and iterated etc. in an undeterministic way. Is it possible that these worlds have side effects on the world that is supposed to be deterministic?

[Julio Jerez]

Like I said there was a problem in Invalidatecache and I fix it for 2.04
I know for fact Newton is 100% capable of repeating the same results, I myself use that property for catching bugs by making movies that save the Key and Mosue control,
and so far it has always worked. This is in the SDK. I also know of some apps that do use mutiple worlds on mutiplayer mode and it seems to work fine.
A way to determine the problem is by post and test since I do not have all of the resource to make a test for every one posting a bug.

However if you solved by switching engine, that is perfectly fine too.

[Julio Jerez]

I see you did ported a test.
viewtopic.php?f=9&t=5054&p=36742#p36742

I will check it out.

[Julio Jerez]

OK for whatever it is worth and for my peace of mind I found the Bug and fixed it.
There was another bug in InvalidateCache.
I was not cleaning the Muti Resolution Broad Phase.
I short this is what Happened, In Newton what you add a body to the world, the body is added to the top grid of the broadPhase, Then during the update bodies a place to the grid that the belong.
Your test was a god test because in exposed the problem.
When you added the bodies for the first time they were in the top grid, the solver start moving the around as the simulation progresses.
The when you cal invalidate, bodies and the contacts are all cleaned up, but the Broad Phase was not,
That should not be a problem because the next update should place then in the correct Cell, and in fact is does do that, but here is the problems, the first time it place the bodies for the root cell to the cell the belong, but the second time in move the form various cells to the cell that the belong.

To make clearer let us label the bodies
A, B, C, B and they are all in the Top cell say Cell 0
After the first update let say the scene is organizes
A, B in Cell 15, C in Cell 100, and D in Cell 101

The second Update body B moves to 100 and we get
A in Cell 15, B, C in Cell 100, and D in Cell 101

Now we want to rest the Scene by calling Invalidate Cache,
And we run a new Update

Now the scene start form where the Last update ended, which is
A in Cell 15, B, C in Cell 100, and D in Cell 101

The first update find that body B belong to cell 15, so in moves body B to cell 15,
But since body a and B have the same position it gets
B, A in Cell 15, C in Cell 100, and D in Cell 101

Instead of Getting
A, B in Cell 15, C in Cell 100, and D in Cell 101

And that lead to a generation of the same colliding pairs but in different order.
In the first pass the pair will be, AB, AC, BC, …
While in the second pass they will be BA, BC, AC…

I fixed quite simple, basically I remove all the bodies from the world and re-insert then again, which causes then to go to the top cell, then update will guarantee to move the bodies from the same origin.
Anyway the bug is fixed and if you still want to use Newton you can download 2.04 them it should work fine now.
Here is the summary of you test

start 0
>>> 0
Hash: 0 00000000
end 0
>>> 0
Hash: 31432400 01df9ed0
--------------------------------------------
start 1
>>> 0
Hash: 0 00000000
end 1
Hash: 31432400 01df9ed0

[Marc]

Thx Julio, for caring about my problem. I tried the new version of Newton 2.04 beta and the test works for me as well now.

However, in my app, I still have determinism problems - but they occur less often. So I tried to recreate it by a test. After a bit of fiddeling, I was able to still create undertermined behaviour by changing the newton world initialization in the test from before to the way I do it in my app:

Code: Select all

   NewtonWorld *nWorld=NewtonCreate(0,0);
   NewtonSetThreadsCount(nWorld, 1);

   NewtonSetPlatformArchitecture(nWorld, 0);
   NewtonSetMultiThreadSolverOnSingleIsland(nWorld, 0);
   NewtonSetSolverModel(nWorld, 2);
   NewtonSetFrictionModel(nWorld, 1);


I tried with different number of bodies and everything went fine. But then I tried executing the test multiple times without restarting the programm. With low numbers of bodies (like 10), it's still equal, but with 100 (maybe with less as well, I didn't try) I get this

Code: Select all

start 0
Hash:   0       00000000
end 0
Hash:   4262197200      fe0bf7d0
--------------------------------------------
start 1
Hash:   0       00000000
end 1
Hash:   4262197200      fe0bf7d0
-----------------------------------------------
-----------------------------------------------
start 0
Hash:   0       00000000
end 0
Hash:   4278299797      ff01ac95
--------------------------------------------
start 1
Hash:   0       00000000
end 1
Hash:   4278299797      ff01ac95
-----------------------------------------------
-----------------------------------------------
start 0
Hash:   0       00000000
end 0
Hash:   2214679939      84015583
--------------------------------------------
start 1
Hash:   0       00000000
end 1
Hash:   2214679939      84015583
-----------------------------------------------
-----------------------------------------------
start 0
Hash:   0       00000000
end 0
Hash:   2048059816      7a12e9a8
--------------------------------------------
start 1
Hash:   0       00000000
end 1
Hash:   2048059816      7a12e9a8


Two lines of ----- represent a new call of the test function. So the newtonworld gets recreated completely. So rewinding a newtonworld itself with the use of NewtonInvalidateCache(nWorld); seems to work now, but creating new worlds, they seem to behave differently when the scene is filled with many bodies.

This might be related to the problem in my app because as I said I have multiple worlds and the one that is supposed to be in deterministic also has around 100 bodies when the problem occurs. Also I actually destroy and recreate the world when replaying - I don't just use the same all the time. When replay a simulation, that happens in a new world.

If you are willing to investigate this, I'ld greatly appreciate it. If you can't recreate the new modified test, I can send you my code - maybe I missed some modification but I don't think so.

[Julio Jerez]

I pasted this into the test

Code: Select all

//   int threadsn;
//   NewtonWorld *nWorld=NewtonCreate(0,0);
//   threadsn = NewtonGetThreadsCount(nWorld);
//   NewtonSetThreadsCount(nWorld, 1);
//   NewtonSetSolverModel(nWorld, 1);

   NewtonWorld *nWorld=NewtonCreate(0,0);
   NewtonSetThreadsCount(nWorld, 1);

   NewtonSetPlatformArchitecture(nWorld, 0);
   NewtonSetMultiThreadSolverOnSingleIsland(nWorld, 0);
   NewtonSetSolverModel(nWorld, 2);
   NewtonSetFrictionModel(nWorld, 1);

and I set n to 100, and I get this result

end 0
Hash: 2048059816 7a12e9a8
--------------------------------------------
end 1
Hash: 2048059816 7a12e9a8

I do not get the rest of yuor post, can you change the test to one that generates the problem?
Polisble using mutiples worlds.
There maybe other memory hidden in some place but it is dificul to find without test to reproduce it..

[Marc]

Hi !

Just change the main like this:

Code: Select all

int _tmain(int argc, _TCHAR* argv[])
{
   DeterminismTest0();
   DeterminismTest0();
   DeterminismTest0();
   DeterminismTest0();

   _getwch();

   return 0;
}


1 DeterminismTest0() works deterministically in itself, but the result from the independent DeterminismTest0() calls lead to different results.

[Julio Jerez]

Ha I see, I added this

Code: Select all

int main(int argc, char* argv[])
{
   DeterminismTest0();
   DeterminismTest0();
   DeterminismTest0();
//   DeterminismTest0();

   _getwch();

   return 0;
}

this is the result

end 0
Hash: 2048059816 7a12e9a8
end 1
Hash: 2048059816 7a12e9a8

end 0
Hash: 2048059816 7a12e9a8
end 1
Hash: 2048059816 7a12e9a8

end 0
Hash: 92463216 0582e070
end 1
Hash: 92463216 0582e070

when few world are creaated on worl may lead difrent resul than other.
I will check it out

[Marc]

Exactly that's the point!

Thanks for investigating.

[Julio Jerez]

Well I try the optimization but I mess up. I broke my own rule, never make and optimization while fixing a Bug.
I will leave the optimization for beta 2.05 and I use the simple solution.

Here is the summary of the run now with four worlds, and 100 bodies and lopping 1000 time.

end 0
Hash: 2162975673 80ec63b9
end 1
Hash: 2162975673 80ec63b9

end 0
Hash: 2162975673 80ec63b9
end 1
Hash: 2162975673 80ec63b9

end 0
Hash: 2162975673 80ec63b9
end 1
Hash: 2162975673 80ec63b9

end 0
Hash: 2162975673 80ec63b9
end 1
Hash: 2162975673 80ec63b9

Please try again and let me know if you find some other problem.