Lack of determinism?

[JoeWright]

Julio, is there any reason that when closing and recreating the Newton world, flushing, creating the bodies exactly the same and applying the same forces, that the simulation may not be exactly the same two times running? Do multiple threads cause a lack of determinism?

I'm struggling to explain a lack of determinism that I'm currently seeing.

Thanks, Joe

[Aphex]

Do you have a fixed timestep?

[Stucuk]

If i destroy everything in the world (Except the world), i can have the simulation react the same as it did last time. Never bothered to try destroying the world its self.

[JoeWright]

Yes fixed time stamp. If its not something anyone else has noticed then I'll have to look into my data more closely. Its generally a small effect over, say, a couple of hundered iterations.

[Dave Gravel]

Fixed time and NEWTON_API void NewtonInvalidateCache (const NewtonWorld* newtonWorld); is suposed to work good.
Working good here.

[JoeWright]

That's what I'm using. I guess there's a bug in my code somewhere.

[Dave Gravel]

Generally when you use this command anytimes a simulation is start,
and if you get again the problem it coming from the fixed times major part of the times .

[JoeWright]

Absolutely deterministic with 1 thread, sometimes not with 2 threads.

See http://www.newtondynamics.com/forum/viewtopic.php?f=12&t=5063 for more info.

[Dave Gravel]

Maybe it have a problem somewhere but it working good here with 4 threads.
If the times is not fixed correctly it's sure with 1 or 2 threads the result don't come the same.

[Marc]

For me, this doesn't work.

0. Setting up the scene, iterating 1000 times, then resetting the body positions, calling NewtonInvalidateCache(), reiterating 1000 times, the resulting scene is different.
1. Setting up the scene, iterating 1000 times, then destroying and recreating the bodies like before, calling NewtonInvalidateCache(), reiterating 1000 times, the resulting scene is different.
2. Recreating the whole world instead works => the resulting scenes are equal for me

I used public beta 2.0 for my tests. Interestingly, the scenes were all equal after 100 iterations, this might be related to my test scene consisting of 4 boxes being close to each other and later on being apart. My Threadscount was 1. I also tried calling NewtonInvalidateCache() at several positions between or after recreating of the bodies or at both places as well as using the same NewtonCollision or releasing it and creating a new one. Sometimes this led to different results of the 2nd iteration, but never the same as the 1st one (except for case 2 where It recreated the whole world).

Am I using it wrong? Is NewtonInvalidateCache() actually ment to solve this problem or is it ment to be used for something else?

Best regards
Marc

[JernejL]

what platform architecture setting are you using? to ensure proper determinism you should only use x86 with no extension, cpu floating point unit precision is not guaranteed otherwise.

[Marc]

Is that even possible?

NewtonGetPlatformArchitecture() returns 0 and the strings is filled with "x87".

[JernejL]

There is also a NewtonSetPlatformArchitecture which sets that.

Have you validated that your application always sends identical numeric values to newton? i know a few cases on this forum where people succesfully used newton in deterministic way, but you should make sure your app is applying same forces and not using functions like random() or anything that depends on variable time.

[Marc]

I checked and I couldn't find anything. I post my testcode. At the beginning there is a function to display the current state as well as calculate a hash value of it for easy comparision. Then there are the physics callbacks. DeterminismTest0() is the actual test. It creates a scene consisting of 4 equal bodies, all at (0, 0, 0). The first 3 get the phys callback set that applies 0 force to them, only on the 4th there will be a force of (0, 0, -10). Then there is the actual iteration of the state up to step 1000. enclosed with printf("start 0\n"); and printf("end 0\n"); Afterwards, I try to recreate the state exaclty as it was before. The loop over the 4 bodies is the same, I experimente with setting some things like resetting the PhysCallback, but it didn't matter. After that, there is the iteration again, this time enclosed by printf("start 1\n"); and printf("end 1\n");. I tried calling NewtonInvalidateCache(nWorld); at several places, but it didn't lead to the desired effect. Here is the output:

Code: Select all

0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
>>> 2
00000000  00000000  00000000    00000000  00000000  00000000
0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
>>> 3
00000000  00000000  00000000    00000000  00000000  00000000
0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
Hash:   0       00000000
end 0
>>> 0
4132492e  c0592d65  c2ca1998    3fce8d8a  bef93f0a  c2125e2d
11.142866  -3.393396  -101.049988       1.613694  -0.486809  -36.591969
>>> 1
c07215a8  414c901e  c2c9903d    bf069ed4  3fe9d3d2  c2123681
-3.782572  12.785185  -100.781715       -0.525861  1.826777  -36.553226
>>> 2
c0fefd28  404d1e40  c2ca0ee7    bf9bf06e  3ef756dc  c21298d6
-7.968403  3.204971  -101.029106        -1.218275  0.483085  -36.649254
>>> 3
3fb4b57e  c1b9e18c  414e9b63    3ccec646  bedc6af5  3e75cf97
1.411789  -23.235130  12.912936         0.025241  -0.430504  0.240050
Hash:   88304293        05436aa5
--------------------------------------------
start 1
>>> 0
00000000  00000000  00000000    00000000  00000000  00000000
0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
>>> 1
00000000  00000000  00000000    00000000  00000000  00000000
0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
>>> 2
00000000  00000000  00000000    00000000  00000000  00000000
0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
>>> 3
00000000  00000000  00000000    00000000  00000000  00000000
0.000000  0.000000  0.000000    0.000000  0.000000  0.000000
Hash:   0       00000000
end 1
>>> 0
4139dc7f  403350d3  c2cab903    3fdb0242  3ed77835  c212969d
11.616332  2.801808  -101.361351        1.711006  0.420839  -36.647083
>>> 1
c099be5d  413c42a1  c2c9c0a2    bf37add5  3fe2e116  c2126711
-4.804488  11.766267  -100.876236       -0.717496  1.772494  -36.600651
>>> 2
c10d8677  c06adc6c  c2ca31f3    bf9356fc  bef70856  c212abbc
-8.845328  -3.669703  -101.097557       -1.151092  -0.482485  -36.667709
>>> 3
4035f0a9  c1a7f263  c0c7dccb    3d538558  bec763e0  bdec6c5a
2.842814  -20.993353  -6.245702         0.051641  -0.389434  -0.115441
Hash:   2024876462      78b129ae


You can see the initial states are the same, always everything 0 for the 4 bodies. After running the simulation, the results differ. Note that simulating the scenen only up to 100 iterations, the result is the same. Also note that when I run the program several times, the output is always the same. So it's not completely random somehow. Here is the code:

Code: Select all


#include "stdafx.h"

#include "Newton.h"
#include <float.h>

void OutputOneBody(float *pos, float *vel, float *omega)
{
   int j;

   for (j=0;j<3;j++)
      printf("%08x  ",((unsigned int*)pos)[12+j]);
   printf("\t");
   for (j=0;j<3;j++)
      printf("%08x  ",((unsigned int*)vel)[j]);

   printf("\n");

   for (j=0;j<3;j++)
      printf("%f  ",pos[12+j]);
   printf("\t");
   for (j=0;j<3;j++)
      printf("%f  ",vel[j]);


   printf("\n");
}


const int n=4;
NewtonBody *body[n];


void OutputState()
{
   float pos[16],vel[3],omega[3];
   int i, j;
   unsigned int hash = 0;
   for (i=0;i<n;i++)
   {
      printf(">>> %u\n",i);
      NewtonBodyGetMatrix(body[i],pos);
      NewtonBodyGetVelocity(body[i],vel);
      NewtonBodyGetOmega(body[i],omega);

      OutputOneBody(pos,vel,omega);

      for (j = 0; j < 16; ++j)
      {
         hash ^= ((unsigned int*)pos)[j];
      }
      for (j = 0; j < 3; ++j)
      {
         hash ^= ((unsigned int*)vel)[j];
      }
      for (j = 0; j < 3; ++j)
      {
         hash ^= ((unsigned int*)omega)[j];
      }

   }
   printf("Hash:\t%u\t%08x\n", hash, hash);
}

void PhysicsApplyForceAndTorque0 (const NewtonBody* body, dFloat timestep, int threadIndex)
{
   float f[3];
   f[0]=0;
   f[1]=0;
   f[2]=0;
   NewtonBodyAddForce(body,f);
   int i = (int)NewtonBodyGetUserData(body);
}

void PhysicsApplyForceAndTorque1 (const NewtonBody* body, dFloat timestep, int threadIndex)
{
   float f[3];
   f[0]=0;
   f[1]=0;
   f[2]=-10;
   NewtonBodyAddForce(body,f);
   int i = (int)NewtonBodyGetUserData(body);
}


void DeterminismTest0()
{
   _control87(655391,0xffffffff);

   float pos[16],vel[3],omega[3];
   int i;
   for (i=0;i<16;i++)
      pos[i]=0;
   pos[0]=1;
   pos[5]=1;
   pos[10]=1;
   pos[15]=1;
   for (i=0;i<3;i++)
      vel[i]=0;
   for (i=0;i<3;i++)
      omega[i]=0;


   NewtonWorld *nWorld=NewtonCreate(0,0);
   int threadsn = NewtonGetThreadsCount(nWorld);
   NewtonSetThreadsCount(nWorld, 1);

   NewtonCollision *col=NewtonCreateBox(nWorld,1,1,1,0);

   for (i=0;i<n;i++)
   {
      body[i]=NewtonCreateBody(nWorld,col);
      NewtonBodySetMassMatrix(body[i],1,1,1,1);
      if (i == 3)
         NewtonBodySetForceAndTorqueCallback (body[i], PhysicsApplyForceAndTorque0);
      else
         NewtonBodySetForceAndTorqueCallback (body[i], PhysicsApplyForceAndTorque1);
      NewtonBodySetAutoSleep(body[i],0);
      NewtonBodySetFreezeState(body[i],0);   // NewtonWorldUnfreezeBody(nWorld,body[i]);
      NewtonBodySetMatrix(body[i],pos);
      NewtonBodySetVelocity(body[i],vel);
      NewtonBodySetOmega(body[i],omega);
      NewtonBodySetUserData(body[i], (void *)i);
   }

   NewtonReleaseCollision(nWorld,col);


   printf("start 0\n");
   OutputState();

   NewtonInvalidateCache(nWorld);
   for (i = 0; i < 1000; ++i)
   {
      NewtonUpdate(nWorld,0.016f);
   }

   printf("end 0\n");
   OutputState();

   printf("--------------------------------------------\n");

   NewtonInvalidateCache(nWorld);
   for (i=0;i<n;i++)
   {
      if (i == 3)
         NewtonBodySetForceAndTorqueCallback (body[i], PhysicsApplyForceAndTorque0);
      else
         NewtonBodySetForceAndTorqueCallback (body[i], PhysicsApplyForceAndTorque1);
      NewtonBodySetAutoSleep(body[i],0);
      NewtonBodySetFreezeState(body[i],0);   // NewtonWorldUnfreezeBody(nWorld,body[i]);
      NewtonBodySetMatrix(body[i],pos);
      NewtonBodySetVelocity(body[i],vel);
      NewtonBodySetOmega(body[i],omega);
      //NewtonBodySetUserData(body[i], (void *)i);
   }

   printf("start 1\n");
   OutputState();

   NewtonInvalidateCache(nWorld);
   for (i = 0; i < 1000; ++i)
   {
      NewtonUpdate(nWorld,0.016f);
   }

   printf("end 1\n");
   OutputState();


}


int _tmain(int argc, _TCHAR* argv[])
{
   DeterminismTest0();

   _getwch();

   return 0;
}


[agi_shi]

Get rid of the invalidate cache calls and anything unrelated to the body's starting position/rotation/velocity.

All you should need to do for a deterministic setup is reset the bodies to their initial position/rotation/velocity (this consists of NewtonBodySetMatrix, NewtonBodySetOmega, and NewtonBodySetVelocity). Don't invalidate the cache, don't reset the callbacks, don't touch those.