UpdatePhysics(timestep) in demoSandbox

[misho]

Hi everyone,

I was wondering what is the purpose of timestep parameter in UpdatePhysics(timestep), which is being calculated and passed in DemoEntityManager::RenderScene(), only to be ignored in the actual void DemoEntityManager::UpdatePhysics(dFloat timestep) function?

[Julio Jerez]

I have not seen it, but I think it is not ignored.
There are two time steps in the simulation.
The render time and the physics time.

The render time is the one pass to physics update.
There this time step is accumulsted and each time the accumulator overfload by the physics time step, a physics update of one fix time step is executed and one fix time step is subtracted from the accumulator until the remainder time in the accumulator is less than one descrete physics time step.

[misho]

Hi Julio!

timestep is definitely ignored, and instead, timestepInSecunds is used:

Code: Select all

void DemoEntityManager::UpdatePhysics(dFloat timestep)
{
   // update the physics
   if (m_world && !m_suspendPhysicsUpdate) {
      D_TRACKTIME();

      dFloat timestepInSecunds = 1.0f / MAX_PHYSICS_FPS;

[Julio Jerez]

oh yes you are right.
but is come to the same thing, the time step passed to the function is just that different between the
current time and the last time.
this was done long time ago, and the reason why is not accumulating the passed time in the float is because the time in float losses lots of accuracy since it is the difference between to long long integers converted to a float. instead the code uses the actual value from the microseconds timer.

Code: Select all

   if (m_world && !m_suspendPhysicsUpdate) {
      D_TRACKTIME();

      dFloat timestepInSecunds = 1.0f / MAX_PHYSICS_FPS;
      unsigned64 timestepMicrosecunds = unsigned64 (timestepInSecunds * 1000000.0f);

      unsigned64 currentTime = dGetTimeInMicrosenconds ();
                unsigned64 nextTime = currentTime - m_microsecunds;


[misho]

Ah, I see! I guess, just for clarity's sake, the parameter timestep could be removed from DemoEntityManager::UpdatePhysics(dFloat timestep).

So, for a long while now, I've been struggling to figure out why my physics is running differently depending on my visual scene complexity (visual scene is not rendered by Newton, Newton only takes care of the NewtonBodies). Namely, my rocket has a guidance to orbit which doesn't require any user interaction, so, in theory, all things being equal between simulations, I should end up in the exact same orbit. However, I noticed that the performance of my Newton physics varies with the frame rate of the simulation (which varies depending on camera angle and visual scene complexity) and in each iteration, I end up in a different orbit.

So first, all things being equal in Newton scene, should m_fps (calculated in void DemoEntityManager::CalculateFPS(dFloat timestep)) be constant, regardless of the visual frames per second?

[Julio Jerez]

I removed the argument
the physics time step should always be constant yes.

[misho]

Great, thanks Julio!

There are 2 more things that I think need to be fixed (again for clarity):

In the demo, "solver sub steps" menu option changes the m_solverSubSteps parameter and applies it, but later on a fixed value of "2" is applied. The parameter is set and applied in void DemoEntityManager::ApplyMenuOptions(), but then it is re-applied using a define hard coded to "2" in void DemoEntityManager::Cleanup (), line 577. This line should probably be removed.

Also, a minor thing, NewtonWorldSetUserData(m_world, this); is called twice in void DemoEntityManager::Cleanup ()... probably needs to be called once.

[Julio Jerez]

fixed.

[misho]

Beautiful, thank you Julio!

I am still investigating my fps vs. performance issue...

Just as a simple sanity check - I am assuming DemoEntityManager::UpdatePhysics(void) can be called in a main event loop inside my DLL, and all the timing is taken care of internally, correct?

The reason I ask is, in the demoSandbox, a call to UpdatePhysics() is defined in a callback to GL display class and buried somewhere deep inside of it. I have no idea under what condition it gets called, but I am assuming it is in a simple loop, not dependent on visual frame.

[Julio Jerez]

You can call it from anywhere yes.

[misho]

So - I have made a very simple test scene for this problem:

A rocket is resting on the ground, under gravity, with a thrust force applied to it as soon as the scene starts. The rocket slowly rises, and after 10 seconds, the thrust cuts out and the rocket altitude is printed in the console window.

I tested this with the default, out of the box demoSandbox, and the rocket altitude reaches around 320 meters. The FPS counter displays around 3000 fps.

Then, I inserted a Sleep(50) instruction just after window->RenderScene(); in void DemoEntityManager::RenderDrawListsCallback(ImDrawData* const draw_data):

Code: Select all

   window->RenderScene();

   Sleep(50); // 50 miliseconds is 20 frames per second

This effectively restricts the visual frame rate to below 20 fps. This time, the rocket in my scene reached only 66 meters, and the frame rate was shown (accurately) as between 16 and 20 fps.

Now... unless my understanding of game physics is completely wrong, the physics should NOT be affected by the VISUAL frame rate caused by the scene complexity (which I have simulated by the "Sleep()" instruction)... it should always give the same result. In both cases, my rocket should have reached exactly 320 meters, except in the later case where the visual experience would look "choppy".

If the Sleep(50) instruction is the wrong way of simulating low frame rate due to scene complexity, I can insert thousands of objects into the scene, which would then represent a truly complex scene, and run the rocket test with and without the extra objects and compare altitudes.

Somebody please enlighten me on this. I have not changed anything in demoSandbox code, except the addition of the Sleep(50) instruction.

[Julio Jerez]

That is correct and normal behavior for most applications.
I explain.

The physic update advance independent of the graphic update.
But since the physics update is called from the graphic update, the time from call to call is determine by the graphic update.
This means that every time the graphic update takes more than the time step of the physics update, the physics update catch up by execution consecutive physics steps.

This cause a few problem for most apps when the physic is the slower than the graphics.

One common problem is pause.
When and app pauses, the timer does not, so when the app resumes, the physic has to catch up for many frame .
This is usually resolved by resting the timer.

Another problem is if the physic is the one that is slower than the graphics.
In this case the physics is the one doing many steps to catch up, but in doing so make the time step even slower going into a vicious circle of getting slower and slower.

This is solved by capping the number of step the physic is alower to execute.
In the demos this number is 2.

So the simulation will not be affected by the frame rate as long as the graphics call physics updates at least once every less than 32 mili seconds.

Yo set a delay of 50 ms, so each update is throwing out 18 ms.

You can change the max number on steps the loop icon execute to 4, and them you can have a frame independent update up to 64 ms.

That way you can let the graphics be a little slower say up to 15 fps, and the physic will not be affected.
One the graph I c fall below 15 fps then it will cause the physics to do fewer steps.

I hop this is clear.

Later I show you the code where this happens.

[Julio Jerez]

ok here is how to can control that
in funtion void DemoEntityManager::UpdatePhysics() line 1348 you will fin this

Code: Select all

      unsigned64 currentTime = dGetTimeInMicrosenconds ();
      unsigned64 nextTime = currentTime - m_microsecunds;
      if (nextTime > timestepMicrosecunds * 2) {
         m_microsecunds = currentTime - timestepMicrosecunds * 2;
         nextTime = currentTime - m_microsecunds;
      }


you see that time to advance is an integer multiple of the times step, and is limited to 2.
if (nextTime > timestepMicrosecunds * 2) {

this has been the case since from 2003 when GPU where no as fast as they are now and he solve was not linear time as it is now. so it the simulation felt behind, doing more physics step will make even slower, there fore I keep the number of physics steps, to two to address the pigeon hole problem of one graphics steps needing two physics steps.

since then graphic has gone order of magnitude faster and teh solve is now linear time, but I never changed the code to match that reality, and the number of physics step remand limited at 2 max.
I just made it 10.

I made the loop the same as it is in newton 4.0 now
if you sync and try again it should be fine for up to a delay of sleep (160) which is a little over that 6 fps. after that the physics just slow down by throwing away time.

please try that and see if it gets better results.

[Julio Jerez]

so now the loop is much simpler, is basically two loops.
first add the time to teh time accumulator.
next check if the time accumulator is large than teh max number of step allow.
if so, find out how many steps the physics will have to execute beyond the max allow,
and subtract that from the time accumulator. this is the part that confused you before.
finally the reminding time in the accumulator is reduce step by step until it is lower that one time step.

in most cases this will execute one or two steps, but if the fps get slow it will catch up upto maxSteps allow.
Notice that this is only valid as long as the physics step is not what makes the system slow (both graphs and physics) because you will be trying to compensate the simulation speed with what is making it slow.

Code: Select all

      const dFloat descreteStep = (1.0f / MAX_PHYSICS_FPS);
      
      dInt32 maxSteps = 10;
      m_timeAccumulator += timestep;
      // if the time step is more than max timestep par frame, throw away the extra steps.
      if (m_timeAccumulator > descreteStep * maxSteps)
      {
         dFloat steps = dFloor (m_timeAccumulator/descreteStep) - maxSteps;
         dAssert (steps >= 0.0f);
         m_timeAccumulator -= descreteStep * steps;
      }

      while (m_timeAccumulator > descreteStep)
      {
         if (m_asynchronousPhysicsUpdate)
         {
            NewtonUpdateAsync(m_world, descreteStep);
         }
         else
         {
            NewtonUpdate(m_world, descreteStep);
         }
         m_timeAccumulator -= descreteStep;
      }


[misho]

Hi Julio!

Thank you so much for taking time to explain this, and make appropriate modifications within hours! I have compiled and tested the new version, and the results are indeed what I expected! I am a very, very happy guy today! (the difference in altitude reached in two cases is miniscule, 320 meters vs 318.6)

This issue has been haunting me for 4 years! My spaceflight sim has been performing differently under different fps circumstances (be it hardware or the complexity of the visual scenery). I always assumed this was due to the way I implemented my scene structure and the update loop (slightly differently than in demoSandbox), and I tasked myself to re-write this portion in line with what is in the demo. I did the re-write a few weeks ago, and to my surprise, it didn't help things - and that's when I started investigating. This development is SUCH a relief!

As one of my attempts to fix this on my own, I wrote a quick piece of code that spawned a thread outside my DLL event loop (my sim is a DLL that connects to a server which is basically a visual display engine) in which I called RenderScene(); in a while(true) loop. This worked, but I noticed some weird "jerking" of objects, probably due to a very small time step that was used inside the loop. This, however, begs a question: Why not run physics in a separate thread, at some user-defined fixed frame rate, let's say 200 fps, and then have the visual world simply obtain physics world state (body positions and orientations) and render it? This would be basically similar to "filming" real world... camera taking a snapshot (let's say, 60 times a second) of real world (running at 200 fps)...

I know this is an obvious concept to you and I am not proposing any new groundbreaking ideas but I was just wondering why the rendering engine isn't set up that way?