UpdatePhysics(timestep) in demoSandbox

[Julio Jerez]

Hi Julio!

I know this is an obvious concept to you and I am not proposing any new groundbreaking ideas but I was just wondering why the rendering engine isn't set up that way?

like I said, the loop was designed about 20 years ago, at a time frame rate was not what it is today,
you struggled to make 60 stable, under that circumstances you can not afford to spend 10 simulation loops to catch with the frame rate, you only allow one or two update at a fix time step and the extra is to make for the case where it needs two physic consecutive physic updates and you throw the rest of teh time away. today the picture is very different.

but in any case, you has the problem solved now.

[misho]

Hi Julio, and everyone!

I have been working on my spaceflight simulator, and with the latest changes discussed in this thread (timestep vs. varying fps), my flight dynamics have vastly improved! Things are behaving very consistently and predictably, with respect to flight dynamics. I dare to say that there is a certain hugely popular commercial spaceflight simulator out there with literally millions of users that would be envious of astrodynamics model as provided by Newton Dynamics!

However (and this is a very small "however", almost to the point of being nitpicky )

There is still a slight discrepancy in the physics engine performance between low and high fps scenarios. I have mentioned it in the results in my test case, where my simple rocket would reach 318 meters in low FPS situation, and 320 meters in high FPS situations. This slight discrepancy results, in my sim, in rather large differences in final orbital parameters. Basically, with the exact same (automated) guidance of the rocket, the flight ends in the orbital apogee (highest point of the orbit reached) between 370 and 430 km (60 km diff). This seems large, but when taken into the context of Earth diameter of ~6400 km, 60 km is rather small and in line with the results of the test case I mentioned above. This difference is based SOLELY on the slight fps differences, depending on how I move the camera around during the powered flight phase.

I would (for obvious reasons) prefer that my physics would consistently yield the same orbital parameters at each end of the powered flight and the ascent to the orbit. I can just imagine my users asking me why, with all things being equal as far as guidance and rocket performance, aren't they getting the consistent results, as they should, and me having to explain to them that my physics engine is slightly sensitive to changes in fps and this results in variations of the resulting orbital parameters.

So - I was just wondering, is there anything else I could tweak to fine tune the timestep processing and narrow this already small dicrepancy? Perhaps something specific to my application of Newton Dynamics: small number of discrete bodies (~15 or so, all the parts in the rocket held together) and high velocities of the object "assembly". Any thoughts would be appreciated.

The parameters I am using are identical to the demoSandbox, except I am using PhysicsUpdate() in synchronous mode.

[Julio Jerez]

What I do not understand is why you keep talking about different simulation rate. Are you running the simulation at variable frame rate?
That's question one.

Question two.
For what you said, error is actually quite small.
You have to remember that a numerical simulation will always have some integration error.
There are two classes of numerical integration methods.
Mainly implicit and explicit.
Explicitly are easiest and these are when to advance the integration you take the derivative at the current time and move along the derivative as if it was a straight line.
These methods have the property that they assually add energy to the system. This is because integration, mathematically means to calculate the area of the trapezoid under under the small step bound by the line of the derivayive. And since the line along the derivative moves away from the real path of the curve, the areas under the small step is larger than the actual area.

The other method is implicit.
This is when you estimate the derivative at time t +dt, and use that as the derivative at time t.
Just there, we start with accepting a numerical integration error, but this is acceptable in return of more stability, but not more accuracy.
Basically by taking the derivative at time t +dt and using it to calculate the area under the tangent, now this areas will be always smaller than the actual area under the curve. This is the secret that explains why implicit method are more stable but not more accurate, they throw away energy by definitions.

Newton uses a mix of the two methods.
This is, for arrangement of conected bodies, Newton uses an explicit method name Runge Kutta 4, that calculates the derivative four time along the path, but at each step the derivative are calculate at time t + dt/4
Then these derivatives is used at time t.
Then after the entire arrangement is advanced four time at dt/4, these derivatives are averaged and used as the derivative at time t, to advance arrangement.

This is by far the more accurate and stable method I have developed over the time I have develop the engine.
This method has allowed for Newton to reproduce even the more complex behaviors of the dynamic of rigid bodies while still allowing for the flexibility that reduced coordinate systems methods are known for.
Newton can reproduce simulation with large mass ratios, corriolis and gyroscope acceleration. Even complex behaviors like teeptops and rattleback are possible.
But the only thing it can not guarantee is exact numerical integration accuracy.

So I am afraid the only way to improve numerical accuracy is by having a high simulation rate.

So unless you can show that there is an actual bug, I am not sure what to say other than to simulate at a high enought fix frame rate.
Your Sim seem small enought than the physics can be run at a high number of sub steps.
We talk about this before. Run at some step like 60 fps and set a number of subsets like 4, 5 or whatever you need. There is always a point of diminishing return, but on the bright side, more substeps has fewer computational cost.

This let you come close variable fps, but instead is variable frame rate. You do a fix frame rate with an adaptive some of sub steps.

The only part that is difficult is to determine how to measure the error so that you can determine when to adjust the number of substeps.

Frankly, I would just run at a fix fps and a fix substep that is high enought to meet your accuracy expectation.
For the demos you has show me this should not be a problem.

[Julio Jerez]

...This, however, begs a question: Why not run physics in a separate thread, at some user-defined fixed frame rate, let's say 200 fps, and then have the visual world simply obtain physics world state (body positions and orientations) and render it? This would be basically similar to "filming" real world... camera taking a snapshot (let's say, 60 times a second) of real world (running at 200 fps)...

I know this is an obvious concept to you and I am not proposing any new groundbreaking ideas but I was just wondering why the rendering engine isn't set up that way?

oh I forget to answer this, yes that's an obvious decision and it is what I would do is I was making a game, but newton demos are meant to be teaching tool.
making it work like that add complexities that are outside the scope of the demos. It fact doing the the frame interpolation is already outside the demos, and to many users make it seems as part of the engine when it is not. I do it because the alternative is jerky movement.

but yes, you do that all you need to do is create a thread and in the thread loop just call NewtonUpdate indefinitely every time step, an have some signal to tell the thread to exit the loop.
some people do use that approach.

you still need to add logic for when to add, remove or manipulate stuff like teleporting or set engine parameters, but for the there is the NewtonWaitToFinish to help with that before you pause the update.

[misho]

Hi Julio! Sorry for taking the time to respond, I wanted to digest and implement your suggestions! Thanks again for taking time and write a detailed explanation! To start off,

What I do not understand is why you keep talking about different simulation rate. Are you running the simulation at variable frame rate?

No - not running the sim at the variable frame rate. When I say "simulation frame rate", I'm talking about visual frames per second, not physics FPS. I was calling the UpdatePhysics() in the main dll loop event that is called when VISUAL frame update happens - just like in the demoSandbox. I run physics at a fixed 60fps time step.

I have since moved the UpdatePhysics() out of the Visual frame update event, and into the main DLL loop, which executes at a much higher rate. The error, as small as it was, shrunk even further, and the way you explained it (thanks!) makes perfect sense! And to be clear - there IS NO BUG. All I was asking is if there is a way to further shrink the already small discrepancy (which, in astrodynamics terms, seems large by the virtue of how orbital mechanics work)

Other thing I realized is that I am running some of my guidance algorithms on a visual update frequency, and I suspect that these also contribute to the discrepancy, based on the sheer fact that their response/reaction time varies based on visual FPS. This is totally on my side, and up to now I didn't think it mattered, but more I think about it, I realize that it does.

So, in short, things look great at this point! As for playing with MAX_PHYSICS_FPS and MAX_PHYSICS_SUB_STEPS settings, right now they are at a demoSandbox "default" setting of 60 and 2 respectively. I will follow your advice and play around with a few combinations until I find that "sweet spot" Thanks again!

[Julio Jerez]

In my opinion you should try with more than two sub step.
In the demos I use two. But the demos the bodies are expected to move at a relatively low velocities.

Your simulation the velocities are quite high and because you have few connected bodies with joints the internal force are very non linear.
The only way to reduce integration error is
1. If the funtion are smooth, then it can take a long time step.
2 if the function are not smooth, ex high velocity and omega, the using smaller step.

Your are in case two.
Deviding the time step by two results in a four time improve in accuracy, so you can probably get it with 3 or 4 substeps.