New Newton PEEL integration

[Julio Jerez]

ncreasing the stack size permanently is not the best way to fix this. It would be better to either replace this call with a regular allocation. If performance is an issue, a good middle ground is simply to use an _alloca() wrapper that uses either a real _alloca() or a regular allocation depending on required size.
You may have other, similar unbounded _alloca() calls elsewhere in the codebase. I would recommend fixing them all while you're at it.

that or you chane this define

Code: Select all

#define DG_ENGINE_STACK_SIZE        (1024 * 1024)

to

Code: Select all

#define DG_ENGINE_STACK_SIZE        (4 * 1024 * 1024)

stack is the best memory that you wan to use.
that define is only telling the OS to reserve a number of pages because it may use that memory, the memory is not committed unless it is used.

The reason why OS do not set the stack to all available memory is because there can be an unlimited number of thread in the single program, and if they all do that then virtual table directory will be huge.

has I set that value to 4 meg as the default you would never find out, The reason I do no is because in window the Main thread stack size is one Meg, and people use the engine single treaded. so what ever works in the newton thread should also works in a single treaded mode.

[Julio Jerez]

They are not "experts", they are just students. It is incomprehensible to me why you care about that paper, since it does not seem to mention Newton anywhere anyway. They cannot have made incorrect claims about your engine, so I just don't know what you are complaining about.
...
"PhysX consumes more GPU" doesn't even mean anything (the PhysX versions in PEEL are entirely running on the CPU anyway).

I tell why I care, I care because the internet has become what the newspapers us to be, if something is in the internet in the form of a peer reviews publication, people will trust is true. It is not the information it I the volume of Misinformation that no one seem to clarify.
For example it is papers like that that make thongs like this happens.

We are looking at **** with NAO model for a large new EU-funded robotics research project, however it would require multiple simulations to be run at once on the same box.
Do you now support, or planning to, support using a physics engine with GPU acceleration (such as PHysX) to provide sufficient power to make this possible?
I know that Bullet has bits of GPU acceleration in beta releases and so on, but this has been "just round the corner" for several years and has not really progressed.

Apparently Microsoft Robotic center, Nasa and GPL are using Physx, Bullet and ODE for designing Robot Exploration.
These falsehoods gets spread all over the place as if they were true.
and it is reason they think other libraries are not a even choice worth considering is that misinformation?
Basically the place a Fence around, that say Phsyx simulation is an exclusive club and they tell you who is invited.

[Pierre]

I would suggest my solution instead of increasing the stack size with the define, since my version fixes the issue robustly once and for all, while your version may still crash with an even larger scene one day. But it's your engine, feel free to fix it the way you want.

BTW I noticed something "funny". Remember that I was using a preview of 3.13 in my PEEL Release 1.01, right? Well... that scene with the 255*255 boxes ("SeaOfStaticBoxes3") DOES NOT CRASH there.

That's the direct proof that all your comments about how I added that last scene just to embarrass Newton were completely wrong and unjustified.

[Pierre]

I tell why I care, I care because the internet has become what the newspapers us to be, if something is in the internet in the form of a peer reviews publication, people will trust is true.

Yes. I'm aware. And PhysX has an order of magnitude more unjustified bad press than Newton online.

But these poor students just wrote some kind of report for their university courses, it's not a peer-reviewed paper at all. Just ignore them...

[Pierre]

I mean just look at the countless headlines:

https://www.google.ch/?gfe_rd=cr&ei=RoC ... ff&start=0

It took ONE stupid article online to create this mess.

So that's one reason why we focus a lot on performance these days. I am not "obsessed", we just don't have a choice. One suspicious performance problem and tons of morons come complaining that we did it on purpose.

[Julio Jerez]

its funny when the shoe is in some one else foot. One expert somehow from that deduced this.

The FPU (x87) instructions usually store intermediate results with 80 bits of precision. When legacy FPU software algorithms are ported to SSE2, certain combinations of math operations or input datasets can result in measurable numerical deviation: this is of critical importance to scientific computations, if the calculation results must be compared against results generated from a different machine architecture.

A simple recompilation of PhysX using SSE2 instruction set might lead to some incorrect calculation results, maybe this is an element of anwser…

[Pierre]

Report 6:

- had a super quick look at the "wiggle room" offset (no time) and I don't understand it. From the comment, I thought that you just wanted some space between the planks. But in fact the code now pulls the planks together instead of spacing them out. So they're created in an initially overlapping state. That cannot be correct.... What am I missing?

I would understand if the '"-" below would be a "+":

Code: Select all

// you must allow a wiggle room otherwise the bridge the mass matrix is singular leading to a explosion of engine with force base solvers
Pos += (PosOffset - Point (Extents.x * 0.01f, 0.0f, 0.0f)) *2.0f;


I tried to run with & without the change anyway, but I can't see a difference (in any of the engines).

So you were correct that I don't even understand the problem.

[Julio Jerez]

Report 6:
I would understand if the '"-" below would be a "+":

Code: Select all

// you must allow a wiggle room otherwise the bridge the mass matrix is singular leading to a explosion of engine with force base solvers
Pos += (PosOffset - Point (Extents.x * 0.01f, 0.0f, 0.0f)) *2.0f;


So you were correct that I don't even understand the problem.

to make the system stable the minus sign Is the more correct way.

yes that's the problem and that was how Kenneth Bodined design his moronic Battery of test, knowing that knowing that a direct solver will crash trying to invert a single matrix.

I make picture to explain it. Imagine you have only one plank, (the case for many planks is the same)
the mage below shows what happen.
the picture in the left is what happen when the plank fully aligned, in that location the system lose one degree of freedom, so to make general you formulate a close form solution when the a small perturbation happens. that it the image on right.

Untitled1.png (28.2 KiB) Viewed 6136 times

The mass matrix for the system in the right is given by the expression I wrote below the image.
(ignoring the inertia because is will so be part of the makes the system explode).

as you extend the system with more plank what you get is a tri-diagonal matrix in which all the diagonal element are 1.0 + e and the off diagonal element are -1 + e

basically a matrix where all minors are singular(a matrix of rank 1 embedded in a higher dimension), very much the worse that you can possible get.
In real life these things works, first that system cannot be made without the wiggle room because it will require infinite tension, but assuming you have a strong cable and it is very tight, it just happens that cables are elastic and they deform and this is a different kind of solution.

Ideally you way you would it without having to pulling the together is by placing the planks alone the path of a Catenary curve (look that up), but I did not wanted to do that. because by pulling the planks together the simulation converge to the same Catenary curve after few iterations.

This I explained to both Kenneth Bodin and Adrian Boeing, I even tried to call them out the Bullet forum when the opportunist and dishonest Erwin Coummans and his side kid Dirk Gregory, decided to cut all of my post and place then out of context in one thread.
http://www.bulletphysics.org/Bullet/php ... f=4&t=1334

so to me I am not surprised you do not know that, but the people who are making these moronic test year after year, ought to know better. Mr Kenneth Bodin does that for a living he knew what he was doing some extremely dishonest, specially after I called him out it.

He did the same with the Gyroscopic demo, The sin wave was the 100% correct solution just like newton produced, Novodex and ODE failed because they lost all the angular momentum,
mr Kenneth students where a triple of morons that did not even know what conservation of moment means.

Conservation of momentum can only happens in the absent of external forces or torque, like and asteroid floating in space.
Procession you get when there are conservative forces or torque. example a planet in orbit or a top or a gyroscope. In these cases there the is a conversion from kinetic to potential over over time, and you see the exactly the sine wave what newton showed.

These idiots did not know that conservation of momentum means the conservation happen in all three axis independent of each other. But these three morons decided to calculate the Magnitude of the momentum on tree different axis.
that's why you do not see the negative momentum in Novodex, because the three Kennett Bodin Stooges added the these axis together.
It was all a Hoax and Mr Kenneth Bodin let it go and still have not retracted that Garbage.
Instead he let more students use it a references for more test year after year.
That I call maliciously dishonest intentions..

[Julio Jerez]

I would suggest my solution instead of increasing the stack size with the define, since my version fixes the issue robustly once and for all, while your version may still crash with an even larger scene one day. But it's your engine, feel free to fix it the way you want.

it is all relative, you do the same thing I do or at leas at some point you were doing the same,
I vaguely remember that I saw a stack array the was vey large in your plug in, an sure enough there is it. you use a stack array of 2048 objects,

Code: Select all

udword PhysX::Update(float dt)
{
   if(mScene)
   {
      mScene->simulate(dt);
      mScene->fetchResults(true);
//      mScene->fetchResults(false);

/*      mScene->setFlag(PxSceneFlag::eENABLE_MANUAL_QUERY_UPDATE, gSQManualFlushUpdates);
      if(gSQManualFlushUpdates)
         mScene->flushQueryUpdates();*/

      if(0)
      {
         PxActor* Actors[2048];

the only reason you are not crashing is that by design PhysX consider static objects differently, so you do no get report for them.
IN newton 3.12 an lower all object were considered the same, it was not only 3.13 the I added the persistence broad phase that capitalize on static bodies to make some optimizations, in the search is one sided and static-static-pair is never even considered as opposed to before that the are consider just to be rejected.

[Pierre]

it is all relative, you do the same thing I do or at leas at some point you were doing the same,
I vaguely remember that I saw a stack array the was vey large in your plug in, an sure enough there is it. you use a stack array of 2048 objects,

This is not the same at all.

The code is commented out, for starters.

But even if it wouldn't, this is just a C-style array of 2048 object pointers. It's a small fixed-size static array consuming 2048 * sizeof(void*) bytes on the stack. If it works once when I run it locally, it is likely to always work since it does not depend on the total number of objects in the scene or anything.

But yours is an unbounded dynamic stack allocation through _alloca(). The allocated size depends on what users are doing with the SDK. One day or another it's going to crash.

Using unbounded _alloca() calls is not robust. We don't do that. Instead we do what I suggested, through a macro like this:

Code: Select all

#define NV_ALLOCA(var, type, count)                                                                                    \
   nvidia::shdfnd::ScopedPointer<type> var;                                                                           \
   {                                                                                                                  \
      uint32_t size = sizeof(type) * (count);                                                                        \
      var.mOwned = size > 1024;                                                                                      \
      if(var.mOwned)                                                                                                 \
         var.mPointer = reinterpret_cast<type*>(nvidia::shdfnd::TempAllocator().allocate(size, __FILE__, __LINE__));\
      else                                                                                                           \
         var.mPointer = reinterpret_cast<type*>(NvAlloca(size));                                                    \
   }


the only reason you are not crashing is that by design PhysX consider static objects differently

This is incorrect. It has nothing to do with that. It would not crash even if I turn all these objects into dynamic objects.

IN newton 3.12 an lower all object were considered the same

This scene worked fine for me in Newton 3.09, as you can verify in my PEEL release.......

[Julio Jerez]

That Macro is a good idea. I consider for 3.14
Thanks.

[Pierre]

to make the system stable the minus sign Is the more correct way.

So if I understand correctly, this is just to avoid a singularity in the exact solver, i.e. give it some breathing room to let gravity push the planks down, towards configurations that don't make the solver explode?

knowing that knowing that a direct solver will crash trying to invert a single matrix

So that's the part I'm a bit unclear about. Isn't that just a limitation of the solver? I mean in real life you can create a bridge-like structure exactly like in the test (i.e. with an horizontal default configuration), and things don't "explode". So a simulator should be able to cope with the same setup, surely?

I see your point and why the system ends up with a singular matrix, but it looks like a limitation of the mathematical model rather than something "wrong" with the test itself.

In real life these things works, first that system cannot be made without the wiggle room because it will require infinite tension, but assuming you have a strong cable and it is very tight, it just happens that cables are elastic and they deform and this is a different kind of solution.

So this I don't understand. In real life you cannot add the "wiggle room" you added, or at least not in that form, since you are creating the planks in an initially overlapping state (which of course cannot be done in real life).

In real life, I went back to my kid's room again and found a toy that could be made to look pretty much like the bridge in the test (wooden planks connected by joints, to make it horizontal you just pull on both ends with your hands. I'll try to take a picture this evening). It looks reasonable that somebody could try to reproduce this toy in a simulator, so I'm a bit surprised that this simple thing would break exact solvers (disclaimer: generally speaking I'm the collision + optimization guy, not the solver guy).

Ideally you way you would it without having to pulling the together is by placing the planks alone th path of a Catenary curve (look that up), but I did not wanted to do that.

Yeah I know that one ("la chaînette" in our french schools). I started a new test and I will build the new bridges that way - I want to see the differences.

But I'm still not sure why/if the old test is really so unfair. I mean if it exposes a limitation/singularity in some solvers, well, that's fair game isn't it? That's kind of what PEEL is exploring.

Anyway the bit I really don't get is that I still don't see any difference in practice. With or without your change, Newton's iterative solver looks fine to me. And with or without your change, Newton's exact solver seems to explode. So I'm missing something. I really don't mind changing the test to improve things but that particular change doesn't seem to help at all?

[Pierre]

Hey, just for fun.... You were mentioning "damage control" before. Well check this out:

http://www.metacritic.com/game/pc/batman-arkham-knight

Now that is some damage we have to control, because as usual gamers are quick to blame everything on Nvidia (or GameWorks, which includes PhysX) while in fact this mess has nothing to do with us.

Compared to that, and maybe at the risk of offending you (sorry), the company really, really, really doesn't care at all if some other physics engine somewhere happens to be faster or better or whatever. It doesn't affect the company. It has zero impact. They don't care. Even PhysX itself is barely on their radar these days (they consider it a "done" project, and the manpower currently allocated to it is actually quite small). So, no conspiracy...

[Julio Jerez]

So if I understand correctly, this is just to avoid a singularity in the exact solver, i.e. give it some breathing room to let gravity push the planks down, towards configurations that don't make the solver explode?

Yes but it is more than just a direct solver will blow. A direct solve will blow because a direct solve from a solution in a finite number of steps.
An iterative solver In practice does not blow off because is just happen to have singular matrix is a matrix with infinite condition number, and since the converge rate of any iterative solver is a function of the inverse of the condition number , an iterative solver will take an infinite number of steps to solve a singular matrix.
In practice what happens is that the matrix is ill conditioned (a matrix with a very large condition number) so the iterative solver will take steps that randomly may diverge in one step and move to convergence in some ethers.
If you take the bridge and you rattle it you will see that behavior. this is not a limitation of Newton it is a basic principle of numerical analysis.

So that's the part I'm a bit unclear about. Isn't that just a limitation of the solver? I mean in real life you can create a bridge-like structure exactly like in the test (i.e. with an horizontal default configuration), and things don't "explode". So a simulator should be able to cope with the same setup, surely?
I see your point and why the system ends up with a singular matrix, but it looks like a limitation of the mathematical model rather than something "wrong" with the test itself.

In real life, I said those thing appear to work, but in really the reason they work is because real life material has a stress tensor matrix, rather than a constant inertia matrix.
basically in real like materials deform, but the lagragian formulation is not longer the same than you get from the assumption that the inertia is invariant.
Newton original solver was solving an idealization of a body with a rigid material as such it will have those limitations. There are numerical models that can formulate elastic tensor matrix when assuming small deformation alone of direction. One such method is Virtual Word Of Elastic Materials that are capable of solving that problem, but I do not think anyone is doing that. Maybe one day I move Newton there.

So this I don't understand. In real life you cannot add the "wiggle room" you added, or at least not in that form, since you are creating the planks in an initially overlapping state (which of course cannot be done in real life).

yes, but this is not real life, it is an simplified model of real life, and since almost each engine has a way to resolve interpenetration, and not way to resolve singularities. Pulling the objects together is a way quick to result the problem. But I also mention that there are even better ways. In is not that I was trying to get some advantage is that those are not my test.

But I'm still not sure why/if the old test is really so unfair. I mean if it exposes a limitation/singularity in some solvers, well, that's fair game isn't it? That's kind of what PEEL is exploring.
Anyway the bit I really don't get is that I still don't see any difference in practice. With or without your change, Newton's iterative solver looks fine to me. And with or without your change, Newton's exact solver seems to explode. So I'm missing something. I really don't mind changing the test to improve things but that particular change doesn't seem to help at all?

It is unfair, because Rigid bodies are a subset of real bodies.
It is like If I says this calculator A can calculate square roots, but infernally it uses Newton Rapson iterative solvers. and some other calculator B can also calculate square root but is use a close form solver. People can use those two calculator to calculate the square root of any number and never have a problem.
Then here if comes a dishonest critic (the would be Kenneth Bodin) and he is hired by the company making the cheap calculator A that uses Newton Rapson. The they tell the dishonest critic, if you feed a negative number to the other calculator, it will crash and the one way to get back is by tuning off and on. The iterative solver will terminate in a fix number of steps before it also crashes, but they do not disclose that the critic. The Critic go on to write his report trashing the calculation that in fact did the right thing and glorifying the calculator that gave the incorrect answer 100% of the time. That report is unfair because calculators are designed to work with real numbers, had them been designed for complex numbers, calculator B will will work while calculator A will still be wrong 100% of the time.
That's what Mr Kenneth Bodin did with his test, he made test that to be solved require considering elastic materials, and he applied that to a solver for rigid body material. He knew that the iterative solver will terminate without a solution before exploding while the direct solver will crash trying to invert the system matrix.
And like I said you do not see difference because Newton now set the iterative solver as the default solver, but if you increase the number of iterations very high or you with to the exact solve it will either blow up or go in to unstable state.

The exact solve will explode because like I said the matrix does not has to be single it just need to be ill conditioned. I tried to give the smallest amount of room so that the iterative solver barely works, but even the iterative solver will do the same if it do many iterations.
I believe even your solve will blow up if you do sufficient impulse iterations.

Also for the record bridges are not flat, engineers go to great efforts to make them follow an inverted Catenary path. People thinks is parabolic arch but it is not. is it a small Catenary path.
you can use a parabolic arch if you want, is much simpler and not much difference..

[Julio Jerez]

http://www.metacritic.com/game/pc/batman-arkham-knight
Compared to that, and maybe at the risk of offending you (sorry), the company really, really, really doesn't care at all if some other physics engine somewhere happens to be faster or better or whatever. It doesn't affect the company. It has zero impact. They don't care. Even PhysX itself is barely on their radar these days (they consider it a "done" project, and the manpower currently allocated to it is actually quite small). So, no conspiracy...

Hey look on the bright side, one guy really like it, gave it a 10

10
Lucky me I got the game on PS4 and its absolutely stunning. It would look even more amazing at 60fps but I'm not one to complain about that too much. This has turned out to be a great sequel to a great series. Collapse
2 of 59 users found this helpful