New Newton PEEL integration

A place to discuss everything related to Newton Dynamics.

Moderators: Sascha Willems, walaber

New Newton PEEL integration

Postby Pierre » Thu Jun 18, 2015 8:44 am

Hello,

So I'm starting this thread to discuss the issues & findings from the new Newton integration to PEEL.

For the background info, see:

https://github.com/Pierre-Terdiman/PEEL/issues/3
viewtopic.php?f=9&t=8836

So far I downloaded the new 3.13 version (yesterday) and managed to compile it without using SSE4 (so it doesn't crash for me, confirmed by running the demos at home).

I didn't make much progress yet because it takes forever to answer forum posts in other threads :) (and I have little time to devote to PEEL these days).

It might take some time because it looks like even the PEEL code has changed quite a bit (new virtual functions in the PINT interface), and of course my own "trunk" version of PEEL has also changed significantly (with, you guessed it.... new virtual functions in the PINT interface).

I asked a question in the other thread that remained unanswered (which versions should I use?), so I just selected 3.13 because I don't know where to download 3.12 anyway. If you want multiple versions, let me know. My current plan is to keep the old 3.09 from the last integration, and add a new integration for 3.13. That way we can see the improvements on the Newton side, and on my side why PEEL is a cool tool to compare multiple versions of the same library.

So. That's all for now I guess. Stay tuned.
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Re: New Newton PEEL integration

Postby Julio Jerez » Thu Jun 18, 2015 9:14 am

Like I said in the pass I used to make label releases in Google SVN, but google does not allowed that anymore.
so after we move to github, I did not know how to do labels so there were some versions that were lost becasle I only added comment in the commentaries field but not tags.

Now I know how to make tags label in Github, if you go to github you will find version 3.12 and 3.13 tagged as releases
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Julio Jerez » Thu Jun 18, 2015 11:50 am

Pierre wrote:My current plan is to keep the old 3.09 from the last integration, and add a new integration for 3.13. That way we can see the improvements on the Newton side, and on my side why PEEL is a cool tool to compare multiple versions of the same library.
So. That's all for now I guess. Stay tuned.


that will be fine, you can add 3.12, 3.13 which are already there, you can get code from and since you have 3.09 you can put that one too, I myself would like to see how newton progressed form 3.09 to 3.13
other version in between would be difficult since Newton was never that organized it was rather a continue stream of commits, some versions were stable and land marks like 2.06 I think but some like 3.11 came out with serious defect that I have to disown. There is not QA in newton either
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Pierre » Thu Jun 18, 2015 2:52 pm

Edited post:

Obviously we cannot cooperate on this as I would have hoped, but I still want to keep Newton in PEEL in future releases. So I will use this place to write my notes, so that everybody is aware of what I am touching/changing, and there is no surprise when the next version arrives.

While I will take care of it myself and take my own decisions now, I may take constructive criticism and remarks into account. But anything hateful, aggressive, or that does not conform to the agreement I accepted when joining this very forum, will be ignored.

First report:

- the sleeping deactivation does indeed work now.

- the per-object CCD parameter will be removed, replaced with the UI-defined flag (as for the other plugins). The rationale is that I want to evaluate the cost of CCD on heavy/large scenes, not just on the few scenes in the CCD category.

- the comments about the "abomination that are PhysX materials" will of course be removed. Generally speaking the project is not related to PhysX, and while the Pint interface was loosely based on how things worked in PhysX, it is independent and decoupled from it. As it turns out, I already had a note in my TODO list to replace the *PINT* materials with something much closer to how things work in my engine (ICE). It is unlikely to happen for the next version though, since it is quite low priority.
Pierre Posts: 30Joined: 16 Jun 2015 12:40 am

Re: New Newton PEEL integration

Postby Pierre » 19 Jun 2015 02:57 pm

Report 2:

- the scene-query functions (raycast, etc) seem to use multiple threads when the user selects multiple threads in the UI. This is not how the other engines work: they all implement single-threaded queries, and PEEL itself calls them from multiple threads if needed (for example in the raytracing test). So I will change that.

- the new shapes should be implemented for all engines supporting them (Havok, Bullet, etc). I don't know how much work that is, so I may remove this for now and keep that for a later version. Same for the new joints.

- there seems to be 2 new unknown caps: mCollisionUniformScaling, mCollisionNonUniformScaling. As far as I can tell there are not used, so they will be removed.

- the BeginCreateObjectHint/EndCreateObjectHint functions in the Pint interfaces don't seem to be used, so they will probably vanish. Similarly, it doesn't look like the new "creationHintHandle" parameters are needed. Will probably vanish unless I see it's used for something useful.

- I see there is a new mSupportVoronoidDecompositionGeneration cap, and new test scenes to Voronoi fracture. I have to look at the code closer. This may or may not go, I don't know yet.

- I see there are limits for the prismatic joint now. This is generally good but I need to implement them in all engines. This one may have to wait.
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Re: New Newton PEEL integration

Postby Pierre » Fri Jun 19, 2015 7:08 pm

Report 3:

- the gyroscopic precession test is good and kept. It reminds me of the old spinning tops demo in Novodex. The test description will change since it contains incorrect remarks.

For example:

no spinning is a sign that the engine is not a physically based simulator only a collision system that resolve inter penetration using impulses


This is incorrect. First, this newly introduced function has only been implemented in Newton:

Code: Select all
pint.SetLinearAndAngularDamp(Handle, 0.0f, 0.0f);


It must be implemented in all engines, obviously, otherwise they use different/non-zero default values.

Second, various engines have other default values for other parameters, that need to be setup. For example in PhysX the maximum angular velocity is limited, and this is the main reason why the tops don't spin. This simple call on dynamic bodies fixes it:

Code: Select all
rigidDynamic->setMaxAngularVelocity(100.0f);


Once properly setup, the PhysX / Havok behaviors are similar to Newton's. Hence, it has nothing to do with using impulses or not, and the text must be changed.
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Re: New Newton PEEL integration

Postby Pierre » Fri Jun 19, 2015 7:14 pm

Report 4:

- contrary to what has been claimed in another thread, the SphereMeshUnitTest has been modified. So PhysX 3.3.1 was not broken after all, its behavior changed simply because the test changed. This will be removed:

Code: Select all
// this is a disingenuous test because some engine use solve contra threshold tolerances at the at this side sphere, so setting the small
// size object an so setting the tolerance lead to a apparent malfunctions when is reality it is no the case.
// Newton 3.12 and lower will fail the test because the contact tolerances have a fix threshold that will have to be set a compilation time,
// Newton 3.13 solves that problem because the tolerance is calculate form the geometry size.
// for the test I am setting the value a little larger, because the test is about contact generation no internal settings.
//const float Radius = 0.036977537f;
const float Radius = 0.07f;


Contrary to what the comment says, the test is not "disingenuous" (?). This is a repro case for a bug, sent to us as-is by a customer (mesh data & sphere size). We cannot change that, for obvious reasons.

But it seems to be a great test since it exposed issues in both PhysX and Newton. FWIW with the original radius put back, the sphere still jitters a bit in Newton 3.13.

- the "wiggle room" offset in the bridge scene is interesting. I will have a look at the behaviors with and without the offset in all engines, to see the effect. It is likely to stay, unless I spot some other issues created by the offset.
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Re: New Newton PEEL integration

Postby Julio Jerez » Fri Jun 19, 2015 7:49 pm

I knew you will be back, you are a warrior.
You can change all those things, just remember but you making the cuts, you are the facto changing the plugin to limit the competitor to only who what make you look good and ignore the rest.
That is why I asked “if speed was the only part that count”. I also said Newton was a far more feature rich engine than any one on that list. (particles soft body stuff is not physics)
Now you are deciding again to make it a "low common denominator comparison".
I only added a limited set of standard functionality.

You may think to yourself you are doing a fair comparison, but when you added that last 255 x 255 array, you quickly jumped to celebrate how cool your stuff was because no one else did that.
That was why I added the Hint stuff so that the high level let the engine know that those objects were special. That is how it would work in 3.12
By 3.13 I added the persisted broad phase because I know the scene collision only users how are invested in the engine use it, the casual user only want to add bodies and see how it work, so I had to have a way to deal with that, and this is why it not used in that 3.13 plug in.
Let the record show that the comparison is a "Low common denomination feature set show".
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Julio Jerez » Fri Jun 19, 2015 8:05 pm

Pierre wrote:This is incorrect. First, this newly introduced function has only been implemented in Newton:
Code: Select all
pint.SetLinearAndAngularDamp(Handle, 0.0f, 0.0f);

Not true, It is in newton since day one and I added to the plug in I turned in.
I made a big deal of that, I want to make sure that what happen 9 years ago.
buy mr Kenneth "Battery of test" did not happens again
why I said that you guys want to see what you want to see.
That was how back then the experts falsely discredit Newton to favor ODE and Noveded, by not setting the angular damp to zero. and the concluding this:
Kenneth students wrote:As shown by the results this test is hard for ODE and Newton. When running the simulation a little longer, the Newton toolkit will make the object stop moving. The
simulation with ODE will explode. The documentation for ODE(section 12.12) explains
this problem and gives some suggestions on how to make a simulation more stable. The
suggestions are not used in this test since then the physical aspects wont be fair to
the other engines. Novodex handles this test best, but still does not conserve angular
momentum.
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Julio Jerez » Fri Jun 19, 2015 8:39 pm

Pierre wrote:- the "wiggle room" offset in the bridge scene is interesting. I will have a look at the behaviors with and without the offset in all engines, to see the effect. It is likely to stay, unless I spot some other issues created by the offset.

So in the end you too are into cam of Boeing and Kenneth, if you do not add room for the structure to deflect the system will explode because the system matrix is singular.
Would you like to bet money on that? Do I have to make the mathematical prove that the test is Garbage without the the wiggle room?
That too was how back them Kenneth put on that paper concluded on the garbage paper the newton engine was inferior to novodex, when was Novodex that fail all the test and newton the who passed.
I though you guys knew that and remain silenced because it favored you, but now I am starting to believe you guys do not even understand the problem. If you do not add the wiggle room all you are doing is proving how good your solver is at poor convergence.


Pierre wrote:- contrary to what has been claimed in another thread, the SphereMeshUnitTest has been modified. So PhysX 3.3.1 was not broken after all, its behavior changed simply because the test changed. This will be removed:

Code: Select all
// this is a disingenuous test because some engine use solve contra threshold tolerances at the at this side sphere, so setting the small
// size object an so setting the tolerance lead to a apparent malfunctions when is reality it is no the case.
// Newton 3.12 and lower will fail the test because the contact tolerances have a fix threshold that will have to be set a compilation time,
// Newton 3.13 solves that problem because the tolerance is calculate form the geometry size.
// for the test I am setting the value a little larger, because the test is about contact generation no internal settings.
//const float Radius = 0.036977537f;
const float Radius = 0.07f;

No true either when you pass a radio in engines like Physx, havoc and Bullet the radio is augmented by an implicit margin. Newton does not add collision margin. So I look at the default collision margin and added the extra size so the two ball are the same size.
So yes the test was disingenuous.
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Julio Jerez » Fri Jun 19, 2015 9:29 pm

Pierre wrote:- I see there is a new mSupportVoronoidDecompositionGeneration cap, and new test scenes to Voronoi fracture. I have to look at the code closer. This may or may not go, I don't know yet.

- I see there are limits for the prismatic joint now. This is generally good but I need to implement them in all engines. This one may have to wait.


No even joint limit, want what else, are you going to change?
I was jut warming up with standard stuff before moving to more avoidance things.
but I see it takes a big whiff to match newton in' it?
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Julio Jerez » Fri Jun 19, 2015 10:22 pm

Pierre wrote:- the comments about the "abomination that are PhysX materials" will of course be removed. Generally speaking the project is not related to PhysX, and while the Pint interface was loosely based on how things worked in PhysX, it is independent and decoupled from it. As it turns out, I already had a note in my TODO list to replace the *PINT* materials with something much closer to how things work in my engine (ICE). It is unlikely to happen for the next version though, since it is quite low priority.


I hope you realize that by limiting the competitors to the reduced set of features supported by PhysX, the test is in fact not independent at all, I mean we have to beat you in your own turf.
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Pierre » Sat Jun 20, 2015 8:42 pm

No new report yet. Didn't have time.

you are the facto changing the plugin to limit the competitor to only who what make you look good and ignore the rest.


With all due respect, you are making assumptions, which is what you criticized in others. You are assuming that my only goal is to make PhysX look good. This is wrong.

Moreover, you do not seem to be reading what I write. I said that things like new shapes would have to wait for the next version, because I have to implement them in all engines. For a shape like a cone, implementing them in all engines except PhysX is the opposite of making PhysX look good. Thus, this is wrong again.

Please note that PEEL is useful to compare engine A to engine B. To do a comparison, you need to have at least 2 engines exposing the same feature. Otherwise this becomes just a sample or tech demo for Newton. And that's your job, not mine.

I am forced to tell you the same as what I already told the Bullet people who complained before you: if you are not happy with my work, feel free to create your own framework with your own tests and your own comparisons. It is probably the quickest and most efficient way to prove your points.

I also said Newton was a far more feature rich engine than any one on that list. (particles soft body stuff is not physics)

Out of curiosity, what feature do you have in mind here, that others do not have?

In any case, a feature only supported by a single engine is of little interest to me for PEEL, since, again, I would have nothing to compare it to. Obviously I cannot add to PEEL each unique feature of each supported engine.


when you added that last 255 x 255 array, you quickly jumped to celebrate how cool your stuff was because no one else did that.

It is easy to prove this wrong. Just go back to my initial blog posts from 2 years ago:
http://www.codercorner.com/blog/?p=864

Even Bullet (and others) can handle that scene, so there would have been no reason to "celebrate" anything.

The Newton integration did not exist at that time (you complained loudly enough about it), so there was no way for me to know that Newton would crash here.

Let the record show that the comparison is a "Low common denomination feature set show"

I am fine with that. To do a comparison, you need to have 2 elements indeed, i.e. some common feature set. Otherwise, again, it's just a show case for one particular engine.

Not true, It is in newton since day one and I added to the plug in I turned in.

It is in Newton, yes. But it is not in other plugins. Look for that function in your own PEEL master branch, it is only implemented in Newton. So unless I'm missing something, what I wrote was true.

That was how back then the experts falsely discredit Newton to favor ODE and Noveded, by not setting the angular damp to zero

As far as I can see this is exactly what you just did as well. You did set the angular damping to zero in Newton, while others kept using their default non-zero value. I'm sorry but from where I sit you just did exactly the same.


No true either when you pass a radio in engines like Physx, havoc and Bullet the radio is augmented by an implicit margin. Newton does not add collision margin. So I look at the default collision margin and added the extra size so the two ball are the same size.

The balls are not the same size. If you change the test in the PEEL code, the same modified radius will be passed to all engines. Newton will still not add a collision margin. The others will still add a collision margin. Thus the radius will be different anyway - no matter what initial radius you put in the test code.

But that's fine. That's exactly the point of PEEL: checking what different engines do when given the same input data. That's a comparison.

So yes the test was disingenuous.

No, it cannot be: we did not even write that test ourselves. You are again not reading what I write. The data came verbatim from a customer. It's their values from their game. We cannot change that. If they would switch their physics engine from PhysX to Newton out of "desperation", they would see exactly what PEEL is exposing here. That's the point of the tool. The tool is working.

No even joint limit, want what else, are you going to change?

You are once again not paying attention. I wrote that this addition was good. But I need time to implement it in all engines. As for what else I am going to change, I will let you know here in my next reports.

With all due respect, you are damaging your reputation on your own forum, in front of your own users. Please stop. I am still trying to work on this with you here.
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Re: New Newton PEEL integration

Postby Julio Jerez » Sat Jun 20, 2015 11:28 pm

Did not you just said you were cutting many of the Newton demos because you guys do not support them? In my book that means changing the goalpost in middle of the game
You had not problems adding those support feature flags so that you can embarrass those other engines that did not supported the things physx did, but when you see that Newton supported them and added a couple of more, then you do not like it and you change the rules. it not that hypocrisy?

The thing is that it was not the Blow out you expect to be, you see that some unknown Joe blow working alone on weekends an with very little, gives your Multimillion Professional library a run for its money, and you are in Damage control and try to make it seems that It did it wrong.
That is how people who use the Newton engine use it. Are you try to say that they should use differently?

The self appointed experts has already started making pseudo paper and Master thesis base on your results from you PEEL tool. from the paper:
http://web.cse.ohio-state.edu/~boggus/5 ... tation.pdf
● Physics Engine Evaluation Lab (PEEL)
compared engines memory consumption
○ Bullet 2.8.1 is worse than PhysX 3.3
○ PhysX 2.8.4 and earlier was substantially
better at sweep tests than Bullet
● PhysX consumes more GPU, Havok
consumes more CPU


Of curse as Usual the same way it happened with those other papers, No matter how much I protest and Say that the test is bias, the "expert" will no see as legitimate and you cannot have that.
David has a PAL version that demonstrate what Adrian did was dishonest, but he ignored pretending it does not exist.
Julio Jerez
Moderator
Moderator
 
Posts: 12426
Joined: Sun Sep 14, 2003 2:18 pm
Location: Los Angeles

Re: New Newton PEEL integration

Postby Pierre » Mon Jun 22, 2015 9:43 am

Did not you just said you were cutting many of the Newton demos because you guys do not support them?


Hmmm? No, I don't think I did. I said "because I need time to implement them in all engines". It takes a lot of time for me to do a new PEEL release, first because at best I only have a couple of hours to work on it in the evening, and second because each feature potentially needs to be implemented in 20+ PINT plugins. It takes forever and I'm already lagging behind.

You had not problems adding those support feature flags so that you can embarrass those other engines


You mean the caps bits? They are not here to "embarrass" anybody, obviously, they are here to avoid crashes, random behaviors, and pollution of performance graphs with bogus results. They are not just a reflection of what each engine supports, but also (and mainly) a reflection of what the PINT plugin currently exposes. Setting something to "false" in there is nothing to be ashamed of.

when you see that Newton supported them and added a couple of more

I never added anything there because of Newton. It should be obvious that I add new caps there when I add a new feature to PEEL. For example in PEEL's "Known issues & TODO" text file, there is this TODO:

- add support for heightfields (all engines)

When I do that, I will probably add a new cap to the structure for heightfields, and it will be set to false for everybody at first. Then I will implement it for PhysX first, simply because that's the engine I'm most familiar with. At that point, if it remains "false" for Newton, it obviously doesn't mean that Newton does not support heightfields. It just means that I may not have the time to add proper support for it myself (this is especially true for Newton since that's the API I know the least).

The thing is that it was not the Blow out you expect to be, you see that some unknown Joe blow working alone on weekends an with very little, gives your Multimillion Professional library a run for its money, and you are in Damage control and try to make it seems that It did it wrong.


(shrug) You are wrong.

There was no "blow" (expected or perceived).

I fully expect that hobby projects can sometimes be better than large ones (having been there myself, and being old enough to know that adding more people to a project usually makes things worse, not better).

There is no "damage" to control either, I'm not quite sure what you are talking about. Newton is a good engine, but your own PEEL_master branch exposes its limits and weaknesses (for example performance of large scenes and memory usage, which are two common things PhysX users seem to care about). I think Newton & PhysX may have a different target audience.

In any case, I'd say competition is good, not something to be "afraid" of. If engine X is significantly better than "us" at something our users care about, it's a good drive for us to go back to the drawing board and improve our product. Everybody wins. Without competitors, we may not have any incentive to do so.

Finally, people, please keep in mind that we are the ones releasing PEEL. We are the ones giving you a direct way to see for yourselves, instead of believing online myths/rumors. Obviously, we are not afraid of anything you can find. I will even point out that we didn't bother removing the tests where PhysX performs poorly. Contrary to what you may have been led to believe, we are honest and fair people. For example look up the "capsule stack", "high mass ratio" or "initial penetration" tests: both Havok & Newton behave better than PhysX there, and yet we didn't try to hide it.

That is how people who use the Newton engine use it. Are you try to say that they should use differently?

I'm afraid I don't know what you're talking about.

The self appointed experts has already started making pseudo paper and Master thesis base on your results from you PEEL tool.

They are not "experts", they are just students. It is incomprehensible to me why you care about that paper, since it does not seem to mention Newton anywhere anyway. They cannot have made incorrect claims about your engine, so I just don't know what you are complaining about.

● Physics Engine Evaluation Lab (PEEL)
compared engines memory consumption
○ Bullet 2.8.1 is worse than PhysX 3.3
○ PhysX 2.8.4 and earlier was substantially
better at sweep tests than Bullet
● PhysX consumes more GPU, Havok
consumes more CPU

FWIW they don't know what they're talking about. PEEL compares more than memory usage, there was no earlier version than PhysX 2.8.4 in PEEL, and "PhysX consumes more GPU" doesn't even mean anything (the PhysX versions in PEEL are entirely running on the CPU anyway).

David has a PAL version that demonstrate what Adrian did was dishonest, but he ignored pretending it does not exist.

I have no idea who "David" is, or why you keep talking to me about this stuff...
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Re: New Newton PEEL integration

Postby Pierre » Mon Jun 22, 2015 9:49 am

Report 5:

- the 255*255 boxes scene crashes because of a stack overflow due to an unbounded _alloca call in this function:

Code: Select all
void dgWorldDynamicUpdate::UpdateDynamics(dgFloat32 timestep)


Probably that line (I only looked at the disassembly so can't say for sure):

Code: Select all
m_islandMemory = (dgIsland*) alloca ((masterList.GetCount() + 256) * sizeof (dgIsland));


Increasing the stack size permanently is not the best way to fix this. It would be better to either replace this call with a regular allocation. If performance is an issue, a good middle ground is simply to use an _alloca() wrapper that uses either a real _alloca() or a regular allocation depending on required size.

You may have other, similar unbounded _alloca() calls elsewhere in the codebase. I would recommend fixing them all while you're at it.
Pierre
 
Posts: 21
Joined: Tue Jun 16, 2015 4:40 am

Next

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 0 guests

cron