Constant AABBoverlap calls

[rvangaal]

I understand the problem of freezed objects penetrating the level once they're hit. I run my (and Newton's) physics at 1000Hz so that is often not a real problem; a cone being hit by a car may penetrate the level the first step (when colliding), but after 1 millisecond the next step will have the cone unfrozen and it will get AABB calls again (now seeing the level tree). It will not move much in that 1 millisecond normally. It would be nice if frozen objects could (optionally) be kept out of broadphase detection though (hm, how would you detect an active object hitting a frozen object then?).

On multithreading, in Newton.cpp I see the comment:

// Name: NewtonSetThreadsCount
...
// Remarks: The maximum number of threaded is set on initialization to the maximum number of CPU in the system.

but in dgWorld.cpp I see:

SetHardwareMode (1);
SetThreadsCount (DG_MAXIMUN_THREADS);
m_maxTheads = m_numberOfTheads;

SetHardwareMode (0);
SetThreadsCount (1);

So it does try to turn on multithreading (detecting the number of cores, so the max number of threads) but then sets things back to a safe no-SIMD, 1 thread environment. Is this intentional?

[Julio Jerez]

SetHardwareMode (1);
SetThreadsCount (DG_MAXIMUN_THREADS);
m_maxTheads = m_numberOfTheads;

SetHardwareMode (0);
SetThreadsCount (1);

So it does try to turn on multithreading (detecting the number of cores, so the max number of threads) but then sets things back to a safe no-SIMD, 1 thread environment. Is this intentional?

yes this is intentional in 2.00 the engine need to create the threads structures for data paraller multithreading.
This changed in core 300 it does not do that anymore.

ON the freeze I beleive I can fix it, I just the to write the function that check if a frezze body was alread has a valid contact cache and skip the broadphase update.
It will be like an sleep body, I do not have the function written yet.

[rvangaal]

I see you modified dgBodyMasterList.cpp in SVN. I'll try that. In RemoveConstraint I now see this though:

body0->m_equilibrium = dgUnsigned32 (body0->m_invMass.m_w ? false : true);
body1->m_equilibrium = dgUnsigned32 (body1->m_invMass.m_w ? false : true);

body0->m_masterNode->GetInfo().Remove(constraint->m_link0);
body1->m_masterNode->GetInfo().Remove(constraint->m_link1);

body0->m_equilibrium = body0->m_invMass.m_w ? false : true;
body1->m_equilibrium = body1->m_invMass.m_w ? false : true;

Seems like you're updating m_equilibrium twice. Probably not needed?

EDIT: btw using 4 threads slowed things down on my quadcore. But that may just be cause I have just a few contacts normally.

[Julio Jerez]

no it uptate equlibrium on the two bodies, not twice.

and yes 4 threads in some systems makes it run slower, I beleive is because quad core do not have local cache.

however in core 300 those mutithreading issues are all resolved, teh threading is diffrnet and it never slow down even if you use 100 of threads,
I limited to 32 but it is only an artificail limit.

you can try core tree hundred, see if it work for you, I beleive you do not use any of the funtionality I have no writen yet.
core 300 is way faster and more table tham core 200, and it was buidl from the ground to be mutihreaded.

bascially in core300 you can simple set it to be mutiothread and forget about it, so peopel with single core will run sevral threas juts as effiecnt as if it was not mutthreaded they get 1x performce
people with say 4 cores will get almost 4x performance (maybet 3.0x)
people with 4 core hypethreaded get the full power of four cores plus the extra percent of the hyper threaded (way more than 4x, arriund 5x performace for very dense secenes)

[rvangaal]

I tried core300 but it uses different collision trees when saving it seems (I get funny collisions when loading Newton 2.34 trees in 3.0). For now, we have a lot of content that uses the 2.34 (actually 2.29) format, so I'll leave that for now. The next version of our software (probably in half a year) would be a good point to upgrade to Newton 3.0.
Thanks for all the help!

[Julio Jerez]

Oh yes core 300 is very different, than core 200 but the collision tree is the same.
I guess you will have to reserialize teh original data, unfortunally Netwon serialization does not have revision control.

I said start core 300 as an experiment so you can see eh different, and get prepared fo rteh updtae.
at some point everything will move to core 300 and maintaneance of core 200 will stops as it did for core 100

there are many many improvement in core 300.

I slowed down because I am trying to make a GUI that is cross platform and do not have all the baggage of guis like xwidged and QT. most casual users discard the engine based on teh look of the demos.
QT is the best GUI I ever found but it has some serius problem with arbritary refresh time updates, the controllers and keyboard reponds, plus it is huge, and the the source code is a restritive licence.

as soon as I resolve the GUI issues I will resume development of core 300, and core 200 I will only fix critical bugs until core 300 replace it completly.

[rvangaal]

GUI's can be quite a pain with all the events being passed, replicated, thrown away. Sometimes quite a nightmare. But simple GUI's can stay quite simple.

I did my own revision control for the tree saving to detect incompatible tree saves. However, tree generation can take a bit of time and we haven't automated it yet, so for now I'll stick with v2.x (have a deadline coming up soon, so I want to avoid too many changes).
Hope you get through the GUI (don't try to overdo it, programming a GUI such as Windows turns out to be very, very complicated, I've been there).

[rvangaal]

I'm coming back to this thread while doing a long hunt for sudden long hangs (>10 secs) somewhere in my program. It might be the graphics card, still investigating as it takes around a day or 2 days to appear each time.

From SVN, I used v2.34 for some time. I just did an SVN update and a LOT of v2.x files have been changed. Still, newton.h states v2.34. Is there a changelist perhaps somewhere? It seems there's so many changes it would hardly be called 2.34 anymore. Bugfixes perhaps? Seemingly a lot of it is due to formatting though (new layout?).

I also tried v3.00 for a moment; but that gives so many problems with convex hull vs collision tree, it's unusable. I've taken the v3 latest update, after detecting this problem in a version from a few months back. The problem is still there; the convex hull of my cars collide in strange ways with the collision tree, creating collisions where there is none. It generates bumps that don't seem deterministic but rather they jitter when I try to drive. I can make a test environment of Racer in which you can check the problems with newton.dll yourself if you want.
I tried turning off optimize the tree, but that just makes the problem lots worse... From the wireframe I can see quads being generated; are those a primitive (I thought it would only generate triangles).

I also see a similar problem it seems as what we talked about in v2.29; callbacks keep being called for some objects (convex hull vs tree). Some cones never go to rest.

[Julio Jerez]

yes there are many chnage in core 200, but they are only formatting, a patch was added to change the format. and I forget to up the version number on core 200.
I do not think any them make change in funtionality. I also added some minor bug fixes after that, but I stope completetle since is is hard to compare 200 and 300 after the formatting was added.
after the patch is is harder to compare versions and see whan is different, but it should be possible if is is neesesary to find a new bug.

in core 300 than yes you are right I make soem chnages and I beleiev I added a bug that make contacts when there is none.
It had being difficult to reproduce because the GUI QT did not let me capture the keyboard and mouse, so than I can make a replay file when the bug happens.
I just change GUI to FOX which is open source, I should be able to fix that bug after I add the replay util I had in 1.0


are you using core 200?
also did we fix thr false call to sleeping objects before?
cal you send me a demo so than I can see what it is?

if is possible can you mak ethe demo with core 300?
It is true that core 300, is still in development but it is very far alone, and the benefict of core 300 far outweight the stability of core 200.
I rather work on solving the problem in the new core.
if you want you can make so that I build the collision tree on start and tha way will be compatible with both core 200 and 300.

[Julio Jerez]

I beileve the jiter bug with convex shapes in core 300, is because a new cool feature I added to the collision system.

basically I added a lever two cache, in newton contact persint form frame to frame, but oly when bodies are at rest.
this makes the collison very stable and predictable. however teh momen teh body start to move contact need to be flush and recalculated.

the was bothering me for a long time. and I was thinking is there was a way to extne teh persintace contact joint beyond restion contacts.
and I beleiev it is possible.

Basially the way the collision works in based on fhe Minksoky sum of two convex shapes.
in general the Minkosky theorem states that two shapes interset each other if origin is inside the minkousky sum of the two shapes.
It just happen that the Minkousky sum of two convex shapes is also a convex shape, so this special case can be used to find the first point of interstion of two convex shapes.
The minskosky sum of two shape can be build using a techique that can be use each time you have a convex functions. The method goes by different names.
Steepest decent, Hill Climbimb, Convex Programing, least square optimization, etc
(every person who came up with an algorithm of finding a gradient of a convex function, gave a different name)
you may know it from you elementary calculus when the told you that you the minimum or the maximun of a quadratic function happens when the derivative of the function becomes zero.
This cook book procedure is the most used concept is science, more than 99.99% of all mathamtical or physics problems are confine in one way or another to the solution of a constrained cuadratic function.

anyway, going back to the collsion problems, the way I build the minkoussky of the two convex shape is by buiding internal tetrahedras. this is the same a for example fanning a cpnve polygon is two d.
In two d you can make a fan of triangles for any conve polygon. so if you are making a collsion system in two D, to find the collision with plane all you need to do, is find the point closest to teh plane,
and get the triangle from that point that are closest to the ground, the collision of the polygopn will be same as teh collision of the triangle that is closest to the ground.
In thee D the triangle becomes tretraherum and the principle is the same.

so if you want to have a dynamic cache, (a First Guess to find the closest tera hedrum to the origion) all I need to do is save the last data used to find the last tretraherum of the last call.
so I added a cache that stores the 4 directions of the last trerathedrum of the minksky sum, and it uses that as the first guess next time.
It makes the contact solve about 3 to 4 time faster and more stable, since it take about 6 to 7 iterations to find the closest tetraherum to the ground,
and with the cache it takes 1 or 2 if the two bodies has not rotated too much relative to each other.

To add this I made a sustancial amount of changes to the engine, and I believe it is in one fo those changes that I intruduced the bug.

[rvangaal]

Sounds complex still. I've mailed you a link with a test setup.

[Julio Jerez]

I no longer have the old email, you can send it here
jerezjulio0@gmail.com

[Julio Jerez]

got it, what is fmodex.dll?

I will debug tonight, have to go work now.

edit oh never mind, I was running the newton forlder and I should run the main fordel.
anyway It will have to be tonigh.

[Julio Jerez]

can you remind me how to drive? try few keys but nothing happened

[Julio Jerez]

I emailed this to you. but just in case

Al right sync to svn and try the latest core 300.
I did the next best thing, basically I single out the offending function and replace with a call to the x87 counterpart, and now it runs fine.
At least I was able to run around the track without the nasty collision bug.

I did notice some vibration of the car when running in first gear and 70+ mph, but I am guessing that is part of your simulation because
I do not get any call to contact joint generations.
Please let me know if this fixes that bug.
Also I noticed that you made the double side face real geometry with thickness , that is much better very food.

as I todl you before, teh simd code depending on eh resul I get from the testion new Intel SDK, if I get equal to better preformance and teh SDK in general enough and no somethong
that will only work ona subeset of CPU, the I will deprecate teh simd path.
The simd path required too much maintance and I rther work of algorith than in teh hardwered.

So after I complete the rest of eth Core (collsion, Contineu collsion and teh Parallel solver for core 300)
I will move to evaluate openCL and the simd core will be the baselline to decide if if OpenCL is mature enought.

Hey the demo is really cool, I was driving around and I decide to go a slow speed so tha I can see teh enviromnet.
very cool.