Static Mesh missing collisions

[Esharc]

Good morning Julio.

For a while I have been trying to figure out why our vehicles fall through the static mesh occasionally. But I have not been able to figure it out. Today I was able to reproduce the issue in one of your Sandbox demos.

I have attached the two files that I modified to get the condition. When you load the "Basic Compound Shapes" demo, you will see that I have added a wall to the static bvh collision shape and removed all other collisions except for 1 camel.

You will see that the camel falls through the static bvh mesh. But if you remove the two lines at 181 and 182 of ndMakeStaticMap.cpp in the BuildFlatPlane function then the camel will not fall through the mesh.

You will also notice that I made the optimize flag false for the test. It seems to happen when there is a height in the AABB, or maybe it is because the object is already in the AABB of the mesh? I am not sure.

I at first thought it was the optimize flag that I had to make true in my code, and that did work for certain collisions, but not all, there are certain areas of the terrain where it still misses collisions. The collision mesh that we are loading is quite large 8000 x 8000 x 350.

If I have missed any important information in my summary of the issue, please let me know. I hope this is just a silly mistake on my side when creating the mesh on my side

ChangedFiles.7z

(5.28 KiB) Downloaded 242 times

[Esharc]

Sorry, here are updated files, I updated my version of Newton and saw that there were quite a bunch of changes, so I had to update the files.

These should just work if you replace the files in the "toolbox" and "demos" folders

ChangedFiles.7z

(5.33 KiB) Downloaded 214 times

[Julio Jerez]

ok
I take a look.

edit:
yes, I can reproduce it, it seems that simply adding the extra vertical wall causes malfunctions.
that looks really bad.
I debug it.

[Julio Jerez]

Wow, my friend,

You found a true needle-in-a-haystack bug!

The issue lies in this function:

ndInt32 ndContactSolver::CompoundToShapeStaticBvhContactsDiscrete()

When the function calculates the distance to the closest node, it computes the squared distance. However, upon exiting, it should take the square root of that value.

It turns out that instead of correctly applying the square root, the function mistakenly copies the squared distance directly.

I wondered why this issue didn’t appear in the other variations of the same function. The answer? This was the only function in the class with the bug.

The reason this bug rarely caused noticeable failures is that, in most cases, the computed distance is less than one unit. Since squaring a number less than one results in an even smaller number, the impact was minimal. Essentially, the function was performing unnecessary work, but it didn’t cause major issues.

However, when the distance is significantly larger—like in the demo—the computed value became much bigger than the actual distance. As a result, the broad-phase logic incorrectly classified the joint contact as inactive.

it is rare that a bug fix also translates to performance gain

if you sync, it souel be fine now.

[Esharc]

Thank you Julio,

I have updated and tested it on my side and it is working correctly now. Such a simple fix, it was staring at me in the face and I could not see it.

I am glad we got more performance out of the fix as well.

[Esharc]

Good afternoon Julio,

I have found another problem. This time I have added the actual static mesh that we are using that we get the problem on.

In this demo, the box starts on a platform that is inside the AABB bounds of the static mesh. When I move it after two seconds to a spot on the static mesh, it falls through the mesh a couple of times before it starts colliding with it.

The file StaticMeshData.txt has the vertex and index data for the static mesh and should be placed in the ndSandbox folder to be loaded.

Also for this demo to work I had to make some changes in ndContactSolver.cpp, because the ndAssert(closestDist2 < ndFloat32(1000.0f)); and ndAssert(closestDist2 < ndFloat32(1.0e6f)); that you have in for the CompoundToShapeStaticBvhContactsDiscrete() and CompoundToConvexContactsDiscrete() respectively was triggering. I assume it is because I am moving the body a large distance at a time?

Unfortunately for our AI, we have to move them large distances occasionally. Is there a better way that I should be doing that?

The folder is too big to load here, so I have added it to a Google drive folder that you can access here: https://drive.google.com/drive/folders/ ... sp=sharing

[Julio Jerez]

some questions.
do I have to use the old demo.

also this line

Also for this demo to work I had to make some changes in ndContactSolver.cpp, because the ndAssert(closestDist2 < ndFloat32(1000.0f)); and ndAssert(closestDist2 < ndFloat32(1.0e6f)); that you have in for the CompoundToShapeStaticBvhContactsDiscrete() and

what that mean is that if tow objects are too far apart, the contact solver is no longer accurate.
if you teleport an object and the brad phase is not aware, them the proxy aabb on the object in the broa phase will not match that aabb of the body. that why that assert is there.

I agree 1000 is probably too small, the sqrt(1000) is about 30 units, so I will chat that to 10000
so that test will be 100 units.
1e-6 is certainly and unreasonable value

Unfortunately for our AI, we have to move them large distances occasionally. Is there a better way that I should be doing that?

how are you moving the object?
is the object static?
how do I reproduce that?

[Esharc]

Hi Julio,

No the old demo will not reproduce this issue, you will have to copy the files that I uploaded to the Google drive folder.

When we move the object, we are using SetMatrix function of the body to move it, it is not static.

You will not have to do anything, when you copy over the files from the Google drive folder the object will move from its current position to a new position automatically after 2 seconds, and will keep getting placed at the new position every 2 seconds.

Here is the link to the Google drive folder again just for ease of access, let me know if you cannot access it.

https://drive.google.com/drive/folders/ ... drive_link

[Julio Jerez]

Ah, I see the issue now.

As I suspected, setting the matrix is not updating the broad phase. I was sure there was a way to handle this, but perhaps some refactoring was missed. It’s not as simple as just notifying the broad phase.

While the logic is now more efficient, it has also become more complex due to the need for careful synchronization. Newton 4 differs from previous versions, where the broad phase was built from the top down. In Newton 4, it's constructed from the bottom up, incorporating redundancy and pruning techniques.

A simple object can have multiple proxies in the broad phase. The key idea is that the AABB proxies in the broad phase collectively cover the body's AABB volume. As the body moves and crosses boundaries, a new proxy with a new tag is added, while garbage collection removes old proxies. This approach allows the broad phase to be updated in linear time using radix sort, with removals being virtually free since outdated proxies are grouped together at the end of the array.

However, this makes debugging significantly more challenging.

I’ll debug it tonight. In the meantime, I might add a function to invalidate the broad phase if the teleport distance becomes too large.

[Julio Jerez]

Ah, yes, the function does exist. Here it is:

Code: Select all

auto mNewPosition = ndGetIdentityMatrix();
mNewPosition.m_posit = ndVector(690.9637, 48.655 + 1.5f, 1092.3562, 1.0);

// m_pBodyToMove->SetMatrix(mNewPosition);
m_pBodyToMove->SetMatrixUpdateScene(mNewPosition);

m_pBodyToMove->SetSleepState(false);


I changed the distance check from 1000 to 5000.
Ideally, calling that function should work as expected, and it does in a release build.
However, the issue is that when the body is moved,
a new box proxy is spawned, but the old one remains.

Here's the order of operations:

New pairs are updated.
Contacts are calculated.
The broad phase removes dead contacts.

Because of this sequence, the new pair triggers a warning.
The broad phase relies on the distance between close bodies,
so it needs to call the contact solver to determine the distance.
After that, it creates the AABB proxy, and only then is the contact deleted.

When bodies are created, there's a deeper build process that recalculates the AABB of all bodies.
As a result, it doesn't immediately trigger the contact call, but this recalculation is an expensive operation.

Please sync and try using
dyToMove->SetMatrixUpdateScene(mNewPosition);

Let me know if that works

edit:
oh, I have a question. the distance you are teleporting is larger that 1500 units.
if a unit is a meter, that approximately a mile distance.
Is that normal of is that to expose the problem.
in the demo ther is no floor below the object, so it just falls to the abyss.

[Esharc]

Thank you for that explanation Julio,

The changes you made fixes the assertions when moving the body, but it is still missing collisions for the first few times it falls through, but after it is "spawned" there about 3 times it starts colliding with the mesh.

If you turn on debug visuals you will see the static mesh. The mesh is quite large, so it takes a while to load. The body falling through the mesh is the actual problem that I am having.

I do not normally move the vehicles such a large distance at a time, it is just the initial movement. What we do is create all the vehicles and place them on a platform under the terrain where they are not visible. Then when the scenario starts and the AI controller starts moving the vehicle, I place them near the operator vehicle. And this distance can sometime be large depending where the operator starts.

Now that I know that the distance is the problem with that assert, I can place the initial position of the vehicles closer to the operator so that that is not a problem anymore.

[Julio Jerez]

Oh, I see it now.
I put the asset mesh in the wrong folder.

Yes I see the issue.
We can rule out the contact solver.
It looks like the porting to such long distance breaks the incremental broad phace reconstruction algorithm.
I tested in a double precision build to make sure it is not
a rounding issue, it is not.

It seems that that kind of teleporting requires a full reconstruction.
I am debugging it.

[Julio Jerez]

ok, if sync it should work now.

there was a bug in this function.
void ndBodyKinematic::SetMatrixUpdateScene(const ndMatrix& matrix)

it was doing the broad expansion fine.
but it never invalidated the contacts.
I guess that Carrie from older version where broad phace used to work by aabb overlap test.
Now it works by integrating the closest distance between aabb until it reaches a small value.

essentially what it should do is,
create more new proxies and add them to the tree,
them, garbage collection destroy the old ones.
but since the contacts distances were never clean,
the broad phace interprets them as still valid and keep
integrating them until the distance reaches zero.
them try again, that's why you see it working after a while.
clearing the contact fixes the problem.

[Esharc]

Thank you for the fix Julio, the collision are working correctly now.