(3.08) Multithread - NewtonWorldCriticalSectionLock

[Dragonic]

Hello,

I have an error (m_threadSanity, dgMemory.cpp) since I tried to create some NewtonBody in differents threads !
I can explain how my code works a little : it's about a voxel engine, and I have to load a chunk of 32x32x32 blocks, and edit it (add or delete blocks).

I have 3 threads :
- the main one (rendering, ...)
- another one is used to load the blocks (from a save file or with procedural generation)
- the last one is used to build the mesh and a NewtonTree/NewtonBody with it

At startup everything seems ok, I have my chunk and the NewtonBody associated.
I use a NewtonWorldRayCast in order to find a pointed block that I can edit.
But when I tried to delete a block, and let the last thread rebuild my mesh and the NewtonBody associated, I have an error (m_threadSanity, dgMemory.cpp, ...).
(not all the time, but at least for 1 edit/2)

I checked the forum, find some things about NewtonWorldCriticalSectionLock, but :
When I look at the wiki, it show :

Code: Select all

void NewtonWorldCriticalSectionLock (const NewtonWorld* newtonWorld);

But in my current NewtonDynamics version (3.08), i have this :

Code: Select all

void NewtonWorldCriticalSectionLock (const NewtonWorld* const newtonWorld, int threadIndex);


I have a NewtonMaterialSetCollisionCallback for a test, where I put the NewtonWorldCriticalSectionLock with the threadIndex, but this doesn't correct the error !
The other callback in my code is the NewtonWorldRayFilterCallback, but I don't think that I need to use NewtonWorldCriticalSectionLock with it !
Finally I don't know if the problem is about a callback and NewtonWorldCriticalSectionLock, or maybe directly in the mesh builder thread !

So thank you in advance if you can explain me how to use multithreading with Newton Dynamics !

(another problem who needs another topic ^^ : If someone know how to build the latest version of NewtonDynamics with MinGW only (without VisualStudio), this would be really helpfull for me ^^ !)

[Julio Jerez]

NewtonWorldCriticalSectionLock is not going to help you because that function is mean to be call form a call back. That function blocks one micro thread. the thread index in the micro thread number.
To make object you only need to follow two rules.
1-do not create or destroy Newton objects from call back, this is because in Newton the memory manager is no thread safe.
2-if you run Newton concurrent with some the thread (similar to D3d or OpenGL) then the NetwonUpdate will return immediately and will run in parallel with you other threads.
if you are using Newton in that mode, you need and you wan to after you create of destroy an object, you need to call NewtonWaitForUpdateToFinish(world);
this function will block that calling thread until the Newton thread finish the update.

This is the most efficient way to run the engine, but is the most difficult.
For that reason there is a C++ wrapper to help people to integrate this model of operation.

[Dragonic]

Thank you for the fast response ^^ !

1-do not create or destroy Newton objects from call back, this is because in Newton the memory manager is no thread safe.

I create new NewtonObjects from another threads, not from any callback in my current code !

2-if you run Newton concurrent with some the thread (similar to D3d or OpenGL) then the NetwonUpdate will return immediately and will run in parallel with you other threads.
if you are using Newton in that mode, you need and you wan to after you create of destroy an object, you need to call NewtonWaitForUpdateToFinish(world);
this function will block that calling thread until the Newton thread finish the update.

I don't understand very well this part ! I need to put NewtonWaitForUpdateToFinish everywhere I tried to delete or create a new NewtonObject in another thread than the principal ?

For that reason there is a C++ wrapper to help people to integrate this model of operation.

Really ? I didn't know that !
But I hope it is about a core3 version !?

EDIT : I tried to debug exactly where the error occur, this is most of the time during the NewtonTree optimization in the third thread of my program :

Code: Select all

//thread 3 : after mesh building
//no error here
NewtonTreeCollisionEndBuild(treeCollision, 1);
//error m_threadCheckSanity, dgMemory.cpp l.458 before this line


[Julio Jerez]

NewtonTreeCollisionEndBuild(treeCollision, 1); will definily mess up newton if it is call concurrenmtly with a newto update.
this is because the memory manage is not thread safe.

you only nee to call NewtonWaitForUpdateToFinish if you call NewtonAsyncrounUpdate.

however of you for soem reason call a newion funtion form another thread, and it just happen thet ther is a NetwonUpdate going on,
the you will get umpredicable results, yo uwill need to add some code to prove the newton update and maje sure it is not running

This is probable a design flaw in netwon but unfornamly this is how it was made, if I have do do again I would make a thread safe memory manager. but hindside is alway 20/20

[Bird]

I'm not sure if it would work for you but you could store a list of bodies to create/delete and make the creation/deletion calls from a NewtonPreListener or NewtonPostListener that's you've set up. That's what I do.

-Bird

[Dragonic]

Hum, ok ^^ !

I can't find anything about NewtonAsynchronousUpdate, NewtonPreListener or NewtonPostListener, so I don't know what you tried to explain to me, sorry ^^' !

I think I will just do things really simple for now, and let everything about Newton working with the main thread !
But I will need later to use a thread in order to build this NewtonTree because it takes too much time, and this is something that is often recreated !
(because I don't have such maths skills to understand how to use and prepare a UserMesh rather than a NewtonTree for now ^^').

[Julio Jerez]

it is these funtions

Code: Select all

   NEWTON_API void NewtonUpdate (const NewtonWorld* const newtonWorld, dFloat timestep);
   NEWTON_API void NewtonUpdateAsync (const NewtonWorld* const newtonWorld, dFloat timestep);
   NEWTON_API void NewtonWaitForUpdateToFinish (const NewtonWorld* const newtonWorld);

but if you do not knwo abput NewtonUpdateAsync, then you do no need to call NewtonWaitForUpdateToFinish.
My guess is that you are runing the engien oa thread, tha is running concurrent with some othe thread, and you try to create objects from
another threrad.

if this is the case it will lead to umpredicatble result.
but is is verty eassy to result with a hand made semaphs of mutex.

bascially create a variable, teh is visble to both thread.

the variabel is atimically set to one beafor update and rest after update.

in the Netwon update thread you can do this

Code: Select all

while (!AtomicExchange(mylock, 1))
{
Sleep()
}
NetwonUpdate()
AtomicExchange(mylock, 0);

and simular code in the place where you create newton objects.
that code will make sure that a newton update and any memory asset are mutually exclusive, even whn eth are run form seprate threads.

[Julio Jerez]

The secund point is that why do you recrate collision trees at run time?

[Dragonic]

About the first point, I understand that I can use some mutexes or semaphores in order to keep the creation of Newton objects in another threads. I will try that !

The secund point is that why do you recrate collision trees at run time?

Because I don't know how to use the UserMesh solution, whereas I fully understand how the NewtonTree works for now ^^' !
But for a custom mesh which is often edited (my project is a Minecraft-like game), I know that the UserMesh is better !

You already tried to explain me how it works but this is really too difficult for me to understand without a minimal and simple example where the only variable needs to be a custom mesh (like the mesh of my chunk).
I tried to look at something like that : http://newtondynamics.com/wiki/index.ph ... _collision
And this seems really complicated >< ! And if it's not the case, it's just that my english isn't so good to fully and directly understand everything ^^' !

[Julio Jerez]

in the demos folder there is an example of a user mesh
C:\Development\newton-dynamics\applications\demosSandbox\sdkDemos\toolBox\UserPlaneCollision.cpp

[Dragonic]

in the demos folder there is an example of a user mesh
C:\Development\newton-dynamics\applications\demosSandbox\sdkDemos\toolBox\UserPlaneCollision.cpp

The only thing I can understand from this is that I need to do my own calculations with the UserMesh solution !
(I think, it's difficult to decode what you do in PlaneCollisionCollideCallback or PlaneMeshCollisionRayHitCallback for example)

And I can understand that calculations aren't so difficult for a plane ^^ ! But in my case I think it is much more complicated to make calculations with my collision mesh rather than a simple plane ^^' !
That's why I'm using a TreeCollision ! I don't have enough skills to code my own collision or raycast function for a complex mesh, even if it's just composed of squares for now ^^ !

[Julio Jerez]

it is not as diffcult as you think, here is an explanation of how you populate this function callbacks
viewtopic.php?f=9&t=7786&hilit=userMesh&start=75

[Dragonic]

it is not as diffcult as you think, here is an explanation of how you populate this function callbacks
viewtopic.php?f=9&t=7786&hilit=userMesh&start=75

Thank you, I'll look at it in detail ^^ !