How to remove ndModel

[JoeJ]

If i close my app, or if i delete and recreate a model to reset it, i get the assert in the function below.
I guess over the years i've already put dozens of hours into this, but i will never figure out how to do this correctly.
So why is the assert here, and how can i avoid it?
I don't see what i do differently than the demos.

Code: Select all

void ndWorld::DeleteDeferredObjects()
{
   // clean up all batched deleted objects, before update
   while (m_deletedModels.GetCount())
   {
      D_TRACKTIME();
      ndAssert(0); // <-
      ndModel* const model = m_deletedModels.GetFirst()->GetInfo();
      m_deletedModels.Remove(m_deletedModels.GetFirst());
      model->m_deletedNode = nullptr;

      ndModelList::ndNode* const worldNode = model->m_worldNode;
      m_modelList.RemoveModel(worldNode->GetInfo());
   }


Btw, updating Newton yesterday, there were compiler errors regarding threading. I could avoid them by using thread emulation, but now i get linker errors. So i can't run the demos to see what's new.

Code: Select all

Build started...
1>------ Build started: Project: ndSandbox, Configuration: Release x64 ------
1>ndLeakTracker.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndConvexFractureModel_2.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndConvexFractureModel_4.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndDebugDisplay.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndExplodeConvexShapeModel.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndBasicParticleFluid.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndDemoEntity.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndDemoEntityManager.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>ndPhysicsWorld.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) private: void __cdecl ndSpinLock::Delay(int &)" (__imp_?Delay@ndSpinLock@@AEAAXAEAH@Z)
1>C:\dev\newton-dynamics-master\newton-4.00\applications\ndSandbox\Release\ndSandbox.exe : fatal error LNK1120: 1 unresolved externals
1>Done building project "ndSandbox.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 11 up-to-date, 0 skipped ==========
========== Build started at 8:29 AM and took 07.208 seconds ==========


I'm using default settings for CMake.

[Julio Jerez]

void Delay(ndInt32& exp);
has D_CORE_API export, it should build.

on this void ndWorld::DeleteDeferredObjects()
yes that is becoming quite messy.

a while back I made a shared pointer and start using with bodes and joints.
I have to add it for models too, but I run int the problems with circular references.
models and loop joints, where a joint reference a body, and the body refenced a joint
and that's a hard problem. so I continue to use the raw point, in those cases.

what I have to do is to use a weak pointer, (which I really hate)
but I have not got around to finish the implementation.

I would use the standard C++, but it does not make any difference,
It is too bad that C++ never implemented a Garbage collected pointer, instead of the proliferation of all those others.
Any way Let see if I add some work on that.

what happens if you just comment out the assert?

[JoeJ]

what happens if you just comment out the assert?

Seems working. I have the problem that the ragdoll is most robust after application startup. But when i reset it (recreating everything), it's much more likely to fall on its nose. So i thought the reason could be related to the assert that i get.
But now that i know it's wip i will just ignore it.

I saw you have added the old exact solver for skeleton contacts. That's why i did the update, but i have not yet looked into this. Will report later.

[Julio Jerez]

I saw you have added the old exact solver for skeleton contacts. That's why i did the update, but i have not yet looked into this. Will report later.

Yes, that's how the solver worked in version 1.xx. Basically, it was one giant exact solver that took all the acyclic joints and then added all the loops and contacts.
However, that approach was too slow. At that time my impression was that joint were more important than contacts. But the public showed me that people preferred lots of bad contacts than reliable contraptions.
So then, I started separating them into islands, like in versions 2 and 3, but only included the first layer of contacts to the joints.

Now, I realized I could make additional passes after the first contacts, adding one more layer each time, which worked well. However, while testing with a robot, I noticed it could only handle a two-layer stack, with the last layer being too soft.

That made me wonder— it does no has to be one row of contact or all row of contacts,
what’s stopping me from adding more layers? So, I set it up for five-layer passes.
After that, the last row remains soft, which I think is a good compromise that any application can handle.
In the end, it’s coming almost full circle to the original 1.53 solver, but with significant improvements.

Anyway, have you tried it?

[JoeJ]

Anyway, have you tried it?

Confusion incoming...
There is neither
ndSkeletonContainer::m_useFullContactModel
nor
ndModelNotify::m_contactModel
in my recent copy of Newton.
But i see those additions on the related, older github commit:https://github.com/MADEAPPS/newton-dynamics/commit/16f8bb39706a51de39bdae11c1ed55330802592a#diff-5b79791664badb7807cec237fd25a5d3292d0d110e801dc8ca99edf1ae4c7aeb
So they were there, but now they are gone?
May you have reverted to some backup? Or do i confuse some things?

[Julio Jerez]

I guess that option is no longer needed. It general options are ways to kludge the simulation, I try to remove them as much as possible.

The layout is simpler now.
there is one array for each type of loop joint.
one for the permanent loops which are part of the skeleton, so they are bilateral joints.
one for the extra bilateral, which are added with a joint is attached to a skeleton.
one for contact pickup in the simulation.

this makes it simpler to understand and I think in the end is even better coded.

also, I will start replacing that dreaded deferred delete with share pointers.
I has always bothered me a lot, so I guess this is as good time as any.
I have to do it is stages.
That's one thing I really miss for cshar and Java.

[JoeJ]

Oh, so i do not have to activate the better contacts - it's already on.
But this also means i can not turn on/off to see the difference. And because i have not used default contacts in a while, it's hard to say. (I use a single custom 'contact' joint instead, which gives be smoother and more accurate results, but causes its own problems.)
However, testing it with the simple 2 bodies inverted pendulum model, i would say it became better. The measured center of pressure from the contacts seems the follow the target more closely than before.

I need to test with the ragdoll as well, and switching to release mode now gives me the same compiler errors than the demo did:

Code: Select all

ndThread::ndThread()
   :ndClassAlloc()
   ,ndSemaphore()
#ifndef D_USE_THREAD_EMULATION
   ,ndAtomic<bool>(true)
   ,std::condition_variable()
   ,std::thread(&ndThread::ThreadFunctionCallback, this)
#endif
{
   strcpy (m_name.m_name, "newtonWorker");

#ifndef D_USE_THREAD_EMULATION
   store(false);
   #ifdef WIN32
      native_handle_type nativeHandle = native_handle(); // <-


gives the errors:

Code: Select all

Severity   Code   Description   Project   File   Line   Suppression State
Error   C2385   ambiguous access of 'native_handle_type'   Realtime   C:\dev\pengII\newton4\dCore\ndThread.cpp   48   

Severity   Code   Description   Project   File   Line   Suppression State
Error   C2385   ambiguous access of 'native_handle'   Realtime   C:\dev\pengII\newton4\dCore\ndThread.cpp   48   

Severity   Code   Description   Project   File   Line   Suppression State
Error   C2385   ambiguous access of 'native_handle'   Realtime   C:\dev\pengII\newton4\dCore\ndThread.cpp   56   


Just fyi. I'll find a way around that i guess...

[Julio Jerez]

Severity Code Description Project File Line Suppression State
Error C2385 ambiguous access of 'native_handle_type' Realtime C:\dev\pengII\newton4\dCore\ndThread.cpp 48

the code is under a #ifdef WIN32 closure
are you using visual studio?
maybe I should make VISUAL_STUDIO specific

sync again I commented it out until later.
I am working on the weak pointer, so that I can remove the deferred remove form the engine.
them we can test that.

[JoeJ]

I've fixed the compiler error using GetCurrentThread() instead the handle object, e.g.:
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_NORMAL)
I'm using VS 2022 on Win10.

There is another unrelated issue i can report. Disabling collisions for certain bodies of my ragdoll does not work. I'm using this function, added some comments from debugging:

Code: Select all

   void EnOrDisableCollisions ()
   {
      //todo: does not work, but similar code works for simple pendulum model (?)
      
      bool enableCollisions = (dbg_contactMode == DEFAULT_CONTACTS);
      for (int i=0; i<2; i++)
      {
         for (int j=0; j<CharacterRig::CharacterController::SENSOR_BODIES_PER_LIMBS_COUNT; j++)
         {
            int sensorI = m_controller->stanceLimbs[i].sensorBodies[j];
            int bodyI = m_controller->sensorBodies[sensorI].bodyIndex; // correct and verified
            ndBodyDynamic *body = m_bodies[bodyI]->GetAsBodyDynamic();
            body->GetCollisionShape().SetCollisionMode(enableCollisions); // flag was already false, but collisions still happen
            body->SetSleepState(false);
         }
      }
   }


I call the function on model creation or when i change settings at runtime, but it does not turn off collisions. (Very rarely it does work, which is strange.)
Similar code works for other models i have, but not for the ragdoll. I guess it's a Newton bug, but maybe i do something wrong. One of the bodies is a compound, maybe this matters.

It's not a big problem. Currently i disable all collisions globally as a workaround.

[Julio Jerez]

I'm really torn about this:

I've fixed the compiler error by using GetCurrentThread() instead of the handle object, for example: SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_NORMAL).

I don't really want to do it this way, but I’ve come to realize that if I don’t, I’ll be the only one losing out.

While working on an Unreal Engine plugin, I noticed that running multi-read async operations caused the engine to become incredibly choppy and slow—so bad, in fact, that Unreal has a built-in time killer that forcefully terminates the app.

To move forward with the plugin, I enabled thread emulation. Then, I realized the real issue: the graphics engine was completely out of control. Unreal creates around 200 threads, with about 100 of them being high-priority. DX12 alone spawns around 20 threads.
This means that if you have a multithreaded application that doesn’t use Unreal's threading system, it gets completely overshadowed.
If your app’s threads don’t match or exceed Unreal’s thread priority, they get preempted because they’re competing with all those high-priority threads. It’s ridiculous.

I started thinking—if Unreal does this, then most other game engines probably do the same.

So, the only viable option is to adopt the same bad habit

[JoeJ]

Hehe,
game devs writing a book:
'Create as many permanent threads as there are cores, and then distribute all your work to those.'
Game devs writing a game:
'I've managed to run each GUI page in it's own thread in the background! How cool is that? Buddy list and chat has its own thread too.'

[Julio Jerez]

ok, I added part one of refactoring object management.
I have to start form the top, because it becomes very too much work if I just remove the deferred
delete object lists.
I refactored about half the demos. I will remove some demos that are just too old.

also, I moved three thread priority to the to the proper place.
void ndThread::ThreadFunctionCallback()

if you have time see if it build for you now.

[JoeJ]

Now i can build and run the sandbox, but i still get the same compiler errors about the handle objects.
This works, and seems a proper fix:

Code: Select all

void ndThread::ThreadFunctionCallback()
{
#ifndef D_USE_THREAD_EMULATION
   #ifdef WIN32
      std::thread::native_handle_type handle = std::thread::native_handle(); // added std::thread::
      ndInt32 priority = GetThreadPriority(handle);


The demos all work.
But it seems rendering is out of sync. Very noticable while driving, the cars jitter one frame back and forth. Problem persists if i change to async update and using more threads.

[Julio Jerez]

ah cool, I added the thread type change.

on eh rendering yet, I broke the rendering a while back when I added shadows.
I do not know what I made the mistake. It is quiet annoying.
after I get all the demos back, I see if I can fix that.

[Julio Jerez]

I just removed that dreadful deferred delete list.
now bodies have references,