ndWorld::Sync?

[Lax]

Hi Julio,

I make further tests with ND4. As I posted on other topics, i have often strange asserts in ND4. Now I found a totally different reason, what could be wrong and its strange. I post my OgreNewt::World update code:

Code: Select all

void World::flushDeferred()
{
    std::vector<std::function<void()>> pending;
    {
        std::lock_guard<std::mutex> g(m_deferMutex);
        pending.swap(m_deferred);
    }

    for (auto& fn : pending)
    {
        if (fn) fn();
    }
}

void World::update(Ogre::Real t_step)
{
    if (m_paused.load())
    {
        if (!m_doSingleStep.exchange(false))
            return;
    }

    // clamp to avoid spiral-of-death
    if (t_step > (m_fixedTimestep * m_maxTicksPerFrames))
        t_step = m_fixedTimestep * m_maxTicksPerFrames;

    m_timeAccumulator += t_step;

    while (m_timeAccumulator >= m_fixedTimestep)
    {
        // 1) Kick the step (ndWorld::Update will first Sync with any *previous* step,
        //    then start this step asynchronously via TickOne()).
        ndWorld::Update(static_cast<ndFloat32>(m_fixedTimestep));  // starts async work

        // Causes weird asserts in ND4
        // 2) Wait for the step we just started to finish -> this makes stepping synchronous.
        // ndWorld::Sync();

        // 3) (Optional) If you kept any deferral queue, you can flush immediately here,
        //    but with a purely synchronous model you won’t need deferral any more.
        flushDeferred();

        m_timeAccumulator -= m_fixedTimestep;

        if (m_paused.load())
            break;
    }

    const Ogre::Real interp = m_timeAccumulator * m_invFixedTimestep;
    postUpdate(interp);
}

void World::postUpdate(Ogre::Real interp)
{
    const ndBodyListView& bodyList = GetBodyList();
    const ndArray<ndBodyKinematic*>& view = bodyList.GetView();

    for (ndInt32 i = ndInt32(view.GetCount()) - 1; i >= 0; --i)
    {
        ndBodyKinematic* const ndBody = view[i];
        if (!ndBody->GetSleepState())
        {
            if (auto* notify = ndBody->GetNotifyCallback())
            {
                if (auto* ogreNotify = dynamic_cast<BodyNotify*>(notify))
                {
                    if (auto* ogreBody = ogreNotify->GetOgreNewtBody())
                        ogreBody->updateNode(interp);
                }
            }
        }
    }
}

void World::recover()
{
    // Called from logic/render thread -> just schedule
    this->deferAfterPhysics([this]()
    {
        this->recoverInternal();
    });
}

void OgreNewt::World::recoverInternal()
{
    const ndBodyListView& bodyList = GetBodyList();
    const ndArray<ndBodyKinematic*>& view = bodyList.GetView();

    for (ndInt32 i = ndInt32(view.GetCount()) - 1; i >= 0; --i)
    {
        ndBodyKinematic* const b = view[i];
        if (!b || b == GetSentinelBody())
            continue;

        ndBodyDynamic* const dyn = b->GetAsBodyDynamic();
        if (!dyn)
            continue;

        // mark the body/scene as dirty (invalidates contact cache & aabb)
            //    This is the ND4 way to say "something changed, don’t keep me asleep".
        dyn->SetMatrixUpdateScene(b->GetMatrix());

        // actually wake it
        dyn->SetAutoSleep(true);      // keep normal autosleep behavior
        dyn->SetSleepState(false);    // force out of equilibrium for the next step
    }
}

So: I get the asserts if ndWorld::Sync() is set in the main loop. I documentated it out and the asserts are gone. I tested some of my more complex ND4 scenarios and they seem to work so far.

But in the ND4 ndDemoEntityManager this is set to true:

m_synchronousPhysicsUpdate = true;

Hence in your example:

Code: Select all

void ndPhysicsWorld::AdvanceTime(ndFloat32 timestep)
{
   D_TRACKTIME();
   const ndFloat32 descreteStep = (1.0f / MAX_PHYSICS_FPS);

   if (m_acceleratedUpdate)
   {
      Update(descreteStep);
      RemoveDeadEntities();
   }
   else
   {
      ndInt32 maxSteps = MAX_PHYSICS_STEPS;
      m_timeAccumulator += timestep;

      // if the time step is more than max timestep par frame, throw away the extra steps.
      if (m_timeAccumulator > descreteStep * (ndFloat32)maxSteps)
      {
         ndFloat32 steps = ndFloor(m_timeAccumulator / descreteStep) - (ndFloat32)maxSteps;
         ndAssert(steps >= 0.0f);
         m_timeAccumulator -= descreteStep * steps;
      }

      while (m_timeAccumulator > descreteStep)
      {
         Update(descreteStep);
         m_timeAccumulator -= descreteStep;
         RemoveDeadEntities();
      }
   }

   {
      ndScopeSpinLock Lock(m_lock);
      ndFloat32 param = m_timeAccumulator / descreteStep;
      m_manager->m_renderer->InterpolateTransforms(param);
   }

   if (m_manager->m_synchronousPhysicsUpdate)
   {
      Sync();
   }
}


So what is going on? Or is my Physics updates loop wrong? I have no idea what is right or wrong. I orientated on the old OgreNewt3 (ND3) code and added the peaces for ND4. Also I have to use deffered functions. For example: I have a Leveleditor (NOWA-Design). I press the play button -> OgreNewt4 starts running. Loop is processed. Then I press the stop button. OgreNewt4 stops. Then i call recover(). I found out it must be called in a seperate step, after ND4 has processed everything.

Perhaps you can shed some light on the matter.

Thanks!

Best Regards
Lax

[Julio Jerez]

this is strange, there is not assert in the path of the call.
I just added a sync inside the loop to see is it assert, but I do not get any.
when you get the assert next time, can you post the code where it happened.

but in any case.
if you insert a sync update in that loop, it will defeat the purpose of caching up if the frame rate falls behind. he is a pseudo code of how to implement your loop.

Code: Select all

void ndPhysicsWorld::AdvanceTime(ndFloat32 timestep)
{
   D_TRACKTIME();
   const ndFloat32 descreteStep = (1.0f / MAX_PHYSICS_FPS);

   ndInt32 maxSteps = MAX_PHYSICS_STEPS;
   m_timeAccumulator += timestep;

   // if the time step is more than max timestep par frame, throw away the extra steps.
   if (m_timeAccumulator > descreteStep * (ndFloat32)maxSteps)
   {
      ndFloat32 steps = ndFloor(m_timeAccumulator / descreteStep) - (ndFloat32)maxSteps;
      ndAssert(steps >= 0.0f);
      m_timeAccumulator -= descreteStep * steps;
   }

   while (m_timeAccumulator > descreteStep)
   {
      Update(descreteStep);

      m_timeAccumulator -= descreteStep;
      RemoveDeadEntities();
   }

   {
      ndScopeSpinLock Lock(m_lock);
      ndFloat32 param = m_timeAccumulator / descreteStep;
      m_manager->m_renderer->InterpolateTransforms(param);
   }

   if (m_manager->m_synchronousPhysicsUpdate)
   {
      Sync();
   }


notice that syn is optional and at the end of the function, only when you want to run asynchronous with the application.

[Lax]

So what I do not understand. Has newtondynamics4 so much changed from newtondynamics3?

using this update loop now:

Code: Select all

void World::update(Ogre::Real t_step)
{
    // Remember which thread is allowed to directly call ndWorld APIs
    if (m_physicsThreadId == std::thread::id())
    {
        m_physicsThreadId = std::this_thread::get_id();
    }

    m_isSimulating.store(true);

    if (m_paused.load())
    {
        if (!m_doSingleStep.exchange(false))
        {
            m_isSimulating.store(false);
            return;
        }
    }

    // clamp to avoid spiral-of-death
    if (t_step > (m_fixedTimestep * m_maxTicksPerFrames))
    {
        t_step = m_fixedTimestep * m_maxTicksPerFrames;
    }

    m_timeAccumulator += t_step;

    while (m_timeAccumulator >= m_fixedTimestep)
    {
        // IMPORTANT: no Sync() here (keeps async benefit when FPS falls behind)
        ndWorld::Update(static_cast<ndFloat32>(m_fixedTimestep));

        m_timeAccumulator -= m_fixedTimestep;

        if (m_paused.load())
        {
            break;
        }
    }

    // One Sync per frame: all ND4 workers finish, now it's safe to touch the world.
    ndWorld::Sync();

    // Safe point: run all engine requests (remove body/joint, jobs, etc.)
    flushDeferred();

    const Ogre::Real interp = m_timeAccumulator * m_invFixedTimestep;
    postUpdate(interp);

    m_isSimulating.store(false);
}


I added deferred closures for raycasts, convexcasts, addBody, removeBody, addJoint, removeJoint. So must i call for any write operation in newtondynamics4 (via OgreNewt4) a deffered closure? Because right now its destroying my NOWA-Engine. Its getting to complicated. I have already much todo with Ogre-Next running stuff on own render-thread. And now OgreNewt will become to complex.

In the past with OgreNewt3 and newtondynamics3. There was no such complexity...

Best Regards
Lax

[Julio Jerez]

I post the loop that works, why are trying something different?

[Lax]

Ok, i adapted my code to use nearly the same update loop.

So again: What i do not understand:

- I run Ogre-Next stuff on render thread and protect the access by a queue to be executed on that thread
- Logic is run on main thread
- I update OgreNewt4 on main thread and have access to ND4 functions mutations via main thread (Like it was in the past in OgreNewt3).
- ND4 itself runs on its physics threads.

Do I need to protect each ND4 call in OgreNewt4 with a queue so that the call is quaranteed to be executed on a physics thread? Because that would be a lot of more work todo...

[Julio Jerez]

- I update OgreNewt4 on main thread and have access to ND4 functions mutations via main thread (Like it was in the past in OgreNewt3).

this should work.

Do I need to protect each ND4 call in OgreNewt4 with a queue so that the call is quaranteed to be executed on a physics thread? Because that would be a lot of more work todo...

I do not understand where are you getting the idea that you have to protect everything.
Must functionality in newton are reentrance and state less, raycast, collision, all of the scene manage functions do not have to deferred.

It looks like there are compound mistakes here.
management stuff can be called from anywhere.
The only rule, is not to delete or create objects in the middle of an update.

My suggestion is to solve one issue at a time.

For that, you can build the SDK single threaded threads, that will run on the main thread, so that and you can check that everything is right.
Then you can try thread emulation and check that the mechanic is right.
then you can try threaded and keep adding.

I also proposed to you, if you let me check it out, and see if I can help with some pointers and ideas.
But you declined.

[Lax]

Hi Julio,

thanks for the offer. I want not decline. Its just, my Engine is right now a mess. Its not easy to compile it etc. And you are very busy. So for me its ok, if you from time to time answer some questions.

And your last reply was worth its weight in gold

Now i fully understand and that were the issues. I sometimes call ND4 Body, Joint creation/destruction from render thread. That was the issue. So I now have a moodycamel::ConcurrentQueue which gathers those commands and processes them at a safe place.

The rest access for ND4 function via OgreNewt4 wrapper are now the same as in OgreNewt3. And now it seems to work!

Best Regards
Lax

[Julio Jerez]

Now i fully understand and that were the issues. I sometimes call ND4 Body, Joint creation/destruction from render thread. That was the issue. So I now have a moodycamel::ConcurrentQueue which gathers those commands and processes them at a safe place.

The rest access for ND4 function via OgreNewt4 wrapper are now the same as in OgreNewt3. And now it seems to work!
Lax

Oh nice.
You can think of the engine like a rendering API that you communicate via a command queue.
You can create objects, or issue destroy command. From anywhere. Including render thread.
But you do not add the to the world.

The you can execute the queue in the post update came back.