TreeCollision corrupt?

[Lax]

Hi julio,

I finally found time, to port OgreNewt for newtondynamics4 with the help of AI and my knowledge . So its now called OgreNewt4.0 which uses the newest Ogre-Next version.

So I'm back and excited to see which new Newton Dynamics features I can integrate into my engine.

I'm now struggling issue by issue and make progress.
So i now stuck on TreeCollision. I tried lots of stuff, but TreeCollision is not really working for any mesh: Either other shapes are not colliding with it, or i get a crash in newtondynamics:

Stacktrace:

Code: Select all

ndNewton_d.dll!ndVector::ndVector(const ndVector & copy) Zeile 1167 C++ ndNewton_d.dll!ndShapeConvexPolygon::CalculatePlaneIntersection(const ndVector & normalIn={...}, const ndVector & origin={...}, ndVector * const contactsOut=0x0000009f5a5f2750) Zeile 95 C++ ndNewton_d.dll!ndShapeInstance::CalculatePlaneIntersection(const ndVector & normal={...}, const ndVector & point={...}, ndVector * const contactsOut=0x0000009f5a5f2750) Zeile 557 C++ ndNewton_d.dll!ndContactSolver::CalculateContacts(const ndVector & point0={...}, const ndVector & point1={...}, const ndVector & normal={...}) Zeile 4428 C++ ndNewton_d.dll!ndContactSolver::ConvexToConvexContactsDiscrete() Zeile 2547 C++ ndNewton_d.dll!ndShapeConvexPolygon::CalculateContactToConvexHullDescrete(const ndShapeInstance * const parentMesh=0x0000009f5a5f15a0, ndContactSolver & contactSolver={...}) Zeile 839 C++ ndNewton_d.dll!ndContactSolver::CalculatePolySoupToHullContactsDescrete(ndPolygonMeshDesc & data={...}) Zeile 4221 C++ ndNewton_d.dll!ndContactSolver::ConvexToStaticMeshContactsDiscrete() Zeile 4272 C++ ndNewton_d.dll!ndContactSolver::ConvexContactsDiscrete() Zeile 2469 C++ ndNewton_d.dll!ndContactSolver::CalculateContactsDiscrete() Zeile 2429 C++ ndNewton_d.dll!ndScene::CalculateJointContacts(int threadIndex=0, ndContact * const contact=0x00000236c2f5e0c0) Zeile 487 C++ ndNewton_d.dll!ndScene::CalculateContacts(int threadIndex=0, ndContact * const contact=0x00000236c2f5e0c0) Zeile 938 C++ ndNewton_d.dll!ndScene::CalculateContacts::__l5::<Lambdafunktion>(int groupId=1, int threadIndex=0) Zeile 1684 C++ ndNewton_d.dll!ndFunction<void <Lambdafunktion>(int, int)>::operator()(int threadIndex=1, int threadCount=0) Zeile 138 C++ ndNewton_d.dll!ndThreadPool::ParallelExecute<ndFunction<void <Lambdafunktion>(int, int)>>(const ndFunction<void <Lambdafunktion>(int, int)> & function={...}, int workGroupCount=4, int groupsPerThreads=8) Zeile 306 C++ ndNewton_d.dll!ndScene::CalculateContacts() Zeile 1689 C++ ndNewton_d.dll!ndWorld::SubStepUpdate(float timestep=0.00416666688) Zeile 430 C++ ndNewton_d.dll!ndWorld::MainUpdate() Zeile 370 C++ ndNewton_d.dll!ndWorld::ThreadFunction() Zeile 390 C++ ndNewton_d.dll!ndWorldScene::ThreadFunction() Zeile 54 C++ ndNewton_d.dll!ndThread::ThreadFunctionCallback() Zeile 156 C++ [Externer Code] I get for the copy ndVector really * values in newtondynamics4: inline ndVector(const ndVector& copy) :m_type(copy.m_type) { } 0x0000009f5a5ef860 {m_f=0x0000009f5a5ef860 {1.56921612e+16, 2.228e-43#DEN, -3.00642766e+26, 4.591e-41#DEN} ...}


This is the TreeCollision Code for an Ogre::v1::Entity:

Code: Select all

namespace
{
   inline bool isFiniteVec(const Ogre::Vector3& v)
   {
      return std::isfinite(v.x) && std::isfinite(v.y) && std::isfinite(v.z);
   }

   inline bool buildFaceTriplet(const Ogre::Vector3& a,
      const Ogre::Vector3& b,
      const Ogre::Vector3& c,
      ndVector outFace[3],
      FaceWinding fw)
   {
      if (!isFiniteVec(a) || !isFiniteVec(b) || !isFiniteVec(c))
         return false;

      const Ogre::Vector3 e0 = b - a;
      const Ogre::Vector3 e1 = c - a;
      const Ogre::Real area2 = e0.crossProduct(e1).squaredLength();
      if (area2 <= Ogre::Real(1e-12))
         return false; // reject degenerate

      outFace[0] = ndVector(a.x, a.y, a.z, 1.0f);
      outFace[1] = ndVector(b.x, b.y, b.z, 1.0f);
      outFace[2] = ndVector(c.x, c.y, c.z, 1.0f);

      if (fw == FW_REVERSE)
         std::swap(outFace[1], outFace[2]);

      return true;
   }
}

TreeCollision::TreeCollision(const World* world,
   Ogre::v1::Entity* obj,
   bool optimize,
   unsigned int id,
   FaceWinding fw)
   : Collision(world)
   , m_faceCount(0)
   , m_categoryId(id)
{
   Ogre::Vector3 scale(1, 1, 1);
   if (auto* node = obj ? obj->getParentNode() : nullptr)
      scale = node->_getDerivedScaleUpdated();

   std::vector<Ogre::Vector3> vertices;
   std::vector<int> indices;

   if (obj)
   {
      Ogre::v1::MeshPtr mesh = obj->getMesh();
      if (!mesh.isNull())
      {
         const unsigned short subCount = mesh->getNumSubMeshes();
         bool sharedAdded = false;

         for (unsigned short si = 0; si < subCount; ++si)
         {
            Ogre::v1::SubMesh* sub = mesh->getSubMesh(si);
            if (!sub) continue;
            if (sub->operationType != Ogre::OperationType::OT_TRIANGLE_LIST)
               continue;

            Ogre::v1::VertexData* vdata = nullptr;
            if (sub->useSharedVertices)
            {
               if (!sharedAdded)
               {
                  vdata = mesh->sharedVertexData[0];
                  sharedAdded = true;
               }
            }
            else
            {
               vdata = sub->vertexData[0];
            }
            if (!vdata) continue;

            const Ogre::v1::VertexElement* posElem =
               vdata->vertexDeclaration->findElementBySemantic(Ogre::VES_POSITION);
            if (!posElem) continue;

            const size_t vertexOffset = vertices.size();
            const size_t vStart = vdata->vertexStart;
            const size_t vCount = vdata->vertexCount;

            Ogre::v1::HardwareVertexBufferSharedPtr vbuf =
               vdata->vertexBufferBinding->getBuffer(posElem->getSource());
            unsigned char* base = static_cast<unsigned char*>(
               vbuf->lock(Ogre::v1::HardwareBuffer::HBL_READ_ONLY));

            const size_t stride = vbuf->getVertexSize();
            for (size_t j = 0; j < vCount; ++j)
            {
               unsigned char* ptr = base + (vStart + j) * stride;
               float* p = nullptr;
               posElem->baseVertexPointerToElement(ptr, &p);
               Ogre::Vector3 v(p[0], p[1], p[2]);
               if (!isFiniteVec(v))
                  continue;
               vertices.push_back(v * scale);
            }
            vbuf->unlock();

            Ogre::v1::IndexData* iData = sub->indexData[0];
            if (!iData) continue;

            Ogre::v1::HardwareIndexBufferSharedPtr ib = iData->indexBuffer;
            const size_t iCount = iData->indexCount;
            const size_t iStart = iData->indexStart;
            if (iCount < 3) continue;

            if (ib->getType() == Ogre::v1::HardwareIndexBuffer::IT_32BIT)
            {
               const unsigned int* baseIdx =
                  static_cast<const unsigned int*>(ib->lock(Ogre::v1::HardwareBuffer::HBL_READ_ONLY));
               const unsigned int* iptr = baseIdx + iStart;
               for (size_t k = 0; k < iCount; ++k)
                  indices.push_back(int(iptr[k]) + int(vertexOffset));
               ib->unlock();
            }
            else
            {
               const unsigned short* baseIdx =
                  static_cast<const unsigned short*>(ib->lock(Ogre::v1::HardwareBuffer::HBL_READ_ONLY));
               const unsigned short* iptr = baseIdx + iStart;
               for (size_t k = 0; k < iCount; ++k)
                  indices.push_back(int(iptr[k]) + int(vertexOffset));
               ib->unlock();
            }
         }
      }
   }

   if (m_col)
      m_col->Release();

   ndPolygonSoupBuilder meshBuilder;
   meshBuilder.Begin();

   const size_t triCount = indices.size() / 3;
   for (size_t t = 0; t < triCount; ++t)
   {
      const int i0 = indices[t * 3 + 0];
      const int i1 = indices[t * 3 + 1];
      const int i2 = indices[t * 3 + 2];
      if (i0 < 0 || i1 < 0 || i2 < 0 ||
         i0 >= (int)vertices.size() ||
         i1 >= (int)vertices.size() ||
         i2 >= (int)vertices.size())
         continue;

      ndVector face[3];
      if (!buildFaceTriplet(vertices[i0], vertices[i1], vertices[i2], face, fw))
         continue;

      for (int j = 0; j < 3; ++j)
         face[j].m_w = 1.0f;

      meshBuilder.AddFace(&face[0].m_x, sizeof(ndVector), 3, static_cast<ndInt32>(id));
      ++m_faceCount;
   }

   meshBuilder.End(optimize);
   m_shapeInstance = new ndShapeInstance(new ndShapeStatic_bvh(meshBuilder));
   m_col = m_shapeInstance->GetShape();
}


Maybe you see, what I'm doing wrong?

Best Regards
Lax

[Lax]

Ah I found the issue. I used:

meshBuilder.End(optimize);

optimize was set to true. That causes the weird behavior. Using

meshBuilder.End(false);

Does work. So maybe there is a bug using mesh optimization?

[Julio Jerez]

Oh, that's awesome,
Are you on the last commit.

The optimizer is usually very good.
One of the points of the optimizer, is that it generates, Delaney meshes.
That it, the mesh is converted to an equivalent that maximize face surface area.
This is a big help for generation of better-quality contact points

I am curious as to why it fails.
I look at the script, and it seem to be built a triangle list form a vertex list to index, list.
but it is difficult to determine what is wrong, without running it.

not sure what to build a face and check the normal, that shouldn't be necessary.
-are you meshed double faces?
-does fail in all meshes?
-also, can I get a repro?

[Lax]

Hm its hard to say, i'm not deep in this topic. The ai helped analyzing it and says:

the optimize = true path tries to rebuild an internal face-adjacency map, but it assumes contiguous triangle indexing (as in the Newton demos). When Ogre submeshes append disjoint vertex batches, that assumption fails → empty m_faceMap → ndArray::operator[] crash exactly where you saw it.

The strange thing is:
- For OgreNewt3 the optimize did work, so Ogre-Next api has not changed. I just adapted to feed newtondynamics4
- Setting optimization to true, it refuses to work also for a really simple model like a quader

I let the ai analyze deeper, this is the result:

“Using ndPolygonSoupBuilder → End(true) → ndShapeStatic_bvh, a simple unit cube made of 12 triangles fails/crashes (or silently produces no collision), while End(false) works. The same triangle list succeeds in ND3 tree collision. This reproduces with all faces sharing one attribute ID.”

“From ndPolygonSoupBuilder.cpp, the optimize path runs faces through FinalizeAndOptimize and strict filters (1e-6 edge, 1e-8 area, collinearity, etc.). For even simple meshes, if windings are flipped by a negative determinant transform or if the geometry scale is small enough, the filters reject every face in the bucket, leaving an empty soup which later crashes when building/using ndShapeStatic_bvh.”

[Julio Jerez]

the big problem with AI is that it is incapable of saying "I don't know".
Instead, when that answerer is no close to the training, it extrapolates from what the training set and for the most part it gets it wrong.
Ai is full of confirmation bias and circular reasoning. simple because these fallacies are built in the training data.

anyway, I can see kernels of truth on what it is telling you, but for the most past in is not correct.
The optimized does not reject entire buckets for adjacent faces.
also, it will not fail on a simple cube. So after that you can't trust that that analysis.

here is what we know as fact.
-The optimizer works, and least in some test problems. we see working in the demos.
-the algorithm has not change much since 3.14
-the function work in previous Ogre Newt

the conclusion must be that there must be a bug in the conversion, so the best way to figure out is
debugging with a simple example like a cube. and see where the mistake is.

Again, is there a way I can get a repro test, that I can try.
I would like to see OgreNewt in ogre.

[Lax]

the big problem with AI is that it is incapable of saying "I don't know".

Yes indeed, thats an illness of ai. I'm also cautious with ai. But on some issues, it did help. So i thought,maybe there is some truth in it with the tree collision.

So I now use your newest version of newtondynamics4. I also strictly orientated on the code for OgreNewt3 TreeCollision for OgreNewt4, then i used winmerge for comparison, and besides the new newtondynamcs4 commands, everything is the same. So there must be a bug on your side. I see nothing on my side.

Or do you see something suspicious here in this code, how your api is fed:

Code: Select all

TreeCollision::TreeCollision(const World* world, Ogre::v1::Entity* obj, bool optimize, unsigned int id, FaceWinding fw)
   : Collision(world)
{
   Ogre::Vector3 scale;

   m_categoryId = id;

   ndPolygonSoupBuilder meshBuilder;
   meshBuilder.Begin();

   Ogre::v1::MeshPtr mesh = obj->getMesh();

   Ogre::Node* node = obj->getParentNode();
   if (node)
   {
      scale = node->_getDerivedScaleUpdated();
   }

   unsigned short sub = mesh->getNumSubMeshes();

   for (unsigned short cs = 0; cs < sub; cs++)
   {
      Ogre::v1::SubMesh* sub_mesh = mesh->getSubMesh(cs);

      //vertex data!
      Ogre::v1::VertexData* v_data;

      if (sub_mesh->useSharedVertices)
      {
         v_data = mesh->sharedVertexData[0];
      }
      else
      {
         v_data = sub_mesh->vertexData[0];
      }

      Ogre::v1::VertexDeclaration* v_decl = v_data->vertexDeclaration;
      const Ogre::v1::VertexElement* p_elem = v_decl->findElementBySemantic(Ogre::VES_POSITION);

      Ogre::v1::HardwareVertexBufferSharedPtr v_sptr = v_data->vertexBufferBinding->getBuffer(p_elem->getSource());
      unsigned char* v_ptr = static_cast<unsigned char*>(v_sptr->lock(Ogre::v1::HardwareBuffer::HBL_READ_ONLY));

      Ogre::v1::IndexData* i_data = sub_mesh->indexData[0];
      size_t index_count = i_data->indexCount;
      size_t poly_count = index_count / 3;

      Ogre::v1::HardwareIndexBufferSharedPtr i_sptr = i_data->indexBuffer;

      bool uses32bit = (i_sptr->getType() == Ogre::v1::HardwareIndexBuffer::IT_32BIT);
      unsigned long* i_Longptr;
      unsigned short* i_Shortptr;

      if (uses32bit)
      {
         i_Longptr = static_cast<unsigned long*>(i_sptr->lock(Ogre::v1::HardwareBuffer::HBL_READ_ONLY));

      }
      else
      {
         i_Shortptr = static_cast<unsigned short*>(i_sptr->lock(Ogre::v1::HardwareBuffer::HBL_READ_ONLY));
      }


      //now loop through the indices, getting polygon info!
      int i_offset = 0;

      for (size_t i = 0; i < poly_count; i++)
      {
         Ogre::Vector3 poly_verts[3];
         unsigned char* v_offset;
         float* v_Posptr;
         int idx;

         if (uses32bit)
         {
            for (int j = 0; j < 3; j++)
            {
               idx = i_Longptr[i_offset + j];        // index to first vertex!
               v_offset = v_ptr + (idx * v_sptr->getVertexSize());
               p_elem->baseVertexPointerToElement(v_offset, &v_Posptr);
               //now get vertex position from v_Posptr!
               poly_verts[j].x = *v_Posptr; v_Posptr++;
               poly_verts[j].y = *v_Posptr; v_Posptr++;
               poly_verts[j].z = *v_Posptr; v_Posptr++;

               poly_verts[j] *= scale;
            }
         }
         else
         {
            for (int j = 0; j < 3; j++)
            {
               idx = i_Shortptr[i_offset + j];       // index to first vertex!
               v_offset = v_ptr + (idx * v_sptr->getVertexSize());
               p_elem->baseVertexPointerToElement(v_offset, &v_Posptr);
               //now get vertex position from v_Posptr!

               // switch poly winding.
               poly_verts[j].x = *v_Posptr; v_Posptr++;
               poly_verts[j].y = *v_Posptr; v_Posptr++;
               poly_verts[j].z = *v_Posptr; v_Posptr++;

               poly_verts[j] *= scale;
            }
         }

         if (fw == FW_DEFAULT)
         {
            meshBuilder.AddFace(
               reinterpret_cast<const ndFloat32*>(&poly_verts[0].x),
               static_cast<ndInt32>(sizeof(Ogre::Vector3)),
               3,
               static_cast<ndInt32>(id)
            );
         }
         else
         {
            Ogre::Vector3 rev_poly_verts[3];
            rev_poly_verts[0] = poly_verts[0];
            rev_poly_verts[1] = poly_verts[2];
            rev_poly_verts[2] = poly_verts[1];

            meshBuilder.AddFace(
               reinterpret_cast<const ndFloat32*>(&rev_poly_verts[0].x),
               static_cast<ndInt32>(sizeof(Ogre::Vector3)),
               3,
               static_cast<ndInt32>(id)
            );
         }

         i_offset += 3;
      }

      v_sptr->unlock();
      i_sptr->unlock();

   }

   meshBuilder.End(true);
   m_shapeInstance = new ndShapeInstance(new ndShapeStatic_bvh(meshBuilder));
   m_col = m_shapeInstance->GetShape();
}


What i also found out: I debugged and painted the collision hull and it looks correct. So there is something wrong with the contact points. So for one mesh i get no collision at all. At the second other mesh, it does collide one time but if i through another gameobject at it again, then it will crash. I made a video. Here is the link:

https://www.lukas-kalinowski.com/Homepage/wp-content/uploads/NOWA/TreecollisionBug.mp4

Again, is there a way I can get a repro test, that I can try.
I would like to see OgreNewt in ogre.

At the moment not sorry. My Engine is a mess for now. I also integrated multithreaded rendering and game loop. So that Ogre is rendered in its own thread. (Ogre itself uses x-threads) and then there is the main thread. And now newtondynamics uses a threadpool. So I'm fixing stuff bit by bit. If i have a state, which is presentable. I will provide ready to compile solution.

Best Regards
Lax

[Julio Jerez]

I just check the demos to see if they were passing a false for optimize flag.
but not, they are all optimized.

It is possible that there is a bug, yes.
what is no possible is that in mesh with the same topologies it working in the demos, and other apps, and fail in you port.
That indicate that there must be a bug in the port.

I see that you have a use 32 flag. are you building in 32 big?
I has been almost five years since I do not build in 32 bit.
I abandon it because many of the new third parties' libraries are not supporting 32 bit anymore.

had you try 64 bit?

without debugging is going to be very difficult to figure out what is wrong.
It is nor just the optimized, but by looking at the videos, The mesh build is wrong too.