Source Code for Newton 2.00

[michDS]

Hi,

Is it possible to get the source code for version 2.00 of Newton for Mac? (Specifically, the source for the lib that you can download from downloads.php called "MacSDK_SDK_2.00")

In 2009 we built an iPhone game using Newton 2.00 libs (before it was open source). Apple has recently retired armv6 support, and we need to recompile the Newton library.

[Julio Jerez]

what do you mean?, the archive contain all the source code for all platforms.
I do not have any code other that what is in SVN. I use to use visual source safe until Visual studion 2003, but after window vista no longer works and I do not have that database anymore
the source of the earliest version of core 200, and all core 100 does not exist.
why don't you use core 300 anyway.

[Julio Jerez]

core 300 compile with xcode just fine, and it is even more compaitible.
you will only need to make minor change to move to core 300 form core 200

[michDS]

I doesn't look like core 300 supports Continuous Collision Mode yet, which I use in my game. Is that right?

[Julio Jerez]

CCD will be in probably some time today. Continue collsion in core 300 will be much better that in any other version

also the lastest version of core 200 has the sorce, coudl you uese that?
Even I I hav eteh sorce control alchive, I do no really have the sorce for each release, one I publishes I continueworking on it.
the latest version or core 200 with source would be the first version that was realesed as open source

[michDS]

Hi Julio,

While using the latest version of core 200 I encountered a bug with it that you've actually started helping me with in the past. I didn't have a PC demo for you to test with back then, but last night I created a very simple PC demo demonstrating the problem (based on the NewtonDemos that comes with Newton).

I'm not sure if you remember, but my game is a shuffleboard game and the bug has to do with the puck veering left or right when it slides down the table (original forum post: viewtopic.php?f=9&t=6828&hilit=Sliding+Object+Behavior).

My demo shows a puck (cylinder) sliding on a flat surface, and at some point veering to the right for no apparent reason. It even does this after setting the friction coefficients to 0:
NewtonMaterialSetContactFrictionCoef(pMaterial, 0.0f, 0.0f, 0);
NewtonMaterialSetContactFrictionCoef(pMaterial, 0.0f, 0.0f, 1);

The demo also draws the friction force components and the contacts.

If you could download and test my demo to see if there's a bug in the Newton engine, that would be very helpful. Here is the demo:
http://www.10pinshuffle.com/test-demo-n ... s-2.35.rar

After extracting, you can find the demo project file at:
newton-dynamics-2.35\applications\newtonDemos\projects\visualStudio_2008\newtonDemos.sln

You just need to compile and run, and you can see what I'm talking about.

Please let me know if you have any questions. I'm at the point where I need to update the game on iOS, and Apple removed armv6 support, so I need to stop using the old version of Newton (2.0) that came as a pre-compiled lib, and use the newer open-source version of Newton. Unfortunately this bug is holding me back from submitting the update, so any help would be greatly appreciated.

Thanks,
Michelle

[Julio Jerez]

Ok I download it, and I see the problem.
I beleieve the problem happen because the ditibution of the contacts when the puck move alone the edge.
The problem is that the solver tah calculate the force to drvbe the puck is a quadratic contrained optimization.

Ther are three type of symnate confioguartion tha can by imput to the solver.
-Under determine, when ther are few unknon than teh monimu to mak eteh systme stable
-Demetnined wn eteh numbe of unnkon is teh numbe of Degree of fredom that mak eth system stable
-Over detremone whne you have more unknonow that DOF

In the first and secudn case a quadric optimization can find a unique solution.
it is the the thord scase that poses the bigest problem, when you have more unknow that the mimun nessesary, the system solve problem by dustribute the force is such way teh minimized the energy
the probem is tah tehr are an infinite number of solutions to teh problems, each one is correct.
It is onle afte teh body is integrated that you cna detremine that one solution was no the better tha oeth because afet teh integration it increase eneregy.

There are pivoting methods that can remove the unsessary variable from the system from the formulation, but they are very expensive for a realtime phusics engine.
Basically these method performe linear combination of rows until they find what row are equivalnt to anoeth rwo and therefore no essencial for a solution and can be set to zero.
This is very costly

However in soem case liek teh one in the demo what can solve the problem from a callback, basically all we need to do cehc i steh contact form a conev plane, if teh do then we can remove contact that are no symetric
leaving only the 4 most essencial contacts. the we align the contacts tangent to the diretion of the track, this will remove all sideway forces generated by the solver

I will try to do that, but no for core 200, I will do for core 300 this sartuday then you can try make teh change for your game and move to core 300.
But the the CCD will be ready

All in All for case like this is no really a big problem, and very eassy to solve.

[michDS]

That's interesting, thanks.

I'd prefer to stay with core 2.35 for now because we've already done extensive testing of our game with that version (we'd like to update to 300 when we have more time). Would it be possible to fix this using 2.35 for now?

[Julio Jerez]

Ok, I started to refactore the demo to work in core 300, and I found some obvious problems in your demo. I am surprise it even work as well as it does.

The first problem is that the puck shape is a cylinder, however cylinder defined alone the x axis, that would be OK if you use the inertia matrix calculator. but you calculate yourself as if the axis was alone Y
The make a irregular body with and inertia matrix that does no math the collision shape, This is not wrong, the body is actually behaving as it should, but the visual representation do no match the physics.
The problem is that since the inertia is asymmetric the solver will distribute the forces and torques according to eth mass distributions, so after each step the sum of all torque will not be zero.
To solve that it is better the pass a offset matrix when you create the cylinder.
I did that and it work much better.

The second problem is the rounding errors, I was mention before, The cylinder has more contact the the minimum necessary to find a optimal solution.
I mention that we can remove contacts, to make it better but I also mention that you should align the friction tangent with the direction of the table.
I thought were doing that and it was still deviating to the side, but when I saw the contact call back you were only making the friction zero.
This is theoretically correct but In the Newton engine that is no who you make a collision frictionless.
if you look at function NewtonMaterialSetContactFrictionCoef you will see that the minimun friction value is clamped to 0.01, This In the Newton engine you cannot have a constraint with zero force,
The result of this is that there will be some residual side forces that will rotate push the puck to the side just a little in each step. Thsi samll force when acting for long time, is enought to make it deviate significantly from the trajectory.

Even with those ill setup, the deviation should not be as severe as I see in the demo. I check the contact call back I see that you do not align the tangent friction to the table direction.
I saw that you tryed to do it but it was not right, I simple added that call and now the deviation is less than one part in 1 thousand which is well inside of 32 bit float accuracy.
The deviation cannot be zero because as I mention before you are no making the motion over the table friction less, the tangent friction have a tiny values, this make the solver work with large number and small number and the and 32 bit float lose Precision.
If you want perfect frictionless motion you need to make the material between the table and the Puck frictionless using the function in the call back
NewtonMaterialSetContactFrictionState (material, 0, 0);
NewtonMaterialSetContactFrictionState (material, 0, 1);

This will remove the friction row for the solver. I did not do that but you should try and see how it works

Here is the code with the changes I mention please see if it works now.

Code: Select all

/* Copyright (c) <2009> <Newton Game Dynamics>
*
* This software is provided 'as-is', without any express or implied
* warranty. In no event will the authors be held liable for any damages
* arising from the use of this software.
*
* Permission is granted to anyone to use this software for any purpose,
* including commercial applications, and to alter it and redistribute it
* freely
*/

#include <toolbox_stdafx.h>
#include "../DemoEntityManager.h"
#include "../DemoCamera.h"
#include "PhysicsUtils.h"
#include "../DemoMesh.h"
#include "../toolBox/OpenGlUtil.h"

#define PHYSICS_WORLD_SCALE         50.0f                                 // conversion from meters to world units

#define TABLE_WIDTH               (0.508f * PHYSICS_WORLD_SCALE)               // 20 inches
#define TABLE_LENGTH            (5.08f * PHYSICS_WORLD_SCALE)               // 200 inches (16' 8")
#define TABLE_HEIGHT            (0.0762f * PHYSICS_WORLD_SCALE)               // 3 inches

#define WEIGHT_DIAMETER            (0.062f * PHYSICS_WORLD_SCALE)
#define WEIGHT_RADIUS            (WEIGHT_DIAMETER * 0.5f)
#define WEIGHT_HEIGHT            (0.025f * PHYSICS_WORLD_SCALE)

#define WEIGHT_MASS               0.360f                                 // weight of a puck in Kg (mass doesn't need scaling)

#define CAMERA_X               (-11.0f)
#define CAMERA_Y               (0.26f * PHYSICS_WORLD_SCALE)
#define CAMERA_Z               (-3.0f * PHYSICS_WORLD_SCALE)

// Material data
enum eSBMaterials
{
   SBMaterial_WEIGHT,
   SBMaterial_SURFACE,

   SB_NUM_MATERIALS
};

NewtonBody* gPuckBody = NULL;
NewtonWorld* gWorld = NULL;

static void   NewtonRigidBodySetForceCB(const NewtonBody* const body, float timestep, int threadIndex);
static void PhysicsNewton_CollisionPuckSurfaceCB(const NewtonJoint *pContactJoint,dFloat fTimeStep,int ThreadIndex);
static void RenderBodyContactsAndTangentDiretions (NewtonBody* const body, float length);

// create physics scene
void PuckSlide (DemoEntityManager* const scene)
{
   NewtonWorld* const world = scene->GetNewton();
   gWorld = world;

   int   materialGroupIDs[SB_NUM_MATERIALS];

   // Create groups
   for (int i = 0; i < SB_NUM_MATERIALS; i++)
   {
      materialGroupIDs[i] = NewtonMaterialCreateGroupID(world);
   }
   
   // Setup the material data
   NewtonMaterialSetDefaultSoftness(world, materialGroupIDs[SBMaterial_WEIGHT], materialGroupIDs[SBMaterial_SURFACE], 0.15f);
   NewtonMaterialSetDefaultElasticity(world, materialGroupIDs[SBMaterial_WEIGHT], materialGroupIDs[SBMaterial_SURFACE], 0.30f);
   NewtonMaterialSetDefaultFriction(world, materialGroupIDs[SBMaterial_WEIGHT], materialGroupIDs[SBMaterial_SURFACE], 0.05f, 0.04f);

   // setup callbacks for collisions between two material groups
   NewtonMaterialSetCollisionCallback(world,materialGroupIDs[SBMaterial_WEIGHT],materialGroupIDs[SBMaterial_SURFACE],NULL,NULL,PhysicsNewton_CollisionPuckSurfaceCB);

   ///////
   // Add table
   {
      dVector tableSize(TABLE_LENGTH, TABLE_HEIGHT, TABLE_WIDTH, 0.0f);

      // create the shape and visual mesh as a common data to be re used
      NewtonCollision* const collision = CreateConvexCollision (world, GetIdentityMatrix(), tableSize, _BOX_PRIMITIVE, materialGroupIDs[SBMaterial_SURFACE]);

      DemoMesh* const geometry = new DemoMesh("cylinder_1", collision, "wood_3.tga", "wood_3.tga", "wood_3.tga");

      dMatrix matrix = GetIdentityMatrix();
      matrix.m_posit.m_x = 0.0f;
      matrix.m_posit.m_z = 0.0f;
      matrix.m_posit.m_y = 0.0f;
      NewtonBody *tableBody = CreateSimpleSolid (scene, geometry, 0.0, matrix, collision, materialGroupIDs[SBMaterial_SURFACE]);

      NewtonBodySetMassMatrix(tableBody, 0.0f, 1.0f, 1.0f, 1.0f);
      NewtonBodySetMaterialGroupID(tableBody, materialGroupIDs[SBMaterial_SURFACE]);
      NewtonBodySetContinuousCollisionMode(tableBody, 1);

      // do not forget to release the assets   
      geometry->Release();
//      NewtonDestroyCollision (collision);
   }
   ///////

   // Add puck
   {
      dVector puckSize(WEIGHT_DIAMETER, WEIGHT_HEIGHT, 0.0f, 0.0f);

      // create the shape and visual mesh as a common data to be re used
      dMatrix offsetMatrix = dRollMatrix(3.141592f/2.0f);
      //NewtonCollision* const collision = CreateConvexCollision (world, GetIdentityMatrix(), puckSize, _CYLINDER_PRIMITIVE, materialGroupIDs[SBMaterial_WEIGHT], &offsetMatrix[0][0]);
      NewtonCollision* const collision = NewtonCreateCylinder (world, puckSize.m_x * 0.5f, puckSize.m_y, materialGroupIDs[SBMaterial_WEIGHT], &offsetMatrix[0][0]);

      DemoMesh* const geometry = new DemoMesh("cylinder_1", collision, "smilli.tga", "smilli.tga", "smilli.tga");

      //dMatrix matrix = dRollMatrix(3.141592f/2.0f);
      dMatrix matrix (GetIdentityMatrix());
      matrix.m_posit.m_x = -TABLE_LENGTH*0.5f+WEIGHT_DIAMETER;
      matrix.m_posit.m_z = -11.8f;
      matrix.m_posit.m_y = 5.0f;
      gPuckBody = CreateSimpleSolid (scene, geometry, WEIGHT_MASS, matrix, collision, materialGroupIDs[SBMaterial_WEIGHT]);

      // Set moment of inertia
      dVector I;
      float Mass = WEIGHT_MASS;
      float Radius = WEIGHT_RADIUS;
      float Height = WEIGHT_HEIGHT;
      I.m_x = I.m_z = Mass*(3.0f*Radius*Radius+Height*Height)/12.0f;
      I.m_y = Mass*Radius*Radius/2.0f;
      NewtonBodySetMassMatrix(gPuckBody,Mass, I.m_x, I.m_y, I.m_z);   

      NewtonBodySetMaterialGroupID(gPuckBody, materialGroupIDs[SBMaterial_WEIGHT]);
      NewtonBodySetContinuousCollisionMode(gPuckBody, 1);
      NewtonBodySetAutoSleep(gPuckBody, 1);

      // Set callbacks
      NewtonBodySetForceAndTorqueCallback(gPuckBody, NewtonRigidBodySetForceCB);

      // do not forget to release the assets
      geometry->Release();
//      NewtonDestroyCollision (collision);
   }

   // place camera into position
   dMatrix camMatrix (dPitchMatrix(20.0f * 3.1416f /180.0f));
   dQuaternion rot (camMatrix);
   dVector origin (CAMERA_Z, CAMERA_Y, CAMERA_X, 0.0f);
   scene->SetCameraMatrix(rot, origin);
}

void NewtonRigidBodySetForceCB(const NewtonBody* const body, float timestep, int threadIndex)
{   
   float mass;
   float Ixx;
   float Iyy;
   float Izz;
   NewtonBodyGetMassMatrix(body, &mass, &Ixx, &Iyy, &Izz);
   
   float force[3];
   force[0] = 0.0f;
   force[1] = mass * (-9.81f * PHYSICS_WORLD_SCALE);
   force[2] = 0.0f;
   NewtonBodySetForce(body, force);
}

static void PhysicsNewton_CollisionPuckSurfaceCB(const NewtonJoint *pContactJoint,dFloat fTimeStep,int ThreadIndex)
{
   dVector Position;
   
   // Get pointer to body
   NewtonBody* body = NewtonJointGetBody0(pContactJoint);            
   float mass, Ixx, Iyy, Izz;
   NewtonBodyGetMassMatrix(body, &mass, &Ixx, &Iyy, &Izz);
   if (mass == 0.0f)
   {
      body = NewtonJointGetBody1(pContactJoint);
   }

   // Test to see if it is the friction calculation that is causing the side force
   // With this the Puck must go straight because it will be frictionless, the net force should not change direction
   dVector tableDir(0.0f, 0.0f, 1.0f, 0.0f);
   for (void* contact = NewtonContactJointGetFirstContact (pContactJoint); contact; contact = NewtonContactJointGetNextContact (pContactJoint, contact))
   {
      NewtonMaterial* const material = NewtonContactGetMaterial (contact);
      // align fiction tangent to the puck motion
      NewtonMaterialContactRotateTangentDirections(material, &tableDir[0]);

      // uncoimen thsi to see ho eth tange are rotated
      //dVector dir0;
      //dVector dir1;
      //NewtonMaterialGetContactTangentDirections (material, body, &dir0[0], &dir1[0]);
      NewtonMaterialSetContactFrictionCoef (material, 0.0f, 0.0f, 0);
      NewtonMaterialSetContactFrictionCoef (material, 0.0f, 0.0f, 1);
   }
}

void LaunchPuck()
{
   static bool bLaunched = false;

   if (bLaunched)
      return;

   if (gPuckBody)
   {
      bLaunched = true;

      NewtonInvalidateCache (gWorld);
      dVector zeros(0.0f, 0.0f, 0.0f, 0.0f);
      NewtonBodySetVelocity(gPuckBody, &zeros.m_x);
      NewtonBodySetOmega(gPuckBody, &zeros.m_x);
      NewtonBodySetForce(gPuckBody, &zeros.m_x);
      NewtonBodySetTorque(gPuckBody, &zeros.m_x);

      dVector vel(171.299469f, 0.0f, 0.0f);
      NewtonBodySetVelocity(gPuckBody, &vel.m_x);
   }
}

void UpdateStuff(DemoEntityManager *scene)
{
   if (gPuckBody)
   {
      int sleepState = NewtonBodyGetSleepState(gPuckBody);
      if (sleepState)
      {
         LaunchPuck();
      }

      // Update the camera to follow puck
      dMatrix rotMatrix (dPitchMatrix(20.0f * 3.1416f /180.0f));
      dQuaternion rot (rotMatrix);
   
      DemoEntity* demoEntity = (DemoEntity*)NewtonBodyGetUserData (gPuckBody);
      float camZ = demoEntity->GetCurrentMatrix().m_posit.m_x - 30.0f;

      DemoCamera* demoCamera = scene->GetCamera___();
      dMatrix camMatrix = demoCamera->GetNextMatrix();
      camMatrix.m_posit.m_x = camZ;

      demoCamera->SetMatrix(*scene, rot, camMatrix.m_posit);
   }
}

Later I will port this demo to core 300, so tha you can see how you cn amigrate to the new version, There I will try to do the ccontact reduction I mention before if it is needed, But I thonk it will no be nesssary
I hope this solve those problems.

[Julio Jerez]

Ok I ported your demo to core 300 and it work flawlessly without vey monoir refactoring.

a new feature of Newton core 300 are pre and post listeners, these are really good for schedualing game play logic
I make teh Puck update use a pre lisnener, an dth awa I can work with all teh oethe demos.
It also solves the memory leack that are left vehind in the core 200 demo

I did not have to do anything to the engine else and and it seems to work very well.
if you sync to SVN you cna see hwo it is set up, an dmaybe you cna migrate to core 300, ther are lot of new features.

Core 300 now has the continues collision, There is an assert in debug but I will fix that this weekend.
the big diffrentce with CCD in core 300 with repsect to core 200 is that Now continue collision is physically accurare, It preserve Consrvation of Momentum and energy and the moment of contact,
it also calcualte exact momnet of impact. In core 200 the contact were calculated auside the collision shapes, and that sometime lead to vilation of consrvation of momenetum.
These are the thing that makes game like your behave as ther player expect all the time. I am placing a lot of effort on making CCD as acurate as the descrete collision and not a check box or a fail safe feature as it is in all physics engines.
you can just test it and see if works.

[michDS]

Thanks Julio. I will try everything today. One thing I noticed in your modified code of the PuckSlide demo is that you have commented out the setting of the puck to continuous collision mode, so I'm not sure if you actually ran it with continuous collision turned on:

//NewtonBodySetContinuousCollisionMode(PuckBody, 1);

Also, I actually don't want frictionless motion, I only added it for testing purposes. So I will remove the following lines:

NewtonMaterialSetContactFrictionCoef (material, 0.0f, 0.0f, 0);
NewtonMaterialSetContactFrictionCoef (material, 0.0f, 0.0f, 1);

[Julio Jerez]

The CCD comment was because I has an assert which I fixed and I forget to uncomnet that line.
you can remove the comment, and it should not make any difference
NewtonBodySetContinuousCollisionMode(m_puckBody, 1);

you can also remove the friction line too, but you msut set teh friction fo some realistic low value, not zero
NewtonMaterialSetContactFrictionCoef (material, 0.0f, 0.0f, 0);
NewtonMaterialSetContactFrictionCoef (material, 0.0f, 0.0f, 1);

[michDS]

I tried migrating to core 300 and I'm getting a crash in dgMutexThread::TickCallback right away. It looks like the dgMutexThread and dgAsyncThread start executing too early? It seems like they shouldn't start until the newton world is fully initialized.

Also for testing purposes I added the following lines to see what happens with frictionless motion (in my demo), but the Puck doesn't launch. It slowly moves to the left until it falls off the table.

NewtonMaterialSetContactFrictionState (material, 0, 0);
NewtonMaterialSetContactFrictionState (material, 0, 1);

Any ideas?

[Julio Jerez]

Ok the problem when setting mtion to be firtion less in that teh body never go to sleep, it simpel slide on teh table and demo use teh sleep state to apply the impules to teh puck

Code: Select all

   void UpdateStuff(DemoEntityManager* const scene)
   {
// commnet thsi if you mak eteh motion funrionless.
//      int sleepState = NewtonBodyGetSleepState(m_puckBody);
//      if (sleepState)
      {
         LaunchPuck();
      }
   }
 

The second problem, yes the engine is now run in a separate thread that execute immediately.
This allow the engine to run concurrently with the application and give almost driver like behavior similar to OpenGL and D3D, this has being tested in Linux Mac and Windows, Iphone and Google Phones
what OS are you using see this problem?

There is a define that replace the thread with sequential code
//#define DG_USE_THREAD_EMULATION
in file ..\coreLibrary_300\source\core\dgThread.h
if you uncomnet it it will disable all the threading code.
I will only do that fo debugging, because the tread make teh code run faster even on systmes with a single core. I can no explained but this is wha my test show.


if you sinc again, the demo now has those changes

[michDS]

Hi,

I'm curious, why doesn't the body go to sleep in the frictionless case, and why does it slide to the left?

My other question is about setting the inertia. If I pass an offset matrix when creating the cylinder collision (like you have done, and I also do this in my game), would the following inertia calculation be correct, or is this still wrong? I understand that newton 300 calculates this for you, but am wondering how I would do it correctly using newton 200:

dVector I;
float Mass = WEIGHT_MASS;
float Radius = WEIGHT_RADIUS;
float Height = WEIGHT_HEIGHT;
I.m_x = I.m_z = Mass*(3.0f*Radius*Radius+Height*Height)/12.0f;
I.m_y = Mass*Radius*Radius/2.0f;
NewtonBodySetMassMatrix(gPuckBody,Mass, I.m_x, I.m_y, I.m_z);

I'll try disabling DG_USE_THREAD_EMULATION now.

Thanks!