Request: Add NewtonUserJointSetRowForce

[Julio Jerez]

ok I just recreated that the bug, I modified the swing hinges to be more or less like the contraction that you showed, and if I apply a gravity of 100 horizontally, dismember a lithe but is recover, if I apply vertically it state stable.
the if I apply a gravity of 200 horizontally is separates a lot, and after that is simple blow up. this show that the is a bad bug there that can not be justified by just numerical error, of the pick force.
I will debug this over the week end because I need verified the math, there most be a sign wrong some where.
I believe this bug is not related to the skeleton, is part of the solver right hand side vector, some how I have a term wrong there.

[Julio Jerez]

Ok AntonSynytsia
I look at the code, and I can not say I found a real problem with the code,
I do find there problems:
-The first is the eassy one and this is in newton since newton 2.0 there with a bug with the way Joint stiffness is set. I am actually scale a dimention less value for 0, to 1 to a value form 1-e3 to I a cap of one hundred. The problem was that is should have being from 0 to 1.e-3 this was making the joint soft even at the setting. I fix that but I do not know all of the implications, I believe is should be fine. Joint should be a little better and collision also more snappy.

the secund bug is the one that I can fix. Basically newton uses a semi implicit integrator.
This integration has an O(dt ^ 3) accuracy which is better that simple euler which is O(dt ^2)
The solve is a RH order 4, and there is no real problem there. the calculation are as accurate as there can be.
It is the euler integrator that fail when when operate on highly not linear equation.
so basically even when the calculation of all of the internal reaction forces are calculated wit a small error, because there forces lest are still too large, then the integrator can not converge to a solution fast enough.
what this mean is that even more iterations count can not fix the bug. the only thing that can make is better is to run at a higher frame rate, or incorporate the integrator wit the solver to integrate position and orientation, that's a very big task

The last problem is the one we can control you said that you are using same function that I am using to pick objects. my function is this

Code: Select all

oid CalculatePickForceAndTorque (const NewtonBody* const body, const dVector& pointOnBodyInGlobalSpace, const dVector& targetPositionInGlobalSpace, dFloat timestep)

if you look at the place when the force and torque is calculates:

dVector force ((veloc1 - veloc0).Scale (mass * invTimeStep));
dVector torque (angularMomentum.Scale(invTimeStep));

you can see what I was talking about before. The force is base on the different between tow position scale by the mass of the body. the toque is a function of how much the angle error has changed, therefore the torque is always much smaller than actual force.

This is why you see as if the angular constraint are stronger but in reality there what you are seeing is a much more stronger force.

a way to fix the problem is no to scale the acceleration by the mass, but by a suer specific sensitivity
like this

Code: Select all

dVector force ((veloc1 - veloc0).Scale (UserSensitivity * invTimeStep));

finally, I made a demo similar to your but my links are 1.5 and there are 15 link long, this mean the last mass is a 22.5 meters away from the origin and weight 50 times more.
as along as I do not pull it away it works fine.
I did that for testing, the version I checking I set the mass to 10. and it seem very robust.

also, there one more thing, your test is very extreme. Usually mechanism are not made to lift object that are 50 as massive. by you is even worse that that, you are placing several links away for the origin, this has a amplification effect, basically is to 50 to 1, this is 50 time the number of links to 1.
but is even more insidious because in addition you also set the gravity to 100, instead to 10.
it is very unlikely you have a mechanical system with those ratios.
I am glad you should me because I gave the change to investigate the problem some more.

last I did try position correction again, by this suffer form the same problem, mainly the impulse to correct there error, has the same degree of non linearity, so it will suffer from the same problems.

The only way that I know that problem can be solve is using a Langragian or Hamiltonian generalized coordinate system formulation, when you get equation of motion of the system.
Bu them Langragian systems are incompatible with Newtonian system, because while Langragian gives the equation of motions along the degree of freedom, Newtonian give you a set of forces along all 6 degree of freedom, an dis hard to mix them.

Anyway please sync and try again see if this is a lithe better.

[AntonSynytsia]

Hi Juleo, it does behave quite better now. It's true that the forces applied were quite too high. And as you said, I too don't think that linked bodies with such low mass would withstand such force in real world scenarios. But the test was kind of worth it, as it did help you improve the skeleton. Thanks for the improvement.

[Julio Jerez]

could you make another video, so that I can see the difference?

[JoeJ]

The idea i have to solve the huge force problem is this:

For every joint use RowMaxFriction to set the maximum force the joint can apply,
so if the force would go higher, the joint drifts appart but does not jitter.

In every joint measure the distance of both parts to see if they drifted appart.
If this distance is higher than given threshold, break the joint.
For sure this works only where game design allows dismemberment.

Downsides:

Deleting a broken joint means deleting the skeleton and recreating two new skeletons for the remaining stuff - i have not tried to see if this has bad effect on simulation.
It may also work to keep the joint but do nothing inside it.

Enabling max friction for everything has performance cost.

[Julio Jerez]

I think we can also do something else. I refuse to accept that this is the end of the story.
I found the problem but what I meant is that is outside of the Newton strategy to deal with it,
It needs a different approach and I believe we can do much better that what we do now.

In newton the joint and the contact are made such that the constraint error is resolved in such a way that the error is reduced by adding a penalty force that moves the two bodies as a contact velocity.
basically add a fix extra velocity based potential energy.
This has worked well if you have a good constraint solver and a poor velocity integrator like Newton has.

What many physics engines do is that the add what is called Burgante penalty, some other use the so call mixed constraint solver and some others do two solver passes one for force and one for position error. All three of these approached are quite horrible to the point that I do not even consider doing a position correction pass a physical based solution and why I have refuse to use those methods for years.

This is why I also say that engine like Physx and Havoc and all the others are not really physic engines they are collision system with a mix of poor dynamics and poor inverse kinematic (position correction is in fact a special case of Holonomic inverse kinematics).
What this mean is inverse kinematic when the number of variable is equal to the number of degree of freedom.

The problem with the Newton method of reducing the errors by adding a constant velocity is that it breaks down if the integration is not capable of reducing there error faster than what the dynamics of system the can generates. This is what I had forgotten and why I am having such a difficult time tuning the vehicle made of pure rigid bodies.
It had worked so far because I always assumed that when two joints violate constraint, the solver will clip the excessive forces and after that these forces will not be there anymore, example of this are contacts and joint that move a relatively low speed.
however if you have cases where the excessive force come out of the integrator itself, then the error becomes a positive feedback and is beyond what the method can solve.

however now that I have identified the error, I believe the solution is adding the option to have elastic constraint rows behave as a stiff spring damping system.

Joe as I mentioned before to you, basically this is the equivalent of making the constraint equation

Code: Select all

f * trans(a) = transp (Fe) + transp (J) * lambda
J dot a + ks * lamda = kc * constraintError


instead of

Code: Select all

f * trans(a) = transp (Fe) + transp (J) * lambda
J dot a = ConstraiontVelocityError * (constantvelocity)


As you can see in my method, the second system, the velocity error is reduced at a fix rate, you can test that by polling to joints and see how the come back together at a constant speed.
The way I arrived at that solution was by using a Langragian Formulation of two separated system
one is a contact point that constraint only one degree of freedom to be fixed to both bodies.
the second constraint one angular degree of freedom to be fixed to both bodies.
then differential equation that you get is used for the linear and angular rows.
I derived that about 15 years ago long before I made Newton public and now I need to refresh myself my memory again because the Lagrangian equation that you get you need to do some interpretation to get the meaning of the turns.
BTW this is why Newton also handles Coriolis and Gyroscopic forces naturally, they all come out for the Lagrangian derivation which is based of the Kenetic Energy and Virtual potential energy in the close system. No other engine that I know can handle that because either they simply use a penalty that clip the error or use a spring damper.
In Newton what I decided was that I will add a fixed extra amount of a Velocity Based Potential Energy to the Lagrangian and derive the equations of motion along that degree of freedom.
Since Velocity based potential energy are negative, they drain a fix amount of energy from the system, and as long as that amount is larger that what Numerical error can add to the system, them the solution is unconditionally stable. This has worked so well for many problems.

In the first equation you can see what is going on, the masses are artificially relaxed, so the diagonal will be larger that is really is so the calculated force will be smaller that what is needed to maintain the constraint. On the right side you it does not consider velocity or acceleration, instead is uses the position error and a penalty constant, this in fact is a acceleration base spring.
What I do not like about that is that the system to find an equilibrium point it requires a error.
Bu the error can be arbitrarily small at the expense of high frequencies instability, all in all is a bad system, but a right tool for the right problem I guess.

you can make by comparing the dimension of a spring and what at the term is
for a spring F = -Ks * x
therefore Ks = f / x -> [kg / sec ^ 2]

for the constraint the right side is the acceleration error so we have
a = - ks * x
ks = a / x -> [ 1 / sec ^ 2]

similarly the added tum to the left side must be a damper units because the unit are not a damper because for a damper we have
F = - Kc * V
therefore Kc = F / V -> [Kg / s]

but we know that Lambda has units of forces. therefore the penalty must have using is
inv (mass) * lambda + kc * lambda = a

the dimensional analyst yield that

inv (mass) * lambda + kc * lambda -> K must have units of [1/ kg]
basically is artificially increase the inverse mass ration of seen by the constraint

What I can do now that I know that for extreme systems like the one in the videos,
I can formulate another Langragian using, both a spring and a velocity based potential energy, and
from there derive a new set of equations that can added using the newton function
NewtonUserJointSetRowSpringDamperAcceleration which is already there.

I am working on getting that feature in because I am having a difficult time with the vehicle model and tire moving a high angular velocity while the mass ratios is also extreme.

[JoeJ]

Sadly i'm far not expert enough to discuss this with you ,
but i have this strong belly feeling that this will be big progress

[Julio Jerez]

Ok I believe now I have support for elastic spring damper joints.
I quickly replaced the function to calculate the tire spring forces for the vehicle with a spring damper row and seems to produce the same result, but I believe this is more robust and stable than when I do it but calculation the force myself and adding as external forces.
with these mode the force are precisely calculated and simply the model a lot.

The car now slide all over the place because now the tire model is broken, this is because when I calculate the tire spring forces I know what the tire load is, and I can use that for the tire lateral and longitudinal forces.

Now That valise is not known until the solve finish, however I will add and new option to the contact call back that aloe to pass the tire longitudinal and lateral stiffness this way the tire model will do the calculation at ether correct tire with the correct values. as oppose to what is doing not which is and approximation base if the calculate spring force.
for example is a tire hit tow contacts, the model will be wrong because it will apply tire force twice, but with the \solve doing it if a tire hit tow contacts, the tire load is smaller and the fore will be appropriately calculated.
I have no tested with the larger mass ratios I will do it later.

[JoeJ]

Wanted to see if the spring damper improves my bad contact forces with skeleton problem,
but any ragdoll bahaviour is broken on my side and fixing would drag me away from current task,
so i only can quick test a T-pose ragdoll.

Surprisingly, using skeletion works much better here than not using skeleton, there is no contact or any other problem.
Now i can't remember if it did show up only with behaviour enabled, but it works also in the older Newton i've used the last. So if it's an improvement on your side, it's some months old.

Anyways, it seems skeleton is ready for controlled ragdoll, so i'll switch to it when i get back on this...

[AntonSynytsia]

could you make another video, so that I can see the difference?

Hi Juleo, I was a little too busy and couldn't post it before. But here it is. I actually compared two revisions:

1. Revision 2503 (23b8907 at github), the revision before you started "adding the support for elastic joints rows and force". I.E when the custom joints without skeletons still worked.
2. Revision 2517 (574d7e5 at github), the revision after you started "adding the support for elastic joints rows and force" where custom joints without skeletons no longer work (only joints with skeletons work).

https://drive.google.com/open?id=0B3qg8 ... G85Q2E5cTA

Just to clarify, the gravity is zero. The force of 5000 newtons is only applied to the green box. The results in the videos are not much different compared to the prior videos. So actually, linear rows haven't got much better compared to what they were before. However, they do realign a little faster than in prior versions. The force applied is too high, so I'm not asking for improvement of the skeleton container. The video is just to display the difference between the prior video as you requested.

[Julio Jerez]

Just to clarify, the gravity is zero. The force of 5000 newtons is only applied to the green box. The results in the videos are not much different compared to the prior videos. So actually, linear rows haven't got much better compared to what they were before. However, they do realign a little faster than in prior versions. The force applied is too high, so I'm not asking for improvement of the skeleton container. The video is just to display the difference between the prior video as you requested.

yes I appreciated that, yes I added support fop the elastic joint, but the interface is not there yet,
I now have to go over the all joint so that the funtionality for controlling elasticity and softness.
All I did was to set the default a lithe higher and you see the it tends to do a better job at recovering but it is still trying to resolve the joints error at a constant velocity rather than at a speed proportional to the error. Let me finish what I doing now and I will get back to the demos and set them up right.

This is a extra bonus but if we can solve it will only make the engine stronger

[AntonSynytsia]

Okay Julio. I'm looking forward to it.

[Julio Jerez]

[*]Revision 2517 (574d7e5 at github), the revision after you started "adding the support for elastic joints rows and force" where custom joints without skeletons no longer work (only joints with skeletons work).

is this true?, they should all be working

[AntonSynytsia]

Oops, I have to correct myself. The standard custom joints still work, but the flexible mode of my custom joints no longer works. I have 3 modes to control the flexibility and robustness. I use the flexible mode for chains/tracks as, in my experience, they seem to behave better than without applying the spring damper acceleration.

Code: Select all

const Joint::LINEAR_STIFF = 1.5e8f;
const Joint::LINEAR_DAMP = 0.1e8f;
...
NewtonUserJointAddLinearRow(joint, &pos0[0], &pos1[0], &matrix0.m_front[0]);
NewtonUserJointSetRowStiffness(joint, joint_data->stiffness);
if (joint_data->ctype == CT_FLEXIBLE)
  NewtonUserJointSetRowSpringDamperAcceleration(joint, Joint::LINEAR_STIFF, Joint::LINEAR_DAMP);
else if (joint_data->ctype == CT_ROBUST)
  NewtonUserJointSetRowAcceleration(joint, NewtonUserCalculateRowZeroAccelaration(joint));


I think the flexible mode no longer works because the NewtonUserJointSetRowSpringDamperAcceleration function was changed in the new revisions.

[AntonSynytsia]

I figured how to fix the effect. I noticed that new dgBilateralConstraint::SetSpringDamperAcceleration function controls joint stiffness desc.m_jointStiffness[index] = - den / DG_PSD_DAMP_TOL ;. I figured that calling NewtonUserJointSetRowStiffness after NewtonUserJointSetSpringDamperAcceleration function rather than prior to returns the original effect. So, anyway it works perfectly now.