Speed up Newton

[arkdemon]

Hello everyone. I am currently testing Newton's performance by creating thousands of boxes and I see that it gets laggy after around 1000 boxes (Model = 2). So I wanted to know if there was a way to speed up the physics step to include it in a project that I am trying to make. (I really love Newton, it is accurate, flexible and allows me to do whatever I want) even if it means less accurate physics.

I also wanted to know if Newton uses acceleration structures like BVH.

Thank you very much for your answers.

[AntonSynytsia]

I'm not an expert, but I do have some experience, currently with Newton 3.12.

1. Your major performance issue could be threads. If your application is unstable with threads, recompile Newton with threads disabled. I did this in my project and performance became way better. On the other hand, if your application is stable with threads, I would use threads as they are meant exactly for this purpose (threads are enabled by default); although, I wouldn't mind comparing the both. See this post for instructions on how to compile Newton with the threads disabled: viewtopic.php?f=9&t=8508#p55371
2. Set solver model to 1 (adaptive). Adaptive solver is where accuracy is less important than speed. Solver model of 2 wouldn't make any performance differences when compared to solver model of 1. Make sure solver model is not 0 (exact) though. See NewtonSetSolverModel.
3. Set friction model to 1. Friction model doesn't affect performance much, but the docs say, adaptive coulomb is about 10% faster than exact coulomb. See NewtonSetFrictionModel.
4. Verify that auto sleep is enabled for all bodies. See NewtonBodySetAutoSleep
5. Use simulation islands. Simulation islands allow you to decide whether or not to process the body. For instance, you could use simulation islands to process bodies that are in the camera view. See this post: viewtopic.php?f=9&t=6923&hilit=frozen+bodies#p47966

[arkdemon]

Hi Anton, thank you for replying and giving those good advices
1. I do not understand your statement "unstable"/"stable with threads". Do you mean that my application already uses threads or that it is "safe" using threads? I also think threads impact the performance a bit :p. Thank you
2. Thanks for the tip
3. I already knew this by searching on the internet :p
4. I think my NewtonBodies were already enabled with auto-sleep (but I now added NewtonBodySetAutoSleep when they were initialized so thanks :p)
5. I never understood and heard what simulation islands were(except by seeing it in the source of newton) and what they were used for but I think from what you said, it could be quite interesting to use them by approximating even more the physics for objects that are not in the camera view. Thanks . However, I think that excluding the entire bodies from being updated might be too much for the project I am trying to make which is an RTS game (the genre of Starcraft I and II but with space included, if you see what I mean).

Thank you very much for the advices you gave me . I will try to look the simulation islands in more depth.

PS: if ever Newton needs some acceleration structures, I found some quite interesting research papers on creating BVH (quickly) while searhing to implement photon mapping (which is a bit too complicated for me). Here are the links:
BVH creation - https://research.nvidia.com/publication ... -k-d-trees
Radix Sorting on the GPU with source code (on CUDA though :/ ) - https://code.google.com/p/back40computi ... dixSorting


Edit: I tested the code given in the link but there is a problem because when the objects are not in the camera frustum and they get into the camera frustum they are not updated to the current frame but to the frame they should have done one frame after if not behind the frustum. By that, I mean that if the newtonbody is behind the frustum for let's say 10 frames and then he gets back into the frustum, he will act as if only one frame has passed and not the 10 frames. Is it possible to change this behavior? Like by running NewtonUpdate not with the difference of time between each frame or 1 but with a timer which would resolve the island issue (or normal behavior).
For example, we would do this:

Code: Select all

NewtonUpdate(world, timer.elapsed()/*since the first frame*/);

and not:

Code: Select all

NewtonUpdate(world, timer.elapsed()/*since the last frame*/);

(which can also allow us to go in the past for replays)
Thank you

[AntonSynytsia]

1. I do not understand your statement "unstable"/"stable with threads". Do you mean that my application already uses threads or that it is "safe" using threads? I also think threads impact the performance a bit :p. Thank you

Newton 3 has threading feature. Threading feature allows you to split blocks of code into separate threads (processes) and execute/calculate them at the same time. In Newton, if threads are enabled, simulation bodies are distributed into threads. For instance, if you have 50 bodies and a maximum of 5 threads, your going to end-up with only 10 bodies being calculated in each thread, which may guarantee 5 times performance boost; however, the performance is altered if we include the time it takes to transfer data between threads and the application. By saying 'unstable' I mean the threads are slow at transferring/receiving data to/from application. And by saying 'stable with threads' I mean the threads are fast at transferring/receiving data.

5. I never understood and heard what simulation islands were(except by seeing it in the source of newton) and what they were used for but I think from what you said, it could be quite interesting to use them by approximating even more the physics for objects that are not in the camera view.

You may also use NewtonBodyEnableSimulation or NewtonBodyDisableSimulation. These functions may become very handy in gaming. If your camera is far from an object or a star planet then an object simulation can be disabled, and enabled when camera is in the desired distance to the object. These are new functions and they are only visible in source

[AntonSynytsia]

PS: if ever Newton needs some acceleration structures, I found some quite interesting research papers on creating BVH (quickly) while searhing to implement photon mapping (which is a bit too complicated for me). Here are the links:
BVH creation - https://research.nvidia.com/publication ... -k-d-trees
Radix Sorting on the GPU with source code (on CUDA though :/ ) - https://code.google.com/p/back40computi ... dixSorting

I barely have any knowledge about acceleration structures. Lets wait and see what Juleo, and other people have to say about this.

I tested the code given in the link but there is a problem because when the objects are not in the camera frustum and they get into the camera frustum they are not updated to the current frame but to the frame they should have done one frame after if not behind the frustum. By that, I mean that if the newtonbody is behind the frustum for let's say 10 frames and then he gets back into the frustum, he will act as if only one frame has passed and not the 10 frames. Is it possible to change this behavior? Like by running NewtonUpdate not with the difference of time between each frame or 1 but with a timer which would resolve the island issue (or normal behavior).
For example, we would do this:

Well, that's what simulation islands are for. When it comes to replays, well, I think it would be better to have all objects updating at a normal timestep. Simulation islands are quite of an interesting feature, but they could be useless for your game.

[arkdemon]

Hi. Thank you for the clarifications
I also think that islands might be useless in my case but it might help me in future, we just never know. ^^

For the acceleration structures, to be honest I do not have the knowledge to understand them fully but they should be useful for many things so once implemented they could be very useful (it would be cool if someone could implement them ).

[Julio Jerez]

newton has BVH yes, that what broad phase, the compound and the collision tree are.
has you implemented the performance counter to see where you bottleneck is

if you have a work with lot of bodies and they are not cluster together a lot the solver soul no be a problem, but is difficult to know with a profiler.
In the absent of a profiler, new has this performance ounter that measure the performance of one thread.

the one thong you can do is to run Newton asynchronous, this take some time to implement, by the engine have a mode that run concurrent with the you game, so as long as the game take say 1/60 fps
the physics step is free if you have more than one core.
if the physic step take longer that 1/60 then the fps is which ever of the two system takes longer.
you can see the asynchronous mode in the demos. try and see if you find it.

[arkdemon]

Hi. Thank you for answering .
Yes, I have profiled and I am sure it is Newton ( it is comprehensible because, correct me if I am wrong, updating the physics is of time complexity O(n^2)).

There was a peak of 408ms for a bit more than 2000 bodies (4 pyramids of n=35, taken from the simple stack demo ) and an average of 67.63 ms (to update) per frame. I used NewtonUpdateAsync with 1 thread, solver model = 1 and friction model = 1.

Here is what I do: I initialize the 4 pyramids and I shoot around 10 spheres at them with continuous collision mode = 1 and when those spheres hit something i set their continuous collision mode to 0. (This is why I think you might have been surprised why it was "laggy").

By the way, I do not know why but sometimes (quite rarely) the planks of the pyramids "bounce" on the floor which is static(mass = 0). Either, it bounces all the time for everything(~10%), either it does not bounce at all(~90%).

[Julio Jerez]

I know is Newton but what part of newton? there are few subsystems.

No the time complexity is O(N) for the solver and O(n lo(n)) for collision broaphase and O(N) for collision narrow phase.
however when you pile lot of objects the narrow phase get explained because out can have up to n * (n - 1) worse contact pairs
if you are making 35 box high pyramid, that will definitely be slow. because ether solve will work a lot, so will the collision
what kind of game does that?
your say you are making and RTS why some many large pryramid stacks?

[Julio Jerez]

I just run the simple Box stack demo, which have 706 boxes and I get 2.5 miliscecund per update you say you get 67.6 on average?

are you sure you are no running on debug? My system is and Intel ICore 7 2600k 3.4 ghz which is 4 years old now.
what system you have?

[arkdemon]

Hi. Well it is currently in early phase . Before, when I tested the library and created around 50 ships that attacked each other, it went laggy (lots of bullets were thrown). Here, I created some box pyramids as a test and I just wanted to see the maximum performance I could achieve.

I get 67.6 ms for 4 pyramids with bullets thrown at them and with one pyramid, I get from 0 to 31ms and an average of 25ms when a block "moves". I have an Intel i5 4570 3.20GHz. I think I may be misconfiguring Newton or not using it properly.

I use Qt Creator as IDE, with MinGW 4.8.2 on Windows 7 and Newton 3.12 static.
I could not compile the lib properly (with CMake or the Makefile) so I made changes to the makefile. I am not sure if the lib was on Release or Debug mode but I think it was on release.
Sometimes I also get some strange errors with the library. One of them is what seems to be that the lib is corrupted, the compiler sometimes tells me that there are some duplicate section errors but still lets the application run. Another problem is when I debug the app, it crashes on startup when having multiple threads (NewtonSetThreadsCount(world, n)) and it crashes systematically on exit at line 104 of file Newton.cpp when running NewtonDestroy(world). I tried updating to r1667 but it does not seem to compile , the major error seems to come from the impossibility of using sse.

[Julio Jerez]

Non of those errors should happens, all the libraries should work out of the box.


try getting the project form source control and Build the demo. they should just build and run.

[arkdemon]

Ok. I will do this.