Newton 2.0x Archemedia Open Beta

[Julio Jerez]

In windows 32 virtual menory is limited to 2gbytes for a simple virtual adress space, so run put of memory is quite possible.
That may be, it engine should not crash, but I never had a file so big.
The solution, i beleive, would be to stimate the memory size and partition the database so that intemediate memory allocation do not blow up the heap.

could you post sned me the test demo I can test that, and work a solution?
It sodul be simple to fix.

[rvangaal]

I sent you a private email with the test load which exhibits the problem.

[JernejL]

You should partition your collision large trees into smaller ones if you can - i did this and with serialization it works much more faster to build and load the serialized meshes (with optimization on), plus you can stream loading of these, to otimize runtime performance.

This gives me a idea.. Julio: is it possible to build a compound collision of trimeshes?

[Julio Jerez]

This gives me a idea.. Julio: is it possible to build a compound collision of trimeshes?

No, but you can build a scene collision, whis is evne better because you can add, remove and move collison tree, and all ohe kind of collisions.

[Julio Jerez]

Ok I test it, an dI added teh fix, but I still have one bug to fix.
I will check in the fix tonight.

the mesh is huge, it has 2.7 million triangle,
the temp allocation is around 1.5 gb when unpacking the vertices as they are read.
I added the Packing function. and it word.

The reason this happen is that I change the format of the dgMeshEffect to use double instead of floats,
I did that because float are do not have enought precision for making vorornid decomposition of 3d bodies.
Using extended floats slow down lot of other stuff so I use doulbe as intemediate storage, so that double the amount of memory used for in some of the utilities.

the fix should solve the problem, at least for that size mesh, but if you for example double the mesh in size it will blow up again.
are meshes that big or bigger normal?

[JernejL]

can i serialize a scene collision and store it as one complete world file?

[rvangaal]

Fortunately this is one of the bigger tracks. Also, there are too many polygons being defined as collidable, which would otherwise make the collision tree smaller. However, it is currently built with a lot of objects combined; the 3D modeler would have to cut some parts up to avoid some objects being collidable.
Good to hear you have it working! I don't think doubling the mesh is coming for a few years yet. But probably by then I'll have moved to 64-bits.

[Julio Jerez]

Well I try to fix the problem by packing the data, but still does not work.
this is what happens,
after the mesh is preprocessed tshi sare the stats:
2.7 million triangles
1.8 million normals
2.9 million vertices

the requiress 141 temporary storage
plus 187 megbit of storage just for the info

at this point is crashes if running forn teh debugger because ther total memory allocated by newton is 328 meg,
If I run form eh desktop, the is past that point a buidl teh collison but
still crash in the part


Sat Jun 25 08:51:04 (INFO): [racer/6148] --- application start ---
Sat Jun 25 08:51:04 (INFO): [racer/6148] Racer version: 0.8.34 (Jun 23 2011/10:54:14) - customer: Internet
Sat Jun 25 08:51:04 (INFO): [racer/6148] Physics engine: NEWTON v2.34
Sat Jun 25 08:51:04 (INFO): [racer/6148] Loading track 'rallytrack'
Sat Jun 25 08:51:07 (WARN): [racer/6148] No shader found for material 'concretebare' in 'data/tracks/rallytrack/concretebare_54_steiger.dof' (default 'concretebare_flat.tga')
Sat Jun 25 08:51:08 (WARN): [racer/6148] No shader found for material 'material_3986' in 'data/tracks/rallytrack/material3986_3_farm_house02.dof' (default 'farm_door.tga')
Sat Jun 25 08:51:08 (WARN): [racer/6148] No surfaces found that match pattern 'dirt_grass_2*'
Sat Jun 25 08:51:08 (WARN): [racer/6148] No surfaces found that match pattern 'road_asphalt_snow*'
Sat Jun 25 08:51:54 (INFO): [racer/6148] Crash detected - attempting to recover some data before displaying the crash dialog
Sat Jun 25 08:52:03 (FATAL): [racer/6148] Exception 0xC0000005, flags 0, Address 0x00B1FBBD
(this dialog text is stored in QLOG.txt)

so even if it build the mesh, it is still too big, for widow 32, I am guessing that the amount of textures and the Game memory allocation still manage to push it over teh 2gbyte memory limit on win32
I believe that mesh is just to big.
This is what you can do, although I do not think it will be any good for that size mesh since game memory is still too big, is to serialize and then load the preserialized data, avoiding the intermediate memory allocation.

Sync to SVN

[JernejL]

rvangaal: if you wish to use such meshes you can use them directly with use of NewtonCreateUserMeshCollision , it's like trimesh but your application handles the internal data and calls such as raycasting/getting localized triangles, etc..

That way you can use the huge mesh, althru this isn't a sane solution (splitting the level into smaller pieces) it will work.

[rvangaal]

I tried the SVN update here as well and also still had a crash somewhere in the process. Instead of attempting to fix this the software way I've instructed the 3D modeler to be more conservative with generating so much collidable mesh.
In the 64-bits days this might improve naturally. What's the status anyway on Newton in 64-bits apps? (not that my app it 64-bit ready yet).

[Julio Jerez]

on the problem, the thing is that Newton is using about 400 m of temporary memory to build the mesh to preprocess the mesh,
this happens after I change the format of teh dgMeshEffect to use double instead of float, however even it was using floats the saving would by about 340 to 550 meg, so I think I will run out of memory anyway.
The mesh per see is not so big that is cannot be build, the problem is that the game by the time it calls to build collision on that mesh , it has allocated more than 1 gbyte of memory already, because of texture and other things.
so even if the mesh was successfully build the memory budget is very tight already.

The one thing you could do is to build the collision off line, and serialize it, loading serialization does not uses temporary memory buffers when loading the assets.

you can make a command line utility the you can call to build the mesh and serialize it, this will work because the 2 gb is per applications in win32, so that way the engine will have all the space to build the mesh collision of even
tow or tree time larger than that, then you load the serialized output of the command line.

[lovatwest]

Ok, so being a relative newbee to this sort of software, I have some questions that may be obvious to most, so please bear with me. I have used the 2.33 demos to proof a concept of using the physics engine in a simulation environment. Essentially, Newton runs as an asynchronous task to a larger simulation. Now, I need to get more complex objects into the mix and I wonder what software created the ".DAT" files used in the Entity class. I'd rather not have to create a new (whole) interface to some 3D graphics engine at this time. Any advice at this stage would be good. There appear to be many good 3D graphics programs/engines out there, but I would like to keep it simple (or simpler) for the time being.
Thanks.

[KingSnail]

Hey Julio, I took a bit of break from programming because I have a lot of university work. How is it going?
Any progress on the proxy controller?

[PJani]

Hy, i was recently busy working on some other project. Any new news on contact multi sampling?

[PJani]

Julio you should keep changelog of all changes per fersion in txt file on svn, because its hard to track all the changes from forum.

PS: now i notice there in svn that i can see change logs...

isn't SVN check in sufficient?

It has being a while since I do not touch the engine, I am working on a new project called Newton Script system.