Multithreading random memory overwrites?

[pHySiQuE]

I have been struggling with some strange memory problems. Malloc has failed frequently, and I get frequent GL_OUT_OF_MEMORY errors. (This error can indicate system memory, not just VRAM.) I also experienced errors sometimes in the Newton memory allocation function. These occur when the application is nowhere near the 3GB limit for 32-bit apps. I've been quite strict about locking critical sections.

Previously, I was setting the number of Newton threads to the max(16,NewtonGetMaxThreadCount()). When I set the thread count to 1, the problem goes away. This isn't conclusive, but I am reporting my experience here.

It should also be noted the application in question never calls NewtonUpdate(), but does create bodies, collisions, etc.

[Julio Jerez]

you are running out of memory. the memory address space in windows 7 is about 2+ gibytes
some version of windows scan go up to 3+ gbyte but after that malloc return NULL
http://msdn.microsoft.com/en-us/library ... 12(v=vs.85).aspx

your only solution is 64 bit.

[pHySiQuE]

No, this would occur with much lower memory usage, like less than 500 mb. I thought it was a problem of not enough contiguous memory, but it was occurring at pretty low mem usage, even after a restart. So I hypothesize the memory address space was being corrupted by a buffer overflow or something so that malloc could not work properly.

Although the editor never calls NewtonUpdate(), setting the NewtonWorld thread count to 1 fixed the problem.

Again, I am not sure Newton is the cause here, but I am just reporting this to help.

[Julio Jerez]

you say you do not call newtonUpdate, that narrow it down.
it is reproducible?

[pHySiQuE]

Unfortunately, the behavior is very inconsistent. I just know my editor has been getting frequent malloc() fails and GL_OUT_OF_MEMORY errors under loads that really should not stress it. It is completely possible there is an error in my code causing this, and disabling multithreading just moved the bug somewhere else, but right now this is my best guess. The problem has not occurred when Newton multithreading was disabled.

[manny]

I would recommend you to link against the debug CRT and use the microsoft tools to debug your applications. They provide awesome tools to debug all kinds of memory related issues, from heap corruptions to memory leaks.
I think one of those great tools is called "Application Verifier" while it sounds simple, it's really awesome, but better than any open source alternative like "valgrind".

[pHySiQuE]

THanks, I should probably do this anyways.

[Julio Jerez]

you say that you are no calling newton update, because this is in the tool.
I assume you are no using Newton thread pool system either.

if this the case them it do no see how newton can cause a memory corruption like that one you are describing. because ether thread people all threads are suspended, and the main engine thread is also suspended.

The only activity is when you call the collision function, from your main thread. therefore If there is a memory corruption it must be trazable form your main thread.

It is only when you make call form with it the NewtonUpdate that the code does internal memory allocation and most allocation are no even allocation they come from a free list based memory pool.

Like Manny say my suggestion is to use some memory tracking Toll.

Incidentally that mamager is no thread safe, at some point I will \make thread safe to see if I can remove some of the manual lock that I have scattered over in the engine, but that si a different issue.

[pHySiQuE]

Not sure if it's related, but I do use frequent raycasts in the editor.

[Julio Jerez]

yes but that function does no allocate memory.