OOM with plenty of free RAM

[Demious]

I'm using CleanMem with Prepar3d v2.4. Prepar3d is a 32-bit program, capable of using max of 4.5GB RAM, including system used memory. However, since I started using CleanMem, I noticed several times already, that Prepar3d closes because of OOM, with even less then 3GB used totally.

What can I do to make sure that I dont get any OOMs before the actual max amount of RAM is used?

[Willy2]

- What do you mean by "OOM" ?
- Sometimes some programs do something "illegal", something what doesn't comply with the memory management usage rules as laid out by Microsoft and then it's no surprise that a program crashes. Cleanmem uses an API incorporated in Windows to reduce memory usage. So, if a program doesn't adhere to the memory usage protocols laid out by MS, then one shouldn't blame Cleanmem.

[Shane]

OOM?

32 bit doesn't allow programs to use more than 3 GB of memory (Normally 4 GB is the limit but Windows normally doesnt let it reach that much since it is using part of it). So I am not sure how their program can use up to 4.5 GB when 32 bit cant even handle that.

So while there is a PAE switch that is suppose to allow it to use more than 4 GB
http://en.wikipedia.org/wiki/RAM_limit

Limits on physical memory for 32-bit platforms also depend on the Physical Address Extension (PAE), which allows 32-bit systems to use more than 4 GB of physical memor

The operating system itself depends if PAE will work
http://msdn.microsoft.com/en-us/library/aa366778.aspx

You didnt tell me any of your system info, such as what version of Windows and all that, so I cant get more specific info.

But that info is if you are on a 32 bit version of Windows. 64 bit version of Windows handles memory differently and even though there is still a limit on 32 bit programs it can go above the 4 GB mark with a few registry tweaks.

Cleanmem simply triggers Windows to do the memory cleanup, so normally there isnt ever a problem with any programs. To be sure have cleanmem skip that program and see how it does. With it using so much memory it is possible it isnt properly marking the memory that is in use and Windows is clearing memory that it shouldnt.

Shane

[Demious]

Sorry about the missing information. I'm so used to having my system specs posted in my profile, that I often forget to specify when I'm on other forums, unless it has a direct connection to a specific part of the system. Specs are at the bottom.

OOM is short for Out Of Memory.

Prepar3D is actually Microsoft FSX, only further developed by another developer, but it's still using the same game engine, so it kind of hard to believe Microsoft doesnt take into account their own structures.
And it's not that I'm blaming CleanMem, I already posted on other forums, but noone knows CleanMem. So to get help, the CleanMem forums seemed like the logical place to go.
And knowing that some setting in an unexpected corner of the system can make or brake the combination of two parts in a system, I'm not putting blame on anything at all. I'm just hoping to find which setting to change to get this small bug eliminated.

I'm using a pretty standard system, running Windows7 64-bit and from experience I can tell that Prepar3d can use about 3.5GB RAM on top of what the system uses, which is about 1GB. Normally around 4.3/4.4GB total RAM use I get the beeps telling me that RAM is running low and Prepar3D will close with an OOM message. That's what I use CleanMem for, to keep RAM use under 4GB, because 32-bit can use a max of 4GB RAM and above 4GB its getting in the 'danger-zone' of causing OOMs.
Only since I've been using CleanMem, it mostly works fine, reaching 3.8/3.9GB of RAM max, but sometimes it happens that Prepar3D closes with an OOM message, with even less than 3GB used.
Problem with FSX/Prepar3D is that it doesnt always clean up memory that is used for textures that arent in use anymore, and should have been deleted already, driving up RAM use higher than it actually needs to be.

Mostly RAM-cleaners are big problem makers with flight simulators, causing stutters, CTDs(Crash To Desktop) and unstable behavior. But then I read that CleanMem works different from other RAM-cleaners, so I thought to give it a go. And so far, over the whole, it seems to work perfectly.
If I can find where those premature OOMs come from, I think a lot of people are going to be interested in CleanMem.

My system:

Asus Z87-k
Intell i7-4770k @ 4.3GHz
Corsair Vengeance 8GB @ 2133MHz
2x SSD, one for Windows and one for Prepar3D, 1x HDD for other software
Gigabyte GTX780 3GB
XFX 750W PSU
Saitek yoke/rudder/trim wheel/panels
CH throttle
VRInsight T&T

Software:

Windows7 64-bit
Prepar3D 2.4
ORBX FTX Global/Vector/OpenLC/Regions/Airports
Pilot's FSG2010
ASNext
REX4
SPAD
Plan-G

[Willy2]

- Open "Resource Monitor" and see how the "Standby", "Free" & "Modified" part of the memory "behave". Are they growing at a (very fast) rate ? Is Windows willing to release parts of the "Standby" memory (conflict with the pagefile/SSD ?)
- Did you disable the pagefile (because of the SSD) ? Perhaps you can enable the pagefile and direct it to the HDD. Perhaps that would solve the problem ? Windows tries to write a (small) part (e.g. after having used Cleanmem) of the reclaimed memory ("Modified") to the pagefile. Without a pagefile all that "Modified" memory remains in the memory itself and that can grow to a VERY high memory usage and could lead to a "OOM" fault.
- Yes, the developer certainly needs to improve the memory management of the program.

[Shane]

I wonder if the OOM error you are getting now isnt for the memory it is using but instead the virtual memory it is using. Which is usually the commit size reported in the task manager and has to do with the page file.

If the program is horrible at cleaning up its memory then I wouldnt be surprised if it is using a lot of the page file as well.

Shane

[Demious]

@Shane -I indeed noticed the paging files getting filled, which also made the OOM-beeps sound, but I could fix that easily enough by increasing the paging file, from 3GB to 8GB.

@Willy2 -I use a paging files of 8GB on the C: drive (SSD for Windows). I've tested if there is any difference between the C: drive and the HDD-drive, but this doesnt seem to make any difference.

I've been keeping an eye on Resource Monitor and that was quite a revelation.
I normally use Performance tab in Task Manager and with CleanMem the Mini Monitor (which show the same value, In Use RAM and Modified RAM), and those look very nice, especially with CleanMem.
'In Use' and 'Modified' RAM, both go up and down with the loading/unloading, very smoothly and neither one of them is doing anything extreme.
On the other hand, what the 'Standby' RAM does was something I didnt expect. Slowly the Standby RAM keeps rising, until there's no free RAM left and at that point the OOM-beeps start sounding and once there's no free RAM left for longer than a couple of seconds, the program crashes with an OOM message.

I havent checked this before, so at this point I can only start guessing... Since the way it happens, which is different then before (normally it's the In Use Ram that fills up and crashes the program), I do have to ask. Can this constant rising of the Standby RAM be an effect that is caused by CleamMem, or can I now finally see this effect, because the In Use RAM doesnt crash the program anymore?
And maybe you guys know, is there something I can do to limit the use of Standby RAM? (If not, it would be fixed easily enough, by adding some extra RAM...)

[Shane]

Standby ram is ok and this is what we want. Ram is so much faster than the page file because the page file is is on the hard drive where standby is in memory itself. When Windows cleanups memory of a process it all gets moved to standby, the process takes what it needs back and what ever is left stays in standby for a while before windows says ok you can go. This is why there is never a hicup when cleanmem is used, it is all happening in memory.

Windows will move a little of the memory to the page file. it moves memory to a page file it thinks wont be needed for a long time.

So the OOM error you are still getting is most likly related to the page file, just increase the size of it and see how it does. Windows does limit the amount of virtual memory a process can have but if I remember right there is a registry key to tell Windows to increase it.

Shane

[Willy2]

- Windows keeps growing/filling the "Standby" memory (that's OK). But when all the "Free" memory is down to zero then Windows should move data from "Standby" to the pagefile. Did you allow Windows to manage the size of the pagefile instead of a fixed size of the pagefile ?
- I have seen this error before. Somehow a combination of programs (a registry setting issue ??) causes Windows to not release/cleanup parts of the memory, move data to the pagefile. It also means the program itself should improve it memory management. There's a program that can release/collapse the "Standby" memory as well. Install it and add it to Task Scheduler and you're done. Although this is only a "workaround". There should be better permanent solution.
http://www.mediafire.com/download/epo2qh9s1ic4q6f/EmptyStandbyList.zip
- I still would redirect as much as files/(temporary) folders away from the SSD & towards the HDD.
http://blog.superuser.com/2011/05/10/maximizing-the-lifetime-of-your-ssd/

[Demious]

Thanks for all the info, guys!

I also found something about memory leaks, and with what you've told me I'm going to do some experimenting and see what I can do. I'll get back to you with the results.

[Demious]

This was a lot simpler than I thought.

From when I used FSX, I always learned to limit virtual memory and I still did that for Prepar3D. Mentioning that Standby memory is moved to the pagefile made me think the easiest thing to try was to let Windows manage the virtual memory, although I didnt have much hope it would be this simple (I'm always out of luck when it comes to simple solutions solving my computer issues )
I've been flying all day and tried to have the memory work as hard as I could, I had 0-15MB of Free RAM constantly, but still everything has been running great all day! :D
I checked over at the Prepar3D forums, to see if it still needs the limited virtual memory and it doesnt, so that's perfect. Problem solved!

Thanks a whole lot for your input, guys!

[Willy2]

- Frankly, it was an "Educated guess". I wasn't sure whether or not Windows would move data from "Standby" to the pagefile but it seems Windows certainly does. But Windows tries to reduce "Modified" and that DOES involve writing some data to the pagefile.
- "Standby" memory is the filecache in the memory.

Glad you solved your problem !!!!!