CleanMem pushing info to page file

[Davey126]

I am a longtime CleanMem user (over a dozen pro licenses) but recently noted a behavior that is inconsistent with its stated purpose. On a Win 8.1 x64 system with 4GB memory executing CleanMem consistently pushes information to the page file often expanding it to many times its original size. The system itself is modestly loaded, primarily browser pages with a 2-4GB total commit level. The drive containing the page file is an SSD which Windows seems to understand. Superfetch is enabled which drives a fairly aggressive system cache.

CleanMem appears to be functioning correctly as I can observe pages being moved to the Standby list upon execution. However, the contents of the Standby list (or some portion of it) are written to the page file each time CleanMem runs. If I disable automatic cleaning the system stabilizes at 40-60% memory utilization with the balance being allocated to the Standby list. Task Manager reports little free memory with a substantial cache which is to be expected. There is some limited page file activity as occurs in all recent versions of Windows - regardless of the amount of available memory.

As the main drive is a SSD I do not experience a performance hit related to this behavior (read/writes are very fast) but it does generate GBs of additional page file writes over the course of a typical day. With automatic cleaning disabled 'Current Usage' of the page file typically remains under 1GB. I have rebooted several times during the most recent observation window (about a week).

I don't fault CleanMem as it is simply calling a Microsoft API to flush data to the Standby list. However, I wonder if the some of the memory management algorithms have changed in Windows 8.1 and/or the use of a SSD is influencing how aggressively information is written to the page file. Another possibility is the information being reported (eg: page file utilization) is incorrect. Note my system only supports SATA II. While the drive itself is very fast the interface limits overall performance which remains a fraction of memory speed. There is really no reason to aggressively page ... but the system does so on each execution of CleanMem.

I am monitoring other Win 8.1 systems as well as a handful of Win 7, Vista and (gasp!) XP rigs to see if these boxes are performing the same way. Many of those systems still have traditional HDDs so it is not an apple to apples comparison.

Posting more for FYI. Curious what you guys think.

[Shane]

I think it depends on the program that is being cleaned. Since Windows is doing the memory cleanup and cleanmem is simply having windows do the cleanup it is hard to tell. But I do know on my system that when I run cleanmem about 500 mb of memory gets free and I notice about 20 mb gets pushed to the page file for some reason, but not a lot.

Not matter what a little is always pushed to the page file by windows, but normally not much since the memory is suppose to be in the standby and not the page file.

One test i would is to try to find out which processes windows is moving to the page file. Do you know how much seems to be pushed to the page file?

But MS did change how Windows 8 handles memory and the page file. They now included a swapfile.sys
http://www.thewindowsclub.com/hiberfil-pagefile-swapfile-sys-windows

So even on my system where I have the page file on a 2nd drive and not on my ssd drive there is still a swapfile.sys on my main drive.

Form what I can tell on that page

Why do we need another virtual page file in Windows 8 ?

The Swapfile.sys in Windows 8 is a special type of pagefile used internally by the system to make certain types of paging operations more efficient. It is used to Suspend or Resume Metro or Modern Windows 8 apps.

Shane

[Davey126]

Thanks. I was aware of the different page/swap files in Windows 8.x. I have hibernation (and hence fast startup) disabled. My swapfile.sys is always 16MB (not sure why) while pagefile.sys varies depending on settings and system load. I never try to cap pagefile.sys but sometimes set a minimum size depending on the type of media where it resides.

Opera (or any blink based browser) appears to be the most consistent culprit. System commit levels rapidly rise as tabs are opened but rarely contract until the browser is fully closed. I can easily increase system commit levels by 4GB or more by simply opening a half dozen tabs. The first time CleanMem runs it behaves as expected: standby allocation jumps dramatically with a few dozen MB being written out to the page file. On subsequent invocations the 'in use' and 'standby' allocations update as expected but the utilization of the system pagefile often jumps by several GBs (as reported by CleanMem). This pattern continues until I close the browser. Closing tabs has little effect on commit levels. My total memory used rarely exceeds 60%; Windows seems to hold values between 40 and 55% during steady state operations.

While it's easy to point a finger at Opera 25 I can easily recreate on Chrome 38 and (to a lesser degree) IE 11. I don't run a lot of extensions and have tried 'privacy mode' which disables all extensions; no difference. I do not use Firefox or Safari.

As you said, CleanMem is not doing anything 'wrong', but its utility for me is diminished given recent observations. I will still use the mini-monitor and tray icon to track memory utilization and other interesting metrics and occasionally invoke a manual scrub. However, I have disabled the task that performs scheduled cleaning as it appears counterproductive on my rig.

[Shane]

Doesnt surprise me to much on the browsers. They have practically become min OS's themselves with html5 and such. And ALL of them suffer from very high memory usage, and the more tabs open the more memory it takes. I use firefox myself and it is the same way with high memory.

Keep in mind, you dont have to stop using cleanmem from the task scheduler, you can use the ignore list and add the browsers to it and have cleanmem not touch them, that way you can at least keep the rest of the processes cleaned up.

But no matter what I think there is always a small amount that gets moved to the page file because that is how windows does it, but the amount that goes to the page file is far less than normal since cleanmem is making windows stay on top of it more often.

I am curious how Windows 10 is going to be with memory as well. Every OS version MS seems to change how memory is handled by a large amount, normally for the better, but at the same time, systems are coming out with a ton more memory than ever before, hopefully MS wont make the mistake of making memory management worse because they think users will always have a ton installed.

Shane

[Davey126]

Thanks. At some point I may try the ignore list. Currently I am monitoring how windows behaves with Cleanmem disabled over an extended period. Yesterday memory utilization spiked over 70% for a while but there was no adverse performance impact despite the constant shuffling. Love those SSDs!

I also want to do some testing on machines where hibernation is enabled. It's my understanding the Standby list does NOT get written to the hibernation file so executing a clean just prior to hibernation (shrinking 'In Use' memory) should reduce both hibernation and recovery times.

The current generation of Browsers with pre-caching enabled appear to request huge blocks of memory for each open tab and rarely release it when the tab is closed. Opera also seems to monitor the percentage of 'in use' memory and moderate incremental requests when utilization is high. However, if I run Cleanmem it then seems to say "holly crap, there's more...I want it all...now" and up go the utilization figures. At some point there is not enough Standby memory to accommodate the humongous caches and lower priority stuff gets written out to the page file. Chrome behaves the same way. IE is a bit more considerate - but also more unpredictable w.r.t memory use.

Well, that's my laymens explanation with a little color thrown in

p.s. I don't think MS will mess with memory management in an adverse way given the continued drive toward merging Windows Phone and 'desktop/metro' into a common core. They get it on mobile where lower-end Windows phones run just fine with 512MB of memory. Andorid and iOS choke on such paltry amounts. We are also starting to see Windows based Chromebook clones that have to aggressively compete on price. Many are equipped with 2GB RAM which is satisfactory on Win 8.1 (given their purpose) and should be even better with Win 10. At least that's what the glossy says.

[Davey126]

Quick follow-up: I seem to be having some success using the advanced monitor to target specific modules instead of doing a blanket cleanup. This moderates the amount of info paged to the SSD while keeping a handle on memory hogs. A good compromise - but obviously only available to 'pro' users.

[Shane]

The free version of cleanmem though lets you do it as well, just just a pro feature. Open the CleanMem_Settings.exe file and you will see the settings you can put for cleanmem, including the lists

Shane

[Davey126]

Quick follow-up: I seem to be having some success using the advanced monitor to target specific modules instead of doing a blanket cleanup. This moderates the amount of info paged to the SSD while keeping a handle on memory hogs. A good compromise - but obviously only available to 'pro' users.

After a week of running with this configuration I can confirm unwanted 'dumps' to the system page file have all but stopped. Clearly some process, most likely related to Opera (Chrome/Chromium/Blink), went nuts when regularly 'cleaned' causing commit levels to soar and ultimately huge page outs. Now, the proper way to deal with this would be to identify the offending process(es) and exclude them from Cleanmem. Unfortunately, Opera spawns a separate process for every extension and tab making identification a time consuming trial/error process. The good news is each process is named opera.com. My solution is to check 'opera.com' every 5 min and clean any that >100 MB. This works exceptionally well without unwanted side effects. I have another trigger set to clean all processes if overall memory utilization >75%. So far I have never hit that mark even with aggressive browser use (a dozen+ tabs open with highly graphical content).

Thanks for making CleanMem such a useful and configurable tool.

[Davey126]

A bit more information for the curious. I have found Win 8.1 makes more intelligent use of system resources than Win 7/Vista/XP, at least on my modest rig, reducing the need for 'helper' apps. I now use Cleanmem primarily as a monitoring tool with very high intervention thresholds in case some process goes nuts. Active memory utilization typically hovers in the mid 50s with minimal page/swap file pushes. That would never happen on XP! I still manually 'clean' memory on occasion (something satisfying about seeing low numbers) but find regular/scheduled cleaning actually yields higher page file utilization over time which is counterproductive. While the performance hit with SSDs is minimal there is no need to generate extra writes without a corresponding benefit. In a previous post I noted Opera (or any Blink browser) as a potential contributor to excess paging but have since found it simply stabilizes at a higher memory use threshold if left alone. I don't fault Cleanmem but rather credit MS for improved memory management algorithms and they move towards a common Window core on traditional/mobile platforms. Cleanmem is still a great utility but probably one that is better suited to legacy operating systems.

[Shane]

That is what is interesting, CleanMem is simply triggering the Windows Memory cleanup, so it is still windows doing the job but it makes me wonder if MS has added new things to the memory management that is not used by their API.

Of course MS says the rule of thumb is it is up to the program itself to cleanup after itself, but a lot of programs don't do a good job of that. Firefox is still my fav to use yet their memory usage has always gone out of control and has always been a problem. I can have 10 tabs open and see memory setting at about 800 MB, close all the tabs except for 1 and the memory doesn't shrink. The tabs are not cleaning up after themselves.

It also matters how memory is marked by a program, this tells windows how to handle it when it goes to cleanup or manage it. So I bet the memory you see getting pushed to the page file is memory that has been marked as permanent by the process but hasn't been used in a while. Since Windows only moves memory it doesn't think will be needed for a long time to the page file.

Memory management has improved with each new version of Windows that is for sure.

Shane

[Davey126]

Firefox is still my fav to use yet their memory usage has always gone out of control and has always been a problem. I can have 10 tabs open and see memory setting at about 800 MB, close all the tabs except for 1 and the memory doesn't shrink. The tabs are not cleaning up after themselves.

Opera and Chrome behave the same way. A small amount of memory is released with each closed tab but the vast majority remains locked until the browser is closed. Recent versions of Chrome maintain persistent background process by default when the browser is closed; you can change that in advanced settings but most users wouldn't know to look.

Cleaning Opera (Chrome/Chromium) process seems to aggravate the problem. Commit levels quickly rise which ultimately drives page file writes. Leave those processes alone and commit levels remain stable or rise much more slowly. Something has changed in Win 8.1 as I do not see this behavior on Win 7. It could also be interaction between Chromium and Win 8.x that doesn't manifest on Win 7.

[Davey126]

Another quick update: The latest release of Opera utilizes Chromium 39 which seems to reduce the number of spontaneous pageouts. Per tab memory consumption is just as high (if not greater) than in earlier versions of Chromium but the way memory is 'marked' may be different as noted by Shane. I didn't test the Chrome browser itself but suspect it would behave similarly. While it's only been a few days it would seem Win 8.1 x64 is now behaving closer to Win 7 with respect to the issues discussed in this thread.

As for Cleanmem, I continue to run with selective cleaning only whacking browser processes (typically tabs) that exceed 250 MB. Typically one or two tabs get cleaned per hour with nothing going to the page file. Active memory utilization hovers in the 55-65% range with only a few hundred MBs in the page file. My system is as responsive as ever with dramatically reduced writes to the SSD (page file). That said, without Cleanmem a rouge tab can easily explode to 500 MB or more which drives all sorts of unwanted system behaviors. A good configuration for my needs.   

[Shane]

Good to see you are finding the right setup to have cleanmem help you the most. This is good information for anyone who needs it

Shane