7

(EDIT: I have been asked to explain why my question is not a duplicate of this question. I thought the reason would be obvious from reading the two questions but to summarise: I don't have a memory leak. I know exactly what is using my memory. I am wanting to know why Windows is choosing to compress my memory (and hence use CPU cycles and render my system slow and unresponsive) when my system appears to have sufficient memory to store the entire application footprint in memory uncompressed.)

I do a lot of scientific/engineering computing simulation with bespoke and memory hungry applications. Perhaps not memory hungry in a 8tb supercomputer sense, but memory hungry in the 'more than word/email/youtube and even chrome' sense. Historically my worksets have occupied anywhere from 4-30 gb of RAM.

Since the anniversary update I have experienced significant performance issues with my machine, and while investigating I noticed that windows appears to be fairly aggressively compressing memory. For example, when opening a particular model I experience this:

enter image description here

Just to summarise the pertinent points from that screenshot:

  • 15388 MB of memory in use
  • 6855 MB of memory compressed
  • If uncompressed, this would use 16657 MB
  • 16.6 GB of memory free
  • 32 GB of memory total

In short: If I didn't compress, by my understanding my memory requirements would be 15388-6855+16657 = 25190 MB. This is less than my available memory.

I have very often noticed that during times of reduced responsiveness my CPU is burning cycles doing memory compression like so: enter image description here

I know that the concept of memory usage is extremely nebulous because of features like shared pages, reserved memory, copy on write etc etc but to me this situation looks like I'm just wasting CPU cycles and system responsiveness unnecessarily. My desire would be to simply not perform any memory compression and instead use my available RAM.

My questions are:

  • Is there something I am misunderstanding that explains what Windows 10 is doing (because the current behaviour seems irrational)?
  • Has there been a change in the behaviour/policy of memory compression since the anniversary update? I ask because I have only started experiencing these performance issues since the update.
  • Is there a way to disable the memory compression? I have tried the technique described here without any luck.

Full disclosure: a version of this question was posted to the microsoft forum but only received canned replies, so I am reposting here.

Community
  • 1
fostandy
  • 1,079
  • Because the Windows 10 WMM uses compression, has nothing to do with out much system memory your system has. What problem do you believe you have exactly? Compression that WMM does cannot be disabled – Ramhound Oct 01 '16 at 02:00
  • 2
    My problem is as stated in the post. I have 32gb of memory. It is using 15388 MB of memory. Of that it is compressing 6855 MB of memory. It is using a very large amount of CPU cycles to do this compression, which is rendering my application slow and unresponsive. To me the more sensible behaviour would be to use all of my available memory (which would require 25190 MB) and not waste CPU cycles compressing memory unnecessarily. – fostandy Oct 04 '16 at 01:44
  • 1
    Also what do you mean by 'WMM uses compression, has nothing to do with out much system memory our system has'? For WMM to have a compression policy/behaviour that does not take into account system memory seems quite crazy. – fostandy Oct 04 '16 at 01:45
  • 1
    @magicandre1981 : I believe my problem is different. I know exactly what is taking up the memory, it is my application. What I am trying to undersatnd is why is Windows compressing memory (and using CPU cycles) when I have sufficient memory to store the entire application footprint uncompressed – fostandy Oct 04 '16 at 01:46
  • 1
    @fostandy - Windows 10 WMM will always compress the memory its managing, there is NO WAY to disable it, and it would happen NO MATTER HOW MUCH memory you had. Its a core feature of Windows 10 which cannot be disabled. "is using a very large amount of CPU cycles to do this compression" - NO It is not. "Since the anniversary update I have experienced significant performance issues with my machine" - Which is evidence that the WMM using compression is not the source of your problems. Windows 10 WMM has used compression since its original release build 10240 (and earlier) i.e. always – Ramhound Oct 04 '16 at 04:30
  • you have a leak, otherwise Windows won't compress it. Open a cmd.exe a admin, run this for 1 minute: wpr.exe -start CPU -start ReferenceSet -filemode && timeout -1 && wpr.exe -stop C:\HighCPUAndMemoryUsage.etl and press a key to stop logging after 1 minute, zip the ETL+PDB folder and share it – magicandre1981 Oct 04 '16 at 06:31
  • have you captured the trace? otherwise I can't see why your windows compresses the data – magicandre1981 Oct 10 '16 at 15:51
  • I gave it up to help you. disable the compression (http://superuser.com/a/1133340/174557) but this will hurt your performance because get a lot of Disk IO when Windows pages data out. – magicandre1981 Oct 15 '16 at 08:22
  • 1
    @magicandre1981 sorry for not replying. I missed this and when I saw it I wasn't sure if I can run it. What information does the WPR record? Is it metadata or will it record the actual value of the data? Unfortunately with the datasets I am under legal obligation/CA. Obviously the risk of disclosure is farcically far-fetched but I have to abide by statutory requirements.. – fostandy Nov 15 '16 at 21:05
  • it only includes statistic data no real file data. upload it to my dropbox request so that only me can see the file: https://www.dropbox.com/request/0H6gjbZxEbrs7LyulX88 – magicandre1981 Nov 16 '16 at 05:18
  • Are you sure the system could handle the entire memory footprint of your app, plus everything else that's running? Use Process Explorer to look at your process's Virtual Size counter. – Jamie Hanrahan Jun 28 '17 at 03:42
  • A few things: 1) the only memory Windows will ever compress is stuff that is being moved off of the modified page list. It may also be on the standby list in not-compressed form, but that stuff can be "repurposed" for other uses. But the point is that Windows is not yanking RAM away from your process just so it can compress it. It's compressing it because it's already been removed from your process - or was never assigned there in the first place. – Jamie Hanrahan Jun 28 '17 at 03:53
  • re CPU time... yes, the memory compression thread is using about 12% of your CPU, but the Idle process shows you have almost 50% of total CPU capacity free. So it's not impacting your app AT ALL.
    • – Jamie Hanrahan Jun 28 '17 at 04:01