Very large execution random behaviour

Question

I have an executable which calls a function from DLL project.

DLL project function has one for loop which gets executed for 3300 times. At the start of loop, I have called below function for testing purpose:

double getUsedMemoryMB()
{
PROCESS_MEMORY_COUNTERS pmc;
GetProcessMemoryInfo(GetCurrentProcess(), &pmc, sizeof(pmc));
return static_cast<double>(pmc.WorkingSetSize) / static_cast<double>(1024 * 1024);
}

This function returns 170 for first 38 iterations in a loop. For 39th iteration, i got result as 171. For iteration 40 to 73, value is 172 and later at iteration 583, I could observe value as 186.

Question is what this function returns? Is it stack memory used by executable process? Why it is getting increased randomly?

I started logging this information as I have tried executing for loop two times and both times it got crashed at different iterations and hence I thought about memory issue.

How to solve this issue?

Unfortunately, I cannot share code due to privacy constraints, but code inside for loop calls a function which do some processing and generate a file on disk. First one minute of code execution generates around 40 files which gets decreased to 40 in a hour post code execution time reaches around 30 Hours. Any thought to solve this issue will be a big help.

Answer 1

Correct me if I am wrong, but I believe Working Set Size is a measure of RAM that is used. It does not include virtual memory that is swapped out to disk. It appears that you are seeing memory increasing sometimes because virtual memory is getting swapped back into physical RAM whenever possible.

Answer 2

Instead of pmc.WorkingSetSize, see if pmc.PageFileUsage is a better measure of total memory used.

Edit: Maybe pmc.PeakPageFileUsage would be another one to watch. It should stabilize in the long term. If there is a memory leak, then that metric might continually increase.

Answer 3

Thanks for sharing this details...

I was not aware and was just trying this out, but it seems my trial is incorrect.

Can I get some idea through some function to verify that there is memory leak across a different iterations in a for loop ?

Answer 4

And I was calling another exe before the for loop started..

Once I got the working set size for that process , I got very huge difference across iterations

For first iteration , it was around 575 and within 200 iterations , that went to 1770 means almost 250 times than initial....

Isn't this some thing weird from memory leak or this information can't guarantee that ?

Answer 5

Yeah I used peakpagefileusage and pagefileusage as well

All three shows different values but pattern is same

All three values have gone high with each iterations