MinGW › MinGW - User

CRT lib: potential memory leakage

Classic

List

Threaded

16 messages Options

Jannick

CRT lib: potential memory leakage

Hi,

it appears that the CRT lib produces some memory leakage: The small program compiled with mingw32-gcc (5.3.0)

#include <stdlib.h>

int main (int argc, char** argv)

{

free(argv[0]);

return 0;

}

runs through like a charm which is suggesting that the CRT lib’s terminating clean-up work misses the exe path (argv[0]) somehow.

I am wondering if this is intended or am I far off here?

Thanks,

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Eli Zaretskii

Re: CRT lib: potential memory leakage

> From: "Jannick" <[hidden email]>
> Date: Fri, 5 May 2017 14:33:02 +0200
>
> it appears that the CRT lib produces some memory leakage: The small program compiled with mingw32-gcc
> (5.3.0)
>
> #include <stdlib.h>
>
> int main (int argc, char** argv)
>
> {
>
> free(argv[0]);
>
> return 0;
>
> }
>
> runs through like a charm which is suggesting that the CRT lib’s terminating clean-up work misses the exe
> path (argv[0]) somehow.

First, what is the purpose of freeing memory just before the program
exits? That memory will be freed by the OS anyway.

And second, the argv array is not allocated by the MinGW runtime, it
is allocated by MSVCRT.DLL, the Windows CRT shared library. Since we
have no idea how it is allocated, calling 'free' on it in your program
is a bad idea, because you cannot be sure it was allocated by a call
to the same 'malloc' version that corresponds to the 'free' you want
to use.

IOW: don't do that!

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Jannick

Re: CRT lib: potential memory leakage

On Fri, 05 May 2017 16:15:36 +0300, Eli Zaretskii wrote:

> First, what is the purpose of freeing memory just before the program exits?
> That memory will be freed by the OS anyway.

I was wondering if this was a leakage, so I simply plugged the free line into
the sample program for demonstration purposes only. As you say it isn’t leakage
in the MinGW control area. Do you happen to know about any relatively easy
reading material about what the OS is doing at start up and at termination?

> And second, the argv array is not allocated by the MinGW runtime, it is
> allocated by MSVCRT.DLL, the Windows CRT shared library. Since we have
> no idea how it is allocated, calling 'free' on it in your program is a bad idea,
> because you cannot be sure it was allocated by a call to the same 'malloc'
> version that corresponds to the 'free' you want to use.
>
> IOW: don't do that!

OK - understood. Many thanks.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Keith Marshall-3

Re: CRT lib: potential memory leakage

In reply to this post by Jannick

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/05/17 13:33, Jannick wrote:
> it appears that the CRT lib produces some memory leakage:

- From a pedantically theoretical perspective, perhaps; from a practical
perspective, no, it does not.

> The small program compiled with mingw32-gcc (5.3.0)
>
> #include <stdlib.h>
>
> int main (int argc, char** argv)
> {
> free(argv[0]);

I'm sure you realize that this is a bug in _your_ program. You did not
allocate storage for argv[0], (nor indeed for any of argv); by calling
free() here, you introduce undefined behaviour.

> return 0;
> }
>
> runs through like a charm which is suggesting that the CRT lib's
> terminating clean-up work misses the exe path (argv[0]) somehow.

You are assuming that argv[0] is even stored in memory which originated
with a call to malloc(); that calling free(argv[0]) didn't immediately
crash your program suggests that you were lucky in that assumption.
Even so, why would a terminating process bother to release individual
heap fragments, when the entire heap allocation is about to be released
anyway? Even if it did devote unnecessary processing resource to this
task, why do you assume that you would necessarily be alerted to the
attempted free() of an already freed heap fragment? This is undefined
behaviour, so could have _any_ outcome, from being silently ignored,
through crashing the (terminating) process, to wiping your hard disk,
frying your motherboard, killing your cat, or blowing up your domicile.

> I am wondering if this is intended or am I far off here?

You are imagining ramifications when, for all practical purposes, none
arise.

- --
Regards,
Keith.

Public key available from keys.gnupg.net
Key fingerprint: C19E C018 1547 DE50 E1D4 8F53 C0AD 36C6 347E 5A3F
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)

iQIcBAEBAgAGBQJZDH65AAoJEMCtNsY0flo/owkP+gNpdEoUh/v1jM5QZkGqSKrH
xX6hLlC66UF9WXzIV812/cp5T/ulMsK4/m+szi/7OWbmN1e0cqDT3dxL/oldmRMg
m3T9Bol26g2pFkEAhCpM2mUCC9d6T1eHA+2bN7JrhqvCOdz/MPwE1/vUjYLIGuYY
To/WdpHsKN6AdmNHKE6XvDudBPB/3oQsS9FYNvTfLAHG8iPwByn+XyWqLBsMk2TF
hTuCzL/raOxfaemqUuOdPNs0PTc8ffQgS5dzMUUTwgRoZAqTIGqDi96ZwAun3oi7
zjUt4UPK2JbDRuaZfUGJCJmpOJRIyJek+1e2lshn1mYLlNVz74sNDXjQk2Yiw4K4
qAjcidKgtmDO3ygjtnAqLRTQE1UYlq+56M5rXKRpsJPoq9TXZJBWdYzNzrOq36GE
zi6eyqoITErj5d4hJ/XC2XV0YZB+kC+MBxgGqGJbm9j/OKQdt5Sn2X5a0S7oq+zO
8Ry/C3rup6hWnU6N4vW6/O2s9OpY2aP1PapE++tPkq0U/5O48iPrY1i7vq+LnfMT
4TYnQkc6gmkOCefj9HGg2ybsoMEOW04QexLykPy56VHJzoVeeyJM71u6jk6vE2EC
qhk9vdJnDkn0xTl0cSPA70JySq99JL5Ok+HvkxnlbmzTW27eZihc+oVMjKOMlHOI
q4fZjUykRAcSA4WCXU59
=ov9s
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Eli Zaretskii

Re: CRT lib: potential memory leakage

In reply to this post by Jannick

> From: "Jannick" <[hidden email]>
> Date: Fri, 5 May 2017 15:31:10 +0200
>
> Do you happen to know about any relatively easy reading material
> about what the OS is doing at start up and at termination?

This is specific to each OS. For Windows, I suggest "Windows
Internals", it has a chapter on process management, and another on
memory management.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Jannick

Re: CRT lib: potential memory leakage

On Fri, 05 May 2017 16:49:35 +0300, Eli Zaretskii wrote:

> > Do you happen to know about any relatively easy reading material about
> > what the OS is doing at start up and at termination?
>
> This is specific to each OS. For Windows, I suggest "Windows Internals",
it
> has a chapter on process management, and another on memory
> management.

Great - many thanks. Yes, Windows is my system here. I will try to digest
the chapters.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Emanuel Falkenauer

Re: CRT lib: potential memory leakage

In reply to this post by Eli Zaretskii

Eli,

On 05-May-17 15:15, Eli Zaretskii wrote:

>> From: "Jannick" <[hidden email]>
>> Date: Fri, 5 May 2017 14:33:02 +0200
>>
>> it appears that the CRT lib produces some memory leakage: The small program compiled with mingw32-gcc
>> (5.3.0)
>>
>> #include <stdlib.h>
>>
>> int main (int argc, char** argv)
>>
>> {
>>
>> free(argv[0]);
>>
>> return 0;
>>
>> }
>>
>> runs through like a charm which is suggesting that the CRT lib’s terminating clean-up work misses the exe
>> path (argv[0]) somehow.
> First, what is the purpose of freeing memory just before the program
> exits? That memory will be freed by the OS anyway.

Disagree. It is my sad experience that Windows in particular does NOT
free all memory allocated in an exiting binary - it probably should, but
does NOT. We learned it the hard way, getting gigs of RAM "reserved"
(but not freed), until we found a few arrays we didn't free just before
leaving, and the thing was accumulating over multiple runs of the same
binary.

A good part of our job is actually making sure that absolutely
everything we allocate is freed, down to the last byte. It is ESSENTIAL
(under Win at least). And we are running the very latest version of Win10.

> And second, the argv array is not allocated by the MinGW runtime, it
> is allocated by MSVCRT.DLL, the Windows CRT shared library. Since we
> have no idea how it is allocated, calling 'free' on it in your program
> is a bad idea, because you cannot be sure it was allocated by a call
> to the same 'malloc' version that corresponds to the 'free' you want
> to use.
>
> IOW: don't do that!

Agreed!

Best,

Emanuel

>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> MinGW-users mailing list
> [hidden email]
>
> This list observes the Etiquette found at
> http://www.mingw.org/Mailing_Lists.
> We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.
>
> _______________________________________________
> You may change your MinGW Account Options or unsubscribe at:
> https://lists.sourceforge.net/lists/listinfo/mingw-users
> Also: mailto:[hidden email]?subject=unsubscribe

Eli Zaretskii

Re: CRT lib: potential memory leakage

> From: Emanuel Falkenauer <[hidden email]>
> Date: Fri, 5 May 2017 17:20:03 +0200
>
> > First, what is the purpose of freeing memory just before the program
> > exits? That memory will be freed by the OS anyway.
>
> Disagree. It is my sad experience that Windows in particular does NOT
> free all memory allocated in an exiting binary - it probably should, but
> does NOT. We learned it the hard way, getting gigs of RAM "reserved"
> (but not freed), until we found a few arrays we didn't free just before
> leaving, and the thing was accumulating over multiple runs of the same
> binary.

AFAIK, "reserved" memory is freed once the program exits, because it
is reserved for the process. If you have evidence that this doesn't
happen, please show a program that can be used to demonstrate this.

I think what you saw is that memory freed by calling 'free' still
stays "reserved", and is not returned to the OS. This is normal, but
it only happens as long as the process still runs.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Emanuel Falkenauer

Re: CRT lib: potential memory leakage

On 05-May-17 18:12, Eli Zaretskii wrote:

>> From: Emanuel Falkenauer <[hidden email]>
>> Date: Fri, 5 May 2017 17:20:03 +0200
>>
>>> First, what is the purpose of freeing memory just before the program
>>> exits? That memory will be freed by the OS anyway.
>> Disagree. It is my sad experience that Windows in particular does NOT
>> free all memory allocated in an exiting binary - it probably should, but
>> does NOT. We learned it the hard way, getting gigs of RAM "reserved"
>> (but not freed), until we found a few arrays we didn't free just before
>> leaving, and the thing was accumulating over multiple runs of the same
>> binary.
> AFAIK, "reserved" memory is freed once the program exits, because it
> is reserved for the process. If you have evidence that this doesn't
> happen, please show a program that can be used to demonstrate this.
>
> I think what you saw is that memory freed by calling 'free' still
> stays "reserved", and is not returned to the OS. This is normal, but
> it only happens as long as the process still runs.

Somehow I expected that answer - which is precisely why I said "it
accumulates over multiple runs of the same binary". If it really worked
as you say (and probably should), then the same memory should be reused
in all the runs after the first, i.e. NOT accumulate into gigs of RAM.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Eli Zaretskii

Re: CRT lib: potential memory leakage

> Cc: [hidden email]
> From: Emanuel Falkenauer <[hidden email]>
> Date: Fri, 5 May 2017 19:13:42 +0200
>
> > AFAIK, "reserved" memory is freed once the program exits, because it
> > is reserved for the process. If you have evidence that this doesn't
> > happen, please show a program that can be used to demonstrate this.
> >
> > I think what you saw is that memory freed by calling 'free' still
> > stays "reserved", and is not returned to the OS. This is normal, but
> > it only happens as long as the process still runs.
>
> Somehow I expected that answer - which is precisely why I said "it
> accumulates over multiple runs of the same binary". If it really worked
> as you say (and probably should), then the same memory should be reused
> in all the runs after the first, i.e. NOT accumulate into gigs of RAM.

I don't understand what you mean by "accumulated over multiple runs of
the same binary". The OS has no way of knowing whether the same
binary will be run again once its process terminates, so it cannot
keep that process's memory "reserved" when the process no longer
exists. Moreover, modern Windows systems randomize the memory of a
binary, so each run of the same binary will have different addresses
allocated to it for the same code calling 'malloc'. In this
situation, accumulating memory over multiple runs makes even less
sense.

Once again, please show your evidence, in the form of running a
program and looking at the system memory with some tool, which would
show that some of the memory which was allocated by a process calling
'malloc' is still marked as used after it terminates. I'm quite sure
you draw erroneous conclusions from whatever observations you made,
but it's impossible to point out specific mistakes without knowing the
details.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Emanuel Falkenauer

Re: CRT lib: potential memory leakage

On 05-May-17 21:00, Eli Zaretskii wrote:
>> Somehow I expected that answer - which is precisely why I said "it
>> accumulates over multiple runs of the same binary". If it really worked
>> as you say (and probably should), then the same memory should be reused
>> in all the runs after the first, i.e. NOT accumulate into gigs of RAM.
> I don't understand what you mean by "accumulated over multiple runs of
> the same binary".

Simple: run (and exit!) it ten times and you end up with ten times of
its allocated RAM marked as unavailable.

> The OS has no way of knowing whether the same
> binary will be run again once its process terminates, so it cannot
> keep that process's memory "reserved" when the process no longer
> exists. Moreover, modern Windows systems randomize the memory of a
> binary, so each run of the same binary will have different addresses
> allocated to it for the same code calling 'malloc'. In this
> situation, accumulating memory over multiple runs makes even less
> sense.

I do agree with you - except it's simply NOT what we are experiencing.

I mentioned "the same binary" because there was in the past a "feature"
in Win that kept DLLs in the RAM even after the parent process exited,
supposedly for being able to activate them without the overhead of
loading again from the disc - Borland's DLLs were famous for that. But
I confess I don't know whether that's still a "feature" in the latest Win.

> Once again, please show your evidence, in the form of running a
> program and looking at the system memory with some tool, which would
> show that some of the memory which was allocated by a process calling
> 'malloc' is still marked as used after it terminates. I'm quite sure
> you draw erroneous conclusions from whatever observations you made,
> but it's impossible to point out specific mistakes without knowing the
> details.

I'll try to produce a convincing SIMPLE example (our stuff is not only
classified, but really heavy) - keep you posted.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Eli Zaretskii

Re: CRT lib: potential memory leakage

[Please don't make this a personal email.]

> From: Emanuel Falkenauer <[hidden email]>
> Date: Sun, 7 May 2017 02:02:28 +0200
>
> > How do you see that the relevant RAM is unavailable, please? What
> > tools do you use to show that? That's the missing piece in this
> > discussion. Without it, I cannot tell where is the misunderstanding
> > between us.
>
> When after just two days of uptime 10 of my 12 gigs of RAM are reported
> "in use" by the Task Manager, then the printer disconnects by itself,
> then the Internet disappears, and then running new programs becomes
> generally impossible... you know something is really amiss.

That's not something that happens to me, surely.

> > Even if you are right, and DLLs loaded by a program are kept in memory
> > when the program exits, that has nothing to do with the issue at hand,
> > which is memory allocated by the program via a call to 'malloc'.
> > Memory for DLLs is allocated by the system, not by the application,
> > and so cannot be freed by the application directly. The only thing
> > the application can do is explicitly call FreeLibrary to unload a
> > DLL. That's a different scenario from what started this thread.
>
> Well, you are onto something here: practically all of our stuff is in
> DLLs! So can you explain what you mean by "Memory for DLLs is allocated
> by the system, not by the application", i.e. what is the fundamental
> difference between calling malloc in an exe and a DLL? In both, the
> memory may subsequently be freed...

Simply what I wrote: when a DLL is loaded, the application doesn't
call 'malloc' to allocate the memory needed for the DLL. So it won't
help to call 'free'. If the DLL was loaded with LoadLibrary, you can
unload it with FreeLibrary, but if it was loaded automatically, you
cannot do even that, I think.

But before you decide that this is your problem, I suggest to make
sure that the DLLs your application loads remains in memory when the
application exits.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Emanuel Falkenauer

Re: CRT lib: potential memory leakage

>> When after just two days of uptime 10 of my 12 gigs of RAM are reported
>> "in use" by the Task Manager, then the printer disconnects by itself,
>> then the Internet disappears, and then running new programs becomes
>> generally impossible... you know something is really amiss.
> That's not something that happens to me, surely.
>
>> Well, you are onto something here: practically all of our stuff is in
>> DLLs! So can you explain what you mean by "Memory for DLLs is
>> allocated by the system, not by the application", i.e. what is the
>> fundamental difference between calling malloc in an exe and a DLL? In
>> both, the memory may subsequently be freed...
> Simply what I wrote: when a DLL is loaded, the application doesn't
> call 'malloc' to allocate the memory needed for the DLL. So it won't
> help to call 'free'. If the DLL was loaded with LoadLibrary, you can
> unload it with FreeLibrary,

You are stating the obvious (LoadLibrary and FreeLibrary), which is NOT
what's in stake here: I'm talking about mallocs and frees called by the
functions INSIDE the DLL. Namely, a few of those mallocs were missing
their corresponding frees and, as a result, even after the EXIT of the
DLL, that memory was clearly not returned to the system.
Once again then: is there a fundamental difference between a malloc
called by the parent process (say an .exe) and a malloc called inside a
DLL? I don't see what that difference could be.

> but if it was loaded automatically, you
> cannot do even that, I think.

Indeed, you can't.

> But before you decide that this is your problem, I suggest to make
> sure that the DLLs your application loads remains in memory when the
> application exits.

On the contrary: even when the DLL is properly FREED from memory
(FreeLibrary), somehow the memory leaks INSIDE the DLL stay marked as
"unavailable" for Win. And, as I said before, they do accumulate over
multiple loads of the DLL(*).

Ergo it seems NOT to be the case that Win reclaimes everything that was
allocated when the application (and its DLL) exits.

P.S. (*) Just to make absolutely sure you understand what I mean:
(1) start executable
(2) executable loads DLL
(3) DLL code allocates with malloc or new
(4) for some of the mallocs and news, the corresponding frees and
deletes are NOT called -> MEMORY LEAKS
(5) exit executable, with properly unloading the DLL (FreeLibrary)
(6) repeat from (1) again
... and the MEMORY LEAKS in (4) are not returned to Win for reuse,
accumulating and eventually consuming all the RAM there is and crashing
the machine.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Eli Zaretskii

Re: CRT lib: potential memory leakage

> Cc: [hidden email]
> From: Emanuel Falkenauer <[hidden email]>
> Date: Sun, 7 May 2017 17:55:24 +0200
>
> I'm talking about mallocs and frees called by the
> functions INSIDE the DLL. Namely, a few of those mallocs were missing
> their corresponding frees and, as a result, even after the EXIT of the
> DLL, that memory was clearly not returned to the system.

A DLL cannot EXIT, AFAIK, it can only be unloaded. When it is
unloaded, all of its memory is released.

I do agree that code that will run in a DLL should free memory it
allocates, because a DLL might not be unloaded when the program using
it exits. But that has nothing to do with the issue which started
this thread, which was about memory allocated by a _program_ and
freeing that memory right before the program exits.

IOW, programs and DLLs have different requirements in this regard, and
thus practice that is good for programs might not be good for DLLs
(and vice versa).

> Once again then: is there a fundamental difference between a malloc
> called by the parent process (say an .exe) and a malloc called inside a
> DLL?

Yes, there is, see above.

> > But before you decide that this is your problem, I suggest to make
> > sure that the DLLs your application loads remains in memory when the
> > application exits.
>
> On the contrary: even when the DLL is properly FREED from memory
> (FreeLibrary), somehow the memory leaks INSIDE the DLL stay marked as
> "unavailable" for Win. And, as I said before, they do accumulate over
> multiple loads of the DLL(*).

I think this can only happen if the DLL is not really unloaded from
memory. FreeLibrary doesn't unload a DLL, it only tells the system
that the calling program has no more use for the DLL.

But once again, this discussion was about programs, not about DLLs.
When a program exits, its memory is released.

> (1) start executable
> (2) executable loads DLL
> (3) DLL code allocates with malloc or new
> (4) for some of the mallocs and news, the corresponding frees and
> deletes are NOT called -> MEMORY LEAKS
> (5) exit executable, with properly unloading the DLL (FreeLibrary)
> (6) repeat from (1) again
> ... and the MEMORY LEAKS in (4) are not returned to Win for reuse,
> accumulating and eventually consuming all the RAM there is and crashing
> the machine.

This could happen, but is unrelated to the issue discussed in this
thread. (And I suspect your DLLs do something very special to
"deserve" the results you describe, like maybe lock some of the memory
they allocate.)

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Emanuel Falkenauer

Re: CRT lib: potential memory leakage

> A DLL cannot EXIT, AFAIK, it can only be unloaded.

Sure, that's what I meant (sorry for the sloppy term).

> When it is
> unloaded, all of its memory is released.

NOT sure, obviously (in particular: see below).

> I do agree that code that will run in a DLL should free memory it
> allocates, because a DLL might not be unloaded when the program using
> it exits.

According to
https://msdn.microsoft.com/en-us/library/windows/desktop/ms683152(v=vs.85).aspx
, the DLL is always "removed from the address space of the current
process" - the reference says nothing as to the possibility of not
unloading due to memory leaks.

> But that has nothing to do with the issue which started
> this thread, which was about memory allocated by a _program_ and
> freeing that memory right before the program exits.

Agreed - but the more general conclusion people might get from your
initial comment could well be "bah, don't bother with leaks in your
programs: everything that was allocated will eventually be freed on exit
anyway" - which is definitely not always the case according to our
experience.

> IOW, programs and DLLs have different requirements in this regard, and
> thus practice that is good for programs might not be good for DLLs
> (and vice versa).

IMHO it's ALWAYS a good practice to free everything you allocated. One
Big Classic we experienced a zillion times is to realize that a whole
program can be used as a sub-routine in a "bigger scheme"... e.g. as a
function inside a DLL. At that point, any leaks that were left in the
original program under the excuse of "the OS will clean up after you"
will clearly become pretty dangerous (and good luck finding it all in a
code written by somebody else ten years ago!).

>
>> Once again then: is there a fundamental difference between a malloc
>> called by the parent process (say an .exe) and a malloc called inside a
>> DLL?
> Yes, there is, see above.

Nope, there isn't - not the one you mention that is: unloading a DLL and
returning to the OS all memory it allocated during its use are two
wholly different issues.

Note the mention in that reference saying

"Before unloading a library module, the system enables the module to
detach from the process by calling the module's DllMain function, if it
has one, with the DLL_PROCESS_DETACH value. Doing so gives the library
module an opportunity to clean up resources allocated on behalf of the
current process.

... which strongly suggests that if you MISS the "opportunity to clean
up resources allocated on behalf of the current process", they might
well NOT be cleaned.

Also, if you have a look at what Application Verifier (in Win10 at
least) offers as "Basics", you will find "Leak", which is explained as
"Checks that when a dll is unloaded there are no outstanding resources
allocated by it". If leaving behind "outstanding resources allocated"
was really innocuous, there would IMHO be no reason to even check for
the condition.

>
>> On the contrary: even when the DLL is properly FREED from memory
>> (FreeLibrary), somehow the memory leaks INSIDE the DLL stay marked as
>> "unavailable" for Win. And, as I said before, they do accumulate over
>> multiple loads of the DLL(*).
> I think this can only happen if the DLL is not really unloaded from
> memory. FreeLibrary doesn't unload a DLL, it only tells the system
> that the calling program has no more use for the DLL.

Agreed... and it eventually deletes it from memory completely, when no
more parent process uses it.

> But once again, this discussion was about programs, not about DLLs.
> When a program exits, its memory is released.
>
>> (1) start executable
>> (2) executable loads DLL
>> (3) DLL code allocates with malloc or new
>> (4) for some of the mallocs and news, the corresponding frees and
>> deletes are NOT called -> MEMORY LEAKS
>> (5) exit executable, with properly unloading the DLL (FreeLibrary)
>> (6) repeat from (1) again
>> ... and the MEMORY LEAKS in (4) are not returned to Win for reuse,
>> accumulating and eventually consuming all the RAM there is and crashing
>> the machine.
> This could happen, but is unrelated to the issue discussed in this
> thread.

Please see above.

> (And I suspect your DLLs do something very special to
> "deserve" the results you describe, like maybe lock some of the memory
> they allocate.)

While it is true that we do lock SOME memory for the purposes of IPC,
NONE of the leaks we ended up plugging were locked. And besides, AFAIK
global (i.e. OS-wide) memory we lock for IPC is allocated in a
completely different segment from the process-local heap where malloc
allocates.

But anyway: as I said I'll try to construct a sufficiently SIMPLE
example to exhibit the disastrous consequences memory leaks can have. If
I succeed, I'll report back.

Best,

Emanuel

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe

Earnie Boyd

Re: CRT lib: potential memory leakage

On 5/7/2017 9:08 PM, Emanuel Falkenauer wrote:

>
>> A DLL cannot EXIT, AFAIK, it can only be unloaded.
>
> Sure, that's what I meant (sorry for the sloppy term).
>
>> When it is
>> unloaded, all of its memory is released.
>
> NOT sure, obviously (in particular: see below).
>

A DLL is in all rights a separate executable and controls it's own
resources. Multiple runtime versions can also exacerbate your
experience if the resources cross DLL boundaries. You have multiple
runtime versions when you have various DLL compiled with various
differing compilers.

>
> But anyway: as I said I'll try to construct a sufficiently SIMPLE
> example to exhibit the disastrous consequences memory leaks can have. If
> I succeed, I'll report back.
>

Please do, it will put us at a common reference.

--
Earnie

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same. Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe