httpd memory leak with 3.0.0.4.374.32

[bilboed]

Hi all,

I've recently started to use merlin-wrt. I upgraded to 3.0.0.4.374.32 and I'm getting some dns/dhcp/... hangs after a week or so (happened twice already).
I don't use any special/extra services (no usb drive attached, no sharing, no nfs, no vpn, ...) except for ssh login.

The last time it happened (an hour ago) I was still able to query the web interface, the system log showed some error regarding vfs running out of memory.
Logging in via ssh I saw that httpd was taking 120+ MB of VSZ, essentially exhausting all memory and making other services jammed.

After rebooting... it started initially with 3898 ... and after an hour it's already up to 4128 (seen with top, I'm assuming the units are kB)

Two questions:
* Is this a known issue ?
* Until a fix is available, is there a way to restart just httpd ?

Update: Actually, it's really easy to trigger. Just reload the index page in your browser, and watch how it's leaking 12kB every time you refresh.

Logging out/in has no effect either.

 

[ryzhov_al]

Two questions:
* Is this a known issue ?

Looks like no.

# pmap -d `pidof httpd` | grep mapped
mapped: 4072K    writeable/private: 692K    shared: 100K
# one refresh
# pmap -d `pidof httpd` | grep mapped
mapped: 4084K    writeable/private: 692K    shared: 112K
# another one
# pmap -d `pidof httpd` | grep mapped
mapped: 4096K    writeable/private: 692K    shared: 124K
# reload WebIU twice
# pmap -d `pidof httpd` | grep mapped
mapped: 4120K    writeable/private: 692K    shared: 148K

Confirmed%)

* Until a fix is available, is there a way to restart just httpd ?

# service restart_httpd

 

[RMerlin]

I can also reproduce it on the RT-AC56U - it's 12KB each time.

The issue also appears in Asus's FW.

Thanks for the report. I'll try to track it down.

 

[RMerlin]

BTW ryzhov_al, the Entware version of valgrind crashes as soon it tries to run any process. Tested with both httpd and fdisk.

 

[RMerlin]

Fixed: https://github.com/RMerl/asuswrt-merlin/commit/35d752cf5b3462d155ae61906ff3fb213069136d

 

[ryzhov_al]

BTW ryzhov_al, the Entware version of valgrind crashes as soon it tries to run any process. Tested with both httpd and fdisk.

Please, try older version from this mirror.

I suspect there is some kind of regression, because there was no problem with valgrind before.

 

[rearden]

Thanks for finding this. I have had to reboot my RT-N66u running 3.0.0.4.374.32 about every 24hrs as it appears to have lost httpd (I can't get to the web interface) and pptp (pptp login does not work, but openvpn does) after that time frame. Hopefully this will have fixed it.
rearden

 

[netware5]

I am with 32.beta3 and never experienced such memory leak.

 

[RMerlin]

That memory leak only occurred if loading a page that accessed a specific function of the httpd daemon. The main index page was one of these pages. It's not a leak-over-time, it only leaked 12 Kb every time you loaded one of these pages, so it would be unrelated to any other issue that could cause resource depletion over time.

 

[netware5]

I am with 32.beta3 and never experienced such memory leak.

I am sorry, seems to be I'm wrong. The memory leak of 12 KB confirmed. Service restart fixes the problem.

RMerlin, do you plan to fix it in next releases?

 

[RMerlin]

I am sorry, seems to be I'm wrong. The memory leak of 12 KB confirmed. Service restart fixes the problem.

RMerlin, do you plan to fix it in next releases?

The fix is already on Github, so it will be included in the next release.

 

[bilboed]

Fixed: https://github.com/RMerl/asuswrt-merlin/commit/35d752cf5b3462d155ae61906ff3fb213069136d

Smashing ! Thanks for that (and for your awesome work in general)