Memory Leak in WebServices

[emmulator]

I have been testing creating and sending Mailings using the webservices. One problem is that it takes 2 webservice calls for each subscriber I want to add (one to create and one to 'bind'). Then, while doing some load testing, I started getting OOM Exceptions, even though OpenEMM was running with xmx at 1.5 Gb. So I did a little profiling, and have noticed that after about 50k webservice calls (creating a mailing with 25k subscribers), the 'usedConnections' Hashtable managed by the WebServiceBase is holding on to 500Mb of RAM!

I imagine this Hashtable is maintained for some kind of webservice troubleshooting -- it appears to put a stacktrace in the Hashtable for each webservice call. But for some reason, the 'remove' is either not getting called or not working.

This probably wouldn't have shown up as a problem if there were an API that could be called to add more than one subscriber at a time. I was always dubious of using this double-ws call per subscriber, and now it looks like I'll be going around the API for this, as well as removing the usedConnections Hashtable itself, unless someone can demonstrate that it does something useful.

[sp]

We ran into the same issue. Is this recognized as a problem and if, is someone working on a fix?

[emmulator]

I don't know if it's been fixed in the official release, but I just removed the 'usedConnections' Hashtable and related code from the WebServiceBase, and it's been working fine ever since. We're still using version 5.5.1.

[sp]

Sadly it hasn't been fixed, we did the same with good results.

[joostvb]

I'm using 6.0.1 and the 'java' process is running out of memory after making many calls to the webservice. It looks like the same problem as described above.
I patched the WebServiceBase.java file (removed all usage of usedConnections) and recompiled. With the new .class file in place, the memory usage of the 'java' process still consumes more memory after each call, but it's less then before.

[maschoff]

Since so many of you experienced this problem we will look into this issue and post our findings here.

[mkalen]

I don't know if it's been fixed in the official release, but I just removed the 'usedConnections' Hashtable and related code from the WebServiceBase, and it's been working fine ever since. We're still using version 5.5.1.

It seems the Hashtable "usedConnections" is a remnant of some sort of local connection pooling or connection tracking. Since AgnUtils.retrieveDataSource is now used to get a proper datasource from the servlet context (using Apache DBCP for connection pooling), the "usedConnections" have no use as far as I can see. (No references from code other than locally in WebServiceBase.java.)

I just ran into the same issue with out of memory after a lot of SOAP calls to OpenEMM (5.5.1 on CentOS) and profiled the server with YourKit Java Profiler. The results show that memory is filling up with string instances that are the values of the "usedConnections" hashtable (string created by the try/catch on rows 79-86).

As others in this thread, I could work around the problem by commenting out all code related to "usedConnections". Line numbers 52, 70-87 and 94 in the 5.5.1 version.

[maschoff]

The hashtable seems to be a leftover from a big debugging session and can be removed safely. We will fix that in the next release.