OpenEMM Technical Support Forum

I have just starting using OpenEMM with a mailing list of 500,000 email addresses. I have followed all the forums posts and install guide tips I could find one tweaking performance on both openemm and mysql. The problem I am having is that almost 48 hours in to the first mailshot, the system has only queued 38,000 emails (only 1000 an hour). I know bandwidth isn't an issue as we have a 100Mbit leased line with no QoS or anything getting in the way. From the pickdist log, I can see a continuous "Queue is already filled up" message, and the occasional message that processes the next batch.

The problem I think is that there are alot of old addresses on the list, and connections are timing out and the messages are being deferred. Greylisting is also happening alot, and I guess this doesn't help things either. All these timeout messages and greylisted messages stay in the queue and clog it up.

Is there any way of tuning openemm to allow for a faster throughput (maybe increasing the queue size, I dont mind the messages sitting there until they reach expiration). Is there any way of reducing the life of the message from 5 days to 1 day? (Sendmail also seem to only process one message at a time?)

I have seen on another forum post about implementing 2 queues, one for fast transmission and a second for slower transmissions. Would this help and can this be implemented mid send?

I ultimately need to speed this mailshot up without interrupting it. Any assistance or guidance would be appreciated

Is there any way of reducing the life of the message from 5 days to 1 day?

The OpenEMM Install Guide explains how to do this.

I have seen on another forum post about implementing 2 queues

This certainly helps but you have to tweak the source code.

Thanks for the information. I looked in the guide and found the following, and have changed this to 60 minutes (hopefully enough to not falsely bounce hosts that are greylisting)

Is there any way for OpenEMM to send the mail through sendmail in the first instance, and if this timesout or gets deferred in some way, then hand the mail sending over to one of our dedicated mail relays? I realise this will cause a softbounce for each message eventually, but I can live with that.

Thanks,

Chris

I have sped up the mail transfer by tweaking the sendmail config to be less careful if its dropping messages.

I still have a major problem with performance.... doing a <b>tail -f /var/log/maillog</b> it looks as there is only 1 or 2 parallel threads actually sending email. Is there any way to increase this? I am hoping to get a throughput of 200 mails a second assuming all addresses are valid...... basically I need the system to send out so fast its burning as much of our 100Mbit leased line as possible ... without of course getting the IP blacklisted!

(I have a postfix mail relay server cluster in our ISP infrastructure that gets through over a million messages a day, and its a lower spec box than what i'm running OpenEMM on )

Thanks,

Chris

Have you handled the 1000-per hour limitation? If so, please share how!

There is no artificial limitation in OpenEMM. The default blocksize of 1,000 makes sense to optimize throughput, in our opinion. However, you are free to increase it (or the stepping value) to tweak the performance.