Import Limitations for customer email-addresses

kdbock · Post by **kdbock** » Mon Dec 08, 2008 10:18 am

Hi,

we've installed openemm 5.5.1 on a new webserver based on open suse 10.3 with current modules installed.

Our customer, using openemm on that server bought 4 million e-mail addresses, which we tried to import into the openemm. We did it in 10,000 email addresses import steps. Unfortunately it seems we've faced a max. limit for customer entries of 176,000.

Is there a limitation in the tool or a known bug with a way to solve the issue?

Any comments are welcome!

Kind regards KD

emmulator · Post by **emmulator** » Fri May 15, 2009 9:25 pm

In the file:

/home/openemm/webapps/core/WEB-INF/classes/emm.properties

You'll find

import.maxrows=120000
recipient.maxRows=200000

The first one is actually mentioned in the installation guide. I got bit by the second one, which isn't mentioned there. Since we're using the webservices, not 'importing', I wasn't worried about the cap on imports, but then we hit the total rows cap and everything just started going into the nether...

I don't understand these arbitrary caps anyway, especially since the errors I was seeing when I hit the cap were not very informative.

Post by **maschoff** » Mon May 18, 2009 3:24 pm

Let me explain:

import.maxrows limits the number of addresses you can import in one batch. The time it takes to import addresses grows exponentially with the size of the batch until the server stalls.

recipient.maxRows limits the number of addresses the database can hold. If the resources of the server are limited it makes sense to limit the database size because otherwise it can eat up more and more CPU time.

However, we will increase the default values in the next release.

emmulator · Post by **emmulator** » Tue May 19, 2009 1:53 am

Well, if it really is *exponential*, then there's something wrong with the code.

I can see where you might want to have limits like this for your hosted version of the product, but for those of us running our own systems, it doesn't really make sense to me to impose an artificial cap.

My bigger concern though, was that, unlike import.maxRows, recipient.maxRows was never mentioned in the installation guide, so this one came as a surprise to us and caused us to lose several Mailings. Please make sure this is highlighted in the next version of that guide.

Overall, we've been very happy with OpenEMM so far!