OpenEMM 2013 on Windows Server 2012 R2

[Oniled]

Credit to Tim Brown, most of all!

Wanted to share this with you since I needed to get OpenEMM 2013 setup and there were no instructions for Windows Server 2012 R2. I modified the instructions from Tim Brown from his post on Windows Server 2008 R2 (https://forum.openemm.org/post6583.html ... 2008#p6583)

Enjoy! Just got this working 30 minutes ago

Step 1)

Preparation
==========

Ensure your server has a minimum of 2GB RAM and that out of that you are not running anything else.

Server Spec Used:
1 vmWare virtual machine
1 vCPU
4GB RAM
40GB Hard Drive
Server 2012 R2

Get rid of any other email handling software that can take port 25!!!!!!!! <- very important!

Ensure you have access to windows Environment Variables Control Panel -> System -> Advanced System Setting -> Advanced -> Environmental Variables

Ensure your firewall will accept inbound port 8080 requests - Control Panel -> Windows Firewall -> Advanced Setting -> Inbound Rules -> Web Server (HTP) -> set the ports to 80, 8080

Make sure that you have the default windows SMTP service installed and running

Install Roles & Features --> Web Server (IIS) --> make sure to check off
Common HTTP Features --> HTTP Redirection
Application Development --> CGI
Application Development --> ISAPI Extensions
Application Development --> ISAPI Filters

Step 2)

Additional Software
================


Install MySQL 5.6.17 via the installer, and do make sure you remember the su password.

http://dev.mysql.com/downloads/installer/5.6.html

Edit your my.ini in C:\ProgramData\MySQL\MySQL Server 5.6.

Ensure that your my.ini file has at least the following configuration, although the location of your MySQL installation will need changing to the correct directory:

Please note that I have added the following markers in to help make things clearer. Just copy and paste the content in between the markers and not the markers themselves.

=====Start of File Content=====
===========================


===========================
======END of File Content=====

=====Start of File Content=====
===========================

[mysqld]
port=3306
datadir="C:/ProgramData/MySQL/MySQL Server 5.6/data\"
default-character-set=latin1
default-storage-engine=INNODB
innodb_log_buffer_size=2048K
innodb_buffer_pool_size = 4096K
innodb_additional_mem_pool_size = 4096K
innodb_log_buffer_size=2048K
innodb_thread_concurrency=4
innodb_flush_log_at_trx_commit=1
innodb_log_file_size=10M
max_connections=150
max_user_connections=30
thread_cache_size=0
thread_stack = 8M
read_buffer_size = 1024K
read_rnd_buffer_size = 1024K
binlog_cache_size=32768
sort_buffer_size = 1024K
net_read_timeout=30
net_retry_count=10
net_write_timeout=30
thread_concurrency = 8
old_passwords=1
open_files_limit = 350
[client]
port=3306

===========================
======END of File Content=====

Install Python 2.7.6 x64 installer
https://www.python.org/ftp/python/2.7.6 ... .amd64.msi

Install Python 2.7 MySQL link
http://www.codegood.com/

Set the following security permissions for python.exe and pythonw.exe
Properties -> Compatibility -> Change Settings for all users -> Run this program as administrator
I know that this is a tad dangerous and you can change it to the iisuser once you have finished, but during setup set all your file and folder permissions to everyone -> full access. It will save you a lot of heart ache in the long run.

Install JDK 6 u45 - http://www.oracle.com/technetwork/java/ ... 19409.html
Set the following environmental variables:
JAVA_HOME = C:\Program Files\Java\jdk1.6.0_45\
PATH = C:\Program Files\Java\jdk1.6.0_45\bin

Install Tomcat 6 (Make sure the 64-bit zip not the installer) - http://download.nextag.com/apache/tomca ... ws-x64.zip

I would advise c:\tomcat as the directory as it will make your life easier later on
Set the following environmental variables:
CATALINA_HOME = C:\tomcat\
TOMCAT_HOME = C:\tomcat\
Again, I know that this is a tad dangerous and you can change it to the iisuser once you have finished, but during setup set all your file and folder permissions to everyone -> full access. It will save you a lot of heart ache in the long run.

maschoff has pointed out an important omission on our part, so thanks for that

In answer to your question, the isapi_redirect.dll is used to allow IIS to redirect request to Tomcat in a production environment. If you are using your server with a domain and have IIS handling the DNS work, IIS will not know how to pass over the port 8080 request to Tomcat. This part of the configuration is not necessary for localhost solutions, but it is for production servers. This is the hardest part of the install and will require a lot of time on the Tomcat website if it does not go smoothly.

Next you will need to create a folder called ISAPI so it looks like c:\tomcat\isapi\

Download and place "isapi_redirect.dll" in this folder. You want the 64 bit version for IIS
http://www.apache.org/dist/tomcat/tomca ... 64-iis.zip

You will also need a properties file called "isapi_redirect.properties"

The content need to be:

=====Start of File Content=====
===========================

extension_uri=/jakarta/isapi_redirect.dll

# You'll need to make sure the following are the correct dir's:
log_file=C:\tomcat\logs\isapi_redirect.log

#debug, info, warn, error or trace
log_level=info

# Paths to the relevant worker files
worker_file=C:\tomcat\conf\workers.properties
worker_mount_file=C:\tomcat\conf\uriworkermap.properties

===========================
======END of File Content=====

You will now need to go to C:\tomcat\conf\ and add the following 2 files:

workers.properties

=====Start of File Content=====
===========================

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Note that the distributed version of this file requires modification
# before it is usable.
#
# Reference documentation: http://tomcat.apache.org/connectors-doc ... rkers.html
#
# As a general note, the characters $( and ) are used internally to define
# macros. Do not use them in your own configuration!!!
#
# Whenever you see a set of lines such as:
# x=value
# y=$(x)\something
#
# the final value for y will be value\something

# Define two status worker:
# - jk-status for read-only use
# - jk-manager for read/write use
worker.list=jk-status
worker.jk-status.type=status
worker.jk-status.read_only=true

worker.list=jk-manager
worker.jk-manager.type=status

# We define a load balancer worker
# with name "balancer"
worker.list=balancer
worker.balancer.type=lb
# error_escalation_time: seconds, default = recover_time/2 (=30)
# Determines, how fast a detected error should switch from
# local error state to global error state
# Since: 1.2.28
worker.balancer.error_escalation_time=0

# - max_reply_timeouts: number, default=0
# If there are to many reply timeouts, a worker
# is put into the error state, i.e. it will become
# unavailable for all sessions residing on the respective
# Tomcat. The number of tolerated reply timeouts is
# configured with max_reply_timeouts. The number of
# timeouts occuring is divided by 2 once a minute and the
# resulting counter is compared against max_reply_timeouts.
# If you set max_reply_timeouts to N and the errors are
# occuring equally distributed over time, you will
# tolerate N/2 errors per minute. If they occur in a burst
# you will tolerate N errors.
# Since: 1.2.24
worker.balancer.max_reply_timeouts=10



# Now we add members to the load balancer
# First member is "node1", most
# attributes are inherited from the
# template "worker.template".
worker.balancer.balance_workers=node1
worker.node1.reference=worker.template
worker.node1.host=localhost
worker.node1.port=8109
# Activation allows to configure
# whether this node should actually be used
# A: active (use node fully)
# D: disabled (only use, if sticky session needs this node)
# S: stopped (do not use)
# Since: 1.2.19
worker.node1.activation=A

# Second member is "node2", most
# attributes are inherited from the
# template "worker.template".
worker.balancer.balance_workers=node2
worker.node2.reference=worker.template
worker.node2.host=localhost
worker.node2.port=8209
# Activation allows to configure
# whether this node should actually be used
# A: active (use node fully)
# D: disabled (only use, if sticky session needs this node)
# S: stopped (do not use)
# Since: 1.2.19
worker.node2.activation=A

# Finally we put the parameters
# which should apply to all our ajp13
# workers into the referenced template
# - Type is ajp13
worker.template.type=ajp13

# - socket_connect_timeout: milliseconds, default=0
# Since: 1.2.27
worker.template.socket_connect_timeout=5000

# - socket_keepalive: boolean, default=false
# Should we send TCP keepalive packets
# when connection is idle (socket option)?
worker.template.socket_keepalive=true

# - ping_mode: Character, default=none
# When should we use cping/cpong connection probing?
# C = directly after establishing a new connection
# P = directly before sending each request
# I = in regular intervals for idle connections
# using the watchdog thread
# A = all of the above
# Since: 1.2.27
worker.template.ping_mode=A

# - ping_timeout: milliseconds, default=10000
# Wait timeout for cpong after cping
# Can be overwritten for modes C and P
# Using connect_timeout and prepost_timeout.
# Since: 1.2.27
worker.template.ping_timeout=10000

# - connection_pool_minsize: number, default=connection_pool_size
# Lower pool size when shrinking pool due
# to idle connections
# We want all connections to be closed when
# idle for a long time in order to prevent
# firewall problems.
# Since: 1.2.16
worker.template.connection_pool_minsize=0

# - connection_pool_timeout: seconds, default=0
# Idle time, before a connection is eligible
# for being closed (pool shrinking).
# This should be the same value as connectionTimeout
# in the Tomcat AJP connector, but there it is
# milliseconds, here seconds.
worker.template.connection_pool_timeout=600

# - reply_timeout: milliseconds, default=0
# Any pause longer than this timeout during waiting
# for a part of the reply will abort handling the request
# in mod_jk. The request will proceed running in
# Tomcat, but the web server resources will be freed
# and an error is send to the client.
# For individual requests, the timeout can be overwritten
# by the Apache environment variable JK_REPLY_TIMEOUT.
# JK_REPLY_TIMEOUT since: 1.2.27
worker.template.reply_timeout=300000

# - recovery_options: number, default=0
# Bit mask to configure, if a request, which was send
# to a backend successfully, should be retried on another backend
# in case there's a problem with the response.
# Value "3" disables retries, whenever a part of the request was
# successfully send to the backend.
worker.template.recovery_options=3

===========================
======END of File Content=====

and uriworkermap.properties

=====Start of File Content=====
===========================

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# uriworkermap.properties - IIS
#
# This file provides sample mappings for example wlb
# worker defined in workermap.properties.minimal
# The general syntax for this file is:
# [URL]=[Worker name]

/admin/*=wlb
/manager/*=wlb
/jsp-examples/*=wlb
/servlets-examples/*=wlb
/examples/*=wlb

# Optionally filter out all .jpeg files inside that context
# For no mapping the url has to start with exclamation (!)

!/servlets-examples/*.jpeg=wlb

#
# Mount jkstatus to /jkmanager
# For production servers you will need to
# secure the access to the /jkmanager url
#
/jkmanager=jkstatus

===========================
======END of File Content=====

Reboot

In IIS manager you will need to add the following configuration:

[Computer Name] -> ISAPI and CGI restrictions -> Add..

Path = C:\tomcat\ISAPI\isapi_redirect.dll
Description = Tomcat
Allow extension path to execute = Tick

Next

[Computer Name] -> Sites -> Default Website/[Domain Name] -> ISAPI Filters -> Add..

Name = Tomcat
Filter = C:\tomcat\ISAPI\isapi_redirect.dll

Next

[Computer Name] -> Sites -> Default Website/[Domain Name] -> Add Virtual Directory

Name = Jakarta
Path = C:\tomcat\ISAPI\

Ensure that execute, read and write permissions are enabled on all filters and virtual directories you have just set up

Reboot

Start tomcat to test it is working.

Open your server browser and go to http:\\localhost:8080

You should see the tomcat "congratulations" page. If not then check the tomcat help files as there and a million and one reasons the blasted thing does not work.

Step 3
Open EMM
========

Extract to c:\openemm

Open the config file

c:\openemm\bin\openemm.py in anything other than notepad

If you have installed tomcat to c:\tomcat then change:

tomcathome = r'C:\tomcat'

else use the path to your install directory. please dont forget the small 'r' at the beginning and also make sure that directory structure is properly formatted.

Set the file and folder permissions to everyone -> full access

This is for testing

If your MySQL su username is not the default one then you can change it in the config file, there are four instances of it you will need to change.

The default is "root", you will need to change this to whatever your SU admin username is.

Since this is a x64 machine, you will need to change the registry key that the setup is looking for.

The default is:

'SOFTWARE\MySQL AB'

Under my install it was at:

'SOFTWARE\Wow6432Node\MySQL AB'

There are two instances of this.

Reboot

Do not run the tomcat startup.bat file. the OpenEMM will start it for you.

Run c:\openemm\setup.bat

Hopefully you will have a clean setup, more than likely not. The output is fairly usless, but if you see any errors with py calls in the, then that is execution permissions.

The logs can be located in c:\openemm\var\log\

You will need these.

Once you have fought your way through the setup reboot

Once you are back up then run:

c:\openemm\start.bat

If this works then tomcat will fire up another console window. That is the point when you know you are getting there.

Problems:

.py issues, check your execution permissions as described in the python setup
.py issues referencing port, your port 25 is in use by another application, you need to get rid of that application

Hanging once you have started. I.e. it does not say anything like "running" after "found database". This is an issue with openemm not being able to find Tomcat. Check you config file, check that the path is correct and that the 'r' is in there. From all the research done this is where most people fall down, we were no exception, having missed the ':' in the 'c:\' of the path, by getting muddled up in all the configuration files flying around.

Tomcat, this will be a configuration issue with either tomcat or your path in the openemm config file is not correct. check those.

Once you have it running, if you get errors on the overview page saying something on the lines of "General Failure, please check you input", then this is a MySQL issue. There is not enough memory being allocated to one of the parts of MySQL. To fix this we quadrupled all memory allocations until it worked, see the my.ini file in the setup part of this. A great tool for checking and modifying your MySQL install is Workbench, which can be found here: http://www.mysql.com/products/workbench/


-----

Once you have had the heart attack of getting it running on http:\\localhost:8080 you need to try and access it through your external browser. If you cant then check your firewall, this is more than likely the problem.

There are two locations for the log files:

c:\openemm\logs

and

c:\openemm\vars\logs\

During the installation its the vars\logs that are the ones that will give you the errors you need. Once you have it running its the \logs that will be your friend.

Once it is up and running it is a truly fantastic tool and all credit to the developers for creating such a fantastic piece of software.

Update:
========

Just a little tip for anyone who is testing and something goes very horribly wrong and it starts to send out an email you don't want and you cant find a way of canceling it:

Go to

C:\OpenEMM\var\spool\QUEUE

This is the generated emails list. If you have set it to build first then send, then just delete this list. If you have not then you will need to flush this list as it generates each time and pray like mad that you have caught it all. This was a 50,000 send mistake I made, although thankfully I found the queue before it hit 200,000!

Update 2:
========

It is very important to back up your openemm.py file once you have got it all working, please please please remember to do this. If something goes wrong, this is the only file, configuration wise, that is the pain to setup. For some reason best known to its self my installation decided that it no longer liked Dyn tags, so I went and deleted all the files and folders in the directory, after having backed up the MySQL I dumped those out as well. In restoring the system to its original self I found that this was the only file I needed.

--------------------------------------------------------------------------

Addendum from Oniled:

---
#1

I realized that this uses a version of OpenSSL that is vulnerable to the Heartbleed bug. To fix that vulnerability, you will need to download the latest Tomcat Native Connectors 1.1.30 for x64 from here

http://apache.org/dist/tomcat/tomcat-co ... 32-bin.zip

Extract the tcnative-1.dll from x64 and place it in C:\tomcat\bin\

---
#2

Updated help files can be downloaded from here:

http://sourceforge.net/projects/openemm ... 3/Updates/
---
#3

Files to use Java 7 can be downloaded here. Have not tried this as of 4/15/2014 - will update my post once I try this with Java 7

http://sourceforge.net/projects/openemm ... 3/Updates/

Java 6 has reached its end of life. If you want to run OpenEMM with Java 7 instead, replace library files aspectjrt-1.5.4.jar and aspectjweaver-1.5.4.jar in directories /home/openemm/webapps/openemm/WEB-INF/lib and /home/openemm/webapps/openemm-ws/WEB-INF/lib with aspectjrt-1.7.2.jar and aspectjweaver-1.7.2.jar and restart OpenEMM afterwards.
---

------------------------------------------------------------------------------------------------

[jPeterman]

I wanted to hopefully save someone else a couple of days of headaches but pointing out that it is really important to use the Python MySQL connector that is posted on codebase (the link above). Other valid MySQL connectors will cause the install to fail with a 'no database module found' error.

Good Luck!

[jPeterman]

Another problem I encountered after following the above instructions was an error from Python,

socket.error: [Errno 10013] an attempt was made to access a socket in a way forbidden by its access permissions.

This is caused when Python cannot connect to port 25. This could be a firewall issue or as it was in my case, because inetinfo (IIS) was already listening on that port. Step one of the above instructions say to:

"Make sure that you have the default windows SMTP service installed and running".

This will cause OpenEMM to fail upon startup as it prevents Python from binding to port 25.
I went to Server Manager > Local Server > Services Window and right clicked on SMTP Service to disable it OpenEMM started without problem.

[maschoff]

Thanks for sharing!

[jPeterman]

I am still trying to sort out some issues and it would be helpful if someone could provide a little info.

What actually sends and receives email in a windows environment?

From what I have read, it is either sendmail or the embedded SMTP server in windows.
In my install, Sendmail is not installed and the integrated SMTP server has to be disabled or Python throws an error.
When I look at what is listening on each port using TCPView, Python is listening on port 25.
I can send mail with no problem. I use a smart host that forwards any response messages (delayed bounces) to ext_2@news.mydomain.net.

The DNS is set up properly but the bounce filter doesn't process the responses properly.
Apparently, it doesn't respond to incoming SMTP requests so all messages are dropped.
I have run multiple tests after making sure that the firewall wasn't interfering and the result is always that there was no response from the server.

I have been trying to figure out how to configure Python to handle these responses with no luck. I could configure the Windows SMTP server without a problem but I am not sure how to tell openemm to use it instead of Python. In addition, I am afraid python wouldn't start if it couldn't bind to port 25.

Any help here would be greatly appreciated. I am trying to figure out a workaround for the bounce management issue when using a smart host and being able to process at least the delayed responses through openemm would be helpful.

[maschoff]

Yes, since Sendmail is not available for Windows, OpenEMM uses the integrated SMTP server and port 25.

[ud]

To your problem:

Apparently, it doesn't respond to incoming SMTP requests so all messages are dropped.

The process to handle incoming mail is semu.py which is the process using port 25. What do you mean exactly by "dropped"? Is the message accepted, but not processed? Or can't the message being delivered to the server? In the first case, have a look into the logfile(s) in c:\openemm\var\log, especially for semu and bavd. In the second case try to connect via telnet news.mydomain.net 25 to the server and check if it accepts the connection and what it replies. If the server replies you can submit these SMTP commands to check if the mail would be accepted at all (wait for an answer after each command):

Code: Select all

HELO me
MAIL FROM:<someone@news.mydomain.net>
RCPT TO:<ext_2@news.mydomain.net>

The answer to the last command is the relevant one to see why the program rejects a message.

-- ud

[jPeterman]

Thank you guys for the quick response. When I try to connect to SMTP I get a "could not establish connection to the host on port 25". It is as if nothing is listening.

I looked at the semu log and there are references to outgoing mail and what look like spool checks but nothing to indicate is listening to or managing incoming mail.

There is no bavd log (which could indicate a problem). There is a bav-update log but each one of those contains only a "going down" followed by a "starting up" reference less than a minute later which I assume would be to create a new log file.

I suspect there is a configuration issue with the integrated SMTP server but I have not been able to figure out how or where it is configured.

Again, thank you very much for your help.

[jPeterman]

I have narrowed the problem down a bit. I can connect to port 25 if I do so from localhost. So I know the SMTP server is running. However, it ignores connection attempts from any IP outside of localhost which means it is ignoring the bounce messages relayed from my smart host.

It would seem like there should be a setting that can be adjusted maybe in smtpd.py or semu.py that would tell it to accept connections from any ip. I can then use windows firewall to limit those connections to only the ip of my smart host.

I am getting close here...just need a little more direction as to how to adjust those settings.

[jPeterman]

I have finally figured out the culprit to port 25 incoming requests being dropped. It turns out it was a firewall configuration issue. When I created the firewall rule to allow communication on port 25, it either automatically associated the rule with the SMTP service or I associated it with the SMTP service. Once I cleared that selection and set it to allow all services to use the rule, it started accepting requests. WooHoo!!