OpenEMM Technical Support Forum

Hello,

I have just upgraded my CentOS VBox image from OpenEMM 19 to 20.04. I have also compiled and installed python 3.8.1 into /home/openemm/opt/python3. When I execute OpenEMMInstaller.sh Python 3 is found. Unfortunately the following error is thrown:

Mandatory python mysql module is missing!

The suggested solution is to install MySQL-python. It would appear, though, that MySQL-python does not work with python3. I also receive the "No module named 'ConfigParser'" error mentioned here:

https://stackoverflow.com/questions/537 ... l-4nev4id4

I have successfully installed the following:

configparser-5.0.0-py3.8
mysql_connector_python-8.0.20
PyMySQL-0.9.3

Would it be possible update the OpenEMM code to enable one to connect to MySQL from Python 3?

Best, Richard

Have you read this thread?

https://forum.openemm.org/viewtopic.php ... ing#p15990

Thank you. Once I had executed "python3 -m pip install mysqlclient" then I could start up with "./bin/OpenEMMInstaller.sh".

Unfortunately, I still can't seem to get the backend to start:

================================
= OpenEMM Installer v20.04.001 =
================================
Host: openemm
License: OpenEMM (ID: 0)
Runtime Version: 20.04.000.026
Manual Version: Unknown
OpenEMM Version: 20.04.000.126
System-Url: http://192.168.2.113
OpenEMMBackend Version: 20.04.000.126

Current menu: Show OpenEMM status

OpenEMM status:
Command '/home/openemm/bin/backend.sh status' returned non-zero exit status 1.
Operating System (OS): CentOS 7
Python version: 3.8.1 (default, May 12 2020, 14:19:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Postfix Version: 2.10.1
DB client version: /bin/mysql Ver 15.1 Distrib 10.2.22-MariaDB, for Linux (x86_64) using EditLine wrapper
Database Connection: OK
Database version: MariaDB 10.2.22-MariaDB
OpenEMM database structure exists (Version 20.01.460)
Currently running database threads ratio (Current: 1, Maximum: 5): 0.2
Database table emm_db_errorlog_tbl: OK
Jobqueue status: OK
No DKIM keys
Java version: 1.8.0_252 (OpenJDK)
Java JCE Unlimited-Key-Strength: OK
Tomcat version: 9.0.33.0
Wkhtml version: wkhtmltopdf 0.12.5 (with patched qt)
OpenEMMApplication is running (Tomcat ProcessID: 6266, started at: 21:15 1:32)
OpenEMMBackend is NOT installed or NOT running


It I try to execute /home/openemm/bin/backend.sh manually this is what happens:

[openemm@openemm ~]$ ./bin/backend.sh
Traceback (most recent call last):
File "/home/openemm/scripts/service3.py", line 22, in <module>
from agn3.config import Config
File "/home/openemm/release/backend/V20.04.000.126/scripts/agn3/config.py", line 22, in <module>
from .dblite import DBLite
File "/home/openemm/release/backend/V20.04.000.126/scripts/agn3/dblite.py", line 17, in <module>
from ._db.sqlite import Layout, SQLite3
ImportError: cannot import name 'Layout' from 'agn3._db.sqlite' (/home/openemm/release/backend/V20.04.000.126/scripts/agn3/_db/sqlite.py)


Do you have any suggestions?


Best, Richard

Could it be that package sqlite-devel was not installed before you compiled Python 3? If so, please install the package and compile Python 3 again.

Thank you again, once I installed sqlite-devel I was able to execute ./bin/backend.sh without throwing any error messages.

Unfortunately, though, the Backend still fails to start:

================================
= OpenEMM Installer v20.04.001 =
================================
Host: openemm
License: OpenEMM (ID: 0)
Runtime Version: 20.04.000.026
Manual Version: Unknown
OpenEMM Version: 20.04.000.126
System-Url: http://192.168.3.226
OpenEMMBackend Version: 20.04.000.126

Current menu: Show OpenEMM status

OpenEMM status:
Command '/home/openemm/bin/backend.sh status' returned non-zero exit status 1.
Operating System (OS): CentOS 7
Python version: 3.8.1 (default, May 12 2020, 22:47:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Postfix Version: 2.10.1
DB client version: /bin/mysql Ver 15.1 Distrib 10.2.22-MariaDB, for Linux (x86_64) using EditLine wrapper
Database Connection: OK
Database version: MariaDB 10.2.22-MariaDB
OpenEMM database structure exists (Version 20.01.460)
Currently running database threads ratio (Current: 1, Maximum: 7): 0.143
Database table emm_db_errorlog_tbl: OK
Jobqueue status: OK
No DKIM keys
Java version: 1.8.0_252 (OpenJDK)
Java JCE Unlimited-Key-Strength: OK
Tomcat version: 9.0.33.0
Wkhtml version: wkhtmltopdf 0.12.5 (with patched qt)
OpenEMMApplication is running (Tomcat ProcessID: 6312, started at: 11:48 2:14)
OpenEMMBackend is NOT installed or NOT running

>


And I'm also not able to kick off the Backend mannualy:

[openemm@openemm ~]$ ./bin/backend.sh status
[openemm@openemm ~]$ ./bin/backend.sh start
[openemm@openemm ~]$ ./bin/backend.sh status
[openemm@openemm ~]$


Something I have noticed, though, is the difference between the
current Backend JAVA directory and all previous versions (though not
sure if this is relevant):

/home/openemm/release/backend/V19.10.000.120/JAVA:
total used in directory 16 available 197262900
drwxr-xr-x. 2 openemm openemm 8192 Nov 24 09:47 .
drwxr-xr-x. 6 openemm openemm 55 Nov 19 19:44 ..
lrwxrwxrwx. 1 openemm openemm 60 Nov 24 09:47 ajaxanywhere-1.2.1.jar -> /home/openemm/webapps/emm/WEB-INF/lib/ajaxanywhere-1.2.1.jar
lrwxrwxrwx. 1 openemm openemm 63 Nov 24 09:47 ajaxtags-1.3-beta-rc7.jar -> /home/openemm/webapps/emm/WEB-INF/lib/ajaxtags-1.3-beta-rc7.jar
.
.
.

/home/openemm/release/backend/V20.04.000.126/JAVA:
total used in directory 4 available 197262900
drwxr-xr-x. 2 openemm openemm 25 May 12 23:29 .
drwxr-xr-x. 6 openemm openemm 55 May 7 18:48 ..
-rw-r--r--. 1 openemm openemm 3009 May 7 18:46 Mailout.ini


Thoughts?


Best, Richard

Could you post content of /home/openemm/var/log/ when you restart OpenEMM?

When I restart OpenEMM the following is written to the terminal:

Checking backend directories
Service sanity is marked as inactive

And nothing is written to:

/home/openemm/log/backend.log


Best, Richard

No other file in /home/openemm/log/ ?

After deleting the old /home/openemm/log/backend.log I restarted OpenEMM.

This is what the directory now looks like. All of subdirectories are also empty.

/home/openemm/log:
total used in directory 4 available 197277700
drwxr-xr-x. 7 openemm openemm 71 May 15 06:40 .
drwx--x--x. 22 openemm openemm 4096 May 12 23:21 ..
drwxr-xr-x. 2 openemm openemm 6 May 2 16:09 account
drwxr-xr-x. 2 openemm openemm 6 May 2 16:11 bounce
drwxr-xr-x. 2 openemm openemm 6 Aug 19 2019 data
drwxr-xr-x. 2 openemm openemm 6 Aug 19 2019 done
drwxr-xr-x. 2 openemm openemm 6 Aug 19 2019 fail


Best, Richard

An update:

I have just upgraded to the latest frontend &c. After switching, or downgrading,
the OpenEMM backend version from V20.04.000.126 ---> V19.10.000.128 the
backend appears to be working again (at least for test and admin mails).

Here is the status:

================================
= OpenEMM Installer v20.04.001 =
================================
Host: openemm
License: OpenEMM (ID: 0)
Runtime Version: 20.04.000.029
Manual Version: 19.07.004
OpenEMM Version: 20.04.000.132
System-Url: http://192.168.3.226
OpenEMMBackend Version: 19.10.000.128

Current menu: Show OpenEMM status

OpenEMM status:
Operating System (OS): CentOS 7
Python version: 3.8.1 (default, May 12 2020, 22:47:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Postfix Version: 2.10.1
DB client version: /bin/mysql Ver 15.1 Distrib 10.2.22-MariaDB, for Linux (x86_64) using EditLine wrapper
Database Connection: OK
Database version: MariaDB 10.2.22-MariaDB
OpenEMM database structure exists (Version 20.01.460)
Currently running database threads ratio (Current: 1, Maximum: 5): 0.2
Database table emm_db_errorlog_tbl: OK
Jobqueue status: OK
No DKIM keys
Java version: 1.8.0_252 (Oracle)
Tomcat version: 9.0.33.0
Tomcat-Native version: Undefined
Wkhtml version: wkhtmltopdf 0.12.5 (with patched qt)
OpenEMMApplication is running (Tomcat ProcessID: 11179, started at: 19:20 1:25)
OpenEMMBackend ok: once
OpenEMMBackend running: mailout, update, trigger, generate, mta, pickdist, slrtscn, direct-path, bav-update, bavd, bav, mlcontrol



Best, Richard

Greetings,

I also have to use the OpenEMM 19.10 backend in order for it function. With the 20.04 backend I can't send any Emails or do anything.
When i use "/home/openemm/bin/backend.sh status" there is no output or any error written into the logs.
I also got this error when starting OpenEMM "Service sanity is marked as inactive.".

Kind regards,
Mika Laurer

Hello Mika,

I ended up getting everything to work, but only once I had upgraded to the latest (stable?) releases:

================================
= OpenEMM Installer v20.04.003 =
================================
Host: openemm
License: OpenEMM (ID: 0)
Runtime Version: 20.04.000.043
Manual Version: 19.07.004
OpenEMM Version: 20.04.000.143
System-Url: http://192.168.3.226
OpenEMMBackend Version: 20.04.000.142

Current menu: Show OpenEMM status

OpenEMM status:
Operating System (OS): CentOS 7
Python version: 3.8.1 (default, May 30 2020, 10:19:40) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Postfix Version: 2.10.1
DB client version: /bin/mysql Ver 15.1 Distrib 10.2.22-MariaDB, for Linux (x86_64) using EditLine wrapper
Database Connection: OK
Database version: MariaDB 10.2.22-MariaDB
OpenEMM database structure exists (Version 20.01.460)
Currently running database threads ratio (Current: 1, Maximum: 6): 0.167
Database table emm_db_errorlog_tbl: OK
Jobqueue status: OK
No DKIM keys
Java version: 1.8.0_252 (Oracle)
Tomcat version: 9.0.33.0
Tomcat-Native version: 1.2.23
Wkhtml version: wkhtmltopdf 0.12.5 (with patched qt)
OpenEMMApplication is running (Tomcat ProcessID: 7683, started at: 19:17 1:30)
OpenEMMBackend ok: once
OpenEMMBackend running: mailout, update, trigger, generate, mta, pickdist, slrtscn, direct-path, bav-update, bavd, bav, mlcontrol


I did, though, recompile Python 3.8.1 in line with the recently released manual (pasting excerpts here for convenience):

12 Deployment of Python 3.8 (or later)

RedHat:

At first, install the following required packages:
# yum install wget gcc gcc-c++ bzip2-devel
# yum install gdbm-devel libgcrypt-devel libffi-devel libxml2-devel ncurses-devel
# yum install openssl-devel readline-devel sqlite-devel zlib-devel xz xz-devel

Download the compressed tarball of Python 3.8.x (or later) from the official Python website
python.org to a working directory of your choice like /root/python3.
To create and deploy your customized version of Python, execute the following commands
as user root (with Python 3.8.2 as example):

# mkdir /home/openemm/opt/Python-3.8.2
# rm -f /home/openemm/opt/python3
# ln -s Python-3.8.2 /home/openemm/opt/python3
# tar -xaf Python-3.8.2.tar.xz
# cd Python-3.8.2
# ./configure --prefix=/home/openemm/opt/Python-3.8.2
# make
# make test
# make install
# export PATH="/home/openemm/opt/python3/bin:$PATH"
# which python3

The last command should result in output

/home/openemm/opt/python3/bin/python3

Install Python's package manager PIP:

# python3 -m pip install --upgrade pip

Finally, install some Python modules required by EMM Inhouse:

# python3 -m pip install py3dns
# python3 -m pip install xlrd xlwt xlutils
# python3 -m pip install paramiko pyspf dnspython dkimpy
# python3 -m pip install pycrypto
# python3 -m pip install requests
# python3 -m pip install httpie
# python3 -m pip install setproctitle
# python3 -m pip install inotify
# python3 m pip install aiodns aiohttp aiohttp-xmlrpc aiosmtpd



Best, Richard

Hi Richard,

thank you. That atleast gave me an error message.
Sadly the backend is still not able to start. I now get this error: "*** Error report for sanity check *** Check: <bound method OpenEMM.__crontab of <sanity3.OpenEMM object at 0x7fa24dd5acd0>> failed: ('list index out of range',)
Failed in sanity check"
Seems like some list in that python object is corruped. But I don't know what the class exactly does.

Thank you for your help.
Mika

I had some Problem with Pyhton 3.8 and the Export path.

the command export PATH="/home/openemm/opt/python3/bin:$PATH" is not permanent.
So after a reboot the Phyton3 was not found at all.

I changed the export Path to

echo 'pathmunge /home/openemm/opt/python3/bin' > /etc/profile.d/ree.sh
chmod +x /etc/profile.d/ree.sh
# and Reload profile
. /etc/profile

Maybe this will help.