[ Changing PYTHONHOME causes ImportError: No module named site ]

I am trying to deploy a Flask web app via mod_wsgi on Apache. I cannot use the default Python environment because it was compiled with UCS-2 Unicode instead of UCS-4, and I cannot recompile it for this one case. Thus, a virtual environment. Virtual environments would have been used anyway, but that error means that I can't get away with using the default Python install and just adding the virtual environment's modules to the PYTHONPATH, which otherwise would have let me avoid the current problem entirely by accident.

I found the documentation for mod_wsgi to change which Python executable to use. However, when attempting to do so, the server fails to work properly. /var/log/httpd/error_log rapidly floods with the line ImportError: No module named site.

I have checked every similar question I can find here and elsewhere, and not yet had success. Experimentation has shown that as far as I can tell, the problem occurs when changing PYTHONHOME without activating a virtual environment - and the way the automated deployment works (via Fabric), as far as I can tell I can't activate a virtual environment.

---

Apache config

My current httpd.conf for the app:

WSGIPythonPath /path/to/dir/containing/wsgi/file/and/app:/path/to/virtualenv/lib:/path/to/virtualenv/lib/site-packages

WSGIPythonHome /path/to/virtualenv

WSGISocketPrefix /var/run/wsgi

User user

Group group

<VirtualHost *>

    ServerName servername.generic.com

    DocumentRoot /path/to/dir/containing/wsgi/file/and/app/static_dev/

    WSGIDaemonProcess appname user=user group=group threads=2

    WSGIScriptAlias / /path/to/dir/containing/wsgi/file/and/app/app.wsgi

    <Directory /path/to/dir/containing/wsgi/file/and/app>
        WSGIProcessGroup appname
        WSGIApplicationGroup %{GLOBAL}
        Require all granted
    </Directory>
</VirtualHost>

---

Data I've found from failed attempts

I know that the error is not in my app.wsgi, because when I added the line raise Exception('tried to open the file') at the very top to check that, the existing ImportError kept happening instead of that new Exception.

I have confirmed via ldd that my version of mod_wsgi is for Python 2.7.

I have tried setting WSGIPythonHome /path/to/virtualenv/bin/ and WSGIPythonHome /path/to/virtualenv/bin/python, with the same result as the current state.

I have tried omitting the WSGIPythonHome directive, in which case it loads the app.wsgi as it should, but breaks on a later import as described at the top (the reason I can't just do that).

I have tried omitting the WSGIPythonPath directive and leaving it up to app.wsgi to add things to the PYTHONPATH, with the same result as the current state.

I have tried putting the path-setting as an argument to WSGIDaemonProcess instead of as the WSGIPythonPath directive, with the same result as the current state.

I have confirmed that there is a site.py in /path/to/virtualenv/lib.

I have confirmed that no other non-app-specific Apache .conf files being used (default settings, automatic module loads, etc) contain the string "WSGI", so I don't think there's any conflicts here.

If I activate the virtual environment from the command line I can import site without an error, just for the sake of testing that it does in fact exist in the environment. However, this is insufficient because it needs to start smoothly from a single call to sudo systemctl start httpd.service due to the deployment tools in use, and that seems to not care about the venv of the current shell session.

If, from a default state, I export PYTHONHOME=/path/to/virtualenv, attempting to open the Python REPL exits immediately with ImportError: No module named site.

If I activate the virtual environment and then set PYTHONHOME, I get the same import error.

If I activate the virtual environment and don't touch PYTHONHOME, echo $PYTHONHOME outputs a blank line, and the Python REPL works fine. In the Python REPL while in the virtualenv:

(virtualenv)-bash-4.2$ python
Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/path/to/virtualenv'
>>> sys.exec_prefix
'/path/to/virtualenv'

Even though setting PYTHONHOME to the same value didn't work.

If I try export PYTHONHOME=:/path/to/virtualenv or export PYTHONHOME=/path/to/virtualenv:, explicitly setting only one of prefix and exec_prefix, it fails with the same import error in either case.

If I activate the virtual environment and set PYTHONHOME in one of those latter two ways, the unset one appears to default to / rather than to the usual default value, but the Python REPL runs fine:

# Setting only exec_prefix
(virtualenv)-bash-4.2$ export PYTHONHOME=:/path/to/virtualenv
(virtualenv)-bash-4.2$ python
Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/'
>>> sys.exec_prefix
'/path/to/virtualenv'
>>> quit()
# Setting only prefix
(.virtualenv)-bash-4.2$ export PYTHONHOME=/path/to/virtualenv:
(.virtualenv)-bash-4.2$ python
Python 2.7.8 (default, Aug 14 2014, 13:26:38)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/path/to/virtualenv'
>>> sys.exec_prefix
'/'

Unfortunately, since the deployment script doesn't care what environment is activated, that doesn't solve it. Trying to set WSGIPythonHome in such a fashion makes no difference whatsoever.

I have noticed one further thing: The Python in the virtualenv is 2.7.8. The Python run outside the virtualenv (usr/bin/python) is 2.7.5. I do not know - would this affect the setting of PYTHONHOME somehow? I would hope not - since that seems to defeat the entire purpose of using WSGIPythonHome to run a virtualenv as compared to just setting sys.path inside the app.wsgi file, the ability to start from a different executable - but I cannot rule it out, clueless as I am.

The 2.7.8 Python in /path/to/virtualenv/bin/python has a sys.real_prefix of /network-mounted-drive/sw/python/python-2.7.8.

I changed the deployment to build from /network-mounted-drive/sw/python/python-2.7.5, then did more tests. Results as follows:

Attempting to start httpd gives the same import error as before.

Setting PYTHONHOME to the location of the virtual environment, then running python:

-bash-4.2$ echo $PYTHONHOME
/path/to/virtualenv
-bash-4.2$ python
ImportError: No module named site

Setting PYTHONHOME to the location of the virtual environment, then explicitly running the virtual environment's python binary (activating the virtual environment and then running python gives the same result):

# In the directory just above the virtualenv
-bash-4.2$ ./virtualenv/bin/python
Python 2.7.5 (default, Mar 14 2016, 14:13:09)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/path/to/virtualenv'
>>> sys.exec_prefix
'/path/to/virtualenv'
>>> sys.real_prefix
'/network-mounted-drive/sw/python/python-2.7.5

---

Does anyone have any idea?

Answer 1

---

Resolution found: The issue seems to have been in trying to use a virtual environment built from something other than the local python install on the system.

Solved by pushing the problem of "local python install on the deployment VM doesn't have pip installed" up the chain to someone with the permissions required to install pip, since no attempted workarounds via networked python installs worked.

The issue of actually using a virtual environment chained from a Python install on a network drive for mod_wsgi may be insoluble, or at least I couldn't figure it out in a reasonable amount of time relative to the bureaucratic solution.