./tools/provision fails to find its libraries when run using the fish shell

[gl-yziquel]

Hi.

I tried the direct installation method of zulip. (Because I tend to like building things from source). Instructions here:

https://zulip.readthedocs.io/en/latest/development/setup-advanced.html#installing-directly-on-ubuntu-debian-centos-or-fedora

I'm on a fresh git checkout. Ref: c91d018

The `./tools/provision' script fails. I'd expect it to succeed. Error message:

mini-me@virtucon ~/h/c/zulip (main)> ./tools/provision
Traceback (most recent call last):
  File "/home/mini-me/home/cellar/zulip/tools/./lib/provision.py", line 28, in <module>
    from tools.setup import setup_venvs
ModuleNotFoundError: No module named 'tools.setup'

From I gather, it seems to me that the call to ./lib/provision.py "$@" 2>&1 | tee -a "$LOG_PATH" occurs in the tools folder. Which is likely why it fails to find tools as a python module.

Confirmation here:

mini-me@virtucon ~/h/c/z/tools (main)> /usr/bin/env python3
Python 3.12.2 (main, Xxx xx xxxx, xx:xx:xx) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tools.setup
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'tools.setup'
>>> 
mini-me@virtucon ~/h/c/z/tools (main)> cd ..
mini-me@virtucon ~/h/c/zulip (main)> /usr/bin/env python3
Python 3.12.2 (main, Xxx xx xxxx, xx:xx:xx) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tools.setup
>>> 

I've already tried this a few months ago, and had the same issue.

These direct installation instructions seem broken to me.

[gl-yziquel]

As far as I see it, the script should be run in the tools folder as ./provision. That scripts claims, in the script, to be path independent. It doesn't seem to be.

[gl-yziquel]

Ah ! No ! Seems to go ok-is when ./provision is launched with bash, and not with the fish shell.

Amending the title accordingly.

Suggestion: at least mention in the documentation that one should not try the direct approach from the fish shell. Allows to avoid wasting time.

[timabbott]

I'd rather just make it work than clutter the documentation with coverage of an variant shell. provision has a shebang line of #!/usr/bin/env bash, so I don't see why a different shell should be relevant -- so it's likely not a deep bug.

Can you do a bit of print-debugging to see what precisely about how the fish shell sets up the environment is causing the problem?

[zulipbot]

Hello @zulip/server-development members, this issue was labeled with the "area: provision" label, so you may want to check it out!

[gl-yziquel]

Not really sure what kind of print debugging you want as the information I gave kind of contains the same information. Here it is, though. With a print of current working directory in the shell script, and another in the python code.

mini-me@virtucon ~/h/c/zulip (mini-me)> ./tools/provision
+ '[' 1000 -eq 0 ']'
++ dirname ./tools/provision
+ '[' -e ./tools/../.vagrant ']'
++ . /etc/os-release
+++ PRETTY_NAME='Ubuntu 23.10'
+++ NAME=Ubuntu
+++ VERSION_ID=23.10
+++ VERSION='23.10 (Mantic Minotaur)'
+++ VERSION_CODENAME=mantic
+++ ID=ubuntu
+++ ID_LIKE=debian
+++ HOME_URL=https://www.ubuntu.com/
+++ SUPPORT_URL=https://help.ubuntu.com/
+++ BUG_REPORT_URL=https://bugs.launchpad.net/ubuntu/
+++ PRIVACY_POLICY_URL=https://www.ubuntu.com/legal/terms-and-policies/privacy-policy
+++ UBUNTU_CODENAME=mantic
+++ LOGO=ubuntu-logo
++ echo 'ubuntu 23.10'
+ os='ubuntu 23.10'
+ case "$os" in
++ python3 --version
+ python_version='Python 3.12.2'
+ case "$python_version" in
+ FAIL='\033[91m'
+ WARNING='\033[93m'
+ ENDC='\033[0m'
+++ dirname ./tools/provision
++ cd ./tools
++ pwd -P
+ PARENT_PATH=/home/mini-me/home/cellar/zulip/tools
+ cd /home/mini-me/home/cellar/zulip/tools
+ mkdir -p ../var/log
+ LOG_PATH=../var/log/provision.log
+ echo 'PROVISIONING STARTING.'
+ export PYTHONUNBUFFERED=1
+ PYTHONUNBUFFERED=1
++ pwd
+ echo Current path: /home/mini-me/home/cellar/zulip/tools
Current path: /home/mini-me/home/cellar/zulip/tools
+ ./lib/provision.py
+ tee -a ../var/log/provision.log
Current path: /home/mini-me/home/cellar/zulip/tools
Traceback (most recent call last):
  File "/home/mini-me/home/cellar/zulip/tools/./lib/provision.py", line 29, in <module>
    from tools.setup import setup_venvs
ModuleNotFoundError: No module named 'tools.setup'
+ failed=1
+ '[' 1 -ne 0 ']'
+ echo -e '\033[91m'

+ echo 'Provisioning failed (exit code 1)!'
Provisioning failed (exit code 1)!
+ echo

+ echo '* Look at the traceback(s) above to find more about the errors.'
* Look at the traceback(s) above to find more about the errors.
+ echo '* Resolve the errors or get help on chat.'
* Resolve the errors or get help on chat.
+ echo '* If you can fix this yourself, you can re-run tools/provision at any time.'
* If you can fix this yourself, you can re-run tools/provision at any time.
+ echo '* Logs are here: zulip/var/log/provision.log'
* Logs are here: zulip/var/log/provision.log
+ echo -e '\033[0m'

+ exit 1
mini-me@virtucon ~/h/c/zulip (mini-me) [1]>

[timabbott]

I guess I'm specifically interested in how PYTHONPATH and PATH differ between fish and bash on your system.