Running a script via a symlink in a different directory fails to load module files

[jinwoo]

As an example, I have two source files -- main.pl and greetings.pl in the same directory, say ~/src.

main.pl:

#!/usr/bin/env swipl

:- initialization(main, main).

:- use_module(greetings).

main :-
    greetings('world', S),
    writeln(S).

greetings.pl:

:- module(greetings, [greetings/2]).

greetings(Name, Result) :-
    format(string(Result), 'Hello, ~w!', Name).

If I run main.pl directly, it runs fine. I can run it from any other directories.

But I put symlinks in my ~/bin directory for some of my utilities so that I can run them easily. Those symlinks run fine as long as the linked prolog scripts are simple and don't import other module files. But when they do, as my example above, the module import fails. The error message is like below.

ERROR: /Users/jinwoo/bin/main:5:
ERROR:    source_sink `greetings' does not exist
Warning: /Users/jinwoo/bin/main:5:
Warning:    Goal (directive) failed: user:use_module(greetings)
ERROR: /Users/jinwoo/bin/main:3: user:main main/0: Unknown procedure: greetings/2

I think SWI-Prolog tries to find module files from the directory where the symlink exists. It should probably follow the symlink and find the module files from the directory where the original script exists.

[JanWielemaker]

As is, file search doesn't do anything special wrt symlinks. I don't know whether it should. Is there current practice from other languages?

[jinwoo]

In my experience with other languages, all worked fine with symlinks. For example, in Python, I created two files similar to my Prolog example above.

main.py:

#!/usr/bin/env python3

import greetings

def main():
    print(greetings.greetings('world'))

if __name__ == '__main__':
    main()

greetings.py:

def greetings(name):
    return 'Hello, {}!'.format(name)

And I create a symlink from ~/bin/main to ~/src/hello/main.py`. And running through the symlink works fine.

[JanWielemaker]

Sounds like something worth considering. Do you happen to know a description on how Python deals with links to source files? Does this only apply for the main script? Does it in general try to find imported modules relative to the physical file? Does it also look relative tot the link? What if there are multiple chained links? It seems hard to find that documentation on the web 😢

[JanWielemaker]

This issue has been mentioned on SWI-Prolog. There might be relevant details there:

https://swi-prolog.discourse.group/t/symbolic-links-to-source-files-and-finding-relative-files/6884/1

[jinwoo]

I think it's just the main script. But I haven't tried importing modules through symlinks so I'm not sure it works with the same principle.

The comments in the discourse thread mention PYTHONHOME and PYTHONPATH but I don't think they are related.

Anyway thanks for considering this. Also I really appreciate your effort in making SWI-Prolog great. I love SWI-Prolog. Thank you!

[JanWielemaker]

For now, I guess the way to go is a shell script using

 exec swipl /path/to/main-file.pl "$@"

[jinwoo]

Good idea. I worked around by compiling the script and making the symlink point to the compiled binary. But a shell script would be better. Thanks!

[jinwoo]

On second thought, it'd be a nice feature to follow the symlinks even for the regular module import cases. It would help share the common code among code in different directories.

If I have foo.pl in dir1, which imports bar.pl in the same directory, and if I want to import foo.pl from client.pl in directory dir2 from a different project, I can simply create a symlink in dir2, which points to dir1/foo.pl. Then I'll be able to import the module, foo, without copying over all its dependencies such as bar.pl.

I think it's just a nice-to-have feature and a poor man's "package management" :) But it'd be useful sometimes.

[kamahen]

As a general rule, I would think that following symlinks would be best -- in other words, treat symlinks the same as hard links. Some specific situations require distinguishing symlinks from non-links (e.g., the Unix commands find, file, ls) but for most situations, symlinks should be fully "derferenced" at each step, the way the readlpath(1) command works (although I couldn't find where SWI-Prolog follows symlinks, when I did a quick scan through the source code).