Where’s that library?

This note is about some difficulties using shared libraries (.dylibs) on macOS.

On Unix-like operating systems, when an executable that uses a shared library loads, the loader (ld.so on Linux, dyld on macOS) uses a search path to find the shared library.

If you don’t do anything else, there’s a standard search path (on macOS, /usr/local/lib:/usr/lib). You can prefix this using the environment variable LD_LIBRARY_PATH on Linux, or DYLD_LIBRARY_PATH on macOS (some say that macOS allows either form, but for me (Monterey) LD_LIBRARY_PATH doesn’t).

If (speaking of macOS only now) you specify -rpath path then that path is baked into the resulting executable or shared library (‘dylib’).

In the case of building cvc4, where the target build structure is

/Volumes/Miscellaneous1/spark2014/install/libexec/spark/bin
                                                        |
                                                        lib

the path baked in (found by otool -l cvc4 | tail) is

/Volumes/Miscellaneous1/spark2014/install/libexec/spark/lib

You find the shared libraries used by cvc4 by

$ otool -L cvc4
cvc4:
	@rpath/libcvc4parser.7.dylib (compatibility version 7.0.0, current version 0.0.0)
	@rpath/libcvc4.7.dylib (compatibility version 7.0.0, current version 0.0.0)
	/opt/gcc-12.0.1/lib/libgmp.10.dylib (compatibility version 15.0.0, current version 15.1.0)
	/opt/gcc-12.0.1/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.30.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)

which means that whenever you run cvc4 it’ll translate the @rpath into the baked-in path.

Unfortunately, not only “whenever” but also “wherever”. You’d hope to be able to copy everything under /Volumes/Miscellaneous1/spark2014/install into an archive so that someone else would be able to install to some convenient place and have it just work without that pesky development disk. No:

$ bin/cvc4
dyld[6029]: Library not loaded: @rpath/libcvc4parser.7.dylib
  Referenced from: /Users/simon/tmp/libexec/spark/bin/cvc4
  Reason: tried: '/Volumes/Miscellaneous1/spark2014/install/libexec/spark/lib/libcvc4parser.7.dylib' (no such file),
  '/Volumes/Miscellaneous1/spark2014/install/libexec/spark/lib/libcvc4parser.7.dylib' (no such file),
  '/usr/local/lib/libcvc4parser.7.dylib' (no such file),
  '/usr/lib/libcvc4parser.7.dylib' (no such file)
Abort trap: 6

(and who wants to require users to set DYLD_LIBRARY_PATH?)

A manual workround is to edit the executable so it doesn’t use the baked-in path.

As well as @rpath, you can use @load_path, which is where dyld found the executable or dylib that needs the referenced dylib, or @executable_path, which is where (in this case) cvc4 was found.

We need to change @rpath/libcvc4parser.7.dylib to @executable_path/../lib/libcvc4parser.7.dylib:

$ install_name_tool \
  -change \
  @rpath/libcvc4parser.7.dylib \
  @executable_path/../lib/libcvc4parser.7.dylib \
  cvc4

and likewise for libcvc4.7.dylib.

What would have made things easier would have been if cvc4 had been linked with -rpath @executable_path/../lib - this works because we know that the executable is in spark/bin and the dylib is in spark/lib.