macOS issues
Compare changes
-mmacosx-version-min
is required at link time as well, otherwise it may link different libraries. Hopefully this is the issue. That would also explain why it works for SciPy - I was already passing the link argument.