Segfaults in re2_regexp_error after libre2 is upgraded #43
Comments
|
Compiling with |
|
Hi @stanhu, Thanks for reporting this. I'm having trouble reproducing it. I tried the following on macOS 10.15.4 and Ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c):
url "https://github.com/google/re2/archive/2020-03-03.tar.gz"
version "20200303"
sha256 "04ee2aaebaa5038554683329afc494e684c30f82f2a1e47eb62450e59338f84d"
# ...
bottle do
cellar :any
sha256 "abb07cfb9def8d6f25a0d5694f0f23ba44b5b500f2d45d25d27515f62802c01b" => :catalina
sha256 "92c4603fad274003e71699376b20a40c6e57c13f878774a894e28a9ef73295c1" => :mojave
sha256 "58ac31c06c851bc5632c3a0703c0d41e55d8b015229cf03de09ee4a8701d92ef" => :high_sierra
end
~> pry -rre2
[1] pry(main)> regexp = RE2::Regexp.new("{", log_errors: false)
=> #<RE2::Regexp /{/>
[2] pry(main)> puts regexp.error unless regexp.ok?
=> nil
url "https://github.com/google/re2/archive/2020-04-01.tar.gz"
version "20200401"
sha256 "98794bc5416326817498384a9c43cbb5a406bab8da9f84f83c39ecad43ed5cea"
# ...
bottle do
cellar :any
sha256 "430e7efee518f5235ea75afc039168ce38c5407b525301199ccbc6c698300bb8" => :catalina
sha256 "67eed1f38907d1d7318b5d95f035ca18ff6186d85ee690560b8b1cd387f8afd4" => :mojave
sha256 "f111b7c906ab090d9db56e1f183f94d02e8cf5ce521e4ae4ff8e64d974cb5cbb" => :high_sierra
end
~> pry -rre2
[1] pry(main)> regexp = RE2::Regexp.new("{", log_errors: false)
=> #<RE2::Regexp /{/>
[2] pry(main)> puts regexp.error unless regexp.ok?
=> nilLooking at ~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> otool -L re2.bundle
re2.bundle:
/usr/local/opt/re2/lib/libre2.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)I know there is a breaking change coming in re2 (removing some deprecated APIs) so the SONAME has changed from 6 to 7, even upgrading to [1] pry(main)> regexp = RE2::Regexp.new("{", log_errors: false)
=> #<RE2::Regexp /{/>
[2] pry(main)> puts regexp.error unless regexp.ok?
=> nilThe bigger picture question here is what is reasonable behaviour for the gem when the underlying library is upgraded? Especially if some internal re2 API changes (as Hopefully there's some smaller issue we can fix here. |
|
You may need to rewind further to https://github.com/Homebrew/homebrew-core/blob/549a5c81c67e401fd3c75339157b3b936ebbbbed/Formula/re2.rb.
That's a good question. We have a check for |
|
Ah, that did the trick: compiling the gem against re2 2020-01-01 and then upgrading to 2020-04-01 causes the segfault when I call Looking at the diff between the two versions I see that there was a breaking ABI change and Google changed the soname from 0 to 6. (I see the latest release has changed again from 6 to 7). However, it looks like Homebrew explicitly changes the soname of the library to 0 regardless of ABI version. Removing the lines from the re2 Formula that change the dylib ID and add symlinks stops Homebrew masquerading libre2.6 and libre2.7 as libre2.0 and causes the gem to fail fast as soon as you require it: # Against re 2020-01-01 without the dylib ID changing or symlinks
~> gem install re2
Fetching re2-1.2.0.gem
Building native extensions. This could take a while...
Successfully installed re2-1.2.0
1 gem installed
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> otool -L re2.bundle
re2.bundle:
/usr/local/opt/re2/lib/libre2.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
# Upgrading to 2020-04-01 without the dylib ID changing or symlinks
~> brew upgrade re2
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> otool -L re2.bundle
re2.bundle:
/usr/local/opt/re2/lib/libre2.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 902.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
# Now libre2.0.dylib is entirely missing
~> ls -la /usr/local/opt/re2/lib
total 2168
drwxr-xr-x 7 mudge admin 224 2 May 10:47 .
drwxr-xr-x 9 mudge admin 288 2 May 10:47 ..
-r--r--r-- 1 mudge admin 348556 2 May 10:47 libre2.6.0.0.dylib
lrwxr-xr-x 1 mudge admin 18 2 May 10:47 libre2.6.dylib -> libre2.6.0.0.dylib
-r--r--r-- 1 mudge admin 755048 2 May 10:47 libre2.a
lrwxr-xr-x 1 mudge admin 18 2 May 10:47 libre2.dylib -> libre2.6.0.0.dylib
drwxr-xr-x 3 mudge admin 96 2 May 10:47 pkgconfig
# Try requiring re2 linked against the now-missing re2 version
~/.gem/ruby/2.6.5/gems/re2-1.2.0/ext/re2> pry -rre2
Traceback (most recent call last):
9: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `<main>'
8: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `load'
7: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/bin/pry:12:in `<top (required)>'
6: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/cli.rb:89:in `parse_options'
5: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:146:in `final_session_setup'
4: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `load_requires'
3: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `each'
2: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:105:in `block in load_requires'
1: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
/Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': cannot load such file -- re2 (LoadError)
13: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `<main>'
12: from /Users/mudge/.gem/ruby/2.7.1/bin/pry:23:in `load'
11: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/bin/pry:12:in `<top (required)>'
10: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/cli.rb:89:in `parse_options'
9: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:146:in `final_session_setup'
8: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `load_requires'
7: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:104:in `each'
6: from /Users/mudge/.gem/ruby/2.7.1/gems/pry-0.13.1/lib/pry/pry_class.rb:105:in `block in load_requires'
5: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:156:in `require'
4: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `rescue in require'
3: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:168:in `require'
2: from /Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.rb:6:in `<top (required)>'
1: from /Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
/Users/mudge/.rubies/ruby-2.7.1/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require': dlopen(/Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundle, 9): Library not loaded: /usr/local/opt/re2/lib/libre2.0.dylib (LoadError)
Referenced from: /Users/mudge/.gem/ruby/2.7.1/gems/re2-1.2.0/lib/re2.bundleLooking on Ubuntu, it seems the If I raise a pull request against Homebrew to stop overwriting the ABI version, this means the gem would fail fast if you try to This wouldn't require any change to the gem itself. What do you think? |
The re2 formula has been overwriting the soname of the libre2 shared library to 0 even though it has changed to 1 [0], 6 [1] and, most recently, 7 [2]. This means that any software linking to the library isn't aware of breaking ABI changes [3]. Fix this by no longer changing the soname to 0 and remove the extra symlinks as re2 itself will generate symlinks from libre2.dylib and libre2.7.dylib to the relevant version. [0]: google/re2@f734980#diff-b67911656ef5d18c4ae36cb6741b7965 [1]: google/re2@d067ca5#diff-b67911656ef5d18c4ae36cb6741b7965 [2]: google/re2@ac65d45#diff-b67911656ef5d18c4ae36cb6741b7965 [3]: mudge/re2#43
The re2 formula has been overwriting the soname of the libre2 shared library to 0 even though it has changed to 1 [0], 6 [1] and, most recently, 7 [2]. This means that any software linking to the library isn't aware of breaking ABI changes [3]. Fix this by no longer changing the soname to 0 and remove the extra symlinks as re2 itself will generate symlinks from libre2.dylib and libre2.7.dylib to the relevant version. [0]: google/re2@f734980#diff-b67911656ef5d18c4ae36cb6741b7965 [1]: google/re2@d067ca5#diff-b67911656ef5d18c4ae36cb6741b7965 [2]: google/re2@ac65d45#diff-b67911656ef5d18c4ae36cb6741b7965 [3]: mudge/re2#43 Closes #54136. Signed-off-by: Bo Anderson <[email protected]>
|
This has now been merged into Homebrew core; could you please re-test with the latest formula (2020-05-01 revision 1) and let me know if it behaves as you expect? Perhaps use the 2020-04-01 formula to install re2 with a soname of 0 (though it should be 6): Then install and compile the gem against that version with Then upgrade to the latest, fixed version of the re2 Formula (which has a soname of 7) and try to run the same command again: |
|
@mudge Awesome, that worked for me as well. Thanks so much! |
|
Maybe I should create a separate issue, but does |
|
I think you need to update to re2 1.2.0 as 2020-05-01 removed the |
|
@mudge Once again, thanks! That did the trick. |
When the system re2 library gets updated, it's often difficult and time-consuming to determine that a `gem pristine re2` is needed. This `gdk doctor` diagnostic attempts to reproduce the seg fault that occurs as a result of mudge/re2#43.
I'm not sure if this is an actual issue with the C extension or whether this just means
libre2needs to be compiled with-fPICfor this not to happen (will test this out). But whenever libre2 is updated (e.g. from v1.1 to v1.2 recently in https://github.com/Homebrew/homebrew-core/blob/40c227a5885561519827edecffd959f6316163d8/Formula/re2.rb#L4), it seems that re2 seg faults iferroris called. Runninggem pristine re2fixes the problem, but I'm wondering if this seg fault is something we can avoid. A simple script that reproduces the problem:Sometimes it fails:
An
objdump -dof thebundlefile shows:The text was updated successfully, but these errors were encountered: