Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vendor re2 #61

Closed
mudge opened this issue Dec 2, 2022 · 3 comments · Fixed by #67
Closed

Vendor re2 #61

mudge opened this issue Dec 2, 2022 · 3 comments · Fixed by #67
Labels
feature
Milestone
2.0: Precompiled ...

Comments

@mudge
Copy link
Owner

mudge commented Dec 2, 2022 •
edited
Loading

As suggested by @tomstuart: so that users don’t need to separately install the underlying re2 library, provide a vendored version that is compiled when the gem is installed.

In order to avoid breaking compatibility for existing users, the system re2 should still be preferred (either automatically found in /usr/local, /opt/homebrew or /usr or specified during install with —with-re2-dir) with the vendored version only built and used as a fallback.

Starting points

Questions

  • How can we reliably compile our own version of re2 that links to the gem successfully?
  • How do we support various different platforms (including Windows)?
  • What are the licensing implications of vendoring the upstream re2 source code inside the gem?
  • Does this put pressure on releasing the gem every time there is an re2 update?
@mudge mudge added the feature label Dec 2, 2022
@stanhu
Copy link
Collaborator

stanhu commented Jan 6, 2023 •
edited
Loading

I came across this issue because every time Homebrew updates re2, this appears to break the gem and users have to run gem pristine re2 to get back into a working state. I was looking to make this gem statically link re2, but it appears that Homebrew only ships the shared library. Homebrew/homebrew-core#119928 fixes that.

How can we reliably compile our own version of re2 that links to the gem successfully?

https://github.com/flavorjones/mini_portile, which Nokogiri uses, should help. I think you also have to keep in mind to include the .tar.gz with the gem because you'll want to avoid downloading the source from an external site every time the gem installs.

Related work with ruby-magic:

  1. Build by vendoring libmagic kwilczynski/ruby-magic#3
  2. Use mini_portile2 to build libmagic kwilczynski/ruby-magic#5

How do we support various different platforms (including Windows)?

mini_portile should handle most of the issues. You'll see Nokogiri has some special handling for Windows.

What are the licensing implications of vendoring the upstream re2 source code inside the gem?

I suspect there's no issue since the re2 is a BSD license, and you're not actually modifying it?

Does this put pressure on releasing the gem every time there is an re2 update?

Perhaps. It looks like re2 updates every few months.

@stanhu stanhu mentioned this issue Jan 6, 2023
Closed
stanhu added a commit to stanhu/re2 that referenced this issue Jul 11, 2023
@stanhu
Vendor re2 and abseil
9003f9c 
If --disable-system-libraries is specified, this change will download
and build abseil and re2. `cmake` and a C++17 compiler is required for
this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
@stanhu stanhu mentioned this issue Jul 11, 2023
Merged
stanhu added a commit to stanhu/re2 that referenced this issue Jul 11, 2023
@stanhu
Vendor re2 and abseil
3e756f7 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
stanhu added a commit to stanhu/re2 that referenced this issue Jul 11, 2023
@stanhu
Vendor re2 and abseil
aa7790f 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
stanhu added a commit to stanhu/re2 that referenced this issue Jul 11, 2023
@stanhu
Vendor re2 and abseil
b4eb2af 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when they are updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
stanhu added a commit to stanhu/re2 that referenced this issue Jul 12, 2023
@stanhu
Vendor re2 and abseil
35c1b19 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when they are updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
stanhu added a commit to stanhu/re2 that referenced this issue Jul 13, 2023
@stanhu
Vendor re2 and abseil
7c5c63a 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when they are updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
stanhu added a commit to stanhu/re2 that referenced this issue Jul 13, 2023
@stanhu
Vendor re2 and abseil
f396485 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when they are updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes mudge#61
mudge pushed a commit that referenced this issue Jul 15, 2023
@stanhu @mudge
Vendor re2 and abseil
c00bcc4 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when they are updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes #61
@mudge mudge added this to the 2.0: Precompiled gems milestone Jul 15, 2023
@mudge
Copy link
Owner Author

mudge commented Jul 15, 2023

The first part of this is now done thanks to #67

@mudge mudge closed this as completed Jul 15, 2023
@tomstuart
Copy link

Hurrah! 🎉

mudge pushed a commit that referenced this issue Sep 13, 2023
@stanhu @mudge
Vendor re2 and abseil
417207c 
If `--disable-system-libraries` is specified, this change will
download and build abseil and re2. `cmake` and a C++17 compiler is
required for this to work.

This makes it possible to ensure all the required dependencies are
contained within the C extension rather than depend on system
libraries, which may break the extension when they are updated.

The building of these libraries uses mini_portile2 and techniques
borrowed from the nokogiri and ruby-magic gems.

Closes #61
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature
Projects
None yet
Milestone
2.0: Precompiled gems
Development

Successfully merging a pull request may close this issue.

Vendor re2 and abseil stanhu/re2
3 participants
@mudge @tomstuart @stanhu