Skip to content

chore: Publish docs to arrow.apache.org#68

Merged
kou merged 27 commits into
apache:mainfrom
Benjamin-Philip:bp/publish-arrow-website
Jul 8, 2026
Merged

chore: Publish docs to arrow.apache.org#68
kou merged 27 commits into
apache:mainfrom
Benjamin-Philip:bp/publish-arrow-website

Conversation

@Benjamin-Philip

@Benjamin-Philip Benjamin-Philip commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

What issue does this PR close?

Closes #55.

What's changed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates CI workflows to publish generated documentation to arrow.apache.org/erlang/ (via an asf-site branch) in addition to GitHub Pages, and reorganizes doc building/deployment responsibilities across workflows.

Changes:

  • Removes the docs-generation test job from Erlang CI.
  • Refactors Docs workflow into a build job plus separate deploy jobs for GitHub Pages and ASF (asf-site) with job-scoped permissions/concurrency.
  • Adds an ASF deployment job that publishes the built docs to the asf-site branch.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
.github/workflows/erlang-ci.yml Removes the standalone doc generation test job from Erlang CI.
.github/workflows/docs.yml Splits docs build/deploy into separate jobs and adds deployment to asf-site for arrow.apache.org publishing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
@Benjamin-Philip
Benjamin-Philip marked this pull request as ready for review June 29, 2026 10:03
@Benjamin-Philip
Benjamin-Philip requested a review from kou June 29, 2026 10:15
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
Comment thread .github/workflows/docs.yml Outdated
@Benjamin-Philip

Benjamin-Philip commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator Author

I forgot to mention that when we publish a release to hex.pm (Erlang and Elixir's defacto package manager), we will also publish versioned docs to hexdocs.

See https://hex.pm/docs/rebar3-publish for the exact process.

With ASF site, GitHub Pages and HexDocs, there will be 3 mirrors for documentation.

@kou

kou commented Jul 1, 2026

Copy link
Copy Markdown
Member

If we want to use hex.pm as our documentation site like we did for Go ( https://arrow.apache.org/go/ redirects to pkg.go.dev ), we can use only it.

@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

I definitely don't want to break the existing pattern, so we can use Hexpm as our primary documentation site.

However, I still want to be able to serve docs until we cut a release. Also, Hexpm will only serve docs for releases much like pkg.go.dev or docs.rs. So, I think it might be useful to have an alternate site serving docs from main?

arrow-rs is serving from main on the ASF site, while docs.rs is handling serving all the versions. Maybe we could replicate this and shutdown GH Pages?

@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

I definitely don't want to break the existing pattern, so we can use Hexpm as our primary documentation site.

However, I still want to be able to serve docs until we cut a release. Also, Hexpm will only serve docs for releases much like pkg.go.dev or docs.rs. So, I think it might be useful to have an alternate site serving docs from main?

arrow-rs is serving from main on the ASF site, while docs.rs is handling serving all the versions. Maybe we could replicate this and shutdown GH Pages?

@kou Thoughts?

@kou

kou commented Jul 3, 2026

Copy link
Copy Markdown
Member

I'm OK that we use Hexpm for released documentations but I don't want to use https://arrow.apache.org/erlang/ for documentation for the main branch because users may access to https://arrow.apache.org/erlang/ .

We should provide documentations of released versions for users. Documentations for main (not released) should be only for Apache Arrow developers.

How about setting redirect to Hexpm for https://arrow.apache.org/erlang/ and putting the documentation for main to https://arrow.apache.org/erlang/main/ or something?

shutdown GH Pages?

Could you do #68 (comment) for it?

@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

Let's serve https://arrow.apache.org/erlang/main/ and redirect the rest to
Hexpm. I'll create an issue in apache/infrastructure-asfyaml to take down the GH
Pages site. I'll also go ahead with using GH Pages in forks as a preview.

@Benjamin-Philip
Benjamin-Philip force-pushed the bp/publish-arrow-website branch from bd9b17d to 6c9f8c4 Compare July 5, 2026 17:53
@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

It looks like we'll need update dtolnay/rust-toolchain to pass ASF allowlist. I'll rebase once the dependabot PR comes along.

Googling around, I read you could trigger one from the Insights > Dependency graph > Dependabot tab?

@kou

kou commented Jul 6, 2026

Copy link
Copy Markdown
Member

I can't find it in the tab.

It seems that our comment (# v1) is wrong:

- uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # v1

Dependabot may not work because of it... Let's update it manually in a separated PR.

It seems that we need to use # stable based on:

@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

My bad. I had put # v1 because that was the last release.

@Benjamin-Philip
Benjamin-Philip force-pushed the bp/publish-arrow-website branch from d78ecfc to 9e3a7fc Compare July 6, 2026 07:02
@Benjamin-Philip
Benjamin-Philip force-pushed the bp/publish-arrow-website branch from 9e3a7fc to b88bf74 Compare July 6, 2026 13:16
@Benjamin-Philip
Benjamin-Philip marked this pull request as draft July 6, 2026 13:18
Comment thread .htaccess Outdated
Comment on lines +18 to +20
ErrorDocument 404 /erlang/main/404.html

RedirectMatch permanent ^/erlang/(?!main)(.*)$ https://arrow.hexdocs.pm/$1

@Benjamin-Philip Benjamin-Philip Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this on httpd version:

Server version: Apache/2.4.68 (Fedora Linux)
Server built:   Jun  9 2026 00:00:00

I also aliased DocumentRoot to /erlang/ in my httpd.conf to better match the ASF site's conditions:

# Map DocumentRoot to /erlang

<Location "/">
    Require all denied
</Location>

Alias "/erlang" "/home/bphilip/GitHub/arrow-erlang/doc"

<Location "/erlang">
    Require all granted
</Location>

Comment on lines +135 to +139
- name: Prepare website
run: |
tar -xf docs/artifact.tar -C asf-site/main
cp .asf.yaml asf-site
cp .htaccess asf-site

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm copying .htaccess here because I don't want to overwrite any configuration on HexDocs's side in-case they use Apache.

Since I'm copying .htaccess here, I decided I'll copy .asf.yaml here as well.

@Benjamin-Philip
Benjamin-Philip marked this pull request as ready for review July 7, 2026 06:44
@Benjamin-Philip
Benjamin-Philip requested review from Copilot and kou and removed request for Copilot July 7, 2026 06:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comment thread .github/workflows/docs.yml
Comment thread .htaccess Outdated
Comment thread .github/workflows/pr_comment.yml
Comment thread CONTRIBUTING.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Comment thread .htaccess Outdated
Comment thread .github/workflows/pr_comment.yml Outdated
Comment thread .github/workflows/pr_comment.yml
Comment thread .github/workflows/docs.yml
Comment thread CONTRIBUTING.md
@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

We can track closing down the existing GitHub Pages site in a separate issue and PR, since a fix upstream in ASF infrastructure may take time.

@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

@kou, I've created an issue upstream: apache/infrastructure-asfyaml#109.

@kou kou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Comment thread .github/workflows/docs.yml
@Benjamin-Philip

Copy link
Copy Markdown
Collaborator Author

I'm thinking even if these docs get published at arrow.apache.org, I'll only update arrow-site to reference it after our first release.

@kou
kou merged commit 7e26121 into apache:main Jul 8, 2026
11 checks passed
@Benjamin-Philip
Benjamin-Philip deleted the bp/publish-arrow-website branch July 8, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish https://arrow.apache.org/erlang/

3 participants