Skip to content
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e6df30e
tmp: A naive github-sync framework
Musicminion Feb 24, 2026
fa29536
dev: update dev yml for github sync, also added with ldap test, Readm…
Musicminion Feb 24, 2026
a48fef0
web: update passport-ldapauth to 3.0.1
Musicminion Feb 24, 2026
382d914
optimize: clean unused code and fix unauthorized github reminder
Musicminion Feb 24, 2026
8ae6703
feat: support import and export for github
Musicminion Feb 26, 2026
9d06105
github-sync: bug fix and new features:
Musicminion Feb 26, 2026
76654fb
feature: support github sync with merge API, also frontend code
Musicminion Feb 27, 2026
9c96646
fix: add clean up for github-sync
Musicminion Feb 28, 2026
e17d2c5
frontend: optimize import while loading
Musicminion Feb 28, 2026
9094979
fix: use fast-forward to apply changes in overleaf if no update in gi…
Musicminion Feb 28, 2026
5ababd6
ce: add github-sync to `server-ce`
Musicminion Feb 28, 2026
370bf05
feat: add proxy for github sync
Musicminion Feb 28, 2026
37f8b84
fix: proxy import from github-sync
Musicminion Feb 28, 2026
7f1a7b7
fix: enhance GitHub sync Modal with need-auth modal for new user
Musicminion Feb 28, 2026
ff2527a
docs: update README
Musicminion Mar 2, 2026
dd03d8f
fix: refactor GitHub sync module imports to use dynamic loading
Musicminion Mar 2, 2026
edd28de
fix: add fetchJson import for GitHub sync in handler
Musicminion Mar 2, 2026
17c0e85
github-sync: remove dev router
Musicminion Mar 2, 2026
4511f94
ci: add `github-sync` to dev image
Musicminion Mar 3, 2026
0886af2
fix: improve error handling in GitHub sync and optimize token storage
Musicminion Mar 3, 2026
bc23e17
fix: filter non-blob items in generateRespURL and update token validi…
Musicminion Mar 3, 2026
b0c869e
fix: enhance validation for GitHub sync state and improve error handling
Musicminion Mar 3, 2026
34f0013
fix: update OLModal properties to disable initialFocus
Musicminion Mar 7, 2026
be50d34
fix: enhance error handling for GitHub branch diffing by checking res…
Musicminion Mar 7, 2026
8745430
Add Star History section to README
Musicminion Mar 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,29 @@
<a href="#license">License</a>
</p>

<img src="doc/screenshot.png" alt="A screenshot of a project being edited in Overleaf Community Edition">
<img src="doc/screenshot-pro.png" alt="A screenshot of a project being edited in Overleaf Community Edition">
<p align="center">
Figure 1: A screenshot of a project being edited in Overleaf Community Edition.
Figure 1: A screenshot of a project being edited in Overleaf Pro Edition.
</p>

## Overleaf Pro Edition
Overleaf Pro is an enhanced version of Overleaf with almost all features and capabilities. For details, please check [Overleaf Pro](https://overleaf-pro.ayaka.space) page. Features in Overleaf Pro include:

- GitHub Sync in 2-ways (Features in SaaS)
- Git-Bridge Support (Features in Server Pro)
- Admin Panel (User/Project management)
- SSO with LDAP and SAML or OAuth 2.0
- Unlimited Compile Times (Adjustable in admin panel)
- Self Register (Optional, can be limited by mail domain)
- Sandbox Compile (With [texlive-full](https://github.com/ayaka-notes/texlive-full) Image Support)
- Template System (With Template Gallery)
- Track Changes (With Review and Comment Panel)
- Full Project History
- Symbol Palette
- ARM Support

Last but not least, Overleaf Pro is open-source, free to use and modify. You can self-host it and contribute to the development of Overleaf Pro. For more details, please check [Developer Documentation](https://overleaf-pro.ayaka.space/dev) page.

## Community Edition

[Overleaf](https://www.overleaf.com) is an open-source online real-time collaborative LaTeX editor. We run a hosted version at [www.overleaf.com](https://www.overleaf.com), but you can also run your own local version, and contribute to the development of Overleaf.
Expand Down
1 change: 1 addition & 0 deletions develop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ each service:
| `history-v1` | 9239 |
| `project-history` | 9240 |
| `linked-url-proxy` | 9241 |
| `github-sync` | 9242 |

To attach to a service using Chrome's _remote debugging_, go to
<chrome://inspect/> and make sure _Discover network targets_ is checked. Next
Expand Down
11 changes: 11 additions & 0 deletions develop/docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,17 @@ services:
- ../services/filestore/app.js:/overleaf/services/filestore/app.js
- ../services/filestore/config:/overleaf/services/filestore/config

github-sync:
command: ["node", "--watch", "app.js"]
environment:
- NODE_OPTIONS=--inspect=0.0.0.0:9229
ports:
- "127.0.0.1:9242:9229"
volumes:
- ../services/github-sync/app:/overleaf/services/github-sync/app
- ../services/github-sync/app.js:/overleaf/services/github-sync/app.js
- ../services/github-sync/config:/overleaf/services/github-sync/config

history-v1:
command: ["node", "--watch", "app.js"]
environment:
Expand Down
16 changes: 15 additions & 1 deletion develop/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,13 @@ services:
- filestore-uploads:/overleaf/services/filestore/uploads
- history-v1-buckets:/buckets

github-sync:
build:
context: ..
dockerfile: services/github-sync/Dockerfile
env_file:
- dev.env

history-v1:
build:
context: ..
Expand Down Expand Up @@ -155,6 +162,7 @@ services:
volumes:
- sharelatex-data:/var/lib/overleaf
- web-data:/overleaf/services/web/data
- ./data/certs/:/var/lib/overleaf/certs/
depends_on:
- mongo
- redis
Expand Down Expand Up @@ -232,4 +240,10 @@ services:
ports:
- "8081:8081"
environment:
ME_CONFIG_MONGODB_SERVER: mongo
ME_CONFIG_MONGODB_SERVER: mongo

# For ldap testing
# https://github.com/rroemhild/docker-test-openldap
ldap:
restart: always
image: ghcr.io/rroemhild/docker-test-openldap:v2.5.0
Binary file added doc/screenshot-pro.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions server-ce/config/env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ export PROJECT_HISTORY_HOST=127.0.0.1
export REALTIME_HOST=127.0.0.1
export WEB_HOST=127.0.0.1
export WEB_API_HOST=127.0.0.1
export GITHUB_SYNC_HOST=127.0.0.1

# If SANDBOXED_COMPILES_SIBLING_CONTAINERS is set to true,
# we need to set the TEXLIVE_IMAGE_USER to www-data so that the
Expand Down
12 changes: 12 additions & 0 deletions server-ce/runit/github-sync/run
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/bin/bash

NODE_PARAMS=""
if [ "$DEBUG_NODE" == "true" ]; then
echo "running debug - github-sync"
NODE_PARAMS="--inspect=0.0.0.0:30670"
fi

source /etc/overleaf/env.sh
export LISTEN_ADDRESS=127.0.0.1

exec /sbin/setuser www-data /usr/bin/node $NODE_PARAMS /overleaf/services/github-sync/app.js >> /var/log/overleaf/github-sync.log 2>&1
3 changes: 3 additions & 0 deletions server-ce/services.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ module.exports = [
{
name: 'linked-url-proxy',
},
{
name: 'github-sync',
}
]

if (require.main === module) {
Expand Down
1 change: 1 addition & 0 deletions services/github-sync/.nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
22.18.0
46 changes: 46 additions & 0 deletions services/github-sync/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# This file was auto-generated, do not edit it directly.
# Instead run bin/update_build_scripts from
# https://github.com/overleaf/internal/

FROM node:22.18.0 AS base

WORKDIR /overleaf/services/github-sync

# Google Cloud Storage needs a writable $HOME/.config for resumable uploads
# (see https://googleapis.dev/nodejs/storage/latest/File.html#createWriteStream)
RUN mkdir /home/node/.config && chown node:node /home/node/.config

FROM base AS app

COPY package.json package-lock.json /overleaf/
COPY libraries/access-token-encryptor/package.json /overleaf/libraries/access-token-encryptor/package.json
COPY libraries/fetch-utils/package.json /overleaf/libraries/fetch-utils/package.json
COPY libraries/logger/package.json /overleaf/libraries/logger/package.json
COPY libraries/metrics/package.json /overleaf/libraries/metrics/package.json
COPY libraries/mongo-utils/package.json /overleaf/libraries/mongo-utils/package.json
COPY libraries/o-error/package.json /overleaf/libraries/o-error/package.json
COPY libraries/promise-utils/package.json /overleaf/libraries/promise-utils/package.json
COPY libraries/settings/package.json /overleaf/libraries/settings/package.json
COPY libraries/stream-utils/package.json /overleaf/libraries/stream-utils/package.json
COPY services/github-sync/package.json /overleaf/services/github-sync/package.json
COPY tools/migrations/package.json /overleaf/tools/migrations/package.json
COPY patches/ /overleaf/patches/
COPY tools/migrations/ /overleaf/tools/migrations/

RUN cd /overleaf && npm ci --quiet
COPY libraries/access-token-encryptor/ /overleaf/libraries/access-token-encryptor/
COPY libraries/fetch-utils/ /overleaf/libraries/fetch-utils/
COPY libraries/logger/ /overleaf/libraries/logger/
COPY libraries/metrics/ /overleaf/libraries/metrics/
COPY libraries/mongo-utils/ /overleaf/libraries/mongo-utils/
COPY libraries/o-error/ /overleaf/libraries/o-error/
COPY libraries/promise-utils/ /overleaf/libraries/promise-utils/
COPY libraries/settings/ /overleaf/libraries/settings/
COPY libraries/stream-utils/ /overleaf/libraries/stream-utils/
COPY services/github-sync/ /overleaf/services/github-sync/
COPY tools/migrations/ /overleaf/tools/migrations/

FROM app
USER node

CMD ["node", "--expose-gc", "app.js"]
36 changes: 36 additions & 0 deletions services/github-sync/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# GitHub Sync Service

Overleaf Github Sync Service, @Ayaka-notes.

## Service Overview
This service is only responsible for 2 things:
- export existing Overleaf projects to GitHub repositories
- export existing overleaf changes and merge changes from GitHub repositories to Overleaf projects

## How we do sync bewtween Overleaf and GitHub
We use `sync point` to track the last synced commit in GitHub and the last synced version in Overleaf, which is stored in the mongodb. A `sync point` means a version in overleaf and a commit in github, they are totally the same content.

When we do sync, we will first check the last overleaf version and current overleaf version, if they are different, we will export the changes from overleaf to github, **as a branch**, we call it `overleaf branch` in the later.

Then we will try to merge this branch to the default branch. However, in GitHub, there are 2 kinds of merge:
- fast forward merge: if there is no new commit in the default branch, we can directly merge the branch to the default branch, and update the sync point.
- normal merge: if there are new commits in the default branch, we need to create a merge commit to merge the branch to the default branch, and update the sync point.

Now we have the new commits in the default branch (called newSha in the code), we will try to merge the new commits to overleaf.

Since GitHub stored all the files in the repository, we can get the changed files between the last synced commit and the new commit, we only return changes files with URLs, then web service will download the changed files and update the overleaf project.

## What if there are conflicts?
If there are conflicts, we will not merge the branch to the default branch, and we will return error to the web service, then web service will get sync status and show the conflict to the user, and ask the user to resolve the conflict in GitHub.

If user merge that conflict branch to the default branch, then we will update the sync point and merge the changes to overleaf.

> How we detect if a branch is merged to the default branch?
>
> We will use GitHub API to diff the default branch and the `overleaf branch`, if the default falls behind the branch, it means the branch is not merged, if the default branch is ahead of the branch, it means the branch is merged.
>
> In a corner case, if the overleaf branch is merged but deleted, we will use the latest commit in the default branch as the new sync point, and merge the changes to overleaf.


## Copyright
Copyright (C) 2026 Ayaka-notes.
28 changes: 28 additions & 0 deletions services/github-sync/app.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// Metrics must be initialized before importing anything else
import '@overleaf/metrics/initialize.js'
import logger from '@overleaf/logger'
import Settings from '@overleaf/settings'
import { createServer } from './app/js/server.js'
import { mongoClient } from './app/js/mongodb.js'

const port = Settings.internal?.githubSync?.port
const host = Settings.internal?.githubSync?.host
mongoClient
.connect()
.then(() => {
logger.debug('Connected to MongoDB from GitHub Sync service')
})
.catch(err => {
logger.fatal({ err }, 'Cannot connect to mongo. Exiting.')
process.exit(1)
})

const { server } = createServer()
server.listen(port, host, err => {
if (err) {
logger.fatal({ err }, `Cannot bind to ${host}:${port}. Exiting.`)
process.exit(1)
}

logger.info({ host, port }, 'GitHub Sync service listening')
})
Loading