Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Overthere Project Guidelines

## Architecture

Overthere is a Java library that abstracts remote file manipulation and process execution behind a protocol-agnostic API (think `java.io.File` and `java.lang.Process` as interfaces). Key layers:

- **Public API** (`src/main/java/com/xebialabs/overthere/`): `Overthere` (factory), `OverthereConnection`, `OverthereFile`, `OverthereProcess`, `ConnectionOptions`, `CmdLine`/`CmdLineArgument`
- **SPI** (`spi/`): `OverthereConnectionBuilder`, `BaseOverthereConnection`, `BaseOverthereFile`, `Protocol` annotation — extend these when adding a new protocol
- **Protocol implementations**: `local/`, `ssh/`, `cifs/`, `smb/`, `winrm/`, `winrs/`, `gcp/`, `telnet/`, `proxy/`
- **Utilities** (`util/`): `OverthereUtils`, `OverthereFileTransmitter`, `OverthereFileTranscoder`

**SPI plugin mechanism**: Connection builders are annotated with `@Protocol(name = "...")` and discovered at runtime via classpath scanning (scannit library). `OverthereConnector` instantiates the builder reflectively. See [SshConnectionBuilder.java](../src/main/java/com/xebialabs/overthere/ssh/SshConnectionBuilder.java) and [CifsConnectionBuilder.java](../src/main/java/com/xebialabs/overthere/cifs/CifsConnectionBuilder.java) as reference implementations.

## Build and Test

```bash
# Build (Windows)
gradlew clean build

# Build (Unix)
./gradlew clean build

# Run only unit tests (no cloud infra needed)
gradlew test

# Publish to Maven Central (requires PGP key + Central Publisher Portal tokens)
gradle clean build signArchives uploadArchives closeAndPromoteRepository
```

- **Java 21** required (`sourceCompatibility = JavaVersion.VERSION_21`)
- **Integration tests** (`itest/`) require live cloud hosts configured via [Overcast](https://github.com/xebialabs/overcast) (`src/test/resources/overcast.conf`)
- `LocalConnectionTest` is the only itest that runs without cloud infra

## Code Style

- **Testing framework**: TestNG (not JUnit) — use `org.testng.annotations.@Test`, `@BeforeMethod`, `@AfterMethod`
- **Conditional tests**: Use `@Assumption` from AssumEng (e.g., `@Assumption(methods = "onWindows")`) — see [ItestsBase6Windows.java](../src/test/java/com/xebialabs/overthere/itest/ItestsBase6Windows.java)
- **Assertions**: Hamcrest (`assertThat(..., is(...))` / `equalTo(...)`) preferred over TestNG `assert*`
- **Error handling**: Throw `RuntimeIOException` (not checked exceptions) for I/O failures in `OverthereFile`/`OverthereConnection` implementations
- All source files must carry the GPLv2 copyright header — use the template in [HEADER](../HEADER)

## Project Conventions

- **ConnectionOptions**: All protocol-specific options are declared as `public static final String` constants in the corresponding `*ConnectionBuilder`. Boolean/numeric defaults follow the `*_DEFAULT` naming convention (e.g., `CIFS_PORT_DEFAULT`).
- **Integration test hierarchy**: New connection itests extend the chain `ItestsBase1Utils` → `ItestsBase2Basics` → `ItestsBase3Copy` → `ItestsBase4Size` → `ItestsBase5Unix` → `ItestsBase6Windows`; override `getProtocol()`, `getOptions()`, and `getExpectedConnectionClassName()`
- **Examples**: Standalone Maven project under `examples/` demonstrating library usage; kept intentionally simple

## Integration Points

- **SSH**: [sshj](https://github.com/hierynomus/sshj) library (`com.hierynomus:sshj`)
- **CIFS/SMB**: jcifs (`jcifs:jcifs`) and smbj for SMB 2.x
- **GCP**: Google Cloud SDK — two flavours: OsLogin ([GcpOsLoginSshConnection.java](../examples/src/main/java/com/xebialabs/overthere/GcpOsLoginSshConnection.java)) and Metadata ([GcpMetaSshConnection.java](../examples/src/main/java/com/xebialabs/overthere/GcpMetaSshConnection.java))
- **WinRM**: Internal HTTP implementation (`winrm/`) and native `winrs` (`winrs/`)

## Security

- GCP credentials are configured via `GCP_CREDENTIALS_TYPE` option; supported types in `GcpCredentialsType` enum (ServiceAccountJsonFile, ServiceAccountToken, OsLogin, etc.) — test key files are in `src/test/resources/gcp/` and must **not** contain real credentials
- Kerberos config for WinRM domain auth lives in `src/test/resources/winrm/conf/krb5.conf` — treat as sensitive
190 changes: 190 additions & 0 deletions .github/skills/create-pr/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
---
name: create-pr
description: "Use this skill to create a pull request after dependency upgrades (or any successful, committed changes). Validates repo state, generates a structured PR description, requires explicit approval before pushing."
argument-hint: "create-pr [--base master]"
---

# create-pr Skill

Generic PR creation after successful changes. Works for any repo — not specific to dependency upgrades.

## Prerequisites

### GitHub CLI (`gh`) must be installed and authenticated

**Install GitHub CLI** (if not already installed):

```bash
# macOS (Homebrew)
brew install gh

# Linux (apt)
sudo apt install gh

# Windows — choose one:
winget install GitHub.cli # Windows 11 / winget (recommended)
choco install gh # Chocolatey
scoop install gh # Scoop
# or: download the *.msi installer from https://github.com/cli/cli/releases
```

**Authenticate:**
```bash
gh auth status # check current auth
gh auth login # if not authenticated
```

**Requirements before invoking this skill:**
- Changes must be committed (or staged for commit)
- Build must be green (`./gradlew clean build`)
- You must **not** be on `master` or `main` (or be prepared to create a new branch)

---

## Workflow

### Step 1 — Validate repo state

```bash
git status
git branch --show-current
git log master..HEAD --oneline
```

Check:
- Not on `master`/`main` → if so, go to Step 2 branch handling
- Has at least one commit ahead of base branch
- No unrelated staged changes (warn user if mix of changes detected)

### Step 2 — Branch handling

If already on a feature branch: proceed.

If on `master`/`main`:
> "You are on `master`. A new branch is needed. Suggest: `chore/deps-update-YYYY-MM-DD`. Use this name or provide your own?"

Create branch only after explicit confirmation:
```bash
git checkout -b chore/deps-update-2026-03-27
```

### Step 3 — Build validation

Check the [Jenkinsfile](../../Jenkinsfile) for the canonical build steps. For overthere:

```bash
# Standard build (always run)
./gradlew clean build
# Windows:
gradlew.bat clean build

# Integration tests (run if SSH/SMB/WinRM/CIFS deps changed)
# Requires Overcast cloud infra — skip if not available:
# ./gradlew itest
```

**STOP if the build fails.** Do not create a PR with a broken build.

### Step 4 — Collect change context

```bash
git diff --name-only master...HEAD
git diff master...HEAD
```

Also read (if available):
- `dependency-upgrade-report.md` — summary of applied upgrades
- `dependency-upgrade-report.json` — structured upgrade data

### Step 5 — Generate PR title

Conventional commit format:

```
chore(deps): upgrade {primary-libraries}
```

Examples:
- `chore(deps): upgrade slf4j 2.0.13→2.0.17, bouncycastle 1.80→1.83`
- `chore(deps): upgrade sshj 0.38→0.40, smbj 0.13→0.14`
- `chore(deps): upgrade 8 patch/minor dependencies`

### Step 6 — Generate PR description

Use this template (remove irrelevant lines; keep checkboxes **unchecked**):

```markdown
**Summary**
- Dependency upgrades applied via dependency-scan skill.
- {Briefly describe which libraries were upgraded and why (e.g. security patch, new features needed)}.

**What Changed**
- [ ] Dependency version bumps in `build.gradle`
- [ ] Build/configuration updates (if any)
- [ ] Tests/docs updates (if any)

**Validation**
- [ ] `./gradlew clean build` passes
- [ ] Integration tests considered (SSH/SMB/WinRM paths affected? If yes, run `./gradlew itest`)

**Risk Notes**
- [ ] Backward compatibility considered
- [ ] Security-category updates reviewed against changelogs
- [ ] Major version bumps (if any) reviewed for API changes

**Optional Dependency Notes** _(include only when relevant)_
- [ ] Reviewed `dependency-upgrade-report.md`
- [ ] Related versions kept in sync (e.g. bcprov + bcpkix, slf4j-api + jcl-over-slf4j)

**Checklist**
- [ ] Branch pushed to origin
- [ ] Build is green
- [ ] PR labels added as needed
- [ ] Ready for review
```

**CI labels to consider** (add with `--label`):
- `ci-skip-integration-tests` — when only build plugin or test-scope deps changed
- `ci-run-platform-build` — when runtime deps changed and full verification is needed
- `ci-skip-sonar-analysis` — if sonar quota is low and changes are mechanical

### Step 7 — Show to user and ask for approval

Display the generated title and body in full, then:
> "Create PR with the above title and description? (yes / edit / no)"

If `edit`: open editor or accept freeform corrections before proceeding.

### Step 8 — Push and create PR

```bash
git push --set-upstream origin <branch-name>

gh pr create \
--base master \
--title "<TITLE>" \
--body "<BODY>"

# With labels:
gh pr create \
--base master \
--title "<TITLE>" \
--body "<BODY>" \
--label "ci-skip-integration-tests"
```

### Step 9 — Return PR URL

Report back:
> "PR created: https://github.com/xebialabs/overthere/pull/NNN"

---

## Guardrails

- ❌ Never create PR without explicit user `yes`
- ❌ Never skip build validation
- ❌ Never run destructive git commands (`--force`, `reset --hard`, `push --force`)
- ❌ Never invent versions or change summaries — use actual `git diff` and report data
- ✅ Always check `gh auth status` before attempting PR creation
- ✅ Always warn if there are uncommitted changes before pushing
83 changes: 83 additions & 0 deletions .github/skills/dependency-scan/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
name: dependency-scan
description: "Use this skill to scan overthere's dependencies for available updates. Runs 3 phases: xl-platform BOM scan (optional), repo hardcoded dep scan, and alignment check."
argument-hint: "dependency-scan [--phase all|xl-platform-scan|scan|alignment] [--xl-platform-dir ../xl-platform]"
---

# dependency-scan Skill — overthere

Orchestrates a full dependency scan across all three phases and produces upgrade reports.

## Phases

### Phase 1 — xl-platform scan (delegated to `xl-platform-scan` skill)

Scans the xl-platform BOM for upstream dependency updates. See [`xl-platform-scan` skill](../xl-platform-scan/SKILL.md) for full details.

> **Note**: `overthere` does not consume the xl-platform BOM directly. Run Phase 1 only if you are also maintaining `xl-platform` and want to propagate upstream updates.

Parses:
- `xl-reference/xl-reference.conf` (~100 unique vars)
- `xl-jakartaee-bom/xl-jakartaee-bom.conf` (~30 vars)
- xl-platform's `gradle.properties` (~5 build plugin vars)
- **~135 total unique version variables**

Resolves via `XL_PLATFORM_COORDINATES` (114 entries) + `SKIP_VARIABLES` (24 entries); dynamic inference (46 patterns) handles unmapped vars.

### Phase 2 — repo scan (overthere)

Scans `overthere`'s own dependency files for hardcoded versions not managed by any BOM.

| Source file | What it contains |
|---|---|
| `build.gradle` (buildscript block) | Gradle build plugin versions: nexus-staging, nebula-release, gradle-pom, grgit, license |
| `build.gradle` (dependencies block) | All library versions: sshj, smbj, jcifs, bouncycastle, httpcomponents, dom4j, jaxen, grpc, GCP, testng, mockito, guava, logback, etc. |
| `gradle.properties` | JVM args only — no library versions defined here |

Uses:
- `REPO_COORDINATES` (9 entries) — this repo's Gradle plugin → Maven coordinate mappings
- `CODE_ANALYSIS_COORDINATES` (0 entries) — no code-analysis tool config in overthere

### Phase 3 — alignment check

Verifies that any version overrides in `build.gradle` are consistent with each other (e.g. Bouncy Castle `bcprov` and `bcpkix` should match, `httpclient` and `httpcore` should be compatible). Flags mismatches.

## Interactive workflow

1. Ask user: "Scan xl-platform too? (yes/no)" (defaults to **no** for standalone repos)
2. Run Phase 2 (always) ± Phase 1 (if yes)
3. Run Phase 3 (alignment check)
4. Output reports
5. Offer handoff: "Run dependency-upgrade to apply selected upgrades?"

## Run command

```bash
# All phases (including xl-platform scan)
python3 .github/skills/dependency-scan/scripts/scan-dependencies.py \
--phase all \
--xl-platform-dir ../xl-platform

# Repo scan + alignment only (no xl-platform)
python3 .github/skills/dependency-scan/scripts/scan-dependencies.py \
--phase scan

# Alignment check only
python3 .github/skills/dependency-scan/scripts/scan-dependencies.py \
--phase alignment
```

## Output

Both files are written (or overwritten) in the repo root:

| File | Purpose |
|---|---|
| `dependency-upgrade-report.md` | Human-readable grouped upgrade table |
| `dependency-upgrade-report.json` | Machine-readable; consumed by `execute-upgrade.py` |

## Configuration & credentials

- **Maven Central**: no credentials required
- **Nexus**: optional; set `NEXUS_USER` + `NEXUS_PASSWORD` env vars, or `nexusUserName`/`nexusPassword` in `~/.gradle/gradle.properties`
- Nexus base URL: `https://nexus.xebialabs.com/nexus/content` (searches `releases`, `central`, `public`)
Loading