Mapepire as SQL runner #2940

worksofliam · 2025-10-26T21:01:12Z

Changes

Rips out the legacy SQL runner with Mapepire, which is bundled in the VSIX and installed as a component.

ILE command improvements

runCommand has a lot of new logic. We now we should do all compiles with SQL. This means no more system, no more standard in or standard out from system. Due to that, we no longer have access to the job log or spool file generated by the executed command.

runCommand has four steps now:

Set the library list (using qcmdexc - if required)
Execute the command (using qcmdexc)
Fetch the latest job log entries job log (using qsys2.joblog_info('*'))
- I am not sure if this logic is sound proof, since we are depending on ORDINAL_POSITION and I believe there are some instances where it is not updated correctly from prior commands. The only way to get around that is to always fetch the job log even if it's not needed (skipDetail)
Fetch the spool file (using a complicated statement)

Due to this implementation, runCommand cannot really run asynchronously like system allowed. At the runCommand level, I have had to implement a queue system so that these sequence of events can only happen one at a time. This ensures the right job log and spool file is returned for the correct command.

To do

Update build to run npm run dsc as part of VSIX build
Ensure all test cases are passing under Mapepire
Test different CCSID scenarios
Remove/replace all liblist usage
Ensure that the component is valid with the existing install from the database extension

How to test this PR

Ensure the test cases pass

Checklist

have tested my change
have created one or more test cases
updated relevant documentation
Remove any/all console.logs I added
have added myself to the contributors' list in CONTRIBUTING.md

Signed-off-by: worksofliam <[email protected]>

github-actions · 2025-10-26T21:02:15Z

👋 A new build is available for this PR based on f07c9bc.

Signed-off-by: worksofliam <[email protected]>

This reverts commit 53f1b80.

julesyan · 2025-12-04T20:33:09Z

All test cases now pass

chrjorgensen

@worksofliam Before this PR we could get more than one spoolfile from one command in an action. One example is found in issue #3004.

Will this PR support that?

We don't know how many spoolfiles are created by one command run, and we may need them all. User created build commands may run more than one CRT-command and thus create more than one spoolfile.

chrjorgensen · 2025-12-12T23:28:49Z

src/api/CompileTools.ts

  }
 }
+
+const LAST_SPOOL_STATEMENT = [


@worksofliam This SQL statement will get the latest spoolfile created by the user - but not necessarily created in this job! If the user has other jobs running, also creating spoolfiles, this SQL may pick one of those instead of the intended one...

worksofliam · 2025-12-13T05:53:26Z

@chrjorgensen it is likely we'll need to track previously obtains spools like we do job log messages. It's not something we do in this at the moment, but we can likely do it. It currently just fetches the latest spoil after the command has executed.

worksofliam

Is this a big issue?

worksofliam · 2025-12-13T05:55:49Z

src/api/IBMiContent.ts

          await this.ibmi.runSQL([
            `@QSYS/CPYF FROMFILE(${library}/${sourceFile}) FROMMBR(${member}) TOFILE(QTEMP/QTEMPSRC) TOMBR(TEMPMEMBER) MBROPT(*REPLACE) CRTFILE(*YES);`,
-            `@QSYS/CPYFRMSTMF FROMSTMF('${tempRmt}') TOMBR('${Tools.qualifyPath("QTEMP", "QTEMPSRC", "TEMPMEMBER", undefined)}') MBROPT(*REPLACE) STMFCCSID(1208) DBFCCSID(${this.config.sourceFileCCSID})`,
+            `@QSYS/CPYFRMSTMF FROMSTMF('${tempRmt}') TOMBR('${Tools.qualifyPath("QTEMP", "QTEMPSRC", "TEMPMEMBER", undefined)}') MBROPT(*REPLACE)`,


Why was streamfile CCSID removed?

worksofliam · 2025-12-13T05:56:06Z

src/api/IBMiContent.ts

      }
      else {
        copyResult = await this.ibmi.runCommand({
-          command: `QSYS/CPYFRMSTMF FROMSTMF('${tempRmt}') TOMBR('${path}') MBROPT(*REPLACE) STMFCCSID(1208) DBFCCSID(${this.config.sourceFileCCSID})`,


Same here, why was streamfile CCSID removed?

worksofliam added 10 commits October 21, 2025 11:13

Initial work at adding mapepire

881cbdb

Signed-off-by: worksofliam <[email protected]>

Working connection

46ddb55

Signed-off-by: worksofliam <[email protected]>

Get job log from run command

6b5ef58

Signed-off-by: worksofliam <[email protected]>

Remove all for bit data

eb14250

Signed-off-by: worksofliam <[email protected]>

Always check mapepire first

fac70fd

Signed-off-by: worksofliam <[email protected]>

Fix mapepire running in tests

897d146

Signed-off-by: worksofliam <[email protected]>

Queue system

db3c41d

Signed-off-by: worksofliam <[email protected]>

Fixes for test cases

8e210a4

Signed-off-by: worksofliam <[email protected]>

Performance gains in startup

a5ea4da

Signed-off-by: worksofliam <[email protected]>

Add comments

b41aa76

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 26, 2025 21:01 — with GitHub Actions Failure

worksofliam changed the title ~~Feature/mapepire_internals~~ Mapepire as SQL runner Oct 26, 2025

Clean up some CCSID logic

e08cd5f

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 26, 2025 21:16 — with GitHub Actions Failure

Add server file to build

427ed62

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 26, 2025 21:21 — with GitHub Actions Failure

worksofliam requested review from a team, SanjulaGanepola and sebjulliand October 26, 2025 21:22

Fix issues with test cases and row count returned

8eef92b

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 26, 2025 22:13 — with GitHub Actions Failure

Re-add fake binding support to work around binding parameter issue

eeab0b0

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 28, 2025 17:59 — with GitHub Actions Failure

Speed up test cases a tiny bit

491e4ee

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 28, 2025 22:33 — with GitHub Actions Failure

worksofliam had a problem deploying to testing_environment October 29, 2025 03:56 — with GitHub Actions Failure

Simple log

6f4716f

Signed-off-by: worksofliam <[email protected]>

worksofliam had a problem deploying to testing_environment October 29, 2025 04:06 — with GitHub Actions Failure

Revert back

de730ce

Signed-off-by: worksofliam <[email protected]>

worksofliam and others added 22 commits December 1, 2025 22:19

Always check mapepire first

776f445

Signed-off-by: worksofliam <[email protected]>

Fix mapepire running in tests

9d1cf7c

Signed-off-by: worksofliam <[email protected]>

Queue system

7985345

Signed-off-by: worksofliam <[email protected]>

Fixes for test cases

63dfba6

Signed-off-by: worksofliam <[email protected]>

Performance gains in startup

f9598b0

Signed-off-by: worksofliam <[email protected]>

Add comments

708cddd

Signed-off-by: worksofliam <[email protected]>

Clean up some CCSID logic

8995099

Signed-off-by: worksofliam <[email protected]>

Add server file to build

c004dfe

Signed-off-by: worksofliam <[email protected]>

Fix issues with test cases and row count returned

dcbe25e

Signed-off-by: worksofliam <[email protected]>

Re-add fake binding support to work around binding parameter issue

b1ea99a

Signed-off-by: worksofliam <[email protected]>

Speed up test cases a tiny bit

39250c2

Signed-off-by: worksofliam <[email protected]>

Simple log

c8b1e3e

Signed-off-by: worksofliam <[email protected]>

Revert back

9659967

Signed-off-by: worksofliam <[email protected]>

Fix broken module test

096bc7a

Signed-off-by: worksofliam <[email protected]>

Fix to encoding tests

1a8a20b

Signed-off-by: worksofliam <[email protected]>

Update to allow mapepire reinstall

d1920cd

Fix issues during merge

5b3dadf

Fix tests that are now invalid

53f1b80

Revert "Fix tests that are now invalid"

7d6670c

This reverts commit 53f1b80.

Fix two test cases for new mapepire changes

15a8735

Fix tests to stop manually controlling CCSID

5ceac8f

Merge branch 'mapepire-dependency' into feature/mapepire_internals

f07c9bc

julesyan temporarily deployed to testing_environment December 4, 2025 20:09 — with GitHub Actions Inactive

sebjulliand mentioned this pull request Dec 5, 2025

Error: (SSH) Channel open failure: open failed #2998

Open

julesyan marked this pull request as ready for review December 12, 2025 12:14

chrjorgensen added this to the 3.0.0 milestone Dec 12, 2025

chrjorgensen requested changes Dec 12, 2025

View reviewed changes

worksofliam commented Dec 13, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Mapepire as SQL runner #2940

Mapepire as SQL runner #2940

Uh oh!

worksofliam commented Oct 26, 2025 •

edited by julesyan

Loading

Uh oh!

github-actions bot commented Oct 26, 2025 •

edited

Loading

Uh oh!

julesyan commented Dec 4, 2025

Uh oh!

chrjorgensen left a comment

Uh oh!

chrjorgensen Dec 12, 2025

Uh oh!

worksofliam commented Dec 13, 2025

Uh oh!

worksofliam left a comment

Uh oh!

worksofliam Dec 13, 2025

Uh oh!

worksofliam Dec 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Mapepire as SQL runner #2940

Are you sure you want to change the base?

Mapepire as SQL runner #2940

Uh oh!

Conversation

worksofliam commented Oct 26, 2025 • edited by julesyan Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

ILE command improvements

To do

How to test this PR

Checklist

Uh oh!

github-actions bot commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

julesyan commented Dec 4, 2025

Uh oh!

chrjorgensen left a comment

Choose a reason for hiding this comment

Uh oh!

chrjorgensen Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

worksofliam commented Dec 13, 2025

Uh oh!

worksofliam left a comment

Choose a reason for hiding this comment

Uh oh!

worksofliam Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

worksofliam Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

worksofliam commented Oct 26, 2025 •

edited by julesyan

Loading

github-actions bot commented Oct 26, 2025 •

edited

Loading