fix(macOS): spawn readFilesCommand via shell (posix_spawnp) so gzipped input works#2693
Open
BenjaminDEMAILLE wants to merge 1 commit into
Open
Conversation
…obin#2663) On macOS, alignment with --readFilesCommand (gzcat, gunzip -c, zcat ...) failed with 'Failed spawning readFilesCommand / No such file or directory', so gzipped input could not be read (works on Linux, and reverting to 2.7.10b worked). Two root causes: - The generated readsCommand script gets a #! shebang only when --sysShell is set; by default (sysShell '-') it has none. The old vfork()+execlp() path happened to work because execvp falls back to /bin/sh on ENOEXEC, but posix_spawn (used by the bioconda build) has no such fallback, so exec failed. - vfork()+exit(0) is undefined behavior: exit() in a vfork child flushes the parent's stdio and runs atexit handlers in the shared address space, and it reported success even when exec failed. Spawn the script explicitly as '<shell> <script>' via posix_spawnp, using --sysShell when set else /bin/sh. No shebang or ENOEXEC fallback is needed, and posix_spawnp is the safe, portable process-creation primitive on Linux and macOS. Verified on native arm64 with plain FASTQ, --readFilesCommand 'gunzip -c', and --readFilesCommand gzcat: 5/5 reads in all cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BenjaminDEMAILLE
marked this pull request as ready for review
June 26, 2026 12:56
This was referenced Jun 26, 2026
birdingman0626
added a commit
to birdingman0626/STAR-Cross
that referenced
this pull request
Jun 29, 2026
…ndian support (#7) * fix(macos): spawn readFilesCommand via posix_spawnp The POSIX path used vfork()+execlp()+exit(0) to run the generated readsCommand script. On macOS this fails ("Failed spawning readFilesCommand", upstream issue alexdobin#2663) because the script has no shebang and posix_spawn-style exec has no ENOEXEC->/bin/sh fallback; calling exit() in a vfork child is also undefined behavior. Replace with posix_spawnp invoking "<shell> <script>" explicitly, with proper error checking. POSIX-only branch; the Windows system()-based path is unchanged. Ported from upstream PR alexdobin#2693. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: big-endian support with guarded little-endian fast path STAR read the genome, suffix array and packed arrays through native uint*/uint128* casts, assuming little-endian byte order, which caused "next index is smaller than previous" failures on big-endian hosts (s390x, ppc64). Add byteOrder.h with loadUintLE/storeUintLE helpers and route the four endian-sensitive sites (PackedArray::operator[]/writePacked, funCompareSuffixes, funCalcSAiFromSA) through them. The helpers are guarded by STAR_BIG_ENDIAN: little-endian builds keep the original single-instruction native load (zero performance/behavior change), and only known big-endian compiles take the portable byte-wise path. The existing MSVC byte-access path in funCalcSAiFromSA is reused for BE. The byte-wise implementation is exposed by name and unit-tested (test_byteOrder.cpp) so the BE algorithm is validated on the LE CI runner. Ported from the patch in upstream issue alexdobin#2690. Note: compile-validated only; CI has no big-endian runner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: referenceless CRAM output (--outSAMtype CRAM) Add CRAM as an output container. STAR produces its BAM via the proven output path, then transcodes it to referenceless CRAM at finalization (cramOutput.cpp, using bundled HTSlib with CRAM_OPT_NO_REF, so no external reference FASTA is required). On conversion failure the original BAM is kept and the run continues. - Parameters_runtimeSetup: accept "CRAM" as outSAMtype[0], reusing the BAM Unsorted/SortedByCoordinate logic; set outCRAMbool and CRAM names. - STAR.cpp: transcode Aligned.out.bam / Aligned.sortedByCoord.out.bam to .cram after wiggle output, removing the intermediate BAM. - Hot per-record output code is untouched; the transcriptome BAM stays BAM for RSEM/Salmon. CRAM is incompatible with --outStd. - WebUI: CRAM options in the outSAMtype dropdown, .cram artifacts and MIME type. - Registered cramOutput in CMake build. Implemented properly rather than porting the incomplete upstream draft PR alexdobin#2670 (which wrote a NULL header and rerouted the working BAM path). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * build: Makefile ARM64/macOS/libomp parity and cramOutput Port the platform-detection from upstream PR alexdobin#2694 to the Makefile (CI uses CMake, which already handles ARM64; this is for `make` users): - detect UNAME_S/UNAME_M; select -mavx2 on x86_64, -march=armv8-a+simd on ARM64/AArch64 - macOS + Apple clang OpenMP: -Xpreprocessor -fopenmp + libomp include, -lomp at link; add native STARforMac target (dynamic libomp) - portable BSD/macOS date fallback for the build timestamp - add cramOutput.o to OBJECTS Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: document CRAM output, ARM64/macOS, and big-endian support Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On macOS, alignment with
--readFilesCommand(gzcat,gunzip -c,zcat, ...) fails with:so gzipped input cannot be read (see #2663). Linux is fine; reverting to 2.7.10b works; plain (uncompressed) FASTQ works.
Root cause
The generated
readsCommandscript gets a#!shebang only when--sysShellis set; by default (sysShell==-) it has none. The oldvfork()+execlp()path worked becauseexecvpfalls back to/bin/shonENOEXEC, butposix_spawn(used by the conda build) has no such fallback, so exec of the shebang-less script fails.Separately,
vfork()+exit(0)is undefined behavior:exit()in a vfork child flushes the parent's stdio and runs atexit handlers in the shared address space, and it reported success even when exec failed.Fix
Spawn the script explicitly as
<shell> <script>viaposix_spawnp, using--sysShellwhen set, else/bin/sh. No shebang orENOEXECfallback is needed, andposix_spawnpis the safe, portable process-creation primitive on Linux and macOS.Verification
Native arm64 build, with plain FASTQ,
--readFilesCommand 'gunzip -c', and--readFilesCommand gzcat: 5/5 reads in all cases.Closes #2663.
🤖 Generated with Claude Code