Using globstar in glob pattern prevents dot (hidden) files from being matched

[mojavelinux]

Version

v22.5.0

Platform

Linux razerbook-og 6.11.7-100.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov  8 19:07:28 UTC 2024 x86_64 GNU/Linux

Subsystem

fs

What steps will reproduce the bug?

Create the following files to scan:

scan/
  subdir/
    a.js
    .b.js

Create the following script:

'use strict'

const fsp = require('node:fs/promises')

;(async () => {
  const globs = [
    'scan/subdir/{.,}*.js',
    'scan/*/{.,}*.js',
    'scan/**/{.,}*.js',
    'subdir/{.,}*.js',
    '*/{.,}*.js',
    '**/{.,}*.js',
  ]
  for (const glob of globs) {
    console.log('glob: ' + glob)
    if (glob.startsWith('subdir/')) process.chdir('scan')
    for await (const entry of fsp.glob(glob)) {
      console.dir(entry)
    }
  }
})()

Whenever ** is used in the pattern, the dot (hidden) files are not matched, even though the pattern specifically requests them.

How often does it reproduce? Is there a required condition?

Always reproducible as long as ** is used in the pattern and the pattern is matching dot (hidden) files.

If the pattern is * or an explicit directory name, then the dot (hidden) files are matched. So this is specific to using globstar (**).

What is the expected behavior? Why is that the expected behavior?

The dot (hidden) files should be matched after the globstar because the pattern is specifically requesting them.

What do you see instead?

When globstar is used in the pattern, the dot (hidden) files are not matched.

Additional information

The described scenario works as expected using node-glob and bash, so the behavior of the built-in function in Node.js is inconsistent.

[deedweird]

I've run into the same bug, Node 22.13.1

Always reproducible as long as ** is used in the pattern and the pattern is matching dot (hidden) files.

It looks like the bug occurs when the pattern specifically begins with **. For example, pattern ./**/.test does find the .test files everywhere except CWD.

A possible workaround is to use 2 patterns: one searches in CWD, another one searches for dotfiles in all directories below CWD:
['.test', './**/.test']

[mojavelinux]

According to my tests, that does not solve the problem. Instead, I have to use a single glob for each depth.

['packages/*/test/fixtures/*/.*.js', 'packages/*/test/fixtures/*/*/.*.js', 'packages/*/test/fixtures/*/*/*/.*.js']

...which defeats the whole purpose of globstar.

[dario-piotrowicz]

Hi there 👋

I've had a look into this and fixing it seems to be pretty simple, these changes should do the trick: dario-piotrowicz@2499a0c

However you can see from my code changes that the current behavior for dot files in globstar matches looks pretty intentional, so maybe they should actualy not be included?

More specifically the question on my mind is, should dot files be included in the first place in glob searches or not?

Currently dot files in normal matches are included but those in globstar matches are not, I think this is very inconsistent and incorrect, but which one is the correct/desired implementation?

In the glob package such files are not included in either type of match, but there is a dot option that enables them, I am guessing that matching this behavior in the nodejs glob implementation would be the optimal solution?

The main problem I think would be that that would cause a breaking change, since dot files in normal matches would disappear unless users specify the dot option 😕

So to summarize the above:

• dot files in globstar matches can be easily added (and I don't think this would be considered a breaking change?), but is that the desired behavior?
• dot files in normal matches could be removed, optionally with the possibility to include them (alongside the globstar ones) via an opt-in option, but this would be a breaking change

If someone from the nodejs team can give some guidance here on what the nodejs implementation should work like I'd be happy to help out and implement a solution for this 🙂

@MoLow maybe you could help here? 🙂 (I'm pinging you based on #47653)

[ShogunPanda]

I also incurred into this bug, using matchesGlob.
I would probably add the dot option to match the expectations of people migrating from the glob package.