feat(python): support for pypi parse syntax and string concatenation

[neskk]

🤔 What's changed?

Support the pypi-parse step definition matcher used by behave and pytest-bdd - both python frameworks.

• Added support for string concatenation both for plain and pypi-parse syntaxes.
• Added tree-sitter query to match pypi parse syntax:
  • @step(parsers.parse("step-def"))
  • @step(parse("step-def"))
  • @step(parse("step-" + "d" + "ef"))
  • @step(parse("step-def {date:valid_date}", custom_types))

⚡️ What's your motivation?

I'm currently using pytest-bdd for running cucumber tests, but because I'm using PyPi Parse for step-definitions, the cucumber-plugin won't recognize them.

Supersedes #206.
Closes #205.

🏷️ What kind of change is this?

• ⚡ New feature (non-breaking change which adds new behaviour)

♻️ Anything particular you want feedback on?

I haven't been able to run the cucumber vscode extension, so I couldn't test it "in action".

I have modified the unit-test step-definitions to include at least one test for each different syntax.

📋 Checklist:

• I agree to respect and uphold the Cucumber Community Code of Conduct
• I've changed the behaviour of the code
  • I have added/updated tests to cover my changes.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly.
• Users should know about my change
  • I have added an entry to the "Unreleased" section of the CHANGELOG, linking to this pull request.

[neskk]

@kieran-ryan The step-def syntax works but now the parameter type queries identify named parameters, e.g.: @when(parse("make a request to {url}")) as a parameter type.

[neskk]

@kieran-ryan any ideas on how can we differentiate expressions coming from "parse" definitions from standard cucumber expressions?

# cucumber
@step("a cucumber expressions {custom_param_type}")
# pytest-bdd
@step(parsers.parse("a pypi {param_name_not_a_type}"))
@step(parsers.parse("a pypi {param_name:param_type}"))

I don't know how we can support the named parameters in step-defs that use the pypi parse, at the same time we support custom param types.
These two approaches effectively use the same syntax with different meaning:

• in pypi parse {xxx} is a named parameter
• in cucumber expressions {xxx} is a parameter type
  Because of this, the step-defs with "parse" syntax are discovered and indexed, but they throw errors for not defining the parameter types.

[neskk]

I realized that Pytest-BDD 8.0.0+ adds support to the official gherkin-parser.
Update: I thought this new pytest-bdd version would support cucumber-expressions, but it still doesn't.

This MR still has a problem with Parameter Types. Even though the step-defs are picked up by the language-service, it fails to parse parameters on step-definitions. E.g.:

@when(parse("bla bla bla {arg1}"))
def bla_bla_bla(arg1: str):
  // do something with arg1

This fails with something like: ParameterType "arg1" not found.

[Jomik]

This would be nice to get. For me the variables in the step definitions are too powerful to not have, and the LSP erroring like this is quite annoying :/
Just set up for pytest-bdd and ran into this 🙈

[neskk]

Hey @Jomik,
Unfortunately @kieran-ryan is MIA, other developers like @luke-hill seem too busy to even update vscode-extension with fixes done by users.
The official Cucumber/Gherkin support for VSCode is completely broken for at least a year now.
I tried to collaborate and take initiative to support pytest-bdd on the language-service, but realized there are a lot of things that need to change before that happens. I received very little support during my efforts, and it seems that the project's maintainers don't really have time to spend here.

1. The tree-sitter syntax I adjusted, supports the "parsers.parse()" syntax of pytest-bdd, but then we have absolutely no support to use named parameters. Even though the language-service recognizes the text of the step-definition, once you use a single named variable, vscode-extension LSP breaks and is no longer able to match step-definitions.
2. Adopting named groups in Cucumber-expressions seem to be the next requirement needed, but again, development seems completely stalled.
3. Support for python custom-types on top of named groups found in step-def parameters is also something that needs some work.

[Jomik]

That's really sad. Have you found a solution for yourself, or just given up on using the LSP?

[neskk]

Currently the only IDE/extension that seems to work is from IntelliJ PyCharm. It picks up step-defs even with named groups. Don't ask me how they are able to do it, I guess paying developers actually helps 😅
Me and other colleagues working with Pytest-BDD simply gave up on the official VSCode Cucumber extension/LSP.

[Jomik]

I am using neovim and wanted the LSP integration there, but I suppose I'll probably just disable the language server again :(
I was really happy when I saw this, damn.

[luke-hill]

@neskk I'm not a python developer and as you rightly say I have other things I'm liaising with.

My job (If you can call it that), is to ensure across the board that things are moving. I have tried recently to ensure that fixes are going into this plugin and that there is "some" limit movement. I apologise if that's not meeting your expectations but cucumber is ran by volunteers, all of whom have other full-time jobs.

I am at the moment trying to ensure each flavour of cucumber and all polyglot items are moving along. This item sits in a unique space in that it's not a part of the cucumber ecosystem but an add-on. So it inevitably is a little bit more fiddly.

Hopefully that addresses some of your gripes

[neskk]

Hey @luke-hill, I don't think I said any lies about the current state of the LSP and vscode extension. It seems like no progress has been done for about a whole year by the organizers of the Cucumber project to actually fix the extension for one of the main IDE used. To me this is very frustrating, it's also something that turns people away from using Cucumber with Python, and reveals lack of priorities/reliability. If you have strangers that volunteer wanting to fix things, and then their fixes don't even get merged for months, even volunteers lose interest. That's my 2 cents, wish you the best and great work.

…

On Tue, Apr 15, 2025, 08:48 Luke Hill ***@***.***> wrote: @neskk <https://github.com/neskk> I'm not a python developer and as you rightly say I have other things I'm liaising with. My job (If you can call it that), is to ensure across the board that things are moving. I have tried recently to ensure that fixes are going into this plugin and that there is "some" limit movement. I apologise if that's not meeting your expectations but cucumber is ran by volunteers, all of whom have other full-time jobs. I am at the moment trying to ensure each flavour of cucumber and all polyglot items are moving along. This item sits in a unique space in that it's not a part of the cucumber ecosystem but an add-on. So it inevitably is a little bit more fiddly. Hopefully that addresses some of your gripes — Reply to this email directly, view it on GitHub <#236 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPWGPH4YCRD25KSVVWCKML2ZS2WJAVCNFSM6AAAAABVGE7UXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMBUGE2DEOJVGU> . You are receiving this because you were mentioned.Message ID: ***@***.***> *luke-hill* left a comment (cucumber/language-service#236) <#236 (comment)> @neskk <https://github.com/neskk> I'm not a python developer and as you rightly say I have other things I'm liaising with. My job (If you can call it that), is to ensure across the board that things are moving. I have tried recently to ensure that fixes are going into this plugin and that there is "some" limit movement. I apologise if that's not meeting your expectations but cucumber is ran by volunteers, all of whom have other full-time jobs. I am at the moment trying to ensure each flavour of cucumber and all polyglot items are moving along. This item sits in a unique space in that it's not a part of the cucumber ecosystem but an add-on. So it inevitably is a little bit more fiddly. Hopefully that addresses some of your gripes — Reply to this email directly, view it on GitHub <#236 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPWGPH4YCRD25KSVVWCKML2ZS2WJAVCNFSM6AAAAABVGE7UXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQMBUGE2DEOJVGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[luke-hill]

@neskk if you wish to contribute more here and are happy to review and merge items then I'm more than happy for you to do that.

Cucumber has always been a collaborative effort. So often in Open source people contribute to the things they want to see progress.

If you're able to review and merge items for this plugin, then to me the only issue would then be releasing it to the store. Does this seem like a fair next step to try make things progress?

[neskk]

Yes, let's try to get things moving and I'm available to help.