add a workflow to connect optional workflow inputs to non-optional tool inputs

[bernt-matthias]

Using this workflow as sub-workflow should simplify workflows. I think that having this might be cool irrespective of if we do galaxyproject/tools-iuc#5713 + galaxyproject/galaxy#17271 -- which do more or less the same .. but still its two tools (which have nothing to do with the actual WF vs a single subworkflow witha clearly described purpose). But if we implement them the dataset creation tool could be removed here (which reduces the overhead of 1 tool call).

Edit: not happy yet. Problem is that the tool input requires a datatype, but the output of pick_value sets none. Setting the datatype of a subworkflow is not possible .. which apparently renders the approach useless. Would there be an option to set the output datatype of pick_value in a way that is useable by workflows.

[mvdbeek]

We should be able to infer the output data type, that's probably not that hard to do, as we do that with other outputs as well.

[bernt-matthias]

You mean in the pick value tool? Question is how to set this? Can we do this in the ECMAS script? Is there an example / docs (I'm really missing docs on expression tools .. is there any)?

[mvdbeek]

Yes, format_source should just do it in https://github.com/galaxyproject/tools-iuc/blob/main/tools/pick_value/pick_value.xml#L149

[bernt-matthias]

Will try. But syntax and functionality-wise I'm not entirely sure how to address a dataset that is nested in two conditionals and a repeat (which might have some null datasets) :)

[bernt-matthias]

Syntax-wise style_cond|type_cond|pick_from_0|value will work. But this would take the format always from the first repeat element. For the workflow (editor) this might already be sufficient to determine the formats (even if the dataset is left empty in the first repeat), but I'm wondering what happens then during tool execution if the first dataset is given / not. I have the feeling that it might actually work .. but need a bit of time for testing.

[github-actions]

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	0
Error	0
Failure	2
Skipped	0

Failed Tests

• ❌ Treat-optional-workflow-input.ga_0

  Problems:

  • Output with path /tmp/tmpe1k7_vox/Pick parameter value on data 1__d5461b58-35b9-4a54-9781-478d08eaab15 different than expected
    Unable to find test function associated with XML tag has_n_line. Check your tool file syntax.
    

  • Expected output [output_param_boolean] not found in results.
    

  Workflow invocation details

  • Invocation Messages

  • Steps

    • Step 1: Optional input:

      • step_state: scheduled

    • Step 2: toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_text_file_with_recurring_lines/1.1.0:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is ok

          Command Line:

          • times=1; yes -- "" 2>/dev/null | head -n $times >> /tmp/tmp90ad9g1u/job_working_directory/000/1/outputs/dataset_d5461b58-35b9-4a54-9781-478d08eaab15.dat;

          Exit Code:

          • 0

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"data"
            __workflow_invocation_uuid__	"d6792deadc6711ee80e5639fce0e8f37"
            chromInfo	"/tmp/tmp90ad9g1u/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            token_set	[{"__index__": 0, "line": null, "repeat_select": {"__current_case__": 0, "repeat_select_opts": "user", "times": "1"}}]

    • Step 3: Check if optional input is given:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is ok

          Command Line:

          • cd ../; python _evaluate_expression_.py

          Exit Code:

          • 0

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"d6792deadc6711ee80e5639fce0e8f37"
            chromInfo	"/tmp/tmp90ad9g1u/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            input_param_type	{"__current_case__": 4, "input_param": null, "mappings": [{"__index__": 0, "from": null, "to": "False"}], "type": "data"}
            output_param_type	"boolean"
            unmapped	{"__current_case__": 2, "default_value": "True", "on_unmapped": "default"}

    • Step 4: Make optional input non-optional:

      • step_state: scheduled

      • Jobs

        • Job 1:

          • Job state is ok

          Command Line:

          • cd ../; python _evaluate_expression_.py

          Exit Code:

          • 0

          Job Parameters:

          • Job parameter	Parameter value
            __input_ext	"input"
            __workflow_invocation_uuid__	"d6792deadc6711ee80e5639fce0e8f37"
            chromInfo	"/tmp/tmp90ad9g1u/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
            dbkey	"?"
            style_cond	{"__current_case__": 0, "pick_style": "first", "type_cond": {"__current_case__": 4, "param_type": "data", "pick_from": [{"__index__": 0, "value": null}, {"__index__": 1, "value": {"values": [{"id": 1, "src": "hda"}]}}]}}

  • Other invocation details

    • history_id

      • 01f6b5c13128c50c

    • history_state

      • ok

    • invocation_id

      • 01f6b5c13128c50c

    • invocation_state

      • scheduled

    • workflow_id

      • 4f8ae09dd08217cd

❌ Treat-optional-workflow-input.ga_1

Problems:

• Expected file properties for output [output_param_boolean]
  

Workflow invocation details

• Invocation Messages

• Steps

  • Step 1: Optional input:

    • step_state: scheduled

  • Step 2: toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_text_file_with_recurring_lines/1.1.0:

    • step_state: scheduled

    • Jobs

      • Job 1:

        • Job state is ok

        Command Line:

        • times=1; yes -- "" 2>/dev/null | head -n $times >> /tmp/tmp90ad9g1u/job_working_directory/000/5/outputs/dataset_e833a6a5-927c-465f-85df-59ef1beed2d7.dat;

        Exit Code:

        • 0

        Job Parameters:

        • Job parameter	Parameter value
          __input_ext	"data"
          __workflow_invocation_uuid__	"f23289afdc6711ee80e5639fce0e8f37"
          chromInfo	"/tmp/tmp90ad9g1u/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
          dbkey	"?"
          token_set	[{"__index__": 0, "line": null, "repeat_select": {"__current_case__": 0, "repeat_select_opts": "user", "times": "1"}}]

  • Step 3: Check if optional input is given:

    • step_state: scheduled

    • Jobs

      • Job 1:

        • Job state is ok

        Command Line:

        • cd ../; python _evaluate_expression_.py

        Exit Code:

        • 0

        Job Parameters:

        • Job parameter	Parameter value
          __input_ext	"input"
          __workflow_invocation_uuid__	"f23289afdc6711ee80e5639fce0e8f37"
          chromInfo	"/tmp/tmp90ad9g1u/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
          dbkey	"?"
          input_param_type	{"__current_case__": 4, "input_param": {"values": [{"id": 4, "src": "hda"}]}, "mappings": [{"__index__": 0, "from": null, "to": "False"}], "type": "data"}
          output_param_type	"boolean"
          unmapped	{"__current_case__": 2, "default_value": "True", "on_unmapped": "default"}

  • Step 4: Make optional input non-optional:

    • step_state: scheduled

    • Jobs

      • Job 1:

        • Job state is ok

        Command Line:

        • cd ../; python _evaluate_expression_.py

        Exit Code:

        • 0

        Job Parameters:

        • Job parameter	Parameter value
          __input_ext	"input"
          __workflow_invocation_uuid__	"f23289afdc6711ee80e5639fce0e8f37"
          chromInfo	"/tmp/tmp90ad9g1u/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"
          dbkey	"?"
          style_cond	{"__current_case__": 0, "pick_style": "first", "type_cond": {"__current_case__": 4, "param_type": "data", "pick_from": [{"__index__": 0, "value": {"values": [{"id": 4, "src": "hda"}]}}, {"__index__": 1, "value": {"values": [{"id": 5, "src": "hda"}]}}]}}

• Other invocation details

  • history_id

    • 4f8ae09dd08217cd

  • history_state

    • ok

  • invocation_id

    • 4f8ae09dd08217cd

  • invocation_state

    • scheduled

  • workflow_id

    • 4f8ae09dd08217cd