[package-spec] - Added support for '*.csv' and '*.gz' file patterns in terraform deployer spec

[ShourieG]

What does this PR do?

Adds support for '.csv' and '.gz' file patterns in terraform deployer spec.

Why is it important?

Unblocks integration PRs which utilise such patterns.

Checklist

• I have added test packages to test/packages that prove my change is effective.
• I have added an entry in spec/changelog.yml.

Related issues

• Closes [Change Proposal] - Add support for more file types in terraform deployer #925
• Relates [Netskope] Add Alerts v2 data stream integrations#14443

[mrodm]

This changelog entry can be moved to 3.4.1-next version section.

[mrodm]

Could it be added text/plain as content media type ?

Suggested change

	- description: Csv files to add into terraform resources
	type: file
	pattern: '^.*\.csv$'
	required: false
	- description: Csv files to add into terraform resources
	type: file
	contentMediaType: "text/plain"
	pattern: '^.*\.csv$'
	required: false

[ShourieG]

mime type for csv is text/csv, would that suit better here ?

[mrodm]

Yes, you're right that would suit better.

Currently, text/csv mime type is not supported in package-spec:

package-spec/code/go/internal/validator/content.go

Lines 16 to 34 in a50d2bc

	func validateContentType(fsys fs.FS, path string, contentType spectypes.ContentType) error {
	switch contentType.MediaType {
	case "application/x-yaml":
	v, _ := contentType.Params["require-document-dashes"]
	requireDashes := (v == "true")
	if requireDashes {
	err := validateYAMLDashes(fsys, path)
	if err != nil {
	return err
	}
	}
	case "application/json":
	case "text/markdown":
	case "text/plain":
	default:
	return fmt.Errorf("unsupported media type (%s)", contentType)
	}
	return nil
	}

One option is to add another case for text/csv into that function (as it is text/markdown or text/plain) for instance. And I guess it is not needed to add any specific validation for that media type.

[ShourieG]

@mrodm, made the suggested changes

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 0495776

History

• 💚 Build #1246 succeeded eec2451

cc @ShourieG

[moxarth-rathod]

@ShourieG another related PR elastic/integrations#14524.

[ShourieG]

@mrodm, I'm not getting the option to merge, can you help me to get it merged.

[mrodm]

@mrodm, I'm not getting the option to merge, can you help me to get it merged.

Sure, I'll merge it 👍