SigmaExpansion in combination with ConvertTypeTransformation results in incorrect queries

[marcelkwaschny]

Hey!

I am using the ConvertTypeTransformation to convert fields to strings in the netwitness backend because netwitness heavily relies on the correct data type for values in fields. I noticed issues when a field uses a SigmaExpansion data type (for example when using the windash modifier).

So when using a rule with a windash modifier:

title: Test
status: test
logsource:
    product: windows
    category: process_creation
detection:
    sel:
        param|windash|contains:
          - '-f'
    condition: sel

and a processing item in a pipeline like this:

ProcessingItem(
    identifier="netwitness_transform_fields_to_string",
    transformation=CustomConvertTypeTransformation(target_type="str"),
    field_name_conditions=[IncludeFieldCondition(fields=["param"])],
)

A conversion will result in a rule that looks like this:

param = "SigmaExpansion(values=[SigmaString((<SpecialChars.WILDCARD_MULTI: 1>, \'-f\', <SpecialChars.WILDCARD_MULTI: 1>)), SigmaString((<SpecialChars.WILDCARD_MULTI: 1>, \'/f\', <SpecialChars.WILDCARD_MULTI: 1>)), SigmaString((<SpecialChars.WILDCARD_MULTI: 1>, \'–f\', <SpecialChars.WILDCARD_MULTI: 1>)), SigmaString((<SpecialChars.WILDCARD_MULTI: 1>, \'—f\', <SpecialChars.WILDCARD_MULTI: 1>)), SigmaString((<SpecialChars.WILDCARD_MULTI: 1>, \'―f\', <SpecialChars.WILDCARD_MULTI: 1>))])"

A simple fix that I figured out is to update the ConvertTypeTransformation to take care of the SigmaExpansion type like this:

@dataclass
class CustomConvertTypeTransformation(ValueTransformation):
    """
    Convert type of value. The conversion into strings and numbers is currently supported.
    """

    target_type: Literal["str", "num"]

    def apply_value(self, field: str, val: SigmaType) -> Optional[Union[SigmaString, SigmaNumber, SigmaExpansion]]:
        if self.target_type == "str":
            if isinstance(val, SigmaExpansion):
                for entry in val.values:
                    entry = SigmaString(str(entry))
                return val
            return SigmaString(str(val))
        elif self.target_type == "num":
            try:
                if isinstance(val, SigmaExpansion):
                    for entry in val.values:
                        entry = SigmaNumber(str(entry))
                    return val
                return SigmaNumber(str(val))
            except SigmaValueError:
                raise SigmaValueError(f"Value '{val}' can't be converted to number for {str(self)}")

Is this intended behavior? And is the update of the ConvertTypeTransformation implementation the right way or am I missing something?

Let me know what you think 😊. If this fix is okay for you I would be happy to open a PR.

[thomaspatzke]

Hi! Thanks for the proposed fix. I'm unsure if it really works because the calculated result of this iteration that is stored in entry is not really used by the following code. Instead the SigmaExpansion contained in val is simply returned.

[marcelkwaschny]

You're right, my bad. Here is a PR that should fix this issue: #329