|
| 1 | +# Splunk lookup export |
| 2 | + |
| 3 | +`splunk_export` creates a UTF-8 CSV lookup from one or more MISP events. It includes |
| 4 | +both event attributes and attributes inside objects. It needs no Splunk credentials, |
| 5 | +network access or extra Python packages. Existing MISP objects such as `domain-ip`, |
| 6 | +`file`, `email` and `network-connection` can supply the attributes; no new templates |
| 7 | +are required. |
| 8 | + |
| 9 | +## Configure and export |
| 10 | + |
| 11 | +Enable **Splunk Lookup Export** in MISP's export module settings. By default only |
| 12 | +attributes with `to_ids=true` are exported. The optional module configuration |
| 13 | +`include_non_ids=true` also includes contextual attributes. Boolean values and |
| 14 | +strings `true`, `false`, `1` and `0` are accepted. Deleted attributes, objects and |
| 15 | +events are always excluded. |
| 16 | + |
| 17 | +The API accepts `data` as a list of standard MISP events (`{"Event": {...}}), bare |
| 18 | +event dictionaries, or legacy export envelopes with `Attribute` and `Object` |
| 19 | +alongside `Event` metadata. For example, POST this JSON to `/query`: |
| 20 | + |
| 21 | +```json |
| 22 | +{ |
| 23 | + "module": "splunk_export", |
| 24 | + "config": {"include_non_ids": false}, |
| 25 | + "data": [{ |
| 26 | + "Event": { |
| 27 | + "uuid": "f1ef0409-c80a-4012-b401-b7d1050f4ed4", |
| 28 | + "info": "Example indicators", |
| 29 | + "Attribute": [{ |
| 30 | + "uuid": "aa09819b-f2d8-4661-83ea-d4897cafebc8", |
| 31 | + "type": "ip-dst", |
| 32 | + "value": "192.0.2.1", |
| 33 | + "to_ids": true |
| 34 | + }] |
| 35 | + } |
| 36 | + }] |
| 37 | +} |
| 38 | +``` |
| 39 | + |
| 40 | +Decode the response's base64 `data` field and save it as `misp_indicators.csv`. |
| 41 | +Unsupported attribute types and malformed values are skipped; the API response's |
| 42 | +optional `warnings` list reports their counts. The MISP download UI may not display |
| 43 | +these warnings. An export with no eligible indicators produces the CSV header only. |
| 44 | + |
| 45 | +## Supported indicators |
| 46 | + |
| 47 | +| MISP type | Lookup `indicator_type` | Suggested CIM field(s) | |
| 48 | +| --- | --- | --- | |
| 49 | +| `ip-src`, `ip-dst` | `ip` | `src_ip`, `dest_ip` respectively | |
| 50 | +| `domain` | `domain` | DNS `query` | |
| 51 | +| `hostname` | `hostname` | `dest` | |
| 52 | +| `url` | `url` | `url` | |
| 53 | +| `email-src`, `email-dst` | `email` | `src_user`, `recipient` respectively | |
| 54 | +| `filename` | `filename` | `file_name` | |
| 55 | +| `md5`, `sha1`, `sha256`, `sha512` | Corresponding hash type | `file_hash` | |
| 56 | +| `domain\|ip` | Separate `domain` and `ip` rows | `query`; `src_ip,dest_ip` | |
| 57 | +| `ip-src\|port`, `ip-dst\|port`, `hostname\|port` | `ip` or `hostname` | As above, with a separate `port` column | |
| 58 | +| `filename\|md5`, `filename\|sha1`, `filename\|sha256`, `filename\|sha512`, `malware-sample` | Separate `filename` and hash rows | `file_name`, `file_hash` | |
| 59 | + |
| 60 | +`cim_fields` contains mapping hints, not a complete CIM event schema or a Splunk |
| 61 | +Enterprise Security threat-intelligence import schema. Choose the field appropriate |
| 62 | +to your data model and sourcetype. A composite attribute produces independent |
| 63 | +hunting pivots, not a detection requiring all components to match. Its |
| 64 | +`original_value`, `attribute_uuid` and optional `port` preserve that association. |
| 65 | + |
| 66 | +Values are preserved exactly, including IPv6 addresses, letter case and Unicode; |
| 67 | +this module is not a type validator. Lookup matching is exact by default, not CIDR, |
| 68 | +wildcard, URL-host or subdomain matching. Normalize both lookup values and telemetry |
| 69 | +where needed. A domain lookup against `query` does not match every URL containing |
| 70 | +that domain. |
| 71 | + |
| 72 | +## Columns and provenance |
| 73 | + |
| 74 | +The stable header is: |
| 75 | + |
| 76 | +```text |
| 77 | +indicator,indicator_type,cim_fields,port,misp_type,original_value,event_uuid,event_id,event_info,attribute_uuid,object_uuid,object_name,object_relation,category,comment,to_ids,tags,first_seen,last_seen |
| 78 | +``` |
| 79 | + |
| 80 | +`tags` is a sorted JSON array combining supplied event, object and attribute tags, |
| 81 | +including TLP and confidence tags when present. No confidence is inferred. |
| 82 | +`first_seen` and `last_seen` are observations copied from the attribute, not expiry |
| 83 | +times. Rows are deduplicated only when all columns agree; separate source events, |
| 84 | +attributes and object contexts remain distinguishable. CSV quoting preserves |
| 85 | +commas, quotes and newlines without changing the underlying indicator values. |
| 86 | + |
| 87 | +The lookup contains the data supplied to the module. Apply your intended sharing |
| 88 | +policy when selecting events and distributing the resulting file. Tags remain |
| 89 | +metadata; Splunk does not automatically enforce their handling rules. |
| 90 | + |
| 91 | +## Upload and hunt |
| 92 | + |
| 93 | +Upload the CSV under **Settings > Lookups > Lookup table files**, and grant the |
| 94 | +intended Splunk app and roles access. These examples use the lookup filename |
| 95 | +directly. For indicators shared by multiple MISP sources, create a lookup definition |
| 96 | +with sufficient `max_matches` to return all sources; a file lookup's default match |
| 97 | +limit may return only the first source. Use the definition name in searches instead |
| 98 | +of the filename when using custom settings. |
| 99 | + |
| 100 | +This bounded network search assumes CIM-normalized `dest_ip` values in an index |
| 101 | +named `network`. Adapt the index and field names to your deployment: |
| 102 | + |
| 103 | +```spl |
| 104 | +index=network earliest=-24h latest=now |
| 105 | +| eval misp_match_type="ip" |
| 106 | +| lookup misp_indicators.csv indicator_type AS misp_match_type indicator AS dest_ip |
| 107 | + OUTPUT event_uuid AS misp_event_uuid attribute_uuid AS misp_attribute_uuid |
| 108 | +| where isnotnull(misp_attribute_uuid) |
| 109 | +| table _time src_ip dest_ip misp_event_uuid misp_attribute_uuid |
| 110 | +``` |
| 111 | + |
| 112 | +For MD5 file telemetry, use the corresponding algorithm as the lookup key. This |
| 113 | +avoids matching a file name or a different indicator type with the same text: |
| 114 | + |
| 115 | +```spl |
| 116 | +index=endpoint earliest=-24h latest=now |
| 117 | +| eval misp_match_type="md5" |
| 118 | +| lookup misp_indicators.csv indicator_type AS misp_match_type indicator AS file_hash |
| 119 | + OUTPUT event_uuid AS misp_event_uuid attribute_uuid AS misp_attribute_uuid |
| 120 | +| where isnotnull(misp_attribute_uuid) |
| 121 | +| table _time dest file_name file_hash misp_event_uuid misp_attribute_uuid |
| 122 | +``` |
| 123 | + |
| 124 | +These searches perform indicator matching only. The network example does not |
| 125 | +enforce the optional port constraint or MISP direction; add those conditions if your |
| 126 | +detection requires them. Refresh or replace the lookup as indicators change: this |
| 127 | +module does not synchronize deletions or expire indicators in Splunk. |
0 commit comments