Skip to content

Latest commit

 

History

History
184 lines (132 loc) · 11.3 KB

11_using_fields.md

File metadata and controls

184 lines (132 loc) · 11.3 KB

Back to index

[11 Using Fields]

Introduction (0:44)

  • Use fields in Splunks' Search processing language to filter data to return needed results.
  • Select which fields are extracted from the index versus search time.
  • Learn the difference between persistent and temporary fields.
  • Do more whit data through the use of knowledge objects.

Using the fields Sidebar (01:42)

Shows all the fields extracted at search time split into two categories

  • Selected fields. Show at the events' bottom together with the default fields (host, source, and sourcetype)

  • interesting fields. are fields that have values in at least 20% of the events. image

    a in front of the field name denotes a string field. # in front of the field name denotes a numeral field. Clicking on a field show values for the field, a count of the values, and a percentage of the events the value shows up in. image image

You can add a field value pair Country=Germany to the search by clicking on it.

image

You can launch quick reports based on the field (top/Rare Values or top values by time as shown below)

image

When you add a field to the selected fields list image

aside from showing up at the event, it will persist for subsequent searches image

From the fields sidebar, either with the all-fields button

image or the more fileds link image

we can see all fields for the search, filter them, see the number of distinct values in the field, the coverage or percentage of the events that have a value in the field, and add a field to the selected fields. image

Using Fields in Search (04:02)

You can refine and run more efficient searches using fields. Field names are case-sensitive while values are not image

Operator Field Operator Usage
=, != with Numerical or string values
>, >0, <, <= wiht numerical values

Wild cards and booleans are allowed image For fields containing an IP address, wildcards are subnet and CIDR aware image

!= is not equal field operator versus NOTboolean

image

They do not always return the same results or the same number of events. The tops search only returns events where the status field is not 200 while the bottom search returns all events that do not have a field where status equals 200. Meaning that if an event does not have a status field at all, it will be included in the results. Me parece que esto es debido a que implicitamente el operador AND anda por ahí.

IN operator can be used as an alternative to chaining several operators

image image

fields command

Can be used to include (+ plus operator) or exclude (- minus operator) fields from the search. We can make our search more efficient by adding a field command to include only the needed fields. image

Filtering as early as possible in a search is a best practice. Fields default to inclusion, so if not operator is specified the supplied fields will be included in the search

renamecommand.

Can be used to rename fields in the search to give them more meaningful or user-friendly names. image

Fields in Search Results (7:05)

Splunk returns

Indexing versus search time field extraction

Some fields are extracted at index time while others are extracted at search time. At data ingestion, a selected number of fields are automatically extracted (metadata: host, source, sourcetype, _time ( Event's timestamp), and _raw (original raw data of an event) At search time, field discovery extracts additional fields from the raw event data based on its assigned sourcetype, and key-value pairs found in the data. These fields are persistent and will be extracted every time a search is run containing the same search terms unless no inclusion is specified.

temporary fields

Are created on an ad-hoc basis with the command eval, which is used to calculate and manipulate field values. Results of eval can be written at search time to a new temporary field or to replace an existing field value. image Since we used a non-existing field name for the results, a new temporary field bandwidth is created.

Field Extraction

The utility field extractor can be used to extract new, persistent across searches, data fields that were not automatically extracted from the data accordingly with the assigned sourcetype. The commands rex and rex can also be used to extract temporal fields at search time.

erex command

erex new_field_name fromfield (is the field in our data to match on) examples (a list of sample data) image

When the search is run, Splunk builds a regular expression based on the sample data, checks it against _raw event data field, and ad matched values to the character field. `Erex' only knows what to look for based on the sample i gave it.

image

image

Using the where command with the isnull function sw can see some missed character names we can add to the sample data list.

We can see RegEx generated by erex in the job menu image

rex command

we will use the capture groups to extract values at search time from field values or raw data. Rex allows matching multiple groups resulting in multiple fields. Let's extract USER and CHARACTER name values rex field= field_to_match_on "regular expresion between double quotes _raw field will be used by default if not field argument is given. Run rex over _raw can have a performance impact. Is better to used it wiht an already extracted field. image this regex captures email addresses. We continue the expression to return a Characte field. Behind our capture group, we add an apostrophe ', followed by a \s to match the white space. To match caracter name [a-zA-Z:]+ follows by another ' apostrophe. then we create another capture grodp `(?P[a-zA-Z0-9.-]+) image

Erex versus rex

erex rex
Usage easier sofisticated
Requires Regular expression knowledge no yes
Requires sample data yes no
Generates Regular Expresion yes no
Help can use Erex-generated Regular expression as a starting point
Advice When possible use rex

My own try image

Enriching Data with Knowledge objects (03:02)

calculated fields

With the eval command, we created a temporary field to convert bytes into megabytes. As it is not a persistent solution we need to write such a conversion every time we want to make it. The calculated field is a better option for that:

From settings menu

image in the fields option

image we find the calculated fields dialog

image

that if filled properly image will allow Splunk to create the field at search time every time the search contains the bytes field in the run.

Calculated fields can only reference fields that are already present in the event returned by a search. So for calculated fields to perform correctly make sure they are configured to reference a field that has already been extracted.

field aliases

Allow to assign alternate names to fields in the data. If you have fields from multiple sourcetypes that all contain similar values image the field alias group together similar values. This allows us to search for all values at once in the shared field alias. field aliases do not replace or remove the original field name so you can search data usin the original name or its alias.

Lookups

Adds to the events other fields and values that are not part of the indexed data From settings menu

image in the lookups option image

the value pairs can be configured to automatically append to events in the search at search time allowing to add related information

image

Search time operations order

image

Field extractions are evaluated first in the pipeline. That means that Calculated fields can add additional context to extracted fields, but a field alias cannot reference a value from a lookup

Back to index