- Use fields in Splunks' Search processing language to filter data to return needed results.
- Select which fields are extracted from the index versus search time.
- Learn the difference between persistent and temporary fields.
- Do more whit data through the use of knowledge objects.
Shows all the fields extracted at search time split into two categories
-
Selected fields. Show at the events' bottom together with the default fields (host, source, and sourcetype)
-
interesting fields. are fields that have values in at least 20% of the events.
a
in front of the field name denotes a string field.#
in front of the field name denotes a numeral field. Clicking on a field show values for the field, a count of the values, and a percentage of the events the value shows up in.
You can add a field value pair Country=Germany
to the search by clicking on it.
You can launch quick reports based on the field (top/Rare Values or top values by time as shown below)
When you add a field to the selected fields list
aside from showing up at the event, it will persist for subsequent searches
From the fields sidebar, either with the all-fields
button
we can see all fields for the search, filter them, see the number of distinct values in the field, the coverage or percentage of the events that have a value in the field, and add a field to the selected fields.
You can refine and run more efficient searches using fields.
Field names are case-sensitive while values are not
Operator | Field Operator Usage |
---|---|
=, != | with Numerical or string values |
>, >0, <, <= | wiht numerical values |
Wild cards and booleans are allowed
For fields containing an IP address, wildcards are subnet and CIDR aware
They do not always return the same results or the same number of events. The tops search only returns events where the status field is not 200 while the bottom search returns all events that do not have a field where status equals 200. Meaning that if an event does not have a status field at all, it will be included in the results. Me parece que esto es debido a que implicitamente el operador AND anda por ahí.
Can be used to include (+
plus operator) or exclude (-
minus operator) fields from the search.
We can make our search more efficient by adding a field command to include only the needed fields.
Filtering as early as possible in a search is a best practice. Fields default to inclusion, so if not operator is specified the supplied fields will be included in the search
Can be used to rename fields in the search to give them more meaningful or user-friendly names.
Splunk returns
Some fields are extracted at index time while others are extracted at search time. At data ingestion, a selected number of fields are automatically extracted (metadata: host, source, sourcetype, _time ( Event's timestamp), and _raw (original raw data of an event) At search time, field discovery extracts additional fields from the raw event data based on its assigned sourcetype, and key-value pairs found in the data. These fields are persistent and will be extracted every time a search is run containing the same search terms unless no inclusion is specified.
Are created on an ad-hoc basis with the command eval
, which is used to calculate and manipulate field values.
Results of eval can be written at search time to a new temporary field or to replace an existing field value.
Since we used a non-existing field name for the results, a new temporary field
bandwidth
is created.
The utility field extractor
can be used to extract new, persistent across searches, data fields that were not automatically extracted from the data accordingly with the assigned sourcetype.
The commands rex
and rex
can also be used to extract temporal fields at search time.
erex
new_field_name fromfield
(is the field in our data to match on) examples
(a list of sample data)
When the search is run, Splunk builds a regular expression based on the sample data, checks it against _raw event data field, and ad matched values to the character
field.
`Erex' only knows what to look for based on the sample i gave it.
Using the where
command with the isnull
function sw can see some missed character names we can add to the sample data list.
We can see RegEx generated by erex
in the job menu
we will use the capture groups to extract values at search time from field values or raw data.
Rex allows matching multiple groups resulting in multiple fields.
Let's extract USER and CHARACTER name values
rex
field= field_to_match_on "regular expresion between double quotes
_raw field will be used by default if not field argument is given.
Run rex over _raw can have a performance impact. Is better to used it wiht an already extracted field.
this regex captures email addresses.
We continue the expression to return a Characte field. Behind our capture group, we add an apostrophe
'
, followed by a \s
to match the white space. To match caracter name [a-zA-Z:]+
follows by another '
apostrophe. then we create another capture grodp `(?P[a-zA-Z0-9.-]+)
erex | rex | |
---|---|---|
Usage | easier | sofisticated |
Requires Regular expression knowledge | no | yes |
Requires sample data | yes | no |
Generates Regular Expresion | yes | no |
Help | can use Erex-generated Regular expression as a starting point | |
Advice | When possible use rex |
With the eval command, we created a temporary field to convert bytes into megabytes. As it is not a persistent solution we need to write such a conversion every time we want to make it. The calculated field is a better option for that:
From settings
menu
we find the
calculated fields
dialog
that if filled properly
will allow Splunk to create the field at search time every time the search contains the bytes field in the run.
Calculated fields can only reference fields that are already present in the event returned by a search. So for calculated fields to perform correctly make sure they are configured to reference a field that has already been extracted.
Allow to assign alternate names to fields in the data. If you have fields from multiple sourcetypes that all contain similar values
the field alias group together similar values. This allows us to search for all values at once in the shared field alias. field aliases do not replace or remove the original field name so you can search data usin the original name or its alias.
Adds to the events other fields and values that are not part of the indexed data
From settings
menu
the value pairs can be configured to automatically append to events in the search at search time allowing to add related information
Field extractions
are evaluated first in the pipeline. That means that Calculated fields
can add additional context to extracted fields
, but a field alias
cannot reference a value from a lookup