Skip to content

Commit b431f3c

Browse files
manotothmanototh
authored
Add series_iir, series_log, series_magnitude, series_max, series_min, series_multiply, series_pearson_correlation, series_pow, series_sign, series_stats, series_stats_dynamic, series_subtract (#439)
1 parent 0145080 commit b431f3c

15 files changed

+1965
-0
lines changed

‎apl/apl-features.mdx‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,10 +322,22 @@ keywords: ['axiom documentation', 'documentation', 'axiom', 'APL', 'axiom proces
322322
| Time series function | [series_greater](/apl/scalar-functions/time-series/series-greater) | Returns the elements of a series that are greater than a specified value. |
323323
| Time series function | [series_greater_equals](/apl/scalar-functions/time-series/series-greater-equals) | Returns the elements of a series that are greater than or equal to a specified value. |
324324
| Time series function | [series_ifft](/apl/scalar-functions/time-series/series-ifft) | Performs an Inverse Fast Fourier Transform on a series, converting frequency-domain data back into time-domain representation. |
325+
| Time series function | [series_iir](/apl/scalar-functions/time-series/series-iir) | Applies an Infinite Impulse Response filter to a series. |
325326
| Time series function | [series_less](/apl/scalar-functions/time-series/series-less) | Returns the elements of a series that are less than a specified value. |
326327
| Time series function | [series_less_equals](/apl/scalar-functions/time-series/series-less-equals) | Returns the elements of a series that are less than or equal to a specified value. |
328+
| Time series function | [series_log](/apl/scalar-functions/time-series/series-log) | Returns the natural logarithm of each element in a series. |
329+
| Time series function | [series_magnitude](/apl/scalar-functions/time-series/series-magnitude) | Calculates the Euclidean norm (magnitude) of a series. |
330+
| Time series function | [series_max](/apl/scalar-functions/time-series/series-max) | Returns the maximum value from a series. |
331+
| Time series function | [series_min](/apl/scalar-functions/time-series/series-min) | Returns the minimum value from a series. |
332+
| Time series function | [series_multiply](/apl/scalar-functions/time-series/series-multiply) | Performs element-wise multiplication of two series. |
327333
| Time series function | [series_not_equals](/apl/scalar-functions/time-series/series-not-equals) | Returns the elements of a series that aren’t equal to a specified value. |
334+
| Time series function | [series_pearson_correlation](/apl/scalar-functions/time-series/series-pearson-correlation) | Calculates the Pearson correlation coefficient between two series. |
335+
| Time series function | [series_pow](/apl/scalar-functions/time-series/series-pow) | Raises each element in a series to a specified power. |
336+
| Time series function | [series_sign](/apl/scalar-functions/time-series/series-sign) | Returns the sign of each element in a series. |
328337
| Time series function | [series_sin](/apl/scalar-functions/time-series/series-sin) | Returns the sine of a series. |
338+
| Time series function | [series_stats](/apl/scalar-functions/time-series/series-stats) | Computes comprehensive statistical measures for a series. |
339+
| Time series function | [series_stats_dynamic](/apl/scalar-functions/time-series/series-stats-dynamic) | Computes statistical measures and returns them in a dynamic object format. |
340+
| Time series function | [series_subtract](/apl/scalar-functions/time-series/series-subtract) | Performs element-wise subtraction between two series. |
329341
| Time series function | [series_sum](/apl/scalar-functions/time-series/series-sum) | Returns the sum of a series. |
330342
| Time series function | [series_tan](/apl/scalar-functions/time-series/series-tan) | Returns the tangent of a series. |
331343
| Type function | [iscc](/apl/scalar-functions/type-functions/iscc) | Checks whether a value is a valid credit card (CC) number. |

‎apl/scalar-functions/time-series/overview.mdx‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,22 @@ The table summarizes the time series functions available in APL.
3030
| [series_greater](/apl/scalar-functions/time-series/series-greater) | Returns the elements of a series that are greater than a specified value. |
3131
| [series_greater_equals](/apl/scalar-functions/time-series/series-greater-equals) | Returns the elements of a series that are greater than or equal to a specified value. |
3232
| [series_ifft](/apl/scalar-functions/time-series/series-ifft) | Performs an Inverse Fast Fourier Transform on a series, converting frequency-domain data back into time-domain representation. |
33+
| [series_iir](/apl/scalar-functions/time-series/series-iir) | Applies an Infinite Impulse Response filter to a series. |
3334
| [series_less](/apl/scalar-functions/time-series/series-less) | Returns the elements of a series that are less than a specified value. |
3435
| [series_less_equals](/apl/scalar-functions/time-series/series-less-equals) | Returns the elements of a series that are less than or equal to a specified value. |
36+
| [series_log](/apl/scalar-functions/time-series/series-log) | Returns the natural logarithm of each element in a series. |
37+
| [series_magnitude](/apl/scalar-functions/time-series/series-magnitude) | Calculates the Euclidean norm (magnitude) of a series. |
38+
| [series_max](/apl/scalar-functions/time-series/series-max) | Returns the maximum value from a series. |
39+
| [series_min](/apl/scalar-functions/time-series/series-min) | Returns the minimum value from a series. |
40+
| [series_multiply](/apl/scalar-functions/time-series/series-multiply) | Performs element-wise multiplication of two series. |
3541
| [series_not_equals](/apl/scalar-functions/time-series/series-not-equals) | Returns the elements of a series that aren’t equal to a specified value. |
42+
| [series_pearson_correlation](/apl/scalar-functions/time-series/series-pearson-correlation) | Calculates the Pearson correlation coefficient between two series. |
43+
| [series_pow](/apl/scalar-functions/time-series/series-pow) | Raises each element in a series to a specified power. |
44+
| [series_sign](/apl/scalar-functions/time-series/series-sign) | Returns the sign of each element in a series. |
3645
| [series_sin](/apl/scalar-functions/time-series/series-sin) | Returns the sine of a series. |
46+
| [series_stats](/apl/scalar-functions/time-series/series-stats) | Computes comprehensive statistical measures for a series. |
47+
| [series_stats_dynamic](/apl/scalar-functions/time-series/series-stats-dynamic) | Computes statistical measures and returns them in a dynamic object format. |
48+
| [series_subtract](/apl/scalar-functions/time-series/series-subtract) | Performs element-wise subtraction between two series. |
3749
| [series_sum](/apl/scalar-functions/time-series/series-sum) | Returns the sum of a series. |
3850
| [series_tan](/apl/scalar-functions/time-series/series-tan) | Returns the tangent of a series. |
3951

‎apl/scalar-functions/time-series/series-iir.mdx‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
---
2+
title: series_iir
3+
description: 'This page explains how to use the series_iir function in APL.'
4+
---
5+
6+
The `series_iir` function applies an Infinite Impulse Response (IIR) filter to a numeric dynamic array (series). This filter processes the input series using coefficients for both the numerator (feedforward) and denominator (feedback) components, creating a filtered output series that incorporates both current and past values.
7+
8+
You can use `series_iir` when you need to apply digital signal processing techniques to time-series data. This is particularly useful for smoothing noisy data, removing high-frequency components, implementing custom filters, or applying frequency-selective transformations to time-series measurements.
9+
10+
## For users of other query languages
11+
12+
If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.
13+
14+
<AccordionGroup>
15+
<Accordion title="Splunk SPL users">
16+
17+
In Splunk SPL, signal processing typically requires external tools or complex manual calculations with `streamstats`. In APL, `series_iir` provides built-in digital filtering capabilities for array data.
18+
19+
<CodeGroup>
20+
```sql Splunk example
21+
... | streamstats window=5 avg(value) as smoothed_value
22+
... (limited to basic moving averages)
23+
```
24+
25+
```kusto APL equivalent
26+
datatable(values: dynamic)
27+
[
28+
dynamic([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
29+
]
30+
| extend filtered = series_iir(values, dynamic([0.25, 0.5, 0.25]), dynamic([1.0, -0.5]))
31+
```
32+
</CodeGroup>
33+
34+
</Accordion>
35+
<Accordion title="ANSI SQL users">
36+
37+
In SQL, implementing IIR filters requires complex recursive queries or user-defined functions. In APL, `series_iir` provides this functionality as a built-in operation on array data.
38+
39+
<CodeGroup>
40+
```sql SQL example
41+
-- Complex recursive CTE required for IIR filtering
42+
WITH RECURSIVE filtered AS (...)
43+
SELECT * FROM filtered;
44+
```
45+
46+
```kusto APL equivalent
47+
datatable(values: dynamic)
48+
[
49+
dynamic([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
50+
]
51+
| extend filtered = series_iir(values, dynamic([0.25, 0.5, 0.25]), dynamic([1.0, -0.5]))
52+
```
53+
</CodeGroup>
54+
55+
</Accordion>
56+
</AccordionGroup>
57+
58+
## Usage
59+
60+
### Syntax
61+
62+
```kusto
63+
series_iir(array, numerator, denominator)
64+
```
65+
66+
### Parameters
67+
68+
| Parameter | Type | Description |
69+
| ------------- | ------- | -------------------------------------------------------------- |
70+
| `array` | dynamic | A dynamic array of numeric values (input series). |
71+
| `numerator` | dynamic | A dynamic array of numerator (feedforward) coefficients. |
72+
| `denominator` | dynamic | A dynamic array of denominator (feedback) coefficients. |
73+
74+
### Returns
75+
76+
A dynamic array containing the filtered output series after applying the IIR filter defined by the numerator and denominator coefficients.
77+
78+
## Use case examples
79+
80+
<Tabs>
81+
<Tab title="Log analysis">
82+
83+
In log analysis, you can use `series_iir` to smooth noisy request duration measurements, making trends and patterns more visible.
84+
85+
**Query**
86+
87+
```kusto
88+
['sample-http-logs']
89+
| summarize durations = make_list(req_duration_ms) by id
90+
| extend smoothed = series_iir(durations, dynamic([0.2, 0.6, 0.2]), dynamic([1.0]))
91+
| take 5
92+
```
93+
94+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20summarize%20durations%20%3D%20make_list(req_duration_ms)%20by%20id%20%7C%20extend%20smoothed%20%3D%20series_iir(durations%2C%20dynamic(%5B0.2%2C%200.6%2C%200.2%5D)%2C%20dynamic(%5B1.0%5D))%20%7C%20take%205%22%7D)
95+
96+
**Output**
97+
98+
| id | durations | smoothed |
99+
| ---- | -------------------------- | -------------------------- |
100+
| u123 | [50, 120, 45, 200, 60] | [50, 91, 62, 128, 88] |
101+
| u456 | [30, 35, 80, 40, 45] | [30, 33, 54, 46, 45] |
102+
103+
This query applies an IIR filter to smooth request duration measurements, reducing noise while preserving the underlying trend.
104+
105+
</Tab>
106+
<Tab title="OpenTelemetry traces">
107+
108+
In OpenTelemetry traces, you can use `series_iir` to filter span duration data, removing high-frequency noise to better identify sustained performance trends.
109+
110+
**Query**
111+
112+
```kusto
113+
['otel-demo-traces']
114+
| extend duration_ms = duration / 1ms
115+
| summarize durations = make_list(duration_ms) by ['service.name']
116+
| extend filtered = series_iir(durations, dynamic([0.1, 0.8, 0.1]), dynamic([1.0, -0.3]))
117+
| take 5
118+
```
119+
120+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'otel-demo-traces'%5D%20%7C%20extend%20duration_ms%20%3D%20duration%20%2F%201ms%20%7C%20summarize%20durations%20%3D%20make_list(duration_ms)%20by%20%5B'service.name'%5D%20%7C%20extend%20filtered%20%3D%20series_iir(durations%2C%20dynamic(%5B0.1%2C%200.8%2C%200.1%5D)%2C%20dynamic(%5B1.0%2C%20-0.3%5D))%20%7C%20take%205%22%7D)
121+
122+
**Output**
123+
124+
| service.name | durations | filtered |
125+
| ------------ | --------------------------- | --------------------------- |
126+
| frontend | [100, 150, 95, 200, 120] | [100, 130, 108, 152, 133] |
127+
| checkout | [200, 250, 180, 300, 220] | [200, 230, 202, 248, 232] |
128+
129+
This query applies an IIR filter with feedback to span durations, smoothing out transient spikes while maintaining sensitivity to sustained changes.
130+
131+
</Tab>
132+
<Tab title="Security logs">
133+
134+
In security logs, you can use `series_iir` to filter request rate data, separating sustained traffic changes from brief anomalies.
135+
136+
**Query**
137+
138+
```kusto
139+
['sample-http-logs']
140+
| summarize request_counts = make_list(req_duration_ms) by status
141+
| extend filtered = series_iir(request_counts, dynamic([0.15, 0.7, 0.15]), dynamic([1.0, -0.4]))
142+
| take 5
143+
```
144+
145+
[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20summarize%20request_counts%20%3D%20make_list(req_duration_ms)%20by%20status%20%7C%20extend%20filtered%20%3D%20series_iir(request_counts%2C%20dynamic(%5B0.15%2C%200.7%2C%200.15%5D)%2C%20dynamic(%5B1.0%2C%20-0.4%5D))%20%7C%20take%205%22%7D)
146+
147+
**Output**
148+
149+
| status | request_counts | filtered |
150+
| ------ | -------------------------- | -------------------------- |
151+
| 200 | [100, 105, 300, 110, 95] | [100, 103, 180, 142, 120] |
152+
| 401 | [10, 12, 50, 15, 11] | [10, 11, 27, 20, 16] |
153+
154+
This query uses IIR filtering to smooth security event patterns, helping distinguish between brief anomalies and sustained attack patterns.
155+
156+
</Tab>
157+
</Tabs>
158+
159+
## List of related functions
160+
161+
- [series_sum](/apl/scalar-functions/time-series/series-sum): Returns the sum of series elements. Use for simple aggregation instead of filtering.
162+
- [series_stats](/apl/scalar-functions/time-series/series-stats): Returns statistical measures. Use for statistical analysis instead of signal processing.
163+
- [series_abs](/apl/scalar-functions/time-series/series-abs): Returns absolute values. Often used after IIR filtering to analyze magnitude.
164+
- [make_series](/apl/tabular-operators/make-series): Creates time-series from tabular data. Often used before applying `series_iir` for signal processing.
165+

0 commit comments

Comments
 (0)