Skip to content

Add series_iir, series_log, series_magnitude, series_max, series_min, series_multiply, series_pearson_correlation, series_pow, series_sign, series_stats, series_stats_dynamic, series_subtract #439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 5, 2025
Merged

Add series_iir, series_log, series_magnitude, series_max, series_min, series_multiply, series_pearson_correlation, series_pow, series_sign, series_stats, series_stats_dynamic, series_subtract #439

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
77bc33b
First draft
manototh Oct 13, 2025
dc5f052
Fix Vale
manototh Oct 13, 2025
690cfa0
Update series-magnitude.mdx
manototh Oct 13, 2025
1946ed6
Fix
manototh Oct 13, 2025
7ddd1c1
Fix min and max
manototh Oct 13, 2025
dfb322f
Fix stats
manototh Oct 13, 2025
6d0b2e4
Fix min, max
manototh Oct 13, 2025
d3896d1
Fix stats
manototh Oct 13, 2025
b074687
Fix stats
manototh Oct 13, 2025
acace65
Fixes
manototh Oct 17, 2025
66294bd
Merge branch 'main' into mano/apl-oct3
manototh Nov 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions apl/apl-features.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,10 +322,22 @@ keywords: ['axiom documentation', 'documentation', 'axiom', 'APL', 'axiom proces
| Time series function | [series_greater](/apl/scalar-functions/time-series/series-greater) | Returns the elements of a series that are greater than a specified value. |
| Time series function | [series_greater_equals](/apl/scalar-functions/time-series/series-greater-equals) | Returns the elements of a series that are greater than or equal to a specified value. |
| Time series function | [series_ifft](/apl/scalar-functions/time-series/series-ifft) | Performs an Inverse Fast Fourier Transform on a series, converting frequency-domain data back into time-domain representation. |
| Time series function | [series_iir](/apl/scalar-functions/time-series/series-iir) | Applies an Infinite Impulse Response filter to a series. |
| Time series function | [series_less](/apl/scalar-functions/time-series/series-less) | Returns the elements of a series that are less than a specified value. |
| Time series function | [series_less_equals](/apl/scalar-functions/time-series/series-less-equals) | Returns the elements of a series that are less than or equal to a specified value. |
| Time series function | [series_log](/apl/scalar-functions/time-series/series-log) | Returns the natural logarithm of each element in a series. |
| Time series function | [series_magnitude](/apl/scalar-functions/time-series/series-magnitude) | Calculates the Euclidean norm (magnitude) of a series. |
| Time series function | [series_max](/apl/scalar-functions/time-series/series-max) | Returns the maximum value from a series. |
| Time series function | [series_min](/apl/scalar-functions/time-series/series-min) | Returns the minimum value from a series. |
| Time series function | [series_multiply](/apl/scalar-functions/time-series/series-multiply) | Performs element-wise multiplication of two series. |
| Time series function | [series_not_equals](/apl/scalar-functions/time-series/series-not-equals) | Returns the elements of a series that aren’t equal to a specified value. |
| Time series function | [series_pearson_correlation](/apl/scalar-functions/time-series/series-pearson-correlation) | Calculates the Pearson correlation coefficient between two series. |
| Time series function | [series_pow](/apl/scalar-functions/time-series/series-pow) | Raises each element in a series to a specified power. |
| Time series function | [series_sign](/apl/scalar-functions/time-series/series-sign) | Returns the sign of each element in a series. |
| Time series function | [series_sin](/apl/scalar-functions/time-series/series-sin) | Returns the sine of a series. |
| Time series function | [series_stats](/apl/scalar-functions/time-series/series-stats) | Computes comprehensive statistical measures for a series. |
| Time series function | [series_stats_dynamic](/apl/scalar-functions/time-series/series-stats-dynamic) | Computes statistical measures and returns them in a dynamic object format. |
| Time series function | [series_subtract](/apl/scalar-functions/time-series/series-subtract) | Performs element-wise subtraction between two series. |
| Time series function | [series_sum](/apl/scalar-functions/time-series/series-sum) | Returns the sum of a series. |
| Time series function | [series_tan](/apl/scalar-functions/time-series/series-tan) | Returns the tangent of a series. |
| Type function | [iscc](/apl/scalar-functions/type-functions/iscc) | Checks whether a value is a valid credit card (CC) number. |
Expand Down
12 changes: 12 additions & 0 deletions apl/scalar-functions/time-series/overview.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,22 @@ The table summarizes the time series functions available in APL.
| [series_greater](/apl/scalar-functions/time-series/series-greater) | Returns the elements of a series that are greater than a specified value. |
| [series_greater_equals](/apl/scalar-functions/time-series/series-greater-equals) | Returns the elements of a series that are greater than or equal to a specified value. |
| [series_ifft](/apl/scalar-functions/time-series/series-ifft) | Performs an Inverse Fast Fourier Transform on a series, converting frequency-domain data back into time-domain representation. |
| [series_iir](/apl/scalar-functions/time-series/series-iir) | Applies an Infinite Impulse Response filter to a series. |
| [series_less](/apl/scalar-functions/time-series/series-less) | Returns the elements of a series that are less than a specified value. |
| [series_less_equals](/apl/scalar-functions/time-series/series-less-equals) | Returns the elements of a series that are less than or equal to a specified value. |
| [series_log](/apl/scalar-functions/time-series/series-log) | Returns the natural logarithm of each element in a series. |
| [series_magnitude](/apl/scalar-functions/time-series/series-magnitude) | Calculates the Euclidean norm (magnitude) of a series. |
| [series_max](/apl/scalar-functions/time-series/series-max) | Returns the maximum value from a series. |
| [series_min](/apl/scalar-functions/time-series/series-min) | Returns the minimum value from a series. |
| [series_multiply](/apl/scalar-functions/time-series/series-multiply) | Performs element-wise multiplication of two series. |
| [series_not_equals](/apl/scalar-functions/time-series/series-not-equals) | Returns the elements of a series that aren’t equal to a specified value. |
| [series_pearson_correlation](/apl/scalar-functions/time-series/series-pearson-correlation) | Calculates the Pearson correlation coefficient between two series. |
| [series_pow](/apl/scalar-functions/time-series/series-pow) | Raises each element in a series to a specified power. |
| [series_sign](/apl/scalar-functions/time-series/series-sign) | Returns the sign of each element in a series. |
| [series_sin](/apl/scalar-functions/time-series/series-sin) | Returns the sine of a series. |
| [series_stats](/apl/scalar-functions/time-series/series-stats) | Computes comprehensive statistical measures for a series. |
| [series_stats_dynamic](/apl/scalar-functions/time-series/series-stats-dynamic) | Computes statistical measures and returns them in a dynamic object format. |
| [series_subtract](/apl/scalar-functions/time-series/series-subtract) | Performs element-wise subtraction between two series. |
| [series_sum](/apl/scalar-functions/time-series/series-sum) | Returns the sum of a series. |
| [series_tan](/apl/scalar-functions/time-series/series-tan) | Returns the tangent of a series. |

Expand Down
165 changes: 165 additions & 0 deletions apl/scalar-functions/time-series/series-iir.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
---
title: series_iir
description: 'This page explains how to use the series_iir function in APL.'
---

The `series_iir` function applies an Infinite Impulse Response (IIR) filter to a numeric dynamic array (series). This filter processes the input series using coefficients for both the numerator (feedforward) and denominator (feedback) components, creating a filtered output series that incorporates both current and past values.

You can use `series_iir` when you need to apply digital signal processing techniques to time-series data. This is particularly useful for smoothing noisy data, removing high-frequency components, implementing custom filters, or applying frequency-selective transformations to time-series measurements.

## For users of other query languages

If you come from other query languages, this section explains how to adjust your existing queries to achieve the same results in APL.

<AccordionGroup>
<Accordion title="Splunk SPL users">

In Splunk SPL, signal processing typically requires external tools or complex manual calculations with `streamstats`. In APL, `series_iir` provides built-in digital filtering capabilities for array data.

<CodeGroup>
```sql Splunk example
... | streamstats window=5 avg(value) as smoothed_value
... (limited to basic moving averages)
```

```kusto APL equivalent
datatable(values: dynamic)
[
dynamic([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
]
| extend filtered = series_iir(values, dynamic([0.25, 0.5, 0.25]), dynamic([1.0, -0.5]))
```
</CodeGroup>

</Accordion>
<Accordion title="ANSI SQL users">

In SQL, implementing IIR filters requires complex recursive queries or user-defined functions. In APL, `series_iir` provides this functionality as a built-in operation on array data.

<CodeGroup>
```sql SQL example
-- Complex recursive CTE required for IIR filtering
WITH RECURSIVE filtered AS (...)
SELECT * FROM filtered;
```

```kusto APL equivalent
datatable(values: dynamic)
[
dynamic([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
]
| extend filtered = series_iir(values, dynamic([0.25, 0.5, 0.25]), dynamic([1.0, -0.5]))
```
</CodeGroup>

</Accordion>
</AccordionGroup>

## Usage

### Syntax

```kusto
series_iir(array, numerator, denominator)
```

### Parameters

| Parameter | Type | Description |
| ------------- | ------- | -------------------------------------------------------------- |
| `array` | dynamic | A dynamic array of numeric values (input series). |
| `numerator` | dynamic | A dynamic array of numerator (feedforward) coefficients. |
| `denominator` | dynamic | A dynamic array of denominator (feedback) coefficients. |

### Returns

A dynamic array containing the filtered output series after applying the IIR filter defined by the numerator and denominator coefficients.

## Use case examples

<Tabs>
<Tab title="Log analysis">

In log analysis, you can use `series_iir` to smooth noisy request duration measurements, making trends and patterns more visible.

**Query**

```kusto
['sample-http-logs']
| summarize durations = make_list(req_duration_ms) by id
| extend smoothed = series_iir(durations, dynamic([0.2, 0.6, 0.2]), dynamic([1.0]))
| take 5
```

[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20summarize%20durations%20%3D%20make_list(req_duration_ms)%20by%20id%20%7C%20extend%20smoothed%20%3D%20series_iir(durations%2C%20dynamic(%5B0.2%2C%200.6%2C%200.2%5D)%2C%20dynamic(%5B1.0%5D))%20%7C%20take%205%22%7D)

**Output**

| id | durations | smoothed |
| ---- | -------------------------- | -------------------------- |
| u123 | [50, 120, 45, 200, 60] | [50, 91, 62, 128, 88] |
| u456 | [30, 35, 80, 40, 45] | [30, 33, 54, 46, 45] |

This query applies an IIR filter to smooth request duration measurements, reducing noise while preserving the underlying trend.

</Tab>
<Tab title="OpenTelemetry traces">

In OpenTelemetry traces, you can use `series_iir` to filter span duration data, removing high-frequency noise to better identify sustained performance trends.

**Query**

```kusto
['otel-demo-traces']
| extend duration_ms = duration / 1ms
| summarize durations = make_list(duration_ms) by ['service.name']
| extend filtered = series_iir(durations, dynamic([0.1, 0.8, 0.1]), dynamic([1.0, -0.3]))
| take 5
```

[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'otel-demo-traces'%5D%20%7C%20extend%20duration_ms%20%3D%20duration%20%2F%201ms%20%7C%20summarize%20durations%20%3D%20make_list(duration_ms)%20by%20%5B'service.name'%5D%20%7C%20extend%20filtered%20%3D%20series_iir(durations%2C%20dynamic(%5B0.1%2C%200.8%2C%200.1%5D)%2C%20dynamic(%5B1.0%2C%20-0.3%5D))%20%7C%20take%205%22%7D)

**Output**

| service.name | durations | filtered |
| ------------ | --------------------------- | --------------------------- |
| frontend | [100, 150, 95, 200, 120] | [100, 130, 108, 152, 133] |
| checkout | [200, 250, 180, 300, 220] | [200, 230, 202, 248, 232] |

This query applies an IIR filter with feedback to span durations, smoothing out transient spikes while maintaining sensitivity to sustained changes.

</Tab>
<Tab title="Security logs">

In security logs, you can use `series_iir` to filter request rate data, separating sustained traffic changes from brief anomalies.

**Query**

```kusto
['sample-http-logs']
| summarize request_counts = make_list(req_duration_ms) by status
| extend filtered = series_iir(request_counts, dynamic([0.15, 0.7, 0.15]), dynamic([1.0, -0.4]))
| take 5
```

[Run in Playground](https://play.axiom.co/axiom-play-qf1k/query?initForm=%7B%22apl%22%3A%22%5B'sample-http-logs'%5D%20%7C%20summarize%20request_counts%20%3D%20make_list(req_duration_ms)%20by%20status%20%7C%20extend%20filtered%20%3D%20series_iir(request_counts%2C%20dynamic(%5B0.15%2C%200.7%2C%200.15%5D)%2C%20dynamic(%5B1.0%2C%20-0.4%5D))%20%7C%20take%205%22%7D)

**Output**

| status | request_counts | filtered |
| ------ | -------------------------- | -------------------------- |
| 200 | [100, 105, 300, 110, 95] | [100, 103, 180, 142, 120] |
| 401 | [10, 12, 50, 15, 11] | [10, 11, 27, 20, 16] |

This query uses IIR filtering to smooth security event patterns, helping distinguish between brief anomalies and sustained attack patterns.

</Tab>
</Tabs>

## List of related functions

- [series_sum](/apl/scalar-functions/time-series/series-sum): Returns the sum of series elements. Use for simple aggregation instead of filtering.
- [series_stats](/apl/scalar-functions/time-series/series-stats): Returns statistical measures. Use for statistical analysis instead of signal processing.
- [series_abs](/apl/scalar-functions/time-series/series-abs): Returns absolute values. Often used after IIR filtering to analyze magnitude.
- [make_series](/apl/tabular-operators/make-series): Creates time-series from tabular data. Often used before applying `series_iir` for signal processing.

Loading