diff --git a/content/develop/data-types/timeseries/configuration.md b/content/develop/data-types/timeseries/configuration.md index bf311626c..839df4307 100644 --- a/content/develop/data-types/timeseries/configuration.md +++ b/content/develop/data-types/timeseries/configuration.md @@ -14,67 +14,105 @@ linkTitle: Configuration title: Configuration Parameters weight: 3 --- -{{< note >}} -As of Redis Community Edition 8.0, configuration parameters for the time series data structure are now set in the following ways: -* At load time via your `redis.conf` file. -* At run time (where applicable) using the [`CONFIG SET`]({{< relref "/commands/config-set" >}}) command. +## Redis Community Edition - set configuration parameters -Also, Redis CE 8.0 persists probabilistic configuration parameters just like any other configuration parameters (e.g., using the [`CONFIG REWRITE`]({{< relref "/commands/config-rewrite/" >}}) command). -{{< /note >}} +Before version 8.0, all time series configuration parameters are load-time parameters. +Use one of the following methods to set the values of load-time configuration parameters: + +- Pass them as command-line arguments following the `loadmodule` argument when starting `redis-server`: + + `redis-server --loadmodule ./{modulename}.so [OPT VAL]...` + +- Add them as arguments to the `loadmodule` directive in your configuration file (for example, `redis.conf`): + + `loadmodule ./{modulename}.so [OPT VAL]...` + +- Use the `MODULE LOAD path [arg [arg ...]]` command. + +- Use the `MODULE LOADEX path [CONFIG name value [CONFIG name value ...]] [ARGS args [args ....]]` command. + +Starting with Redis 8.0, most time series configuration parameters are runtime parameters. +While you can set runtime parameters at load time, using the Redis `CONFIG` command is easier and works the same way as with Redis runtime configuration parameters. + +This means: + +- `CONFIG SET parameter value [parameter value ...] ` + + Set one or more configuration parameters. + +- `CONFIG GET parameter [parameter ...]` + + Read the current value of one of more parameters. + +- `CONFIG REWRITE` + + Rewrite your Redis configuration file (for example, the `redis.conf` file) to reflect the configuration changes. + +Starting with Redis 8.0, you can specify time series configuration parameters directly in your Redis configuration file the same way you would for Redis configuration parameters. + +Once a value is set with `CONFIG SET` or added manually to your configuration file, it will overwrite values set with `--loadmodule`, `loadmodule`, `MODULE LOAD`, or `MODULE LOADEX`. + +In a cluster, you must run `CONFIG SET` and `CONFIG REWRITE` on each node separately. + +In Redis 8.0, new names for the time series configuration parameters were introduced to align the naming with the Redis configuration parameters. +You must use the new names when using the `CONFIG` command. ## Time series configuration parameters | Parameter name
(version < 8.0) | Parameter name
(version ≥ 8.0) | Run-time | Redis
Software | Redis
Cloud | | :------- | :------- | :------- | :------- | :------- | -| CHUNK_SIZE_BYTES | [ts-chunk-size-bytes](#ts-chunk-size-bytes) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | -| COMPACTION_POLICY | [ts-compaction-policy](#ts-compaction-policy) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | -| DUPLICATE_POLICY | [ts-duplicate-policy](#ts-duplicate-policy) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | -| ENCODING | [ts-encoding](#ts-encoding) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | -| IGNORE_MAX_TIME_DIFF | [ts-ignore-max-time-diff](#ts-ignore-max-time-diff-and-ts-ignore-max-val-diff) | :white_check_mark: ||| -| IGNORE_MAX_VAL_DIFF | [ts-ignore-max-val-diff](#ts-ignore-max-time-diff-and-ts-ignore-max-val-diff) | :white_check_mark: ||| -| NUM_THREADS | [ts-num-threads](#ts-num-threads) | :white_large_square: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | -| RETENTION_POLICY | [ts-retention-policy](#ts-retention-policy) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | -| OSS_GLOBAL_PASSWORD | Deprecated in v8.0.0. Replace with the `masterauth` password. | :white_check_mark: ||| - -{{< note >}} -Parameter names for Redis CE versions < 8.0, while deprecated, will still be supported in version 8.0. -{{< /note >}} +| CHUNK_SIZE_BYTES | [ts-chunk-size-bytes](#chunk_size_bytes--ts-chunk-size-bytes) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | +| COMPACTION_POLICY | [ts-compaction-policy](#compaction_policy--ts-compaction-policy) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | +| DUPLICATE_POLICY | [ts-duplicate-policy](#duplicate_policy--ts-duplicate-policy) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | +| RETENTION_POLICY | [ts-retention-policy](#retention_policy--ts-retention-policy) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | +| ENCODING | [ts-encoding](#encoding--ts-encoding) | :white_check_mark: | ✅ Supported

 | ✅ Flexible & Annual
❌ Free & Fixed | +| IGNORE_MAX_TIME_DIFF | [ts-ignore-max-time-diff](#ignore_max_time_diff--ts-ignore-max-time-diff-and-ignore_max_val_diff--ts-ignore-max-val-diff) | :white_check_mark: ||| +| IGNORE_MAX_VAL_DIFF | [ts-ignore-max-val-diff](#ignore_max_time_diff--ts-ignore-max-time-diff-and-ignore_max_val_diff--ts-ignore-max-val-diff) | :white_check_mark: ||| +| NUM_THREADS | [ts-num-threads](#num_threads--ts-num-threads) | :white_large_square: | ✅ Supported

 | ❌ Flexible & Annual
❌ Free & Fixed | +| [OSS_GLOBAL_PASSWORD](#oss_global_password) | Deprecated in v8.0 | :white_check_mark: ||| --- -### ts-chunk-size-bytes +### CHUNK_SIZE_BYTES / ts-chunk-size-bytes -Default initial allocation size, in bytes, for the data part of each new chunk. -This default value is applied to each new time series upon its creation. -Actual chunks may consume more memory. +The initial allocation size, in bytes, for the data part of each new chunk. Actual chunks may consume more memory. +Changing this value does not affect existing chunks. Type: integer Valid range: `[48 .. 1048576]`; must be a multiple of 8 -Default: `4096` +#### Precedence order -### ts-compaction-policy +Because the chunk size can be provided at different levels, the actual precedence of the chunk size is: -Default compaction rules for newly created keys with [`TS.ADD`]({{< relref "/commands/ts.add/" >}}), [`TS.INCRBY`]({{< relref "/commands/ts.incrby/" >}}), and [`TS.DECRBY`]({{< relref "/commands/ts.decrby/" >}}). +1. Key-level policy, as set with [`TS.CREATE`]({{< relref "/commands/ts.create/" >}})'s and [`TS.ALTER`]({{< relref "/commands/ts.alter/" >}})'s `CHUNK_SIZE` optional argument. +1. The `ts-chunk-size-bytes` configuration parameter. +1. The hard-coded default: `4096` -Type: string +#### Example -Default: No compaction rules. +Set the default chunk size to 1024 bytes: -**Discussion** +Version < 8.0: -#### Example +``` +$ redis-server --loadmodule ./redistimeseries.so CHUNK_SIZE_BYTES 1024 +``` + +Version >= 8.0: ``` -$ redis-server --loadmodule ./redistimeseries.so NUM_THREADS 3 +redis> CONFIG SET ts-chunk-size-bytes 1024 ``` -### COMPACTION_POLICY +### COMPACTION_POLICY / ts-compaction-policy -Default compaction rules for newly created key with [`TS.ADD`]({{< relref "commands/ts.add/" >}}), [`TS.INCRBY`]({{< relref "commands/ts.incrby/" >}}), and [`TS.DECRBY`]({{< relref "commands/ts.decrby/" >}}). +Default compaction rules for newly created keys with [`TS.ADD`]({{< relref "/commands/ts.add/" >}}), [`TS.INCRBY`]({{< relref "/commands/ts.incrby/" >}}), and [`TS.DECRBY`]({{< relref "/commands/ts.decrby/" >}}). + +Type: string -Note that `COMPACTION_POLICY` has no effect on keys created with [`TS.CREATE`]({{< relref "commands/ts.create/" >}}). To understand the motivation for this behavior, consider the following scenario: Suppose a `COMPACTION_POLICY` is defined, but then one wants to manually create an additional compaction rule (using [`TS.CREATERULE`]({{< relref "commands/ts.createrule/" >}})) which requires first creating an empty destination key (using [`TS.CREATE`]({{< relref "commands/ts.create/" >}})). But now there is a problem: due to the `COMPACTION_POLICY`, automatic compactions would be undesirably created for that destination key. +Note that this configuration parameter does not affect keys you create with [`TS.CREATE`]({{< relref "commands/ts.create/" >}}). To understand why, consider the following scenario: Suppose you define a default compaction policy but then want to manually create an additional compaction rule (using [`TS.CREATERULE`]({{< relref "commands/ts.createrule/" >}})), which requires you to first create an empty destination key (using `TS.CREATE`). This approach creates a problem: the default compaction policy would cause Redis to automatically create undesired compactions for the destination key. Each rule is separated by a semicolon (`;`), the rule consists of multiple fields that are separated by a colon (`:`): @@ -124,13 +162,13 @@ Each rule is separated by a semicolon (`;`), the rule consists of multiple field * h - hour * d - day - Assure that there is a bucket that starts at exactly _alignTimestamp_ after the epoch and align all other buckets accordingly. Default value: 0 (aligned with the epoch). Example: if _bucketDuration_ is 24 hours, setting _alignTimestamp_ to `6h` (6 hours after the Epoch) will ensure that each bucket’s timeframe is [06:00 .. 06:00). + Ensure that there is a bucket that starts at exactly _alignTimestamp_ after the Epoch and align all other buckets accordingly. Default value: 0 (aligned with the Epoch). Example: if _bucketDuration_ is 24 hours, setting _alignTimestamp_ to `6h` (6 hours after the Epoch) will ensure that each bucket’s timeframe is [06:00 .. 06:00). {{% warning %}} -In a clustered environment, if you set `COMPACTION_POLICY`, you must use [hash tags]({{< relref "/operate/oss_and_stack/reference/cluster-spec" >}}#hash-tags) for all time series key names. This ensures that Redis will create each compaction in the same hash slot as its source key. If you don't, the system may fail to compact the data without displaying any error messages. +In a clustered environment, if you set this configuration parameter, you must use [hash tags]({{< relref "/operate/oss_and_stack/reference/cluster-spec" >}}#hash-tags) for all time series key names. This ensures that Redis will create each compaction in the same hash slot as its source key. If you don't, the system may fail to compact the data without displaying any error messages. {{% /warning %}} -When a compaction policy is defined, compaction rules will be created automatically for newly created time series, and their key would be set to: +When a compaction policy is defined, compaction rules are created automatically for newly created time series, and the compaction key name would be: * If the time bucket alignment is 0: @@ -140,46 +178,35 @@ When a compaction policy is defined, compaction rules will be created automatica _key_agg_dur_aln_ where _key_ is the key of the source time series, _agg_ is the aggregator (in uppercase), _dur_ is the bucket duration in milliseconds, and _aln_ is the time bucket alignment in milliseconds. Example: `key_SUM_60000_1000`. -**Examples** +#### Precedence order + +1. The `ts-compaction-policy` configuration parameter. +1. No compaction rules. + +#### Example rules - `max:1M:1h` - Aggregate using `max` over one-minute windows and retain the last hour - `twa:1d:0m:360M` - Aggregate daily [06:00 .. 06:00) using `twa`; no expiration -### ts-duplicate-policy +#### Example -The default policy for handling insertion ([`TS.ADD`]({{< relref "/commands/ts.add/" >}}) and [`TS.MADD`]({{< relref "/commands/ts.madd/" >}})) of multiple samples with identical timestamps, with one of the following values: +Set a compaction policy composed of 5 compaction rules: -#### Example +Version < 8.0: ``` $ redis-server --loadmodule ./redistimeseries.so COMPACTION_POLICY max:1m:1h;min:10s:5d:10d;last:5M:10m;avg:2h:10d;avg:3d:100d ``` -### RETENTION_POLICY - -Default retention period, in milliseconds, for newly created keys. - -Retention period is the maximum age of samples compared to highest reported timestamp, per key. Samples are expired based solely on the difference between their timestamp and the timestamps passed to subsequent [`TS.ADD`]({{< relref "commands/ts.add/" >}}), [`TS.MADD`]({{< relref "commands/ts.madd/" >}}), [`TS.INCRBY`]({{< relref "commands/ts.incrby/" >}}), and [`TS.DECRBY`]({{< relref "commands/ts.decrby/" >}}) calls. - -The value `0` means no expiration. - -When both `COMPACTION_POLICY` and `RETENTION_POLICY` are specified, the retention of newly created compactions is according to the retention time specified in `COMPACTION_POLICY`. - -#### Default - -0 - -#### Example - -Setting the default retention to 300 days: +Version >= 8.0: ``` -$ redis-server --loadmodule ./redistimeseries.so RETENTION_POLICY 25920000000 +redis> CONFIG SET ts-compaction-policy max:1m:1h;min:10s:5d:10d;last:5M:10m;avg:2h:10d;avg:3d:100d ``` -### DUPLICATE_POLICY +### DUPLICATE_POLICY / ts-duplicate-policy -Is policy for handling insertion ([`TS.ADD`]({{< relref "commands/ts.add/" >}}) and [`TS.MADD`]({{< relref "commands/ts.madd/" >}})) of multiple samples with identical timestamps, with one of the following values: +The default policy for handling insertion ([`TS.ADD`]({{< relref "/commands/ts.add/" >}}) and [`TS.MADD`]({{< relref "/commands/ts.madd/" >}})) of multiple samples with identical timestamps, with one of the following values: | policy | description | | ---------- | ---------------------------------------------------------------- | @@ -194,30 +221,85 @@ The default value is applied to each new time series upon its creation. Type: string -Default: `BLOCK` +#### Precedence order -**Precedence order** - -Since the duplication policy can be provided at different levels, the actual precedence of the used policy will be: +Because the duplication policy can be provided at different levels, the actual precedence of the duplication policy is: 1. [`TS.ADD`]({{< relref "/commands/ts.add/" >}})'s `ON_DUPLICATE_POLICY` optional argument. 1. Key-level policy, as set with [`TS.CREATE`]({{< relref "/commands/ts.create/" >}})'s and [`TS.ALTER`]({{< relref "/commands/ts.alter/" >}})'s `DUPLICATE_POLICY` optional argument. 1. The `ts-duplicate-policy` configuration parameter. -1. The default policy. +1. The hard-coded default: `BLOCK` + +### RETENTION_POLICY / ts-retention-policy + +The default retention period, in milliseconds, for newly created keys. + +The retention period is the maximum age of samples compared to the highest reported timestamp, per key. Samples are expired based solely on the difference between their timestamps and the timestamps passed to subsequent [`TS.ADD`]({{< relref "commands/ts.add/" >}}), [`TS.MADD`]({{< relref "commands/ts.madd/" >}}), [`TS.INCRBY`]({{< relref "commands/ts.incrby/" >}}), and [`TS.DECRBY`]({{< relref "commands/ts.decrby/" >}}) calls. + +Type: integer + +Valid range: `[0 .. 9,223,372,036,854,775,807]` + +The value `0` means no expiration. + +When both `COMPACTION_POLICY` / `ts-compaction-policy` and `RETENTION_POLICY` / `ts-retention-policy` are specified, the retention of newly created compactions is according to the retention time specified in `COMPACTION_POLICY` / `ts-compaction-policy`. + +#### Precedence order -### ts-encoding +Because the retention can be provided at different levels, the actual precedence of the retention is: + +1. Key-level retention, as set with [`TS.CREATE`]({{< relref "/commands/ts.create/" >}})'s and [`TS.ALTER`]({{< relref "/commands/ts.alter/" >}})'s `RETENTION` optional argument. +1. The `ts-retention-policy` configuration parameter. +1. No retention. + +#### Example + +Set the default retention to 300 days: + +Version < 8.0: + +``` +$ redis-server --loadmodule ./redistimeseries.so RETENTION_POLICY 25920000000 +``` + +Version >= 8.0: + +``` +redis> CONFIG SET ts-retention-policy 25920000000 +``` + +### ENCODING / ts-encoding Note: Before v1.6 this configuration parameter was named `CHUNK_TYPE`. -Default chunk encoding for automatically created time series keys when [ts-compaction-policy](#ts-compaction-policy) is configured. +Default chunk encoding for automatically created compactions when [ts-compaction-policy](#ts-compaction-policy) is configured. Type: string Valid values: `COMPRESSED`, `UNCOMPRESSED` -Default: `COMPRESSED` +#### Precedence order -### ts-ignore-max-time-diff and ts-ignore-max-val-diff +1. The `ts-encoding` configuration parameter. +1. The hard-coded default: `COMPRESSED` + +#### Example + +Set the default encoding to `UNCOMPRESSED`: + +Version < 8.0: + +``` +$ redis-server --loadmodule ./redistimeseries.so ENCODING UNCOMPRESSED +``` + +Version >= 8.0: + +``` +redis> CONFIG SET ts-encoding UNCOMPRESSED +``` + +### IGNORE_MAX_TIME_DIFF / ts-ignore-max-time-diff and IGNORE_MAX_VAL_DIFF / ts-ignore-max-val-diff Default values for newly created keys. @@ -229,12 +311,6 @@ Valid ranges: - `ts-ignore-max-time-diff`: `[0 .. 9,223,372,036,854,775,807]` - `ts-ignore-max-val-diff`: `[0 .. 1.7976931348623157e+308]` -Defaults: -- `ts-ignore-max-time-diff`: 0 -- `ts-ignore-max-val-diff`: 0.0 - -**Discussion** - Many sensors report data periodically. Often, the difference between the measured value and the previous measured value is negligible and related to random noise or to measurement accuracy limitations. In such situations it may be preferable not to add the new measurement to the time series. A new sample is considered a duplicate and is ignored if the following conditions are met: @@ -247,58 +323,56 @@ A new sample is considered a duplicate and is ignored if the following condition where `max_timestamp` is the timestamp of the sample with the largest timestamp in the time series, and `value_at_max_timestamp` is the value at `max_timestamp`. -### ts-num-threads +#### Precedence order -The maximum number of per-shard threads for cross-key queries when using cluster mode ([`TS.MRANGE`]({{< relref "/commands/ts.mrange/" >}}), [`TS.MREVRANGE`]({{< relref "/commands/ts.mrevrange/" >}}), [`TS.MGET`]({{< relref "/commands/ts.mget/" >}}), and [`TS.QUERYINDEX`]({{< relref "/commands/ts.queryindex/" >}})). The value must be equal to or greater than `1`. Note that increasing this value may either increase or decrease the performance! +1. The `ts-ignore-max-time-diff` and `ts-ignore-max-val-diff` configuration parameters. +1. The hard-coded defaults: `0` and `0.0`. -Type: integer +#### Example -Valid range: `[1..16]` +Version < 8.0: -Redis CE default: `3` +``` +$ redis-server --loadmodule ./redistimeseries.so IGNORE_MAX_TIME_DIFF 10 IGNORE_MAX_VAL_DIFF 0.1 +``` -Redis Software default: Set by plan, and automatically updates when you change your plan. +Version >= 8.0: -Redis Cloud defaults: -- Flexible & Annual: Set by plan -- Free & Fixed: `1` +``` +redis> CONFIG SET ts-ignore-max-time-diff 10 ts-ignore-max-val-diff 0.1 +``` -### ts-retention-policy +### NUM_THREADS / ts-num-threads -Default retention period, in milliseconds, for newly created keys. +The maximum number of per-shard threads for cross-key queries when using cluster mode ([`TS.MRANGE`]({{< relref "/commands/ts.mrange/" >}}), [`TS.MREVRANGE`]({{< relref "/commands/ts.mrevrange/" >}}), [`TS.MGET`]({{< relref "/commands/ts.mget/" >}}), and [`TS.QUERYINDEX`]({{< relref "/commands/ts.queryindex/" >}})). The value must be equal to or greater than `1`. Note that increasing this value may either increase or decrease the performance! Type: integer -Valid range: `[0 .. 9,223,372,036,854,775,807]` - -Default: `0` - -Retention period is the maximum age of samples compared to highest reported timestamp, per key. Samples are expired based solely on the difference between their timestamps and the timestamps passed to subsequent [`TS.ADD`]({{< relref "/commands/ts.add/" >}}), [`TS.MADD`]({{< relref "/commands/ts.madd/" >}}), [`TS.INCRBY`]({{< relref "/commands/ts.incrby/" >}}), and [`TS.DECRBY`]({{< relref "/commands/ts.decrby/" >}}) calls. - -The value `0` means no expiration. +Valid range: `[1..16]` -When both `ts-compaction-policy` and `ts-retention-policy` are specified, the retention of newly created compactions is according to the retention time specified in `ts-compaction-policy`. +Redis CE default: `3` -## Setting configuration parameters on module load (deprecated) +Redis Software default: Set by plan, and automatically updates when you change your plan. -These methods are deprecated beginning with Redis CE v8.0.0. +Redis Cloud defaults: +- Flexible & Annual: Set by plan +- Free & Fixed: `1` -Setting configuration parameters at load-time is done by appending arguments after the `--loadmodule` argument when starting a server from the command line or after the `loadmodule` directive in a Redis config file. For example: +#### Example -In [redis.conf]({{< relref "/operate/oss_and_stack/management/config" >}}): +Version < 8.0: -```sh -loadmodule ./redistimeseries.so [OPT VAL]... +``` +$ redis-server --loadmodule ./redistimeseries.so NUM_THREADS 3 ``` -From the [Redis CLI]({{< relref "/develop/tools/cli" >}}), using the [MODULE LOAD]({{< relref "/commands/module-load" >}}) command: +Version >= 8.0: ``` -127.0.0.6379> MODULE LOAD redistimeseries.so [OPT VAL]... +redis> redis-server --loadmodule ./redistimeseries.so ts-num-threads 3 ``` -From the command line: -```sh -$ redis-server --loadmodule ./redistimeseries.so [OPT VAL]... -``` \ No newline at end of file +### OSS_GLOBAL_PASSWORD + +Prior to version 8.0, when using time series in a cluster, you had to set the `OSS_GLOBAL_PASSWORD` configuration parameter on all cluster nodes. As of version 8.0, Redis no longer uses this parameter and ignores it if present. Redis now uses a new shared secret mechanism to send internal commands between cluster nodes.