object-expiration-rules-52e2c18.md

Object Expiration Rules

Object Expiration allows you to schedule removal of your objects after a defined time period. It helps define rules to expire objects within a bucket based on which Amazon S3 deletes expired objects.

Object Store service allows you to use Amazon's Object Lifecycle Management feature for setting expiration rules for objects. To know more about expiration feature, please refer to Amazon's documentation on Object Lifecycle Management.

Default configurations

A default expiration rule is set that ensures that all non-current versions are deleted after 14 days.

How to modify default lifecycle configurations

The default settings can be modified via instance update operation. It is advised to fetch the existing lifecycle rule using get-service operation and modify the same. Also, it is recommended to send the complete rule in the instance update operation.

List of supported configuration parameters for modifying rules

Parameter	Type	Mandatory	Description
lifecycle Configuration	array of lifecycle configuration objects	yes	A list of rules to be configured on a service instance. Must contain at least one rule.

LifecycleConfiguration object has the following parameters:

Parameter	Type	Mandatory	Description
id	String	no	ID of the lifecycle rule configured for the storage space. Consists of at most 255 bytes. If not specified, or the value is empty, OSS will automatically generate a unique ID.
status	String	yes	Indicates the state the lifecycle rule is in. Value can be "Enabled" or "Disabled"(by default). If the rule is disabled, OSS will not perform any operations defined in the rule.
expirationInDays	Number	yes	Specify that the Object expires the storage type after N days of its last modification time.
expirationDate	Date	no	Specify an absolute date, and execute the expiration of storage type operation according to the Object whose last modification time is before this date.
filter	Object	yes	If specified, refers to the relative expiration time, indicating the retention period between the current version becoming a non-current version and the permanent deletion.
noncurrentVersionExpiration	Object	yes	If specified, refers to the relative expiration time, indicating the retention period between the current version becoming a non-current version and the permanent deletion.

Note:

A valid LifecycleConfiguration object must have either expirationInDays or noncurrentVersionExpiration, or both.

Filter object has two parameters: predicate and and.

Parameter	Type	Mandatory	Description
Predicate	Predicate object	no	Each S3 Lifecycle rule includes a filter that you can use to identify a subset of objects in your bucket to which the S3 Lifecycle rule applies
and	And object	no	Each S3 Lifecycle rule includes a filter that you can use to identify a subset of objects in your bucket to which the S3 Lifecycle rule applies

Predicate object has the following parameters:

Parameter	Type	Mandatory	Description
Prefix	String	no	The rule will be applied to the specified object that matches the prefix.
tag	Object	no	The rule will be applied to the objects that matches the tag value.

And object has the following parameters:

Parameter	Type	Mandatory	Description
prefix	String	no	The rule will be applied to the specified object that matches the prefix.
tags	Object	no	The rule will be applied to the objects that matches the tag value.

Note:

Filter object can have a predicate parameter or an and parameter.

Tag object has the following parameters:

Parameter	Type	Mandatory	Description
key	String	Yes	Key of the tag element
value	String	Yes	Value of the tag element

Tags object is a list and inside that it can have multiple key and value pairs.

NoncurrentVersionExpiration object has the following parameter:

Parameter	Type	Mandatory	Description
days	Number	Yes	Days after which the non-current object will be expired
newerNoncurrentVersions	Number	No	Number of non-current versions to retain

Predicate: If the predicate parameter is used, then either the prefix or the tag can be specified, but not both. Refer to the example below.

Example with prefix:

Sample Code:

{
  "autoExpiration": [
    {
      "id": "logs-rule",
      "status": "Enabled",
      "filter":{
        "predicate":{
            "prefix":"logs"
         }
      },
      "expirationInDays": 90,
      "expirationDate": null,
      "noncurrentVersionExpiration": {
        "days": 90,
        "newerNoncurrentVersions": 1
      },
      "abortIncompleteMultipartUpload": null
    }
  ]
}

Example with Tag:

Sample Code:

{
  "autoExpiration": [
    {
      "id": "logs-rule",
      "status": "Enabled",
      "filter":{
        "predicate":{
            "tag": {
              "key": "key1",
              "value": "value1"
            }
         }
      },
      "expirationInDays": 90,
      "expirationDate": null,
      "noncurrentVersionExpiration": {
        "days": 90,
        "newerNoncurrentVersions": 1
      },
      "abortIncompleteMultipartUpload": null
    }
  ]
}

Note:

Do not use both a prefix and a tag together inside a predicate as it may cause unexpected behaviour.

And: If the and parameter is used, then at most one prefix and any number of tags can be specified. Refer to the example below.

Sample Code:

{
  "autoExpiration": [
    {
      "id": "logs-rule",
      "status": "Enabled",
      "filter":{
        "and": {
          "prefix": "logs",
          "tags": [
            {
              "key": "key1",
              "value": "value1"
            },
            {
              "key": "key1",
              "value": "value1"
            }
          ]
        }
      },
      "expirationInDays": 90,
      "expirationDate": null,
      "noncurrentVersionExpiration": {
        "days": 90,
        "newerNoncurrentVersions": 1
      },
      "abortIncompleteMultipartUpload": null
    }
  ]
}

Example:

Sample Code:

cf update-service <service_instance_name> -c SampleRule.json

SampleRule.json

Sample Code:

{
  "autoExpiration": [
    {
      "id": "SampleRule1",
      "status": "Enabled",
      "filter":{
        "predicate":{
            "prefix":"test/"
         }
      },
      "expirationInDays": 90,
      "expirationDate": null,
      "noncurrentVersionExpiration": {
        "days": 90,
        "newerNoncurrentVersions": 1
      },
      "abortIncompleteMultipartUpload": null
    }
  ]
}

How to get existing lifecycle rule?

Sample Code:

cf service <service_instance_name> --params

List of supported configuration parameters

Parameter	Type	Mandatory	Description
autoExpiration	Array of Rule Objects	no	Existing rule set on the bucket

Deleting configured Lifecycle rules

Objectstore Service now allows deleting Lifecycle rules that have been configured using Amazon's Object Lifecycle Management.

SUPPORTED CONFIGURATION PARAMETERS

Parameter	Type	Mandatory	Description
deleteAutoExpiration	String Array of Rule Ids	yes	Existing rules set on the bucket

Example:

Sample Code:

cf update-service <service_instance_name> -c SampleDeleteRule.json

SampleDeleteRule.json

Sample Code:

{
    "deleteAutoExpiration": [
        "SampleRule1",
        "SampleRule2"
    ]
}

Note:

• Do not pass any other parameter in the update call along with deleteAutoExpiration, as we do not allow any other operation along with this.

• It is advised to fetch the existing lifecycle rule(s) using get-service operation and then provide us with the latest rule id(s) that are to be deleted in the above format.

• If any of the rule id(s) provided do not exist in the bucket, then the request will fail with the following error:

  Following rule identifiers passed: [SampleRule3, SampleRule4] is/are incorrect. Please do the instance GET operation to fetch the existing rule identifiers.
  

• If deleting a lifecycle rule is attempted when there are no rules present in the bucket, the request will fail with the following error:

  No Existing lifecycle configurations present in IaaS to delete.
  

Note:

• Transitioning objects to other storage classes is not supported.
• Object Store service does not support transitioning objects to other storage classes (other than Standard storage class). Even though it is technically still possible to set transition rules, we highly discourage you to set such rules.
• If objects are transitioned to other storage classes, users will not be able to retrieve those back.
• Also, if objects are transitioned to other storage classes by setting transition rules, the storage cost will not be reduced from what is charged for standard storage class currently.

How to handle an error while updating a lifecycle expiration rule

You may see the following error while updating lifecycle on your AWS instance:

Filter element can only be used in Lifecycle V2. (Service: Amazon S3; Status Code: 400; Error Code: InvalidRequest

Why does this error occur?

If you previously set an expiration rule using Lifecycle V1 (when you had permission to directly set the rule through AWS S3 CLI/SDK/API) and now you are trying to update the rule through the update-instance operation, then you would see this error.

The reason is that, when you request the rule modification through update-service, the Object Store service uses Lifecycle V2 (the newer version). Therefore, the conflict between V1 and V2 happens.

This is a valid error message from AWS. You must adapt your lifecycle configuration to the new format, using one of the following options:

1. Suggested option: Fetch and delete all existing rules using the service instance update operation. Then apply the desired rules with the V2 format of lifecycle rules again through service instance operation.
2. Workaround: Setting a rule through update-service call using V1 lifecycle will also work, as long as some of the properties that were introduced with V2 lifecycle are not used in the update-service call.

An example of a V1 lifecycle expiration rule that will work through update-service appears below:

Sample Code:

"autoExpiration": [ { "abortIncompleteMultipartUpload": null, "expirationDate": null, "expirationInDays": -1, "expiredObjectDeleteMarker": false, "filter": null, "id": "Example-id", "noncurrentVersionExpiration": { "days": 15, "newerNoncurrentVersions": -1 }, "prefix": "", "status": "Enabled" } ]