AssertionError: Starting restore 2024-05-08T12:31:35.961157697Z to point-in-time 2024-05-08T12:30:03 from '2024-05-08T12:28:58Z'...Started logical restore. Waiting to finish...Error: operation failed with: reply oplog: replay chunk 1715171346.1715171401: apply oplog for chunk: applying a transaction entry: apply txn: { "Timestamp": { "T": 1715171371, "I": 18 }, "Term": 1, "Hash": null, "Version": 2, "Operation": "c", "Namespace": "admin.$cmd", "Object": [ { "Key": "applyOps", "Value": [ [ { "Key": "op", "Value": "c" }, { "Key": "ns", "Value": "config.$cmd" }, { "Key": "ui", "Value": { "Subtype": 4, "Data": "tarb5SVqT5+GyZftH4KEzw==" } }, { "Key": "o", "Value": [ { "Key": "create", "Value": "databases" }, { "Key": "idIndex", "Value": [ { "Key": "v", "Value": 2 }, { "Key": "key", "Value": [ { "Key": "_id", "Value": 1 } ] }, { "Key": "name", "Value": "_id_" } ] } ] } ], [ { "Key": "op", "Value": "i" }, { "Key": "ns", "Value": "config.databases" }, { "Key": "ui", "Value": { "Subtype": 4, "Data": "tarb5SVqT5+GyZftH4KEzw==" } }, { "Key": "o", "Value": [ { "Key": "_id", "Value": "test" }, { "Key": "primary", "Value": "rs1" }, { "Key": "partitioned", "Value": false }, { "Key": "version", "Value": [ { "Key": "uuid", "Value": { "Subtype": 4, "Data": "kMiyCz3DSb+o6f0SjZDh4Q==" } }, { "Key": "timestamp", "Value": { "T": 1715171371, "I": 16 } }, { "Key": "lastMod", "Value": 1 } ] } ] }, { "Key": "o2", "Value": [ { "Key": "_id", "Value": "test" } ] } ], [ { "Key": "op", "Value": "i" }, { "Key": "ns", "Value": "config.placementHistory" }, { "Key": "ui", "Value": { "Subtype": 4, "Data": "blFVmTvZQHqshZPtYgS2KA==" } }, { "Key": "o", "Value": [ { "Key": "_id", "Value": "663b702bf1843dade7b058c3" }, { "Key": "nss", "Value": "test" }, { "Key": "timestamp", "Value": { "T": 1715171371, "I": 16 } }, { "Key": "shards", "Value": [ "rs1" ] } ] }, { "Key": "o2", "Value": [ { "Key": "_id", "Value": "663b702bf1843dade7b058c3" } ] } ] ] } ], "Query": null, "UI": null, "LSID": "ZgAAAAVpZAAQAAAABI5CKSRZyEqviuHLwo9Bp4MFdWlkACAAAAAAY5mrDaxi8gv8RmdTsQ+1j7fmkr7JUsabhNmXAheU0fgFdHhuVVVJRAAQAAAABF03VgUwQUyCp1pnOPCgokQA", "TxnNumber": 0, "PrevOpTime": "HAAAABF0cwAAAAAAAAAAABJ0AP//////////AA==" }: applying transaction op: op: {"Timestamp":{"T":1715171371,"I":18},"Term":1,"Hash":null,"Version":0,"Operation":"i","Namespace":"config.databases","Object":[{"Key":"_id","Value":"test"},{"Key":"primary","Value":"rs1"},{"Key":"partitioned","Value":false},{"Key":"version","Value":[{"Key":"uuid","Value":{"Subtype":4,"Data":"kMiyCz3DSb+o6f0SjZDh4Q=="}},{"Key":"timestamp","Value":{"T":1715171371,"I":16}},{"Key":"lastMod","Value":1}]}],"Query":[{"Key":"_id","Value":"test"}],"UI":{"Subtype":4,"Data":"tarb5SVqT5+GyZftH4KEzw=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (UnknownError) applyOps had one or more errors applying ops

AssertionError: Error: backup '2024-05-08T12:35:16Z' didn't finish successfully

AssertionError: Starting restore 2024-05-08T12:54:43.921246881Z to point-in-time 2024-05-08T12:54:13 from '2024-05-08T12:53:24Z'...Started logical restore. Waiting to finish....Error: operation failed with: reply oplog: replay chunk 1715172851.1715172881: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715172852,"I":3},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"test.system.views","Object":[{"Key":"_id","Value":"test.test2"},{"Key":"viewOn","Value":"system.buckets.test2"},{"Key":"pipeline","Value":[[{"Key":"$_internalUnpackBucket","Value":[{"Key":"timeField","Value":"timestamp"},{"Key":"metaField","Value":"data"},{"Key":"bucketMaxSpanSeconds","Value":3600}]}]]}],"Query":[{"Key":"_id","Value":"test.test2"}],"UI":{"Subtype":4,"Data":"E45Qva0US++06cARq2gBjA=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (UnknownError) applyOps had one or more errors applying ops

AssertionError: Starting restore 2024-05-08T13:04:19.845508139Z from '2024-05-08T13:03:50Z'...Started logical restore. Waiting to finish....Error: operation failed with: waiting for dumpDone: cluster failed: waiting for dumpDone: cluster failed: reply oplog: replay chunk 1715173430.1715173440: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715173430,"I":7},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"test.system.buckets.test2","Object":[{"Key":"_id","Value":"663b7804f8d25b23bf201ca4"},{"Key":"control","Value":[{"Key":"version","Value":1},{"Key":"min","Value":[{"Key":"_id","Value":"663b78363276785bad529aff"},{"Key":"timestamp","Value":"2024-05-08T13:03:00Z"}]},{"Key":"max","Value":[{"Key":"_id","Value":"663b78363276785bad529aff"},{"Key":"timestamp","Value":"2024-05-08T13:03:50.375Z"}]}]},{"Key":"meta","Value":8},{"Key":"data","Value":[{"Key":"timestamp","Value":[{"Key":"0","Value":"2024-05-08T13:03:50.375Z"}]},{"Key":"_id","Value":[{"Key":"0","Value":"663b78363276785bad529aff"}]}]}],"Query":[{"Key":"meta","Value":8},{"Key":"_id","Value":"663b7804f8d25b23bf201ca4"}],"UI":null,"LSID":"SAAAAAVpZAAQAAAABMbiw0r8KkvFklBEtviZaLMFdWlkACAAAAAAY5mrDaxi8gv8RmdTsQ+1j7fmkr7JUsabhNmXAheU0fgA","TxnNumber":1,"PrevOpTime":"HAAAABF0cwAAAAAAAAAAABJ0AP//////////AA=="} | merr <nil>: applyOps: (NamespaceNotFound) cannot apply insert or update operation on a non-existent namespace test.system.buckets.test2: { ts: Timestamp(1715173430, 7), t: 1, v: 2, op: "i", ns: "test.system.buckets.test2", o: { _id: ObjectId('663b7804f8d25b23bf201ca4'), control: { version: 1, min: { _id: ObjectId('663b78363276785bad529aff'), timestamp: new Date(1715173380000) }, max: { _id: ObjectId('663b78363276785bad529aff'), timestamp: new Date(1715173430375) } }, meta: 8, data: { timestamp: { 0: new Date(1715173430375) }, _id: { 0: ObjectId('663b78363276785bad529aff') } } }, o2: { meta: 8, _id: ObjectId('663b7804f8d25b23bf201ca4') }, lsid: { id: UUID("c6e2c34a-fc2a-4bc5-9250-44b6f89968b3"), uid: BinData(0, 6399AB0DAC62F20BFC466753B10FB58FB7E692BEC952C69B84D997021794D1F8) }, txnNumber: 1, prevOpTime: { ts: Timestamp(0, 0), t: -1 } }

pymongo.errors.OperationFailure: got stale shardVersion response from shard rs2 at host rs201:27017 :: caused by :: timestamp mismatch detected for test.test2, full error: {'ok': 0.0, 'errmsg': 'got stale shardVersion response from shard rs2 at host rs201:27017 :: caused by :: timestamp mismatch detected for test.test2', 'code': 13388, 'codeName': 'StaleConfig', 'ns': 'test.test2', 'vReceived': {'e': ObjectId('663b7877c9e97893757db7cd'), 't': Timestamp(1715173495, 59), 'v': Timestamp(1, 1)}, 'vWanted': {'e': ObjectId('000000000000000000000000'), 't': Timestamp(0, 0), 'v': Timestamp(0, 0)}, 'shardId': 'rs2', '$clusterTime': {'clusterTime': Timestamp(1715173539, 22), 'signature': {'hash': b"\x00\x02\xad\xc6\xb5'\x94\xb5\xb9\x0b\x87\xcb\xe2Y\x9d$\xf1#c6", 'keyId': 7366613982091673623}}, 'operationTime': Timestamp(1715173539, 22)}

AssertionError: Starting restore 2024-05-08T12:31:40.490820348Z to point-in-time 2024-05-08T12:30:08 from '2024-05-08T12:29:03Z'...Started logical restore. Waiting to finish...Error: operation failed with: reply oplog: replay chunk 1715171352.1715171406: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715171377,"I":2},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"config.databases","Object":[{"Key":"_id","Value":"test"},{"Key":"primary","Value":"rs1"},{"Key":"partitioned","Value":true},{"Key":"version","Value":[{"Key":"uuid","Value":{"Subtype":4,"Data":"QScUTEVGTTmmHzZjuvvVQg=="}},{"Key":"timestamp","Value":{"T":1715171376,"I":16}},{"Key":"lastMod","Value":1}]}],"Query":null,"UI":{"Subtype":4,"Data":"+iQBsZWKSFGtx/RNYzj+5g=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (NamespaceNotFound) cannot apply insert or update operation on a non-existent namespace config.databases: { ts: Timestamp(1715171377, 2), t: 1, v: 2, op: "i", ns: "config.databases", o: { _id: "test", primary: "rs1", partitioned: true, version: { uuid: UUID("4127144c-4546-4d39-a61f-3663bafbd542"), timestamp: Timestamp(1715171376, 16), lastMod: 1 } }, ui: UUID("fa2401b1-958a-4851-adc7-f44d6338fee6") }

AssertionError: Starting restore 2024-05-08T12:56:04.576983391Z to point-in-time 2024-05-08T12:55:34 from '2024-05-08T12:54:44Z'...Started logical restore. Waiting to finish....Error: operation failed with: reply oplog: replay chunk 1715172932.1715172962: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715172933,"I":2},"Term":2,"Hash":null,"Version":2,"Operation":"i","Namespace":"test.system.views","Object":[{"Key":"_id","Value":"test.test2"},{"Key":"viewOn","Value":"system.buckets.test2"},{"Key":"pipeline","Value":[[{"Key":"$_internalUnpackBucket","Value":[{"Key":"timeField","Value":"timestamp"},{"Key":"metaField","Value":"data"},{"Key":"bucketMaxSpanSeconds","Value":3600},{"Key":"exclude","Value":[]}]}]]}],"Query":null,"UI":{"Subtype":4,"Data":"0Yls8LPkRpGy/d0oV8MZcw=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (NamespaceNotFound) Failed to apply operation due to missing collection (d1896cf0-b3e4-4691-b2fd-dd2857c31973): { ts: Timestamp(1715172933, 2), t: 2, v: 2, op: "i", ns: "test.system.views", o: { _id: "test.test2", viewOn: "system.buckets.test2", pipeline: [ { $_internalUnpackBucket: { timeField: "timestamp", metaField: "data", bucketMaxSpanSeconds: 3600, exclude: [] } } ] }, ui: UUID("d1896cf0-b3e4-4691-b2fd-dd2857c31973"), h: 0, wall: new Date(0) }

AssertionError: Starting restore 2024-05-08T12:32:07.841646048Z to point-in-time 2024-05-08T12:30:24 from '2024-05-08T12:29:19Z'...Started logical restore. Waiting to finish...Error: operation failed with: reply oplog: replay chunk 1715171368.1715171422: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715171393,"I":2},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"config.databases","Object":[{"Key":"_id","Value":"test"},{"Key":"primary","Value":"rs1"},{"Key":"partitioned","Value":false},{"Key":"version","Value":[{"Key":"uuid","Value":{"Subtype":4,"Data":"6kf5W7oVR+CczhvQlyDDXw=="}},{"Key":"timestamp","Value":{"T":1715171392,"I":15}},{"Key":"lastMod","Value":1}]}],"Query":[{"Key":"_id","Value":"test"}],"UI":{"Subtype":4,"Data":"dpcmJeTvTAuASeKnEx8cGg=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (NamespaceNotFound) cannot apply insert or update operation on a non-existent namespace config.databases: { ts: Timestamp(1715171393, 2), t: 1, v: 2, op: "i", ns: "config.databases", o: { _id: "test", primary: "rs1", partitioned: false, version: { uuid: UUID("ea47f95b-ba15-47e0-9cce-1bd09720c35f"), timestamp: Timestamp(1715171392, 15), lastMod: 1 } }, o2: { _id: "test" }, ui: UUID("76972625-e4ef-4c0b-8049-e2a7131f1c1a") }

AssertionError: Error: backup '2024-05-08T12:35:56Z' didn't finish successfully

AssertionError: Starting restore 2024-05-08T12:56:22.282216333Z to point-in-time 2024-05-08T12:55:52 from '2024-05-08T12:55:02Z'...Started logical restore. Waiting to finish...Error: operation failed with: reply oplog: replay chunk 1715172949.1715172979: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715172950,"I":2},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"test.system.views","Object":[{"Key":"_id","Value":"test.test2"},{"Key":"viewOn","Value":"system.buckets.test2"},{"Key":"pipeline","Value":[[{"Key":"$_internalUnpackBucket","Value":[{"Key":"timeField","Value":"timestamp"},{"Key":"metaField","Value":"data"},{"Key":"bucketMaxSpanSeconds","Value":3600}]}]]}],"Query":[{"Key":"_id","Value":"test.test2"}],"UI":{"Subtype":4,"Data":"eAzlZqCYTUaZuL6K9SKhLQ=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (NamespaceNotFound) Failed to apply operation due to missing collection (780ce566-a098-4d46-99b8-be8af522a12d): { ts: Timestamp(1715172950, 2), t: 1, v: 2, op: "i", ns: "test.system.views", o: { _id: "test.test2", viewOn: "system.buckets.test2", pipeline: [ { $_internalUnpackBucket: { timeField: "timestamp", metaField: "data", bucketMaxSpanSeconds: 3600 } } ] }, o2: { _id: "test.test2" }, ui: UUID("780ce566-a098-4d46-99b8-be8af522a12d"), h: 0, wall: new Date(0) }

AssertionError: Starting restore 2024-05-08T13:06:15.148925737Z from '2024-05-08T13:05:49Z'...Started logical restore. Waiting to finish...Error: operation failed with: waiting for dumpDone: cluster failed: reply oplog: replay chunk 1715173549.1715173558: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715173549,"I":4},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"test.system.buckets.test2","Object":[{"Key":"_id","Value":"663b787cebe74bfaac150e25"},{"Key":"control","Value":[{"Key":"version","Value":1},{"Key":"min","Value":[{"Key":"_id","Value":"663b78adf1e40ab090592091"},{"Key":"timestamp","Value":"2024-05-08T13:05:00Z"}]},{"Key":"max","Value":[{"Key":"_id","Value":"663b78adf1e40ab090592091"},{"Key":"timestamp","Value":"2024-05-08T13:05:49.205Z"}]}]},{"Key":"meta","Value":9},{"Key":"data","Value":[{"Key":"timestamp","Value":[{"Key":"0","Value":"2024-05-08T13:05:49.205Z"}]},{"Key":"_id","Value":[{"Key":"0","Value":"663b78adf1e40ab090592091"}]}]}],"Query":[{"Key":"meta","Value":9},{"Key":"_id","Value":"663b787cebe74bfaac150e25"}],"UI":null,"LSID":"SAAAAAVpZAAQAAAABE8z91EyHkVDiHmszLTvFrsFdWlkACAAAAAAY5mrDaxi8gv8RmdTsQ+1j7fmkr7JUsabhNmXAheU0fgA","TxnNumber":1,"PrevOpTime":"HAAAABF0cwAAAAAAAAAAABJ0AP//////////AA=="} | merr <nil>: applyOps: (NamespaceNotFound) cannot apply insert or update operation on a non-existent namespace test.system.buckets.test2: { ts: Timestamp(1715173549, 4), t: 1, v: 2, op: "i", ns: "test.system.buckets.test2", o: { _id: ObjectId('663b787cebe74bfaac150e25'), control: { version: 1, min: { _id: ObjectId('663b78adf1e40ab090592091'), timestamp: new Date(1715173500000) }, max: { _id: ObjectId('663b78adf1e40ab090592091'), timestamp: new Date(1715173549205) } }, meta: 9, data: { timestamp: { 0: new Date(1715173549205) }, _id: { 0: ObjectId('663b78adf1e40ab090592091') } } }, o2: { meta: 9, _id: ObjectId('663b787cebe74bfaac150e25') }, lsid: { id: UUID("4f33f751-321e-4543-8879-acccb4ef16bb"), uid: BinData(0, 6399AB0DAC62F20BFC466753B10FB58FB7E692BEC952C69B84D997021794D1F8) }, txnNumber: 1, prevOpTime: { ts: Timestamp(0, 0), t: -1 } }

pymongo.errors.OperationFailure: got stale shardVersion response from shard rs2 at host rs201:27017 :: caused by :: timestamp mismatch detected for test.test2, full error: {'ok': 0.0, 'errmsg': 'got stale shardVersion response from shard rs2 at host rs201:27017 :: caused by :: timestamp mismatch detected for test.test2', 'code': 13388, 'codeName': 'StaleConfig', 'ns': 'test.test2', 'vReceived': {'t': Timestamp(1715173614, 52), 'e': ObjectId('663b78ee6acd4a0d9fadec90'), 'v': Timestamp(1, 3)}, 'vWanted': {'t': Timestamp(0, 0), 'e': ObjectId('000000000000000000000000'), 'v': Timestamp(0, 0)}, 'shardId': 'rs2', '$clusterTime': {'clusterTime': Timestamp(1715173666, 26), 'signature': {'hash': b"md\x18\xbb\xf2(h\x88\x14Y\xc7\xc8'\x14`\xc8\xd8\xcb\xc1\xb1", 'keyId': 7366614471717945370}}, 'operationTime': Timestamp(1715173666, 26)}

Failed: Timeout >300.0s

AssertionError: Starting restore 2024-05-08T12:36:23.36849817Z to point-in-time 2024-05-08T12:34:51 from '2024-05-08T12:33:46Z'...Started logical restore. Waiting to finish...Error: operation failed with: reply oplog: replay chunk 1715171634.1715171689: apply oplog for chunk: applying an entry: op: {"Timestamp":{"T":1715171660,"I":3},"Term":1,"Hash":null,"Version":2,"Operation":"i","Namespace":"config.databases","Object":[{"Key":"_id","Value":"test"},{"Key":"primary","Value":"rs1"},{"Key":"partitioned","Value":false},{"Key":"version","Value":[{"Key":"uuid","Value":{"Subtype":4,"Data":"aUuKT3BoRcid5CH1PSf2Jg=="}},{"Key":"lastMod","Value":1}]},{"Key":"lastMovedTimestamp","Value":{"T":1715171660,"I":1}}],"Query":null,"UI":{"Subtype":4,"Data":"fncZfJi+QI61Xx8nv8lXFA=="},"LSID":null,"TxnNumber":null,"PrevOpTime":null} | merr <nil>: applyOps: (NamespaceNotFound) cannot apply insert or update operation on a non-existent namespace config.databases: { ts: Timestamp(1715171660, 3), t: 1, v: 2, op: "i", ns: "config.databases", o: { _id: "test", primary: "rs1", partitioned: false, version: { uuid: UUID("694b8a4f-7068-45c8-9de4-21f53d27f626"), lastMod: 1 }, lastMovedTimestamp: Timestamp(1715171660, 1) }, ui: UUID("7e77197c-98be-408e-b55f-1f27bfc95714") }

Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: github/codeql-action/upload-sarif@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.