Generalize expiry based de-duplication, dsmr #1810

tsachiherman · 2024-11-22T15:35:09Z

What ?

Integrate the generic de-deduplication logic into the dsmr

x/dsmr/block.go

x/dsmr/node.go

x/dsmr/node_test.go

tsachiherman · 2024-11-25T18:52:13Z

All sounds like great ideas, thanks!

…

On Mon, Nov 25, 2024, 1:27 PM aaronbuchwald ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In x/dsmr/node.go <#1810 (comment)>: > @@ -172,19 +193,28 @@ func (n *Node[T]) BuildChunk( return chunk, n.storage.AddLocalChunkWithCert(chunk, &chunkCert) } -func (n *Node[T]) BuildBlock(parent Block, timestamp int64) (Block, error) { - if timestamp <= parent.Timestamp { +// BuildBlock(ctx context.Context, parentView state.View, parent *ExecutionBlock) (*ExecutionBlock, *ExecutedBlock, merkledb.View, error) can we remove this comment? ------------------------------ In x/dsmr/node_test.go <#1810 (comment)>: > +func newTestingLog() logging.Logger { + return logging.NewLogger("dsmr") +} we can replacing this with logging.NoLog{} ------------------------------ In x/dsmr/node_test.go <#1810 (comment)>: > +func newTestingTracer(t *testing.T) trace.Tracer { + tracer, err := trace.New(trace.Config{}) + require.NoError(t, err) + return tracer +} we can replace this with trace.Noop ------------------------------ In x/dsmr/node_test.go <#1810 (comment)>: > +type testingChainIndex struct{} + +func (*testingChainIndex) GetExecutionBlock(context.Context, ids.ID) (validitywindow.ExecutionBlock[*ChunkCertificate], error) { + return nil, nil +} + +func newChainIndexer() ChainIndex { + return &testingChainIndex{} +} We should add unit tests for the newly added functionality here. What do you think of the following cases? - skip duplicate chunks in build block - return an error if all available chunks are duplicates in build block - duplicate chunk within a block - duplicate chunk that between the block we are executing and the last accepted block - duplicate chunk inside of the accepted validity window - propagate an error if we fail to fetch a block during de-duplication — Reply to this email directly, view it on GitHub <#1810 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF2OOH6AKK7BDRFH2ESZGB32CNT2VAVCNFSM6AAAAABSJTMZI2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDINJZGMZDKMZVGM> . You are receiving this because you were assigned.Message ID: ***@***.***>

aaronbuchwald · 2024-11-29T17:53:18Z

internal/validitywindow/validitywindow.go

+	// make sure we have no repeats within the block itself.
+	blkTxsIDs := make(map[ids.ID]bool, len(blk.Txs()))
+	for _, tx := range blk.Txs() {
+		id := tx.GetID()
+		if _, has := blkTxsIDs[id]; has {
+			return fmt.Errorf("%w: duplicate in block", ErrDuplicateContainer)
+		}
+		blkTxsIDs[id] = true
+	}


We do this check twice now in chain since we previously handled this within NewExecutionBlock

We should only apply the check once, which do you think is the better place for it? The other addition within NewExecutionBlock is pre-populating the signature job. It would probably be better to remove that from the execution block and handle it in AsyncVerify. One good reason to do this is that handling this within ParseBlock can be a DoS vector.

I don't see why testing this twice is an issue on it's own, and I think that I have a better solution here;
Testing it in NewExecutionBlock is not ideal from my perspective, as it should be construction step ( i.e. no errors ).
How do you feel about the following:
in ExecutionBlock, we would add the following function:

func (b *ExecutionBlock) ValidateDuplicateTransactions() error { if len(b.Txs) != b.txsSet.Len() { ErrDuplicateTx } return nil

than in validitywindow.go VerifyExpiryReplayProtection, we can just call this method:

... if err := blk.ValidateDuplicateTransactions(); err != nil { return err } ...

and remove the test from NewExecutionBlock

x/dsmr/block.go

x/dsmr/node.go

aaronbuchwald · 2024-11-29T18:04:03Z

x/dsmr/node.go

+	if block.Tmstmp <= parentBlock.Tmstmp && parentBlock.Hght > 0 {
+		return ErrTimestampNotMonotonicallyIncreasing
+	}
+


Can we remove this check from this PR?

We should enforce timestamp >= parent timestamp (should allow them to be equal in case a block builder pushes the timestamp ahead of wall clock time for some nodes.

We should not allow the case that a malicious node builds a block with a timestamp that we consider valid less than 1s ahead of our wall clock time, but still ahead of our wall clock time, such that when we build a block on top of it, we fail because the current timestamp is ahead of our local timestamp.

We should update the check applied in BuildBlock imo.

We should also never execute the genesis block, so the check for parentBlock.Hght > 0 should be removed.

aaronbuchwald · 2024-11-29T18:05:35Z

x/dsmr/node_test.go

+	if blk, has := ti.blocks[id]; has {
+		return blk, nil
+	}
+	return nil, nil


Fetching a non-existing block should return an error, can we return database.ErrNotFound if the block is not available?

I find the ux of us using the sentinel database.ErrNotFound experience to be very awkward. This is due to the existing interface definition and not the changes in this PR though.... if we had something like GetExecutionBlock() (ExecutionBlock[T], bool, error) we wouldn't have to introspect the error and could just return an error if error is non-nil and check the bool if it just wasn't there.

joshua-kim · 2024-12-03T18:37:42Z

internal/validitywindow/validitywindow.go

@@ -69,6 +70,15 @@ func (v *TimeValidityWindow[Container]) VerifyExpiryReplayProtection(
 	if dup.Len() > 0 {
 		return fmt.Errorf("%w: duplicate in ancestry", ErrDuplicateContainer)
 	}
+	// make sure we have no repeats within the block itself.
+	blkTxsIDs := make(map[ids.ID]bool, len(blk.Txs()))


Can we use set.Set here?

joshua-kim · 2024-12-03T18:41:44Z

x/dsmr/certificate.go

+func (c ChunkCertificate) GetExpiry() int64 { return c.Expiry }

 func (c *ChunkCertificate) GetSlot() int64 { return c.Expiry }


I think these are duplicated - GetSlot is meant to be Expiry

joshua-kim · 2024-12-03T18:43:57Z

x/dsmr/node.go

+	log logging.Logger,
+	tracer trace.Tracer,
+	chainIndex ChainIndex,
+	validityWindowDuration int64,


Should we use time.Duration here?

joshua-kim · 2024-12-03T18:44:57Z

x/dsmr/node.go

+	oldestAllowed := timestamp - n.validityWindowDuration
+	if oldestAllowed < 0 {
+		oldestAllowed = 0
+	}


nit: min(0, oldestAllowed)

joshua-kim · 2024-12-03T18:46:12Z

x/dsmr/node_test.go

+	blocks map[ids.ID]validitywindow.ExecutionBlock[*ChunkCertificate]
+}
+
+func (ti *testingChainIndex) GetExecutionBlock(_ context.Context, id ids.ID) (validitywindow.ExecutionBlock[*ChunkCertificate], error) {


Nits:

receiver name should just be t

id -> blkID or blockID

joshua-kim · 2024-12-03T19:21:00Z

x/dsmr/block.go

 func (b Block) GetID() ids.ID {
 	return b.blkID
 }
+
+func (b Block) Parent() ids.ID {


I don't care which we pick, but we should be consistent on the naming of either Foo() or GetFoo().

I'm going to dodge this one by saying that this method is no longer needed.

joshua-kim · 2024-12-03T19:21:14Z

x/dsmr/block.go

+func (b Block) Txs() []*ChunkCertificate {
+	return b.ChunkCerts
+}


This seems weird because these are not returning txs

yes, I agree. let's discuss this as a group ?

Can we use the term containers?

joshua-kim · 2024-12-03T19:21:27Z

x/dsmr/node.go

+	log logging.Logger,
+	tracer trace.Tracer,


nit: make these (logger + tracer) the first args in this fn

joshua-kim · 2024-12-03T19:22:38Z

x/dsmr/node.go

+type (
+	ChainIndex         = validitywindow.ChainIndex[*ChunkCertificate]
+	timeValidityWindow = *validitywindow.TimeValidityWindow[*ChunkCertificate]
+)


Seems like we're doing this to avoid the caller depending on an internal package. I'm wondering if it even makes sense for validitywindow to be in internal at all... should this just be merged into dsmr?

@aaronbuchwald asked to remove this section completely.

joshua-kim · 2024-12-03T19:26:11Z

x/dsmr/block.go

 	"github.com/ava-labs/hypersdk/utils"
 )

 const InitialChunkSize = 250 * 1024

 type Tx interface {
-	GetID() ids.ID
-	GetExpiry() int64
+	emap.Item


This exposes the internal/emap package into the caller. I think the previous wrapping pattern where we wrapped this interface w/ a type that implemented the emap interface actually looked cleaner.

once merged, we won't need wrapping interface anymore. . unless I'm missing something ?

I think @joshua-kim 's point is to have the internal type implemented as makes sense in this package and then wrap it with another type that converts between that structure and the required interface when we need to use it in the validity window or expiry map where we need a specific interface. Correct me if I'm wrong @joshua-kim

aaronbuchwald

As discussed offline, can we re-write the tests with the following assumptions:

validity window is a separate component from DSMR in practice and will be responsible for keeping itself up to date
therefore, it should be tested independently as unit tests for the validity window and within integration tests when used in conjunction with DSMR/chain
we should add test cases for DSMR package that treat the validity window as a blackbox ie. return a mocked error from VerifyExpiryReplayProtection and IsRepeat and a mocked value of duplicates from IsRepeat that indicates one of the chunks is a duplicate and should be eliminated
DSMR tests should not be responsible for setting the execution block index (treated as a separate component)

aaronbuchwald · 2025-01-10T16:03:55Z

internal/validitywindow/validitywindowtest/executionblock.go

+
+func int64ToID(n int64) ids.ID {
+	var id ids.ID
+	binary.LittleEndian.PutUint64(id[0:8], uint64(n))


Could we use BigEndian as we use it throughout the rest of the repo?

aaronbuchwald · 2025-01-10T16:04:16Z

internal/validitywindow/validitywindowtest/executionblock.go

+	return false
+}
+
+func NewExecutionBlock(parent int64, timestamp int64, height uint64, contrainers ...int64) ExecutionBlock {


Can we fix the typo contrainers ?

This constructor is fairly difficult to read in the tests as it is a sequence of similar numbers including a variadic final parameter. Could we switch the last argument to being a slice, so that it's clearer to read which numbers are the containers?

aaronbuchwald · 2025-01-10T16:06:31Z

internal/validitywindow/validitywindowtest/container.go

+	return c.Expiry
+}
+
+func NewContainer(expiry int64) Container {


Can we move this package to the validity window test file if this test package is not required externally?

aaronbuchwald · 2025-01-10T16:06:47Z

internal/validitywindow/validitywindowtest/container.go

+// method for returning an id of the item
+func (c Container) GetID() ids.ID {
+	return c.ID
+}
+
+// method for returning this items timestamp


Can we remove comments that do not add any new information?

aaronbuchwald · 2025-01-10T16:07:05Z

internal/validitywindow/validitywindow_test.go

+	}
+}
+
+// testing structures.


can we remove comments that do not add any new information?

aaronbuchwald · 2025-01-10T16:26:38Z

x/dsmr/block.go

+func (e validityWindowBlock) Timestamp() int64 {
+	return e.Block.Timestamp
+}
+
+func (e validityWindowBlock) Height() uint64 {
+	return e.Block.Height
+}
+
+func (e validityWindowBlock) Contains(id ids.ID) bool {
+	return e.certs.Contains(id)
+}
+
+func (e validityWindowBlock) Parent() ids.ID {
+	return e.Block.ParentID
+}
+


Can we add those functions directly to the block type? It will need to implement them as well as part of the snow refactor where it must fulfill the snow.Block type

aaronbuchwald · 2025-01-10T17:01:53Z

x/dsmr/node_test.go

+		4,
+		codec.Address{},
+	))
+	_, err = node.Accept(context.Background(), blk)


We should never call Accept on the same block twice

good point. This Accept was not needed, so I've just dropped it.

aaronbuchwald · 2025-01-10T17:05:09Z

x/dsmr/node_test.go

+	r.NoError(node.BuildChunk(
+		context.Background(),
+		[]dsmrtest.Tx{
+			{
+				ID:     ids.GenerateTestID(),
+				Expiry: 4,
+			},
+		},
+		4,
+		codec.Address{},
+	))
+
+	blk, err := node.BuildBlock(context.Background(), node.LastAccepted, 2)
+	r.NoError(err)
+	r.NoError(node.Verify(context.Background(), node.LastAccepted, blk))


I'm not sure this is testing what it's intended to. This should test that the validity window correctly eliminates chunks that are in the chunk pool from block building. However, we build, verify, and accept the block, which will remove the chunk from the chunk pool. Therefore, no chunks are actually "eliminated" by the validity window in this test.

I added coverage for that in TestNode_BuildBlock_IncludesChunks

The current flow here is build -> verify -> accept and then build another block and confirm that the chunks included in the already accepted block are correctly eliminated so BuildBlock returns the expected error.

At that point, the node is in a state of having an empty chunk pool and an updated last accepted block. I guess this is just testing that BuildBlock does not build an invalid block with the same chunk that was already included. Isn't that the same as the no available chunk certs in the test above?

…s/hypersdk into tsachi/refactor_validity_window3

tsachiherman · 2025-01-13T21:07:27Z

DSMR tests should not be responsible for setting the execution block index (treated as a separate component)

Definitely doable, but would require passing in the underlying TimeValidityWindow, so that we could mock the success case. Than, no indexer would be needed.

aaronbuchwald · 2025-01-14T17:41:43Z

internal/validitywindow/validitywindow.go

+	// make sure we have no repeats within the block itself.
+	blkContainerIDs := set.NewSet[ids.ID](len(blk.Containers()))
+	for _, container := range blk.Containers() {
+		id := container.GetID()
+		if blkContainerIDs.Contains(id) {
+			return fmt.Errorf("%w: duplicate in block", ErrDuplicateContainer)
+		}
+		blkContainerIDs.Add(id)
+	}


This change makes it inconsistent across tx deduplication and chunk deduplication how this is handled. Currently tx deduplication will make sure that there are no duplicate transactions included in a block when it parses or creates it, so that the validitywindow package does not need to check for duplicates within the requested block.

We should align on one way or the other. I think it's a smaller change to this PR to go with the current style for execution block of checking for duplicates when parsing/creating the block, but think either version is fine.

You're correct that it's being performed twice : one in validitywindow.VerifyExpiryReplayProtection and one in chain.NewExecutionBlock. Yet, the functionality is missing in dsmr.NewValidityWindowBlock.

This functionality is logically needed for the correctness of the replay protection, although an early testing would work just as well.

my preference would be to keep it in validitywindow so that we can claim that the entire replay-protection is encapsulated in one place.

That makes sense, could we remove the check for this in ExecutionBlock in that case, so that we don't duplicate the check?

x/dsmr/block.go

x/dsmr/node_test.go

aaronbuchwald · 2025-01-14T18:15:04Z

internal/validitywindow/validitywindow_test.go

+		Name          string
+		Accepted      []executionBlock
+		VerifyBlock   executionBlock
+		OldestAllowed int64
+		ExpectedError error


Can we unexport the field names here?

Can we add an Accepted field to indicate the last block considered accepted as opposed to in processing as we do for TestValidityWindowIsRepeat below?

not sure exactly how to do that, since we really need the included chunks to be set correctly, as it's being tested.

Why can't we do the exact same as we do below and just test VerifyExpiryReplayProtection instead of IsRepeat ?

r := require.New(t) chainIndex := &testChainIndex{} tvw := NewTimeValidityWindow(&logging.NoLog{}, trace.Noop, chainIndex) r.NotNil(tvw) for i, blk := range test.blocks { if i <= int(test.accepted) { tvw.Accept(blk) } chainIndex.set(blk.GetID(), blk) }

oh - now I understood what you meant. You want that the last entry in the blocks would be the one used for the VerifyExpiryReplayProtection

internal/validitywindow/validitywindow_test.go

x/dsmr/node_test.go

aaronbuchwald · 2025-01-14T18:51:32Z

x/dsmr/node_test.go

+	r.NoError(node.BuildChunk(
+		context.Background(),
+		[]dsmrtest.Tx{
+			{
+				ID:     ids.GenerateTestID(),
+				Expiry: 4,
+			},
+		},
+		4,
+		codec.Address{},
+	))
+
+	blk, err := node.BuildBlock(context.Background(), node.LastAccepted, 2)
+	r.NoError(err)
+	r.NoError(node.Verify(context.Background(), node.LastAccepted, blk))


The current flow here is build -> verify -> accept and then build another block and confirm that the chunks included in the already accepted block are correctly eliminated so BuildBlock returns the expected error.

At that point, the node is in a state of having an empty chunk pool and an updated last accepted block. I guess this is just testing that BuildBlock does not build an invalid block with the same chunk that was already included. Isn't that the same as the no available chunk certs in the test above?

tsachiherman · 2025-01-14T21:02:00Z

The current flow here is build -> verify -> accept and then build another block and confirm that the chunks included in the already accepted block are correctly eliminated so BuildBlock returns the expected error.

At that point, the node is in a state of having an empty chunk pool and an updated last accepted block. I guess this is just testing that BuildBlock does not build an invalid block with the same chunk that was already included. Isn't that the same as the no available chunk certs in the test above?

No, it's not exactly the same. Please note of two things:

in both node.BuildBlock we called for a block with a timestamp of 3. The state of the node wasn't altered by the call since BuildBlock returned an error.
after adding a new chunk and calling node.BuildBlock again, we had no error being returned.

That allows us to test that the source for the error is the absence of chunks. ( note that all the chunks in this test have expiry of 4, far beyond the block time ).

internal/validitywindow/validitywindow_test.go

Co-authored-by: aaronbuchwald <[email protected]> Signed-off-by: Tsachi Herman <[email protected]>

…s/hypersdk into tsachi/refactor_validity_window3

eliminate emapChunk

e5ecf69

tsachiherman self-assigned this Nov 22, 2024

tsachiherman added 2 commits November 22, 2024 11:52

update

ba644c3

update

2939543

aaronbuchwald reviewed Nov 25, 2024

View reviewed changes

x/dsmr/block.go Outdated Show resolved Hide resolved

x/dsmr/node.go Outdated Show resolved Hide resolved

x/dsmr/node.go Outdated Show resolved Hide resolved

x/dsmr/node.go Outdated Show resolved Hide resolved

x/dsmr/node.go Outdated Show resolved Hide resolved

tsachiherman added 6 commits November 25, 2024 10:09

Merge branch 'main' into tsachi/refactor_validity_window3

08f0d59

update per CR

659f1fb

lint

921b908

apply requested changes.

508cc92

fix code + test.

7a3fed6

rollback unneeded changes.

b312a3c

tsachiherman marked this pull request as ready for review November 25, 2024 18:20

tsachiherman requested a review from joshua-kim as a code owner November 25, 2024 18:20

rename testing chain indexer.

1238460

aaronbuchwald reviewed Nov 25, 2024

View reviewed changes

x/dsmr/node.go Outdated Show resolved Hide resolved

x/dsmr/node_test.go Outdated Show resolved Hide resolved

x/dsmr/node_test.go Outdated Show resolved Hide resolved

x/dsmr/node_test.go Outdated Show resolved Hide resolved

tsachiherman added 8 commits November 25, 2024 15:04

update

8c1ab98

add unit test.

f8e42a8

update unit test.

b000edf

Merge branch 'main' into tsachi/refactor_validity_window3

1d2536e

lint

6911f26

update

f2816e7

lint

e78f847

update

270245a

aaronbuchwald reviewed Nov 29, 2024

View reviewed changes

joshua-kim mentioned this pull request Dec 2, 2024

Generate Warp Signatures #1818

Merged

joshua-kim reviewed Dec 3, 2024

View reviewed changes

tsachiherman added 3 commits December 5, 2024 11:25

Merge branch 'main' into tsachi/refactor_validity_window3

144ddd4

fix few CR feedback.

2fa701d

add certSet

2220b5e

Merge branch 'main' into tsachi/refactor_validity_window3

84157c2

aaronbuchwald requested changes Jan 10, 2025

View reviewed changes

tsachiherman added 6 commits January 13, 2025 10:39

Merge branch 'main' into tsachi/refactor_validity_window3

58ed219

update testing

8f243b2

Merge branch 'tsachi/refactor_validity_window3' of github.com:ava-lab…

69c6996

…s/hypersdk into tsachi/refactor_validity_window3

Merge branch 'main' into tsachi/refactor_validity_window3

2e879bc

update

43d5d4c

update

c088d7e

tsachiherman added 3 commits January 13, 2025 16:08

lint

fd641ae

update

2db87c4

lint

fa3fc67

tsachiherman requested a review from aaronbuchwald January 13, 2025 22:39

rollback few unneeded empty lines

c513464

aaronbuchwald requested changes Jan 14, 2025

View reviewed changes

tsachiherman added 2 commits January 14, 2025 16:02

address CR feedback.

4dfe80b

lint

f56f017

aaronbuchwald reviewed Jan 15, 2025

View reviewed changes

internal/validitywindow/validitywindow_test.go Outdated Show resolved Hide resolved

aaronbuchwald reviewed Jan 15, 2025

View reviewed changes

internal/validitywindow/validitywindow_test.go Outdated Show resolved Hide resolved

tsachiherman and others added 7 commits January 15, 2025 10:43

Update internal/validitywindow/validitywindow_test.go

919e852

Co-authored-by: aaronbuchwald <[email protected]> Signed-off-by: Tsachi Herman <[email protected]>

Update internal/validitywindow/validitywindow_test.go

e0a910c

Co-authored-by: aaronbuchwald <[email protected]> Signed-off-by: Tsachi Herman <[email protected]>

remove unit test

b0f1782

Merge branch 'main' into tsachi/refactor_validity_window3

7f21c2b

remove the testing for duplicate containerid from chain.ExecutionBlock

23219b0

Merge branch 'tsachi/refactor_validity_window3' of github.com:ava-lab…

081381b

…s/hypersdk into tsachi/refactor_validity_window3

update

3f81853

aaronbuchwald approved these changes Jan 15, 2025

View reviewed changes

aaronbuchwald merged commit 143eaea into main Jan 15, 2025
17 checks passed

aaronbuchwald deleted the tsachi/refactor_validity_window3 branch January 15, 2025 18:13

		func (c ChunkCertificate) GetExpiry() int64 { return c.Expiry }

		func (c *ChunkCertificate) GetSlot() int64 { return c.Expiry }

Generalize expiry based de-duplication, dsmr #1810

Generalize expiry based de-duplication, dsmr #1810

Conversation

tsachiherman commented Nov 22, 2024 • edited Loading

What ?

tsachiherman commented Nov 25, 2024 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronbuchwald left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsachiherman commented Jan 13, 2025

Choose a reason for hiding this comment

tsachiherman Jan 14, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsachiherman commented Jan 14, 2025

tsachiherman commented Nov 22, 2024 •

edited

Loading

tsachiherman Jan 14, 2025 •

edited

Loading