Skip to content

Commit d3a7fd5

Browse files
committed
Disable existence checks in S3OutputFile.create
Restore the behavior before as it was before support for conditional writes was introduced. File existence checks are not implemented for multi-part upload code path (the `FileAlreadyExistsException` is throw only when multi-part did not take place). Furthermore, file existence checks can break file writes when the call is retried on the AWS SDK level. Fixing both these issues is possible, but not necessary: users of `TrinoOutputFile.create` cannot assume particular behavior and file system implementation is allowed to overwrite existing files. Atomic writes are necessary for Delta's connector transaction log synchronizer `S3ConditionalWriteLogSynchronizer`, but is explicitly invokes `TrinoOutputFile.createExclusive` API.
1 parent 1de55e9 commit d3a7fd5

File tree

3 files changed

+7
-7
lines changed

3 files changed

+7
-7
lines changed

lib/trino-filesystem-s3/src/main/java/io/trino/filesystem/s3/S3OutputFile.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ public void createExclusive(byte[] data)
7171
@Override
7272
public OutputStream create(AggregatedMemoryContext memoryContext)
7373
{
74-
return create(memoryContext, context.exclusiveWriteSupported());
74+
return create(memoryContext, false);
7575
}
7676

7777
private OutputStream create(AggregatedMemoryContext memoryContext, boolean exclusive)

lib/trino-filesystem-s3/src/test/java/io/trino/filesystem/s3/AbstractTestS3FileSystem.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,12 @@ protected final Location getRootLocation()
9494
return Location.of("s3://%s/".formatted(bucket()));
9595
}
9696

97+
@Override
98+
protected boolean isCreateExclusive()
99+
{
100+
return false;
101+
}
102+
97103
@Override
98104
protected final boolean supportsRenameFile()
99105
{

lib/trino-filesystem-s3/src/test/java/io/trino/filesystem/s3/TestS3FileSystemS3Mock.java

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -38,12 +38,6 @@ public class TestS3FileSystemS3Mock
3838
private static final S3MockContainer S3_MOCK = new S3MockContainer("3.0.1")
3939
.withInitialBuckets(BUCKET);
4040

41-
@Override
42-
protected boolean isCreateExclusive()
43-
{
44-
return false; // not supported by s3-mock
45-
}
46-
4741
@Override
4842
protected boolean supportsCreateExclusive()
4943
{

0 commit comments

Comments
 (0)