Skip to content

[Storage] Fix 103/104 Live Test Failures - #48376

Merged
Peter Wu (weirongw23-msft) merged 6 commits into
Azure:mainfrom
weirongw23-msft:weirongw23/103-104-fixes
Jul 31, 2026
Merged

[Storage] Fix 103/104 Live Test Failures#48376
Peter Wu (weirongw23-msft) merged 6 commits into
Azure:mainfrom
weirongw23-msft:weirongw23/103-104-fixes

Refreshed test recordings

3a09010
Select commit
Loading
Failed to load commit list.
Azure Pipelines / python - storage - tests failed Jul 31, 2026 in 2h 17m 59s

Build #20260731.2 had test failures

21 errors / 78 warnings

Details

Tests

  • Failed: 51 (0.22%)
  • Passed: 22,647 (96.93%)
  • Other: 666 (2.85%)
  • Total: 23,364

Annotations

Check failure on line 7065 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

Build log #L7065 

PowerShell exited with code '1'.

Check failure on line 21 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

Build log #L21 

There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.

Check failure on line 21 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

Build log #L21 

There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.

Check failure on line 27169 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

Build log #L27169 

PowerShell exited with code '1'.

Check failure on line 1 in test_append_data_streaming_large[True]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

test_append_data_streaming_large[True] 

azure.core.exceptions.HttpResponseError: (OperationTimedOut) Operation could not be completed within the specified time.
RequestId:5a4d6bf6-101f-0014-4026-21e1d2000000
Time:2026-07-31T19:58:03.1714402Z
Code: OperationTimedOut
Message: Operation could not be completed within the specified time.
RequestId:5a4d6bf6-101f-0014-4026-21e1d2000000
Time:2026-07-31T19:58:03.1714402Z
Raw output 
self = <test_content_validation.TestStorageContentValidation object at 0x0000015354F0A0C0>
a = True
kwargs = {'__aggregate_cache_key': ('EnvironmentVariableLoader',), 'datalake_storage_account_key': <devtools_testutils.envvaria...ronmentVariable object at 0x0000015358C5FE60>, 'storage_data_lake_soft_delete_account_name': 'dtlksdt460ec354a6f545ae'}
datalake_storage_account_name = 'dtlkt460ec354a6f545ae'
file = <azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient object at 0x0000015358C7D460>
data1 = b'abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdea...cdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde'
data2 = b'12345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451...512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345abcdefg'
data3 = b'12345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234...3456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678'
assert_method = <function assert_content_md5 at 0x000001535453C860>

    @DataLakePreparer()
    @pytest.mark.parametrize("a", [True, "md5", "crc64"])  # a: validate_content
    @pytest.mark.live_test_only
    def test_append_data_streaming_large(self, a, **kwargs):
        datalake_storage_account_name = kwargs.pop("datalake_storage_account_name")
    
        self._setup(datalake_storage_account_name)
        file = self.file_system.get_file_client(self._get_file_reference())
    
        data1 = b"abcde" * 1024 * 1024  # 5 MiB
        data2 = b"12345" * 2 * 1024 * 1024 + b"abcdefg"  # 10 MiB + 7
        data3 = b"12345678" * 8 * 1024 * 1024  # 64 MiB
        assert_method = assert_structured_message if a == "crc64" else assert_content_md5
    
        # Act
        file.create_file()
        file.append_data(BytesIO(data1), 0, flush=True, validate_content=a, raw_request_hook=assert_method)
        file.append_data(BytesIO(data2), len(data1), flush=True, validate_content=a, raw_request_hook=assert_method)
>       file.append_data(
            BytesIO(data3), len(data1) + len(data2), flush=True, validate_content=a, raw_request_hook=assert_method
        )

tests\test_content_validation.py:193: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\tracing\decorator.py:119: in wrapper_use_tracer
    return func(*args, **kwargs)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\storage\filedatalake\_data_lake_file_client.py:541: in append_data
    process_storage_error(error)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\storage\filedatalake\_deserialize.py:253: in process_storage_error
    exec("raise error from None")  # pylint: disable=exec-used # nosec
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   azure.core.exceptions.HttpResponseError: (OperationTimedOut) Operation could not be completed within the specified time.
E   RequestId:5a4d6bf6-101f-0014-4026-21e1d2000000
E   Time:2026-07-31T19:58:03.1714402Z
E   Code: OperationTimedOut
E   Message: Operation could not be completed within the specified time.
E   RequestId:5a4d6bf6-101f-0014-4026-21e1d2000000
E   Time:2026-07-31T19:58:03.1714402Z

<string>:1: HttpResponseError

Check failure on line 1 in test_append_data_streaming_large[md5]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

test_append_data_streaming_large[md5] 

azure.core.exceptions.HttpResponseError: (OperationTimedOut) Operation could not be completed within the specified time.
RequestId:b6b4c8cb-a01f-0073-5627-21f12e000000
Time:2026-07-31T20:01:46.5414455Z
Code: OperationTimedOut
Message: Operation could not be completed within the specified time.
RequestId:b6b4c8cb-a01f-0073-5627-21f12e000000
Time:2026-07-31T20:01:46.5414455Z
Raw output 
self = <test_content_validation.TestStorageContentValidation object at 0x0000015354F0A1E0>
a = 'md5'
kwargs = {'__aggregate_cache_key': ('EnvironmentVariableLoader',), 'datalake_storage_account_key': <devtools_testutils.envvaria...ronmentVariable object at 0x00000153569E4E00>, 'storage_data_lake_soft_delete_account_name': 'dtlksdt460ec354a6f545ae'}
datalake_storage_account_name = 'dtlkt460ec354a6f545ae'
file = <azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient object at 0x000001535640A270>
data1 = b'abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdea...cdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde'
data2 = b'12345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451...512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345abcdefg'
data3 = b'12345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234...3456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678'
assert_method = <function assert_content_md5 at 0x000001535453C860>

    @DataLakePreparer()
    @pytest.mark.parametrize("a", [True, "md5", "crc64"])  # a: validate_content
    @pytest.mark.live_test_only
    def test_append_data_streaming_large(self, a, **kwargs):
        datalake_storage_account_name = kwargs.pop("datalake_storage_account_name")
    
        self._setup(datalake_storage_account_name)
        file = self.file_system.get_file_client(self._get_file_reference())
    
        data1 = b"abcde" * 1024 * 1024  # 5 MiB
        data2 = b"12345" * 2 * 1024 * 1024 + b"abcdefg"  # 10 MiB + 7
        data3 = b"12345678" * 8 * 1024 * 1024  # 64 MiB
        assert_method = assert_structured_message if a == "crc64" else assert_content_md5
    
        # Act
        file.create_file()
        file.append_data(BytesIO(data1), 0, flush=True, validate_content=a, raw_request_hook=assert_method)
        file.append_data(BytesIO(data2), len(data1), flush=True, validate_content=a, raw_request_hook=assert_method)
>       file.append_data(
            BytesIO(data3), len(data1) + len(data2), flush=True, validate_content=a, raw_request_hook=assert_method
        )

tests\test_content_validation.py:193: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\tracing\decorator.py:119: in wrapper_use_tracer
    return func(*args, **kwargs)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\storage\filedatalake\_data_lake_file_client.py:541: in append_data
    process_storage_error(error)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\storage\filedatalake\_deserialize.py:253: in process_storage_error
    exec("raise error from None")  # pylint: disable=exec-used # nosec
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   azure.core.exceptions.HttpResponseError: (OperationTimedOut) Operation could not be completed within the specified time.
E   RequestId:b6b4c8cb-a01f-0073-5627-21f12e000000
E   Time:2026-07-31T20:01:46.5414455Z
E   Code: OperationTimedOut
E   Message: Operation could not be completed within the specified time.
E   RequestId:b6b4c8cb-a01f-0073-5627-21f12e000000
E   Time:2026-07-31T20:01:46.5414455Z

<string>:1: HttpResponseError

Check failure on line 1 in test_append_data_streaming_large[crc64]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

test_append_data_streaming_large[crc64] 

azure.core.exceptions.AzureError: Expected structured message header in response does not match request. Request: XSM/1.0; properties=crc64, Response: None
Raw output 
self = <test_content_validation.TestStorageContentValidation object at 0x0000015354F0A360>
a = 'crc64'
kwargs = {'__aggregate_cache_key': ('EnvironmentVariableLoader',), 'datalake_storage_account_key': <devtools_testutils.envvaria...ronmentVariable object at 0x0000015356C06840>, 'storage_data_lake_soft_delete_account_name': 'dtlksdt460ec354a6f545ae'}
datalake_storage_account_name = 'dtlkt460ec354a6f545ae'
file = <azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient object at 0x0000015356A57140>
data1 = b'abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdea...cdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde'
data2 = b'12345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451...512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345123451234512345abcdefg'
data3 = b'12345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234...3456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678123456781234567812345678'
assert_method = <function assert_structured_message at 0x000001535466C860>

    @DataLakePreparer()
    @pytest.mark.parametrize("a", [True, "md5", "crc64"])  # a: validate_content
    @pytest.mark.live_test_only
    def test_append_data_streaming_large(self, a, **kwargs):
        datalake_storage_account_name = kwargs.pop("datalake_storage_account_name")
    
        self._setup(datalake_storage_account_name)
        file = self.file_system.get_file_client(self._get_file_reference())
    
        data1 = b"abcde" * 1024 * 1024  # 5 MiB
        data2 = b"12345" * 2 * 1024 * 1024 + b"abcdefg"  # 10 MiB + 7
        data3 = b"12345678" * 8 * 1024 * 1024  # 64 MiB
        assert_method = assert_structured_message if a == "crc64" else assert_content_md5
    
        # Act
        file.create_file()
        file.append_data(BytesIO(data1), 0, flush=True, validate_content=a, raw_request_hook=assert_method)
        file.append_data(BytesIO(data2), len(data1), flush=True, validate_content=a, raw_request_hook=assert_method)
>       file.append_data(
            BytesIO(data3), len(data1) + len(data2), flush=True, validate_content=a, raw_request_hook=assert_method
        )

tests\test_content_validation.py:193: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\tracing\decorator.py:119: in wrapper_use_tracer
    return func(*args, **kwargs)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\storage\filedatalake\_data_lake_file_client.py:539: in append_data
    return self._client.path.append_data(**options)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\tracing\decorator.py:119: in wrapper_use_tracer
    return func(*args, **kwargs)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\storage\filedatalake\_generated\operations\_path_operations.py:2680: in append_data
    pipeline_response: PipelineResponse = self._client._pipeline.run(  # pylint: disable=protected-access
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\pipeline\_base.py:242: in run
    return first_node.send(pipeline_request)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\pipeline\_base.py:98: in send
    response = self.next.send(request)
..\..\..\.venv\azure-storage-file-datalake\.venv_whl\Lib\site-packages\azure\core\pipeline\_base.py:98: in send
    response = self.next.send(request)
..\..\..\.venv\a
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]

Check failure on line 1 in test_standard_blob_tier_set_tier_api

See this annotation in the file changed.

@azure-pipelines azure-pipelines / python - storage - tests

test_standard_blob_tier_set_tier_api 

azure.core.exceptions.HttpResponseError: A required feature for this operation is not enabled on the account.
RequestId:cabd5070-201e-0043-7226-21a67e000000
Time:2026-07-31T19:56:28.2418019Z
ErrorCode:FeatureNotEnabled
featurename:SmartTier
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>FeatureNotEnabled</Code><Message>A required feature for this operation is not enabled on the account.
RequestId:cabd5070-201e-0043-7226-21a67e000000
Time:2026-07-31T19:56:28.2418019Z</Message><FeatureName>SmartTier</FeatureName></Error>
Raw output 
args = (<test_blob_storage_account.TestBlobStorageAccount object at 0x7f2c576328c0>,)
kwargs = {'__aggregate_cache_key': ('EnvironmentVariableLoader',), 'blob_storage_account_key': <devtools_testutils.envvariable_...remium_storage_account_key': <devtools_testutils.envvariable_loader.EnvironmentVariable object at 0x7f2c55255810>, ...}
trimmed_kwargs = {'__aggregate_cache_key': ('EnvironmentVariableLoader',), 'blob_storage_account_key': <devtools_testutils.envvariable_...remium_storage_account_key': <devtools_testutils.envvariable_loader.EnvironmentVariable object at 0x7f2c55255810>, ...}

    def record_wrap(*args, **kwargs):
        # ---- your existing trimming/early-exit logic ----
        trimmed_kwargs = {k: v for k, v in kwargs.items()}
        trim_kwargs_from_test_function(test_func, trimmed_kwargs)
    
        if is_live_and_not_recording():
>           return test_func(*args, **trimmed_kwargs)

../../../.venv/azure-storage-blob/.venv_whl/lib/python3.10/site-packages/devtools_testutils/proxy_testcase.py:306: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_blob_storage_account.py:70: in test_standard_blob_tier_set_tier_api
    blob.set_standard_blob_tier(tier)
../../../.venv/azure-storage-blob/.venv_whl/lib/python3.10/site-packages/azure/core/tracing/decorator.py:119: in wrapper_use_tracer
    return func(*args, **kwargs)
../../../.venv/azure-storage-blob/.venv_whl/lib/python3.10/site-packages/azure/storage/blob/_blob_client.py:1986: in set_standard_blob_tier
    process_storage_error(error)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

storage_error = None

    def process_storage_error(storage_error) -> NoReturn:  # type: ignore [misc] # pylint:disable=too-many-statements, too-many-branches
        raise_error = HttpResponseError
        serialized = False
        if isinstance(storage_error, AzureSigningError):
            storage_error.message = (
                storage_error.message
                + ". This is likely due to an invalid shared key. Please check your shared key and try again."
            )
        if not storage_error.response or storage_error.response.status_code in [200, 204]:
            raise storage_error
        # If it is one of those three then it has been serialized prior by the generated layer.
        if isinstance(
            storage_error,
            (PartialBatchErrorException, ClientAuthenticationError, ResourceNotFoundError, ResourceExistsError),
        ):
            serialized = True
        error_code = storage_error.response.headers.get("x-ms-error-code")
        error_message = storage_error.message
        additional_data = {}
        error_dict = {}
        try:
            error_body = ContentDecodePolicy.deserialize_from_http_generics(storage_error.response)
            try:
                if error_body is None or len(error_body) == 0:
                    error_body = storage_error.response.reason
            except AttributeError:
                error_body = ""
            # If it is an XML response
            if isinstance(error_body, Element):
                error_dict = {child.tag.lower(): child.text for child in error_body}
            # If it is a JSON response
            elif isinstance(error_body, dict):
                error_dict = error_body.get("error", {})
            elif not error_code:
                _LOGGER.warning(
                    "Unexpected return type %s from ContentDecodePolicy.deserialize_from_http_generics.", type(error_body)
                )
                error_dict = {"message": str(error_body)}
    
            # If we extracted from a Json or XML response
            # There is a chance error_dict is just a string
            if error_dict and isinstance(error_dict, dict):
          
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]
View more details on Azure Pipelines