Skip to content

Commit 7161114

Browse files
hive-botclaude
andcommitted
fix: raise on failed translation memory jobs and local file errors
wait_until_translation_memory_job_done() raises DeepLException with the API's error message when the job fails or expires and returns only a completed job. With timeout_s, the last sleep is cut to the deadline. import_translation_memory_from_filepath() reads the file before creating the import job, and export_translation_memory_to_filepath() checks the output path before creating the export job. Local I/O failures raise DeepLException with the OSError as cause, and a failed export removes its partially written file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 455d8ac commit 7161114

4 files changed

Lines changed: 236 additions & 53 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
153153
`DeepLException`; before they were always `False` and `None`.
154154

155155
### Changed
156+
- `wait_until_translation_memory_job_done()` raises `DeepLException` with the
157+
API's error message when the job fails or expires, instead of returning the
158+
job. It returns only a completed job. With `timeout_s`, the last sleep is cut
159+
short so the wait no longer runs past the timeout.
160+
- `import_translation_memory_from_filepath()` reads the file before creating
161+
the import job; a file that cannot be read raises `DeepLException` without
162+
sending a request. `export_translation_memory_to_filepath()` checks that the
163+
output directory exists and the path is writable before creating the export
164+
job, and raises `DeepLException` otherwise.
156165
- `translate_text()` accepts `glossary` and `glossary_ids` without
157166
`source_lang`; DeepL detects the source language and uses the glossary
158167
dictionary for the detected language pair. Document translation with a
@@ -245,6 +254,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
245254
- Fixed errors from the pre-signed translation memory upload and download URLs
246255
reporting `should_retry` as false for HTTP 429 and 5xx, and dropping the
247256
response body of a failed download from the message.
257+
- Fixed `import_translation_memory_from_filepath()` and
258+
`export_translation_memory_to_filepath()` raising a raw `OSError` for local
259+
file failures; they now raise `DeepLException` with the `OSError` as cause.
260+
A failed export no longer leaves a partially written file behind.
248261
- HTTP 401 responses now raise `AuthorizationException`, like 403. Before they
249262
raised `DeepLException` with "Unexpected status code: 401".
250263
- `DeepLException.should_retry` is now `True` for every HTTP 5xx error. It was

‎README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1238,6 +1238,11 @@ Importing takes a while: the API detects the uploaded file asynchronously, so
12381238
the job keeps reporting `awaiting_input` for roughly half a minute after the
12391239
upload before completing. Pass `timeout_s` to bound how long to wait.
12401240

1241+
The file is read before the import job is created: a missing file raises
1242+
`ValueError`, and a file that cannot be read raises `DeepLException` (with the
1243+
`OSError` as its cause), in both cases without sending a request. A failed or
1244+
expired import job raises `DeepLException` with the API's error message.
1245+
12411246
The three steps are also available separately, for example to upload the file
12421247
yourself or to poll for progress. `create_translation_memory_import()` returns
12431248
an upload URL that the file must be uploaded to before processing starts, then
@@ -1268,6 +1273,12 @@ job = deepl_client.wait_until_translation_memory_job_done(
12681273
)
12691274
```
12701275

1276+
`wait_until_translation_memory_job_done()` polls every 5 seconds and returns
1277+
the job once it is completed. If the job fails or expires it raises
1278+
`DeepLException` with the API's error message. With `timeout_s` it raises
1279+
`DeepLException` once the timeout is exceeded, cutting the last sleep short so
1280+
the wait does not run past the timeout.
1281+
12711282
Errors from the pre-signed upload and download URLs are not DeepL API errors:
12721283
`upload_translation_memory_file()` and `download_translation_memory_export()`
12731284
raise a plain `DeepLException` with the response body in the message,
@@ -1296,6 +1307,12 @@ job = deepl_client.export_translation_memory_to_filepath(
12961307
)
12971308
```
12981309

1310+
The output path is checked before the export job is created: if its directory
1311+
does not exist, or the path is a directory or not writable,
1312+
`DeepLException` is raised without sending a request. If the export job
1313+
fails, or the download or writing the file fails, `DeepLException` is raised
1314+
and a partially written file is removed.
1315+
12991316
As with import, the individual steps are available separately. Note that the
13001317
API may reuse a previously completed export of an unchanged translation memory,
13011318
indicated by `reused_existing`.

‎deepl/deepl_client.py‎

Lines changed: 109 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,11 @@
2929
TranslationMemoryJob,
3030
TranslationMemorySegments,
3131
)
32-
from deepl.exceptions import DeepLException, GlossaryNotFoundException
32+
from deepl.exceptions import (
33+
ConnectionException,
34+
DeepLException,
35+
GlossaryNotFoundException,
36+
)
3337
from deepl.translator import (
3438
Translator,
3539
_DOWNLOAD_CHUNK_SIZE,
@@ -59,6 +63,20 @@
5963
_T = TypeVar("_T")
6064

6165

66+
def _tm_job_action(job: TranslationMemoryJob) -> str:
67+
if job.operation == TranslationMemoryJob.OPERATION_IMPORT:
68+
return "importing"
69+
if job.operation == TranslationMemoryJob.OPERATION_EXPORT:
70+
return "exporting"
71+
return "processing"
72+
73+
74+
def _tm_job_error_message(job: TranslationMemoryJob) -> str:
75+
return (
76+
job.result.error_message if job.result else None
77+
) or "unknown error"
78+
79+
6280
def _parse_improvements(json: Any, text_count: int) -> List[WriteResult]:
6381
improvements = json.get("improvements") if isinstance(json, dict) else None
6482
if not isinstance(improvements, list) or len(improvements) < text_count:
@@ -2153,8 +2171,8 @@ def wait_until_translation_memory_job_done(
21532171
job: Union[str, TranslationMemoryJob],
21542172
timeout_s: Optional[int] = None,
21552173
) -> TranslationMemoryJob:
2156-
"""Polls a translation memory job until it finishes, sleeping between
2157-
requests, and returns the final status.
2174+
"""Polls a translation memory job until it completes, sleeping 5
2175+
seconds between requests, and returns the completed job.
21582176
21592177
Note that an import job keeps reporting "awaiting_input" for a while
21602178
after its file has been uploaded, because the API detects the upload
@@ -2163,33 +2181,43 @@ def wait_until_translation_memory_job_done(
21632181
its own, so pass timeout_s when that is a possibility.
21642182
21652183
:param job: Job ID string or TranslationMemoryJob object.
2166-
:param timeout_s: (Optional) Maximum time to wait before raising an
2167-
error. Note that this is not accurate to the second, but only
2168-
polls every 5 seconds.
2169-
:return: TranslationMemoryJob containing the status when finished.
2184+
:param timeout_s: (Optional) Maximum time in seconds to wait before
2185+
raising an error. The last sleep is cut short so that the wait
2186+
does not run past the timeout.
2187+
:return: TranslationMemoryJob containing the completed status.
2188+
2189+
:raises DeepLException: If the job fails or expires, with the API's
2190+
error message, or if timeout_s is exceeded.
21702191
"""
21712192
# Validates the reference; get_translation_memory_job() encodes it.
21722193
self._job_id(job)
2173-
status = self.get_translation_memory_job(job)
21742194
start_time_s = time.time()
2175-
while not status.done:
2176-
if (
2177-
timeout_s is not None
2178-
and time.time() - start_time_s > timeout_s
2179-
):
2195+
status = self.get_translation_memory_job(job)
2196+
while True:
2197+
if not status.ok:
21802198
raise DeepLException(
2181-
f"Manual timeout of {timeout_s}s exceeded for"
2182-
+ " translation memory job",
2199+
f"Error occurred while {_tm_job_action(status)} "
2200+
f"translation memory: {_tm_job_error_message(status)}",
21832201
should_retry=False,
21842202
)
2203+
if status.done:
2204+
return status
21852205
secs = 5.0
2206+
if timeout_s is not None:
2207+
remaining_s = start_time_s + timeout_s - time.time()
2208+
if remaining_s <= 0:
2209+
raise DeepLException(
2210+
f"Manual timeout of {timeout_s}s exceeded for"
2211+
+ " translation memory job",
2212+
should_retry=False,
2213+
)
2214+
secs = min(secs, remaining_s)
21862215
util.log_info(
21872216
f"Rechecking translation memory job status after sleeping "
21882217
f"for {secs:.3f} seconds."
21892218
)
21902219
time.sleep(secs)
21912220
status = self.get_translation_memory_job(job)
2192-
return status
21932221

21942222
def download_translation_memory_export(
21952223
self,
@@ -2250,9 +2278,9 @@ def import_translation_memory_from_filepath(
22502278
:param input_path: Path to the TMX file to import.
22512279
:param display_name: (Optional) Name for the resulting translation
22522280
memory, defaults to the file name.
2253-
:param timeout_s: (Optional) Maximum time to wait for the import to
2254-
finish. Note that this is not accurate to the second, but only
2255-
polls every 5 seconds.
2281+
:param timeout_s: (Optional) Maximum time in seconds to wait for the
2282+
import to finish. The last 5-second sleep between status checks is
2283+
cut short so that the wait does not run past the timeout.
22562284
:param content_type: (Optional) MIME type of the file, declared when
22572285
creating the import job and sent with the upload. Defaults to
22582286
"application/xml".
@@ -2262,38 +2290,40 @@ def import_translation_memory_from_filepath(
22622290
:return: TranslationMemoryJob for the completed import; its result
22632291
carries the new translation memory ID.
22642292
2265-
:raises DeepLException: If the import fails.
2293+
:raises ValueError: If input_path does not exist.
2294+
:raises DeepLException: If the file cannot be read, the upload is
2295+
rejected, the import job fails or expires, or timeout_s is
2296+
exceeded.
22662297
"""
22672298
file_path = pathlib.Path(input_path)
22682299
if not file_path.exists():
22692300
raise ValueError(f"file does not exist: {input_path}")
2301+
# Read the file before creating the job, so an unreadable file fails
2302+
# without leaving an import job waiting for its upload.
2303+
try:
2304+
with open(file_path, "rb") as input_file:
2305+
content = input_file.read()
2306+
except OSError as e:
2307+
raise DeepLException(
2308+
f"Error reading translation memory file {input_path}: {e}"
2309+
) from e
22702310

22712311
created = self.create_translation_memory_import(
22722312
file_name=file_path.name,
2273-
content_length=file_path.stat().st_size,
2313+
content_length=len(content),
22742314
content_type=content_type,
22752315
display_name=display_name,
22762316
extra_body_parameters=extra_body_parameters,
22772317
)
2278-
with open(file_path, "rb") as input_file:
2279-
self.upload_translation_memory_file(
2280-
created,
2281-
input_file,
2282-
content_type=content_type or "application/xml",
2283-
)
2318+
self.upload_translation_memory_file(
2319+
created,
2320+
content,
2321+
content_type=content_type or "application/xml",
2322+
)
22842323

2285-
job = self.wait_until_translation_memory_job_done(
2324+
return self.wait_until_translation_memory_job_done(
22862325
created.job_id, timeout_s
22872326
)
2288-
if not job.ok:
2289-
error_message = (
2290-
job.result.error_message if job.result else None
2291-
) or "unknown error"
2292-
raise DeepLException(
2293-
"Error occurred while importing translation memory: "
2294-
f"{error_message}"
2295-
)
2296-
return job
22972327

22982328
def export_translation_memory_to_filepath(
22992329
self,
@@ -2308,30 +2338,56 @@ def export_translation_memory_to_filepath(
23082338
:param translation_memory: Translation memory ID string or
23092339
TranslationMemoryInfo object.
23102340
:param output_path: Path to write the exported TMX file to.
2311-
:param timeout_s: (Optional) Maximum time to wait for the export to
2312-
finish. Note that this is not accurate to the second, but only
2313-
polls every 5 seconds.
2341+
:param timeout_s: (Optional) Maximum time in seconds to wait for the
2342+
export to finish. The last 5-second sleep between status checks is
2343+
cut short so that the wait does not run past the timeout.
23142344
:return: TranslationMemoryJob for the completed export.
23152345
2316-
:raises DeepLException: If the export fails.
2346+
:raises DeepLException: If output_path cannot be written (checked
2347+
before the export job is created), the export job fails or
2348+
expires, the download fails, or timeout_s is exceeded. A partially
2349+
written output file is removed.
23172350
"""
2351+
path = pathlib.Path(output_path)
2352+
# Check the output path before creating the job, so a path that
2353+
# cannot be written fails without exporting first.
2354+
directory = path.parent
2355+
if not directory.is_dir():
2356+
raise DeepLException(
2357+
f"Output directory does not exist: {directory}"
2358+
)
2359+
if path.is_dir():
2360+
raise DeepLException(f"Output path is a directory: {output_path}")
2361+
if not os.access(path if path.exists() else directory, os.W_OK):
2362+
raise DeepLException(f"Output path is not writable: {output_path}")
2363+
23182364
created = self.create_translation_memory_export(translation_memory)
23192365
job = self.wait_until_translation_memory_job_done(
23202366
created.job_id, timeout_s
23212367
)
2322-
if not job.ok:
2323-
error_message = (
2324-
job.result.error_message if job.result else None
2325-
) or "unknown error"
2326-
raise DeepLException(
2327-
"Error occurred while exporting translation memory: "
2328-
f"{error_message}"
2329-
)
23302368

2331-
with open(output_path, "wb") as output_file:
2332-
self.download_translation_memory_export(
2333-
job, output_file, chunk_size=8192
2334-
)
2369+
try:
2370+
with open(path, "wb") as output_file:
2371+
self.download_translation_memory_export(
2372+
job, output_file, chunk_size=8192
2373+
)
2374+
except BaseException as e:
2375+
# Do not leave a partial file behind.
2376+
try:
2377+
path.unlink()
2378+
except OSError:
2379+
pass
2380+
if isinstance(e, requests.exceptions.RequestException):
2381+
# The download stream broke off; not a local I/O failure.
2382+
raise ConnectionException(
2383+
f"Error downloading translation memory export: {e}"
2384+
) from e
2385+
if isinstance(e, OSError):
2386+
raise DeepLException(
2387+
"Error writing translation memory export to "
2388+
f"{output_path}: {e}"
2389+
) from e
2390+
raise
23352391
return job
23362392

23372393
def _asset_call(

0 commit comments

Comments
 (0)