Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Spaces removed in translation request. #278

Open
joey-bolts opened this issue Jun 22, 2023 · 3 comments · May be fixed by #288
Open

[BUG] Spaces removed in translation request. #278

joey-bolts opened this issue Jun 22, 2023 · 3 comments · May be fixed by #288
Assignees
@NarkNiro
Labels
bug Something isn't working

Comments

@joey-bolts
Copy link

Describe the bug
After sending a request with the following text as content in TextMedia bodytext:

<p>Important species in blueberry include the Western flower thrips (<em>Frankliniella occidentalis</em>) and Chilli thrips (<em>Scirtothrips dorsalis</em>).</p>

the result received from the API is:

<p>Belangrijke soorten in blauwe bes zijn de californische trips<em>(Frankliniella occidentalis</em>) en chilipeper trips<em>(Scirtothrips dorsalis</em>).</p>

tags have been moved, and there is a space missing. I checked this directly with xdebug in the $response->translations->[0]->text response.

To Reproduce

  • Create a content element
  • Add the source text
  • Request translation
  • Result has some spaces removed before text in ()

Expected behavior
Text is translation with same formatting

Screenshots
If applicable, add screenshots to help explain your problem.

TYPO3 Setup

  • TYPO3 Version - V10.4.37
  • Server (Apache)
  • Database type and version (MySQL)
  • PHP version - 7.4
  • DeepL free
  • composer

Additional context / Optional solution

    public function translateContent(
        string $content,
        array $targetLanguageRecord,
        string $customMode,
        array $sourceLanguageRecord
    ): string {

        $content = htmlentities($content);

        // mode deepl
        if ($customMode == 'deepl') {
            $response = $this->deeplService->translateRequest(
                $content,
                $targetLanguageRecord['language_isocode'],
                $sourceLanguageRecord['language_isocode']
            );

            if (!empty($response) && isset($response['translations'])) {
                foreach ($response['translations'] as $translation) {
                    if ($translation['text'] != '') {
                        $content = htmlspecialchars_decode($translation['text'], ENT_QUOTES | ENT_SUBSTITUTE | ENT_HTML5);
                        break;
                    }
                }
            }
        }

Adding $content = htmlentities($content); solved the issue and the formatting stays the same.

I could XClass this as a solution for now, but i would like to have this fixed in the extension itself. It might not be the correct location for the fix, but anywhere you prefer.
@joey-bolts joey-bolts added the bug Something isn't working label Jun 22, 2023
@joey-bolts joey-bolts changed the title [BUG] Spaces removed in transaltion request. [BUG] Spaces removed in translation request. Jun 29, 2023
@NarkNiro NarkNiro linked a pull request Jul 21, 2023 that will close this issue
Draft
@NarkNiro NarkNiro self-assigned this Jul 21, 2023
@NarkNiro
Copy link
Member

Hello @joey-beech-it ,

The PR is currently not yet finished, I forgot to change/commit a part.
This I still have to pull on Friday, so the PR is still in a DRAF status to consider.

But I could reproduce your error and write a test for it. There are simply more Translate Request parameters missing, which lead to a better result with XML tags.
From a htmlentities() of the content I would advise against, because this might lead to unsightly translation.

@joey-bolts
Copy link
Author

@NarkNiro

Thanks for the info. Great to hear. I will keep an eye on your PR. Feel free to mention me if you want to have it tested again.

joey-bolts pushed a commit to beechit/wv_deepltranslate that referenced this issue Jul 25, 2023
[TASK] Use HTML entities so customer can start using translation api
42517f4 
until web-vision#278 is
fixed - SEE PR
@joey-bolts
Copy link
Author

I tested this, but still some issues occurs. Spaces still get stripped.
I have multiple cases, here is an example.

Source:
<p>The parasitic wasps<em>&nbsp;Diglyphus</em>&nbsp;<em>isaea</em>&nbsp;(<a href="t3://page?uid=current">Miglyphus</a>) and<em>&nbsp;Dacnusa sibirica</em>&nbsp;(<a href="t3://page?uid=current">Minusa</a>), are effective natural enemies of leaf miner larvae. Additionally, the beneficial nematode&nbsp;<em>Steinernema feltiae</em>&nbsp;(<a href="t3://page?uid=current">Entonem</a>), can contribute to the reduction of leaf miner larvae, when applied on the leaves of the tomato plant.</p>

image

Result:
<p>Las avispas parásitas<em>Diglyphus</em><em>isaea</em>(<a href="t3://page?uid=current">Miglyphus</a>) y<em>Dacnusa sibirica</em>(<a href="t3://page?uid=current">Minusa</a>), son enemigos naturales eficaces de las larvas del minador. Además, el nematodo beneficioso<em>Steinernema feltiae</em>(<a href="t3://page?uid=current">Entonem</a>), puede contribuir a la reducción de las larvas del minador, cuando se aplica sobre las hojas de la planta de tomate.</p>

image

Expected
<p>Las avispas parásitas<em> Diglyphus</em> <em>isaea</em> (<a href="t3://page?uid=current">Miglyphus</a>) y<em> Dacnusa sibirica</em> (<a href="t3://page?uid=current">Minusa</a>), son eficaces enemigos naturales de las larvas del minador. Además, el nematodo beneficioso <em>Steinernema feltiae</em> (<a href="t3://page?uid=current">Entonem</a>), puede contribuir a la reducción de las larvas del minador, cuando se aplica en las hojas de la planta del tomate.</p>

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NarkNiro NarkNiro

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Respect rte allow tags in translate request web-vision/deepltranslate-core
2 participants
@NarkNiro @joey-bolts