[Plato imports] Encoding problems for imported publication abstracts

[nicolasfranck]

Bug description

Some imported publications from Plato have encoding problems
in their abstracts.

Steps to Reproduce

Case 1

1. Import the latest candidate records
2. Go to /candidate-records
3. Open record with title A Concrete State: Constructing Materials and Building Ambitions in the (Belgian) Congo
4. Go abstract

I have imported this in https://backoffice.bibliotest.ugent.be/publication/01J1CGV8G7PDYEW6TNS999AM9T#abstracts to have a look

Plato ID: 1972

Case 2

1. Go to https://backoffice.bibliodev.ugent.be/candidate-records
2. Find "Unravelling urban soil properties and their spatial patterns to assess urban soil functioning", 2023, LA20 & WE12
3. Preview

[nicolasfranck]

this is how it looks like in the plato api

It looks like a <92> is put there, which is a curly quote from the Windows-1252 character encoding

cf. https://stackoverflow.com/questions/19984396/utf-8-or-92

[mietcls]

Input gathered live via Nicolas Franck: probably not an issue for us, but for Plato.

To do: contact Michiel

[mietcls]

Michiel contacted:

Michiel:

Unicodes \u0091, \u0092 en \u0097 zijn respectievelijk 'PRIVATE USE ONE', 'PRIVATE USE TWO' en 'End of Guarded Area'; bedoeld voor zelf te definiëren karakters.

Nicolas S:

Best to fix it on display, cfr. HTML stripping.

[mietcls]

Related issue with a rejected record:

[mietcls]

TODO: ask source – "any chance this can get fixed on your side"