Skip to content

Add an example to show how to handle a non-UTF-8 page #39385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,59 @@ browser.webRequest.onBeforeRequest.addListener(
);
```

This example shows, how to handle a non-UTF-8 page:

```js
// This example uses https://github.com/t-kouyama/fast-sjis-encoder
// Note: Make sure the file you paste in has the encoding set to UTF-8
function listener(details) {
if (details.statusCode != 200) {
return;
}

const filter = browser.webRequest.filterResponseData(details.requestId);
let decoder = new TextDecoder();

const data = [];
filter.ondata = (event) => {
data.push(new Uint8Array(event.data));
};

filter.onstop = (event) => {
const combinedLength = data.reduce((acc, buffer) => acc + buffer.length, 0);
const combinedArray = new Uint8Array(combinedLength);
let writeOffset = 0;
for (const buffer of data) {
combinedArray.set(buffer, writeOffset);
writeOffset += buffer.length;
}
// Get the charset from the page meta tag
const part = decoder.decode(combinedArray.slice(0, 1000));
const charset = part.match(/<meta charset="(.+?)">/)[1];
// Creates a new TextDecoder object with the label "shift_jis"
decoder = new TextDecoder(charset);

let str = decoder.decode(combinedArray);
// Translated with Google Translate
str = str.replace("ノートパソコン", "laptop");
str = str.replace("タブレット", "tablet");
str = str.replace("ハードディスク", "hard disk");
str = str.replace("PCパーツ", "PC parts");
str = str.replace("周辺機器", "Peripheral equipment");

// Need to use an external library because TextEncoder only supports "UTF-8"
filter.write(sjis(str));
filter.close();
};
}

browser.webRequest.onHeadersReceived.addListener(
listener,
{ urls: ["https://kakaku.com/"], types: ["main_frame"] },
["blocking"],
);
```

{{WebExtExamples}}

## Browser compatibility
Expand Down