Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: 导出 CSV 时特殊处理逗号等字符 #3091

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions packages/s2-core/__tests__/unit/utils/export/export-pivot-spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -686,4 +686,74 @@ describe('PivotSheet Export Test', () => {
expect(result.split(TAB_SEPARATOR)).toHaveLength(81);
},
);

// https://github.com/antvis/S2/issues/2880
it('should escape CSV Field', async () => {
const data = clone<DataItem[]>(originData);

data.unshift({
number: 7789,
province: 'ac, abs, moon',
city: 'Venture "Extended Edition"',
type: 'Venture "Extended Edition, Very Large"',
sub_type: 'MUST SELL!\nair, moon roof, loaded',
});

const s2 = new PivotSheet(
getContainer(),
assembleDataCfg({
data,
fields: {
valueInCols: true,
columns: ['province', 'city'],
rows: ['type', 'sub_type'],
values: ['number'],
},
meta: [
{
field: 'number',
name: '数,量',
formatter: (value) => {
return Number(value)
.toFixed(3)
.toString()
.replace(/(\d)(?=(\d{3})+.)/g, '$1,');
},
},
{
field: 'province',
name: '省份',
formatter: (value) => `${value},`,
},
{
field: 'city',
name: '城市',
formatter: (value) => `${value}\t`,
},
{
field: 'type',
name: '类别',
formatter: (value) => `${value}\n`,
},
{
field: 'sub_type',
name: '子类别',
formatter: (value) => `${value}"`,
},
],
}),
assembleOptions(),
);

await s2.render();
const result = await asyncGetAllPlainData({
sheetInstance: s2,
split: CSV_SEPARATOR,
formatOptions: { formatHeader: true, formatData: true },
});

expect(result).toContain(`,"ac, abs, moon,",`);
expect(result).toContain(`,"7,789.000",`);
expect(result).toContain(`,"Venture ""Extended Edition""",`);
});
});
48 changes: 48 additions & 0 deletions packages/s2-core/__tests__/unit/utils/export/method-spec.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import {
convertString,
escapeCSVField,
keyEqualTo,
trimTabSeparator,
} from '../../../../src/utils/export/method';
Expand Down Expand Up @@ -29,3 +30,50 @@ describe('method test', () => {
expect(trimTabSeparator(1 as unknown as string)).toBe(1);
});
});

type SimpleData = string | number | null | undefined;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为啥要额外定义一次

describe('escapeCSVField', () => {
it('should return the same value for non-string types', () => {
const testData: SimpleData[] = [42, null, undefined];

testData.forEach((input) => {
expect(escapeCSVField(input)).toBe(input);
});
});

it('should return the same string if no special characters are present', () => {
const testStrings = ['hello', '123', 'test'];

testStrings.forEach((str) => {
expect(escapeCSVField(str)).toBe(str);
});
});

it('should escape double quotes by replacing with two double quotes', () => {
const input = 'hello "world"';
const expected = '"hello ""world"""';

expect(escapeCSVField(input)).toBe(expected);
});

it('should wrap strings containing commas in double quotes', () => {
const input = 'hello,world';
const expected = '"hello,world"';

expect(escapeCSVField(input)).toBe(expected);
});

it('should wrap strings containing newlines in double quotes', () => {
const input = 'hello\nworld';
const expected = '"hello\nworld"';

expect(escapeCSVField(input)).toBe(expected);
});

it('should wrap strings containing tabs in double quotes', () => {
const input = 'hello\tworld';
const expected = '"hello\tworld"';

expect(escapeCSVField(input)).toBe(expected);
});
});
16 changes: 14 additions & 2 deletions packages/s2-core/src/utils/export/copy/base-data-cell-copy.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { map } from 'lodash';
import {
AsyncRenderThreshold,
CSV_SEPARATOR,
TAB_SEPARATOR,
type DataItem,
type Formatter,
Expand All @@ -14,7 +16,11 @@ import type {
import { CopyMIMEType } from '../../../common/interface/export';
import { Node } from '../../../facet/layout/node';
import type { SpreadSheet } from '../../../sheet-type';
import { getHeaderList, getHeaderMeasureFieldNames } from '../method';
import {
escapeCSVField,
getHeaderList,
getHeaderMeasureFieldNames,
} from '../method';
import { unifyConfig } from './common';

export abstract class BaseDataCellCopy {
Expand Down Expand Up @@ -50,8 +56,14 @@ export abstract class BaseDataCellCopy {
dataMatrix: SimpleData[][],
separator: string,
): CopyablePlain {
let escapeDataMatrix: SimpleData[][] = dataMatrix;

if (separator === CSV_SEPARATOR) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

考虑下是否去掉 convertString, 理论上刷选复制/全部复制/导出 可以走统一的转换

escapeDataMatrix = map(dataMatrix, (row) => map(row, escapeCSVField));
}

return this.config.transformers[CopyMIMEType.PLAIN](
dataMatrix,
escapeDataMatrix,
separator,
) as CopyablePlain;
}
Expand Down
24 changes: 24 additions & 0 deletions packages/s2-core/src/utils/export/method.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,30 @@ export const trimTabSeparator = (text: string) => {
return replace(text, new RegExp(TAB_SEPARATOR, 'g'), '');
};

/**
* https://en.wikipedia.org/wiki/Comma-separated_values#Example
* 根据 CSV 规范,按以下规则处理字段内容:
* 若字段包含 ,、"、\n 或 \t → 用双引号包裹字段。
* 若字段中的双引号 → 转义为两个双引号 ""。
* @param field
*/
export const escapeCSVField = (field: SimpleData): SimpleData => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The escapeCSVField function is a critical addition to handle CSV field escaping. Ensure that this function is thoroughly tested and validated to prevent any data corruption during CSV export.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议和 trimTabSeparator 统一下, trimTabSeparator 去掉,然后 escapeCSVField => escapeField

if (typeof field !== 'string') {
return field;
}

// 检查是否需要转义:包含逗号、双引号或换行符
if (/[",\n\t]/.test(field)) {
// 转义双引号 -> 两个双引号
field = field.replace(/"/g, '""');

// 用双引号包裹字段
return `"${field}"`;
}

return field;
};

export const getHeaderMeasureFieldNames = (
fields: string[],
spreadsheet: SpreadSheet,
Expand Down
Loading