Skip to content

Commit 1978a49

Browse files
authored
Merge pull request #12 from matlab-deep-learning/remove_unnecessary_split
Remove unnecessary split
2 parents ea9fed4 + 99b3045 commit 1978a49

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

+bert/+tokenizer/+internal/WordPieceTokenizer.m

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@
9898
c = fread(fid,Inf);
9999
fclose(fid);
100100
c = native2unicode(c,'utf-8');%#ok
101-
words = split(splitlines(c')).';
101+
words = splitlines(c').';
102102
empties = cellfun(@isempty,words);
103103
words(empties) = [];
104104
vocab = wordEncoding(words);

0 commit comments

Comments
 (0)