@@ -7,7 +7,7 @@ One of the best ways to contribute is by writing templates!
7
7
A template is a piece of code written in a templating language called
8
8
[ Jinja] ( https://jinja.palletsprojects.com/en/3.0.x/ ) . A template defines
9
9
a function that maps an example from a dataset in the
10
- [ HuggingFace library] ( https://huggingface.co/datasets ) to two strings of
10
+ [ Hugging Face datasets library] ( https://huggingface.co/datasets ) to two strings of
11
11
text. The first is called the _ prompt_ which provides all information that
12
12
will be available to solve a task, such as the instruction and the context.
13
13
The second piece is called the _ output_ , which is the desired response to the
@@ -23,7 +23,7 @@ and find an unclaimed one. Put your name under "Who's Prompting it?" and
23
23
mark it yellow to show it's in progress.
24
24
1 . ** Examine the dataset.** Select or type the dataset into the dropdown in the app.
25
25
If the dataset has subsets (subsets are not the same as splits), you can select
26
- which one to work on. Note that templates are subset specific. You can find
26
+ which one to work on. Note that templates are subset- specific. You can find
27
27
out background information on the dataset by reading the information in the
28
28
app. The dataset is a collection of examples, and each example is a Python
29
29
dictionary. The sidebar will tell you the schema that each example has.
@@ -42,10 +42,10 @@ applied to the current example will appear in the right sidebar.
42
42
through a handful of examples of the prompted dataset using the
43
43
"Prompted dataset viewer" mode.
44
44
1 . ** Write between 5 and 10 templates** . Repeat the steps 4 to 8 to create between 5
45
- and 10 (more if you want!) templates per dataset. Feel free to introduce some diversity
45
+ and 10 (more if you want!) templates per dataset/subset . Feel free to introduce some diversity
46
46
both in the format and the formulation.
47
47
1 . ** Duplicate the template(s).** If the dataset you have chosen bear the same
48
- format as other datasets (for instance ` MNLI ` and ` SNLI ` have identical format ),
48
+ format as other datasets (for instance, ` MNLI ` and ` SNLI ` have identical formats ),
49
49
you can simply claim these datasets and duplicate the templates you have written
50
50
to these additional datasets.
51
51
1 . ** Upload the template(s).** Submit a PR using the instructions
@@ -56,7 +56,7 @@ to these additional datasets.
56
56
Here is a quick crash course on using [ Jinja] ( https://jinja.palletsprojects.com/en/3.0.x/ )
57
57
to write templates. More advanced usage is in the [ cookbook] ( #jinja-cookbook ) .
58
58
59
- Generally in a template, you'll want to use a mix of hard-coded data that is
59
+ Generally, in a template, you'll want to use a mix of hard-coded data that is
60
60
task-specific and stays the same across examples, and commands that tailor the
61
61
prompt and output to a specific example.
62
62
@@ -115,7 +115,7 @@ Is this a piece of news regarding {{"world politics"}}, {{"sports"}}, {{"busines
115
115
116
116
A few miscellaneous things:
117
117
118
- * ** Writing outputs.** When writing a template for an task that requires outputting
118
+ * ** Writing outputs.** When writing a template for a task that requires outputting
119
119
a label, don't use articles or other stop words before the label name in the output.
120
120
For example, in TREC, the output should be "Person", not "A person". The reason
121
121
is that evaluations often look at the first word of the generated output to determine
@@ -145,7 +145,7 @@ Is this question asking for a {{"definition"}}, a {{"description"}}, a {{"manner
145
145
```
146
146
* ** Conditional generation format.** Always specify the output label ` y ` and separate it from the prompt
147
147
by indicating the vertical bars ` ||| ` . The ` y ` will be generated by a generative model
148
- conditionned on the prompted input you wrote. You can always transform an "infix" prompt format
148
+ conditioned on the prompted input you wrote. You can always transform an "infix" prompt format
149
149
``` jinja2
150
150
Given that {{premise}}, it {{ ["must be true", "might be true", "must be false"][label] }} that {{hypothesis}}
151
151
```
@@ -158,7 +158,7 @@ Given that {{premise}}, it {{ "must be true, might be true, or must be false" }}
158
158
## Uploading Templates
159
159
160
160
Once you save or modify a template, the corresponding file inside the ` templates `
161
- directory in the repo will be modified. To upload it, following these steps:
161
+ directory in the repo will be modified. To upload it, follow these steps:
162
162
1 . Run ` make style ` and ` make quality ` .
163
163
2 . Commit the modified template files (anything under ` templates ` ) to git.
164
164
3 . Push to your fork on GitHub.
0 commit comments