You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Language models use very well the context, and the column name can give useful context. For instance the value "mouse" should probably be embedded differently if the column name is "peripheral" versus "animal".
My suggestion is to have an option to embed entries with "{column_name}: {entry}" to enable exploiting this piece of context.
Of course, ideally we should benchmark this option, to see how / when it brings value.
The benefit would be to disambiguate "State: New York" and "City: New York"
The text was updated successfully, but these errors were encountered:
jeromedockes
changed the title
New option to table encoder to prefix with the column name
New option to TextEncoder to prefix with the column name
Jan 13, 2025
Language models use very well the context, and the column name can give useful context. For instance the value "mouse" should probably be embedded differently if the column name is "peripheral" versus "animal".
My suggestion is to have an option to embed entries with "{column_name}: {entry}" to enable exploiting this piece of context.
Of course, ideally we should benchmark this option, to see how / when it brings value.
The benefit would be to disambiguate "State: New York" and "City: New York"
The text was updated successfully, but these errors were encountered: