-
Notifications
You must be signed in to change notification settings - Fork 386
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
648b14f
commit 7eec970
Showing
4 changed files
with
92 additions
and
1 deletion.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,78 @@ | ||
| C#/.NET binding of llama.cpp, provided with Web API and Native UI. | ||
| # LLamaSharp - .NET Bindings for llama.cpp | ||
|
|
||
|  | ||
|
|
||
| The C#/.NET binding of llama.cpp. It provides APIs to inference the LLaMa Models and deploy it on native environment or Web. It works on | ||
| both Windows and Linux and does NOT require compiling the library yourself. | ||
|
|
||
| ## Installation | ||
|
|
||
| Just search `LLama` in nuget package manager and install it! | ||
|
|
||
| ``` | ||
| PM> Install-Package LLama | ||
| ``` | ||
|
|
||
| ## Usages | ||
|
|
||
| Currently, `LLamaSharp` provides two kinds of model, `LLamaModelV1` and `LLamaModel`. Both of them works but `LLamaModel` is more recommended | ||
| because it provides better alignment with the master branch of [llama.cpp](https://github.com/ggerganov/llama.cpp). | ||
|
|
||
| Besides, `ChatSession` makes it easier to wrap your own chat bot. The code below is a simple example. For all examples, please refer to | ||
| [Examples](./LLama.Examples). | ||
|
|
||
| ```cs | ||
|
|
||
| var model = new LLamaModel(new LLamaParams(model: "<Your path>", n_ctx: 512, repeat_penalty: 1.0f)); | ||
| var session = new ChatSession<LLamaModel>(model).WithPromptFile("<Your prompt file path>") | ||
| .WithAntiprompt(new string[] { "User:" ); | ||
| Console.Write("\nUser:"); | ||
| while (true) | ||
| { | ||
| Console.ForegroundColor = ConsoleColor.Green; | ||
| var question = Console.ReadLine(); | ||
| Console.ForegroundColor = ConsoleColor.White; | ||
| var outputs = _model.Call(question); | ||
| foreach (var output in outputs) | ||
| { | ||
| Console.Write(output); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Demo | ||
|
|
||
|  | ||
|
|
||
| ## Roadmap | ||
|
|
||
| ✅ LLaMa model inference. | ||
|
|
||
| ✅ Embeddings generation. | ||
|
|
||
| ✅ Chat session. | ||
|
|
||
| 🔳 Quantization | ||
|
|
||
| 🔳 ASP.NET core Integration | ||
|
|
||
| 🔳 WPF UI Integration | ||
|
|
||
| ## Assets | ||
|
|
||
| The model weights is too large to include in the project. However some resources could be found below: | ||
|
|
||
| - [eachadea/ggml-vicuna-13b-1.1](https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/tree/main) | ||
| - [TheBloke/wizardLM-7B-GGML](https://huggingface.co/TheBloke/wizardLM-7B-GGML) | ||
| - Magnet: [magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352&dn=LLaMA](magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352&dn=LLaMA) | ||
|
|
||
| The weights included in the magnet is exactly the weights from [Facebook LLaMa](https://github.com/facebookresearch/llama). | ||
| The prompts could be found below: | ||
| - [llama.cpp prompts](https://github.com/ggerganov/llama.cpp/tree/master/prompts) | ||
| - [ChatGPT_DAN](https://github.com/0xk1h0/ChatGPT_DAN) | ||
| - [awesome-chatgpt-prompts-zh](https://github.com/PlexPt/awesome-chatgpt-prompts-zh) | ||
| ## License | ||
|
|
||
| This project is licensed under the terms of the MIT license. |