AI Toolkit لـ VS Code يجمع بين مجموعة من النماذج من Azure AI Studio Catalog وكاتالوجات أخرى مثل Hugging Face. يُبسط هذا الأدوات عملية تطوير تطبيقات الذكاء الاصطناعي باستخدام أدوات ونماذج الذكاء الاصطناعي التوليدية من خلال:
- البدء باكتشاف النماذج وتجربة الأدوات.
- ضبط النماذج واستخدامها للاستدلال باستخدام الموارد الحاسوبية المحلية.
- الضبط والاستدلال عن بُعد باستخدام موارد Azure.
قم بتثبيت AI Toolkit لـ VS Code
[معاينة خاصة] إعداد بنقرة واحدة لتطبيقات Azure Container Apps لتشغيل ضبط النماذج والاستدلال في السحابة.
لنبدأ الآن في تطوير تطبيقك الذكي:
- تأكد من تثبيت برنامج تشغيل NVIDIA على الجهاز المضيف.
- قم بتشغيل
huggingface-cli loginإذا كنت تستخدم HF لاستغلال مجموعة البيانات. Oliveشرح إعدادات المفتاح لأي شيء يؤثر على استخدام الذاكرة.
نظرًا لأننا نستخدم بيئة WSL وهي مشتركة، تحتاج إلى تفعيل بيئة Conda يدويًا. بعد هذه الخطوة، يمكنك تشغيل الضبط أو الاستدلال.
conda activate [conda-env-name] لتجربة النموذج الأساسي بدون ضبط، يمكنك تشغيل هذا الأمر بعد تفعيل Conda.
cd inference
# Web browser interface allows to adjust a few parameters like max new token length, temperature and so on.
# User has to manually open the link (e.g. http://0.0.0.0:7860) in a browser after gradio initiates the connections.
python gradio_chat.py --baseonlyبمجرد فتح مساحة العمل في حاوية التطوير، افتح نافذة الأوامر (المسار الافتراضي هو جذر المشروع)، ثم قم بتشغيل الأمر أدناه لضبط نموذج LLM على مجموعة البيانات المختارة.
python finetuning/invoke_olive.py سيتم حفظ نقاط التحقق والنموذج النهائي في models folder.
Next run inferencing with the fune-tuned model through chats in a console, web browser or prompt flow.
cd inference
# Console interface.
python console_chat.py
# Web browser interface allows to adjust a few parameters like max new token length, temperature and so on.
# User has to manually open the link (e.g. http://127.0.0.1:7860) in a browser after gradio initiates the connections.
python gradio_chat.pyلاستخدام prompt flow in VS Code, please refer to this Quick Start.
Next, download the following model depending on the availability of a GPU on your device.
To initiate the local fine-tuning session using QLoRA, select a model you want to fine-tune from our catalog.
| Platform(s) | GPU available | Model name | Size (GB) |
|---|---|---|---|
| Windows | Yes | Phi-3-mini-4k-directml-int4-awq-block-128-onnx | 2.13GB |
| Linux | Yes | Phi-3-mini-4k-cuda-int4-onnx | 2.30GB |
| Windows Linux |
No | Phi-3-mini-4k-cpu-int4-rtn-block-32-acc-level-4-onnx | 2.72GB |
Note You do not need an Azure Account to download the models
The Phi3-mini (int4) model is approximately 2GB-3GB in size. Depending on your network speed, it could take a few minutes to download.
Start by selecting a project name and location. Next, select a model from the model catalog. You will be prompted to download the project template. You can then click "Configure Project" to adjust various settings.
We use Olive to run QLoRA fine-tuning on a PyTorch model from our catalog. All of the settings are preset with the default values to optimize to run the fine-tuning process locally with optimized use of memory, but it can be adjusted for your scenario.
- Fine tuning Getting Started Guide
- Fine tuning with a HuggingFace Dataset
- Fine tuning with Simple DataSet
- To run the model fine-tuning in your remote Azure Container App Environment, make sure your subscription has enough GPU capacity. Submit a support ticket to request the required capacity for your application. Get More Info about GPU capacity
- If you are using private dataset on HuggingFace, make sure you have a HuggingFace account and generate an access token
- Enable Remote Fine-tuning and Inference feature flag in the AI Toolkit for VS Code
- Open the VS Code Settings by selecting File -> Preferences -> Settings.
- Navigate to Extensions and select AI Toolkit.
- Select the "Enable Remote Fine-tuning And Inference" option.
- Reload VS Code to take effect.
- Execute the command palette
AI Toolkit: Focus on Resource View. - Navigate to Model Fine-tuning to access the model catalog. Assign a name to your project and select its location on your machine. Then, hit the "Configure Project" button.
- Project Configuration
- Avoid enabling the "Fine-tune locally" option.
- The Olive configuration settings will appear with pre-set default values. Please adjust and fill in these configurations as required.
- Move on to Generate Project. This stage leverages WSL and involves setting up a new Conda environment, preparing for future updates that include Dev Containers.
- Click on "Relaunch Window In Workspace" to open your remote development project.
Note: The project currently works either locally or remotely within the AI Toolkit for VS Code. If you choose "Fine-tune locally" during project creation, it will operate exclusively in WSL without remote development capabilities. On the other hand, if you forego enabling "Fine-tune locally", the project will be restricted to the remote Azure Container App environment.
To get started, you need to provision the Azure Resource for remote fine-tuning. Do this by running the AI Toolkit: Provision Azure Container Apps job for fine-tuning from the command palette.
Monitor the progress of the provision through the link displayed in the output channel.
If you're using private HuggingFace dataset, set your HuggingFace token as an environment variable to avoid the need for manual login on the Hugging Face Hub.
You can do this using the AI Toolkit: Add Azure Container Apps Job secret for fine-tuning command. With this command, you can set the secret name as HF_TOKEN and use your Hugging Face token as the secret value.
To start the remote fine-tuning job, execute the AI Toolkit: Run fine-tuning command.
To view the system and console logs, you can visit the Azure portal using the link in the output panel (more steps at View and Query Logs on Azure). Or, you can view the console logs directly in the VSCode output panel by running the command AI Toolkit: Show the running fine-tuning job streaming logs.
Note: The job might be queued due to insufficient resources. If the log is not displayed, execute the
AI Toolkit: Show the running fine-tuning job streaming logscommand, wait for a while and then execute the command again to re-connect to the streaming log.
During this process, QLoRA will be used for fine-tuning, and will create LoRA adapters for the model to use during inference. The results of the fine-tuning will be stored in the Azure Files.
After the adapters are trained in the remote environment, use a simple Gradio application to interact with the model.
Similar to the fine-tuning process, you need to set up the Azure Resources for remote inference by executing the AI Toolkit: Provision Azure Container Apps for inference from the command palette.
By default, the subscription and the resource group for inference should match those used for fine-tuning. The inference will use the same Azure Container App Environment and access the model and model adapter stored in Azure Files, which were generated during the fine-tuning step.
If you wish to revise the inference code or reload the inference model, please execute the AI Toolkit: Deploy for inference command. This will synchronize your latest code with Azure Container App and restart the replica.
Once deployment is successfully completed, you can access the inference API by clicking on the "Go to Inference Endpoint" button displayed in the VSCode notification. Or, the web API endpoint can be found under ACA_APP_ENDPOINT in ./infra/inference.config.json وفي لوحة الإخراج. الآن أنت جاهز لتقييم النموذج باستخدام هذه النقطة النهائية.
لمزيد من المعلومات حول التطوير عن بُعد باستخدام AI Toolkit، راجع ضبط النماذج عن بُعد والاستدلال باستخدام النموذج المضبوط.
إخلاء المسؤولية:
تمت ترجمة هذا المستند باستخدام خدمات الترجمة الآلية بالذكاء الاصطناعي. بينما نسعى جاهدين لتحقيق الدقة، يرجى العلم أن الترجمات الآلية قد تحتوي على أخطاء أو عدم دقة. يجب اعتبار المستند الأصلي بلغته الأصلية المصدر الموثوق. للحصول على معلومات حاسمة، يُوصى بالاستعانة بترجمة بشرية احترافية. نحن غير مسؤولين عن أي سوء فهم أو تفسيرات خاطئة تنشأ عن استخدام هذه الترجمة.