Conversation
Signed-off-by: Haiming94 <[email protected]>
Summary of ChangesHello @Haiming94, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the documentation for the LongContext-ICL Annotation challenge by adding a new Chinese README, making the competition details and setup instructions accessible to a Chinese-speaking audience. Concurrently, the English README has been refined to be more concise, focusing on immediate news and quick start steps, thereby improving overall clarity and user experience for participants. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new Chinese README file (READMD_cn.md) and modifies the existing English README.md. The English README.md has been significantly streamlined by removing detailed sections such as 'Introduction', 'Objectives', and 'Challenge Details', as well as a specific instruction for NVIDIA environment setup. While the addition of Chinese documentation is beneficial, the removal of comprehensive English content might impact the clarity and completeness for English-speaking users. Consider ensuring that all essential information is accessible in English, either by restoring the removed sections or by providing clear references to equivalent English resources.
| <!-- END NEWS --> | ||
|
|
||
| ## Introduction | ||
|
|
||
| The LongContext-ICL-Annotation Challenge focuses on automatic data annotation under long-context settings using Large Language Models (LLMs). The competition is built upon the Qwen3-4B model and adopts the In-context Learning (ICL) paradigm to investigate scalable and high-quality automated annotation methods. | ||
|
|
||
| Participating teams are required to use the officially provided datasets and design effective ICL-based annotation solutions tailored for ultra-long context scenarios. All submissions will be evaluated on a unified benchmark dataset. The Organizing Committee will conduct standardized evaluations and determine the final rankings based on the official evaluation results. | ||
|
|
||
| ## Objectives | ||
|
|
||
| This challenge takes Large Language Models (LLMs) as the core technical foundation and targets automated data annotation under ultra-long context constraints, aiming to explore novel paradigms that balance annotation efficiency and annotation accuracy. The competition focuses on the following key scientific and engineering challenges: | ||
|
|
||
| - 1. Instruction and Prompt Design: | ||
|
|
||
| How can effective model instructions and prompt strategies be designed in ultra-long context scenarios to guide LLMs toward stable and high-quality data annotation? | ||
| - 2. Ultra-Long Context Construction: | ||
|
|
||
| When the number of available annotation examples significantly exceeds the model’s context capacity, how can information-dense and structurally coherent ultra-long context inputs be constructed for target data annotation? | ||
| - 3. Multi-Turn and Continuous Annotation: | ||
|
|
||
| In automated multi-round dialogue or continuous interaction settings, how can ultra-long contexts be efficiently leveraged to achieve both consistency and scalability in data annotation? | ||
|
|
||
| ## Challenge Details | ||
|
|
||
| - Participating teams are expected to independently design a complete LLM-based automatic data annotation pipeline and validate their approach under a unified dataset and evaluation protocol. Evaluation scores and rankings will be published on a standardized leaderboard. | ||
|
|
||
| - In addition to prediction results, teams must submit a technical report and fully reproducible source code in accordance with the competition requirements. The Organizing Committee will reproduce submitted solutions and review the technical design. The final score will be calculated as a weighted combination of prediction performance and technical solution evaluation, with detailed rules specified by the competition. | ||
|
|
||
| - Teams are required to submit their technical reports and complete source code to the official OpenSeek GitHub repository designated by the competition. | ||
|
|
||
| - For additional details, please refer to [FlagOS platform](https://flagos.io/RaceDetail?id=296fmsd8&lang=en). All competition-related information is subject to the announcements published on the official platform. | ||
| --- | ||
|
|
There was a problem hiding this comment.
The removal of the 'Introduction', 'Objectives', and 'Challenge Details' sections from the English README significantly reduces the information available to English-speaking users. While a Chinese README has been added, it's important to maintain comprehensive documentation in English as well, or at least provide a clear link to an equivalent English resource if these details are moved elsewhere. Consider restoring these sections or providing a clear reference.
| flagScale | ||
| ``` | ||
|
|
||
| On NVIDIA platforms, it is recommended to create the environment using: `cd src && bash create_env_nvidia.sh` |
There was a problem hiding this comment.
The instruction for setting up the environment on NVIDIA platforms (cd src && bash create_env_nvidia.sh) was removed. Since the create_env_nvidia.sh script still exists in the src directory, this instruction remains relevant for users with NVIDIA GPUs. It should be restored to guide users on the correct environment setup.
1 participant
No description provided.