❌ use vllm create_final_community_reports error

[xxll88]

only when create_final_community_reports , display error almost when is finished 。

          ❌ create_final_community_reports

None
⠙ GraphRAG Indexer
├── Loading Input (InputFileType.text) - 1 files loaded (0 filtered) 100%
├── create_base_text_units
├── create_base_extracted_entities
├── create_summarized_entities
├── create_base_entity_graph
├── create_final_entities
├── create_final_nodes
├── create_final_communities
├── join_text_units_to_entity_ids
├── create_final_relationships
├── join_text_units_to_relationship_ids
└── create_final_community_reports
❌ Errors occurred during the pipeline run, see logs for more details.

03:52:10,989 graphrag.index.reporting.file_workflow_callbacks INFO Error Invoking LLM details={'input': '\nYou are a helpful assistant responsible for >
01:07:09,532 graphrag.index.graph.extractors.community_reports.community_reports_extractor ERROR error generating community report

01:07:10,885 graphrag.index.reporting.file_workflow_callbacks INFO Community Report Extraction Error details=None
01:07:10,885 graphrag.index.verbs.graph.report.strategies.graph_intelligence.run_graph_intelligence WARNING No report found for community: 15
01:07:10,930 datashaper.workflow.workflow INFO executing verb window
01:07:10,931 datashaper.workflow.workflow ERROR Error executing verb "window" in create_final_community_reports: 'community'

vllm serve /mnt/vdb1/model/Qwen2-72B-Instruct-GPTQ-Int4
--host 0.0.0.0
--port 8000
--served-model-name qwen2-72b
--max-model-len 12000
--gpu-memory-utilization 0.8
--dtype bfloat16
--enforce_eager

llm:
api_key: "xx"
type: openai_chat
model: qwen2-72b
api_base: http://10.x.x.x:8000/v1
model_supports_json: true
max_tokens: 4000
request_timeout: 180.0

another question:
only when create community_reports, GPU is at low load about 5% , query once only every 180s , so speed is very slow.

[xxll88]

11:17:11,575 graphrag.index.run ERROR error running workflow create_final_community_reports
Traceback (most recent call last):
File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\graphrag\index\run.py", line 323, in run_pipeline
result = await workflow.run(context, callbacks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\datashaper\workflow\workflow.py", line 369, in run
timing = await self._execute_verb(node, context, callbacks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\datashaper\workflow\workflow.py", line 410, in _execute_verb
result = node.verb.func(**verb_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\datashaper\engine\verbs\window.py", line 73, in window
window = __window_function_mapwindow_operation

File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\frame.py", line 4102, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\shrnema\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\range.py", line 417, in get_loc
raise KeyError(key)
KeyError: 'community'
11:17:11,578 graphrag.index.reporting.file_workflow_callbacks INFO Error running pipeline! details=None

[maslenkovas]

I was having similar issue, for me the solution was in changing the community_report.txt prompt. In my case the model was generating community reports in a broken json format because of double curly braces in the prompt. When I changed the prompt by putting single curly braces instead of double ones - the error got solved.

Here are my error logs):
11:09:56,781 graphrag.index.reporting.file_workflow_callbacks INFO Community Report Extraction Error details=None 11:09:56,781 graphrag.index.verbs.graph.report.strategies.graph_intelligence.run_graph_intelligence WARNING No report found for community: 0 11:10:23,321 httpx INFO HTTP Request: POST api_base/chat/completions "HTTP/1.1 200 OK" 11:10:23,322 graphrag.llm.openai.utils ERROR error loading json, json={{ "title": "Machine Learning and its Applications",

and here is the updated prompt part:

- TITLE: community's name that represents its key entities - title should be short but specific. When possible, include representative named entities in the title.
- SUMMARY: An executive summary of the community's overall structure, how its entities are related to each other, and significant information associated with its entities.
- IMPACT SEVERITY RATING: a float score between 0-10 that represents the severity of IMPACT posed by entities within the community.  IMPACT is the scored importance of a community.
- RATING EXPLANATION: Give a single sentence explanation of the IMPACT severity rating.
- DETAILED FINDINGS: A list of 5-10 key insights about the community. Each insight should have a short summary followed by multiple paragraphs of explanatory text grounded according to the grounding rules below. Be comprehensive.

Return output as a well-formed JSON-formatted string with the following format:
    {
        "title": <report_title>,
        "summary": <executive_summary>,
        "rating": <impact_severity_rating>,
        "rating_explanation": <rating_explanation>,
        "findings": [
            {
                "summary":<insight_1_summary>,
                "explanation": <insight_1_explanation>
            },
            {
                "summary":<insight_2_summary>,
                "explanation": <insight_2_explanation>
            }
        ]
    }```

[natoverse]

Consolidating alternate model issues here: #657

[keakon]

I switched to lmdeploy. It's about 2x faster then vllm when serving qwen2-7b and won't be stuck during create_final_community_reports.

[l4b4r4b4b4]

I was actually able to use functionary vllm server with a llama-3.1 AWQ quant and indexing as well as both local and global search works like a charm.