Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ES Error when importing large graph - total fields exceeded #10580

Closed
chiatt opened this issue Feb 5, 2024 · 1 comment
Closed

ES Error when importing large graph - total fields exceeded #10580

chiatt opened this issue Feb 5, 2024 · 1 comment

Comments

@chiatt
Copy link
Member

chiatt commented Feb 5, 2024

Importing a graph with a lot of nodes using the Graph Manager results in an error from ElasticSearch:

Traceback (most recent call last):
  File "/web_root/arches/arches/app/utils/data_management/resource_graphs/importer.py", line 128, in import_graph
    graph.save()
  File "/web_root/arches/arches/app/models/graph.py", line 510, in save
    self.update_es_node_mapping(node, datatype_factory, se)
  File "/web_root/arches/arches/app/models/graph.py", line 464, in update_es_node_mapping
    se.create_mapping("resources", body=datatype_mapping)
  File "/web_root/arches/arches/app/search/search.py", line 164, in create_mapping
    self.es.indices.put_mapping(index=index, **body)
...
elasticsearch.BadRequestError: BadRequestError(400, 'illegal_argument_exception', 'Limit of total fields [1000] has been exceeded')

Increasing the limit with the following command resolves the issue:

curl -H "Content-Type: application/json" -XPUT http://localhost:9202/[my project name]_resources/_settings -d  "{\"index.mapping.total_fields.limit\": 2000}"

We may consider raising this limit to 2000 when defining the mapping if this is becomes a common issue.

@chiatt chiatt added this to pipeline Feb 5, 2024
@chiatt chiatt moved this to 🆕 Under Review/Discussion in pipeline Feb 5, 2024
@chiatt
Copy link
Member Author

chiatt commented Feb 7, 2025

Haven run into this issue in over a year - closing

@chiatt chiatt closed this as completed Feb 7, 2025
@github-project-automation github-project-automation bot moved this from 🆕 Under Review/Discussion to ✅ Done in pipeline Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

1 participant