Failed to run koboldai on NixOS unstable

[pasqui23]

; nix run github:nixified-ai/flake#koboldai-nvidia                                                                                                                                                                                                         # ❌1 took 2s zsh 
warning: binary cache 'http://127.0.0.1:12304' is for Nix stores with prefix 'Nix::Store::getStoreDir', not '/nix/store'
Traceback (most recent call last):
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1076, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/onnx/config.py", line 35, in <module>
    from PIL import Image
  File "/nix/store/ndcb708sh41sjbszxj27a78jl6d2amcz-python3-3.10.11-env/lib/python3.10/site-packages/PIL/Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: /nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/0d4xl0xk1g0w41yqyd64jvzbip5lhfig-libXdmcp-1.1.3/lib/libXdmcp.so.6)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1076, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/nix/store/iw1vmh509hcbby8dbpsaanbri4zsq7dj-python3-3.10.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 47, in <module>
    from .configuration_gpt2 import GPT2Config
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/models/gpt2/configuration_gpt2.py", line 23, in <module>
    from ...onnx import OnnxConfigWithPast, PatchingSpec
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1066, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1078, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.onnx.config because of the following error (look up to see its traceback):
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/0d4xl0xk1g0w41yqyd64jvzbip5lhfig-libXdmcp-1.1.3/lib/libXdmcp.so.6)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/nix/store/513r43w6qkrv1ri5m49dawhjvrp205dw-koboldAi-patchedSrc/aiserver.py", line 62, in <module>
    from transformers import StoppingCriteria, GPT2Tokenizer, GPT2LMHeadModel, GPTNeoForCausalLM, GPTNeoModel, AutoModelForCausalLM, AutoTokenizer, PreTrainedModel, modeling_utils
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1067, in __getattr__
    value = getattr(module, name)
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1066, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/nix/store/blsb0ajywpv3ahbzxwaf2d23r77cxb5n-python3-3.10.10-env/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 1078, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.gpt2.modeling_gpt2 because of the following error (look up to see its traceback):
Failed to import transformers.onnx.config because of the following error (look up to see its traceback):
/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224/lib/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/0d4xl0xk1g0w41yqyd64jvzbip5lhfig-libXdmcp-1.1.3/lib/libXdmcp.so.6)

[MatthewCroughan]

This can happen for a few reasons, such as setting LD_PRELOAD or LD_LIBRARY_PATH incorrectly. Sometimes I've had this happen when I have upgraded NixOS and not rebooted (some variables need to be set in my environment correctly, but don't until I reboot). So I do recommend rebooting the system just to see if this is the case.

You can give a go at running koboldai with a newer glibc like this

NIXPKGS_ALLOW_UNFREE=1 nix run --impure --override-input nixpkgs github:nixos/nixpkgs/nixos-unstable github:nixified-ai/flake#koboldai-nvidia

We need NIXPKGS_ALLOW_UNFREE because one of the library dependencies of koboldai have recently been marked unfree in Nixpkgs.

[max-privatevoid]

It's probably because our glibc is too old for the graphics drivers on your system. Overriding our nixpkgs input will solve that problem, but it might introduce new ones.

[max-privatevoid]

Closing as we've deprecated our support for KoboldAI. The root cause is being tracked in #35.