OSError: xxx does not appear to have a file named config.json. Checkout ‘https://huggingface.co//root/.cache/huggingface/hub/models–THUDM–glm-4-9b-chat//tree/None’ for available files.

问题

OSError: /root/.cache/huggingface/hub/models--THUDM--glm-4-9b-chat/ does not appear to have a file named config.json. Checkout 'https://huggingface.co//root/.cache/huggingface/hub/models--THUDM--glm-4-9b-chat//tree/None' for available files.

原因

 CUDA_VISIBLE_DEVICES=1 python3 -m vllm.entrypoints.openai.api_server --host 0.0.0.0 \--port 11001 \--gpu-memory-utilization 1.0 \--model /root/.cache/huggingface/hub/models--THUDM--glm-4-9b-chat/ \--tensor-parallel-size 1 \--trust-remote-code

上面的--model传入值，指向的目录下没有config.json文件。

解决方案

指向包含config.json存在的目录
如
/root/.cache/huggingface/hub/models--THUDM--glm-4-9b-chat/snapshots/04419001bc63e05e70991ade6da1f91c4aeec278/

 CUDA_VISIBLE_DEVICES=1 python3 -m vllm.entrypoints.openai.api_server --host 0.0.0.0 \--port 11001 \--gpu-memory-utilization 1.0 \--model /root/.cache/huggingface/hub/models--THUDM--glm-4-9b-chat/snapshots/04419001bc63e05e70991ade6da1f91c4aeec278/ \--tensor-parallel-size 1 \--trust-remote-code