离线加载huggingface模型

huggingface 本地加载模型

源码位置：

/home/anaconda3/envs/Cap3D/lib/python3.8/site-packages/huggingface_hub/file_download.py

阅读里面的函数，可以知道下载的文件 url 和存储位置

def hf_hub_download(
	...
) -> str:
    """Download a given file if it's not already present in the local cache.

    """
	...
	
    storage_folder = os.path.join(cache_dir, repo_folder_name(repo_id=repo_id, repo_type=repo_type))
    os.makedirs(storage_folder, exist_ok=True)
    print('storage dir is ', storage_folder)
	
	...
	
    url = hf_hub_url(repo_id, filename, repo_type=repo_type, revision=revision, endpoint=endpoint)
	
	print('url ', url)
 
    return pointer_path

根据输出可以知道是

>> storage_folder
'/home/.cache/huggingface/hub/models--google--flan-t5-xxl'

>> url 
https://huggingface.co/google/flan-t5-xxl/resolve/main/tokenizer_config.json

创建本地storage_folder

mkdir /home/.cache/huggingface/hub/models--google--flan-t5-xxl

huggingface文件下载

根据 repo_id 和 filename 从 huggingface官网搜索并下载文件

>> repo_id
'google/flan-t5-xxl'
>> filename
'tokenizer_config.json'