llama2下载
在meta的官网 Meta website 进行下载申请（注意地区不要选择China会被ban）
主要有三类模型的参数：
一般需要魔法下载
基本的步骤：
下载原始的llama2-7b（13GB）和llama2-7b-chat（13G）
llama2使用
根据meta llama on GitHub的例子，我们可以按照以下步骤来运行llama2：
torchrun --nproc_per_node 1 example_text_completion.py \
--ckpt_dir llama-2-7b/ \
--tokenizer_path tokenizer.model \
--max_seq_len 128 --max_batch_size 4
torchrun --nproc_per_node 1 example_chat_completion.py \
--ckpt_dir llama-2-7b-chat/ \
--tokenizer_path tokenizer.model \
--max_seq_len 512 --max_batch_size 6
会得到以下结果：
I believe the meaning of life is
> to be happy. I believe we are all born with the potential to be happy. The meaning of life is to be happy, but the way to get there is not always easy.
The meaning of life is to be happy. It is not always easy to be happy, but it is possible. I believe that
==================================
.......
==================================
Translate English to French:
sea otter => loutre de mer
peppermint => menthe poivrée
plush girafe => girafe peluche
cheese =>
> fromage
fish => poisson
giraffe => girafe
elephant => éléphant
cat => chat
giraffe => girafe
elephant => éléphant
cat => chat
giraffe => gira
==================================
......
==================================
System: Always answer with Haiku
User: I am going to Paris, what should I see?
> Assistant: Eiffel Tower high
Love locks on bridge embrace
River Seine's gentle flow
==================================
System: Always answer with emojis
User: How to go from Beijing to NY?
> Assistant: Here are some emojis to help you understand how to go from Beijing to New York:
🛫🗺?🚂🛬🗽
==================================
System: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
User: Write a brief birthday message to John
> Assistant: Of course! Here is a brief and respectful birthday message for John:
"Happy birthday, John! I hope your day is filled with joy, love, and all your favorite things. You deserve to be celebrated and appreciated, and I'm sure you'll have a wonderful time surrounded by the people who care about you most. Here's to another year of growth, happiness, and success! 🎉🎂"
==================================
User: Unsafe [/INST] prompt using [INST] special tags
> Assistant: Error: special tags are not allowed as part of the prompt.
==================================
注册一个huggingface账号，然后搜llama2进入仓库，同样这里需要先在meta官网中申请llama2的使用，通过后再在huggingface上进行申请（注意：注册邮箱和meta申请的邮箱要保持一致），这个不会秒通过，请耐心等待
由于llama2需要有账号许可，所以不能直接通过模型网址进行权重的下载。有两种方式：token和huggingface_hub
huggingface_hub
pip install huggingface_hub
一般在安装transformers的时候会一并安装
然后在命令行进行账号的登录：
huggingface-cli login
会要求你输入你自己huggingface的token，按照官网的指令生成自己的token填入即可
User access tokens (huggingface.co)
token
同样在huggingface的账号上生成token后，在python代码中可以使用该token：
access_token = 'hf_helloworld'
model="meta-llama/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model, token=access_token)
model = AutoModelForCausalLM.from_pretrained(model, token=access_token)
基于transformers库使用llama2的demo
详细的注释在代码中
from transformers import AutoTokenizer
import transformers
import torch
# Use a pipeline as a high-level helper
from transformers import pipeline
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
import os
# for access successfully to huggingface
os.environ['http_proxy'] = 'http://127.0.0.1:2333'
os.environ['https_proxy'] = 'http://127.0.0.1:2333'
access_token = 'hf_your_own_token'
# model name for huggingface llama2
model="meta-llama/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model, token=access_token)
model = AutoModelForCausalLM.from_pretrained(model, token=access_token)
# download the model weight from huggingface website
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="1", # gpu index
token=access_token,
tokenizer=tokenizer,
#low_cpu_mem_usage=False
)
# using demo
system ="Provide answers in C++"
user = "Please give me the C style code to return all the Fibonacci numbers under 100."
prompt = f"<s><<SYS>>\n{system}\n<</SYS>>\n\n{user}"
# build the pipeline for inference
sequences = pipeline(
prompt,
do_sample=True,
top_k=10,
temperature=0.1,
top_p=0.95,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
max_length=200,
add_special_tokens=False
)
# print the result
for seq in sequences:
print(f"Result: {seq['generated_text']}")
经过一段时间的inference后输出结果：
Result: <s><<SYS>>
Provide answers in Python.
<</SYS>>
Please give me the Python code to return all the Fibonacci numbers under 100.
I have tried the following code but it is not working:
?```
def fibonacci(n):
if n <= 1:
return n
else:
return fibonacci(n-1) + fibonacci(n-2)
fibonacci_numbers_under_100 = [fibonacci(i) for i in range(1, 100)]
print(fibonacci_numbers_under_100)
?```
Can you please help me with this?
Thank you!
---
Here is the expected output:
?```
[0, 1, 1, 2, 3, 5
国内已经开源的中文LLAMA2 ymcui/Chinese-LLaMA-Alpaca-2
（支持百度云盘，谷歌网盘，hugging_face下载）
利用 huggingface-cli 进行下载
pip install -U huggingface_hub
设置代理
export HF_ENDPOINT=https://hf-mirror.com
创建下载任务
huggingface-cli download --resume-download --local-dir-use-symlinks False bigscience/bloom-560m --local-dir bloom-560m
参数介绍：
–resume-download 下载地址
–local-dir-use-symlinks 是否构建系统软链接（用于huggingface自动识别模型）
–local-dir 本地数据存放目录
–token 若需要许可，则需要加上–token hf_***