seamless-m4t-v2-large transformers 使用gpu推理

GPU version:

step 1, model???? = model.to('cuda:0')???? or? model???? = model.to('cuda')

step2:? audio_inputs = processor(audios=audio, return_tensors="pt").to('cuda:0')? or audio_inputs = processor(audios=audio, return_tensors="pt").to('cuda')

CPU version:

step 1 not valid for cpu , so marked this line,???? #model???? = model.to('cuda:0')???? or? model???? = model.to('cuda')

step2:? audio_inputs = processor(audios=audio, return_tensors="pt")

records:

推理3s的音频转音频性能对比， rtx4090 vs cpu 11870

rtx4090:? 0.5s

cpu 11870:? 3.25s

conclusion:???? rtx4090 is 650% better

issues:

model.to('cuda:1')?? 出现下面问题，可能是因为gpu显存不足问题

t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking#ip=1