Skip to main content

环境安装

以python3.11、cuda12.8、pytorch2.8为例

pip install modelscope
pip install packaging
pip install transformers peft diffusers
pip install flash-attn --no-build-isolation

huggingface下载配置​

国内可以通过hfd分布式下载与

wget https://hf-mirror.com/hfd/hfd.sh
chmod a+x hfd.sh
export HF_ENDPOINT=https://hf-mirror.com

# 默认wget
./hfd.sh 

# 基于aria2
pip install aria2
./hfd.sh yeliudev/VideoMind-Dataset --dataset --include "ego4d/*" --include "ego4d/**" --local-dir /data0/hh_data/ --tool aria2c -x 4

下载出现401 Client Error Unauthorized for url：命令行输入export HF_TOKEN=xxxx（自己去hf网上申请一个就行了）

transformers dev安装​

git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e .

deepspeed 在 model, self.optimizer, self.lr_scheduler = self.accelerator.prepare 卡死​

参考：https://github.com/deepspeedai/DeepSpeed/issues/4176#issuecomment-2324519931

屏幕optimizer

    "optimizer": {
        "type": "AdamW",
        "params": {
            "lr": "auto",
            "betas": "auto",
            "eps": "auto",
            "weight_decay": "auto"
        }
    },