Hi @li-plus! Thanks for your contribution!

Can you please enable gradient checkpointing tests for this model to make sure it works properly? I see these tests are skipped on main, however, I was able to run them without issues locally.

Hi @li-plus! Thanks for your contribution!

Can you please enable gradient checkpointing tests for this model to make sure it works properly? I see these tests are skipped on main, however, I was able to run them without issues locally.

@qubvel Thanks for advice. I've re-enable these gradient checkpointing tests for Qwen2VL in the latest commit. They run just fine on my machine.

Thanks! Can you please also push an empty commit with the message [run-slow] qwen2_vl to trigger all model tests? We should be fine here, cause test_training_gradient_checkpointing is not a slow test, however just to double check everything else is fine 🙂

Thanks. Just pushed!

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qubvel It seems those failures are not related to this PR. Any idea?

@li-plus

Thanks your commit!
when I use gradient_checkpointing for Qwen2VisionTransformerPretrainedModel, I meet this Error:

AttributeError: 'Qwen2VisionTransformerPretrainedModel' object has no attribute '_gradient_checkpointing_func'. Did you mean: 'gradient_checkpointing'?

Is it necessary to implement this '_gradient_checkpointing_func' in 'Qwen2VisionTransformerPretrainedModel' ?

@li-plus

Thanks your commit! when I use gradient_checkpointing for Qwen2VisionTransformerPretrainedModel, I meet this Error:

AttributeError: 'Qwen2VisionTransformerPretrainedModel' object has no attribute '_gradient_checkpointing_func'. Did you mean: 'gradient_checkpointing'?

Is it necessary to implement this '_gradient_checkpointing_func' in 'Qwen2VisionTransformerPretrainedModel' ?

My transfromers version is latest, and it is:
transformers ==4.47.0

@ShuaibinQi Hi, I did not reproduce this error using transformers 4.47.0 using this demo code:

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info


processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-7B-Instruct")

model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-7B-Instruct",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map="cuda",
)
model.gradient_checkpointing_enable()
model.train()

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to("cuda")

print(f'before forward {torch.cuda.memory_allocated()/1e9=:.3f} GB, {torch.cuda.memory_reserved()/1e9=:.3f} GB')
output = model(**inputs, use_cache=False)
print(f'after forward {torch.cuda.memory_allocated()/1e9=:.3f} GB, {torch.cuda.memory_reserved()/1e9=:.3f} GB')
output.logits.sum().backward()
print(f'after backward {torch.cuda.memory_allocated()/1e9=:.3f} GB, {torch.cuda.memory_reserved()/1e9=:.3f} GB')

Did you use gradient_checkpointing_enable to enable gradient checkpointing?