avoid many failures for ImageGPT #34071
Conversation
| @unittest.skip( | ||
| reason="After #33533, this still passes, but many subsequential tests fail with `device-side assert triggered`" | ||
| ) |
There was a problem hiding this comment.
just a thought, but can it is be because we need to self.asserts(ValueError) and not RuntimeError? Similar test for video/image_inputs are not causing massive failures
There was a problem hiding this comment.
But this test itself is still passing, so what the code gives (expected) RuntimeError. Not sure if we can adjust the test to use ValueError.
I can give it a try without any asserts to see if other tests are affected.
There was a problem hiding this comment.
I think the desired error (here RuntimeError) from such input (remove some images from inputs) causing the cdua in a bad state (I tried without assert it). Not sure what would be best approach if we want such test case. But before we have an idea, just skip it 😅
There was a problem hiding this comment.
oke, thanks! Will have it noted for such tests in the future
There was a problem hiding this comment.
@zucchini-nlp Running that test in a subprocess could avoid the issue
But the necessary change is not super trivial
* skip * [run-slow] imagegpt * skip * [run-slow] imagegpt * [run-slow] imagegpt,video_llava * skip * [run-slow] imagegpt,video_llava --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* skip * [run-slow] imagegpt * skip * [run-slow] imagegpt * [run-slow] imagegpt,video_llava * skip * [run-slow] imagegpt,video_llava --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* skip * [run-slow] imagegpt * skip * [run-slow] imagegpt * [run-slow] imagegpt,video_llava * skip * [run-slow] imagegpt,video_llava --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
What does this PR do?
See comment in the change