Abstrakt
Recently, the CLIP model demonstrated impressive performance in text-image search and zero classification tasks. Hence, CLIP was used as a primary model in many cross-modal search tools at evaluation campaigns.
In this paper, we show a study performed with the model integrated to a successful video search tool at the respected Video Browser Showdown competition. The tool allows more complex querying actions on top of the primarymodel.
Specifically, temporal querying and Bayesian like relevance feedback were tested as well as their natural combination temporal relevance feedback. In a thorough analysis of the tool's performance, we show current limits of cross-modal searching with CLIP and also the impact of more advanced query formulation strategies.
We conclude that current cross-modal search models enable users to solve some types of tasks trivially with a single query, however, for more challenging tasks it is necessary to rely also on interactive search strategies.