PicSOM experiments in TRECVID 2007
Our experiments in TRECVID 2007 include participation in the high-level feature extraction, search, and video summarization tasks, using a common system framework based on multiple parallel Self-Organizing Maps (SOMs). In the high-level feature extraction task, we applied a method of representing semantic concepts as class models on parallel SOMs, combined with external text search results. This year, we introduced a further post-processing stage in which the concepts' temporal and inter-concept co-occurrences were analyzed.We submitted the following six runs: A_PicSOM_1_6: Required visual baseline A_PicSOM_2_5: Visual features and text search A_PicSOM_3_3: Visual features using variable convolution and text search A_PicSOM_4_4: Visual features using variable convolution A_PicSOM_5_2: Visual features, text search, and temporal context based on training set A_PicSOM_6_1: Visual features, text search, and temporal context based on validation set The results show that the temporal and inter-concept co-occurrence analysis improved the results considerably. On the other hand, inclusion of the text search worsened the results, leading to overall degradation of performance also on the subsequent runs. For this reason, we later executed additional runs in which the co-occurrence post-processing stage was employed without the text search. In the search task, we submitted a total of six fully-automatic runs. In this year's experiments, we augmented the baseline ASR/MT search and content-based retrieval runs with high-level semantic concepts and pseudo relevance feedback. The overall settings for the six runs were as follows: F_A_1_PicSOM_1_6: Required text search baseline F_A_1_PicSOM_2_5: Required visual baseline F_A_2_PicSOM_3_4: Text search and visual features F_A_2_PicSOM_4_3: Text search and visual features, with pseudo relevance feedback F_A_2_PicSOM_5_2: Text search and visual features, semantic concepts F_A_2_PicSOM_6_1: Text search and visual features, semantic concepts, with pseudo relevance feedback In this year's experiments, retrieval based on the visual features performed very poorly, and consequently the text baseline outperformed also the combined run with both visual and text features. In the further experiments, the inclusion of both the semantic concepts and pseudo relevance feedback resulted in performance improvement.