ABSTRACT:
Video summarization is the procedure to extract the most significant contents of a video and to represent it concise form. The existing strategies for video summarization couldn’t achieve the satisfactory result for a video with camera movement, and critical illumination changes. To tackle these issues, in this project new system for video summarization is proposed based on Eye Tracker data as human eyes can track moving objects accurately in these cases.
The smooth pursuit is the condition of eye movement when a client follows after a moving object in a video. This motivates us to implement new technology to recognize smooth pursuit from another kind of gaze points, for example, fixation and saccade. The smooth pursuit gives just the area of moving objects in a video frame; in any case, it doesn’t show whether the locate moving objects are very attractive (i.e. salient regions) to viewers or not and in addition the amount of the movement of the moving objects. The amount of salient regions and object motions are two essential highlights to measure the viewer’s attention level for determining the keyframes for video summarization.
To discover the most attractive items, another spatial saliency prediction strategy is likewise proposed by building a saliency map around each smooth pursuit gaze point in view of the human visual field, for example, fovea, parafoveal, and perifovea regions. To recognize the measure of protest movements, the aggregate separations between the current and the previous gaze points of viewers during a smooth pursuit is estimated as a motion saliency score. The motivation is that the movement of eye gaze is identified with the motion of the objects during smooth pursuit.
Finally, both spatial and motion saliency maps are joined to get an accumulated saliency score for each frame and an arrangement of keyframes are chosen based on client chose or system default skimming ratio. The proposed strategy is implemented on Office video dataset that contains videos with camera movements and illumination changes. Experimental results confirm the superior performance of the proposed Spatial and Motion Saliency Prediction (SMSP) method compared with the state of the art strategies.
