In this paper we propose a novel algorithm for video summarization, OB-MMR (Optimized Balanced Audio Video Maximal Marginal Relevance). This algorithm is suitable to summarize both single and multiple videos. OB-MMR is achieved by optimizing the parameters in Balanced AV-MMR (Balanced Audio Video Maximal Marginal Relevance), namely the balance factor between audio information and visual information in the video, but also the importance of face and audio transitions among audio segments with different genres. Therefore, OB-MMR achieves a better result than previous algorithms, Video-MMR and Balanced AV-MMR. Furthermore, it is possible to select the optimized parameters for each genre of videos, which leads to promising automatic algorithms for video summarization in the future large-scale experiments.