Last week, I made a brief post updating on how I solved my object detection problem, and could continue working on other aspects of this project. I actually had a most of the work done I am about to talk about, but just wanted to tidy up code, and make a presentable data dump of the recent SSBM major tournament Genesis 6. Note that all code is available on Github. A detailed breakdown of the stats below can be found here.

Step 1: Determine when percent sprites are present in the video.

Firstly, I would need to determine when a match “exists” within a video file. To do this, I used the percent symbols as an “anchor”, since it took relatively low computation power. The numbers at the top left represent the confidence of the bounding boxes in order of color (Blue, Green, Red, Cyan). Notice how percent symbols have a confidence above 0.8, which is the value I used to threshold whether melee is on screen or not.

However, there are a few shortcuts I make that are hidden from the gif above. Firstly, I scan a random number of frames in the video to find the most accurate size of percent symbol I should be searching for. Currently, I am working with 360p footage, so I assume the width of the percent symbol varies between (24, 30). Next, I make sure not to scan every frame, since it would be take way too long. I currently run with step sizes of 60 frames, which corresponds to 2 seconds at 30fps. Also, notice how bounding boxes are only appearing in the bottom quarter of the screen. This is another shortcut to save computation time, since percent symbols will always be in the lower section of the screen.

Step 2: Find the Stage Corresponding to each match

As per part 1 of this series, I used the Darkflow repository to search for stages within each video. Some useful shortcuts I took were only searching a limited number of frames within each match range that I determined from the previous step. I also ran into the issue of “Highlight Reels” being mis-categorized as matches. To deal with this, I removed all matches that had multiple stages found from the random sampling I performed.

Conclusions

I compared the results between Genesis 5 and 6 Top 8 as shown in the image at the beginning of this article. For a more detailed report check out this pastebin. My next plans are to start working on text detection for the timer and percent counters, so I can get some valuable damage info to present.

I know that I’m doing work similar to what metascouter has already done, but I’m focusing on speed and the possibility to perform this analysis AUTOMATICALLY on ANY video so it can hopefully be applied to the SmashVods database. Props to those guys though, they have been giving me a bunch of inspiration.