This page shows visualizations of some width-3 1-d convolutional filters from Google's lm_1b language model. Each column corresponds to one position in the filter, and shows the characters with the most positive weights. Use the checkbox in the bottom-right to also see the most negative weights (may be slow).

Below that are examples of words for which the filter emits the highest values. A filter's response is its maximum value over all substrings it sees in the word. So if a filter has high weights on 'c' in the first position, then 'a', then 't', it will assign equally high scores to 'cat', 'fatcat', 'concatenate', etc. The portion of the string in blue is the substring the filter is responding to.

'^' and '$' represent beginning and end of word markers, respectively. '_' is a padding character. Literal versions of those characters are escaped with a backslash.

Use the links at the top to see filters of other widths.

Check out my blog post here for a bit more context.