Design Representation

visual

textual

structural

interactive

For each app, Rico exposes Google Play Store metadata, a set of user interaction traces, and a list of all the unique UIs discovered. The Play Store metadata includes an app’s category, average rating, number of ratings, and number of downloads. Each user trace is composed of a sequence of UIs and user interactions that connect them. Each UI comprises a screenshot, an augmented Android view hierarchy, a set of explored user interactions, a set of animations capturing transition effects in response to user interaction, and a learned vector representation of the UI’s layout. View hierarchies capture all of the elements comprising a UI, their properties, and relationships between them. For each element, Rico exposes itsproperties such as screen position, dimensionality, and visibility,properties such as class name, id, and displayed text,properties such as a list of its children in the hierarchy, andproperties such as the ways a user can interact with it. Additionally, we annotate elements with any Android superclasses that they are derived from (e.g., TextView), which can help third-party applications reason about element types.