Finally, we report a long-term field study, in which our version of the Android API instrumented with usage patterns and associated examples was made available to public access. During 17 months, it received 77,863 visits, coming from 160 countries. We analyze these visits to answer two research questions: (a) Do API users search for source code examples? (b) Do API users search for examples of usage patterns?

Next, we describe the methodology followed to extract usage patterns for the Android API and we characterize such patterns, in terms of their representativeness and complexity. We also show the results of a validation of the extracted usage patterns with two professional Android developers.

First, we report a characterization study on the usage of the Android API by client applications. Our central goal is to check whether the Android API and the proposed dataset of Android clients are indeed interesting objects of study.

This paper reports the first (to the best of our knowledge) large-scale field study on the instrumentation of API documents with usage patterns. The study is based on the Android API, which is selected due to its complexity, size, and relevance to Android developers ( Syer et al., 2011 ; Ruiz et al., 2014 ). We consider an API usage pattern as set of API methods that are used together with a certain frequency ( Robillard et al., 2013 ). We extend a tool, called APIMiner ( Montandon et al., 2013 ), to mine usage patterns from a dataset of Android open-source applications. This tool also instruments the original API documents with information on the extracted usage patterns.

Methods in modern APIs are not used independently of each other, but according to some patterns ( Robillard et al., 2013 ; Long, Wang & Cai, 2009 ). For example, the Android JavaDoc page that documents the beginTransaction method explicitly reports that it is usually used together with setTransactionSucessful and endTransaction. However, this page is an exception and most usage patterns are not documented at all. We reach this conclusion after inspecting the Android documentation, searching for 100 popular usage patterns, mined from a dataset of 396 applications. We found that only 12 patterns are somehow documented.

Materials & Methods

In this section, we present the tool we used to enhance API documents with information on usage patterns, the dataset used to mine these patterns, and the support and confidence thresholds used by the mining algorithm.

APIMiner APIMiner (http://apiminer.org) is a tool that instruments JavaDocs with code examples, extracted from API clients (Montandon et al., 2013). As illustrated in Fig. 1, an “Example Button” is included in the original documentation, before the signature of each API method. By clicking on these buttons, developers are presented with source code examples for the documented API methods. A detailed presentation of the algorithms used by APIMiner to extract, summarize, and rank examples is out of the scope of this paper and we refer the interested reader to our previous work (Montandon et al., 2013). Figure 1: JavaDoc instrumented by APIMiner (Montandon et al., 2013). For this article, we extend APIMiner with a capability to provide examples for API methods that are often called together. An API usage pattern has the following form: M ⇒ M 1 , M 2 , … , M n where M and M i , 1 ≤ i ≤ n, are methods from the API of interest. This pattern expresses that when the method M (antecedent term) is called by a given client method C, methods M 1 , M 2 , …, M n (consequent terms) are usually called by C, not necessarily in this order. The extension adds a new module to APIMiner for inferring usage patterns. This module relies on the FP-Growth association rules mining algorithm, as implemented by the Weka data mining library (http://www.cs.waikato.ac.nz/ml/weka). The transactions used as input to FP-Growth are the methods of the client systems; the items are the Android methods called by these client methods (all calls are statically extracted, based on the static types of the target objects). Transactions with a single call to an API method are not relevant for our purposes. Therefore, they are discarded before executing the FP-Growth algorithm. To evaluate the significance of the extracted patterns we rely on two measures: support (number of transactions that include the methods in the pattern) and confidence (the probability of finding the methods from the consequent term in the subset of the transactions including the antecedent method). We also extend the JavaDoc interface to show examples of usage patterns, as presented in Fig. 2. In the usage scenario illustrated by this figure, the API user initially requested examples for the beginTransaction() method, using the original interface provided by APIMiner. The window presenting the examples has a bottom panel with the usage patterns that have beginTransaction as the antecedent term. For instance, after selecting the first pattern the user sees examples that include not only beginTransaction() but also endTransaction(). Figure 2: Interface for presenting examples for API usage patterns. We also instrumented the JavaDoc sections that document the API methods with information on other methods they are frequently called with (if any), as presented in Fig. 3. Figure 3: Usage patterns are presented in the detailed documentation of methods.

Mining dataset Different from work that processes Android bytecode downloaded from Google Play (Ruiz et al., 2014), in our case it is important to have the original source code, to guarantee the extraction of examples with a minimum level of legibility. For this reason, we relied on GitHub to construct a dataset with the source code of Android systems, which we use for mining the usage patterns considered in this paper. We downloaded Android projects from GitHub that have at least 50 commits, to restrict the analysis to projects with a minimum level of activity, and that are not forks of other projects, to avoid many similar projects in the dataset. By considering these requirements, we create a mining dataset with 396 projects, including well-known applications, such as WordPress, Astrid, K9, and ConnectBot. Considering all projects, the dataset includes 57,658 classes and 450,762 methods. For this study, we use version 4.1.2-r1 of the Android API. We initially evaluate the usage of the API by the systems in the proposed dataset. Considering just the methods that call methods from the Android API, 59% call a second API method, which shows the feasibility of searching for usage patterns. Moreover, in the dataset, 40% of the public or protected methods from the Android API are never called. We also analyzed the distribution of the number of API calls in our dataset. Figure 4 shows two histograms with the frequency of the number of calls. For each value n in the x-axis, the y-axis represents the number of methods with exactly n calls. To ease visualization, the first histogram shows methods with at most 40 calls; and the second histogram with at most 300 calls (in the full dataset, the number of calls ranges from 1 to 6,729). The histogram is right-skewed, meaning that while most methods are called few times (median equal to 5), we also have high and very high values. This finding implies that centrality and dispersion statistics measures (e.g., mean and standard deviation) should not be used to describe our empirical data on number of calls. Instead, the histogram suggests the data best fits a heavy-tailed distribution, possibly a power law. To check this possibility, we use the statistical framework proposed by Clauset, Shalizi & Newman (2009) for discerning power-law behavior in empirical data. By following this framework, we reject the (null) hypothesis that power-laws are a plausible explanation for our data, for a significance level of 5% (p-value = 0.00). However, this is not the same as concluding that the number of calls do not match a heavy-tailed distribution. In fact, Fig. 4 suggests a heavy-tailed behavior, possibly matching an alternative distribution (e.g., stretched exponential or log-normal). Figure 4: Histogram of number of calls. (A) Up to 40 method calls; (B) up to 300 method calls. Summary: Client methods tend to make multiple calls to the Android API (59% of the methods calling an API method, call at least a second one), which shows that there is space for mining methods that are frequently called together. However, the usage of the Android API follows a highly right-skewed distribution, which suggests that we should not expect to derive usage patterns for most API methods.