1. Network delay

Using the debug drawer we can change the network delay which is usually a good test when combined with orientation change or app killed in the background.

While a network request is in progress, how would the app handle screen orientation changes or leaving the app (and then coming back to it after a while)?

2. Error rate

Using the debug drawer we can change the error rate to check how the code handles unexpected errors.

Does the app handle HTTP errors and other errors as expected? Does it notify the user or log the error to Crashlytics as non-fatal crash?

3. Offline mode

Simply by turning OFF the WiFi or turning ON the airplane mode (for total connectivity loss).

We want to test things like:

Will the app crash if a network request was already in progress?

Notify the user that the connection was lost?

Is it caching what should have been cached?

4. Orientation change

Activities and non-retained fragments get destroyed and recreated on orientation change (assuming configChanges orientation was not set in the manifest, check Handling Runtime Changes).

We want to check the impact of orientation change on the performance and behavior of the app:

What happened to any ongoing requests?

Was the app state restored correctly?

Was the correct related landscape/portrait layout/resources loaded?

5. App killed while in the background

Sometimes Android will kill an app while it’s in the background to cleanup memory for other running apps. When we go back in the app all the dependencies required by the current screen should be re-initialized if needed (does the current view depend on something initialized in a previous screen).

Ways to reproduce:

Open the app Press device Home button Run the following command in terminal

adb shell am kill YOUR_PACKAGE_NAME

Another way would be to go to device Settings > Developer Options > look for “Do not keep activities” and check it

Optimizing for Doze and App Standby, is an interesting, somewhat related read.

6. App Update

Did the app sustain any previous data after the update? In other words, is this update backward compatible?

You need to worry about Serializable objects stored in the SharedPreferences, for example: will they get deserialized (changing the fields order can break deserialization), database scheme changes, etc…

7. Key Bashing

Vigorous swiping and tapping can produce some strange errors.

You can use Monkey for that, it is a command-line tool that you can run with the following command:

adb shell monkey -p YOUR_PACKAGE_NAME

8. Multi-Window mode (Android N)

If we enable multi-window support, we need to verify the following behavior under both multi-window and standard modes.

Launch the app in standard mode, then switch to multi-window mode by long-pressing the Overview button. Verify that the app switches properly.

Launch the app directly in multi-window mode, and verify that the app launches properly.

Resize your app in multi-window mode by dragging the divider line. Verify that the app resizes without crashing, and that necessary UI elements are visible.

If you have specified minimum dimensions for your app, attempt to resize the app below those dimensions. Verify that you cannot resize the app to be smaller than the specified minimum.

Through all tests, verify that your app’s performance is acceptable. For example, verify that there is not too long a lag to update the UI after the app is resized.

Testing checklist

We tried these operations in both standard and multi-window mode to verify app’s performance.

Enter and leave multi-window mode.

Switch from your app to another app, and verify that the app behaves properly while it is visible but not active. For example, if your app is playing video, verify that the video continues to play while the user is interacting with another app.

In multi-window mode, try moving the dividing bar to make your app both larger and smaller. Try these operations in both side-by-side and one-above-the-other configurations. Verify that the app does not crash, essential functionality is visible, and the resize operation doesn’t take too long.

Perform several resize operations in rapid succession. Verify that your app doesn’t crash or leak memory.

Use your app normally in a number of different window configurations, and verify that the app behaves properly. Verify that text is readable, and that UI elements aren’t too small to interact with.

More info about this here: https://developer.android.com/preview/features/multi-window.html

9. TransactionTooLargeException

If you are targeting Android Nougat, you need to be careful not to store more than 1MB (in theory, less in practice) in Binder transaction.

For example, saving data in onSaveInstance when moving between activities.

Technically the 1Mb limit is shared by all transactions in progress for the process so you might hit the limit way before that.