Native Apps - Inspect elements by using UI Automator Viewer and Appium Desktop Inspector

There are many different tools that help you inspect elements in mobile apps. But the most popular and widely used tools are:

1. UIAutomatorViewer: This is a tool provided by Android Studio that lets you inspect elements in the mobile app

NOTE: The UIAutomatorViewer can be used only for android native apps.

2. Appium Desktop Inspector: The latest version of Appium GUI (Appium Desktop) comes with an inspector. We can use this inspector for both Android and iOS apps (for iOS apps, you would need a Mac).

How to open UIAutomatorViewer:
UIAutomatorViewer comes with Android Studio installation package. So when we install Android Studio, UIAutomatorViewer would automatically get installed with it. There are 2 ways in which we can open UIAutomatorViewer. Follow any of these methods to open it.

Method 1: Open UIAutomatorViewer from Command Prompt This is the easiest way of opening UIAutomatorViewer. Just follow the steps given below to open it –

Step 1.1: Open command prompt

Step 1.2: Type uiautomatorviewer and then hit Enter

Wait for a couple of seconds, The UIAutomatorViewer  window will open

Method 2: Open UIAutomatorViewer from its folder location UIAutomatorViewer is installed on your machine as a .bat file. Using this method, you will navigate to the folder location where it is installed and then open it. Follow the steps given below to use this approach –

Step 2.1: Open the folder location where UIAutomatorViewer is installed. Generally, this location is – C:\Users\akumar\AppData\Local\Android\sdk\tools (Replace ‘akumar’ with the username that you have on your machine)

Step 2.2: Check that this bin folder contains uiautomatorviewer

Step 2.3: Double click on uiautomatorviewer batch file, the uiautomatorviewer window will open.

NOTE: Connect your mobile phone to your machine using USB (OR) Launch Android Emulator from Android studio.

How to launch Android Emulator:

1. Launch Android studio

2. Go to Tools >> Android >> AVD Manager

3. Create virtual device.

Connect your mobile phone to your machine using USB 

Before we start inspecting elements of the mobile app, first need to connect a mobile device to our machine via USB. 

Step 1: To make sure that the phone is connected properly, open command prompt and run adb devices command. 

Step 3: Now need to open the app which we want to inspect.

Eg: Care.com app

Inspect mobile app using UI Automator Viewer

Follow the steps given below to inspect mobile app with UI Automator Viewer –

Step 1: Make sure that the mobile phone is connected properly with USB or Launch android emulator, ensure that it is unlocked and Care app is opened

Step 2: Now click on Device Screenshot icon in UI Automator Viewer (this is the second icon as shown in the below image)

Step 3: will see a popup message which shows the progress information as shown below.

Step 4: Wait for a few seconds for UI Automator Viewer to process all the information on the screen. Once this is done, UI Automator Viewer will load the screenshot of the phone.

Step 5: Now, hover the mouse pointer anywhere on the device screenshot. will see that some information gets displayed on the right side panes

Important Note: Technically speaking, UI Automator Viewer just captures the mobile screen. So whatever is displayed on the device, UI Automator Viewer would capture that.

Main sections in UI Automator Viewer

From the below image, We can see that UI Automator Viewer has 3 main sections –

How to use these properties to identify elements for automation:

Well, we cannot use the properties directly, each property has other names. Let's see how to use those properties values to work. Following attributes can be used to identify the care app elements.

text: attribute can be used as "name"

Ex:

MobileElement ele1 = (MobileElement) driver.findElement(By.name("Sign in"));

resource-id: attribute can be used as "id"

class: attribute can be used as "className"

content-desc: attribute can be used as "AccessibilityId"

Examples:

1) If we have className as android.widget.ImageView only for that particular element...we can directly use it by By.className("android.widget.ImageView");

2) If we have multiple classNames... you can return them into a List, and then click based on index

List buttons = driver.findElements(By.className("android.widget.RelativeLayout"));

buttons.get(0).click(); or buttons.get(1).click();

Appium Desktop Inspector – Steps to get started

Inspecting elements in the mobile app using Appium Desktop inspector is a multi-step process. You would need to do the below-mentioned tasks to get rolling.

• Open Appium Desktop and start Appium server
• Provide Desired Capabilities for care app.
• Start New Session (i.e., Start Appium Desktop Inspector)
• Inspect elements on the mobile app

Since there are many steps involved here and to make this entire process easy to understand.

Open Appium Desktop app and start Appium Server

1. Open Appium from Start > All Programs
2. Once Appium Desktop opens, navigate to its Simple tab

3. Enter the following values in the fields as shown in the below image –

• Server Address – 0.0.0.0
• Server Port – 4723

4. Click on Start Server v1.7.2 button

5. Appium server would start and you would see the screen as shown below

Appium Desired Capabilities Screen

Till now, We have opened Appium Desktop and started the server. Let us now will look at the Desired Capabilities screen.

1. Click on Start Inspector Session button (the first button with search icon) from Appium logs screen

2. A new Appium window would be displayed. Have a look at each of the four tabs in this window. When we select any of these tabs, then only the top section of the window changes. The bottom section, starting at Desired Capabilities stays the same way.

As of now will just talk about Automatic Server and Sauce Labs and Test Object tabs

Automatic Server: Whenever we run scripts on local, choose this tab and then add Desired capabilities.

Sauce Labs and Test Object tabs: We can use these tabs when we run mobile tests on Sauce Labs infrastructure

Provide Desired Capability details for Automatic Server

 Let's see how we can provide the Desired Capability details to start Appium Desktop Inspector. With Appium Desktop Inspector, we will need to provide all the Desired Capabilities that we write in our test case to launch your app

1. To add the first capability, type ‘deviceName’ in Name field and ‘Android emulator’ in Value field. Your screen would now look like this (note that the JSON Representation block would show the capability that you have added just now)

2. Now click on the ‘+’ icon, so that a new blank row gets displayed

3. Keep on adding new rows till you add all the capabilities to launch app in the appium desktop server.

Eg: Android care app:

{
  "appiumVersion": "1.7.2",
  "platformName": "Android",
  "deviceName": "Android Emulator",
  "platformVersion": "7.1",
  "app": "E:\\selenium\\qa-vtest\\src\\test\\resources\\platform\\android\\CareDebug.apk",
  "deviceOrientation": "portrait",
  "appPackage": "com.care.android.careview.debug",
  "appActivity": "com.care.android.careview.ui.startup.SplashActivity"
}

Eg: IOS care app:

{
 “appiumVersion”: “1.6.4",
 “platformName”: “iOS”,
 “deviceName”: “iPhone 7 Plus Simulator”,
 “platformVersion”: “10.3",
 “app”: “/Users/amit.kumar/care/vtest/qa-vtest/src/test/resources/platform/iOS/CareDebug.app”,
 “deviceOrientation”: “portrait”,
 “automationName”: “XCUITest”,
 “platform”: “MAC”,
 “udid”: “FAF13F24-54AD-433B-BADA-9EF62FB0AEE6"
}

Save the Desired Capabilities Set


It’s a good approach to save the Desired Capabilities set especially when we are using multiple devices. This is due to the fact that once you save a Desired Capability set, then we can directly select it from the Saved Capabilities Set section and start the inspector. So if we use multiple devices or android versions, we can save these capabilities as say ‘Samsung Galaxy S7’, ‘Nexus 6P’, ‘Android 7.0 Device’ etc.

Let’s quickly see the steps that you have to follow to save the Desired Capabilities:


From the Automatic Server tab where you entered all the Desired Capabilities, click on ‘Save As…’ button
You would see a popup asking for the name of the Capability set






3. Enter any name and click on Save button. The capability set would get saved
4. Click on Saved Capabilities Sets tab. 








Start Appium Desktop Inspector

First of all make sure that mobile device is connected to your machine. Also see that its unlocked. To start Appium Desktop Inspector, click on Start Session button. we can do this either from the Desired Capabilities tab, or from the Saved Capability Sets tab.

Once you click on the button, we would see that the screen would start getting refreshed. Then after few moments, Appium window would display the screenshot of Care app together with its Object Hierarchy Dump. The Appium window screen would like this




Note: Whenever we want inspect elements in a particular screen then make sure that the app is open and click on "Refresh screen and screenshot" on appium desktop server.