0x00 前言
UiAutomator是Android 4.1以上提供的一个UI自动化测试工具，4.3升级到了UiAutomator2.0，实现方式也从UiTestAutomationBridge变成了UiAutomation。
0x01 UiAutomation实现分析
UiAutomation类位于android.app包下面，是API18新增的类。
public final class UiAutomation {
private final IAccessibilityServiceClient mClient;
private final IUiAutomationConnection mUiAutomationConnection;
public void connect();
public void disconnect();
public final boolean performGlobalAction(int action);
public final AccessibilityServiceInfo getServiceInfo();
public AccessibilityNodeInfo getRootInActiveWindow();
public boolean injectInputEvent(InputEvent event, boolean sync);
public boolean setRotation(int rotation);
public AccessibilityEvent executeAndWaitForEvent(Runnable command, AccessibilityEventFilter filter, long timeoutMillis);
public void waitForIdle(long idleTimeoutMillis, long globalTimeoutMillis);
public Bitmap takeScreenshot();
public void setRunAsMonkey(boolean enable);
｝
以上是UiAutomation类中最重要的成员变量和函数，所有的接口基本都是通过调用mClient和mUiAutomationConnection这两个对象实现的。
public UiAutomation(Looper looper, IUiAutomationConnection connection) {
if (looper == null) {
throw new IllegalArgumentException("Looper cannot be null!");
}
if (connection == null) {
throw new IllegalArgumentException("Connection cannot be null!");
}
mUiAutomationConnection = connection;
mClient = new IAccessibilityServiceClientImpl(looper);
}
interface IUiAutomationConnection {
void connect(IAccessibilityServiceClient client);
void disconnect();
boolean injectInputEvent(in InputEvent event, boolean sync);
boolean setRotation(int rotation);
Bitmap takeScreenshot(int width, int height);
boolean clearWindowContentFrameStats(int windowId);
WindowContentFrameStats getWindowContentFrameStats(int windowId);
void clearWindowAnimationFrameStats();
WindowAnimationFrameStats getWindowAnimationFrameStats();
void executeShellCommand(String command, in ParcelFileDescriptor fd);
void grantRuntimePermission(String packageName, String permission, int userId);
void revokeRuntimePermission(String packageName, String permission, int userId);
// Called from the system process.
oneway void shutdown();
}
UiAutomation构造函数需要传入两个参数，一个是线程Looper对象，用于发送消息，一个是IUiAutomationConnection接口实例。
public void connect() {
if (mHandlerThread.isAlive()) {
throw new IllegalStateException("Already connected!");
}
mHandlerThread.start();
mUiAutomation = new UiAutomation(mHandlerThread.getLooper(),
new UiAutomationConnection());
mUiAutomation.connect();
}
这是UiAutomationShellWrapper类中的一个方法，正好可以看到UiAutomation如何初始化。构造函数的第二个参数实际使用的是UiAutomationConnection类实例，这个类也是在android.app包下面，正是继承自IUiAutomationConnection.Stub类。
UiAutomation中一个重要的成员变量是mClient，它的类型是IAccessibilityServiceClient。
oneway interface IAccessibilityServiceClient {
void init(in IAccessibilityServiceConnection connection, int connectionId, IBinder windowToken);
void onAccessibilityEvent(in AccessibilityEvent event);
void onInterrupt();
void onGesture(int gesture);
void clearAccessibilityCache();
void onKeyEvent(in KeyEvent event, int sequence);
}
IAccessibilityServiceClient是一个AIDL接口定义，在抽象类AccessibilityService中实现了一个该接口的实现类IAccessibilityServiceClientWrapper。UiAutomation中定义了一个IAccessibilityServiceClientWrapper的子类IAccessibilityServiceClientImpl，主要是重写了onAccessibilityEvent方法，用于获取AccessibilityEvent事件。UiAutomation的构造函数中实例化的正是IAccessibilityServiceClientImpl实例。
UiAutomation的初始化过程主要是在connect方法中。
/**
* Connects this UiAutomation to the accessibility introspection APIs.
*
* @hide
*/
public void connect() {
synchronized (mLock) {
throwIfConnectedLocked();
if (mIsConnecting) {
return;
}
mIsConnecting = true;
}
try {
// Calling out without a lock held.
mUiAutomationConnection.connect(mClient);
} catch (RemoteException re) {
throw new RuntimeException("Error while connecting UiAutomation", re);
}
synchronized (mLock) {
final long startTimeMillis = SystemClock.uptimeMillis();
try {
while (true) {
if (isConnectedLocked()) {
break;
}
final long elapsedTimeMillis = SystemClock.uptimeMillis() - startTimeMillis;
final long remainingTimeMillis = CONNECT_TIMEOUT_MILLIS - elapsedTimeMillis;
if (remainingTimeMillis <= 0) {
throw new RuntimeException("Error while connecting UiAutomation");
}
try {
mLock.wait(remainingTimeMillis);
} catch (InterruptedException ie) {
/* ignore */
}
}
} finally {
mIsConnecting = false;
}
}
}
这里主要是调用了mUiAutomationConnection.connect(mClient)。
public void connect(IAccessibilityServiceClient client) {
if (client == null) {
throw new IllegalArgumentException("Client cannot be null!");
}
synchronized (mLock) {
throwIfShutdownLocked();
if (isConnectedLocked()) {
throw new IllegalStateException("Already connected.");
}
mOwningUid = Binder.getCallingUid();
registerUiTestAutomationServiceLocked(client);
storeRotationStateLocked();
}
}
private void registerUiTestAutomationServiceLocked(IAccessibilityServiceClient client) {
IAccessibilityManager manager = IAccessibilityManager.Stub.asInterface(
ServiceManager.getService(Context.ACCESSIBILITY_SERVICE));
AccessibilityServiceInfo info = new AccessibilityServiceInfo();
info.eventTypes = AccessibilityEvent.TYPES_ALL_MASK;
info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC;
info.flags |= AccessibilityServiceInfo.FLAG_INCLUDE_NOT_IMPORTANT_VIEWS
| AccessibilityServiceInfo.FLAG_REPORT_VIEW_IDS;
info.setCapabilities(AccessibilityServiceInfo.CAPABILITY_CAN_RETRIEVE_WINDOW_CONTENT
| AccessibilityServiceInfo.CAPABILITY_CAN_REQUEST_TOUCH_EXPLORATION
| AccessibilityServiceInfo.CAPABILITY_CAN_REQUEST_ENHANCED_WEB_ACCESSIBILITY
| AccessibilityServiceInfo.CAPABILITY_CAN_REQUEST_FILTER_KEY_EVENTS);
try {
// Calling out with a lock held is fine since if the system
// process is gone the client calling in will be killed.
manager.registerUiTestAutomationService(mToken, client, info);
mClient = client;
} catch (RemoteException re) {
throw new IllegalStateException("Error while registering UiTestAutomationService.", re);
}
}
UiAutomationConnection.connect函数接着调用了registerUiTestAutomationServiceLocked(client)，而registerUiTestAutomationServiceLocked主要是调用了IAccessibilityManager的registerUiTestAutomationService注册了一个Accessibility服务。
由此可见，UiAutomation最终也是使用了AccessibilityManagerService。
0x02 如何使用UiAutomation
UiAutomator的常见使用方式是调用uiautomator命令，或是将uiautomator.jar导入到自己的工程中。为了更加自由地使用UiAutomation提供的能力，可以考虑直接创建UiAutomation对象实例使用。由于UiAutomation的构造函数以及其它一些重要方法设置了@hide，因此无法直接使用，需要用反射的方式获取。
Object connection = null;
HandlerThread mHandlerThread = new HandlerThread("UiAutomationThread");
mHandlerThread.start();
try{
Class<?> UiAutomationConnection = Class.forName("android.app.UiAutomationConnection");
Constructor<?> newInstance = UiAutomationConnection.getDeclaredConstructor();
newInstance.setAccessible(true);
connection = newInstance.newInstance();
Class<?> IUiAutomationConnection = Class.forName("android.app.IUiAutomationConnection");
Constructor<?> newUiAutomation = UiAutomation.class.getDeclaredConstructor(Looper.class, IUiAutomationConnection);
UiAutomation mUiAutomation = (UiAutomation)newUiAutomation.newInstance(mHandlerThread.getLooper(), connection);
Method connect = UiAutomation.class.getDeclaredMethod("connect");
connect.invoke(mUiAutomation);
Log.i(TAG, ""+mUiAutomation);
mUiAutomation.waitForIdle(1000, 1000 * 10);
AccessibilityNodeInfo nodeInfo = mUiAutomation.getRootInActiveWindow();
Log.i(TAG, ""+nodeInfo);
}catch(Exception e){
e.printStackTrace();
return;
}
这是一段最简单的使用代码，编译到apk里面，运行后报错了。
Caused by: java.lang.SecurityException: You do not have android.permission.RETRIEVE\_WINDOW_CONTENT required to call registerUiTestAutomationService from pid=3676, uid=10040
at android.os.Parcel.readException(Parcel.java:1599)
at android.os.Parcel.readException(Parcel.java:1552)
at android.view.accessibility.IAccessibilityManager$Stub$Proxy.registerUiTestAutomationService (IAccessibilityManager.java:352)
at android.app.UiAutomationConnection.registerUiTestAutomationServiceLocked(UiAutomationConnection.java:337)
at android.app.UiAutomationConnection.connect(UiAutomationConnection.java:89)
at android.app.UiAutomation.connect(UiAutomation.java:197)
看来是需要android.permission.RETRIEVE_WINDOW_CONTENT权限，尝试将该权限添加到AndroidManifest.xml中，但是提示这个是系统权限，普通应用不能申请。因此，应用中不能使用UiAutomation的，同时发现，编译为jar包后使用shell权限执行，是可以正常跑过的。这也是为什么uiautomator工具可以正常运行的原因。
0x03 AccessibilityManagerService
如前所述，UiAutomation底层使用的还是AccessibilityManagerService，这里简单分析一下AccessibilityManagerService的实现。
该类内部包含了一个Service类：
/**
* This class represents an accessibility service. It stores all per service
* data required for the service management, provides API for starting/stopping the
* service and is responsible for adding/removing the service in the data structures
* for service management. The class also exposes configuration interface that is
* passed to the service it represents as soon it is bound. It also serves as the
* connection for the service.
*/
class Service extends IAccessibilityServiceConnection.Stub implements ServiceConnection, DeathRecipient;
从描述上看，每个Service实例代表了一个Accessibility服务，同时它实现了ServiceConnection接口，因此它也是一个Service客户端。
Service类中定义了一个bool类型的变量mIsAutomation，表示当前服务是否是UiAutomation。在AccessibilityManagerService的registerUiTestAutomationService方法中，将当前服务的组件名称设置为sFakeAccessibilityServiceComponentName，而mIsAutomation就是通过组件名称来判断是否是UiAutomation。
accessibilityServiceInfo.setComponentName(sFakeAccessibilityServiceComponentName);
mIsAutomation = (sFakeAccessibilityServiceComponentName.equals(componentName));
由于UiAutomation与AccessibilityService实现方式不同，因此，AccessibilityManagerService在很多地方都会根据mIsAutomation执行不同的逻辑。
根据前面的分析，在UiAutomation发起connect请求后，会进入AccessibilityManagerService的registerUiTestAutomationService。然后经过一系列的函数调用，进到Service的bindLocked函数。
/**
* Binds to the accessibility service.
*
* @return True if binding is successful.
*/
public boolean bindLocked() {
UserState userState = getUserStateLocked(mUserId);
if (!mIsAutomation) {
if (mService == null && mContext.bindServiceAsUser(
mIntent, this,
Context.BIND_AUTO_CREATE | Context.BIND_FOREGROUND_SERVICE_WHILE_AWAKE,
new UserHandle(mUserId))) {
userState.mBindingServices.add(mComponentName);
}
} else {
userState.mBindingServices.add(mComponentName);
mService = userState.mUiAutomationServiceClient.asBinder();
mMainHandler.post(new Runnable() {
@Override
public void run() {
// Simulate asynchronous connection since in onServiceConnected
// we may modify the state data in case of an error but bind is
// called while iterating over the data and bad things can happen.
onServiceConnected(mComponentName, mService);
}
});
userState.mUiAutomationService = this;
}
return false;
}
bindLocked会发送一个异步消息，调到onServiceConnected回调。
@Override
public void onServiceConnected(ComponentName componentName, IBinder service) {
synchronized (mLock) {
mService = service;
mServiceInterface = IAccessibilityServiceClient.Stub.asInterface(service);
UserState userState = getUserStateLocked(mUserId);
addServiceLocked(this, userState);
if (userState.mBindingServices.contains(mComponentName) || mWasConnectedAndDied) {
userState.mBindingServices.remove(mComponentName);
mWasConnectedAndDied = false;
try {
mServiceInterface.init(this, mId, mOverlayWindowToken);
onUserStateChangedLocked(userState);
} catch (RemoteException re) {
Slog.w(LOG_TAG, "Error while setting connection for service: "
+ service, re);
binderDied();
}
} else {
binderDied();
}
}
}
这里的mServiceInterface其实就是UiAutomation构造函数中实例化的mClient的远程对象。接着，调用了IAccessibilityServiceClient的init函数，该函数的实现位于AccessibilityService.IAccessibilityServiceClientWrapper类中。
public void init(IAccessibilityServiceConnection connection, int connectionId,
IBinder windowToken) {
Message message = mCaller.obtainMessageIOO(DO_INIT, connectionId,
connection, windowToken);
mCaller.sendMessage(message);
}
case DO_INIT: {
mConnectionId = message.arg1;
SomeArgs args = (SomeArgs) message.obj;
IAccessibilityServiceConnection connection =
(IAccessibilityServiceConnection) args.arg1;
IBinder windowToken = (IBinder) args.arg2;
args.recycle();
if (connection != null) {
AccessibilityInteractionClient.getInstance().addConnection(mConnectionId,
connection);
mCallback.init(mConnectionId, windowToken);
mCallback.onServiceConnected();
} else {
AccessibilityInteractionClient.getInstance().removeConnection(
mConnectionId);
mConnectionId = AccessibilityInteractionClient.NO_ID;
AccessibilityInteractionClient.getInstance().clearCache();
mCallback.init(AccessibilityInteractionClient.NO_ID, null);
}
} return;
这样，初始化完成后，AccessibilityInteractionClient实例中保存了IAccessibilityServiceConnection实例，而AccessibilityManagerService中对应的Service对象中也保存了IAccessibilityServiceClient实例，从而建立起双向的Binder通信。
在发生Accessibility事件后，AccessibilityManagerService会通过IAccessibilityServiceConnection的onAccessibilityEvent方法将事件通知给UiAutomation。
0x04 UiAutomator与AccessibilityService
AccessibilityService是一个继承自Service的抽象服务类，用户在使用时需要实现一个自己的子类。该类很大程度上依赖于AccessibilityInteractionClient类提供的接口，AccessibilityInteractionClient类内部保存了一个静态的LongSparseArray<AccessibilityInteractionClient> sClients对象，用来实现线程相关的单例对象。
/**
* @return The client for the current thread.
*/
public static AccessibilityInteractionClient getInstance() {
final long threadId = Thread.currentThread().getId();
return getInstanceForThread(threadId);
}
/**
* <strong>Note:</strong> We keep one instance per interrogating thread since
* the instance contains state which can lead to undesired thread interleavings.
* We do not have a thread local variable since other threads should be able to
* look up the correct client knowing a thread id. See ViewRootImpl for details.
*
* @return The client for a given <code>threadId</code>.
*/
public static AccessibilityInteractionClient getInstanceForThread(long threadId) {
synchronized (sStaticLock) {
AccessibilityInteractionClient client = sClients.get(threadId);
if (client == null) {
client = new AccessibilityInteractionClient();
sClients.put(threadId, client);
}
return client;
}
}
前面提到，AccessibilityInteractionClient对象中保存了IAccessibilityServiceConnection实例，因此，可以调用该接口提供的功能。
/**
* Interface given to an AccessibilitySerivce to talk to the AccessibilityManagerService.
*
* @hide
*/
interface IAccessibilityServiceConnection {
void setServiceInfo(in AccessibilityServiceInfo info);
boolean findAccessibilityNodeInfoByAccessibilityId(int accessibilityWindowId,
long accessibilityNodeId, int interactionId,
IAccessibilityInteractionConnectionCallback callback, int flags, long threadId);
boolean findAccessibilityNodeInfosByText(int accessibilityWindowId, long accessibilityNodeId,
String text, int interactionId, IAccessibilityInteractionConnectionCallback callback,
long threadId);
boolean findAccessibilityNodeInfosByViewId(int accessibilityWindowId,
long accessibilityNodeId, String viewId, int interactionId,
IAccessibilityInteractionConnectionCallback callback, long threadId);
boolean findFocus(int accessibilityWindowId, long accessibilityNodeId, int focusType,
int interactionId, IAccessibilityInteractionConnectionCallback callback, long threadId);
boolean focusSearch(int accessibilityWindowId, long accessibilityNodeId, int direction,
int interactionId, IAccessibilityInteractionConnectionCallback callback, long threadId);
boolean performAccessibilityAction(int accessibilityWindowId, long accessibilityNodeId,
int action, in Bundle arguments, int interactionId,
IAccessibilityInteractionConnectionCallback callback, long threadId);
AccessibilityWindowInfo getWindow(int windowId);
List<AccessibilityWindowInfo> getWindows();
AccessibilityServiceInfo getServiceInfo();
boolean performGlobalAction(int action);
oneway void setOnKeyEventResult(boolean handled, int sequence);
}
与UiAutomation不同的是，AccessibilityService的初始化是和普通的Service一致的。由于AccessibilityService比较特殊的地方在于需要在设置的辅助功能里开启对应的服务，点击开启后，会执行到BindService逻辑，进而执行到AccessibilityService的onBind回调，并触发AccessibilityManagerService中Service的onServiceConnected回调。
/**
* Implement to return the implementation of the internal accessibility
* service interface.
*/
@Override
public final IBinder onBind(Intent intent) {
return new IAccessibilityServiceClientWrapper(this, getMainLooper(), new Callbacks() {
@Override
public void onServiceConnected() {
AccessibilityService.this.onServiceConnected();
}
@Override
public void onInterrupt() {
AccessibilityService.this.onInterrupt();
}
@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
AccessibilityService.this.onAccessibilityEvent(event);
}
@Override
public void init(int connectionId, IBinder windowToken) {
mConnectionId = connectionId;
mWindowToken = windowToken;
// The client may have already obtained the window manager, so
// update the default token on whatever manager we gave them.
final WindowManagerImpl wm = (WindowManagerImpl) getSystemService(WINDOW_SERVICE);
wm.setDefaultToken(windowToken);
}
@Override
public boolean onGesture(int gestureId) {
return AccessibilityService.this.onGesture(gestureId);
}
@Override
public boolean onKeyEvent(KeyEvent event) {
return AccessibilityService.this.onKeyEvent(event);
}
});
}
相对于UiAutomation只能在shell环境中执行，AccessibilityService是可以运行在app环境中的，但是需要用户手动开启服务会略显麻烦。
0x05 总结
UiAutomator和AccessibilityService作为两种不同的实现形式，拥有各自的优缺点，这两年流行的抢红包工具基本也是基于这两种方式实现的。在自动化中使用它们也能起到一些辅助作用。