Posts tagged ‘reverse engineering’
Thanks to everyone who came out to catch David Teitelbaum’s class at Droidcon UK, “Who Needs Thumbs? Android Instrumentation and Reverse Engineering.” We had a fantastic time mixing it up with all the Android Devs in London and appreciate all the great feedback on Twitter! For all of you who’ve asked, you can find the slides here.
Are you thinking to yourself, “I couldn’t attend DroidCon UK and I’ll never forgive myself because this class sounds SO interesting. I wish there was somewhere else I could go to learn how to reverse engineer my favorite apps!?” Well you’re in luck! Strap on your clogs because we’re headed back across the Atlantic to bring this amazing class to DroidCon NL! Here are the details:
WHO? David Teitelbaum presenting at DroidCon NL
WHAT? Who Needs Thumbs? Android Instrumentation and Reverse Engineering
WHEN? Friday, November 23 from 11:30 am – 12:15 pm
WHERE? Science Park Amsterdam
Don’t miss your chance to learn about the fundamentals of APK reverse engineering, including Dalvik executable disassembly/reassembly, code injection, and view hacking using tools such as Smali/Baksmali. We hope to see you there!
When it comes to learning Android, it’s amazing how easy it is to find tutorials, code samples, and documentation to immerse yourself into. Interestingly, I’ve found the inverse to be true for the, dare I say, way cooler world of hacking Android. Reverse engineering Android applications can be really fun and give you a decent knowledge for the inner workings of the Dalvik Virtual Machine. This post will be an all-out, start-to-finish, beginners* tutorial on the tools and practices of reverse engineering Android through the disassembly and code injection of the Android Hello World application.
*Beginner means that you know a bit about Android and Java in general, if not, learn a bit first and come back. Experience in the terminal environment on your machine is also probably necessary.
The Apk
In order to start reverse engineering, you must first understand what you’re working with. So what exactly is an apk? (hint: not American Parkour.) An Android package, or apk, is the container for an Android app’s resources and executables. It’s a zipped file that contains simply:
- AndroidManifest.xml (serialized, non human readable)
- classes.dex
- res/
- lib/ (sometimes)
- META-INF/
The meat of the application is the classes.dex file, or the Dalvik executable (get it, dex) that runs on the device. The application’s resources (i.e. images, sound files) reside in the res directory, and the AndroidManifest.xml is more or less the link between the two, providing some additional information about the application to the OS. The lib directory contains native libraries that the application may use via NDK, and the META-INF directory contains information regarding the application’s signature.
You can grab the HelloWorld apk we will be hacking here. The source to this apk is available from the developer docs tutorial, and when compiled looks something like this:
Flashy, huh
The Tools
In order to complete this tutorial, you’ll need to download and install the following tools:
Disassembling the Apk
Once you’ve installed apktool, go ahead and open up your terminal and change directory into where you’ve placed the downloaded apk.
$ cd ~/Desktop/HelloWorld
Execution of the apktool binary without arguments will give you its usage, but we will only use the ‘d’ (dump) and ‘b’ (build) commandline options for this tutorial. Dump the apk using the apktool ‘d’ option:
$ apktool d HelloWorld.apk
This will tell the tool to decode the assets and disassemble the .dex file in the apk. When finished, you will see the ./HelloWorld directory, containing:
- AndroidManifest.xml (decoded, human readable)
- res/ (decoded)
- smali/
- apktool.yml
The AndroidManifest.xml is now readable, the resources have been decoded, and a smali directory has been created (ignore the apktool.yml as it’s just a configuration for the tool itself). The smali directory is probably the most important of the three, as it contains a set of smali files, or bytecode representation of the application’s dex file. You can think of it as an intermediate file between the .java and the executable.
So let’s take a look at what’s in the smali directory , ‘ls’ yields:
$ ls HelloWorld/smali/com/test/helloworld/ HelloWorldActivity.smali R$attr.smali R$drawable.smali R$layout.smali R$string.smali R.smali
Immediately we notice that the smali directory contains subdirectories defining the application’s namespace (com.test.helloworld). Additionally, we can see an individual smali file for each java class. There’s one catch – any ‘$’ in the smali file’s name means it’s an inner class in Java. Here we see the bytecode representation of the following classes:
- HelloWorldActivity.java
- R.java
Where R.java contains inner classes attr, string, and so on. It’s evident that HelloWorldActivity is the activity that’s displayed when the app launches, so what exactly is R?
R.java is an automatically generated file at application build time that maps resources to an associated id. When a developer wants to use anything in the res folder, he/she must use the R class to appropriately reference that resource. Because of this, we’ll omit the R.java from our investigation, as it really only contains a bunch of constants that no one cares about.
Reading the Smali
Now that we’ve disassembled our apk, let’s take a look at the java and smali representations of our impressive HelloWorldActivity.
package com.test.helloworld;
import android.app.Activity;
import android.os.Bundle;
import android.widget.TextView;
public class HelloWorldActivity extends Activity {
/** Called when the activity is first created. */
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
TextView text = new TextView(this);
text.setText("Hello World, Android");
setContentView(text);
}
}
.class public Lcom/test/helloworld/HelloWorldActivity;
.super Landroid/app/Activity;
.source "HelloWorldActivity.java"
# direct methods
.method public constructor ()V
.locals 0
.prologue
.line 7
invoke-direct {p0}, Landroid/app/Activity;->()V
return-void
.end method
# virtual methods
.method public onCreate(Landroid/os/Bundle;)V
.locals 2
.parameter "savedInstanceState"
.prologue
.line 11
invoke-super {p0, p1}, Landroid/app/Activity;->onCreate(Landroid/os/Bundle;)V
.line 13
new-instance v0, Landroid/widget/TextView;
invoke-direct {v0, p0}, Landroid/widget/TextView;->(Landroid/content/Context;)V
.line 14
.local v0, text:Landroid/widget/TextView;
const-string v1, "Hello World, Android"
invoke-virtual {v0, v1}, Landroid/widget/TextView;->setText(Ljava/lang/CharSequence;)V
.line 15
invoke-virtual {p0, v0}, Lcom/test/helloworld/HelloWorldActivity;->setContentView(Landroid/view/View;)V
.line 17
return-void
.end method
It should be pretty evident which one of these files is written in java, nonetheless, the smali representation shouldn’t be too intimidating.
Let’s break down whats going on here in java first. In line 07, we define our HelloWorldActivity class that extends android.app.Activity, and within that class, override the onCreate() method. Inside the method, we create an instance of the TextView class and call the TextView.setText() method with our message. Finally, in line 15 we set the view by calling setContentView(), passing in the TextView instance.
In smali, we can see that we have a bit more going on. Let’s break it up into sections, we have:
- class declarations from lines 01-03
- a constructor method from lines 07-15
- a bigger onCreate() method from lines 19-43
Declarations and Constructor
The class declarations in smali are essentially the same in java, just in a different syntax. They give the virtual machine their class and superclass name via the .class and .super tags. Additionally, the compiler throws in the source file name for…shits and gigs? Nope, stack traces.
The constructor has seemingly appeared out of no where, but really was inserted by the compiler because we extended another class. You can see that in line 12 the virtual machine is to make a direct invokation of the super classes constructor – this follows the nature of subclasses, they must call their superclasses constructor.
Data Types
In the onCreate() method beginning on line 19, we can see that the smali method definition isn’t that far off from its java counterpart. The method’s parameter types are defined within the parenthesis (semicolon separated) with the return type discreetly placed on the end of the .method line. Object return types are easy to recognize, given they begin with an L and are in full namespace. Java primitives, however, are represented as capital chars and follow the format:
V void Z boolean B byte S short C char I int J long (64 bits) F float D double (64 bits)
So for our onCreate() definition in smali, we can expect a void return value.
Registers
Moving one line down, on line 20 we see the ‘.locals’ directive. This determines how many registers the Dalvik vm will use for this method _without_ including registers allocated to the parameters of the method. Additionally, the number of parameters for any virtual method will always be the number of input parameters + 1. This is due to an implicit reference to the current object that resides in parameter register 0 or p0 (in java this is called the “this” reference). The registers are essentially references, and can point to both primitive data types and java objects. Given 2 local registers, 1 parameter register, and 1 “this” reference, the onCreate() method uses an effective 4 registers.
For convenience, smali uses a ‘v’ and ‘p’ naming convention for local vs. parameter registers. Essentially, parameter (p) registers can be represented by local (v) registers and will always reside in the highest available registers. For this example, onCreate() has 2 local registers and 2 parameter registers, so the naming scheme will look something like this:
v0 - local 0 v1 - local 1 v2/p0 - local 2 or parameter 0 (this) v3/p1 - local 3 or parameter 1 (android/os/Bundle)
Note: You may see the .registers directive as oppose to the .locals directive. The only difference is that the .registers directive includes parameter registers (including “this”) into the count. Given the onCreate() example, .locals 2 == .registers 4
Opcodes
Dalvik opcodes are relatively straightforward, but there are a lot of them. For the sake of this post’s length, we’ll only go over the basic (yet important) opcodes found in our example HelloWorldActivity.smali. In the onCreate method in HelloWorldActivity the following opcodes are used:
- invoke-super vx, vy, … invokes the parent classes method in object vx, passing in parameter(s) vy, …
- new-instance vx creates a new object instance and places its reference in vx
- invoke-direct vx, vy, … invokes a method in object vx with parameters vy, … without the virtual method resolution
- const-string vx creates string constant and passes reference into vx
- invoke-virtual vx, vy, … invokes the virtual method in object vx, passing in parameters vy, …
- return-void returns void
Hacking the App
Now that we know what we’re looking at, lets inject some code and rebuild the app. The code we will inject is only one line in java and presents the user with the toast message “hacked!”.
Toast.makeText(getApplicationContext(), "Hacked!", Toast.LENGTH_SHORT).show();
How do we do this in smali? Easy, let’s just compile this into another application and disassemble. The end result is something like this:
.line 18
invoke-virtual {p0}, Lcom/test/helloworld/HelloWorldActivity;->getApplicationContext()Landroid/content/Context;
move-result-object v1
const-string v2, "Hacked!"
const/4 v3, 0x0
invoke-static {v1, v2, v3}, Landroid/widget/Toast;->makeText(Landroid/content/Context;Ljava/lang/CharSequence;I)Landroid/widget/Toast;
move-result-object v1
invoke-virtual {v1}, Landroid/widget/Toast;->show()V
Now, let’s ensure we have the right amount of registers in our original onCreate() to support these method calls. We can see that the highest register in the code we want to patch is v3, which we have but will require us to overwrite both of our parameter registers. Given we won’t be using either of those registers after setContentView(), this number is appropriate. Our final patched HelloWorldActivity.smali should look like:
.class public Lcom/test/helloworld/HelloWorldActivity;
.super Landroid/app/Activity;
.source "HelloWorldActivity.java"
# direct methods
.method public constructor ()V
.locals 0
.prologue
.line 8
invoke-direct {p0}, Landroid/app/Activity;->()V
return-void
.end method
# virtual methods
.method public onCreate(Landroid/os/Bundle;)V
.locals 2
.parameter "savedInstanceState"
.prologue
.line 12
invoke-super {p0, p1}, Landroid/app/Activity;->onCreate(Landroid/os/Bundle;)V
.line 14
new-instance v0, Landroid/widget/TextView;
invoke-direct {v0, p0}, Landroid/widget/TextView;->(Landroid/content/Context;)V
.line 15
.local v0, text:Landroid/widget/TextView;
const-string v1, "Hello World, Android"
invoke-virtual {v0, v1}, Landroid/widget/TextView;->setText(Ljava/lang/CharSequence;)V
.line 16
invoke-virtual {p0, v0}, Lcom/test/helloworld/HelloWorldActivity;->setContentView(Landroid/view/View;)V
# Patches Start
invoke-virtual {p0}, Lcom/test/helloworld/HelloWorldActivity;->getApplicationContext()Landroid/content/Context;
move-result-object v1
const-string v2, "Hacked!"
const/4 v3, 0x0
invoke-static {v1, v2, v3}, Landroid/widget/Toast;->makeText(Landroid/content/Context;Ljava/lang/CharSequence;I)Landroid/widget/Toast;
move-result-object v1
invoke-virtual {v1}, Landroid/widget/Toast;->show()V
# Patches End
return-void
.end method
Lines 40+ contain the injected code.
Rebuilding the Apk
Now all that’s left is to rebuild the app!
$ apktool b ./HelloWorld
This will instruct apktool to rebuild the app, however, this rebuilt app will not be signed. We will need to sign the app before it can be successfully installed on any device or emulator.
Signing the Apk
In order to sign the apk, you’ll need jarsigner and keytool (or a platform specific alternative, like signapk for windows). With jarsigner and keytool, however, the steps are pretty easy. First create the key:
$ keytool -genkey -v -keystore my-release-key.keystore -alias alias_name -keyalg RSA -validity 10000
Then use jarsigner to sign your apk, referencing that key:
$ jarsigner -verbose -keystore my-release-key.keystore ./HelloWorld/dist/HelloWorld.apk alias_name
Then you’re done! Install the app onto your device or emulator and impress the shit out of yourself!
damn…impressive
That’s it for this tutorial, but stay tuned. There will definitely be more in the future. Feel free to leave any questions in the comment section, or contact me with any questions.
Happy hacking,
David Teitelbaum
We’re taking this show on the road, folks. Our VP of Engineering, David Teitelbaum, is presenting a class at DroidCon London! That’s right, ladies and gents. We’re bringing our Android hacking love across the pond to show all the devs in the UK how to hack their favorite Android apps straight off Google Play. The class will teach developers the fundamentals of APK reverse engineering, including Dalvik executable disassembly/reassembly, code injection, and view hacking using tools such as Smali/Baksmali and Romain Guy’s awesome ViewServer.
Here are the deets:
Who?
David Teitelbaum presenting at Droidcon London
What?
Who Needs Thumbs? Android Instrumentation and Reverse Engineering
When?
Friday, October 26, 14:00 – 14:45
Where?
Business Design Centre
52 Upper Street
Islington
London
N1 0QH
So what do you say, UK? Want to meet up with us and find out what Apkudo is all about? Put the lager on ice (we’re talking to you, @dnlkbox) and give us a shout at SayHi@apkudo.com. Hey – we might even bring you a rad “Hacker” t-shirt!
Come on…you know you want one.
The HierarchyViewer is an Android SDK tool that gives developers the ability to introspect on all aspects of an application’s layout at runtime. The tool can be extremely useful for developers when debugging the view state of an application across a realm of devices. Unfortunately, the Android SDK has limited this tool to “devices running a developer version of the Android system”. I personally had no idea what Android meant by this statement, so we spent some time investigating the implementation of the ViewServer in Android OS and fortunately found that the HierarchyViewer can be enabled with any Android device as long as it has root access.
Android HierarchyViewer in Tree Mode on the Galaxy SII
So how does it actually work? The HierarchyViewer utilizes a service running on the device called ViewServer located in
frameworks/base/services/java/com/android/server/ViewServer.java
When launched, ViewServer opens up a socket on local port 4939 and receives commands from a client (usually HierarchyViewer) to dump the current view state of the device. The ViewServer dispatches these calls via binder to the ViewRoot class, which serializes the view state and transmits it to the client over the socket. The WindowManagerService manages the ViewServer and provides a hook (via binder service call) to spawn the service.
A client can only launch the ViewServer if the device’s properties ro.debuggable=1 and ro.secure=0, and the client contains manifest.dump permission (in the usual case, HierarchyView, as the client, has these permissions via adb). Although Google Engineer Romain Guy has been very adamant that the ViewServer is not enabled for production devices, we’ve found a pretty easy way to bypass its restrictions if you have root. The portion of the code dictating the restrictions can be found in line 5501 of WindowManagerService.java and is shown below.
public boolean startViewServer(int port) {
if (isSystemSecure()) {
return false;
}
if (!checkCallingPermission(Manifest.permission.DUMP, "startViewServer")) {
return false;
}
private boolean isSystemSecure() {
return "1".equals(SystemProperties.get(SYSTEM_SECURE, "1")) &&
"0".equals(SystemProperties.get(SYSTEM_DEBUGGABLE, "0"));
}
One can go about bypassing these restrictions in two ways: patch the WindowManagerService to allow anyone to spawn the server _or_ modify the ro.debuggable and ro.secure properties in the default.prop file. Because of the potential negative ramifications of modifying the ro.debuggable and ro.secure properties in Android (not to mention the fact that the default.prop file is part of the ramdisk, which makes editing it a bitch), we decided to patch the WindowManagerService. The patch is minimally invasive and provides a good intro into the power of patching the Android framework.
In order to perform this patch you will need the smali/baksmali tools, dextopt-wrapper, and access to a rooted device with BusyBox installed. If you have no idea what these tools are or how they work, you may want to do some reading first. We’ve only tested this on the HTC One S and a 4.0.3 Galaxy S II, so if you brick your device we’ll feel bad but can’t really do anything about it (hey, maybe we’ll send you a t-shirt).
Step 1 – Backup the phone’s /system/framework/ on your machine
$ mkdir ./system $ mkdir ./system/framework $ adb pull /system/framework/ ./system/framework/
Important: Make sure the directory structure matches the device’s for step 3.
Step 2 – Grab the bootclasspath from the device
$ adb shell # echo $BOOTCLASSPATH
Copy this path.
Step 3 – Disassemble (baksmali) the services odex
$ baksmali -x -a 14 -c <copied bootclasspath> ./system/framework/services.odex
-x = odex
-a = api level 14
-c = classes (loaded from the bootclasspath, separated by colon)
If you’ve done this correctly you will now see a directory called ‘out’, otherwise verify you’ve pulled the jars and bootclasspath correctly.
Step 4 – In your favorite text editor, open up WindowManagerService.smali
$ emacs out/com/android/server/wm/WindowManagerService.smali
Search for isSystemSecure(). This is the method they use (only for ViewServer) to verify the system is not secure. We happened to find this on line 4815. Note: In 4.0 they moved WindowManagerService into the wm directory, it may just be in out/com/android/server/
Step 5 – Patch the return value of isSystemSecure() to always return false
.method private isSystemSecure()Z
.registers 4
.prologue
.line 5965
const-string v0, "1"
const-string v1, "ro.secure"
const-string v2, "1"
invoke-static {v1, v2}, Landroid/os/SystemProperties;->get(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;
move-result-object v1
invoke-virtual {v0, v1}, Ljava/lang/String;->equals(Ljava/lang/Object;)Z
move-result v0
if-eqz v0, :cond_22
const-string v0, "0"
const-string v1, "ro.debuggable"
const-string v2, "0"
invoke-static {v1, v2}, Landroid/os/SystemProperties;->get(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;
move-result-object v1
invoke-virtual {v0, v1}, Ljava/lang/String;->equals(Ljava/lang/Object;)Z
move-result v0
if-eqz v0, :cond_22
const/4 v0, 0x1
:goto_21
return v0
:cond_22
const/4 v0, 0x0
goto :goto_21
.end method
We can see from above that the v0 register contains the return value of this method. Let’s add a simple
const/4 v0, 0x0
in the line 41 above return v0 to ensure this method will always return false, giving the WMS the go ahead to spawn view server. Easy enough, right?
Step 6 – Reassemble into Dex
$ smali ./out -o classes.dex
-o = output file (classes.dex is the default name for apks in Android)
Step 7 – Zip dex
$ zip services_hacked.jar ./classes.dex
Step 8 – Remount /system
$ adb remount
OR
$ adb shell # mount -o remount,rw -t yaffs2 /dev/block/mtdblock3 /system
Step 9 – Push dexopt-wrapper and services_hacked.jar onto /data/local/tmp
$ adb push ./services_hacked.jar /data/local/tmp $ adb push ./dexopt-wrapper /data/local/tmp
Step 10 – Optimize the services_hacked.jar into services_hacked.odex
$ adb shell # cd /data/local/tmp # chmod 777 ./dexopt-wrapper
Important: We need to feed in the bootclasspath into dexopt, but because we have modified a member of that class path, we need to make sure we don’t include it. To do this, copy bootclasspath (via method in step 2) and omit the “:/system/framework/services.jar”. Forgetting this step will likely result in a boot loop.
# ./dexopt-wrapper ./services_hacked.jar ./services_hacked.odex <copied bootclasspath excluding ":/system/framework/services.jar">
Our final command looked something like
./dexopt-wrapper ./services_hacked.jar ./services_hacked.odex /system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/framework2.jar:/system/framework/android.policy.jar:/system/framework/apache-xml.jar:/system/framework/filterfw.jar:/system/framework/sechardware.jar:/system/framework/wimax.jar
A great way to check that this command has worked is to see that the /system/framework/services.odex and the services_hacked.odex are nearly identical in size. The two files shouldn’t differ by more than 1KB.
Step 11 – Copy in the signature
Each odex is signed with a 20 byte signature located at a 52 byte offset within the odex. Let’s trick the OS into thinking our hacked odex matches the system signature by copying it from the original
# busybox dd if=/system/framework/services.odex of=/data/local/tmp/services_hacked.odex bs=1 count=20 skip=52 seek=52 conv=notrunc
if = input file
of = output file
bs = block size (1 byte)
count = number of blocks
skip = input file offset
seek = output file offset
conv=notrunc – don’t truncate the output file.
Step 12 – Replace the original odex with the hacked version
# dd if=/data/local/tmp/services_hacked.odex of=/system/framework/services.odex
Step 13 – Let the device reset
The previous step should result in a software reset, and it may be worthwhile to logcat as this loads. If you see the Dalvik logs stating “some deps went away” or “not all deps represented” then you’ve definitely screwed up step 10. These errors are a result of passing in the wrong bootclasspath to dexopt-wrapper. If this causes a bootloop, see step 16.
Step 14 – Launch ViewServer
WindowManagerService provides a (heavily undocumented) hook into spawning ViewServer. To start the server on port 4939,
$ adb shell service call window 1 i32 4939
To stop the server
$ adb shell service call window 2
To check if the server is running
$ adb shell service call window 3
If ViewServer is running, you should see
Result: Parcel(00000000 00000001 '........')
Step 15 – Launch HierarchyViewer
$ ./sdk/tools/hierarchyviewer
You will now see your rooted phone available in the HierarchyViewer. Double click on a layout to introspect on that view tree.
Step 16 (if you get stuck in a bootloop)
Don’t panic! Simply push the backed up services.odex to system/framework to revert to your previous state.
$ adb push ./system/framework/services.odex /system/framework/services.odex
Note, you should make sure you don’t reboot when if you get a bootloop. In some cases the OS will not be at a state at which your SU binary is functional (I’ve seen it segfault many times). If you don’t have root your only option is to reflash the device via fastboot or other bootloader.
Although this API is not meant to be public, it’s extremely useful to developers when debugging their UI. It’s a shame you have to jump through so many hoops to obtain this functionality. I hope to see Android keep the ViewServer in further builds and provide a means of using the HierarchyViewer on production phones as well.
(As an aside: we refered to HierarchyViewer as the “usual client” of ViewServer above. Other clients are certainly possible. One issue with HierarchyViewer is it’s pretty damn slow – we’ve been working on our own optimized client over here at Apkudo, which will be the subject of a future blog!)
Happy Hacking,
David Teitelbaum
VP of Engineering
@davtbaum
apkudo 