15 September 2009

How to get EMMA code coverage of Android

Here are some basic steps How to get Emma code coverage of Android.

Thanks to Brett Chabot and Gabor for useful support at Android Developer Group

Before doing these steps, you need to get full source code of Android (follow this link for more details )

After that, go to root folder of Android source code and do extractly this instruction below.

A. Generating code coverage using runtest script

1. Firstly, you need to add the target system/framework/emma.jar to the device's boot classpath. So that, modify the BOOTCLASSPATH variable in /system/core/rootdir/init.rc .

a. Open /system/core/rootdir/init.rc

b. The system/framework/emma.jar entry needs to be added in the exact position shown below:
export BOOTCLASSPATH=/system/framework/core.jar:/system/framework/ext.jar:/system/framework/emma.jar:/system/framework/framework.jar:/system/framework/android.policy.jar:/system/framework/services.jar

c. Save and close init.rc file

2. Since EMMA will save the coverage result into SDCard, you need to ensure that the SDCard can be mounted properly. So that,

a. Open this file: build/core/main.mk

b. Ensure that these lines have already added (around line number 212 to 219), if not, do it by yourself!
# Install a vold.conf file is one's not already being installed.
ifeq (,$(filter %:system/etc/vold.conf, $(PRODUCT_COPY_FILES)))
PRODUCT_COPY_FILES += \
development/data/etc/vold.conf:system/etc/vold.conf
ifeq ($(filter eng tests,$(TARGET_BUILD_VARIANT)),)
$(warning implicitly installing vold.conf)
endif
endif

(Please follow this link for more details )

3. Build the bootimage to pick up the init.rc changes
make bootimage

4. Build a full system image
make -j4

5. Make EMMA
export EMMA_INSTRUMENT=true

make emma

6. Start emulator
[path_to_SDK_tools]/emulator -sdcard [path_to_your_sdcard]

7. Next, use the runtest.py script. Runtest will do all the necessary steps to instrument your test and target package, run the test, and generate the code coverage report.
cd [path_to_android_source_code]

python development/testrunner/runtest.py --coverage [Test_package]

[Test_package] can be: apidemos, core, music, email... (Leave this option null to see the list).

8. After finish this script, the coverage report will be generated and saved into

[path_to_Android_source_code]/out/emma/

folder. You can find a html report of EMMA code coverage for [Test_package] there.

=========================================================

B. Generating code coverage for your own test

If you want to running code coverage for your own test, please follow steps as described above, but skip step 6 to 8, continue with step 9 (after step 5) shown below:

9. Setting the environment and additional bash commands. (like m,mm,mmm, choosecombo etc) Notice the space after the dot!
. build/envsetup.sh

10. Set the ANDROID_PRODUCT_OUT directory for the emulator to know the image location
export ANDROID_PRODUCT_OUT=[path_to_Android_source_code]/out/target/product/generic

This step is important. The emulator will know where are the images it need to be synchronized with.

11. Set EMMA_INSTRUMENTATION to true
export EMMA_INSTRUMENT=true

12. Compile the Application would like to instrument
mmm development/samples/[your_project]

(You can use apidemos for example)

After run this step, new images can be generated and saved into ANDROID_PRODUCT_OUT (in step 10)

13. Run emulator
[path_to_SDK_tools]/emulator -sdcard [path_to_your_sdcard]

14. Remount the drive - it is needed to have a writable drive. without that sync wont work
adb remount

15. Synchronize the local content (new images) with the emulator
adb sync

16. Run instrumentation
adb shell am instrument -w -e coverage true [source_code_of_test]

Here is example for apidemos
adb shell am instrument -w -e coverage true com.example.android.apis.tests/android.test.InstrumentationTestRunner

See this for more details of InstrumentationTestRunner options.

17. This command will dump a runtime coverage data file at /sdcard/coverage.ec on the device.

Extract it to local host
adb pull /sdcard/coverage.ec coverage.ec

18. Now generate a coverage report. You'll need to pass in the path to the coverage metadata generated at build time.
java -cp external/emma/lib/emma.jar emma report -r html -in coverage.ec -sp [path_to_your_project_source_code] -in out/target/common/obj/APPS/[your_project]_intermediates/coverage.em

Here is example for apidemos:
java -cp external/emma/lib/emma.jar emma report -r html -in coverage.ec -sp development/samples/ApiDemos/src -in out/target/common/obj/APPS/ApiDemos_intermediates/coverage.em

19. Now, you will get the HTML report of EMMA code coverage for your own project.

Check it out!


Notes: In the first time running instrumentation (step 7 or 16) the Android system may be crashed. But, don't worry, it will automatically restart. Then, you can run the instrumentation again successfully!

2 comments:

Sathy said...

Hello Duykham,

I was trying to replicate the same steps for my code coverage but on my target device. i have set the base/Android.mk with the EMMA_INSTRUMENt=true and am able to see the code genearting the coverage.em files.But now that am working on the target am held up from even trying to run the ApiDemos on the target device.
Could you please help me know the steps that should be followed.

Thanks
Satish

Android app development said...

This is one of the genius post.I like your your blog power.This is one of the supper Information.i like this post.Android app developers

Shellcode that bring back tty input

There are cases that you think you have been able to exploit the bug (e.g. buffer-overflow) but the program is terminated right awa...