Deriving Code Coverage Informationfrom Prof i ling Data Recorded for aTrace-based Just-in-time CompilerChristian HäublInstitute for System SoftwareJohannes Kepler University LinzAustriahaeubl@ssw.jku.atChristian WimmerOracle LabsUSAchristian.wimmer@oracle.comHanspeter MössenböckInstitute for System SoftwareJohannes Kepler University LinzAustriamoessenboeck@ssw.jku.atAbstractCode coverage information is typically recorded by adding instru-mentation to an application before or during execution. However,this has the disadvantage that the instrumentation decreases the per-formance.A high-performance virtual machine (VM), such as Oracle’sJava HotSpot VM, is already designed to record prof iling data forthe executed application. This prof iling data is intended to guideoptimizations during just-in-time (JIT) compilation. In this paper,we propose a runtime system that allows deriving exact code cover-age information from this recorded prof iling data by guaranteeingcertain system properties. With this approach, we avoid the has-sle of instrumenting the application explicitly. Therefore, we mini-mize both the run-time overhead and the implementation effort forrecording code coverage information.We implemented our approach for a variant of the Java HotSpotVM that uses a trace-based JIT compiler. However, our approach issimple and general enough to be applicable to most modern VMs.Our evaluation with industry-strength benchmark suites shows thatthis approach allows recording code coverage information whileretaining almost full peak performance. So, our approach is alsosuitable for collecting code coverage information in day-to-dayoperation environments such as long running server applications.Categories and Subject Descriptors D.2.5 [Software Engineer-ing]: Testing and Debugging—Testing tools (e.g., data generators,coveragetesting); D.3.4[ProgrammingLanguages]:Processors—CompilersGeneral Terms Measurement, PerformanceKeywords Java, code coverage, just-in-time compilation1. IntroductionCode coverage metrics are used to determine the statements,branches or paths that were covered during the execution of a pro-Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor prof it or commercial advantage and that copies bear this notice and the full citationon the f irst page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).PPPJ’13, September 11–13, 2013, Stuttgart, Germany.Copyright is held by the owner/author(s).ACM 978-1-4503-2111-2/13/09.http://dx.doi.org/10.1145/2500828.2500829gram. These metrics are useful for f inding out whether the programwas tested extensively enough and are usually obtained by instru-menting the code.Figure 1 shows the steps that are typically involved to recordcode coverage information for a Java application. The f irst step isto instrument the application either in an off l ine preprocessing stepor online while the application is executed. The code coverage toolinserts probes at all relevant positions of the control f l ow. Whena probe is executed, it marks a certain part of the application ascovered. When the application exits, the coverage data is persisted,either by storing it in a f ile or transmitting it to another machine.The actual computation of the covered code parts is then doneoff l ine by analyzing the recorded coverage data and combining thisinformation with the application’s source code.Theproblemwiththisapproachisthatinstrumentationdegradesthe performance of the executed program and is therefore typicallyonly enabled during testing and switched off in daily operation.For Java, several code coverage tools exist and to the best of ourknowledge, all of them add instrumentation code to the applicationor to the virtual machine (VM). However, instrumenting an appli-cation has some disadvantages such as slowing down the executionor even introducing bugs that cannot be reproduced without instru-mentation. Approaches to reduce the overhead of the code coverageprobes can be divided into two categories (see Section 5). The f irstcategory of optimizations tries to avoid redundant probes so thatfewer probes have to be inserted and less instrumentation code hasto be executed at run time. The second category removes code cov-erage probes when they are no longer needed. Removing no longerrequired probes is eff icient but increases the complexity of the codecoverage tools.We propose a system where no instrumentation code has to beadded explicitly to record exact code coverage information. In-stead, we exploit the fact that modern high-performance VMs al-ready have an instrumented interpreter or baseline compiler thatrecords prof iling data. Normally, this prof iling data is only usedinstrumentationapplicationinsertsVMexecutesprobesraw code coverage dataapplication source coderecordsanalyzes andcomputescode coverage informationFigure 1. Obtaining code coverage information1
