|Censorship, Freedom of Speech, Privacy|
Please join the Electronic Frontier Foundation ( EFF.org ) and the fight for your rights on the Internet.
I have not received any National Security Letter.
|HOME||Software||Lotus Cars||DWARF||Kindle||eeepc||PALM||RPM Building|
libdwarf and dwarfdump are Free Software.
The distribution consists of C and C++ source code that you compile with your C/C++ compiler (documentation is included).
Beginning May 4, 2016 there is a list of critical vulnerabilities in libdwarf. Critical because these lead to serious difficulties where the calling application could crash (segfault, etc).
For a list of current known vulnerabilities and status, see https://www.prevanders.net/dwarfbug.html
For an xml version of the same data one should refer to https://www.prevanders.net/dwarfbug.xml
The DWARF Debugging Information Format is of interest to programmers working on compilers and debuggers (and anyone interested in reading or writing DWARF information). It was developed by a committee (known as the PLSIG at the time) starting around 1991. Starting around 1991 SGI got involved with the committee and then developed the libdwarf and dwarfdump tools for SGI-internal use and as part of SGI IRIX developer tools. From around 1993 dwarfdump and libdwarf were shipped (as an executable and archive respectively, not source) with every release of the SGI MIPS/IRIX C compiler. In 1994 (I think the correct year) SGI agreed (at my request) to open-source libdwarf (and in 1999 to open-source dwarfdump) so anyone could use them.
License terms are mostly GPL (version 2) or LGPL (version 2.1). The details are discussed in the license page.
DWARF specifications and the dwarf email forums are at http://www.dwarfstd.org. You can sign up for an email discussion list there.
Email about libdwarf and dwarfdump may be sent to libdwarf-list -at- linuxmail -dot- org (replace -at- and -dot- with the normal single characters to form the email address).
dwarfdump cpu time spent doing -f or -F is now reduced by 25-50 percent. On one X86_64 machine user cpu went from 508 down to 323 seconds (to run 17K tests of a regression test run). One pattern of libdwarf use by dwarfdump made a small memoization effort inside libdwarf work well. Your milage may vary. (June 13, 2016)
Corrupt relocation records are noticed and prevented from corrupting in-memory data. Thanks to to Yue Liu for the new test case. (May 17, 2016)
Many checks added to catch corrupt DWARF thanks to Yue Liu and his test cases. Thanks to Sture Carlson for pointing out a mistake in the code used to generate tables in the dwarfdump build. Fixes pushed to Sourceforge. (May 12, 2016)
Thanks to Etienne Berg for reporting a missing comma in libdwarf/dwarf_error.c which meant error numbers 264 and higher returned incorrect strings from dwarf_errmsg(). Now that string table is checked by test code. Now, at build time, the array size is checked for correctness. Thanks to Tom Murphy for passing on a report by lieanu of a bug where libdwarf failed to check for a NULL when it really should have checked. Thanks to Yue Liu for providing small test cases showing where libdwarf was not careful to check for erroneous DWARF. Fixes pushed to Sourceforge. (April 27, 2016)
Fixed problems in the -k checking code of DIEs and improved reporting of abbreviations information in DIEs and with -b (try adding -G -v). Fixed a problem where (with a NULL error pointer argument) libdwarf attempted to dereference NULL. (March 12, 2016)
Recent commits to sourceforge have been cleanups provoked by compiler warnings (the --enable-wall configure option adds a bunch of gcc warning options). Cross-compilation of libdwarf is a little easier as a result of makefile/configure suggestions by Hannes Domani. The SHF_COMPRESSED Elf section flag (meaning zlib compression) is supported for reading DWARF2 through DWARF5 in libdwarf. (February 14, 2016)
Thanks to Hannes Domani for providing a link to some work by Jose Fonseca. Work of interest to folks wanting to access DWARF in a PE object file. See https://github.com/jrfonseca/drmingw/tree/master/src/mgwhelp . It presumes one is using MinGW as a Windows development environment. The C++ source file dwarf_pe.cpp shows how to use existing facilities in libdwarf to access an object format that libdwarf knows nothing about. Basically one creates a set of properly defined function pointers and calls dwarf_object_init() and then the libdwarf function calls are available.. The mgwhelp source is LGPL. (February 07, 2016)
Added support for DWARF5 DW_FORM_line_strp in dwarf_formstring().
Added checks for too-small .debug_frame .eh_frame sections. However this is just the tip of the iceberg when it comes to preventing crashes in the face of bogus input. Most places that are reading bytes of DWARF data simply assume the read won't run off the end of anything. With a little macro magic it's not difficult to do MUCH more thorough testing while reading from memory and yet let one turn off all that testing. It's is not at all clear how much a performance-hit thorough checking would be. For example, checking reads of leb numbers would likely mean checking every byte for still-allowed-pointer. Is such protection from damaged objects a critical feature? (January 19, 2016)
libdwarf.h: in one new macro function interface the argument names were not commented out. Normally not a problem but formally a mistake. Sorry. It's fixed on sourceforge. The new DWARF5 (and DWARF4 with currrent gcc) macro section (.debug_macro) interface functions are now documented in version 2.38 of libdwarf2.1.pdf on sourceforge. (January 16, 2016)
Thanks to Tom Hughes for bringing a problem reading a badly-damaged (fuzzed) elf object to my attention: now libdwarf gets an error not a coredump. Thanks to Tom Kittel for suggesting a 4 line fix to a Makefile that lets one easily build and use a shared-library (.so) libdwarf with dwarfdump (or, just as easily, build and use an archive version). Thanks for Emre Kultursay for finding a couple of bugs in libdwarf location list handling and providing the fix. (December 30, 2015)
Libdwarf reads compressed 'zdebug' dwarf sections (transparently, automatically). Thanks to Gernot Klingler for providing an example executable and demonstrating the GNU objcopy option that creates zdebug sections from ordinary DWARF debug sections. Libdwarf now uses zlib if zlib is present at configure time. If zlib is not present at configure time it all still builds but the result won't deal with zdebug sections. (November 27, 2015)
Now allows building libdwarf dwarfdump dwarfgen and dwarfexample separate from the source tree. The build uses configure everywhere now. Thanks to Kubo Takehiro for some configure and Makefile suggestions and for motivating me to complete this feature. (November 15, 2015)
Libdwarf/dwarfdump now supports reading gcc's two-level line tables and split dwarf location lists and some other DWARF5 data.. Thanks to Cary Coutant and Emre Kultursay as their assistance was crucial to getting it done. (November 14, 2015)
Thanks to Edward Williamson, Arnaud Diederen, and Remi Gurski for noticing incompleteness in error condition tests in the libdwarf consumer source code. Fixes for all these issues are in the SourceForge source as of today (one new fix February 13 2015).
Corrections in the use of va_end() pushed to sourceforge on 12 January, 2015. And addition of includes of stdarg.h in dwarfdump source pushed to sourceforge 15 January 2015. New options for checking DWARF use (in dwarfdump) pushed to Sourceforge on 08 January, 2015. The regressiontests file set has new tests. Some improvements in handling malloc-failure are included in the source. Thanks to Carlos Alberto Enciso for doing the dwarfdump enhancement.
There no longer seems much need for dwarfdump2 now that dwarfdump has search code built in (dwarf_tsearch). As of the 20150115 release dwarfdump2 is no longer present.
"Code Testing Through Fault Injection" in :login; magazine (December, 2014. Usenix.org) by Peter Gutmann offered a simple idea from an unnamed friend: instrument malloc() so on call N malloc() returns NULL. Here are the details
Results from tests based on this idea make it clear that having the dwarf_error() return be dependent on malloc is not such a good idea when malloc is out of space. The change creates no change in interfaces or semantics, it just uses a static Dwarf_Error_s struct when the alternative is to not really report an error.
Code using the consumer (reader) calls need not change. Binary and source compatibility is retained for consumer code.
The dwarf_producer_init() function interface changed so existing calls will fail at caller compile time. The producer callback function name changed too. This returns the code to a single producer-init function and one with an easier to understand option list. It also results in the elimination of extremely messy producer configure-time options and code #ifdefs relating to relocation generation in favor of simple run-time choices. Though this change is a problem for a few users it seemed inevitable and waiting for some future date did not seem productive.
A correspondent reports that a Go language implementation emitted DW_FORM_ref_addr according to DWARF V2 for the case of 64 bit addresses with 32 bit DWARF offsets. Libdwarf was assuming no one was going to do that in the odd way DWARF2 documented. Now libdwarf follows the standard (as it really should have all along). Thanks to Arnaud Diederen for pointing out this blunder and for providing a tiny Go test source.
Because tsearch() itself is not universally available (and even when tsearch() is available tdestroy() is sometimes not available) I implemented some tree algorithms using the standard tsearch interface definitions.
The basic four tsearch Standard interface declarations are quite old, traditional, incompletely documented, hard to use, and not at all what anyone would design as an interface today. But these four functions are declared in the Posix/SingleUnixSpecification standard.
The algorithms implemented are binary tree, binary tree with Eppinger delete, balanced binary tree, red black tree, and even a hashing version). The function interfaces implemented include tsearch(), tdelete(), tdestroy(), tfind(), and twalk(). All these are in a directory named tsearch beginning with the 20140131 release of libdwarf. The test data for tsearch testing was moved to the regressiontests git repository as of the 20140208 release (to save space in the release source tar file).
To avoid conflicts with standard library versions all the function names are prefixed with dwarf_. Libdwarf and dwarfdump now use the new tsearch.
I ran all the distribution's source code through the checker called STACK from mit.edu. It checks C/C++ source for many sorts of errors. For example, it finds undefined-effect C code. Code optimizers increasingly delete code with undefined-effect so what used to work in your code can stop working. STACK depends on building llvm and clang with specific options and all this is nicely defined on the STACK web page.
I highly recommend this checker for any code you ship.
With Ubuntu 13.10 I found it easy to run C code through the tests. With C++ there were issues with missing gcc headers which required an annoying workaround. I fixed the two errors STACK found, both of which were in libwarf's producer code. The errors involved using a pointer before testing it for NULL. The tests involved would only have effect if callers passed in invalid arguments and meant callers could coredump instead of seeing an error return.
January 31, 2013: Announcing this incompatible change!
In June 2013 there will be an incompatible change to libdwarf.h which will mean those compiling against the producer code in libdwarf will encounter an error. The Callback function prototype will add 'const' to a char * argument in the Callback functions whose types are named below. This is not a binary incompatibility it is a source compile time incompatibility.
Generally the only people affected are those compiling a compiler that uses libdwarf to generate DWARF2. The function prototypes in libdwarf.h are named Dwarf_Callback_Func_c, Dwarf_Callback_Func_b, and Dwarf_Callback_Func.
It will not affect folks linking against libdwarf but not recompiling the code calling libdwarf.
The fix is simple: add 'const' to the char* argument to your libdwarf callback prototypes and implementation.
This change will let us eliminate several compiler warning messages from the build of libdwarf. It will not affect folks calling only the consumer interfaces of libdwarf. Only those who coded callback functions using the libdwarf producer callbacks are affected.
All libdwarf distributions contain C source plus the DWARF2 specification plus libdwarf specifications. Implementors often extend DWARF by adding new attributes and other things. Those that we are aware of are defined in dwarf.h which is provided here for reference. If you have corrections or additions, please let me know! The file is in every distribution of libdwarf.
Beginning 19 March 2011 libdwarf source code is available
via anonymous access with git.
The git source code is the most recent: it may have
features or fixes not in a tar.gz release, but if so the
features or fixes are not needed by most people.
Top-of-trunk code in the git repository has been fully tested.
"git clone git://git.code.sf.net/p/libdwarf/code"
initializes a git repository in the local directory it will create named "code" and populates it with the most up to date libdwarf source.
The regression tests (which you really don't need or want) are available
"git clone git://git.code.sf.net/p/libdwarf/regressiontests" .
A small indent-checking tool in case you make changes and want
to preserve proper libdwarf indentation is in
"git clone git://git.code.sf.net/p/dicheck-da/code"
but since this one also creates a directory named "code" do this command in a different directory from the one used to hold libdwarf "code".
The libdwarf build process involves a simple traditional approach (at least for personal use, people generating binary releases or a corporate library will have their own internal requirements to consider). In the base directory of the source distribution (either expanded from a tar.gz or from git) do the following:
./configure make dd
If all goes well, this will build libdwarf and dwarfdump.
If you want to try building all the executables, do
which will build libdwarf, dwarfdump, dwarfgen, and dwarfexample in a minute or two. You don't really need dwarfgen or dwarfexample, so if the compile fails in one of those don't worry about it. If libdwarf and dwarfdump complete their build you have most of the functionality you need. dwarfdump (written in C) lets you dump out, in readable form, the DWARF2, DWARF3, DWARF4, and (likely in 2015) DWARF5 data from an object file.
There are some prerequisites you must have installed on your build machine:
C compiler (conformant with, at minimum, the 1989 C standard) C++ compiler (for dwarfgen) libelf (for libdwarf and dwarfdump and dwarfgen).
None of this understands the object files in Apple MacOSX. I don't have time to provide such, and no one else has proposed contributing such support. Contact the libdwarf-list email address before attempting to contribute any code.
None of this understands the object files in Microsoft Windows. A few people have made libdwarf and dwarfdump work on Windows, but none have contributed any support code. Contact the libdwarf-list email address before attempting to contribute any code. Note that libdwarf has specific design features that enable reading of non-Elf object files without very much difficulty, but you will have to write your own code to read those files and will need to do a special build of libdwarf that does not use Elf related headers (which should be relatively easy). Use of one of the POSIX-like environments like mingw or cygwin or the like will likely ease your way considerably.
The regression test build process involves rather more work, and few will want to bother with it. There is no tar.gz available, you have to use git. In the base directory of the regressiontests distribution do the following (if the base of the libdwarf source tree is at ../code this should work, but if not see the regressiontests/README.txt file):
Rather than keeping known-good output in the regression-tests, we keep a dwarfdump.O (O for Old) compare output against the new dwarfdump. We do it this way as the test output is much too large to save.
To run the dwarfextract test successfully, bfd.h needs to be installed in a standard place, so install binutils-dev (or the equivalent for your release) to get bfd.h.
On a modern machine running directly on the host the tests should run in about 40 minutes and the final message should say PASS. Unless the components in the libdwarf source build correctly the tests cannot pass. Unless you are running Ubuntu GNU/Linux or FreeBSD 9.1 you may have more work to do to run the tests, partly because the tests depend on having a known-good version of dwarfdump in the distribution.
If the Ubuntu or FreeBSD objects in the distribution won't run in your test environment you could build dwarfdump.O to run tests:
|filename, download-link||Bytes||Reason for release|
Incorporates code detecting malformed DWARF and malformed Elf object files. Dwarfdump and libdwarf performance reading frame data improved significantly, though the improvements don't apply to all frame data interface functions. The configure/make system now builds libdwarf shared objects (when asked to) with a proper soname. Thanks to Sture Carlson, Hannes Domani, etienneberg, Steve Kaufman, Yue Liu, and Fabian Wolff for their help/comments on libdwarf. Major thanks to Carlos Alberto Enciso for his collaboration.
sha512sum (remove spaces following colon): 430978587e284f4c63e54790a7cbbfdf13c7a4154b 9e130dc788869b6c6584981c5af46b7b363d3b181e aaafd5190bcb17e5383f8600cb1d2627c423a7042dc2
Incorporates many additional checks so that corrupt dwarf will not crash an executable calling libdwarf. Thanks to Yue Liu for providing a number of small and corrupted objects.
sha512sum (remove spaces following colon): 8a4d3721390d4d79fbf8d01fb27995ac9871f1d3e 19d85aa25c108ee4cc45968a5331e2b477f559168 c4597513235fb508bfe4ee1664c6dfbede3a47d48490b5
Now reads and prints DWARF5/4 macro data (The new .debug_macro section). Added some checks of abbreviation codes. Thanks to Emre Kultursay, Tom Kittel, Gernot Klingler, and Kubo Takehiro for suggestions and test cases. See the top-level README for hints on building the package outside of the source tree and building and using the shared-library version of libdwarf.
sha512sum (remove spaces following colon): 594519460d3cd3f60f7e97931ce3e3d775393e03068a 2c932d72ba95d8cdb2c53b622671c5af8a798712937b 62dbf9594e2da8945553a0021a275cf677f4d90a
Support for reading Split Dwarf object files provided. New location and location-list interfaces added for Split Dwarf and DWARF5. Uses new functions to pass location list data to callers, not publicly defined structures, as functions make any future changes to location information easier to support. Existing DWARF 2,3,4 location information interfaces retained for compatibility so existing calling code is not affected..
sha512sum (remove spaces following colon): 22c6a233cf156f3e7a8ad65c6b0f3c6b0de5a7ddc0f 1c71c9b2dc7efa59a4ee1c9714e981bb26d40a0c212 7501a3e853a7605af10be96bdcb0486723d4a1443c
Corrected accidental C99-isms, added missing return statements, and improved checking. Thanks to Carlos Alberto Enciso for these enhancements. I suggest using the git source base, it has some small fixes beyond this release related to new code for the .debug_addr and the .debug_str_offsets sections (DWARF5) and for reading gcc experimental two-level line tables.. Thanks to Emre Kultursay for encouraging me to add some needed DWARF5 features and for reporting bugs so the git source base gets corrected quickly.
sha512sum (remove spaces following colon): a567e653fdad598d911e2ed7e219945adbf0f00ef 8e81806e993916aacfc8075657ec3ca925e1efa85 c7860a0d9515cae2b04677dfb25ac416fb4cc47933441a
Strings are checked thoroughly to ensure they do not run off the end of their section without a termination and cause chaos in libdwarf. Now with DWARF5 (and DWARF4) Debug Fission support, so one can nearly transparently (see dwarf_set_tied_dbg()) extract addresses denoted DW_FORM_addrx in a package file from the executable with the .debug_addr section. Withdrawn as 20150915 has this and more.
sha512sum (remove spaces following colon):
Now with DWARF5 (and DWARF4) Package File reader support. Package Files are a way to keep DWARF debug information in a separate object file. DWARF5 is not a released standard, so the new features must be considered tentative.
sha512sum (remove spaces following colon): 3495f6c17b5ade3a9f38e4d92b63f318e1f69d8841 95e6eafecb99c49e9366ed5c0fbdabd15f6d3b79c9 426565a5960364bb1ac1d1cb185363318872cdf83520
Improved range checking. Fixed bugs in dwgetopt(). Added omitted tags and attributes lists in dwarfdump to make error checking more accurate. Where pc checks make no sense in a line table (meaning on DW_TAG_type_unit and children) such dwarfdump checks are now omitted.
sha512sum (remove spaces following colon): 983c0bb5d70f59e95b8b9de9cda74d714795526220 ac944b6e058554b1f1e831063ae5524d6a2de557e1 fe829ccbf17b1ab71195fa5589504ead3d94396ab0a4
The omission of an include of stdarg.h in the 20150112 release meant that standard-conforming compilers could get an error compiling dwarfdump due to va_list not being defined, though Ubuntu and Freebsd compiles did not see such an error. Now dwarfdump does the include. The dwarfdump2 source has been removed as of this tar.gz file. Dwarfdump2 is no longer needed.
sha512sum (remove spaces following colon): abcc465f3fcc369143cb34976ad2874b5a9d2a6b4f732be2 b83b0e7620799747778947aed0c10872a5fad73443cfa986 48bdc50a3b17c98292a6439b54d60222
New checking options added to dwarfdump. Dwarfdump2 no longer updated or compiled. Improvements in internal checking to handle badly formed Elf and Dwarf files. Better recovery from malloc failure.
sha512sum (remove spaces following colon): 23fe3dec516a90c2cedb851971f2fd902e056bf4471a32 a2237881354d71af866fcdbc9d3ff55d4e83b75fdafd9f 7790bf90165506e9285259c67f342c4eccba
Source incompatibility for users of producer code: dwarf_producer_init() now has a new interface and a sensible way to select the output ABI/ISA for relocation numbers. Part of preparation to emit DWARF3,4,5. The reader code now handles DebugFission, called Split Dwarf objects in the draft DWARF5 standard.
sha512sum (remove spaces following colon): ccf8180b69cdb47902564dda1fca52d15c10239ce6bb8c c9f5af5a67d37f888811572e314414372bfbc2b640c1fff 7cf87542f782f8390f733d884a24d9a16fb
Libdwarf now follows the DWARF2 standard properly in reading DW_FORM_ref_addr. The original DWARF2 standard is on dwarfstd.org so it's gone from the libdwarf source.
sha512sum (remove spaces following colon): 7ecd27b40418fd98bb24ee59b9779efe30dca26384b4a36f5 a1b0a99805f4d8ff281b2e3d4470fb8e8da28045c34bf6b53 fdf85e9dfc5fa76c0eac8462ae8467
Fixed a bug in dwarfdump Makefile.in so parallel make works reliably. Removed remaining trailing-whitespace. Moved tsearch testcases over to the libdwarf regressiontests repository as the tests seemed too large to keep in the source.
sha512sum (remove spaces following colon): d8ba3eeaf36d98a1ee26397208fff1658a2b7a41c 25d3742a81617c74d6359aeff08bb6221f99b9937 9030575578e11cd66be1d906a5832a6edd362229ce2e7e
Radically simplifies libdwarf allocation code. Adds GNU-specific DW_FORM codes so recent gcc objects can be read usefully. Adds tsearch implementations (see the tsearch directory). Removes trailing whitespace (it appeared all over). Adds new functions to libdwarf for new DWARF reference types. The tsearch test cases have bloated this source release, but in future those test cases won't be in the tar file (instead they are in the libdwarf regressiontests repository on sourceforge).
sha512sum (remove spaces following colon): d41ebe4e7b76ad91f93b17e33da878fb0a35d7a35 32d641108b217bf93bcd9f10c1d52f0dd5f2ece08 3152d8dafe637ae343633f894122cbb77825b7a3350ed2
Added AARCH64 relocation support. Fixed some simple compiler warnings.
sha512sum (remove spaces following colon): f9d25cfd6c6b15bebf6cd63c7014ecf4123798fce637c 0da103008758d6a9d5705c3797216a8d1ab3e210c4235 f199ab19d7ed0bf6c3582f49eacba1629c5cc0
Now with a simplified build (see README) and with checks for most compiler errors in producing DW_AT_sibling attributes. Verified on Ubuntu 12.10 and FreeBSD 9.1.
sha512sum (remove spaces following colon): 1cb272f80745a789d592d57e6a64b1b4ec6e1b646653 da2f19c2e2d803b8e90b52f5c69597360bf1703b1c25 7844c53c3f43a80cd9f45a2adb314ad2511c19e9
A mistake in handling DW_OP_GNU_const_type could lead to a libdwarf coredump at times, and even when it appeared to work the value printed was wrong. Thanks to Tom Hughes and Andrew Bernat for pointing out the DW_OP_GNU_const_type problem. This release has libdwarf interfaces compatible with releases before 20130125. Compared to 20130125 this restores the libdwarf.h interface Dwarf_Loc structure and changes the way the lr_number2 member is used for the new location expression operator DW_OP_GNU_const_type.
sha512sum (remove spaces following colon): c9911ce0b9725400ec1a70e809e185b122095c534f05687 ea0be16a0e9bfe3b8128e353834c43b5fe80e1b149b11 f113fcce53c57bad0a12b3b091dd0e31d043
For information on previous releases, see the older release list
Every release of libdwarf/dwarfdump is tested with many options and option-combinations. There is no real reason libdwarf/dwarfdump users should need to redo this testing work. However, if you do wish to try running the regression tests, you will find them in git in sourceforge via anonymous access "git clone git://git.code.sf.net/p/libdwarf/regressiontests". See the file README.txt in the base directory of the tests for an overview of the test process. We do not provide a tar file of the regression tests. The regression tests are updated with every libdwarf release.
Additional tests (simple object files or shared-objects or executables) are accepted here should you wish to submit such, but the goal is to add tests that represent previously-untested aspects of DWARF/libdwarf, not to duplicate existing tests. Smallish object files are preferred. No source need be provided. The submitter has to be sure, and state, that releasing the objects here is appropriate.
Nothing in libdwarf looks at very many sections of an Elf file: the other section contents can be zeroed out without affecting the object use for testing libdwarf (see the directory named 'zero' in the testing distribution for a convenient byte-zeroing helper application in C++ source). Sections like .text and .data (and closely related sections) are of no value for testing libdwarf. It may be that otherwise-proprietary objects can be released for use in this test suite once the instruction and data sections are zeroed out.