|Censorship, Freedom of Speech, Privacy|
Please join the Electronic Frontier Foundation ( EFF.org ) and the fight for your rights on the Internet.
I have not received any National Security Letter.
|HOME||Software||Lotus Cars||DWARF||Kindle||eeepc||PALM||RPM Building|
libdwarf and dwarfdump are Free Software.
The distribution consists of C and C++ source code that you compile with your C/C++ compiler (documentation is included).
The DWARF Debugging Information Format is of interest to programmers working on compilers and debuggers (and anyone interested in reading or writing DWARF information). It was developed by a committee (known as the PLSIG at the time) starting around 1991. Starting around 1991 SGI got involved with the committee and then developed the libdwarf and dwarfdump tools for SGI-internal use and as part of SGI IRIX developer tools. From around 1993 dwarfdump and libdwarf were shipped (as an executable and archive respectively, not source) with every release of the SGI MIPS/IRIX C compiler. In 1994 (I think the correct year) SGI agreed (at my request) to open-source libdwarf (and in 1999 to open-source dwarfdump) so anyone could use them.
License terms are mostly GPL (version 2) or LGPL (version 2.1). The details are discussed in the license page.
DWARF specifications and the dwarf email forums are at http://www.dwarfstd.org. You can sign up for an email discussion list there.
Email about libdwarf and dwarfdump may be sent to libdwarf-list -at- earthlink -dot- net (replace -at- and -dot- with the normal single characters to form the email address).
Libdwarf/dwarfdump now support reading gcc's two-level line tables (29 September 2015). Thanks to Cary Coutant and Emre Kultursay as their assistance was crucial to getting it done.
Thanks to Edward Williamson, Arnaud Diederen, and Remi Gurski for noticing incompleteness in error condition tests in the libdwarf consumer source code. Fixes for all these issues are in the SourceForge source as of today (one new fix February 13 2015).
Corrections in the use of va_end() pushed to sourceforge on 12 January, 2015. And addition of includes of stdarg.h in dwarfdump source pushed to sourceforge 15 January 2015. New options for checking DWARF use (in dwarfdump) pushed to Sourceforge on 08 January, 2015. The regressiontests file set has new tests. Some improvements in handling malloc-failure are included in the source. Thanks to Carlos Alberto Enciso for doing the dwarfdump enhancement.
There no longer seems much need for dwarfdump2 now that dwarfdump has search code built in (dwarf_tsearch). As of the 20150115 release dwarfdump2 is no longer present.
"Code Testing Through Fault Injection" in :login; magazine (December, 2014. Usenix.org) by Peter Gutmann offered a simple idea from an unnamed friend: instrument malloc() so on call N malloc() returns NULL. Here are the details
Results from tests based on this idea make it clear that having the dwarf_error() return be dependent on malloc is not such a good idea when malloc is out of space. The change creates no change in interfaces or semantics, it just uses a static Dwarf_Error_s struct when the alternative is to not really report an error.
Code using the consumer (reader) calls need not change. Binary and source compatibility is retained for consumer code.
The dwarf_producer_init() function interface changed so existing calls will fail at caller compile time. The producer callback function name changed too. This returns the code to a single producer-init function and one with an easier to understand option list. It also results in the elimination of extremely messy producer configure-time options and code #ifdefs relating to relocation generation in favor of simple run-time choices. Though this change is a problem for a few users it seemed inevitable and waiting for some future date did not seem productive.
A correspondent reports that a Go language implementation emitted DW_FORM_ref_addr according to DWARF V2 for the case of 64 bit addresses with 32 bit DWARF offsets. Libdwarf was assuming no one was going to do that in the odd way DWARF2 documented. Now libdwarf follows the standard (as it really should have all along). Thanks to Arnaud Diederen for pointing out this blunder and for providing a tiny Go test source.
Because tsearch() itself is not universally available (and even when tsearch() is available tdestroy() is sometimes not available) I implemented some tree algorithms using the standard tsearch interface definitions.
The basic four tsearch Standard interface declarations are quite old, traditional, incompletely documented, hard to use, and not at all what anyone would design as an interface today. But these four functions are declared in the Posix/SingleUnixSpecification standard.
The algorithms implemented are binary tree, binary tree with Eppinger delete, balanced binary tree, red black tree, and even a hashing version). The function interfaces implemented include tsearch(), tdelete(), tdestroy(), tfind(), and twalk(). All these are in a directory named tsearch beginning with the 20140131 release of libdwarf. The test data for tsearch testing was moved to the regressiontests git repository as of the 20140208 release (to save space in the release source tar file).
To avoid conflicts with standard library versions all the function names are prefixed with dwarf_. Libdwarf and dwarfdump now use the new tsearch.
I ran all the distribution's source code through the checker called STACK from mit.edu. It checks C/C++ source for many sorts of errors. For example, it finds undefined-effect C code. Code optimizers increasingly delete code with undefined-effect so what used to work in your code can stop working. STACK depends on building llvm and clang with specific options and all this is nicely defined on the STACK web page.
I highly recommend this checker for any code you ship.
With Ubuntu 13.10 I found it easy to run C code through the tests. With C++ there were issues with missing gcc headers which required an annoying workaround. I fixed the two errors STACK found, both of which were in libwarf's producer code. The errors involved using a pointer before testing it for NULL. The tests involved would only have effect if callers passed in invalid arguments and meant callers could coredump instead of seeing an error return.
January 31, 2013: Announcing this incompatible change!
In June 2013 there will be an incompatible change to libdwarf.h which will mean those compiling against the producer code in libdwarf will encounter an error. The Callback function prototype will add 'const' to a char * argument in the Callback functions whose types are named below. This is not a binary incompatibility it is a source compile time incompatibility.
Generally the only people affected are those compiling a compiler that uses libdwarf to generate DWARF2. The function prototypes in libdwarf.h are named Dwarf_Callback_Func_c, Dwarf_Callback_Func_b, and Dwarf_Callback_Func.
It will not affect folks linking against libdwarf but not recompiling the code calling libdwarf.
The fix is simple: add 'const' to the char* argument to your libdwarf callback prototypes and implementation.
This change will let us eliminate several compiler warning messages from the build of libdwarf. It will not affect folks calling only the consumer interfaces of libdwarf. Only those who coded callback functions using the libdwarf producer callbacks are affected.
All libdwarf distributions contain C source plus the DWARF2 specification plus libdwarf specifications. Implementors often extend DWARF by adding new attributes and other things. Those that we are aware of are defined in dwarf.h which is provided here for reference. If you have corrections or additions, please let me know! The file is in every distribution of libdwarf.
Beginning 19 March 2011 libdwarf source code is available
via anonymous access with git.
The git source code is the most recent: it may have
features or fixes not in a tar.gz release, but if so the
features or fixes are not needed by most people.
Top-of-trunk code in the git repository has been fully tested.
"git clone git://git.code.sf.net/p/libdwarf/code"
initializes a git repository in the local directory it will create named "code" and populates it with the most up to date libdwarf source.
The regression tests (which you really don't need or want) are available
"git clone git://git.code.sf.net/p/libdwarf/regressiontests" .
A small indent-checking tool in case you make changes and want
to preserve proper libdwarf indentation is in
"git clone git://git.code.sf.net/p/dicheck-da/code"
but since this one also creates a directory named "code" do this command in a different directory from the one used to hold libdwarf "code".
The libdwarf build process involves a simple traditional approach (at least for personal use, people generating binary releases or a corporate library will have their own internal requirements to consider). In the base directory of the source distribution (either expanded from a tar.gz or from git) do the following:
./configure make dd
If all goes well, this will build libdwarf and dwarfdump.
If you want to try building all the executables, do
which will build libdwarf, dwarfdump, dwarfgen, and dwarfexample in a minute or two. You don't really need dwarfgen or dwarfexample, so if the compile fails in one of those don't worry about it. If libdwarf and dwarfdump complete their build you have most of the functionality you need. dwarfdump (written in C) lets you dump out, in readable form, the DWARF2, DWARF3, DWARF4, and (likely in 2015) DWARF5 data from an object file.
There are some prerequisites you must have installed on your build machine:
C compiler (conformant with, at mininum, the 1989 C standard) C++ compiler (for dwarfgen) libelf (for libdwarf and dwarfdump and dwarfgen).
None of this understands the object files in Apple MacOSX. I don't have time to provide such, and no one else has proposed contributing such support. Contact the libdwarf-list email address before attempting to contribute any code.
None of this understands the object files in Microsoft Windows. A few people have made libdwarf and dwarfdump work on Windows, but none have contributed any support code. Contact the libdwarf-list email address before attempting to contribute any code. Note that libdwarf has specific design features that enable reading of non-Elf object files without very much difficulty, but you will have to write your own code to read those files and will need to do a special build of libdwarf that does not use Elf related headers (which should be relatively easy). Use of one of the POSIX-like environments like mingw or cygwin or the like will likely ease your way considerably.
The regression test build process involves rather more work, and few will want to bother with it. There is no tar.gz available, you have to use git. In the base directory of the regressiontests distribution do the following (if the base of the libdwarf source tree is at ../code this should work, but if not see the regressiontests/README.txt file):
Rather than keeping known-good output in the regression-tests, we keep a dwarfdump.O (O for Old) compare output against the new dwarfdump. We do it this way as the test output is much too large to save.
To run the dwarfextract test successfully, bfd.h needs to be installed in a standard place, so install binutils-dev (or the equivalent for your release) to get bfd.h.
On a modern machine running directly on the host the tests should run in about 40 minutes and the final message should say PASS. Unless the components in the libdwarf source build correctly the tests cannot pass. Unless you are running Ubuntu GNU/Linux or FreeBSD 9.1 you may have more work to do to run the tests, partly because the tests depend on having a known-good version of dwarfdump in the distribution.
If the Ubuntu or FreeBSD objects in the distribution won't run in your test environment you could build dwarfdump.O to run tests:
|filename, download-link||Bytes||Reason for release|
Corrected accidental C99-isms, added missing return statements, and improved checking. Thanks to Carlos Alberto Enciso for these enhancements. I suggest using the git source base, it has some small fixes beyond this release related to new code for the .debug_addr and the .debug_str_offsets sections (DWARF5) and for reading gcc experimental two-level line tables.. Thanks to Emre Kultursay for encouraging me to add some needed DWARF5 features and for reporting bugs so the git source base gets corrected quickly.
sha512sum (remove spaces following colon): a567e653fdad598d911e2ed7e219945adbf0f00ef 8e81806e993916aacfc8075657ec3ca925e1efa85 c7860a0d9515cae2b04677dfb25ac416fb4cc47933441a
Strings are checked thoroughly to ensure they do not run off the end of their section without a termination and cause chaos in libdwarf. Now with DWARF5 (and DWARF4) Debug Fission support, so one can nearly transparently (see dwarf_set_tied_dbg()) extract addresses denoted DW_FORM_addrx in a package file from the executable with the .debug_addr section. Withdrawn as 20150915 has this and more.
sha512sum (remove spaces following colon):
Now with DWARF5 (and DWARF4) Package File reader support. Package Files are a way to keep DWARF debug information in a separate object file. DWARF5 is not a released standard, so the new features must be considered tentative.
sha512sum (remove spaces following colon): 3495f6c17b5ade3a9f38e4d92b63f318e1f69d8841 95e6eafecb99c49e9366ed5c0fbdabd15f6d3b79c9 426565a5960364bb1ac1d1cb185363318872cdf83520
Improved range checking. Fixed bugs in dwgetopt(). Added omitted tags and attributes lists in dwarfdump to make error checking more accurate. Where pc checks make no sense in a line table (meaning on DW_TAG_type_unit and children) such dwarfdump checks are now omitted.
sha512sum (remove spaces following colon): 983c0bb5d70f59e95b8b9de9cda74d714795526220 ac944b6e058554b1f1e831063ae5524d6a2de557e1 fe829ccbf17b1ab71195fa5589504ead3d94396ab0a4
The omission of an include of stdarg.h in the 20150112 release meant that standard-conforming compilers could get an error compiling dwarfdump due to va_list not being defined, though Ubuntu and Freebsd compiles did not see such an error. Now dwarfdump does the include. The dwarfdump2 source has been removed as of this tar.gz file. Dwarfdump2 is no longer needed.
sha512sum (remove spaces following colon): abcc465f3fcc369143cb34976ad2874b5a9d2a6b4f732be2 b83b0e7620799747778947aed0c10872a5fad73443cfa986 48bdc50a3b17c98292a6439b54d60222
New checking options added to dwarfdump. Dwarfdump2 no longer updated or compiled. Improvements in internal checking to handle badly formed Elf and Dwarf files. Better recovery from malloc failure.
sha512sum (remove spaces following colon): 23fe3dec516a90c2cedb851971f2fd902e056bf4471a32 a2237881354d71af866fcdbc9d3ff55d4e83b75fdafd9f 7790bf90165506e9285259c67f342c4eccba
Source incompatibility for users of producer code: dwarf_producer_init() now has a new interface and a sensible way to select the output ABI/ISA for relocation numbers. Part of preparation to emit DWARF3,4,5. The reader code now handles DebugFission, called Split Dwarf objects in the draft DWARF5 standard.
sha512sum (remove spaces following colon): ccf8180b69cdb47902564dda1fca52d15c10239ce6bb8c c9f5af5a67d37f888811572e314414372bfbc2b640c1fff 7cf87542f782f8390f733d884a24d9a16fb
Libdwarf now follows the DWARF2 standard properly in reading DW_FORM_ref_addr. The original DWARF2 standard is on dwarfstd.org so it's gone from the libdwarf source.
sha512sum (remove spaces following colon): 7ecd27b40418fd98bb24ee59b9779efe30dca26384b4a36f5 a1b0a99805f4d8ff281b2e3d4470fb8e8da28045c34bf6b53 fdf85e9dfc5fa76c0eac8462ae8467
Fixed a bug in dwarfdump Makefile.in so parallel make works reliably. Removed remaining trailing-whitespace. Moved tsearch testcases over to the libdwarf regressiontests repository as the tests seemed too large to keep in the source.
sha512sum (remove spaces following colon): d8ba3eeaf36d98a1ee26397208fff1658a2b7a41c 25d3742a81617c74d6359aeff08bb6221f99b9937 9030575578e11cd66be1d906a5832a6edd362229ce2e7e
Radically simplifies libdwarf allocation code. Adds GNU-specific DW_FORM codes so recent gcc objects can be read usefully. Adds tseach implementations (see the tsearch directory). Removes trailing whitespace (it appeared all over). Adds new functions to libdwarf for new DWARF reference types. The tsearch test cases have bloated this source release, but in future those test cases won't be in the tar file (instead they are in the libdwarf regressiontests repository on sourceforge).
sha512sum (remove spaces following colon): d41ebe4e7b76ad91f93b17e33da878fb0a35d7a35 32d641108b217bf93bcd9f10c1d52f0dd5f2ece08 3152d8dafe637ae343633f894122cbb77825b7a3350ed2
Added AARCH64 relocation support. Fixed some simple compiler warnings.
sha512sum (remove spaces following colon): f9d25cfd6c6b15bebf6cd63c7014ecf4123798fce637c 0da103008758d6a9d5705c3797216a8d1ab3e210c4235 f199ab19d7ed0bf6c3582f49eacba1629c5cc0
Now with a simplified build (see README) and with checks for most compiler errors in producing DW_AT_sibling attributes. Verified on Ubuntu 12.10 and FreeBSD 9.1.
sha512sum (remove spaces following colon): 1cb272f80745a789d592d57e6a64b1b4ec6e1b646653 da2f19c2e2d803b8e90b52f5c69597360bf1703b1c25 7844c53c3f43a80cd9f45a2adb314ad2511c19e9
A mistake in handling DW_OP_GNU_const_type could lead to a libdwarf coredump at times, and even when it appeared to work the value printed was wrong. Thanks to Tom Hughes and Andrew Bernat for pointing out the DW_OP_GNU_const_type problem. This release has libdwarf interfaces compatible with releases before 20130125. Compared to 20130125 this restores the libdwarf.h interface Dwarf_Loc structure and changes the way the lr_number2 member is used for the new location expression operator DW_OP_GNU_const_type.
sha512sum (remove spaces following colon): c9911ce0b9725400ec1a70e809e185b122095c534f05687 ea0be16a0e9bfe3b8128e353834c43b5fe80e1b149b11 f113fcce53c57bad0a12b3b091dd0e31d043
For information on previous releases, see the older release list
Every release of libdwarf/dwarfdump is tested with many options and option-combinations. There is no real reason libdwarf/dwarfdump users should need to redo this testing work. However, if you do wish to try running the regression tests, you will find them in git in sourceforge via anonymous access "git clone git://git.code.sf.net/p/libdwarf/regressiontests". See the file README.txt in the base directory of the tests for an overview of the test process. We do not provide a tar file of the regression tests. The regression tests are updated with every libdwarf release.
Additional tests (simple object files or shared-objects or executables) are accepted here should you wish to submit such, but the goal is to add tests that represent previously-untested aspects of DWARF/libdwarf, not to duplicate existing tests. Smallish object files are preferred. No source need be provided. The submitter has to be sure, and state, that releasing the objects here is appropriate.
Nothing in libdwarf looks at very many sections of an Elf file: the other section contents can be zeroed out without affecting the object use for testing libdwarf (see the directory named 'zero' in the testing distribution for a convenient byte-zeroing helper application in C++ source). Sections like .text and .data (and closely related sections) are of no value for testing libdwarf. It may be that otherwise-proprietary objects can be released for use in this test suite once the instruction and data sections are zeroed out.