Solaris: Tracing your application

Every Solaris system administrator already know the truss utility, which allow you to trace the system calls as your application is running.

I recently discovered another tool which I found to be really helpful when you want to know what an application is really doing: apptrace.

Apptrace is tracing the _function_ calls instead of the _system_ calls: You can then see which library and function your program is using!

This comes very handy in your daily sysadmin life, when you have to port a software or simply check what function of which library that software is using!

Here is a simple example of this utility against the “hostid” binary:

# apptrace hostid
-> hostid -> libc.so.1:int atexit(int (*)() = 0xeef93ee4)
<- hostid -> libc.so.1:atexit()
-> hostid -> libc.so.1:int atexit(int (*)() = 0x8050bac)
<- hostid -> libc.so.1:atexit()
-> hostid -> libc.so.1:void __fpstart(void)
<- hostid -> libc.so.1:__fpstart() = 0xeebfd484
-> hostid -> libc.so.1:long gethostid(void)
<- hostid -> libc.so.1:gethostid() = 0xcfabcc
-> hostid -> libc.so.1:int printf(const char * = 0x8050bcc "%08lx
", void * = 0xcfabcc, ...)
00cfabcc
<- hostid -> libc.so.1:printf() = 0x9
-> hostid -> libc.so.1:exit(0x0, 0xeec00088, 0xf10c8c04) ** NR

Have you ever wondered “How does that application is getting that system information?”, apptrace is there to help answering.
For the purpose of this post, I was simply wondering how zpool list was using libzfs.so to get the list of existing system pools, here is below the output of “apptrace zpool list”. I’ve removed useless output for clarity:


-> zpool -> libzfs.so.1:libzfs_handle_t * libzfs_init(void)
<- zpool -> libzfs.so.1:libzfs_init() = 0x851b648
-> zpool -> libzfs.so.1:void libzfs_print_on_error(libzfs_handle_t * = 0x851b648, boolean_t = 0x1)
<- zpool -> libzfs.so.1:libzfs_print_on_error() = 0x851b648
...[SNIPPED]...
-> zpool -> libzfs.so.1:int zpool_iter(libzfs_handle_t * = 0x851b648, zpool_iter_f = 0x805d608, void * = 0x8516ec8)
...[SNIPPED]...
-> zpool -> libzfs.so.1:const char * zpool_get_name(zpool_handle_t * = 0x8536e08)
<- zpool -> libzfs.so.1:zpool_get_name() = 0x8536e10
-> zpool -> libzfs.so.1:const char * zpool_get_name(zpool_handle_t * = 0x8536cc8)
<- zpool -> libzfs.so.1:zpool_get_name() = 0x8536cd0
...[SNIPPED]...

We can build the following .c code based on apptrace’s output:

#include <libzfs.h>
#include <stdio.h>
#include <stdlib.h>

extern int zp_iter(zpool_handle_t *, void *);

int main(void) {
  libzfs_handle_t *zh = libzfs_init();
  libzfs_print_on_error(zh, 1);
  zpool_iter(zh, zp_iter, NULL);
  exit(0);
}

int zp_iter(zpool_handle_t *z, void *a) {
  printf("name:\t%s\n", zpool_get_name(z));
  return 0; 
}

Compile and try:


$ gcc zl.c -lzfs -lcryptoutil -m64 -o zl
$ ./zl
name: rpool
name: tank

And voila! If you find other useful usage for apptrace, please leave a comment below 😉

This entry was posted in Solaris and tagged , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *