gdb debugging tips¶
Displaying Julia variables¶
Within gdb
, any jl_value_t*
object obj
can be displayed using
(gdb)calljl_(obj)
The object will be displayed in the julia
session, not in the gdb session.
This is a useful way to discover the types and values of objects being
manipulated by Julia’s C code.
Similarly, if you’re debugging some of Julia’s internals (e.g.,
inference.jl
), you can print obj
using
ccall(:jl_,Void,(Any,),obj)
This is a good way to circumvent problems that arise from the order in which julia’s output streams are initialized.
Julia’s flisp interpreter uses value_t*
objects; these can be displayed
with callfl_print(ios_stdout,obj)
.
Useful Julia variables for Inspecting¶
While the addresses of many variables, like singletons, can be be useful to print for many failures, there are a number of additional variables (see julia.h for a complete list) that are even more useful.
- (when in
jl_apply_generic
)f->linfo
andjl_uncompress_ast(f->linfo,f->linfo->ast)
:: for figuring out a bit about the call-stack jl_lineno
andjl_filename
:: for figuring out what line in a test to go start debugging from (or figure out how far into a file has been parsed)$1
:: not really a variable, but still a useful shorthand for referring to the result of the last gdb command (such asprint
)jl_options
:: sometimes useful, since it lists all of the command line options that were successfully parsedjl_uv_stderr
:: because who doesn’t like to be able to interact with stdio
Useful Julia functions for Inspecting those variables¶
gdblookup($rip)
:: For looking up the current function and line. (use$eip
on i686 platforms)jlbacktrace()
:: For dumping the current Julia backtrace stack to stderr. Only usable afterrecord_backtrace()
has been called.jl_dump_llvm_value(Value*)
:: For invokingValue->dump()
in gdb, where it doesn’t work natively. For example,f->linfo->functionObject
,f->linfo->specFunctionObject
, andto_function(f->linfo)
.Type->dump()
:: only works in lldb. Note: add something like;1
to prevent lldb from printing its prompt over the outputjl_eval_string("expr")
:: for invoking side-effects to modify the current state or to lookup symbolsjl_typeof(jl_value_t*)
:: for extracting the type tag of a Julia value (in gdb, callmacrodefinejl_typeofjl_typeof
first, or pick something short likety
for the first arg to define a shorthand)
Inserting breakpoints for inspection from gdb¶
In your gdb
session, set a breakpoint in jl_breakpoint
like so:
(gdb)breakjl_breakpoint
Then within your Julia code, insert a call to jl_breakpoint
by adding
ccall(:jl_breakpoint,Void,(Any,),obj)
where obj
can be any variable or tuple you want to be accessible in the breakpoint.
It’s particularly helpful to back up to the jl_apply
frame, from which you can display the arguments to a function using, e.g.,
(gdb)calljl_(args[0])
Another useful frame is to_function(jl_lambda_info_t*li,boolcstyle)
. The jl_lambda_info_t*
argument is a struct with a reference to the final AST sent into the compiler. However, the AST at this point will usually be compressed; to view the AST, call jl_uncompress_ast
and then pass the result to jl_
:
#2 0x00007ffff7928bf7 in to_function (li=0x2812060, cstyle=false) at codegen.cpp:584584abort();(gdb)pjl_(jl_uncompress_ast(li,li.ast))
Inserting breakpoints upon certain conditions¶
Loading a particular file¶
Let’s say the file is sysimg.jl
:
(gdb)breakjl_loadifstrcmp(fname,"sysimg.jl")==0
Calling a particular method¶
(gdb)breakjl_apply_genericifstrcmp(F->name->name,"method_to_break")==0
Since this function is used for every call, you will make everything 1000x slower if you do this.
Debugging during Julia’s build process (bootstrap)¶
Errors that occur during make
need special handling. Julia is built in two stages, constructing
sys0
and sys.ji
. To see what commands are running at the time of failure, use makeVERBOSE=1
.
At the time of this writing, you can debug build errors during the sys0
phase from the base
directory using:
julia/base$gdb--args../usr/bin/julia-debug-Cnative--build../usr/lib/julia/sys0sysimg.jl
You might need to delete all the files in usr/lib/julia/
to get this to work.
You can debug the sys.ji
phase using:
julia/base$gdb--args../usr/bin/julia-debug-Cnative--build../usr/lib/julia/sys-J../usr/lib/julia/sys0.jisysimg.jl
By default, any errors will cause Julia to exit, even under gdb. To catch an error “in the act”, set a breakpoint
in jl_error
(there are several other useful spots, for specific kinds of failures, including: jl_too_few_args
,
jl_too_many_args
, and jl_throw
).
Once an error is caught, a useful technique is to walk up the stack and examine the function by inspecting
the related call to jl_apply
. To take a real-world example:
Breakpoint1,jl_throw(e=0x7ffdf42de400)attask.c:802802{(gdb)pjl_(e)ErrorException("auto_unbox: unable to determine argument type")$2=void(gdb)bt10#0 jl_throw (e=0x7ffdf42de400) at task.c:802#1 0x00007ffff65412fe in jl_error (str=0x7ffde56be000 <_j_str267> "auto_unbox:unabletodetermineargumenttype") at builtins.c:39#2 0x00007ffde56bd01a in julia_convert_16886 ()#3 0x00007ffff6541154 in jl_apply (f=0x7ffdf367f630, args=0x7fffffffc2b0, nargs=2) at julia.h:1281...
The most recent jl_apply
is at frame #3, so we can go back there and look at the AST for the function
julia_convert_16886
. This is the uniqued name for some method of convert
. f
in this frame is a
jl_function_t*
, so we can look at the type signature, if any, from the specTypes
field:
(gdb)f3#3 0x00007ffff6541154 in jl_apply (f=0x7ffdf367f630, args=0x7fffffffc2b0, nargs=2) at julia.h:12811281returnf->fptr((jl_value_t*)f,args,nargs);(gdb)pf->linfo->specTypes$4=(jl_tupletype_t*)0x7ffdf39b1030(gdb)pjl_(f->linfo->specTypes)Tuple{Type{Float32},Float64}# <-- type signature for julia_convert_16886
Then, we can look at the AST for this function:
(gdb)pjl_(jl_uncompress_ast(f->linfo,f->linfo->ast))Expr(:lambda,Array{Any,1}[:#s29, :x], Array{Any, 1}[Array{Any, 1}[], Array{Any, 1}[Array{Any, 1}[:#s29, :Any, 0], Array{Any, 1}[:x, :Any, 0]], Array{Any, 1}[], 0], Expr(:body,Expr(:line,90,:float.jl)::Any,Expr(:return,Expr(:call,:box,:Float32,Expr(:call,:fptrunc,:Float32,:x)::Any)::Any)::Any)::Any)::Any
Finally, and perhaps most usefully, we can force the function to be recompiled in order to step through the
codegen process. To do this, clear the cached functionObject
from the jl_lamdbda_info_t*
:
(gdb)pf->linfo->functionObject$8=(void*)0x1289d070(gdb)setf->linfo->functionObject=NULL
Then, set a breakpoint somewhere useful (e.g. emit_function
, emit_expr
, emit_call
, etc.), and run
codegen:
(gdb)pjl_compile(f)...# your breakpoint here
Mozilla’s Record and Replay Framework (rr)¶
Julia now works out of the box with rr, the lightweight recording and deterministic debugging framework from Mozilla. This allows you to replay the trace of an execution deterministically. The replayed execution’s address spaces, register contents, syscall data etc are exactly the same in every run.
A recent version of rr
(3.1.0 or higher) is required.