Darling: Implement APIs to get GDB/LLDB working under Darling

[LubosD]

This has super-high priority, because right now we cannot properly debug Darling or any software running underneath it.

APIs to be implemented

ptrace() operations

• [x] PT_THUPDATE - signal delivery to inferior (after it is received over an exception port)
• [x] PT_PTRACE_ME
• [x] PT_SIGEXC - redirect my own signals to an exception port, only used in ptrace_me mode
• [x] PT_DETACH
• [x] PT_KILL
• [x] PT_ATTACHEXC - attach and do PT_SIGEXC on child

posix_spawn() flags

• [x] _POSIX_SPAWN_DISABLE_ASLR
  
  
  
  • Implement with personality(ADDR_NO_RANDOMIZE)
  
  
• [x] POSIX_SPAWN_START_SUSPENDED
  
  
  
  • This should probably be implemented as a flag to mldr to suspend itself before it jumps into dyld code
  
  

Mach APIs

• [x] task_info(TASK_DYLD_INFO)
  
  
  
  • Information for this needs to be delivered to LKM by mldr - it has to find a section named __DATA,__all_image_info. How exactly TBD.
  
  
• [x] thread_get_state(GDB_X86_THREAD_STATE and GDB_X86_FLOAT_STATE)
• [x] thread_set_state(ditto)
• [x] thread_get_state(X86_DEBUG_STATE) + set
  
  
  
  • Equivalent to PTRACE_POKEUSER for u_debugreg registers - sets HW watchpoints
  
  
• [x] task_threads()
• [x] task_resume() + task_suspend()
• [x] thread_resume() + thread_suspend()
• [x] task_get_exception_ports + set
• [x] mach_vm_region(), mach_vm_region_recurse()
  
  
  
  • task_t->map needs to be given a reasonable value for this
  
  
• [x] task_for_pid()
• [x] pid_for_task() // only used by FSF's gdb, Apple's gdb does things differently
• [x] vm_read() + vm_write() (implement using access_process_vm())
• [x] mach_vm_protect()

LLDB

• task_info(flavor=5) (also used by ps)
• [x] sysctlbyname("hw.cpu64bit_capable")
• [x] task_threads() seems to return duplicates
• [x] sysctl({ CTL_KERN, KERN_OSVERSION })
• [x] thread_info(THREAD_BASIC_INFO) that returns correct run_state
• [x] sysctl({CTL_KERN, KERN_PROC, KERN_PROC_PID, nnn}) that returns kp_proc.p_flag & P_LP64 for 64-bit processes
• [x] Implement sysctl(sysctlnametomib("sysctl.proc_cputype") + ".PID")
• [x] Implement task_info(TASK_VM_INFO) (needs page size)
• [x] Implement proc_pidinfo(PROC_PIDTHREADINFO) (uses pth_name)

Implementation notes

• The biggest complication is signal delivery to GDB over a Mach exception port. My current idea is to spawn a kthread upon PT_SIGEXC or PT_ATTACHEXC and trace the target process in ptrace-like manner from there.
• I think we should build Apple's GDB as part of Darling rather than point users to a repository somewhere (e.g. to avoid confusion between gdb and gdb-apple).
  
  
  
  • There are two different flavors of GDB for macOS: FSF and Apple. FSF's is a lot less demanding for needed APIs, but attaching to running processes is not fully implemented (it doesn't work).
  
  
• We will look into building and running LLDB once GDB works.

[LubosD]

This is how adding a breakpoint currently fails:

2785 +21.027975 sec [0009/1303]: ::read ( 15, 0x7ffef1a987c0, 1024 ) => 21 err = 0x00000000
2786 +0.011023 sec [0009/1303]: read: $Z0,7ffff7ed98f0,1#e9
2787 +0.008166 sec [0009/1303]: getpkt: $Z0,7ffff7ed98f0,1#e9
2788 +0.020084 sec [0009/0307]: RNBRunLoopInferiorExecuting ctx.Events().WaitForSetEvents(0x000000a5) => 0x00000020 (read_packet_available )
2789 +0.004269 sec [0009/0307]: HandleReceivedPacket ("Z0,7ffff7ed98f0,1");
2790 +0.004230 sec [0009/0307]: MachProcess::CreateBreakpoint ( addr = 0x7ffff7ed98f0, length = 1, hardware = 0)
2791 +0.004297 sec [0009/0307]: MachProcess::EnableBreakpoint ( addr = 0x7ffff7ed98f0 )
2792 +0.017011 sec [0009/0307]: ::mach_vm_read ( task = 0x0a03, addr = 0x7ffff7ed98f0, size = 1, data => 0x7ffef4e9b000, dataCnt => 1 ) err = 0x00000000
2793 +0.005927 sec [0009/0307]: MachTask::ReadMemory ( addr = 0x7ffff7ed98f0, size = 1, buf = 0x7fd43ec0a1e0) => 1 bytes read
0x7ffff7ed98f0: 55
2794 +0.013687 sec [0009/0307]: ::mach_vm_region_recurse ( task = 0x0a03, address => 0x7ffff7e98000, size => 1036288, nesting_depth => 0, info => 0x7fffffffa8e4, infoCnt => 12) addr = 0x7ffff7ed98f0  err = 0x00000000
2795 +0.004229 sec [0009/0307]: info = { prot = 4, max_prot = 7, inheritance = 0x00000000, offset = 0x00000000, user_tag = 0x00000000, ref_count = 0, shadow_depth = 0, ext_pager = 0, share_mode = 0, is_submap = 0, behavior = 0, object_id = 0x00000000, user_wired_count = 0x0000 }
2796 +0.005872 sec [0009/0307]: error: ::mach_vm_protect ( task = 0x0a03, addr = 0x7ffff7ed98f0, size = 1, set_max = 0, prot = 3 ) err = (os/kern) failure (0x00000005)
2797 +0.006209 sec [0009/0307]: error: ::mach_vm_protect ( task = 0x0a03, addr = 0x7ffff7ed98f0, size = 1, set_max = 0, prot = 19 ) err = (os/kern) failure (0x00000005)
2798 +0.004219 sec [0009/0307]: Failed to set read/write protections on region for address: [0x7ffff7ed98f0-0x7ffff7ed98f1)
2799 +0.009162 sec [0009/0307]: MachTask::WriteMemory ( addr = 0x7ffff7ed98f0, size = 1, buf = 0x7ffff7b48ae8) => 0 bytes written
error: MachProcess::EnableBreakpoint ( addr = 0x7ffff7ed98f0 ): unable to write breakpoint opcode to memory.
2800 +0.006360 sec [0009/0307]: 21181246 RNBRemote::SendPacket (E09) called
2801 +0.006468 sec [0009/0307]: ::write ( socket = 15, buffer = 0x7fffffffa6b1, length = 7) => 7 err = 0x00000000
2802 +0.010155 sec [0009/0307]: putpkt: $E09#00
2803 +0.000018 sec [0009/0307]: sent: $E09#00
2804 +0.004028 sec [0009/0307]: RNBRunLoopInferiorExecuting ctx.Events().WaitForSetEvents(0x000000a5) ...

[bugaevc]

:tada:

[bugaevc]

For the reference, it's been 505 days since using-machos-experiment (remember that name?) was merged, but we've survived :smile: and even made some amazing progress without a debugger. It's only going to be much much more productive from here.