November 01, 2018 posted by Kamil Rytarowski

I have presented the state of NetBSD sanitizers during two conferences in the San Francisco Bay Area: Google Summer of Code Mentor Summit (Mountain View) and MeetBSDCa (Santa Clara, Intel Campus SC12). I've also made progress in upstreaming of our local patches to LLVM sanitizers and introducing generic NetBSD enhancements there.

The Bay Area

I took part (together with William Coldwell - cryo@) in the GSOC Mentor Summit as a NetBSD delegate. I've presented during the event a presentation with a quick introduction to NetBSD, track history of GSoC involvement and the LLVM Sanitizers work with a stress of sanitizers.

The MeetBSDCa conference is a continuation of the MeetBSD conferences from Poland. I took part there as a speaker talking about Userland Sanitizers in NetBSD. I've also presented the state of virtualization in NetBSD during a discussion panel. Additionally I've prepared a lightning talk about NetBSD Kernel sanitizers and quick status update from The NetBSD Foundation. Unfortunately the schedule was last minute changed (introduction of BSD history talk in the slot of lightning presentations) and the closing ceremony had different proceeding. Nonetheless, I'm sharing these additional quick presentations.

During the former conference it was a great opportunity to meet people from other Open Source projects, a lot of them are in interaction with NetBSD developers during the process of upstreaming local support patches. During the latter conference it was an opportunity to meet BSD people and people closer to hardware companies.

Upstreaming process of LLVM Sanitizers

I've upstreamed a number of patches to the LLVM source tree. The changes can be summarized as:

Further reworking the code and approaching the state of installing of sysctl*() inteceptors.

Fixing or marking failing or hanging tests in the sanitizer test-suites.

Adapting definitions of syscalls and ioctl(2) operations for NetBSD 8.99.25.

Detailed list of commits merged with the upstream LLVM compiler-rt repository:

Update ioctl(2) operations for NetBSD 8.99.25

Update generate_netbsd_ioctls.awk for NetBSD 8.99.25

Diable test suppressions-library for NetBSD/i386

Disable BufferOverflowAfterManyFrees for NetBSD

Mark breaking asan tests on NetBSD

Switch getline_nohang from XFAIL to UNSUPPORTED for NetBSD

Mark vptr-non-unique-typeinfo as a broken test for NetBSD/i386

Mark breaking sanitizer_common tests on NetBSD

Handle NetBSD alias for pthread_sigmask

Cast the return value of _Unwind_GetIP() to uptr

Mark interception_failure_test with XFAIL for NetBSD

Disable ASan test asan_and_llvm_coverage_test for NetBSD

Adapt ASan test heavy_uar_test for NetBSD

Mark breaking TSan tests on NetBSD with XFAIL

Cleanup includes in sanitizer_platform_limits_netbsd.cc

Regenerate syscall hooks for NetBSD 8.99.25

Update generate_netbsd_syscalls.awk for NetBSD 8.99.25

Handle pthread_sigmask in DemangleFunctionName()

Drop now hidden ioctl(2) operations for NetBSD

Handle NetBSD symbol mangling for tzset

Handle NetBSD symbol mangling for nanosleep and vfork

Mark test/tsan/getline_nohang as XFAIL for NetBSD

Disable the GNU strerror_r TSan test for NetBSD

Mark test/tsan/ignore_lib5 as unsupported for NetBSD

Mark intercept-rethrow-exception.cc as XFAIL on NetBSD

Disable failing tests lib/asan/tests on NetBSD

Skip unsupported MSan tests on NetBSD

Mark 4 MSan tests as XFAIL for NetBSD

Mark MSan fork test as UNSUPPORTED on NetBSD

Reflect the current reality and disable lsan tests on NetBSD

Use PTHREAD_STACK_MIN conditionally in a test

Remove remnant code of using indirect syscall on NetBSD

Don't harcode -ldl test/sanitizer_common/TestCases

Disable TestCases/pthread_mutexattr_get on NetBSD

Fix Posix/devname_r for NetBSD

Unwind local macro DEFINE_INTERNAL()

Introduce internal_sysctlbyname in place of sysctlbyname

Frequently asked question

People keep asking me about rationale of some design decisions in sanitizers and whether something could be done better. One of such places is to reuse more of libc internals and to not keep bypassing it whenever possible. The motivation to keep redoing the same work for NetBSD is to keep close to the upstream (mostly Linux & Android) source code with a minimal delta between NetBSD vs others support. Doing some operations in a more convenient way is tempting, but it's a danger that someone will need to keep maintaining a larger diff, especially since upstream developers will focus on their own OSes rather than trying to adapt their patches for potentially alternative approaches.

Plan for the next milestone

I will keep upstreaming local LLVM patches (almost 2500LOC to go!).

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

http://netbsd.org/donations/#how-to-donate [1 comment]