September 03, 2019 posted by Kamil Rytarowski

As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay.

I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite.

NetBSD distribution changes

I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@).

The following changes were committed to the sources:

ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers

966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build

8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data

fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data

40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER

b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER

c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER

030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER

fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER

5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data

5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot

d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk

2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER

ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7

7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7

d8671fba7a78 Set NODEBUG for LLVM sanitizers

242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER

5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers

e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers

231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers

8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers

81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers

a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c

7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer

921ddc9bc97c Set NOSANITIZER in i386 ramdisk image

64361771c78d Enhance MKSANITIZER support

3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests

c27f4619d513 Avoids signedness bit shift in db_get_value()

680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no)

4ecfbbba2f2a Rework the LLVM compiler_rt build rules

748813da5547 Correct the build rules of LLVM sanitizers

20e223156dee Enhance the support of LLVM sanitizers

0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files

Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0.

As a demo, I have crafted a writing on combining RUMPKERNEL, MKSANITIZER with the honggfuzz fuzzer: Rumpkernel assisted fuzzing of the NetBSD file system kernel code in userland.

GDB

I've merged NetBSD distribution downstream GDB patches into my local GDB tree and executed the regression tests (check-gdb):

[...] Test run by kamil on Mon Sep 2 12:36:03 2019 Native configuration is x86_64-unknown-netbsd9.99 === gdb tests === Schedule of variations: unix [...] === gdb Summary === # of expected passes 54591 # of unexpected failures 3267 # of expected failures 35 # of unknown successes 3 # of known failures 59 # of unresolved testcases 29 # of untested testcases 141 # of unsupported tests 399

Full log is here.

This means that there are a lot of more tests and known failures than in 2017-09-05:

$ uname -a NetBSD chieftec 8.99.2 NetBSD 8.99.2 (GENERIC) #0: Sat Sep 2 22:55:29 CEST 2017 root@chieftec:/public/netbsd-root/sys/arch/amd64/compile/GENERIC amd64 Test run by kamil on Tue Sep 5 17:06:28 2017 Native configuration is x86_64--netbsd === gdb tests === Schedule of variations: unix [...] === gdb Summary === # of expected passes 16453 # of unexpected failures 483 # of expected failures 9 # of known failures 28 # of unresolved testcases 17 # of untested testcases 41 # of unsupported tests 25

There are actually some regressions and a set of tests that fails probably due to environment differences like lack of gfortran at hand.

Full log is here

GSoC Mentoring

The Google Summer of Code programme reached the end. My mentees wrote successfully their final reports:

I'm also mentoring the AFL+KCOV work by Maciej Grochowski. Maciej will visit EuroBSDCon-2019 and speak about his work.

Add methods for setting and getting the thread name

I've reached out to the people from standards bodies and I'm working on defining the standard approach for setting and getting the thread name. I have received a proper ID of my proposal and I'm now supposted to submit the text in either PDF or HTML format.

This change will allow to manage the thread name with an uniform interface on all comforming platforms.

Plan for the next milestone

Keep enhancing GDB support. Keep detecting ptrace(2) bugs and addressing them.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

http://netbsd.org/donations/#how-to-donate [1 comment]