



DRAFT 0 - 10/11/18,

DRAFT 1 - 9/16/19, 7:19 PM, I'm still working on this, but it's already useful and I'd like some feedback - so I decided to share it early. Contemporary C tooling for making higher quality C, faster or more safely. DRAFT 2 - 10/1/19, mostly additions to static analysis tools.

DRAFT 3 - 10/4/19, updates on build systems, package management, and complexity analysis.

DRAFT 4 - 10/6/19, run time dynamic verification and instrumentation, sanitizers (asan/ubsan/etc), performance tools, static analyzers.

DRAFT 5 - C interpreter(s).

DRAFT 6 - 11/6/19, mention TermDebug vim, more windows debugging tools, C drawing for intro.







In 2001 or so people started using the phrase "Modern C++". So now that it's 2019, I guess we're in the post modern era? Anyway, this isn't a post about C++ code, but some of this information applies there too.





No logo, but it's used everywhere.



Welcome to the post modern era. Some of the C++ people have pulled off one of the cleverest and sneakiest tricks ever. They required 'modern'



So, we have some pretty new language features in C with C11. But what about tooling?



Tools and protection for our feet. C, whilst a work horse being used in everything from toasters, trains, phones, web browsers, ... (everything basically) - is also an excellent tool for shooting yourself in the foot.

Noun footgun (plural footguns)

( informal , humorous , derogatory ) Any feature whose addition to a product results in the user shooting themselves in the foot. C.

Tools like linters, test coverage checkers, static analyzers, memory checkers, documentation generators, thread checkers, continuous integration, nice error messages, ... and such help protect our feet.



How do we do continuous delivery with a language that lets us do the most low level footgunie things ever? On a dozen CPU architectures, 32 bit, 64bit, little endian, big endian, 64 bit with 32bit pointers (wat?!?), with multiple compilers, on a dozen different OS, with dozens of different versions of your dependencies?



Surely there won't be enough time to do releases, and have time left to eat my vegan shaved ice desert after lunch?







Debuggers Give me 15 minutes, and I'll change your mind about GDB. --

https://www.youtube.com/watch?v=PorfLSr3DDI Firstly, did you know gdb had a curses based 'GUI' which works in a terminal? It's a quite a bit easier to use than the command line text interface. It's called TUI. It's built in, and uses emacs key bindings.



But what if you are used to VIM key bindings?



https://cgdb.github.io/

VIM has integrated gdb debugging with



Also, there's a fairly easy to use web based front end for GDB called

(https://www.gdbgui.com/). For those who don't use an IDE with debugging support built in (such as Visual studio by Microsoft or XCode by Apple).











Reverse debugger Normally a program runs forwards. But what about when you are debugging and you want to run the program backwards?



Set breakpoints and data watchpoints and quickly reverse-execute to where they were hit.

How do you tame non determinism to allow a program to run the same way it did when it crashed? In C and with threads some times it's really hard to reproduce problems.



rr helps with this. It's actual magic.



https://rr-project.org/













LLDB - the LLVM debugger. Apart from the ever improving





IDE debugging Visual Studio by Microsoft, and XCode by Apple are the two heavy weights here.



The free Visual Studio Code also supports debugging with GDB. https://code.visualstudio.com/docs/languages/cpp



Sublime is another popular editor, and there is good GDB integration for it too in the



Windows debugging Suppose you want to do post mortem debugging? With and you can. procdump and WinDbg How to take a procdump " ( https://blogs.msdn.microsoft.com/webdav_101/2018/03/20/how-to-take-a-procdump/ ) is a nice tutorial on how to use procdump.

Launch a process and then monitor it for exceptions:

C:\>procdump -e 1 -f "" -x c:\dumps consume.exe This makes some process dumps when it crashes, which you can then open with





Portable building, and package management C doesn't have a package manager... or does it?



Ever since Debian dpkg, Redhat rpm, and Perl started doing package management in the early 90s people world wide have been able to share pieces of software more easily. Following those systems, many other systems like Ruby gems, JavaScript npm, and Pythons cheese shop came into being. Allowing many to share code easily.



But what about C? How can we define dependencies on different 'packages' or libraries and have them compile on different platforms?



How do we build with Microsofts compiler, with gcc, with clang, or Intels C compiler? How do we build on Mac, on Windows, on Ubuntu, on Arch linux? Sometimes we want to use an Integrated Development Environment (IDE) because they provide lots of nice tools. But maybe also three IDEs (XCode, Microsoft Visual C, CLion, ...) depending on platform. But we don't want to have to keep several IDE project files up to date. But we also want to integrate nicely with different OS packagers like Debian, FreeBSD. We want people to be able to use apt-get install for their dependencies if they want. We also want to cross compile code on our beefy workstations to work on microcontrollers or slower low memory systems (like earlier RaspberryPi systems).







The Meson Build System. In 2001 or so people started using the phrase "Modern C++". So now that it's 2019, I guess we're in the post modern era? Anyway, this isn't a post about C++ code, but some of this information applies there too.Some of the C++ people have pulled off one of the cleverest and sneakiest tricks ever. They required 'modern' C99 and C11 features in 'recent' C++ standards. Microsoft has famously still clung onto some 80s version of C with their compiler for the longest time. So it's been a decade of hacks for people writing portable code in C. For a while I thought we'd be stuck in the 80s with C89 forever. However, now that some C99 and C11 features are more widely available in the Microsoft compiler, we can use these features in highly portable code (but forget about C17/C18 ISO/IEC 9899:2018/C2X stuff!!). Check out the "New" Features in C talk, and the Modern C book for more details.So, we have some pretty new language features in C with C11. But what about tooling?C, whilst a work horse being used in everything from toasters, trains, phones, web browsers, ... (everything basically) - is also an excellent tool for shooting yourself in the foot.Tools like linters, test coverage checkers, static analyzers, memory checkers, documentation generators, thread checkers, continuous integration, nice error messages, ... and such help protect our feet.How do we do continuous delivery with a language that lets us do the most low level footgunie things ever? On a dozen CPU architectures, 32 bit, 64bit, little endian, big endian, 64 bit with 32bit pointers (wat?!?), with multiple compilers, on a dozen different OS, with dozens of different versions of your dependencies?Surely there won't be enough time to do releases, and have time left to eat my vegan shaved ice desert after lunch?Firstly, did you know gdb had a curses based 'GUI' which works in a terminal? It's a quite a bit easier to use than the command line text interface. It's called TUI. It's built in, and uses emacs key bindings.But what if you are used to VIM key bindings? cgdb to the rescue.VIM has integrated gdb debugging with TermDebug since version 8.1.Also, there's a fairly easy to use web based front end for GDB called gdbgui (https://www.gdbgui.com/). For those who don't use an IDE with debugging support built in (such as Visual studio by Microsoft or XCode by Apple).Normally a program runs forwards. But what about when you are debugging and you want to run the program backwards?How do you tame non determinism to allow a program to run the same way it did when it crashed? In C and with threads some times it's really hard to reproduce problems.rr helps with this. It's actual magic.Apart from the ever improving gdb , there is a new debugger from the LLVM people - lldb ( https://lldb.llvm.org/ ).Visual Studio by Microsoft, and XCode by Apple are the two heavy weights here.The free Visual Studio Code also supports debugging with GDB. https://code.visualstudio.com/docs/languages/cppSublime is another popular editor, and there is good GDB integration for it too in the SublimeGDB package (https://packagecontrol.io/packages/SublimeGDB).This makes some process dumps when it crashes, which you can then open with WinDbg (https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugging-using-windbg-preview).C doesn't have a package manager... or does it?Ever since Debian dpkg, Redhat rpm, and Perl started doing package management in the early 90s people world wide have been able to share pieces of software more easily. Following those systems, many other systems like Ruby gems, JavaScript npm, and Pythons cheese shop came into being. Allowing many to share code easily.But what about C? How can we define dependencies on different 'packages' or libraries and have them compile on different platforms?How do we build with Microsofts compiler, with gcc, with clang, or Intels C compiler? How do we build on Mac, on Windows, on Ubuntu, on Arch linux? Sometimes we want to use an Integrated Development Environment (IDE) because they provide lots of nice tools. But maybe also three IDEs (XCode, Microsoft Visual C, CLion, ...) depending on platform. But we don't want to have to keep several IDE project files up to date. But we also want to integrate nicely with different OS packagers like Debian, FreeBSD. We want people to be able to use apt-get install for their dependencies if they want. We also want to cross compile code on our beefy workstations to work on microcontrollers or slower low memory systems (like earlier RaspberryPi systems).



"Meson is an open source build system meant to be both extremely fast, and, even more importantly, as user friendly as possible. The main design point of Meson is that every moment a developer spends writing or debugging build definitions is a second wasted. So is every second spent waiting for the build system to actually start compiling code." It's first major user was GStreamer, a multi platform multimedia toolkit which is highly portable. Now it is especially popular in the FreeDesktop world with projects like gstreamer, GTK, and systemd amongst many others using it.



The documentation is excellent, and it's very fast compared to autotools or cmake. If CMake is modern, then The Meson Build System (https://mesonbuild.com/index.html) is post modern.It's first major user was GStreamer, a multi platform multimedia toolkit which is highly portable. Now it is especially popular in the FreeDesktop world with projects like gstreamer, GTK, and systemd amongst many others using it.The documentation is excellent, and it's very fast compared to autotools or cmake.

CppDepend , a commercial static analyzer based on Clang

The Clang Static Analyzer

The Clang Static Analyzer is a source code analysis tool that finds bugs in C, C++, and Objective-C programs. Currently it can be run either as a standalone tool or within Apple Xcode. The standalone tool is invoked from the command line, and is intended to be run in tandem with a build of a codebase.

brew install llvm

$SCANBUILD=`ls /usr/local/Cellar/llvm/*/bin/scan-build` $SCANBUILD -V python3 setup.py build

apt-get install clang-tools.

cppcheck

Cppcheck is an analysis tool for C/C++ code. It provides unique code analysis to detect bugs and focuses on detecting undefined behaviour and dangerous coding constructs. The goal is to detect only real errors in the code (i.e. have very few false positives).

"You will find more bugs in your software by testing your software carefully, than by using Cppcheck."

To Install cppcheck:

brew install cppcheck bear sudo apt-get install cppcheck bear





To run cppcheck on C code:

# call your build tool, like `bear make` to record. # See cppcheck manual for other C environments including Visual Studio. bear python setup.py build cppcheck --quiet --language=c --enable=all -D__x86_64__ -D__LP64__ --project=compile_commands.json

cppcheck --help

--enable

[src_c/math.c:1134]: (style) The function 'vector_getw' is never used. [src_c/base.c:1309]: (error) Pointer addition with NULL pointer. [src_c/scrap_qnx.c:109]: (portability) Assigning a pointer to an integer is not portable. [src_c/surface.c:832] -> [src_c/surface.c:819]: (warning) Either the condition '!surf' is redundant or there is possible null pointer dereference: surf.

/Analyze in Microsoft Visual Studio

"In GNU C and C++, you can use function attributes to specify certain function properties that may help the compiler optimize calls or check code more carefully for correctness."

Custom static analysis for API usage

Runtime checks and Dynamic Verification

./configure CFLAGS="-fsanitize=address,undefined -g" LDFLAGS="-fsanitize=address,undefined" make make check

Address Sanitizer

AddressSanitizer is a fast memory error detector. It consists of a compiler instrumentation module and a run-time library. The tool can detect the following types of bugs:

Out-of-bounds accesses to heap, stack and globals

Use-after-free

Use-after-return (runtime flag ASAN_OPTIONS=detect_stack_use_after_return=1)

Use-after-scope (clang flag -fsanitize-address-use-after-scope)

Double-free, invalid free

Memory leaks (experimental)

LDFLAGS= " -g -fsanitize=address " CFLAGS= " -g -fsanitize=address -fno-omit-frame-pointer " python3 setup.py install

Undefined Behaviour Sanitizer

App Verifier

"Any Windows developers that are listening to this: if you’re not using App Verifier, you are making a mistake." -- Bruce Dawson

Application Verifier (AppVerif.exe) is a dynamic verification tool for user-mode applications. This tool monitors application actions while the application runs, subjects the application to a variety of stresses and tests, and generates a report about potential errors in application execution or design.

Buffer over runs

Use after free issues

Thread issues including using TLS properly.

Low resource handling

Race conditions

Performance profiling and measurement

“The objective (not always attained) in creating high-performance software is to make the software able to carry out its appointed tasks so rapidly that it responds instantaneously, as far as the user is concerned.” Michael Abrash. “Michael Abrash’s Graphics Programming Black Book.”

Valgrind

Apple Performance Tools

LLVM performance tools.

GNU/Linux tools

Microsoft performance tools.

Intel performance tools.

Caching builds

Distributed building.

Complexity of code.

"Complex is better than complicated. It's OK to build very complex software, but you don't have to build it in a complicated way. Lizard is a free open source tool that analyse the complexity of your source code right away supporting many programming languages, without any extra setup. It also does code clone / copy-paste detection."

# install lizard

python3 -m pip install lizard

# show warnings only and include/exclude some files.

lizard src_c/ -x"src_c/_sdl2*" -w



# Can also run it as a python module.

python3 -m lizard src_c/ -x"src_c/_sdl2*" -w



# Show a full report, not just warnings (-w).

lizard src_c/ -x"src_c/_sdl2*" -x"src_c/_*" -x"src_c/SDL_gfx*" -x"src_c/pypm.c"

Lizard can also make a pretty word cloud from your source.

Testing your code on different OS/architectures.

Code Formatting

Services

Coding standards for C

How are other projects tested?

SQLite

Curl

Python

OpenSSL

libsdl

Linux

Haproxy