Kicking the Debugger habit

This post covers my experience of giving up breakpoint and step-through debugging.

There are some interesting features of the domain I work in that have led me here. Financial analytics tend to compose many algorithms such as curve fitting, statistics, monte carlo and optimisation. The public API on the other hand is very simple: given this portfolio, return the risks associated.

A bug report is more likely to be 'this risk number looks a little odd' rather than 'an exception was thrown and here is the stack trace'. I can't agree with the idea that you should only unit test your public APIs. Rather, this should be: you need to unit test your public APIs, but if your domain is sufficiently complex, you also need to unit test internal modules. What is key is if a bug report queries an API result how quickly could you investigate and resolve any potential issue?

In my field:

It's not scalable - for larger code paths or multiple threads setting breakpoints and stepping through is just not feasible. It's like finding a needle in a haystack.

It's limited in power - even mature debugging frameworks such as in Visual Studio are limited in the kind of conditional logic you can use while debugging.

It's time consuming - many a good hour can be spent pressing F5/F10/F11 in a zombie like state only to restart and try again.

Since starting to use Expecto, I've been using the command line to run unit tests. Expecto does integrate with Visual Studio and can even do live unit testing in VS Code. I've found using a set of commands I've built up in FAKE to be more flexible and productive e.g.

test integration test all test 64 debug -- stress 2

Expecto encourages using normal code for organisation, setup & teardown and parameterisation of tests, instead of a limited framework of attributes.

Now apply this concept to debugging. With a debug module in the core of a codebase that is conditional on the debug configuration, code can be annotated with validation and some debug output. The command line records a history of the test results and validation output. Once complete, compiling in release ensures all diagnostic code is removed.

This started out as simple functions to printfn data being sequenced and piped, but expanded into functions to count calls, check for NaNs globally, serialize function inputs and outputs, test convergence of numbers etc. This is normal code and there is huge scope for adding conditional logic.

Kicking the debugger habit has given me a productivity boost. It forces me to think more logically about how I validate and break down a problem.

It also reduces the complexity of the tooling. Finding and fixing bugs feels more like coding and unit testing. I can use a simpler code editor plus the command line.

The result is I now have more confidence that once I've created the initial bug test I will be able to resolve it quickly.

I've been asked for some sample code from the debug module. The code below should hopefully start to give an idea of what can be done.

//#if DEBUG [< AutoOpen >] module OverflowAndNaNCheck = open Checked let inline private isNaN v = match box v with | :? float as v -> Double . IsNaN v | _ -> false let inline ( / ) a b = let c = a / b in if isNaN c then failwithf "NaN found: %A / %A = %A" a b c else c let inline ( + ) a b = let c = a + b in if isNaN c then failwithf "NaN found: %A + %A = %A" a b c else c let inline ( - ) a b = let c = a - b in if isNaN c then failwithf "NaN found: %A - %A = %A" a b c else c let inline (*) a b = let c = a * b in if isNaN c then failwithf "NaN found: %A * %A = %A" a b c else c let inline ( ** ) a b = let c = a ** b in if isNaN c then failwithf "NaN found: %A ** %A = %A" a b c else c let inline sqrt a = let c = sqrt a in if isNaN c then failwithf "NaN found: sqrt %A = %A" a c else c let inline log a = let c = log a in if isNaN c then failwithf "NaN found: log %A = %A" a c else c let inline log10 a = let c = log10 a in if isNaN c then failwithf "NaN found: log10 %A = %A" a c else c let inline asin a = let c = asin a in if isNaN c then failwithf "NaN found: asin %A = %A" a c else c let inline acos a = let c = acos a in if isNaN c then failwithf "NaN found: acos %A = %A" a c else c let inline atan a = let c = atan a in if isNaN c then failwithf "NaN found: atan %A = %A" a c else c module Dbg = let private rand = Random ( ) let mutable private randN = None type atRandom ( n : int ) = do randN <- Some n interface IDisposable with member __ . Dispose ( ) = randN <- None let write fmt = let sb = let n = DateTime . Now let sb = StringBuilder ( "DEBUG " ) sb . Append ( n . ToString ( "dd MMM HH:mm:ss.fffffff" ) ) |> ignore sb . Append ( "> " ) |> ignore sb Printf . kbprintf ( fun ( ) -> if Option . isNone randN || Option . get randN |> rand . Next = 0 then let old = Console . ForegroundColor try Console . ForegroundColor <- ConsoleColor . Red sb . ToString ( ) |> Console . WriteLine finally Console . ForegroundColor <- old ) sb fmt let writeIf condition fmt = if condition ( ) then write fmt let runIf condition fn = if condition ( ) then fn ( ) let pipe fmt = fun a -> write fmt a ; a let seq desc s = let s = Seq . cache s Seq . iter ( write "%s: %A" desc ) s s let fun1 desc ( f : ' a -> ' b ) = fun a -> let b = f a write "%s - Input: %A\t\t Output: %A" desc a b ; b let fun2 desc ( f : ' a -> ' b -> ' c ) = fun a b -> let c = f a b write "%s - Input: %A\t\t Output: %A" desc ( a , b ) c ; c let fun3 desc ( f : ' a -> ' b -> ' c -> ' d ) = fun a b c -> let d = f a b c write "%s - Input: %A\t\t Output: %A" desc ( a , b , c ) d ; d type counter ( desc : string ) = let mutable count = 0 member __ . Count = count member __ . Increment ( ) = count <- count + 1 member inline m . Calls fn = fun a -> m . Increment ( ) ; fn a interface IDisposable with member __ . Dispose ( ) = write "%s count = %i" desc count let descendingChecker desc = let mutable last = infinity fun x -> if x > last then write "%s - should be descending but %A > %A" desc x last else last <- x let mutable private functionMap = Map . empty let addFun ( key : string ) ( fn : unit -> unit ) = functionMap <- Map . add key fn functionMap let runFun ( key : string ) = Map . find key functionMap ( ) //#endif

namespace System

namespace System.Text

Multiple items

type AutoOpenAttribute =

inherit Attribute

new : unit -> AutoOpenAttribute

new : path:string -> AutoOpenAttribute

member Path : string



--------------------

new : unit -> AutoOpenAttribute

new : path:string -> AutoOpenAttribute

Multiple items

module Checked



from Microsoft.FSharp.Core.Operators



--------------------

module Checked



from Microsoft.FSharp.Core.ExtraTopLevelOperators

val box : value:'T -> obj

Multiple items

val float : value:'T -> float (requires member op_Explicit)



--------------------

type float = Double



--------------------

type float<'Measure> = float

type Double =

struct

member CompareTo : value:obj -> int + 1 overload

member Equals : obj:obj -> bool + 1 overload

member GetHashCode : unit -> int

member GetTypeCode : unit -> TypeCode

member ToString : unit -> string + 3 overloads

static val MinValue : float

static val MaxValue : float

static val Epsilon : float

static val NegativeInfinity : float

static val PositiveInfinity : float

...

end

Double.IsNaN(d: float) : bool

val failwithf : format:Printf.StringFormat<'T,'Result> -> 'T

val sqrt : value:'T -> 'U (requires member Sqrt)

val log : value:'T -> 'T (requires member Log)

val log10 : value:'T -> 'T (requires member Log10)

val asin : value:'T -> 'T (requires member Asin)

val acos : value:'T -> 'T (requires member Acos)

val atan : value:'T -> 'T (requires member Atan)

Multiple items

type Random =

new : unit -> Random + 1 overload

member Next : unit -> int + 2 overloads

member NextBytes : buffer:byte[] -> unit

member NextDouble : unit -> float



--------------------

Random() : Random

Random(Seed: int) : Random

union case Option.None: Option<'T>

Multiple items

val int : value:'T -> int (requires member op_Explicit)



--------------------

type int = int32



--------------------

type int<'Measure> = int

union case Option.Some: Value: 'T -> Option<'T>

type IDisposable =

member Dispose : unit -> unit

Multiple items

type DateTime =

struct

new : ticks:int64 -> DateTime + 10 overloads

member Add : value:TimeSpan -> DateTime

member AddDays : value:float -> DateTime

member AddHours : value:float -> DateTime

member AddMilliseconds : value:float -> DateTime

member AddMinutes : value:float -> DateTime

member AddMonths : months:int -> DateTime

member AddSeconds : value:float -> DateTime

member AddTicks : value:int64 -> DateTime

member AddYears : value:int -> DateTime

...

end



--------------------

DateTime ()

(+0 other overloads)

DateTime(ticks: int64) : DateTime

(+0 other overloads)

DateTime(ticks: int64, kind: DateTimeKind) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int, calendar: Globalization.Calendar) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, kind: DateTimeKind) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, calendar: Globalization.Calendar) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, millisecond: int) : DateTime

(+0 other overloads)

DateTime(year: int, month: int, day: int, hour: int, minute: int, second: int, millisecond: int, kind: DateTimeKind) : DateTime

(+0 other overloads)

property DateTime.Now: DateTime

Multiple items

type StringBuilder =

new : unit -> StringBuilder + 5 overloads

member Append : value:string -> StringBuilder + 19 overloads

member AppendFormat : format:string * arg0:obj -> StringBuilder + 7 overloads

member AppendLine : unit -> StringBuilder + 1 overload

member Capacity : int with get, set

member Chars : int -> char with get, set

member Clear : unit -> StringBuilder

member CopyTo : sourceIndex:int * destination:char[] * destinationIndex:int * count:int -> unit

member EnsureCapacity : capacity:int -> int

member Equals : sb:StringBuilder -> bool

...



--------------------

StringBuilder() : StringBuilder

StringBuilder(capacity: int) : StringBuilder

StringBuilder(value: string) : StringBuilder

StringBuilder(value: string, capacity: int) : StringBuilder

StringBuilder(capacity: int, maxCapacity: int) : StringBuilder

StringBuilder(value: string, startIndex: int, length: int, capacity: int) : StringBuilder

val ignore : value:'T -> unit

module Printf



from Microsoft.FSharp.Core

val kbprintf : continuation:(unit -> 'Result) -> builder:StringBuilder -> format:Printf.BuilderFormat<'T,'Result> -> 'T

module Option



from Microsoft.FSharp.Core

val isNone : option:'T option -> bool

val get : option:'T option -> 'T

type Console =

static member BackgroundColor : ConsoleColor with get, set

static member Beep : unit -> unit + 1 overload

static member BufferHeight : int with get, set

static member BufferWidth : int with get, set

static member CapsLock : bool

static member Clear : unit -> unit

static member CursorLeft : int with get, set

static member CursorSize : int with get, set

static member CursorTop : int with get, set

static member CursorVisible : bool with get, set

...

property Console.ForegroundColor: ConsoleColor

type ConsoleColor =

| Black = 0

| DarkBlue = 1

| DarkGreen = 2

| DarkCyan = 3

| DarkRed = 4

| DarkMagenta = 5

| DarkYellow = 6

| Gray = 7

| DarkGray = 8

| Blue = 9

...

field ConsoleColor.Red: ConsoleColor = 12

Console.WriteLine() : unit

(+0 other overloads)

Console.WriteLine(value: string) : unit

(+0 other overloads)

Console.WriteLine(value: obj) : unit

(+0 other overloads)

Console.WriteLine(value: uint64) : unit

(+0 other overloads)

Console.WriteLine(value: int64) : unit

(+0 other overloads)

Console.WriteLine(value: uint32) : unit

(+0 other overloads)

Console.WriteLine(value: int) : unit

(+0 other overloads)

Console.WriteLine(value: float32) : unit

(+0 other overloads)

Console.WriteLine(value: float) : unit

(+0 other overloads)

Console.WriteLine(value: decimal) : unit

(+0 other overloads)

Multiple items

val seq : sequence:seq<'T> -> seq<'T>



--------------------

type seq<'T> = Collections.Generic.IEnumerable<'T>

module Seq



from Microsoft.FSharp.Collections

val cache : source:seq<'T> -> seq<'T>

val iter : action:('T -> unit) -> source:seq<'T> -> unit

Multiple items

val string : value:'T -> string



--------------------

type string = String

val infinity : float

Multiple items

module Map



from Microsoft.FSharp.Collections



--------------------

type Map<'Key,'Value (requires comparison)> =

interface IReadOnlyDictionary<'Key,'Value>

interface IReadOnlyCollection<KeyValuePair<'Key,'Value>>

interface IEnumerable

interface IComparable

interface IEnumerable<KeyValuePair<'Key,'Value>>

interface ICollection<KeyValuePair<'Key,'Value>>

interface IDictionary<'Key,'Value>

new : elements:seq<'Key * 'Value> -> Map<'Key,'Value>

member Add : key:'Key * value:'Value -> Map<'Key,'Value>

member ContainsKey : key:'Key -> bool

...



--------------------

new : elements:seq<'Key * 'Value> -> Map<'Key,'Value>

val empty<'Key,'T (requires comparison)> : Map<'Key,'T> (requires comparison)

type unit = Unit

val add : key:'Key -> value:'T -> table:Map<'Key,'T> -> Map<'Key,'T> (requires comparison)

val find : key:'Key -> table:Map<'Key,'T> -> 'T (requires comparison)