head

tail

NullPointerException

head

head

tail

FullList a

We stress that the head-of-empty-list errors can be eliminated now, without any modification to the Haskell type system, without developing any new tool. Already Haskell98 can do that. The same technique applies to OCaml and even Java and C++. The only required advancement is in our thinking and programming style.

Thinking of full lists as a separate type from ordinary, potentially empty lists does affect our programming style -- but it does not have to break the existing code. The new style is easy to introduce gradually. Besides safety, its explicitness makes list processing algorithms more insightful, separating out algorithmically meaningful empty list checks from the redundant safety checks. Let us see some examples.

Assume the following interface

type FullList a -- abstract; for an implementation, see below fromFL :: FullList a -> [a] indeedFL :: [a] -> w -> (FullList a -> w) -> w -- an analogue of `maybe' headS :: FullList a -> a tailS :: FullList a -> [a] -- Adding something to a general list surely gives a non-empty list infixr 5 !: class Listable l where (!:) :: a -> l a -> FullList a

headS

tailS

fromFL

FullList

headS []

headS some_expression

some_expression

FullList

head

tail

regular_reverse :: [a] -> [a] regular_reverse l = loop l [] where loop [] accum = accum loop l accum = loop (Prelude.tail l) (Prelude.head l : accum)

safe_reverse :: [a] -> [a] safe_reverse l = loop l [] where loop l accum = indeedFL l accum $ (\l -> loop (tailS l) (headS l : accum)) test1 = safe_reverse [1,2,3]

indeedFL

l

headS

tailS

safe_append :: [a] -> FullList a -> FullList a safe_append [] l = l safe_append (h:t) l = h !: safe_append t l l1 :: FullList Int l1 = 1 !: 2 !: [] -- We can apply safe_append on two FullList without any problems test5 = tailS $ safe_append (fromFL l1) l1 -- [2,1,2]

FullList

data FullList a = FullList a [a] -- carry the list head explicitly fromFL :: FullList a -> [a] fromFL (FullList x l) = x : l headS :: FullList a -> a headS (FullList x _) = x tailS :: FullList a -> [a] tailS (FullList _ x) = x

FullList

FullList

There is another approach of representing FullList , which easily generalizes to other structures.

module NList (FullList, fromFL, headS, tailS, ...) where newtype FullList a = FullList [a] -- data constructor is not exported! fromFL (FullList x) = x -- The following are _total_ functions -- They are guaranteed to be safe, and so we could have used -- unsafeHead# and unsafeTail# if GHC provided them. headS :: FullList a -> a headS (FullList (x:_)) = x tailS :: FullList a -> [a] tailS (FullList (_:x)) = x

FullList

FullList

FullList

One may regard the abstract type FullList as standing for the proposition (invariant) that the represented list is non-empty. Now we have something to prove: we have to verify, manually or semi-automatically, that all operations within the module NList whose return type is FullList respect the invariant and ensure the truth of the non-emptiness proposition. Once we have verified these exported constructors, all operations that consume FullList , within NList or outside, can take this non-emptiness proposition for granted. Therefore, we are justified in using unsafe-head operations in implementing headS . Compared to the first implementation, the fact that FullList represents a non-empty list is no longer obvious and has to be proven. Fortunately, we only have to prove the operations within the NList module, that is, the ones that make use of the data constructor FullList . All other functions, which produce FullList merely by invoking the operations of NList , ensure the non-emptiness invariant by construction and do not need a proof. The advantage of the second implementation is the easy generalization to ByteString s, Text , and other sequences and collections. The data constructor FullList is merely a newtype wrapper, with no run-time overhead. Thus the second implementation provides the safety of head and tail operations without sacrificing efficiency.

In the old (2006) discussion of non-empty lists on Haskell-Cafe, Jan-Willem Maessen wrote: ``In addition, we have this rather nice assembly of functions which work on ordinary lists. Sadly, rewriting them all to also work on NonEmptyList or MySpecialInvariantList is a nontrivial task.'' Backwards compatibility is indeed a serious concern: no matter how better a new programming style may be, the mere thought of re-writing the existing code is a deterrent. Suppose we have a function foo:: [a] -> [a] (whose code, if available, we'd rather not change) and we want to write something like

\l -> [head l, head (foo l)]

\l -> indeedFL l onempty (\l -> [headS l, headS (foo l)])

foo

[a]

FullList a

foo

FullList a

headS

FullList a

fromFL

fromFL

fromIntegral

Int

Integer

Int

CInt

If we are not sure if our foo maps non-empty lists to non-empty lists, we really should handle the empty list case:

\l -> indeedFL l onempty $ \l -> [headS l, indeedFL (foo $ fromFL l) onempty' headS]

foo

\l -> indeedFL l onempty $ \l -> [headS l, indeedFL (foo $ fromFL l) (error msg) headS] where msg = "I'm quite sure foo maps non-empty lists to " ++ "non-empty lists. I'll be darned if it doesn't."

foo

NList

nfoo (FullList x) = FullList $ foo x

\l -> indeedFL l onempty (\l -> [headS l, headS (nfoo l)])

In conclusion, we have demonstrated the programming style that ensures safety without sacrificing efficiency. The key idea is that an abstract data type ensures (possibly quite sophisticated) propositions about the data -- so long as the very limited set of basic constructors satisfy the propositions. This main idea is very old, advocated by Milner and Morris in the mid-1970s. If there is a surprise in this, it is in the triviality of approach. One can't help but wonder why we do not program in this style.