Happy, Alex, and GHC 7.8

As we approach the 7.8 release of GHC, more and more people are running into problems with packages that use Alex and/or Happy for parsing.

The errors look like

templates/GenericTemplate.hs:104:22: Couldn't match expected type ‛Bool’ with actual type ‛Happy_GHC_Exts.Int#’ In the expression: (n Happy_GHC_Exts.<# (0# :: Happy_GHC_Exts.Int#)) In a stmt of a pattern guard for a case alternative: (n Happy_GHC_Exts.<# (0# :: Happy_GHC_Exts.Int#)) In a case alternative: n | (n Happy_GHC_Exts.<# (0# :: Happy_GHC_Exts.Int#)) -> (happyReduceArr Happy_Data_Array.! rule) i tk st where rule = (Happy_GHC_Exts.I# ((Happy_GHC_Exts.negateInt# ((n Happy_GHC_Exts.+# (1# :: Happy_GHC_Exts.Int#))))))

for Happy and

Pattern bindings containing unlifted types should use an outermost bang pattern: ((I# (ord_c))) = ord c In the expression: let (base) = alexIndexInt32OffAddr alex_base s ((I# (ord_c))) = ord c (offset) = (base +# ord_c) .... in case new_s of { -1# -> (new_acc, input) _ -> alex_scan_tkn user orig_input (len +# 1#) new_input new_s new_acc } In a case alternative: Just (c, new_input) -> let (base) = alexIndexInt32OffAddr alex_base s ((I# (ord_c))) = ord c .... in case new_s of { -1# -> (new_acc, input) _ -> alex_scan_tkn user orig_input (len +# 1#) new_input new_s new_acc } In the second argument of ‘seq’, namely ‘case alexGetChar input of { Nothing -> (new_acc, input) Just (c, new_input) -> let (base) = ... .... in case new_s of { -1# -> ... _ -> alex_scan_tkn user orig_input (len +# 1#) new_input new_s new_acc } }’

for Alex. (These are not all the error messages that are produced by GHC, but hopefully enough that this article is googlable.)

First I give instructions on how to fix these problems, and then explain why they arise in the first place.

TL;DR: how do I fix the package?

As a maintainer

Install the latest versions of alex and happy . GHC 7.8 support was added in alex-3.1.0 and happy-1.19.0 , but later versions contain additional bugfixes. Double-check that cabal picks the latest versions of these tools: in your package’s source tree run cabal configure -v | grep -e alex -e happy configure -v-e alex -e happy The output should look like Using alex version 3.1.3 found on system at: /home/feuerbach/bin/alex Using happy version 1.19.3 found on system at: /home/feuerbach/bin/happy Bump the package’s version (the fourth component is enough: e.g. 1.2.3 -> 1.2.3.1 ), build the package and upload: cabal build build cabal sdist sdist cabal upload dist/ $pkg - $version .tar.gz upload dist/.tar.gz

That’s it; no actual source code modification to your package is necessary. If you’re curious as to why this works, read on.

As a user

First of all, check that you have latest alex and happy installed. That by itself can resolve your problem.

If it doesn’t, notify the package maintainer(s) about this problem and send them a link to this article. Only they are in a position to fix the problem properly.

Until the maintainer(s) react, you can fix the problem locally as follows:

Get into the source tree: cabal get $pkg get cd $pkg - $version (If cabal says it doesn’t know about the get command, you have to update it with cabal install cabal-install install cabal-install The command was called unpack before, but since you are using GHC 7.8 now, the older versions of cabal will get you in trouble anyway.) Now that you’re in the source tree, run cabal clean clean You may think «but I’ve just downloaded a fresh copy of the package’s source — surely it is clean!» Not really; read on for the details. Finally, cabal install install should run without any alex - or happy -related errors.

What’s going on here?

Code produced by old Happy and Alex no longer builds

Because Alex and Happy strive to produce the most efficient code, they make use of unboxed types and primitives. And those are affected by certain changes in GHC 7.8:

Happy and Alex were then updated to generate code that builds with the new GHC. So, it seems, just updating happy / alex should do the trick. Not so fast!

cabal includes generated code in the source distribution

When cabal creates a source distribution for uploading to hackage ( cabal sdist ), it includes the files generated by alex and happy in the tarball. So even when you have the new alex and happy installed, cabal install will not see the need to regenerate .hs files from .x and .y sources, and will run into the errors described above.

That’s why maintainers have to re-upload their tarballs generated with new alex and happy ; and until they do, users have to run cabal get and cabal clean .

The rationale behind this cabal behavior is not to force users install alex or happy . Alas, it doesn’t work so well in practice:

Dec 09 19:37:21 dcoutts refold: there's a few problems with our shipped pre-processed sources system Dec 09 19:37:41 dcoutts it doesn't interact well with using build-tools: happy Dec 09 19:38:03 dcoutts if there are shipped sources then obviously we do not need happy Dec 09 19:38:14 dcoutts the shipped sources currently go in dist Dec 09 19:38:18 dcoutts that then fails if you clean Dec 09 19:38:33 dcoutts it only allows one instance of shipped sources Dec 09 19:38:52 dcoutts e.g. for happy & alex, they can produce ghc-specific output or generic output Dec 09 19:39:20 dcoutts this is less of a problem these days since in practice there are not other compilers Dec 09 19:40:19 dcoutts and then this new problem, if we do ship sources, we don't know what version of the pre-processor generated them, so we cannot easily hack around version incompatibilities Dec 09 19:40:21 refold yes, using dist is hack Dec 09 19:40:30 dcoutts the plan was to use a different dir Dec 09 19:40:31 refold also fails with a different --builddir Dec 09 19:40:34 dcoutts right

See also #130 and #1685.

cabal sdist is not “pure”

We usually think of cabal sdist as a pure function taking in the source tree and producing the tarball. It’s not that simple.

Above I wrote that cabal sdist includes alex and happy -generated sources in the tarball. However, as Mikhail Glushenkov explains, it doesn’t actually generate them. It only includes them if they are already present as an artifact of a previous cabal build .

When I uploaded haskell-src-exts-1.16.0, I wasn’t aware of this and apparently ran cabal clean before cabal sdist . As a consequence, the tarball doesn’t have the dist/ subdirectory with generated files as you can easily check.

In order to install that particular version of haskell-src-exts , a user needs to have happy installed (and if her happy is old, she’ll get the exact same error described above). When I learned about it, I made a point release, 1.16.0.1, which does include the happy output.

So this is another thing that maintainers need to be aware of and watch out for.