Gambit-C vs. Bigloo vs. Chicken vs. MzScheme vs. Scheme48

The following tables contain the execution time of the Gambit benchmarks on various implementations of Scheme in different situations. For a given benchmark and situation, the entry in green indicates which Scheme system has the fastest execution and the number given is the CPU time in milliseconds. Other entries give the execution time relative to the green entry. Blank entries indicate that this benchmark was not executed (possibly because the system did not accept to compile the program, or because a given situation could not be simulated with that implementation of Scheme).

The benchmarks were run in four different situations. This was done by using compiler options and/or declarations.

Safe with generic arithmetic and mutable bindings. This situation approximates the R5RS semantics: generic arithmetic operations

mutable bindings for the definitions in the benchmark

mutable predefined bindings (for +, car, ...)

safe execution (i.e. an exception must be signalled on errors) Safe with generic arithmetic. This situation corresponds to the proposed R6RS semantics: generic arithmetic operations

overflow detection on fixnum arithmetic (either produce a bignum or signal an exception)

immutable bindings for the definitions in the benchmark

immutable predefined bindings (for +, car, ...)

safe execution (i.e. an exception must be signalled on errors) Safe with fixnum/flonum specializations. This situation corresponds to the proposed R6RS semantics, with the use of fixnum and flonum specific functions for arithmetic: arithmetic operations are specialized to fixnum or flonum arguments as appropriate (for example turning + into fl+ ) and the fixnum operations may wrap on overflow

into ) and the fixnum operations may wrap on overflow immutable bindings for the definitions in the benchmark

immutable predefined bindings (for +, car, ...)

safe execution (i.e. an exception must be signalled on errors) Unsafe with fixnum/flonum specializations. This situation corresponds to the proposed R6RS semantics, with the use of fixnum and flonum specific functions for arithmetic and the use of the declaration `` (declare (unsafe)) '': like ``Safe with fixnum/flonum specializations'' but errors are not checked

Note: the tables are best viewed with a wide window.

Safe with generic arithmetic and mutable bindings Safe with generic arithmetic Safe with fixnum/flonum specializations Unsafe with fixnum/flonum specializations (R5RS) (R6RS) (R6RS fixflo) (R6RS fixflo unsafe) Program Bigloo Chicken Gambit MzScheme Scheme48 boyer 14.11 2050 5.48 28.08 browse 8.78 5445 5.93 19.24 cpstak 6.14 2267 7.83 17.08 ctak 2.54 1760 87.60 9.46 dderiv 6.51 3306 7.92 18.22 deriv 6.59 1934 10.64 19.55 destruc 10.41 1957 6.41 32.17 diviter 13.11 1911 10.68 39.62 divrec 7.94 3096 7.87 17.94 puzzle 9.85 2537 7.50 24.90 takl 10.36 3936 4.38 27.04 triangl 9.53 3491 5.48 24.29 fft 10.21 1446 7.06 20.95 fib 5.24 4917 4.41 12.27 fibfp 4.56 2499 8.28 11.31 mbrot 6.74 2123 10.79 18.45 pnpoly 10.06 1903 6.52 crash sum 12.30 1453 8.88 35.51 sumfp 6.76 1518 14.00 19.30 tak 8.02 3207 5.16 19.81 ack 10.86 246 6.54 22.80 array1 9.31 1029 5.83 22.76 cat 2.83 1.55 1122 1.83 string 135 2.10 1.73 61.26 sum1 2.75 217 1.16 11.47 sumloop 13.01 1437 10.28 36.00 tail 2.21 1277 1.24 4.94 wc 3.97 863 1.71 6.14 conform 4.24 2628 3.29 13.44 dynamic 3.18 1282 1.24 7.68 earley 11.00 1337 6.79 25.20 fibc 3.27 1744 64.70 10.15 graphs 7.83 2170 8.45 20.73 lattice 13.47 3295 5.08 38.31 matrix 8.84 2162 5.36 21.27 maze 5.87 3138 4.12 12.96 mazefun 6.61 2475 4.65 18.21 nqueens 11.33 2482 6.85 29.58 paraffins 17.38 2164 4.77 14.91 peval 9.09 1895 5.10 21.86 primes 12.83 2006 8.04 25.70 ray 4.00 1980 5.68 104.72 scheme 5.54 2282 3.51 13.76 simplex 7.51 2279 5.82 55.46 slatex 1.65 1528 1.11 2.79 Program Bigloo Chicken Gambit MzScheme Scheme48 boyer 650 2.48 1.56 3.81 28.77 browse 2940 2.64 1.27 5.44 14.53 cpstak 5.72 1.95 1775 8.35 12.03 ctak 60.67 1.85 1729 70.26 8.34 dderiv 3.11 3.27 1579 15.54 23.04 deriv 4.21 4.03 1111 19.00 21.70 destruc 1.20 2.35 1482 3.32 20.72 diviter 5.41 814 1.43 22.89 23.87 divrec 3.15 1.62 1599 14.08 13.63 puzzle 2.32 2.57 1909 3.32 10.75 takl 1.63 491 2.45 4.86 74.03 triangl 1.26 2.99 2243 2.24 14.96 fft 3.19 2.06 1291 3.50 8.81 fib 2.39 3.74 2288 1.25 11.45 fibfp 4.15 2.29 1733 9.58 7.81 mbrot 4.86 1.44 1816 12.20 10.51 pnpoly 8.20 2.64 1522 3.83 crash sum 5.22 5.06 1.43 740 19.66 sumfp 5.30 1.74 1331 12.48 9.96 tak 2.17 3.73 1536 1.52 14.41 ack 3.38 6.20 133 1.47 18.20 array1 1.75 2.65 830 1.47 9.52 cat 230 11.85 7.14 3.66 2.74 string 30 4.50 9.50 9.57 275.33 sum1 80 7.25 2.04 2.96 30.62 sumloop 2.37 4.31 1037 2.83 15.22 tail 640 2.86 1.92 2.09 6.39 wc 230 7.38 3.36 3.77 7.43 conform 800 2.95 1.49 6.92 33.72 dynamic 520 4.72 1.98 2.13 16.67 earley 1.61 5.26 1127 4.62 19.36 fibc 33.08 2.14 1365 70.66 10.56 graphs 2.08 2.08 1358 9.86 20.09 lattice 1.50 1.82 2721 3.26 34.55 matrix 1.37 3.03 1727 4.33 16.03 maze 1410 3.03 1.13 3.53 14.74 mazefun 2.02 2.41 1359 2.44 15.54 nqueens 1.73 2.59 1733 3.12 15.74 paraffins 2.78 5.32 1748 4.50 8.63 peval 880 4.53 1.21 5.47 22.76 primes 2.22 8.24 1339 8.98 19.24 ray 3.42 1.22 1127 5.82 177.25 scheme 1460 1.75 1.26 3.08 14.34 simplex 1.93 2.40 1325 4.70 78.38 slatex 930 1.78 1.57 1.60 3.52 Program Bigloo Chicken Gambit MzScheme Scheme48 boyer 640 1.60 browse 2990 1.27 cpstak 5.31 1660 ctak 59.02 1768 dderiv 2.90 1707 deriv 4.08 1158 destruc 1360 1.06 diviter 3.72 1201 divrec 3.16 1620 puzzle 1.90 1745 takl 830 1.50 triangl 1440 1.56 fft 1.11 1210 fib 1000 2.08 fibfp 1.60 1739 mbrot 240 7.67 pnpoly 4.10 1433 sum 100 6.64 sumfp 100 13.80 tak 1090 1.21 ack 1.30 92 array1 1.35 853 cat 240 6.29 string 40 7.32 sum1 70 2.40 sumloop 220 3.40 tail 630 1.99 wc 160 4.90 conform 800 1.54 dynamic 510 2.05 earley 1.42 1016 fibc 34.71 1339 graphs 1.95 1293 lattice 1.45 2877 matrix 1.17 1772 maze 1.08 973 mazefun 1.16 1279 nqueens 1.01 1563 paraffins 2.70 1829 peval 880 1.21 primes 2.15 1360 ray 1.02 1091 scheme 1490 1.20 simplex 1.02 1145 slatex 930 1.59 Program Bigloo Chicken Gambit MzScheme Scheme48 boyer 470 3.39 1.43 browse 2500 2.57 1.50 cpstak 6.02 1.62 1438 ctak 49.19 1.65 1728 dderiv 3.00 2.72 1500 deriv 3.72 2.91 1179 destruc 1200 1.12 1.06 diviter 5.17 785 1.29 divrec 3.03 1.84 1409 puzzle 3.58 4.28 660 takl 450 1.10 1.96 triangl 950 4.63 1.20 fft 3.17 4.66 328 fib 770 1.49 1.24 fibfp 2.26 2.55 1173 mbrot 230 5.30 4.68 pnpoly 190 14.73 1.51 sum 100 1.48 1.70 sumfp 100 11.15 12.04 tak 740 1.26 1.42 ack 2.05 44 1.34 array1 1.28 2.75 431 cat 200 10.44 7.86 string 20 6.55 14.55 sum1 70 6.26 2.36 sumloop 100 9.55 3.85 tail 560 2.51 2.20 wc 130 9.89 5.58 conform 630 3.75 1.50 dynamic 500 4.25 1.93 earley 1.49 6.55 767 fibc 29.12 1.71 1358 graphs 1.90 2.39 979 lattice 1.75 2.39 2067 matrix 1.17 2.39 1481 maze 1.47 4.38 524 mazefun 1.26 1.25 1000 nqueens 1.23 1.59 1081 paraffins 2.65 5.53 1736 peval 780 4.43 1.10 primes 2.15 1.43 1079 ray 130 10.81 2.94 scheme 1060 2.31 1.44 simplex 350 crash 1.33 slatex 880 1.70 1.60