Can the creators of Quorum write programs in Quorum? A. Sinan Unur June 10, 2015

Quorum is a programming language created by Stefik et al. In a 2011 paper, the authors infamously proclaimed:

Results showed that while Quorum users were afforded significantly greater accuracy compared to those using Perl and Randomo, Perl users were unable to write programs more accurately than those using a language designed by chance.

Quorum seems to be their pet language which they are pushing to replace actual programming languages in widespread use in the industry.

Elsewhere, I showed how the purportedly identical code snippets in Java, Perl, Quorum, Python, and Ruby they used in a follow up paper were strangely inconsistent.

For completeness sake, I thought I should run their Quorum program. I wasn't able to get their code to run using their widget on the Quorum web site.

I finally broke down and downloaded Quorum to a Linux VM. I wanted to see what output their Quorum code produced, as I had failed to run it using their online widget.

Using the image of the reference Quorum code which they gave their participants in both studies, I typed in the following code, and put it in a text file

action Main number x = z(1, 12, 3) end action z(integer a, integer b, integer c) returns number number d = 0.0 number e = 0.0 integer i = a repeat b - a times if i mod c = 0 then d = d + 1 end else then e = e + 1 end i = i + 1 end if d > e then return d end else then return e end end

When I try to run, I get:

$ java -jar Quorum.jar t.quorum Preparing to build t.quorum Building files... done. The code did not build correctly. Here is a list of errors: 1: t.quorum, Line 11, Column 8: d has already been defined. 1: t.quorum, Line 13, Column 6: A loop, conditional, class, or action statement is missing an 'end'. 1: t.quorum, Line 14, Column 8: e has already been defined. 1: t.quorum, Line 16, Column 6: Missing or invalid statement at 'i

Here is a screenshot:

To me, the code I typed looks identical to the code given in the paper. Yet, Quorum thinks the assignment to d on line 11 is a redefinition of that variable. I think the rest of the errors follow from that.

Did I make a mistake? Or, is the Quorum code Stefik et al. used to make their claim flawed? Given that almost no one programs in Quorum, does it matter?

Stefik and friends seem to have found a corner of academia in which they can ponder if eliminating the use of == for equality comparison makes sense (BTW, this is not so novel, e.g. SAS uses = for both assignment and comparison, and most versions of BASIC I touched did this as well).

They use inflammatory language to state benign conclusions, exhibit flawed statistical reasoning, and the code snippets they show in their papers are not consistent with each other and task description.

These observations lead me to doubt the practical significance of their results and research.

Methinks they are not in a good position to lecture others on Questions and Responsibilities for the Programming Language Community.

PS: You can discuss this post on /r/perl.