Item posted by Akim Demaille <akim> on Sat 26 Jan 2019 02:15:08 PM UTC.

We are very happy to announce the release of Bison 3.3!



The new option --update replaces deprecated features with their modern

spelling, but also applies fixes such as eliminating duplicate directives,

etc. It is now possible to annotate rules with their number of expected

conflicts. Bison can be made relocatable. The symbol declaration syntax

was overhauled, and in particular, %nterm, that exists since the origins of

Bison, is now an officially supported (and documented!) feature. C++

parsers now feature genuine symbol constructors, and use noexcept/constexpr.

The GLR parsers in C++ now support the syntax_error exceptions. There are

also many smaller improvements, including a fix for a bug which is at least

31 years old.



Please see the NEWS below for more details.



Many thanks to Askar Safin, Derek Clegg, Étienne Renault, Frank Heckenbach,

Rici Lake, Wolfgang Thaller and the members of the Bison mailing lists for

their feedback during the development of this version.



Here are the compressed sources:

https://ftp.gnu.org/gnu/bison/bison-3.3.tar.gz (4.1MB)

https://ftp.gnu.org/gnu/bison/bison-3.3.tar.xz (2.1MB)



Here are the GPG detached signatures[*]:

https://ftp.gnu.org/gnu/bison/bison-3.3.tar.gz.sig

https://ftp.gnu.org/gnu/bison/bison-3.3.tar.xz.sig



Use a mirror for higher download bandwidth:

https://www.gnu.org/order/ftp.html



[*] Use a .sig file to verify that the corresponding file (without the

.sig suffix) is intact. First, be sure to download both the .sig file

and the corresponding tarball. Then, run a command like this:



gpg --verify bison-3.3.tar.gz.sig



If that command fails because you don't have the required public key,

then run this command to import it:



gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E



and rerun the 'gpg --verify' command.



This release was bootstrapped with the following tools:

Autoconf 2.69

Automake 1.16.1

Flex 2.6.4

Gettext 0.19.8.1

Gnulib v0.1-2382-g34881aff4



NEWS



* Noteworthy changes in release 3.3 (2019-01-26) [stable]



A new mailing list was created, Bison Announce. It is low traffic, and is

only about announcing new releases and important messages (e.g., polls about

major decisions to make).



https://lists.gnu.org/mailman/listinfo/bison-announce



** Backward incompatible changes



Support for DJGPP, which has been unmaintained and untested for years, is

removed.



** Deprecated features



A new feature, --update (see below) helps adjusting existing grammars to

deprecations.



*** Deprecated directives



The %error-verbose directive is deprecated in favor of '%define

parse.error verbose' since Bison 3.0, but no warning was issued.



The '%name-prefix "xx"' directive is deprecated in favor of '%define

api.prefix {xx}' since Bison 3.0, but no warning was issued. These

directives are slightly different, you might need to adjust your code.

%name-prefix renames only symbols with external linkage, while api.prefix

also renames types and macros, including YYDEBUG, YYTOKENTYPE,

yytokentype, YYSTYPE, YYLTYPE, etc.



Users of Flex that move from '%name-prefix "xx"' to '%define api.prefix

{xx}' will typically have to update YY_DECL from



#define YY_DECL int xxlex (YYSTYPE *yylval, YYLTYPE *yylloc)



to



#define YY_DECL int xxlex (XXSTYPE *yylval, XXLTYPE *yylloc)



*** Deprecated %define variable names



The following variables, mostly related to parsers in Java, have been

renamed for consistency. Backward compatibility is ensured, but upgrading

is recommended.



abstract -> api.parser.abstract

annotations -> api.parser.annotations

extends -> api.parser.extends

final -> api.parser.final

implements -> api.parser.implements

parser_class_name -> api.parser.class

public -> api.parser.public

strictfp -> api.parser.strictfp



** New features



*** Generation of fix-its for IDEs/Editors



When given the new option -ffixit (aka -fdiagnostics-parseable-fixits),

bison now generates machine readable editing instructions to fix some

issues. Currently, this is mostly limited to updating deprecated

directives and removing duplicates. For instance:



$ cat foo.y

%error-verbose

%define parser_class_name "Parser"

%define api.parser.class "Parser"

%%

exp:;



See the "fix-it:" lines below:



$ bison -ffixit foo.y

foo.y:1.1-14: warning: deprecated directive, use '%define parse.error verbose' [-Wdeprecated]

%error-verbose

^~~~~~~~~~~~~~

fix-it:"foo.y":{1:1-1:15}:"%define parse.error verbose"

foo.y:2.1-34: warning: deprecated directive, use '%define api.parser.class {Parser}' [-Wdeprecated]

%define parser_class_name "Parser"

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

fix-it:"foo.y":{2:1-2:35}:"%define api.parser.class {Parser}"

foo.y:3.1-33: error: %define variable 'api.parser.class' redefined

%define api.parser.class "Parser"

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

foo.y:2.1-34: previous definition

%define parser_class_name "Parser"

^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

fix-it:"foo.y":{3:1-3:34}:""

foo.y: warning: fix-its can be applied. Rerun with option '--update'. [-Wother]



This uses the same output format as GCC and Clang.



*** Updating grammar files



Fixes can be applied on the fly. The previous example ends with the

suggestion to re-run bison with the option -u/--update, which results in a

cleaner grammar file.



$ bison --update foo.y

[...]

bison: file 'foo.y' was updated (backup: 'foo.y~')



$ cat foo.y

%define parse.error verbose

%define api.parser.class {Parser}

%%

exp:;



*** Bison is now relocatable



If you pass '--enable-relocatable' to 'configure', Bison is relocatable.



A relocatable program can be moved or copied to a different location on

the file system. It can also be used through mount points for network

sharing. It is possible to make symbolic links to the installed and moved

programs, and invoke them through the symbolic link.



*** %expect and %expect-rr modifiers on individual rules



One can now document (and check) which rules participate in shift/reduce

and reduce/reduce conflicts. This is particularly important GLR parsers,

where conflicts are a normal occurrence. For example,



%glr-parser

%expect 1

%%



...



argument_list:

arguments %expect 1

| arguments ','

| %empty

;



arguments:

expression

| argument_list ',' expression

;



...



Looking at the output from -v, one can see that the shift-reduce conflict

here is due to the fact that the parser does not know whether to reduce

arguments to argument_list until it sees the token _after_ the following

','. By marking the rule with %expect 1 (because there is a conflict in

one state), we document the source of the 1 overall shift-reduce conflict.



In GLR parsers, we can use %expect-rr in a rule for reduce/reduce

conflicts. In this case, we mark each of the conflicting rules. For

example,



%glr-parser

%expect-rr 1



%%



stmt:

target_list '=' expr ';'

| expr_list ';'

;



target_list:

target

| target ',' target_list

;



target:

ID %expect-rr 1

;



expr_list:

expr

| expr ',' expr_list

;



expr:

ID %expect-rr 1

| ...

;



In a statement such as



x, y = 3, 4;



the parser must reduce x to a target or an expr, but does not know which

until it sees the '='. So we notate the two possible reductions to

indicate that each conflicts in one rule.



This feature needs user feedback, and might evolve in the future.



*** C++: Actual token constructors



When variants and token constructors are enabled, in addition to the

type-safe named token constructors (make_ID, make_INT, etc.), we now

generate genuine constructors for symbol_type.



For instance with these declarations



%token ':'

<std::string> ID

<int> INT;



you may use these constructors:



symbol_type (int token, const std::string&);

symbol_type (int token, const int&);

symbol_type (int token);



which should be used in a Flex-scanner as follows.



%%

[a-z]+ return yy::parser::symbol_type (ID, yytext);

[0-9]+ return yy::parser::symbol_type (INT, text_to_int (yytext);

":" return yy::parser::symbol_type (’:’);

<<EOF>> return yy::parser::symbol_type (0);



Correct matching between token types and value types is checked via

'assert'. For instance, 'symbol_type (ID, 42)' would abort (while

'make_ID (42)' would not even compile).



*** C++: Variadic emplace



If your application requires C++11 and you don't use symbol constructors,

you may now use a variadic emplace for semantic values:



%define api.value.type variant

%token <std::pair<int, int>> PAIR



in your scanner:



int yylex (parser::semantic_type *lvalp)

{

lvalp->emplace <std::pair<int, int>> (1, 2);

return parser::token::PAIR;

}



*** C++: Syntax error exceptions in GLR



The glr.cc skeleton now supports syntax_error exceptions thrown from user

actions, or from the scanner.



*** More POSIX Yacc compatibility warnings



More Bison specific directives are now reported with -y or -Wyacc. This

change was ready since the release of Bison 3.0 in September 2015. It was

delayed because Autoconf used to define YACC as `bison -y`, which resulted

in numerous warnings for Bison users that use the GNU Build System.



If you still experience that problem, either redefine YACC as `bison -o

y.tab.c`, or pass -Wno-yacc to Bison.



*** The tables yyrhs and yyphrs are back



Because no Bison skeleton uses them, these tables were removed (no longer

passed to the skeletons, not even computed) in 2008. However, some users

have expressed interest in being able to use them in their own skeletons.



** Bug fixes



*** Incorrect number of reduce-reduce conflicts



On a grammar such as



exp: "num" | "num" | "num"



bison used to report a single RR conflict, instead of two. This is now

fixed. This was the oldest (known) bug in Bison: it was there when Bison

was entered in the RCS version control system, in December 1987.



Some grammar files might have to adjust their %expect-rr.



*** Parser directives that were not careful enough



Passing invalid arguments to %nterm, for instance character literals, used

to result in unclear error messages.



** Documentation



The examples/ directory (installed in .../share/doc/bison/examples) has

been restructured per language for clarity. The examples come with a

README and a Makefile. Not only can they be used to toy with Bison, they

can also be starting points for your own grammars.



There is now a Java example, and a simple example in C based on Flex and

Bison (examples/c/lexcalc/).



** Changes



*** Parsers in C++



They now use noexcept and constexpr. Please, report missing annotations.



*** Symbol Declarations



The syntax of the variation directives to declare symbols was overhauled

for more consistency, and also better POSIX Yacc compliance (which, for

instance, allows "%type" without actually providing a type). The %nterm

directive, supported by Bison since its inception, is now documented and

officially supported.



The syntax is now as follows:



%token TAG? ( ID NUMBER? STRING? )+ ( TAG ( ID NUMBER? STRING? )+ )*

%left TAG? ( ID NUMBER? )+ ( TAG ( ID NUMBER? )+ )*

%type TAG? ( ID | CHAR | STRING )+ ( TAG ( ID | CHAR | STRING )+ )*

%nterm TAG? ID+ ( TAG ID+ )*



where TAG denotes a type tag such as ‘<ival>’, ID denotes an identifier

such as ‘NUM’, NUMBER a decimal or hexadecimal integer such as ‘300’ or

‘0x12d’, CHAR a character literal such as ‘'+'’, and STRING a string

literal such as ‘"number"’. The post-fix quantifiers are ‘?’ (zero or

one), ‘*’ (zero or more) and ‘+’ (one or more).

