Go-li language user manual

func Compar(a, b *) int

func Transpose(matrix [], stride int) (o [], newstride int)

func Sort(compar func(*,*), slice [])

Introduction

Go Li also known as go language improved is a prototype programming language to test a limited “generics” features added to go language. It allows to use parameterized functions also known as generic functions. Each such a function is parametrized by exactly one type, known as the wildcard type.

Installation

Visit http://go-li.github.io/xgtest.html# to use an online playground. Or download a source to source translator from the url https://github.com/go-li/transpiler. This translator is able to translate source code packages from go-li to golang. It is invoked ./transpiler -P /source/arbitrary/package/directory -T /destination/package/directory.

Alternatively, you can try the experimental gccgo based compiler below.

Basic usage

package main



import "fmt"



func reverse(args ...) [] {

for i := 0; i < len(args)/2; i++ {

args[i], args[len(args)-1-i] = args[len(args)-1-i], args[i]

}

return args

}



func main(){

fmt.Println(reverse("World", "Hello"))

fmt.Println(reverse(1,2,3,4,5))



}

In this example we demonstrate a reverse(args ...) generic function. Reverse is a generic function, because it has a parameter of a generic type.

What is a generic type? What generic types exist?

Generic type is any type that contains the omitted type placeholder, or one that contains another generic type. The following table lists examples of commonly used generic types:

Type Explanation Reason why generic * Generic pointer to wildcard type Has omitted type placeholder [] Generic slice of wildcard types Has omitted type placeholder ... Generic varargs slice Has omitted type placeholder map[]int Generic key to integer map Has omitted type placeholder map[int] Map containing generic values Has omitted type placeholder map[] Map with same key and value type Has omitted type placeholders chan Generic channel Has omitted type placeholder type T * Named generic type T Contains omitted type placeholder struct { A * } Generic struct containing a field with generic pointer type Contains another generic type func (a *) Generic callback signature Contains generic type param/ret func ()(a *) Generic callback signature Contains generic type param/ret interface { Foo()(*) } Generic interface with generic method Foo with generic pointer result type Contains generic method Foo that has a generic signature due to generic pointer function result.

Any named type with underlying generic type is also a generic type.

What is a generic function or method?

Each generic function must satisfy the following two rules:

It must be toplevel (in the file scope). Generic closures are invalid. It must have at least one parameter with generic type . (Generic result is not sufficient.)

The reason for the first rule is that generic closures are currently disallowed because generic closures are rarely if never useful. The reason for the second rule is that it must be possible to derive the wildcard type from every single callsite. Without a generic function parameter, this is impossible. For this reason top-level functions having only generic typed results and no generic typed parameters are invalid. The following table lists several examples of generic and non generic top-level functions together with validity:

Function Genericity Validity 1 func Add1(a int) int { return a + 1; } not generic valid 2 func passPtr(a *) * { return a; } generic valid 3 func returnEmptyGslice() [] { return nil; } not generic invalid 4 func tryGclosure() { func(*){}((*int)(nil)); } not generic invalid

Suppose we want to return nil slice of arbitrary type? How do we fix the function 3? It’s simple, we add some generic typed parameter. When we call the function, we can pass a typed nil.

package main func returnEmptyGslice(_ *) [] { return nil; } func main() { intslice := returnEmptyGslice((*int)(nil)) // wildcard is int byteslice := returnEmptyGslice((*byte)(nil)) // wildcard is byte println(len(intslice)) println(len(byteslice)) }

Calling top-level generic functions

When calling top-level generic function, for every call site, there are these rules:

It must be possible to derive the wildcard type It must be possible to derive the concrete named return types for every generic named return type (if any). There can not be any wildcard type conflict (with the exception of interface{} wildcard and concrete varargs passed as … typed variadic parameters )

In the following wildcard type derivation examples, we are going to call this top-level function:

func Wildcards(a *, b func(*,*), c []) { }

We are going to call Wildcards function using these callsites. We assume the function Wildcards is called from a non-generic function. Validity and explanation why follow the call site code:

Call site Validity Explanation 1 Wildcards(nil, nil, nil) invalid Untyped nil derived from all args 2 Wildcards((*int)(nil), nil, nil) valid Wildcard is int (from param a) 3 Wildcards(nil, func(*int,*int){}, nil) valid Wildcard is int (from param b) 4 Wildcards(nil, nil, []byte{}) valid Wildcard is byte (from param c) 5 Wildcards((*int)(nil), nil, ([]byte)(nil)) invalid Wildcard is int (from a) or byte (from c) [rule 3]

What follows is we are going to call top-level generic function Varargs.

func Varargs(a *, b ...) { }

Call site Validity Explanation 6 Varargs(nil, 0) valid Wildcard is int (from param b) 7 Varargs(nil, 0, “hello”) invalid Wildcard is int (from b) or string (from b) [rule 3] 8 Varargs((*int)(nil)) valid Wildcard is int (from param a) 9 Varargs(nil) invalid Untyped nil derived from a [rule 1] 10 Varargs((interface{})(nil), 0, “hello”) valid Wildcard is interface{} (from a) and interface{} (from b) [rule 3]

Generic named type callsites with concrete type mappings

It’s possible to pass named concrete types as named generic parameters. This demonstrates this feature. In this scenario, from argument x and parameter a it can be derived that the type Tint is the concrete type that implements the generic type T

package main type T * type Tint *int func accept(a T) T { return a } func main() { var x Tint x = accept(x) }

The following demo is invalid. The translator has no way to know what concrete type should the variable y in main have. In other words, the possibility that generic type Y shall become concrete type Yint is unrelated to the fact that the generic type X becomes the concrete type Xint.

// This demo does not compile package main type X * type Y * type Xint *int type Yint *int func reject(a X) Y { return a } func main() { var x Xint y := reject(x) }

Operations allowed in generic functions

Examples of compile time type safe operations

Operation Operation name Type of a Type of b *a = *b Copy generic value * * *a, *b = *b, *a Swap generic value * * *a = b[0] Copy slice value * [] a = &b[0] Set pointer to a slice value * [] a = b Copy slice header [] [] a[4] = b[5] Copy slice value [] [] a = make([], b) Make a generic slice [] int a = b[4:5] Reslice a generic slice [] [] _ = a[*b] Wildcard keyed map lookup map[]bool * a = make(map[int], b) Make a wildcard valued map map[int] int *a = b[5] Copy map value to ptr elem. * map[int] a(b, nil) Run generic callback or func func(*,*) * a = b((*)(nil)) Pass wildcard type to call * func(*)(*) a <- *b Send on generic chan chan * a = b copy variable to interface{} interface{} Any generic

Examples of run time “type safe” operations

Operation Why not compile time safe Type of a Type of b a = make(map[]bool, b) Panic due incomparable key map[]bool int ((interface{})(a)).(*int) a might point to non-int type *

So what can we use this “generics” for

This is not a well studied area. A surprising uses of this mechanism could be discovered. Let’s first try to summarise what is known to be possible today. By possible we mean it’s done in a compile time type safe way.

Sorting of slices. Binary heap built from a slice. Byte array keyed arbitrary valued sorted sets (AVL tree). Max(a,b), Min(a,b) functions for arbitrary types. Double slice arbitrary keyed arbitrary valued hashtable that is only 20% slower than builtin map. Type safe doubly linked list. Leaky pool. Ring buffer “queue” made from a slice. Map[foo]struct{} set. float64/float32 agnostic math functions like transpose, matrix multiply by constant.

Not compile time type safe:

Deduplicate a slice by throwing things into a map. Convert a slice using function to arbitrary another type slice.

Unknown:

Call traits. Graphs for instance immediate dominator. Bimap. Multiple indexes over a collections. data accessors in Dietmar Kühl's Masters Thesis on generic graph algorithms. Caches. Compile-time type-safe sync.Pool. Set operations over collections.

Errors and solutions

undetermined wildcard from missing varargs in call to foo

You have a varargs only function like:

func foo(args ...) { }

You can’t call it like: foo()

You need to provide at least one argument from which the wildcard type can be determined. For instance you can pass typed nil to the function: foo((*int)(nil)) . Or you can pass some concrete (the same) types, for instance foo("a","b","c") . Or foo(1,2,3)

cannot convert nil (untyped nil value) to untyped void

You can’t call it like: foo(nil)

You need to provide some argument from which the wildcard type can be determined. For instance you can pass typed nil to the function: foo((*int)(nil))

wildcard cannot be untyped nil (untyped nil value)

You have a generic pointer accepting function like:

func accept(ptr *) { }

You can’t call it like: accept(nil)

You need to provide some argument from which the wildcard type can be determined. For instance you can pass typed nil to the function: accept((*int)(nil))

GCCGO-li user manual

Installation - x86-64 Linux only

1. Download the gccgo-8.0.1-006.tar.xz from the generics slack chat channel. The URL of the channel is: https://gophers.slack.com/messages/C88U9BFDZ/

2. After downloading, verify the authenticity of the file.

$ sha256sum gccgo-8.0.1-006.tar.xz

337c04797b7c2c0fc7e586b309fad19cac864dfd9f7ef4420f0f427123db3ccd gccgo-8.0.1-006.tar.xz

3. Unpack the archive ether from the file browser or using command:

$ tar -xf gccgo-8.0.1-006.tar.xz

You can unpack it to any location. But the rest of the guide assumes you moved the unpacked gccgo folder to /opt. The file structure should be like this:

/opt/ ├── gccgo │ ├── bin │ │ ├── c++ │ │ ├── cpp │ │ ├── g++ │ │ ├── gcc │ │ ├── gcc-ar ...

Now it’s installed. You can skip to testing the installation.

Installation from source

You need gcc source from gcc.gnu.org, and the generics patches from github.

For example I’ve downloaded gcc-8-20180325.tar.xz and these patches:

https://github.com/go-li/gofrontend/commit/1f227e14dd73acff48a8909bbef0ff4b9da73f53.patch

https://github.com/go-li/gofrontend/commit/b448b08e273af75ffd007b76230306c88d9e6f50.patch

You also need the backend patch, known as patch zero.patch:

https://pastebin.com/raw/VhNtCvbY

Create a folder gccgo and inside a folder emptydir and gofrontend. Unpack the gcc archive to emptydir. In gofrontend create two folders go and libgo. Put the patches to gofrontend.

This guide assumes you are building in /tmp/gccgo. If you plan to shut down your computer, pick an alternate location for example in your home folder.

/tmp/gccgo/ ├── emptydir │ ├── gcc-8-20180325 │ ... ├── gofrontend │ ├── go │ ├── libgo │ ├── arrayaccess.patch │ ├── generics.patch │ ├── zero.patch ...

Copy over everything from /tmp/gccgo/emptydir/gcc-8-20180325/gcc/go/gofrontend/ to /tmp/gccgo/gofrontend/go/. Also copy everything from /tmp/gccgo/emptydir/gcc-8-20180325/libgo/ to /tmp/gccgo/gofrontend/libgo/

Now run this in the gofrontend folder to apply the changes from github:

patch -p1 < arrayaccess.patch patch -p1 < generics.patch

In folder emptydir run this recipe

cd gcc-8-20180325/ patch -p1 < ../../gofrontend/zero.patch rm -rf gcc/go/gofrontend ln -s /tmp/gccgo/gofrontend/go gcc/go/gofrontend mkdir libgotmp/ mv libgo/go libgotmp/ mv libgo/mics libgotmp/ mv libgo/runtime libgotmp/ rm -rf libgo mkdir libgo/ mv libgotmp/go libgo/ mv libgotmp/mics libgo/ mv libgotmp/runtime libgo/ mkdir libgo/testsuite/ mkdir libgo/config/ cd libgo/testsuite for f in /tmp/gccgo/gofrontend/libgo/testsuite/*; do ln -s $f `basename $f`; done cd .. cd config/ for f in /tmp/gccgo/gofrontend/libgo/config/*; do ln -s $f `basename $f`; done cd .. for f in /tmp/gccgo/gofrontend/libgo/*; do ln -s $f `basename $f`; done cd .. cd ..

Now, in emptydir, you can configure.

CFLAGS='-O3' LDFLAGS='-s -w -O3' CXXFLAGS='-O3' gcc-8-20180325/configure --prefix=/opt/gccgo --disable-bootstrap --disable-libgomp --without-ppl --without-isl --without-cloog --disable-libada --disable-multilib --enable-__cxa_atexit --enable-gnu-indirect-function --enable-languages=c,c++,go --disable-libsanitizer

Now after configure went ok, run make (this could take up to two hours!!)

make -j 4

Once that’s succeeded, run sudo make install. Your gccgo will appear in /opt/gccgo/

Testing

You can download basic test programs to determine if the generics work. These programs are available at https://github.com/go-li/demo.

$ cd /tmp/ $ git clone https://github.com/go-li/demo.git Cloning into 'demo'... remote: Counting objects: 169, done. remote: Compressing objects: 100% (3/3), done. remote: Total 169 (delta 0), reused 1 (delta 0), pack-reused 166 Receiving objects: 100% (169/169), 50.62 KiB | 0 bytes/s, done. Resolving deltas: 100% (81/81), done. Checking connectivity... done. $ /opt/gccgo/bin/gccgo demo/bubblesort.go -static-libgo $ ./a.out 47,35,7,5,3,

This tested the bubblesort demo. Other demos from this repo should work as well.

Gccgo bugs

This gccgo frontend has many bugs. However this does not necessarily mean that your programs will fail in production. If your program successfully compiles, it’s usually safe to run it. These bugs limit the possible generics features that you can use in your program. If you use a buggy feature, your program will fail to compile or the compiler will crash.

Varargs does not work

You cannot create and use generic varargs functions. The demo from the top of this document does not work.

Unsafe packages cannot use generics

If you import unsafe, your generic functions won’t be recognized to be generic. Therefore you should never import unsafe while using this proposal.

Import/export does not work for generics

If you create a generic function with a capitalized name, you are asking for a trouble. It will never work.

Wildcard substitution bugs

The type returned by a generic-type-result function is sometimes not the one it should be. This is because wildcard substitution is buggy. In particular parameter-related named types are not returned and not substituted into the result type. You can workaround by casting the function result variable to the right type manually. Also if you use unusual return types like a generic map it will crash because substitution is not implemented.

Only a few generic operations are supported

Only operations supported is copying wildcard-typed value using a pointer:

*dst = *src

And a generic slice index address dereference expression

pointer = &slice[5]

Furthermore you can make generic slices:

make([], 4)

Cannot assign generic variables to interface{}

This doesn’t work and never will.

Cannot cast interface{} to a generic type, like slice := foo.([])

This doesn’t work and never will.

The protection against using of generic types outside of generic context is missing

You should always first develop and test your generic program using the transpiler / playground. Only when you get no errors, you can attempt to compile using gccgo. If you use generic types outside of generic context (top-level generic function) there is no safety and no guarantees.