Go

Let's build a Go application using the TEN framework.

Creating the TEN App

First, we'll create a basic TEN Go app by integrating several pre-built TEN packages. Follow these steps:

>_ Terminal

tman install app default_app_go
cd default_app_go

tman install protocol msgpack
tman install extension_group default_extension_group

Installing a Default TEN Extension

Next, install a default TEN extension written in Go:

>_ Terminal

tman install extension default_extension_go

Declaring the Prebuilt Graph for Auto-Start

Now, we'll modify the property.json file of the TEN app to include a graph declaration. This will ensure the default extension starts automatically when the TEN app is launched.

.json

"predefined_graphs": [
  {
    "name": "default",
    "auto_start": true,
    "nodes": [
      {
        "type": "extension_group",
        "name": "default_extension_group",
        "addon": "default_extension_group"
      },
      {
        "type": "extension",
        "name": "default_extension_go",
        "addon": "default_extension_go",
        "extension_group": "default_extension_group"
      }
    ]
  }
]

Building the App

Unlike standard Go projects, the TEN Go app uses CGo, so you need to set up certain environment variables before building. A build script is already provided in the TEN runtime Go binding system package, so you can build the app with a single command:

>_ Terminal

go run ten_packages/system/ten_runtime_go/tools/build/main.go

The compiled binary, main, will be generated in the /bin folder.

Starting the App

Since some environment variables need to be set, it is recommended to start the app using the provided script:

>_ Terminal

./bin/start

Debugging

If you are using Visual Studio Code (VSCode) as your development environment, you can use the following configurations to debug Go/C code.

Debugging Go Code

.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Go app (launch)",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "${workspaceFolder}",
            "cwd": "${workspaceFolder}",
            "output": "${workspaceFolder}/bin/tmp",
            "env": {
                "LD_LIBRARY_PATH": "${workspaceFolder}/lib",
                "DYLD_LIBRARY_PATH": "${workspaceFolder}/lib",
                "CGO_LDFLAGS": "-L${workspaceFolder}/lib -lten_runtime_go -Wl,-rpath,@loader_path/lib"
            }
        }
    ]
}

Debugging C Code

.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "C (launch)",
            "type": "lldb",
            "request": "launch",
            "program": "${workspaceFolder}/bin/main",
            "cwd": "${workspaceFolder}",
            "env": {
                "LD_LIBRARY_PATH": "${workspaceFolder}/lib",
                "DYLD_LIBRARY_PATH": "${workspaceFolder}/lib",
                "CGO_LDFLAGS": "-L${workspaceFolder}/lib -lten_runtime_go -Wl,-rpath,@loader_path/lib"
            }
        }
    ]
}

CGO

Generated Code

When interfacing Go with C, the cgo tool is crucial. When a C function is called from Go, cgo converts the Go source files into multiple output files, including both Go and C source files. These generated C source files are then compiled using compilers like gcc or clang.

Use the following command to generate these C/Go source files:

>_ Terminal

mkdir out
go tool cgo -objdir out

Example:

>_ Terminal

go tool cgo -objdir out cmd.go error.go

The generated files include:

.bash

├── _cgo_export.c
├── _cgo_export.h
├── _cgo_flags
├── _cgo_gotypes.go
├── _cgo_main.c
├── _cgo_.o
├── cmd.cgo1.go
├── cmd.cgo2.c
├── error.cgo1.go
└── error.cgo2.c

• _cgo_export.h is key to the interoperability between Go and C within the cgo context. It contains necessary declarations for Go functions accessible from C.

  This file is automatically generated based on Go functions explicitly exported using the //export directive. It also includes definitions for C types corresponding to the Go types used in these functions, ensuring compatibility between the two languages.

  Example of type definitions:

.h

typedef signed char GoInt8;
typedef unsigned char GoUint8;
typedef short GoInt16;
typedef unsigned short GoUint16;
typedef int GoInt32;
typedef unsigned int GoUint32;
typedef long long GoInt64;
typedef unsigned long long GoUint64;
typedef GoInt64 GoInt;
typedef GoUint64 GoUint;
typedef size_t GoUintptr;
typedef float GoFloat32;
typedef double GoFloat64;

• _cgo_gotypes.go contains the corresponding Go types defined in C and used in Go.

  For example, if you define a struct ten_go_error_t in a header file common.h and use C.ten_go_error_t in Go, there will be a corresponding Go type in _cgo_gotypes.go:

.go

package ten

type _Ctype_ten_go_status_t = _Ctype_struct_ten_go_status_t

type _Ctype_struct_ten_go_status_t struct {
    err_no   _Ctype_int
    msg_size _Ctype_uint8_t
    err_msg  [256]_Ctype_char
    _        [3]byte
}

• cmd.cgo1.go is a cgo-generated counterpart to the original Go source file cmd.go. It replaces direct calls to C functions and types with calls to the generated Go functions and types provided by cgo.

  Example:

.go

package ten

func NewCmd(cmdName string) (Cmd, error) {
    // ...
    cStatus := func() _Ctype_struct_ten_go_status_t {
        var _cgo0 _cgo_unsafe.Pointer = unsafe.Pointer(unsafe.StringData(cmdName))
        var _cgo1 _Ctype_int = _Ctype_int(len(cmdName))
        _cgoBase2 := &bridge
        _cgo2 := _cgoBase2
        _cgoCheckPointer(_cgoBase2, 0 == 0)
        return _Cfunc_ten_go_cmd_create_custom_cmd(_cgo0, _cgo1, _cgo2)
    }()
    // ...
}

• cmd.cgo2.c is a wrapper of the original C function called from Go.

  Example:

.c

CGO_NO_SANITIZE_THREAD void _cgo_cb1b98e39356_Cfunc_ten_go_cmd_create_custom_cmd(void *v) {
    struct {
        const void* p0;
        int p1;
        char __pad12[4];
        ten_go_msg_t** p2;
        ten_go_error_t r;
    } __attribute__((__packed__, __gcc_struct__)) *_cgo_a = v;
    char *_cgo_stktop = _cgo_topofstack();
    __typeof__(_cgo_a->r) _cgo_r;
    _cgo_tsan_acquire();
    _cgo_r = ten_go_cmd_create_cmd(_cgo_a->p0, _cgo_a->p1, _cgo_a->p2);
    _cgo_tsan_release();
    _cgo_a = (void*)((char*)_cgo_a + (_cgo_topofstack() - _cgo_stktop));
    _cgo_a->r = _cgo_r;
    _cgo_msan_write(&_cgo_a->r, sizeof(_cgo_a->r));
}

So, the calling sequence of C.ten_go_cmd_create_cmd() from Go is:

_Cfunc_ten_go_cmd_create_custom_cmd (auto-generated Go)
           |
           V
   _cgo_runtime_cgocall (Go)
           |
           V
      entrysyscall     // switch Go stack to C stack
           |
           V
_cgo_cb1b98e39356_Cfunc_ten_go_cmd_create_custom_cmd (auto-generated C)
           |
           V
   ten_go_cmd_create_cmd (C)
           |
           V
      exitsyscall

Incomplete Type

As mentioned earlier, the cgo tool generates corresponding Go types in _cgo_gotypes.go based on C header files imported via import "C". If a struct is opaque in the C header, the cgo tool generates an incomplete type instead.

Example of an opaque C struct:

.h

typedef struct ten_go_msg_t ten_go_msg_t;

The cgo tool will generate an incomplete type in Go:

.go

type _Ctype_ten_go_msg_t = _Ctype_struct_ten_go_msg_t
type _Ctype_struct_ten_go_msg_t _cgopackage.Incomplete

What happens if you use the incomplete type in Go?

• Incomplete types cannot be allocated.

  Since sys.NotInHeap cannot be allocated on the Go heap, operations like new or make won't work. Attempting to create a new instance of an opaque struct in Go will result in a compiler error:

.go

msg := new(C.ten_go_msg_t) // Error: can't be allocated in Go; it is incomplete (or unallocatable)

• Pointers to incomplete types cannot be passed to C directly.

  If you have a C function with a pointer to an opaque struct as a parameter, passing a Go pointer to this incomplete type directly to a C function will not work according to cgo rules. The Go compiler will require the pointer to be "pinned" to ensure it adheres to Go's garbage collector (GC) constraints.

Rules for Using C.uintptr_t Instead of a Pointer to an Opaque Struct

• Benefits:

  • C.uintptr_t and uintptr in Go are integers large enough to hold a C pointer, avoiding memory allocation or conversion when passing from Go to C.

• Limitations:

  • uintptr is an integer, not a pointer. It represents the bit pattern of an address with no pointer arithmetic.

  • uintptr cannot be dereferenced in Go. Converting it to unsafe.Pointer can cause issues with Go's GC.

  • Since uintptr is an integer, nil or NULL cannot be passed to C. Use 0 instead of nil to represent a null address.