Running C code in Go without Cgo using WASM: Part 1
In this series, we're going to explore compiling and running C programs in Go without Cgo. We'll first learn the basic process, building up to more complex examples over time. By the end, we'll have the QuickJS Javascript interpreter running inside a Go server without Cgo.
What's wrong with Cgo?
If you've been around the Go ecosystem long enough, you've inevitably stepped outside of Go's Garden of Eden and fallen off the Cgo cliff. The moment you import "C" you lose Go's cross-compilation support, your builds become slower and Go's built-in performance tools can no longer see the full picture. What's worse is that Cgo can't be isolated in a module somewhere, if you or any of your dependencies import "C", Cgo kicks in. Cgo infects your codebase.
Losing cross-compilation is a big problem for library authors. We want our libraries to be as easy as possible to install and consume. We don't want our library being the reason you can't cross-compile or profile your application.
I collided head-first with this problem while trying to support server-side Javascript rendering (SSR) in Go for an upcoming web framework. There's many ways to support SSR in Go, but most of them involve Cgo. One of the challenges of supporting SSR is that it needs to execute at runtime. I didn't want people to face the Cgo cliff every time they deploy to production. This problem has been bugging me for about a year now. Then I stumbled upon https://www.figma.com/blog/an-update-on-plugin-security/.
WAT? WASM?
Figma is using QuickJS compiled into WASM to power their plugin system. Figma's plugin system needs to be able to safely execute arbitrary code and WASM provides the sandbox capabilities to do that. This solution got my brain buzzing. They've already proven that QuickJS ⇒ WASM is possible and I've seen WASM VMs written in pure Go like Wagon, Life and Gasm. Can we get QuickJS running inside Go via WASM?
Here's what this looks like:
{
"id": "4ddd218a-2b50-44b0-9eea-9c974cdb7116",
"type": "image",
"url": "https://prod-files-secure.s3.us-west-2.amazonaws.com/a0e28483-01ab-4009-81c1-3139fd1424a5/807136d9-3eb5-44d0-8275-ec00fcc44b1b/CleanShot_2021-01-03_at_17.19.402x.png?X-Amz-Algorithm=AWS4-HMAC-SHA256\u0026X-Amz-Content-Sha256=UNSIGNED-PAYLOAD\u0026X-Amz-Credential=ASIAZI2LB466ZIBOAADB%2F20260601%2Fus-west-2%2Fs3%2Faws4_request\u0026X-Amz-Date=20260601T054306Z\u0026X-Amz-Expires=3600\u0026X-Amz-Security-Token=IQoJb3JpZ2luX2VjEDkaCXVzLXdlc3QtMiJGMEQCIAIVwJF6cZb%2FyC2yZLG6v7JcRYHrzSlqDwyuApDbQ%2FbvAiBXQTw9BjnC%2BFO91qUSq7URMoZp0t8Ad8svqAmSGscDKyr%2FAwgCEAAaDDYzNzQyMzE4MzgwNSIM8zmXekVvfIKFSl6MKtwD9VMaCzLe6gFO%2FWHVsS3iO5xVRkwsNl6n%2BN0TkkqFc%2F%2Fo1WoVJY0gQiaZKDSpoYcRJ3BivFzzGOpiI2rNJ7kQy5qF3K0JhkpOOATcFUV7JmKU4H%2BdXhQQDLPJX5hdSGvetFShmyQP4%2FQV9CxWVoBet%2ByhVEYYl1%2FpW5sM9kn%2F28VtnROBTfofVqTrScBv2rPx%2FgHB8qa%2B37HMznrGx6NaaVQyvvEylck8rSp2QaUETYnhp6Kchwr9GGhhexd71SJHGKr5HQS8YEL32YLoPmWHCIlRdnm1zEQJO82XKoEC%2FfesFiBxJ3IXKxFYh95JY%2FaaWDMZCg51CZdp5i76UaNgxi7rh0TWoVpxbbFVEUZYVV%2B3Ipql7yMCQBcT8XZEHM6pdW7221GOs9bIRFjDsrB03LTAiK4id0CFCgoECwENQMVuvs6%2Bezkq2OkXpapsj%2BQGDoa0brRRoxARYFF7IYP0fPn0O1utJKF6VrHV8%2BuKGA4RjoiIMXHwfZgcggK3a4Ca%2BJzFxIduB41oMy3foPiSsNShXSHEn3b6YJ21%2Fm0oEqJ0fCRi3Nq%2BC8zNWrQwiOO1nlfSK2AGDT32FZTerwjF0laCHijcW9hb0mL1%2BcROwlS0w%2F0h59X5pj0qt3kwu6Xz0AY6pgGz24U8sassnmcY688AasZf1EG0Es9BKFj36Z6ezvwMg9UHMlyDa5GN%2BYmd9Md7WbZ7XnArKZgAe0HOL5zmYOzL7tL9FYeeAB6wvqFWiN4xBfujp15XTvYxsFAQtlRkqvjQu2eGkmUMzXTrc1FWNhtoYXucluDKoWZpQpZh2nPGy1WBGhofp2W9xt4F6ObbevE5aPoIkBXz8STQGW63MjIiFG3rJHU8\u0026X-Amz-Signature=7308b98679028805ecf858a1dc5b98eab195d104ad547676f3c4f6b1e0c4db9b\u0026X-Amz-SignedHeaders=host\u0026x-amz-checksum-mode=ENABLED\u0026x-id=GetObject"
}The first step is to compile QuickJS to WASM. Then we'll load that WASM module into a WASM VM. Next we'll pass Javascript source code into that VM and if all goes well, the WASM module will spit out HTML.
This journey will have some side quests as we learn more about how WASM works, so buckle up.
Step 1: quickjs.c to WASM
The first step is to get quick.c compiled to WASM. This turned out to be pretty easy. The wonderful @maple3142 already laid the groundwork for us in github.com/maple3142/wasm-jseval (The example below diverges from the source code slightly)
{
"id": "3f108491-3df9-4744-b3e7-0d8abae3e230",
"type": "code",
"content": [
{
"type": "text",
"content": "// quick.c\n#include \u003cstring.h\u003e\n#include \"quickjs/quickjs.h\"\n\nconst char* eval(const char* str) {\n\tJSRuntime* runtime = JS_NewRuntime();\n\tJSContext* ctx = JS_NewContext(runtime);\n\tJSValue result =\n\t JS_Eval(ctx, str, strlen(str), \"\u003cevalScript\u003e\", JS_EVAL_TYPE_GLOBAL);\n\tif (JS_IsException(result)) {\n\t\tJSValue realException = JS_GetException(ctx);\n\t\treturn JS_ToCString(ctx, realException);\n\t}\n\tJSValue json = JS_JSONStringify(ctx, result, JS_UNDEFINED, JS_UNDEFINED);\n\tJS_FreeValue(ctx, result);\n\treturn JS_ToCString(ctx, json);\n}"
}
],
"language": "c"
}{
"id": "a08602f0-89f8-449b-a209-17d9b4831134",
"type": "code",
"content": [
{
"type": "text",
"content": "emcc -o quick.js quick.c quickjs/quickjs.c quickjs/cutils.c quickjs/libregexp.c quickjs/libbf.c quickjs/libunicode.c -DCONFIG_VERSION=\"\\\"1.0.0\\\"\" -s WASM=1 -s SINGLE_FILE -s MODULARIZE=1 -lm -Oz -s EXPORTED_FUNCTIONS='[\"_eval\"]' -s EXTRA_EXPORTED_RUNTIME_METHODS='[\"ccall\", \"cwrap\"]' --llvm-lto 3 -s AGGRESSIVE_VARIABLE_ELIMINATION=1 --closure 1"
}
],
"language": "javascript"
}This command worked the first time I ran it. How often does that happen?
This particular command spits out javascript, but it's easy enough to get it to spit out WASM directly. Just change -o quick.js to -o quick.wasm.
Step 2: Load our WASM module into a WASM VM
{
"id": "2a6a1348-de36-485f-a216-a0f61c542b95",
"type": "code",
"content": [
{
"type": "text",
"content": "// add.c\n#include \u003cstdint.h\u003e\n\nuint64_t add(uint64_t a, uint64_t b)\n{\n return a + b;\n}"
}
],
"language": "c"
}{
"id": "3effdb35-ea49-4e53-9e72-08d3423a5e24",
"type": "code",
"content": [
{
"type": "text",
"content": "emcc add.c -o add.wasm -s WASM=1 -s \"EXPORTED_FUNCTIONS=['_add']\" --no-entry"
}
],
"language": "c"
}{
"id": "e8e9c673-55a1-4982-8c12-12b7b875c24e",
"type": "code",
"content": [
{
"type": "text",
"content": "package main\n\nimport (\n\t\"bytes\"\n\t\"encoding/binary\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"os\"\n\t\"reflect\"\n\n\t\"github.com/mathetake/gasm/hostfunc\"\n\t\"github.com/mathetake/gasm/wasm\"\n)\n\nfunc main() {\n\tbuf, err := ioutil.ReadFile(\"add.wasm\")\n\tif err != nil {\n\t\tlog.Fatal(\"error reading file: \", err)\n\t}\n\tmod, err := wasm.DecodeModule(bytes.NewBuffer(buf))\n\tif err != nil {\n\t\tlog.Fatal(\"error decoding module: \", err)\n\t}\n\t// Build the modules\n\tb := hostfunc.NewModuleBuilder()\n\tmodules := b.Done()\n\t// Create the new VM\n\tvm, err := wasm.NewVM(mod, modules)\n\tif err != nil {\n\t\tlog.Fatal(\"error creating vm: \", err)\n\t}\n\tret, _, err := vm.ExecExportedFunction(\"add\", 10, 20)\n\tif err != nil {\n\t\tlog.Fatal(\"error running exec: \", err)\n\t}\n\tfmt.Println(ret[0])\n}"
}
],
"language": "go"
}{
"id": "873b9d6c-8018-4b93-925d-515870025f75",
"type": "code",
"content": [
{
"type": "text",
"content": "GOOS=linux GOARCH=amd64 go build eval/main.go\ndocker run -v $(PWD):/src -w /src -t alpine ./main\n30"
}
],
"language": "shell"
}{
"id": "fb775323-c151-4d2c-8930-8c381512d395",
"type": "code",
"content": [
{
"type": "text",
"content": "wasm-objdump -x --section=Function add.wasm\n\nadd.wasm: file format wasm 0x1\n\nSection Details:\n\nFunction[18]:\n - func[0] sig=1\n - func[1] sig=6 \u003cadd\u003e\n - func[2] sig=1 \u003c_initialize\u003e\n - func[3] sig=1 \u003cemscripten_stack_init\u003e\n - func[4] sig=0 \u003cemscripten_stack_get_free\u003e\n - func[5] sig=0 \u003cemscripten_stack_get_end\u003e\n - func[6] sig=0 \u003cstackSave\u003e\n - func[7] sig=2 \u003cstackRestore\u003e\n - func[8] sig=3 \u003cstackAlloc\u003e\n - func[9] sig=3\n - func[10] sig=2\n - func[11] sig=2\n - func[12] sig=2\n - func[13] sig=0\n - func[14] sig=1\n - func[15] sig=3 \u003cfflush\u003e\n - func[16] sig=3\n - func[17] sig=0 \u003c__errno_location\u003e"
}
],
"language": "shell"
}{
"id": "ddfbfdc7-eaca-4b13-843d-2bf2cd18f866",
"type": "code",
"content": [
{
"type": "text",
"content": "wasm-objdump -x --section=Import hello.wasm\n\nhello.wasm: file format wasm 0x1\n\nSection Details:\n\nImport[4]:\n - func[0] sig=5 \u003cwasi_snapshot_preview1.args_sizes_get\u003e \u003c- wasi_snapshot_preview1.args_sizes_get\n - func[1] sig=5 \u003cwasi_snapshot_preview1.args_get\u003e \u003c- wasi_snapshot_preview1.args_get\n - func[2] sig=3 \u003cwasi_snapshot_preview1.proc_exit\u003e \u003c- wasi_snapshot_preview1.proc_exit\n - func[3] sig=14 \u003cwasi_snapshot_preview1.fd_write\u003e \u003c- wasi_snapshot_preview1.fd_write"
}
],
"language": "shell"
}https://github.com/WebAssembly/WASI/blob/master/phases/snapshot/docs.md
That wraps up Part 1! In Part 2, we'll learn how to initialize memory and pass strings into and out of our WASM module. If you'd like to get notified when Part 2 is out, you can join the mailing list below. No spam, I promise.
{
"id": "7d3a8eea-16b7-4ca5-8c33-f4d1f0593161",
"type": "callout",
"icon": {
"type": "emoji",
"emoji": "📩"
},
"content": [
{
"type": "text",
"content": "email "
}
],
"styles": [
{
"type": "background",
"color": "gray"
}
]
}Footnotes
{
"type": "unordered-list",
"id": "c6a2dd6a-f508-4158-a7aa-969a374b674d",
"items": [
{
"id": "c6a2dd6a-f508-4158-a7aa-969a374b674d",
"content": [
{
"type": "link",
"content": "github.com/maple3142/wasm-jseval",
"url": "http://github.com/maple3142/wasm-jseval"
}
]
},
{
"id": "920418e1-077c-4c7f-a305-5de03fb88f12",
"content": [
{
"type": "link",
"content": "https://github.com/justjake/quickjs-emscripten",
"url": "https://github.com/justjake/quickjs-emscripten"
}
]
}
]
}{
"id": "6bd8506b-ac4e-4573-96fb-a019ce284087",
"type": "divider"
}Now, if you're just building a Go binary to distribute, losing cross-compilation isn't the end of the world. We have great tools like https://github.com/techknowlogick/xgo for compiling inside a Docker container for your target platform and Github Actions make building binaries across platforms easier than ever.