Archive for April 2021
Go plugin file size optimization
Now a days most of my pet projects are developed in golang. Recently I started working on a project to read data from some sensors to do some automations at my farm. The program will be running in a raspberry pi. I designed the system to utilize plugin architecture, this approach allow me to expand the functionality with ease. When I start adding plugins, I realized that the plugin binary size is more than I expected. As we have fast internet connectivity these size will not cause much harm but when the system is deployed in a place where internet connections are really slow then the file size really matters. This blog is about how I managed to reduce the plugin file size.
If one designing a pluggable system then real care should be given to the plugin code base. Design the plugin in such a way to reduce import of packages. Golang plugins are self sufficient like any other go binary. So what ever the packages imported will add size to the plugin. To make thing more clear, I wrote a small application which loads several plugins and print hello world.
Lets write some code
Here is the sample plugin looks like
import "fmt"
type Plugin struct {
}
func (d Plugin) Write() {
fmt.Println("Hello world")
}
var P Plugin
Like above created another plugin to write hello world in German
Build the plugin
go build -buildmode=plugin -o writers/plugins/en.so writers/plugins/en/en.go
go build -buildmode=plugin -o writers/plugins/de.so writers/plugins/de/de.go
Now lets examine the size of the plugin
ls -lh ./writers/plugins/*.so
-rw-r--r-- 1 pi pi 3.3M Apr 25 13:57 ./writers/plugins/de.so
-rw-r--r-- 1 pi pi 3.3M Apr 25 13:57 ./writers/plugins/en.so
As you can see each plugin is 3.3mb.
Build again this time let’s use ldflags
go build -ldflags="-s -w" -buildmode=plugin -o writers/plugins/en.so writers/plugins/en/en.go
Check the size again
ls -lh ./writers/plugins/*.so
-rw-r--r-- 1 pi pi 3.3M Apr 25 14:28 ./writers/plugins/de.so
-rw-r--r-- 1 pi pi 2.4M Apr 25 14:28 ./writers/plugins/en.so
Building with ldflags reduces the size of en.so plugin by 1mb
Lets run upx to this binary
sudo apt-get install upx
chmod +x ./writers/plugins/en.so
upx -9 -k ./writers/plugins/en.so
Check the file size again
ls -lh ./writers/plugins/*.so
-rw-r--r-- 1 pi pi 3.3M Apr 25 14:28 ./writers/plugins/de.so
-rwxr-xr-x 1 pi pi 2.1M Apr 25 14:28 ./writers/plugins/en.so
Running upx reduce the size by 0.2mb
This is the max reduction I can get with different build optimization.
Refactor the code
This is where we need to redesign the plugins and should keep refactoring the code to reduce package imports.
Where this size of plugin comes from? Its the import of fmt package. If I comment fmt.Println and build using ldflags and running upx will reduce the plugin size to 893k
ls -lh ./writers/plugins/*.so
-rw-r--r-- 1 pi pi 3.3M Apr 25 14:49 ./writers/plugins/de.so
-rwxr-xr-x 1 pi pi 893K Apr 25 14:49 ./writers/plugins/en.so
So how to keep the file size optimum and achieve the result we need. Interface comes to our rescue.
Lets create an interface, this is just a sample code and not following any naming conventions here
type Plugger interface {
Print(a ...interface{})
}
Every plugin should now relay on this interface to print hello world. See the refactored en plugin
type Plugin struct {
}
func (d Plugin) Write(plugger writers.Plugger) {
plugger.Print("Hello world")
}
var P Plugin
Here is the method that satisfies Plugger interface. This function should be outside of plugins package
import (
"fmt"
)
type PluginUtil struct {
}
func NewPluginUtils() PluginUtil {
return PluginUtil{}
}
func (p PluginUtil) Print(a ...interface{}) {
fmt.Println(a...)
}
Check the size of plugin again
-lh ./writers/plugins/*.so
-rw-r--r-- 1 pi pi 1.5M Apr 25 15:11 ./writers/plugins/de.so
-rwxr-xr-x 1 pi pi 897K Apr 25 15:11 ./writers/plugins/en.so
Source code: https://github.com/sonyarouje/goplugin
+Sony Arouje
Sony Arouje 