Go test inside LXD container

When you’re testing software that interacts with another system, it’s common to mock all the dependencies. But let’s be real — mocks are great for simulating interactions, yet they don’t give you any guarantees that the actual integration between systems will work as expected. What is the purpose of mocking database if your code relies on the database behavior (e.g. unique index constraint)? You should always consider writing integration tests!

But writing integration tests (even small ones) can be tedious, as you need to create an isolated environment where you do a reproducible installation of all the dependencies. Luckily in Go there are Testcontainers that simplify this routine. But what if for whatever reason we can’t use this and we want to use LXD containers instead of Docker?

Test helpers to the rescue!

First we will create our LXDContainer type which takes testing.TB and holds a client that we use to talk to our LXD instance.

type LXDContainer struct {
    t      testing.TB
    client lxdc.InstanceServer
    name   string
}

Simple constructor to create and launch container with a provided image:

func NewLXDContainer(t testing.TB, name, image string) *LXDContainer {
	t.Helper()

    c, err := lxdc.ConnectLXDUnix("/var/snap/lxd/common/lxd/unix.socket", nil)
	if err != nil {
		t.Fatalf("failed to connect to LXD: %v", err)
	}

	req := lxdapi.ContainersPost{
		Name: name,
		Source: lxdapi.ContainerSource{
			Type:        "image",
			Fingerprint: image,
		},
	}

	op, err := c.CreateContainer(req)
	if err != nil {
		t.Fatalf("failed to create LXD container: %v", err)
	}

	// Wait for the operation to complete
	err = op.Wait()
	if err != nil {
		t.Fatalf("failed to create LXD container: %v", err)
	}

	container := LXDContainer{t: t, name: name, client: c}

	container.startLXDContainer()

	t.Cleanup(func() {
		container.stopLXDContainer()

		op, err := c.DeleteContainer(name)
		if err != nil {
			t.Fatalf("failed to delete LXD container: %v", err)
			return
		}

		err = op.Wait()
		if err != nil {
			t.Fatalf("failed to delete LXD container: %v", err)
		}
	})

	return &container
}

func (c *LXDContainer) startLXDContainer() {
	c.t.Helper()

	req := lxdapi.InstanceStatePut{
		Action:  "start",
		Timeout: -1,
	}

	op, err := c.client.UpdateInstanceState(c.name, req, "")
	if err != nil {
		c.t.Fatalf("failed to start LXD container: %v", err)
	}

	err = op.Wait()
	if err != nil {
		c.t.Fatalf("failed to start LXD container: %v", err)
	}
}

func (c *LXDContainer) stopLXDContainer() {
	c.t.Helper()

	req := lxdapi.InstanceStatePut{
		Action:  "stop",
		Force:   true,
		Timeout: -1,
	}

	op, err := c.client.UpdateInstanceState(c.name, req, "")
	if err != nil {
		c.t.Fatalf("failed to stop LXD container: %v", err)
	}

	err = op.Wait()
	if err != nil {
		c.t.Fatalf("failed to stop LXD container: %v", err)
	}
}

Since we want containers for the integration tests, we probably want to have our external applications installed inside the container. We can do it by implementing something like Exec method below, or we can use LXD profiles or cloud-init scripts for this.

// Exec will execute provided command inside container
func (c *LXDContainer) Exec(command []string) {
	c.t.Helper()

	// Setup the exec request
	req := lxdapi.InstanceExecPost{
		Command:   command,
		WaitForWS: true,
	}

	args := lxdc.InstanceExecArgs{
		Stdin:  os.Stdin,
		Stdout: os.Stdout,
		Stderr: os.Stderr,
	}

	op, err := c.client.ExecInstance(c.name, req, &args)
	if err != nil {
		c.t.Fatalf("failed to execute command: %v", err)
	}

	// Wait for it to complete
	err = op.Wait()
	if err != nil {
		c.t.Fatalf("failed to execute command: %v", err)
	}
}

Now we have all the components in place, and we can use our new test helper in our tests. Running go test ./... -run TestExample -v will talk to your local LXD installation, start a container, execute whoami inside the container and then cleanup created resources.

func TestExample(t *testing.T) {
	// t.Name() returns name that is not compatible with LXD
	name := strings.ReplaceAll(t.Name(), "/", "-")
	container := lxdtest.NewLXDContainer(t, name, "jammy")
	container.Exec([]string{"whoami"})
}

Happy testing, and hopefully, it will make writing integration tests a bit more enjoyable for you.

Until next time!

P.S. please don’t use build tags for integration tests.