Control durability guarantees from Go

Golem provides a set of functions components can call to control details of the durable execution engine. For components implemented in Go the Golem Go SDK provides a set of Go wrappers for these functions.

Setting up the Golem Go SDK

If the project was created with golem-cli new, it already has the Golem Go SDK set up, otherwise follow the step at Setting up the Golem Go SDK

General concepts

The library allows controlling four main aspects of the durable execution engine: the current persistence level, the idempotence mode, defining atomic regions and changing retry policies (discussed in the next page).

All these features are regional - they can be changed for a section of the code within a single exported function. To make this easy to use in Go, the library provides lower and higher level functions:

• the functions starting with Set and Get can be used to directly change the current mode of the engine
• the functions starting with With take a function parameter which will be executed with the requested mode, and then they restore the previous settings

Persistence level

The persistence level can be one of the following:

To change the persistence level for a section of the code, use the one of the golemhost.SetPersistenceLevel or golemhost.WithPersistenceLevel functions:

import (
	"github.com/golemcloud/golem-go/golemhost"
)
 
// Setting directly the persistence level
golemhost.SetPersistenceLevel(golemhost.PersistenceLevelPersistNothing)
 
// WithPersistenceLevel stores (using golemhost.GetPersistenceLevel) the current persistence level and
// then restores it after running the provided function
result, err = golemhost.WithPersistenceLevel(
	golemhost.PersistenceLevelPersistNothing,
	func() (string, error) {
		// this section runs PersistNothing level
		return "hello", nil
	},
)

Idempotence mode

Golem assumes that HTTP requests are idempotent by default. This means that in case of a failure, if the system cannot determine whether the request successfully reached the target or not, it will be retried. This behavior can be changed using the golemhost.SetIdempotenceMode and golemhost.WithIdempotenceMode functions:

import (
	"github.com/golemcloud/golem-go/golemhost"
)
 
// Setting directly the idempotence mode
golemhost.SetIdempotenceMode(false)
 
// WithIdempotenceMode stores (using golemhost.GetIdempotenceMode) the current idempotence mode and
// then restores it after running the provided function
result, err = golemhost.WithIdempotenceMode(
	false,
	func() (string, error) {
		// this section runs with idempotence mode disabled
		return "hello", nil
	},
)

With disabled idempotence mode, in case Golem cannot determine if the request was sent or not, it won't retry it but the worker will fail.

Atomic regions

By default, side effects are persisted and retried one by one. It is possible to group them together into atomic regions, in which case the execution is retried for some reason (the worker failed or interrupted within the region), all the side effects will be reexecuted.

The golem-go library exposes the golemhost.MarkBeginOperation and golemhost.MarkEndOperation low level functions, and the higher level golemhost.Atomically function built on top of those to manage atomic regions.

The golemhost.Atomically is preferred and recommended for handling atomic regions, as it guarantees to call the mark functions at the right time and with right values.

import (
	"github.com/golemcloud/golem-go/golemhost"
)
 
result, err = golemhost.Atomically(func() (string, error) {
	firstResult, err := FirstSideEffect()
	if err != nil {
		return "", err
	}
 
	secondResult, err := SecondSideEffect(firstResult)
	if err != nil {
		return "", err
	}
 
	return secondResult, nil
})

Commit oplog

The golemhost.OpLogCommit function waits until the oplog is committed to its persistent storage. The function takes a single argument, replicas, with the desired number of storage replicas the worker's journal is replicated to. The function will block until the oplog is committed to the specified number of replicas, or, if this number is larger than the available number of replicas, until it is written to all the replicas.