package cloudspanner

import "github.com/google/trillian/skylog/storage/gcp/cloudspanner"

Package cloudspanner provides implementation of the Skylog storage API in Cloud Spanner.

Index ¶

• type SequenceOpts
• type SequenceStorage
• func NewSequenceStorage(c *spanner.Client, treeID int64, opts SequenceOpts) *SequenceStorage
• func (s *SequenceStorage) Read(ctx context.Context, begin, end uint64) ([]storage.Entry, error)
• func (s *SequenceStorage) Write(ctx context.Context, begin uint64, entries []storage.Entry) error
• type TreeOpts
• type TreeStorage
• func NewTreeStorage(c *spanner.Client, treeID int64, opts TreeOpts) *TreeStorage
• func (t *TreeStorage) Read(ctx context.Context, ids []compact.NodeID) ([][]byte, error)
• func (t *TreeStorage) Write(ctx context.Context, nodes []storage.Node) error

Types ¶

type SequenceOpts ¶

type SequenceOpts struct {
	BatchSize uint64
	Shards    uint64
}

SequenceOpts configures the sequence storage sharding mechanism.

Log entries are split into the configured number of Shards, where each shard stores a periodic sub-sequence of batches of BatchSize. For example, if Shards is 3, and BatchSize is 2 then the entries are sharded as follows:

0 0 1 1 2 2 0 0 1 1 ...

Such schema optimizes for the case when entries are written in a nearly sequential way. If many concurrent writes are happening, all shards will be involved in parallel, and Cloud Spanner will add splits in between.

TODO(pavelkalinnikov): Store the parameters in per-tree metadata.

type SequenceStorage ¶

type SequenceStorage struct {
	// contains filtered or unexported fields
}

SequenceStorage allows reading from and writing to a sequence storage.

func NewSequenceStorage ¶

func NewSequenceStorage(c *spanner.Client, treeID int64, opts SequenceOpts) *SequenceStorage

NewSequenceStorage returns a new SequenceStorage.

func (*SequenceStorage) Read ¶

func (s *SequenceStorage) Read(ctx context.Context, begin, end uint64) ([]storage.Entry, error)

Read fetches the specified [begin, end) range of entries, and returns them in order. May return a prefix of the requested range if it spans multiple shards or some entries are missing.

func (*SequenceStorage) Write ¶

func (s *SequenceStorage) Write(ctx context.Context, begin uint64, entries []storage.Entry) error

Write stores all the passed-in entries to the sequence starting at the specified begin index.

type TreeOpts ¶

type TreeOpts struct {
	ShardLevels uint // Between 1 and 65.
	LeafShards  int64
}

TreeOpts stores sharding parameters for tree storage.

The sharding scheme is as follows. The lower ShardLevels levels are split into LeafShards shards, where each shard stores a periodic sub-structure of perfect subtrees. For example, if ShardLevels is 2, and LeafShards is 3 then the lower 2 levels are sharded as shown below:

 0   1   2   0   1
/ \ / \ / \ / \ / \
0 0 1 1 2 2 0 0 1 1 ...

Additionally, a single shard number 3 is created for all the nodes from the levels above.

Such schema optimizes for the case when nodes are written to the tree in a nearly sequential way. If many concurrent writes are happening, all shards will be involved in parallel, and Cloud Spanner will add splits in between.

TODO(pavelkalinnikov): Shard higher levels as well for more scalability. TODO(pavelkalinnikov): Achieve better vertical locality with stratification. TODO(pavelkalinnikov): Store the parameters in per-tree metadata.

type TreeStorage ¶

type TreeStorage struct {
	// contains filtered or unexported fields
}

TreeStorage allows reading from and writing to a tree storage.

func NewTreeStorage ¶

func NewTreeStorage(c *spanner.Client, treeID int64, opts TreeOpts) *TreeStorage

NewTreeStorage returns a new TreeStorage for the specified tree and options.

func (*TreeStorage) Read ¶

func (t *TreeStorage) Read(ctx context.Context, ids []compact.NodeID) ([][]byte, error)

Read fetches Merkle tree hashes of the passed in nodes from the storage. TODO(pavelkalinnikov): Add nodes cache.

func (*TreeStorage) Write ¶

func (t *TreeStorage) Write(ctx context.Context, nodes []storage.Node) error

Write stores all the passed-in nodes in the tree storage.

Source Files ¶

sequence.go tree.go