package offset2

import "go.cryptoscope.co/margaret/offset2"

Package offset2 implements a margaret log as persisted sequence of data across multiple files.

Format Defintion

A log consists of three files: data, ofst and jrnl.

* data: a list of length-prefixed data chunks, size is a uint64 (size++[size]byte).

* ofst: a list of uint64, representing entry offsets in 'data'

* jrnl keeps track of the current sequence number, see checkJournal() for more

To read entry 5 in `data`, you follow these steps:

1. Seek to 5*(sizeof(uint64)=8)=40 in `ofset` and read the uint64 representing the offset in `data`

2. Seek to that offset in `data`, read the length-prefix (the uint64 for the size of the entry)

3. Finally, read that amount of data, which is your entry

All uint64's are encoded in BigEndian.

Index ¶

• func LockSink(sink luigi.Sink) luigi.Sink
• type OffsetLog
• func Open(name string, cdc margaret.Codec) (*OffsetLog, error)
• func (log *OffsetLog) Append(v interface{}) (int64, error)
• func (log *OffsetLog) Changes() luigi.Observable
• func (log *OffsetLog) CheckConsistency() error
• func (log *OffsetLog) Close() error
• func (log *OffsetLog) FileName() string
• func (log *OffsetLog) Get(seq int64) (interface{}, error)
• func (log *OffsetLog) Null(seq int64) error
• func (log *OffsetLog) Query(specs ...margaret.QuerySpec) (luigi.Source, error)
• func (log *OffsetLog) Replace(seq int64, data []byte) error
• func (log *OffsetLog) Seq() int64

Functions ¶

func LockSink ¶

func LockSink(sink luigi.Sink) luigi.Sink

Types ¶

type OffsetLog ¶

type OffsetLog struct {
	// contains filtered or unexported fields
}

func Open ¶

func Open(name string, cdc margaret.Codec) (*OffsetLog, error)

Open returns a the offset log in the directory at `name`. If it is empty or does not exist, a new log will be created.

func (*OffsetLog) Append ¶

func (log *OffsetLog) Append(v interface{}) (int64, error)

func (*OffsetLog) Changes ¶

func (log *OffsetLog) Changes() luigi.Observable

func (*OffsetLog) CheckConsistency ¶

func (log *OffsetLog) CheckConsistency() error

CheckConsistency is an fsck for the offset log.

func (*OffsetLog) Close ¶

func (log *OffsetLog) Close() error

func (*OffsetLog) FileName ¶

func (log *OffsetLog) FileName() string

func (*OffsetLog) Get ¶

func (log *OffsetLog) Get(seq int64) (interface{}, error)

func (*OffsetLog) Null ¶

func (log *OffsetLog) Null(seq int64) error

Null overwrites the entry at seq with zeros updating is kinda odd in append-only but in some cases you still might want to redact entries

func (*OffsetLog) Query ¶

func (log *OffsetLog) Query(specs ...margaret.QuerySpec) (luigi.Source, error)

func (*OffsetLog) Replace ¶

func (log *OffsetLog) Replace(seq int64, data []byte) error

Replace overwrites the seq entry with data. data has to be smaller then the current entry.

func (*OffsetLog) Seq ¶

func (log *OffsetLog) Seq() int64

Source Files ¶

data.go journal.go log.go offset.go qry.go

Directories ¶