package marshmallow

import "github.com/perimeterx/marshmallow"

Package marshmallow provides a simple API to perform flexible and performant JSON unmarshalling. Unlike other packages, marshmallow supports unmarshalling of some known and some unknown fields with zero performance overhead nor extra coding needed. While unmarshalling, marshmallow allows fully retaining the original data and access it via a typed struct and a dynamic map.

https://github.com/perimeterx/marshmallow

Index ¶

• Variables
• func EnableCache()
• func EnableCustomCache(c Cache)
• func Unmarshal(data []byte, v interface{}, options ...UnmarshalOption) (map[string]interface{}, error)
• func UnmarshalFromJSONMap(data map[string]interface{}, v interface{}, options ...UnmarshalOption) (map[string]interface{}, error)
• type Cache
• type JSONDataErrorHandler
• type JSONDataHandler
• type Mode
• type MultipleError
• func (m *MultipleError) Error() string
• type MultipleLexerError
• func (m *MultipleLexerError) Error() string
• type ParseError
• func (p *ParseError) Error() string
• type UnmarshalOption
• func WithExcludeKnownFieldsFromMap(excludeKnownFields bool) UnmarshalOption
• func WithMode(mode Mode) UnmarshalOption
• func WithSkipPopulateStruct(skipPopulateStruct bool) UnmarshalOption
• type UnmarshalerFromJSONMap

Examples ¶

• Cache
• JSONDataHandler

Variables ¶

var (
	// ErrInvalidInput indicates the input JSON is invalid
	ErrInvalidInput = errors.New("invalid JSON input")

	// ErrInvalidValue indicates the target struct has invalid type
	ErrInvalidValue = errors.New("unexpected non struct value")
)

Functions ¶

func EnableCache ¶

func EnableCache()

EnableCache enables unmarshalling cache with default implementation. More info at EnableCustomCache.

func EnableCustomCache ¶

func EnableCustomCache(c Cache)

EnableCustomCache enables unmarshalling cache. It allows reuse of refection information about types needed to perform the unmarshalling. A use of such cache can boost up unmarshalling by x1.4. Check out benchmark_test.go for an example.

EnableCustomCache is not thread safe! Do not use it while performing unmarshalling, or it will cause an unsafe race condition. Typically, EnableCustomCache should be called once when the process boots.

Caching is disabled by default. The use of this function allows enabling it and controlling the behavior of the cache. Typically, the use of sync.Map should be good enough. The caching mechanism stores a single map per struct type. If you plan to unmarshal a huge amount of distinct struct it may get to consume a lot of resources, in which case you have the control to choose the caching implementation you like and its setup.

func Unmarshal ¶

func Unmarshal(data []byte, v interface{}, options ...UnmarshalOption) (map[string]interface{}, error)

Unmarshal parses the JSON-encoded object in data and stores the values in the struct pointed to by v and in the returned map. If v is nil or not a pointer to a struct, Unmarshal returns an ErrInvalidValue. If data is not a valid JSON or not a JSON object Unmarshal returns an ErrInvalidInput.

Unmarshal follows the rules of json.Unmarshal with the following exceptions: - All input fields are stored in the resulting map, including fields that do not exist in the struct pointed by v. - Unmarshal only operates on JSON object inputs. It will reject all other types of input by returning ErrInvalidInput. - Unmarshal only operates on struct values. It will reject all other types of v by returning ErrInvalidValue. - Unmarshal supports three types of Mode values. Each mode is self documented and affects how Unmarshal behaves.

func UnmarshalFromJSONMap ¶

func UnmarshalFromJSONMap(data map[string]interface{}, v interface{}, options ...UnmarshalOption) (map[string]interface{}, error)

UnmarshalFromJSONMap parses the JSON map data and stores the values in the struct pointed to by v and in the returned map. If v is nil or not a pointer to a struct, UnmarshalFromJSONMap returns an ErrInvalidValue.

UnmarshalFromJSONMap follows the rules of json.Unmarshal with the following exceptions: - All input fields are stored in the resulting map, including fields that do not exist in the struct pointed by v. - UnmarshalFromJSONMap receive a JSON map instead of raw bytes. The given input map is assumed to be a JSON map, meaning it should only contain the following types: bool, string, float64, []interface, and map[string]interface{}. Other types will cause decoding to return unexpected results. - UnmarshalFromJSONMap only operates on struct values. It will reject all other types of v by returning ErrInvalidValue. - UnmarshalFromJSONMap supports three types of Mode values. Each mode is self documented and affects how UnmarshalFromJSONMap behaves.

Types ¶

type Cache ¶

type Cache interface {
	// Load returns the value stored in the map for a key, or nil if no value is present.
	// The ok result indicates whether value was found in the map.
	Load(key interface{}) (interface{}, bool)
	// Store sets the value for a key.
	Store(key, value interface{})
}

Cache allows unmarshalling to use a cached version of refection information about types. Cache interface follows the implementation of sync.Map, but you may wrap any cache implementation to match it. This allows you to control max cache size, eviction policies and any other caching aspect.

package main

import (
	"github.com/perimeterx/marshmallow"
	"sync"
)

func main() {
	// enable default cache
	marshmallow.EnableCache()

	type exampleStruct struct {
		Foo string `json:"foo"`
		Boo []int  `json:"boo"`
	}
	v := exampleStruct{}
	_, _ = marshmallow.Unmarshal([]byte(`{"foo":"bar","boo":[1,2,3]}`), &v)

	// enable custom cache, you can pass any implementation of the marshmallow.Cache interface
	// this lets you control the size of the cache, eviction policy, or any other aspect of it.
	marshmallow.EnableCustomCache(&sync.Map{})
}

type JSONDataErrorHandler ¶

type JSONDataErrorHandler interface {
	HandleJSONData(data map[string]interface{}) error
}

JSONDataErrorHandler allow types to handle JSON data as maps. Types should implement this interface if they wish to act on the map representation of parsed JSON input. This is mainly used to allow nested objects to capture unknown fields and leverage marshmallow's abilities. If HandleJSONData returns an error, it will be propagated as an unmarshal error

type JSONDataHandler ¶

type JSONDataHandler interface {
	HandleJSONData(data map[string]interface{})
}

Deprecated: use JSONDataErrorHandler instead

package main

import (
	"fmt"
	"github.com/perimeterx/marshmallow"
)

func main() {
	type parentStruct struct {
		Known  string      `json:"known"`
		Nested childStruct `json:"nested"`
	}

	data := []byte(`{"known": "foo","unknown": "boo","nested": {"known": "goo","unknown": "doo"}}`)
	p := &parentStruct{}
	_, err := marshmallow.Unmarshal(data, p)
	fmt.Printf("err: %v\n", err)
	fmt.Printf("nested data: %+v\n", p.Nested.Data)
}

type childStruct struct {
	Known string `json:"known"`

	Data map[string]interface{} `json:"-"`
}

func (c *childStruct) HandleJSONData(data map[string]interface{}) error {
	c.Data = data
	return nil
}

err: <nil>
nested data: map[known:goo unknown:doo]

type Mode ¶

type Mode uint8

Mode dictates the unmarshalling mode. Each mode is self documented below.

const (
	// ModeFailOnFirstError is the default mode. It makes unmarshalling terminate
	// immediately on any kind of error. This error will then be returned.
	ModeFailOnFirstError Mode = iota

	// ModeAllowMultipleErrors mode makes unmarshalling keep decoding even if
	// errors are encountered. In case of such error, the erroneous value will be omitted from the result.
	// Eventually, all errors will all be returned, alongside the partial result.
	ModeAllowMultipleErrors

	// ModeFailOverToOriginalValue mode makes unmarshalling keep decoding even if
	// errors are encountered. In case of such error, the original external value be placed in the
	// result data, even though it does not meet the schematic requirements.
	// Eventually, all errors will be returned, alongside the full result. Note that the result map
	// will contain values that do not match the struct schema.
	ModeFailOverToOriginalValue
)

type MultipleError ¶

type MultipleError struct {
	Errors []error
}

MultipleError indicates one or more unmarshalling errors during JSON map decode

func (*MultipleError) Error ¶

func (m *MultipleError) Error() string

type MultipleLexerError ¶

type MultipleLexerError struct {
	Errors []*jlexer.LexerError
}

MultipleLexerError indicates one or more unmarshalling errors during JSON bytes decode

func (*MultipleLexerError) Error ¶

func (m *MultipleLexerError) Error() string

type ParseError ¶

type ParseError struct {
	Reason string
	Path   string
}

ParseError indicates a JSON map decode error

func (*ParseError) Error ¶

func (p *ParseError) Error() string

type UnmarshalOption ¶

type UnmarshalOption func(*unmarshalOptions)

func WithExcludeKnownFieldsFromMap ¶

func WithExcludeKnownFieldsFromMap(excludeKnownFields bool) UnmarshalOption

WithExcludeKnownFieldsFromMap is an UnmarshalOption function to set the excludeKnownFieldsFromMap option. Exclude known fields flag is set to false by default. When the flag is set to true, fields specified in the input struct (known fields) will be excluded from the result map

func WithMode ¶

func WithMode(mode Mode) UnmarshalOption

WithMode is an UnmarshalOption function to set the unmarshalling mode.

func WithSkipPopulateStruct ¶

func WithSkipPopulateStruct(skipPopulateStruct bool) UnmarshalOption

WithSkipPopulateStruct is an UnmarshalOption function to set the skipPopulateStruct option. Skipping populate struct is set to false by default. If you do not intend to use the struct value once unmarshalling is finished, set this option to true to boost performance. This would mean the struct fields will not be set with values, but rather it will only be used as the target schema when populating the result map.

type UnmarshalerFromJSONMap ¶

type UnmarshalerFromJSONMap interface {
	UnmarshalJSONFromMap(data interface{}) error
}

UnmarshalerFromJSONMap is the interface implemented by types that can unmarshal a JSON description of themselves. In case you want to implement custom unmarshalling, json.Unmarshaler only supports receiving the data as []byte. However, while unmarshalling from JSON map, the data is not available as a raw []byte and converting to it will significantly hurt performance. Thus, if you wish to implement a custom unmarshalling on a type that is being unmarshalled from a JSON map, you need to implement UnmarshalerFromJSONMap interface.

Source Files ¶

cache.go doc.go errors.go options.go reflection.go unmarshal.go unmarshal_from_json_map.go