gocloud.dev – gocloud.dev/blob/s3blob Index | Examples | Files

package s3blob

import "gocloud.dev/blob/s3blob"

Package s3blob provides a blob implementation that uses S3. Use OpenBucket to construct a *blob.Bucket.

URLs

For blob.OpenBucket, s3blob registers for the scheme "s3". The default URL opener will use an AWS session with the default credentials and configuration; see https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ for more details. Use "awssdk=v1" or "awssdk=v2" to force a specific AWS SDK version. To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.

Escaping

Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for s3blob:

As

s3blob exposes the following types for As:

Example (OpenBucketFromURL)

Code:play  

package main

import (
	"context"
	"log"

	"gocloud.dev/blob"
)

func main() {
	// PRAGMA: This example is used on gocloud.dev; PRAGMA comments adjust how it is shown and can be ignored.
	// PRAGMA: On gocloud.dev, add a blank import: _ "gocloud.dev/blob/s3blob"
	// PRAGMA: On gocloud.dev, hide lines until the next blank line.
	ctx := context.Background()

	// blob.OpenBucket creates a *blob.Bucket from a URL.
	bucket, err := blob.OpenBucket(ctx, "s3://my-bucket?region=us-west-1")
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()

	// Forcing AWS SDK V2.
	bucket, err = blob.OpenBucket(ctx, "s3://my-bucket?region=us-west-1&awssdk=v2")
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()
}

Index

Examples

Constants 

const Scheme = "s3"

Scheme is the URL scheme s3blob registers its URLOpener under on blob.DefaultMux.

Variables 

var Set = wire.NewSet(
	wire.Struct(new(URLOpener), "ConfigProvider"),
)

Set holds Wire providers for this package.

Functions

func OpenBucket 

func OpenBucket(ctx context.Context, sess client.ConfigProvider, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucket returns a *blob.Bucket backed by S3. AWS buckets are bound to a region; sess must have been created using an aws.Config with Region set to the right region for bucketName. See the package documentation for an example.

Deprecated: AWS no longer supports their V1 API. Please migrate to OpenBucketV2.

Example

Code:play  

package main

import (
	"context"
	"log"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"gocloud.dev/blob/s3blob"
)

func main() {
	// PRAGMA: This example is used on gocloud.dev; PRAGMA comments adjust how it is shown and can be ignored.
	// PRAGMA: On gocloud.dev, hide lines until the next blank line.
	ctx := context.Background()

	// Establish an AWS session.
	// See https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ for more info.
	// The region must match the region for "my-bucket".
	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("us-west-1"),
	})
	if err != nil {
		log.Fatal(err)
	}

	// Create a *blob.Bucket.
	bucket, err := s3blob.OpenBucket(ctx, sess, "my-bucket", nil)
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()
}

func OpenBucketV2 

func OpenBucketV2(ctx context.Context, client *s3v2.Client, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucketV2 returns a *blob.Bucket backed by S3, using AWS SDK v2.

Example

Code:play  

package main

import (
	"context"
	"log"

	awsv2cfg "github.com/aws/aws-sdk-go-v2/config"
	s3v2 "github.com/aws/aws-sdk-go-v2/service/s3"
	"gocloud.dev/blob/s3blob"
)

func main() {
	// PRAGMA: This example is used on gocloud.dev; PRAGMA comments adjust how it is shown and can be ignored.
	// PRAGMA: On gocloud.dev, hide lines until the next blank line.
	ctx := context.Background()

	// Establish a AWS V2 Config.
	// See https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/ for more info.
	cfg, err := awsv2cfg.LoadDefaultConfig(ctx)
	if err != nil {
		log.Fatal(err)
	}

	// Create a *blob.Bucket.
	clientV2 := s3v2.NewFromConfig(cfg)
	bucket, err := s3blob.OpenBucketV2(ctx, clientV2, "my-bucket", nil)
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()
}

Types

type Options 

type Options struct {
	// UseLegacyList forces the use of ListObjects instead of ListObjectsV2.
	// Some S3-compatible services (like CEPH) do not currently support
	// ListObjectsV2.
	UseLegacyList bool

	// EncryptionType sets the encryption type headers when making write or
	// copy calls. This is required if the bucket has a restrictive bucket
	// policy that enforces a specific encryption type
	EncryptionType typesv2.ServerSideEncryption

	// KMSEncryptionID sets the kms key id header for write or copy calls.
	// This is required when a bucket policy enforces the use of a specific
	// KMS key for uploads
	KMSEncryptionID string
}

Options sets options for constructing a *blob.Bucket backed by fileblob.

type URLOpener 

type URLOpener struct {
	// UseV2 indicates whether the AWS SDK V2 should be used.
	UseV2 bool

	// ConfigProvider must be set to a non-nil value if UseV2 is false.
	ConfigProvider client.ConfigProvider

	// Options specifies the options to pass to OpenBucket.
	Options Options
}

URLOpener opens S3 URLs like "s3://mybucket".

The URL host is used as the bucket name.

Use "awssdk=v1" to force using AWS SDK v1, "awssdk=v2" to force using AWS SDK v2, or anything else to accept the default.

The following S3-specific query options are also supported:

For V1, see gocloud.dev/aws/ConfigFromURLParams for supported query parameters for overriding the aws.Session from the URL. For V2, see gocloud.dev/aws/V2ConfigFromURLParams.

func (*URLOpener) OpenBucketURL 

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens a blob.Bucket based on u.

Source Files

s3blob.go

Version
v0.40.0 (latest)
Published
Oct 10, 2024
Platform
js/wasm
Imports
28 packages
Last checked
1 month ago –

Tools for package owners.