`mc pipe`

Syntax

The mc pipe command streams content from STDIN to a target object.

EXAMPLE

The following command writes contents of STDIN to an S3 compatible storage.

echo "My Meeting Notes" | mc pipe s3/engineering/meeting-notes.txt

SYNTAX

The command has the following syntax:

mc [GLOBALFLAGS] pipe                              \
                 TARGET                            \
                 [--encrypt "string"]              \
                 [--storage-class, --sc "string"]  \
                 [--attr "string"]                 \
                 [--tags "string"]                 \
                 [--encrypt-key "string"]

Brackets [] indicate optional parameters.
Parameters sharing a line are mutually dependent.
Parameters separated using the pipe | operator are mutually exclusive.

Copy the example to a text editor and modify as-needed before running the command in the terminal/shell.

Changed in version RELEASE.2023-01-11T03-14-16Z: mc pipe now supports concurrent uploads for better throughput of large streams.

Parameters

TARGET: Required
The full path to the alias or prefix where the command should run.

--attr

Optional

Add custom metadata for the object.

Specify key-value pairs as KEY=VALUE\;, separating each pair with a back slash and semicolon (\;). For example, --attr key1=value1\;key2=value2\;key3=value3.

--encrypt

Optional

Encrypt or decrypt objects using server-side encryption with server-managed keys. Specify key-value pairs as KEY=VALUE.

Each KEY represents a bucket or object.
Each VALUE represents the data key to use for encrypting object(s).

Enclose the entire list of key-value pairs passed to --encrypt in double-quotes ".

--encrypt can use the MC_ENCRYPT environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.

--encrypt-key

Optional

Encrypt or decrypt objects using server-side encryption with client-specified keys. Specify key-value pairs as KEY=VALUE.

Each KEY represents a bucket or object.
Each VALUE represents the data key to use for encrypting object(s).

Enclose the entire list of key-value pairs passed to --encrypt-key in double quotes ".

--encrypt-key can use the MC_ENCRYPT_KEY environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.

--storage-class, --sc

Optional

Set the storage class for the new object at the TARGET.

See Amazons documentation for more information on S3 storage classes.

--tags

Optional

Applies one or more tags to the TARGET.

Specify an ampersand-separated list of key-value pairs as KEY1=VALUE1&KEY2=VALUE2, where each pair represents one tag to assign to the objects.

Global Flags

This command supports any of the global flags.

Examples

Write Contents of `STDIN` to the Local Filesystem

The following command writes the contents of STDIN to the /tmp folder on the local filesystem.

mc pipe /tmp/hello-world.go

Copy an ISO Image to S3 Storage

The following command first streams the contents of an iso image for Debian and then uses the stream to create the object at an S3 path.

cat debian-live-11.5.0-amd64-mate.iso | mc pipe s3/opensource-isos/debian-11-5.iso

Stream MySQL Database Dump to S3

The following command first streams a MySQL database and uses the stream to create a backup on S3 with mc pipe:

mysqldump -u root -p ******* accountsdb | mc pipe s3/sql-backups/backups/accountsdb-sep-28-2022.sql

Write a File to a Reduced Redundancy Storage Class

The following command takes the STDIN stream and creates an object on the Reduced Redundancy storage class on S3.

 mc pipe --storage-class REDUCED_REDUNDANCY s3/personalbuck/meeting-notes.txt

Copy a File to a MinIO Deployment with Metadata

The following command uploads an MP3 file to a MinIO deployment with an ALIAS of myminio and a music bucket. The object writes with some metadata for Cache-Control and Artist.

cat music.mp3 | mc pipe --attr "Cache-Control=max-age=90000,min-fresh=9000;Artist=Unknown" myminio/music/guitar.mp3

Set Tags on Uploaded Objects

The following command creates an object on a MinIO deployment with an ALIAS of myminio in bucket mybucket with two tags. MinIO supports adding up to 10 custom tags to an object.

tar cvf - . | mc pipe --tags "category=prod&type=backup" myminio/mybucket/backup.tar