Amazon S3 - Put objects

Declaration

<AMAWSS3 ACTIVITY="put_object" SUBFOLDERS="yes/no" ARCHIVETURNOFF="yes/no" MATCHCASE="yes/no" CHECKSUM="yes/no" EXCLUDE="text" RE="yes/no" ACCESSKEY="text" SECRETKEY="text (encrypted)" PROTOCOL="text (options)" USERAGENT="text" MAXERRORRETRY="number" SERVICEURL="text" PROXYHOST="text" PROXYPORT="number" PROXYUSER="text" PROXYPWD="text (encrypted)" FILE="text" ACL="text (options)" STORAGECLASS="text (options)" ENCRYPTIONTYPE="text (options)" BUCKETNAME="text" KEYNAME="text" TIMEOUT="number" MD5="text" RESULTDATASET="text" CONTENTTYPE="text" GENERATEMD5="yes/no" />

Related Topics    

Description

Adds (uploads) one or more objects to a bucket.

IMPORTANT: The AWS S3 activities are performed using Amazon's Simple Storage Service engine, therefore, launching and operating Amazon S3 through Automate Desktop requires a valid Access Key ID and Secret Access Key.

Practical usage

Puts one or more objects into an S3 bucket. You must have write permissions on a bucket to add an object to it. Amazon S3 is a distributed system. If it receives multiple write requests for the same object simultaneously, it overwrites all but the last object written.

Parameters

Connection

Property Type Required Default Markup Description
Connection --- --- --- --- Indicates where AWS user credentials and preferences should originate from. This is a design mode parameter used only during task construction and configuration, thus, comprises no markup. The available options are:
  • Host (default)- Specifies that user credentials and advanced preferences are configured individually for this activity. This option is normally chosen if only a single activity is required to complete an operation.
  • Session - Specifies that user credentials and advanced preferences are obtained from a pre-configured session created in an earlier step with the use of the Amazon S3 - Create session activity. This option is normally chosen if a combination of related activities are required to complete an operation. Linking several activities to a single session eliminates redundancy. Additionally, a single task supports construction and simultaneous execution of multiple sessions, improving efficiency.

Connection - Session

Property Type Required Default Markup Description
Session Text Yes, if Connection is set to Session EC2Session1 SESSION="S3Session1" The name of an existing session to attach this activity to. This parameter is active only if the Connection parameter is set to Session. The default session name is S3Session1.

Connection - Host > Credentials

Property Type Required Default Markup Description
Access key Text Yes, if Connection is set to Host (Empty) ACCESSKEY="022QF06E7MXBSH9DHM02" A 20-character alphanumeric string that uniquely identifies the owner of the AWS service account, similar to a username. This key along with a corresponding secret access key forms a secure information set that AWS uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host.
Secret access key Text Yes, if Connection is set to Host (Empty) SECRETKEY="text (encrypted)" A 40-character string that serves the role as password to access the AWS service account. This along with an associated access key forms a secure information set that EC2 uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host.

Connection - Host > Advanced

Property Type Required Default Markup Description
Protocol Text (options) No HTTP PROTOCOL="HTTPS" The protocol required. The available options are:
  • HTTP (default)
  • HTTPS
User agent Text No Automate USERAGENT="Automate" The name of the client or application initiating requests to AWS, which in this case, is Automate Desktop. This parameter's default value is Automate. 
Service URL Text No (Empty) SERVICEURL="https://s3.eu-west-1.amazonaws.com" The URL that provides the service endpoint. To make the service call to a different region, you can pass the region-specific endpoint URL. For example, entering https://s3.us-west-1.amazonaws.com points to US West (Northern California) region. A complete list of S3 regions, along with associated endpoints and valid protocols can be found below under Amazon S3 regions and endpoints .
Maximum number of retries on error Number No (Empty) MAXERRORRETRY="4" The total amount of times this activity should retry its request to the server before returning an error. Network components can generate errors anytime in the life of a request, thus, implementing retries can increase reliability. 
Proxy host Text No (Empty) PROXYHOST="proxy.host.com" The hostname (for example, server.domain.com) or IP address (for example, 192.168.0.100) of the proxy server to use when connecting to AWS.  
Proxy port Number No (Empty) PROXYPORT="1028" The port that should be used to connect to the proxy server.
Proxy username Text No (Empty) PROXYUSER="username" The username that should be used to authenticate connection with the proxy server (if required).
Proxy password Text No (Empty) PROXYPWD="encrypted" The password that should be used to authenticate connection with the proxy server (if required).

Object

Property Type Required Default Markup Description
Put files Text No  (Empty)
  • FILE="c:\folder1\file.txt"
  • FILE="c:\folder1\*.txt"
  • FILE="c:\folder1\*.*"
 If enabled, specifies the local files to upload onto the S3 bucket. To specify more than one file, use wildcard characters (* or ?). To specify multiple objects or wildcard masks, separate them with a pipe symbol. Example: *.txt|*.bak. If this parameter is enabled, the Put data parameter is ignored (enabled by default).
Put data Text No (Empty) DATA="dataString" If enable, specifies the text string that should be populated into the object being put into the S3 bucket. If this parameter is enabled, the Put files parameter is ignored (disabled by default).
Bucket name Text Yes (Empty) BUCKETNAME="MyBucket" The name of the bucket in which to put the objects into.
Key name Text Yes (Empty)
  • KEYNAME="filename.txt"
  • KEYNAME="*.txt"
  • KEYNAME=*.txt|*.doc"
  • KEYNAME="*.*"
 The key name of the objects to be placed in the bucket. A key is the unique identifier for an object within a bucket. To specify more than one object, use wildcard characters (* or ?).
Canned ACL Text Yes Private
  • ACL="noACL"
  • ACL="Private"
  • ACL="PublicRead"
  • ACL="PublicReadWrite"
  • ACL="AuthenticatedRead"
  • ACL="BucketOwnerRead"
  • ACL="BucketOwnerFullControl"
 Sets the S3 canned access policy associated to the object being placed into the bucket. The available Canned ACL options are:
  • No ACL - No access policies.
  • Private (Default) - Owner gets full control. No one else has access rights.
  • Public read - Owner gets full control and the anonymous principal is granted read access.
  • Public Read Write - Owner gets full control, the anonymous principal is granted read/write access. Useful policy to apply to a bucket, but is generally not recommended.
  • Authenticated read - Owner gets full control, and any principal authenticated as a registered Amazon S3 user is granted read access.
  • Bucket owner read - Object owner gets full control. Bucket owner gets read access. This ACL applies only to objects and is equivalent to Private when used with Create Bucket activity. Use this ACL to let someone other than the bucket owner write content (get full control) in the bucket but still grant the bucket owner read access to the objects.
  • Bucket owner full control - Object owner gets full control. Bucket owner gets full control. Applies only to objects and is equivalent to Private when used with Create Bucket activity. Use this ACL to let someone other than the bucket owner write content (get full control) in the bucket but still grant the bucket owner full rights over the objects.
Timeout (in minutes) Number No 20 TIMEOUT="25" The timeout value (in minutes) that should be set for this activity. The value is assigned to the Timeout properties of the requested object used for S3 Put requests. The default value is 20 minutes.
Storage class Text (options) No Standard
  • STORAGECLASS="standard"
  • STORAGECLASS="reduced_redundancy"
  • STORAGECLASS="glacier"
 The storage class to use. This provides the option to reduce costs by storing non-critical, reproducible data at lower levels of redundancy than Amazon S3's standard storage. The available options are:
  • Standard (default) - Use Amazon's standard storage configuration.
  • Reduced redundancy - Use reduced redundancy storage which stores objects on multiple devices across multiple facilities, providing 400 times the durability of a typical disk drive, but does not replicate objects as many times as standard Amazon S3 storage. Thus, using RRS is even more cost effective.

  • Glacier - To transition objects to the Glacier storage class you can use lifecycle configuration.

Server side encryption Text (options) No AES-256
  • ENCRYPTIONTYPE="none"
  • ENCRYPTIONTYPE="AES256"
 Specifies a server-side encryption algorithm to use when Amazon S3 creates an object. The available objects are:
  • None (default) - No server side encryption algorithm will be used.
  • AES-256 - Server side encryption will be set to AES-256.
MD5 digest (optional) Text No (Empty) MD5="MD5Value" The base64 encoded 128-bit MD5 digest of the message used to ensure that data is not corrupted traversing the network. When you use the Content-MD5 option, Amazon S3 checks the object against the provided MD5 value. If they do not match, Amazon S3 returns an error. Additionally, you can calculate the MD5 while putting an object to Amazon S3 and compare the returned ETag to the calculated MD5 value.
Generate MD5 digest Yes/No No No GENERATEMD5="YES" If selected, generates an MD5 value used as a message integrity check to verify that the data is the same data that was originally sent. Disabled by default.
Content type (optional) Text No (Empty) CONTENTTYPE="text/plain" A standard MIME type describing the format of the contents (for example, text/plain).
Create and populate dataset with S3 object information Text No (Empty) RESULTDATASET="S3ObjectInfo" The name of the dataset to create and populate with information about the objects that were added.

File Options

Property Type Required Default Markup Description
Exclude mask Text No (Empty)
  • EXCLUDE="*.txt"
  • EXCLUDE="*.txt|*.bak"
  • EXCLUDE="c:\foldename"
 Causes this activity to omit files matching the masks specified. File names or wildcard masks may be used. Multiple entries may be specified by separating them with a pipe symbol (|), for example: *.txt|*.bak.
Regular expression Yes/No No No RE="yes" If selected, indicates that the value entered in the Exclude mask parameter will be interpreted as a regular expression. If disabled (default) the value will be interpreted as normal readable text.
Only if newer than Date No (Empty) ISNEWERTHAN="%DateSerial(2001,10,12) + TimeSerial(00,00,00)%" If enabled, causes this activity to only act on files that are newer than the date/time specified. If this parameter is left blank or disabled (default), file dates are ignored. Click the Custom button to select from a list of pre-defined date parameters. Enable the Expression option to allow entry of a date/time expression.
Only if older than Date No (Empty) ISOLDERTHAN="%DateSerial(2001,10,12) + TimeSerial(00,00,00)%" If enabled, causes this activity to only act on files that are older than the date/time specified. If this parameter is left blank or disabled (default), file dates are ignored. Click the Custom button to select from a list of pre-defined date parameters. Enable the Expression option to allow entry of a date/time expression.
Include subfolders Yes/No No No SUBFOLDERS="YES" If selected, denotes that, if present, subfolders should be searched for files matching the mask specified in the Put files or Put data parameter. If disabled (default), subfolders are ignored. Only files that exist in the root of the source folder will be searched.
Turn archive attribute off Yes/No No No ARCHIVETURNOFF="YES" If selected, specifies that the archive attribute of the source files should be switched OFF. The Windows archive attribute is generally used to track whether a file has been backed-up. Turning the source file's archive attribute off indicates to many backup programs that the file has already been backed-up. This parameter is disabled by default.
Validate checksum Yes/No No No CHECKSUM="yes" If selected, causes this activity to validate file checksum before uploading. Disabled by default.
Match case Yes/No No No MATCHCASE="yes" If selected, causes this activity to become case sensitive. Disabled by default.

File Attributes

Property Type Required Default Markup Description
Attributes Text (Options) No (Empty) ATTRFILTER="+R+A-H" (copy read-only & archive files but not hidden files) This group of settings causes this activity to filter which files are affected by the attribute change based on the original attribute settings of the source files. In visual mode, a group of controls are provided to assist in the selection of this parameter. In AML mode, a single text item must be specified that contains the original attribute mask of the files you wish to affect. Available options are:
  • R—Read-only: Specifying "+R" causes files with this attribute turned on to be included, "-R" causes files with this attribute turned off to be included, not specifying the letter (default) causes this attribute to be ignored.
  • A—Archive: Specifying "+A" causes files with this attribute turned on to be included, "-A" causes files with this attribute turned off to be included, not specifying the letter (default) causes this attribute to be ignored.
  • S—System: Specifying "+S" causes files with this attribute turned on to be included, "-S" causes files with this attribute turned off to be included, not specifying the letter (default) causes this attribute to be ignored.
  • H—Hidden: Specifying "+R" causes files with this attribute turned on to be included, "-H" causes files with this attribute turned off to be included, not specifying the letter (default) causes this attribute to be ignored.
  • C—Compression: Specifying "+C" causes files with this attribute turned on to be included, "-C" causes files with this attribute turned off to be included, not specifying the letter (default) causes this attribute to be ignored.

Advanced

Property Type Required Default Markup Description
Name Text No (Empty) HEADER NAME="myHeader"

Specifies the "key" in a key-value pair. This is the handle that you assign to an object. In Amazon S3, details about each file and folder are stored in key value pairs called metadata or headers. System metadata is used and processed by Amazon S3, however, user metadata or custom headers can be specified by you. This adds more flexibility and enables you to better distinguish specific files by adding or editing custom headers on existing S3 objects or assigning custom headers to new objects. Press Click here to add new row... to add a key-value pair. Press the red X to remove an existing key-value pair.

Value Text No (Empty) VALUE="theValue" Specifies the "value" in a key-value pair. This is the content that you are storing for an object. In Amazon S3, details about each file and folder are stored in key value pairs called metadata or headers. System metadata is used and processed by Amazon S3, however, user metadata or custom headers can be specified by you. This adds more flexibility and enables you to better distinguish specific files by adding or editing custom headers on existing S3 objects or assigning custom headers to new objects. Press Click here to add new row... to add a key-value pair. Press the red X to remove an existing key-value pair.

Description

Error Causes

On Error

Additional notes

Amazon S3 regions and endpoints

This table contains a complete list of Amazon Simple Storage Service endpoints, along with their corresponding regions, supported protocols and location constraints.

Endpoint Region Protocol Location Constraints
s3.amazonaws.com US Standard * HTTP and HTTPS (none required)
s3.us-west-2.amazonaws.com US West (Oregon) Region HTTP and HTTPS us-west-2
s3.us-west-1.amazonaws.com US West (Northern California) Region HTTP and HTTPS us-west-1
s3.eu-west-1.amazonaws.com EU (Ireland) Region HTTP and HTTPS EU
s3.ap-southeast-1.amazonaws.com Asia Pacific (Singapore) Region HTTP and HTTPS ap-southeast-1
s3.ap-southeast-2.amazonaws.com Asia Pacific (Sydney) Region HTTP and HTTPS ap-southeast-2
s3.ap-northeast-1.amazonaws.com Asia Pacific (Tokyo) Region HTTP and HTTPS ap-northeast-1
s3.sa-east-1.amazonaws.com South America (Sao Paulo) Region HTTP and HTTPS sa-east-1

* The US Standard region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps.

Example

NOTE:
  • Copy and paste the sample AML code below directly into the Task Builder Steps Panel.
  • To successfully run the sample code, update parameters containing user credentials, files, file paths, or other information specific to the task to match your environment.

Description

Put file "C:\Temp\Book1.xlsx" in bucket "myBucket". Key name is "Book1.xlsx". Use "my_session" S3 session.

Copy 
<AMAWSS3 ACTIVITY="put_object" BUCKETNAME="myBucket" KEYNAME="Book1.xlsx" FILE="C:\Temp\Book1.xlsx" SESSION="my_session" />