Amazon S3 - Get objects

Declaration

<AMAWSS3 OVERWRITE="yes/no" ONLYIFEXIST="yes/no" MATCHCASE="yes/no" CHECKSUM="yes/no" EXCLUDE="text" RE="yes/no" ACCESSKEY="text" SECRETKEY="text (encrypted)" PROTOCOL="text (options)" MAXERRORRETRY="number" SERVICEURL="text" PROXYHOST="text" PROXYPORT="number" PROXYUSER="text" PROXYPWD="text (options)" BUCKETNAME="text" KEYNAME="text" RESULTVARIABLE="text" VERSION="number" RESULTDATASET="text"><HEADER NAME="text" VALUE="text" /></AMAWSS3>

Related Topics    

Description

Retrieves a single object or multiple objects contained in an S3 bucket. You can store an object into a local file or store an object's contents into a variable. To specify more than one object, use wildcard characters (* or ?). To specify multiple objects or wildcard masks, separate them with a pipe symbol (|). Example: *.txt|*.bak.

IMPORTANT: The AWS S3 activities are performed using Amazon's Simple Storage Service engine, therefore, launching and operating Amazon S3 through Automate Desktop requires a valid Access Key ID and Secret Access Key.

Practical usage

Used to download one or more S3 objects. Note that in order for this activity function properly, you must have READ access to the objects you wish to retrieve.

Parameters

Connection

Property Type Required Default Markup Description
Connection --- --- --- --- Indicates where AWS user credentials and preferences should originate from. This is a design mode parameter used only during task construction and configuration, thus, comprises no markup. The available options are:
  • Host (default)- Specifies that user credentials and advanced preferences are configured individually for this activity. This option is normally chosen if only a single activity is required to complete an operation.
  • Session - Specifies that user credentials and advanced preferences are obtained from a pre-configured session created in an earlier step with the use of the Amazon S3 - Create session activity. This option is normally chosen if a combination of related activities are required to complete an operation. Linking several activities to a single session eliminates redundancy. Additionally, a single task supports construction and simultaneous execution of multiple sessions, improving efficiency.

Connection - Session

Property Type Required Default Markup Description
Session Text Yes, if Connection is set to Session EC2Session1 SESSION="S3Session1" The name of an existing session to attach this activity to. This parameter is active only if the Connection parameter is set to Session. The default session name is S3Session1.

Connection - Host > Credentials

Property Type Required Default Markup Description
Access key Text Yes, if Connection is set to Host (Empty) ACCESSKEY="022QF06E7MXBSH9DHM02" A 20-character alphanumeric string that uniquely identifies the owner of the AWS service account, similar to a username. This key along with a corresponding secret access key forms a secure information set that AWS uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host.
Secret access key Text Yes, if Connection is set to Host (Empty) SECRETKEY="text (encrypted)" A 40-character string that serves the role as password to access the AWS service account. This along with an associated access key forms a secure information set that EC2 uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host.

Connection - Host > Advanced

Property Type Required Default Markup Description
Protocol Text (options) No HTTP PROTOCOL="HTTPS" The protocol required. The available options are:
  • HTTP (default)
  • HTTPS
User agent Text No Automate USERAGENT="Automate" The name of the client or application initiating requests to AWS, which in this case, is Automate Desktop. This parameter's default value is Automate. 
Service URL Text No (Empty) SERVICEURL="https://s3.eu-west-1.amazonaws.com" The URL that provides the service endpoint. To make the service call to a different region, you can pass the region-specific endpoint URL. For example, entering https://s3.us-west-1.amazonaws.com points to US West (Northern California) region. A complete list of S3 regions, along with associated endpoints and valid protocols can be found below under Amazon S3 regions and endpoints .
Maximum number of retries on error Number No (Empty) MAXERRORRETRY="4" The total amount of times this activity should retry its request to the server before returning an error. Network components can generate errors anytime in the life of a request, thus, implementing retries can increase reliability. 
Proxy host Text No (Empty) PROXYHOST="proxy.host.com" The hostname (for example, server.domain.com) or IP address (for example, 192.168.0.100) of the proxy server to use when connecting to AWS.  
Proxy port Number No (Empty) PROXYPORT="1028" The port that should be used to connect to the proxy server.
Proxy username Text No (Empty) PROXYUSER="username" The username that should be used to authenticate connection with the proxy server (if required).
Proxy password Text No (Empty) PROXYPWD="encrypted" The password that should be used to authenticate connection with the proxy server (if required).

Object

Property Type Required Default Markup Description
Bucket Name Text Yes (Empty) BUCKETNAME="MyBucket" Indicates the name of the bucket in which to get objects from.
Key Name Text Yes (Empty)
  • KEYNAME="filename.txt"
  • KEYNAME="*.txt"
  • KEYNAME=*.txt|*.doc"
  • KEYNAME="*.*"
 Specifies the key name of the object s to get. A key is the unique identifier for an object within a bucket. To specify more than one object, use wildcard characters (* or ?). To specify multiple objects or wildcard masks, separate them with a pipe symbol (|). Example: *.txt|*.bak.
Store object into local file Text No (Empty)
  • FILE="c:\FolderName\file.txt"
  • FILE="c:\FolderName\*.txt"
  • FILE="c:\FolderName\*.*"
 Specifies the destination folder and (optionally) the file names in which to store the retrieved S3 objects. This can be a fully qualified path or a file name. Folders that do not exist will be automatically created at runtime.
Store object data into variable Text No (Empty) RESULTVARIABLE="myVar" The name of an existing variable in which to populate with the contents of the retrieved object. This property will only work with single object retrieval.
Create and populate dataset with S3 object information Text No (Empty) RESULTDATASET="PublicReadWrite" Specifies the name of the dataset to create and populate with information regarding the retrieved objects. This activity creates and populates a dataset containing a specific set of fields in addition to the standard dataset fields. See Datasets for more information.

Description

Error Causes

On Error

Additional notes

Amazon S3 regions and endpoints

This table contains a complete list of Amazon Simple Storage Service endpoints, along with their corresponding regions, supported protocols and location constraints.

Endpoint Region Protocol Location Constraints
s3.amazonaws.com US Standard * HTTP and HTTPS (none required)
s3.us-west-2.amazonaws.com US West (Oregon) Region HTTP and HTTPS us-west-2
s3.us-west-1.amazonaws.com US West (Northern California) Region HTTP and HTTPS us-west-1
s3.eu-west-1.amazonaws.com EU (Ireland) Region HTTP and HTTPS EU
s3.ap-southeast-1.amazonaws.com Asia Pacific (Singapore) Region HTTP and HTTPS ap-southeast-1
s3.ap-southeast-2.amazonaws.com Asia Pacific (Sydney) Region HTTP and HTTPS ap-southeast-2
s3.ap-northeast-1.amazonaws.com Asia Pacific (Tokyo) Region HTTP and HTTPS ap-northeast-1
s3.sa-east-1.amazonaws.com South America (Sao Paulo) Region HTTP and HTTPS sa-east-1

* The US Standard region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps.

Datasets

A dataset is a multiple column, multiple row container object. This activity creates and populates a dataset containing a specific set of fields in addition to the standard dataset fields.

Name Type Return Value
theDataset.KeyName Text The Key Name of the retrieved object.
theDataset.AmazonID2 Text A special token associated to the retrieved object that helps Amazon Web Service (AWS) troubleshoot problems.
theDataset.ETag Text The object's entity tag. This is a hash of the object that can be used to do conditional operations. The ETag only reflects changes to the contents of an object, not its metadata.
theDataset.Size Number The size of the object, in bytes.
theDataset.ContentType Text A standard MIME type describing the format of the object (for example, text/plain).
theDataset.VersionID Text The version ID of the retrieved object if it has a unique version ID.
NOTE: To get the version ID, Versioning must be enabled using the Amazon S3 - Set bucket versioning activity.

Example

NOTE:
  • Copy and paste the sample AML code below directly into the Task Builder Steps Panel.
  • To successfully run the sample code, update parameters containing user credentials, files, file paths, or other information specific to the task to match your environment.

Description

Get object "testFile.txt" from bucket "myBucket". Store it into file "C:\Temp\testFile.txt". Create and populate dataset "theDataset" with S3 object information. Use "mySession" S3 session.

Copy 
<AMAWSS3 BUCKETNAME="myBucket" KEYNAME="testFile.txt" FILE="C:\Temp\testFile.txt" RESULTDATASET="theDataset" SESSION="mySession" />