Amazon S3 - List buckets |
Declaration
|
<AMAWSS3 ACTIVITY="list_buckets" ACCESSKEY="text" SECRETKEY="text (encrypted)" USERAGENT="text" MAXERRORRETRY="number" SERVICEURL="text" PROXYHOST="text" PROXYPORT="number" PROXYUSER="text" PROXYPWD="text (encrypted)" RESULTDATASET="text"><HEADER NAME="text" VALUE="text" /></AMAWSS3> |
Description
Retrieves all bucket names and their creation dates and stores the results in an Automate dataset.
|
IMPORTANT: The AWS S3 activities are performed using Amazon's Simple Storage Service engine, therefore, launching and operating Amazon S3 through Automate requires a valid Access Key ID and Secret Access Key. |
Practical Usage
Used to retrieve the name and creation date of one or more buckets.
Parameters
connection
|
Property |
Type |
Required |
Default |
Markup |
Description |
|
Connection |
--- |
--- |
--- |
--- |
Indicates where AWS user credentials and preferences should originate from. This is a design mode parameter used only during task construction and configuration, thus, comprises no markup. The available options are:
|
|
Session |
Text |
Yes if connection is session-based |
EC2Session1 |
SESSION="S3Session1" |
The name of an existing session to attach this activity to. This parameter is active only if the Connection parameter is set to Session. The default session name is 'S3Session1'. |
|
Access key |
Text |
Yes if connection is host-based |
(Empty) |
ACCESSKEY= "022QF06E7MXBSH9DHM02" |
A 20-character alphanumeric string that uniquely identifies the owner of the AWS service account, similar to a username. This key along with a corresponding secret access key forms a secure information set that AWS uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host. |
|
Secret Access key |
Text |
Yes if connection is host-based |
(Empty) |
SECRETKEY= "kWcrlUX5JEDGM/LtmEENI/ aVmYvHNif5zB+d9+ct" |
A 40-character string that serves the role as password to access the AWS service account. This along with an associated access key forms a secure information set that EC2 uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host. |
|
Protocol |
Text (options) |
No |
HTTP |
PROTOCOL="HTTPS" |
The protocol required. The available options are:
|
|
User agent |
Text |
No |
Automate |
USERAGENT="Automate" |
The name of the client or application initiating requests to AWS, which in this case, is Automate. This parameter's default value is 'Automate'. |
|
Service URL |
Text |
No |
(Empty) |
SERVICEURL= "https://s3.eu-west-1.amazonaws.com" |
The URL that provides the service endpoint. To make the service call to a different region, you can pass the region-specific endpoint URL. For example, entering https://s3.us-west-1.amazonaws.com points to US West (Northern California) region. A complete list of S3 regions, along with associated endpoints and valid protocols can be found below under S3 Endpoints and Regions. |
|
Maximum retry on error |
Number |
No |
(Empty) |
MAXERRORRETRY="4" |
The total amount of times this activity should retry its request to the server before returning an error. Network components can generate errors anytime in the life of a request, thus, implementing retries can increase reliability. |
|
Proxy host |
Text |
No |
(Empty) |
PROXYHOST="proxy.host.com" |
The host name (i.e. server.domain.com) or IP address (i.e. xxx.xxx.xxx.xxx) of the proxy server to use when connecting to AWS. |
|
Proxy port |
Number |
No |
(Empty) |
PROXYPORT="1028" |
The port that should be used to connect to the proxy server. |
|
Proxy username |
Text |
No |
(Empty) |
PROXYUSER="username" |
The username that should be used to authenticate connection with the proxy server (if required). |
|
Proxy password |
Text |
No |
(Empty) |
PROXYPWD="encrypted" |
The password that should be used to authenticate connection with the proxy server (if required). |
Bucket
|
Property |
Type |
Required |
Default |
Markup |
Description |
|
Create and populate dataset with bucket information |
Text |
Yes |
(Empty) |
RESULTDATASET="myDataset" |
Indicates the name of the dataset in which to populate the bucket's name and creation date with (More on datasets below under Datasets). |
Advanced
|
Property |
Type |
Required |
Default |
Markup |
Description |
|
Name |
Text |
No |
(Empty) |
HEADER NAME="myHeader" |
Specifies the "key" in a key-value pair. This is the handle that you assign to an object. In Amazon S3, details about each file and folder are stored in key value pairs called metadata or headers. System metadata is used and processed by Amazon S3, however, user metadata or custom headers can be specified by you. This adds more flexibility and enables you to better distinguish specific files by adding or editing custom headers on existing S3 objects or assigning custom headers to new objects. Press Click here to add new row... to add a key-value pair. Press the red X to remove an existing key-value pair. |
|
Value |
Text |
No |
(Empty) |
VALUE="theValue" |
Specifies the "value" in a key-value pair. This is the content that you are storing for an object. In Amazon S3, details about each file and folder are stored in key value pairs called metadata or headers. System metadata is used and processed by Amazon S3, however, user metadata or custom headers can be specified by you. This adds more flexibility and enables you to better distinguish specific files by adding or editing custom headers on existing S3 objects or assigning custom headers to new objects. Press Click here to add new row... to add a key-value pair. Press the red X to remove an existing key-value pair. |
Additional notes
S3 endpoints and regions
This table contains a complete list of Amazon Simple Storage Service endpoints, along with their corresponding regions, supported protocols and location constraints.
|
Endpoint |
Region |
Protocol |
Location Constraints |
| s3.amazonaws.com | US Standard * | HTTP and HTTPS | (none required) |
| s3.us-west-2.amazonaws.com | US West (Oregon) Region | HTTP and HTTPS | us-west-2 |
| s3.us-west-1.amazonaws.com | US West (Northern California) Region | HTTP and HTTPS | us-west-1 |
| s3.eu-west-1.amazonaws.com | EU (Ireland) Region | HTTP and HTTPS | EU |
| s3.ap-southeast-1.amazonaws.com | Asia Pacific (Singapore) Region | HTTP and HTTPS | ap-southeast-1 |
| s3.ap-southeast-2.amazonaws.com | Asia Pacific (Sydney) Region | HTTP and HTTPS | ap-southeast-2 |
| s3.ap-northeast-1.amazonaws.com | Asia Pacific (Tokyo) Region | HTTP and HTTPS | ap-northeast-1 |
| s3.sa-east-1.amazonaws.com | South America (Sao Paulo) Region | HTTP and HTTPS | sa-east-1 |
* The US Standard region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps.
Datasets
A dataset is a multiple column, multiple row container object. This activity creates and populates a dataset containing a specific set of fields in addition to the standard dataset fields. The table below describes these fields (assuming the dataset name assigned was theDataset).
|
Name |
Type |
Return Value |
|
theDataset.BucketName |
Text |
Returns the name of the bucket. |
|
theDataset.CreationDate |
Text |
Returns the creation date/time of the bucket. |
Example
- The sample AML code below can be copied and pasted directly into the Steps Panel of the Task Builder.
- Parameters containing user credentials, files, file paths, and/or other information specific to the task must be customized before the sample code can run successfully.
Description
Create and populate dataset "myDataset" with buckets information. Use "Session1" S3 session.
|
<AMAWSS3 ACTIVITY="list_buckets" RESULTDATASET="myDataset" SESSION="Session1" />
|