Amazon S3 - List bucket(s)

Amazon S3 - List Bucket(s)

Declaration

<AMAWSS3 ACTIVITY="list_buckets" ACCESSKEY="text" SECRETKEY="text 
(encrypted)" USERAGENT="text" MAXERRORRETRY="number" 
SERVICEURL="text" PROXYHOST="text" PROXYPORT="number" 
PROXYUSER="text" PROXYPWD="text (encrypted)" 
RESULTDATASET="text"><HEADER NAME="text" 
VALUE="text" /></AMAWSS3>

Description: Retrieves all bucket names and their creation dates and stores the results in an dataset.

IMPORTANT: The AWS S3 activities are performed using Amazon's Simple Storage Service engine, therefore, launching and operating Amazon S3 a valid Access Key ID and Secret Access Key.

Practical Usage

Used to retrieve the name and creation date of one or more buckets.

Connection Parameters

Property	Type	Required	Default	Markup	Description
Connection					Indicates where AWS user credentials and preferences should originate from. This is a design mode parameter used only during task construction and configuration, thus, comprises no markup. The available options are: Host (default) - Specifies that user credentials and/or advanced preferences are configured individually for this activity. This option is normally chosen if only a single activity is required to complete an operation. Session - Specifies that user credentials and/or advanced preferences are obtained from a pre-configured session created in an earlier step with the use of the S3 - Create session activity. This option is normally chosen if a combination of related activities are required to complete an operation. Linking several activities to a single session eliminates redundancy. Additionally, a single task supports construction and simultaneous execution of multiple sessions, improving efficiency.
Session	Text	Yes if connection is session-based	EC2Session1	SESSION="S3Session1"	The name of an existing session to attach this activity to. This parameter is active only if the Connection parameter is set to Session. The default session name is 'S3Session1'.
Access key	Text	Yes if connection is host-based	(Empty)	ACCESSKEY= "022QF06E7MXBSH9DHM02"	A 20-character alphanumeric string that uniquely identifies the owner of the AWS service account, similar to a username. This key along with a corresponding secret access key forms a secure information set that AWS uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host.
Secret Access key	Text	Yes if connection is host-based	(Empty)	SECRETKEY= "kWcrlUX5JEDGM/LtmEENI/ aVmYvHNif5zB+d9+ct"	A 40-character string that serves the role as password to access the AWS service account. This along with an associated access key forms a secure information set that EC2 uses to confirm a valid user's identity. This parameter is active only if the Connection parameter is set to Host.
Protocol	Text (options)	No	HTTP	PROTOCOL="HTTPS"	The protocol required. The available options are: HTTP (default) HTTPS
User agent	Text	No	AutoMate	USERAGENT="AutoMate"	The name of the client or application initiating requests to AWS, which in this case, is AutoMate. This parameter's default value is 'AutoMate'.
Service URL	Text	No	(Empty)	SERVICEURL= "https://s3.eu-west-1.amazonaws.com"	The URL that provides the service endpoint. To make the service call to a different region, you can pass the region-specific endpoint URL. For example, entering https://s3.us-west-1.amazonaws.com points to US West (Northern California) region. A complete list of S3 regions, along with associated endpoints and valid protocols can be found below under S3 Endpoints and Regions.
Maximum retry on error	Number	No	(Empty)	MAXERRORRETRY="4"	The total amount of times this activity should retry its request to the server before returning an error. Network components can generate errors anytime in the life of a request, thus, implementing retries can increase reliability.
Proxy host	Text	No	(Empty)	PROXYHOST="proxy.host.com"	The host name (e.g., server.domain.com) or IP address (e.g., xxx.xxx.xxx.xxx) of the proxy server to use when connecting to AWS.
Proxy port	Number	No	(Empty)	PROXYPORT="1028"	The port that should be used to connect to the proxy server.
Proxy username	Text	No	(Empty)	PROXYUSER="username"	The username that should be used to authenticate connection with the proxy server (if required).
Proxy password	Text	No	(Empty)	PROXYPWD="encrypted"	The password that should be used to authenticate connection with the proxy server (if required).

Bucket Parameters

Property	Type	Required	Default	Markup	Description
Create and populate dataset with bucket information	Text	Yes	(Empty)	RESULTDATASET="myDataset"	Indicates the name of the dataset in which to populate the bucket's name and creation date with (More on datasets below under Datasets).

Advanced Parameters

Each Amazon S3 object has a set of key-value pairs with which it is associated called Headers or Metadata. Metadata can provide important details about an object, such as file name, type, date of creation/modification etc. There are two kinds of metadata in S3; system metadata, and user metadata. System metadata is used and processed by Amazon S3. User metadata (also known as custom header) is specified by you, the user. Amazon S3 simply stores it and passes it back to you upon request. S3 lets you to store your personal information as custom headers or user metadata such as First Name, Last Name, Company Name, Phone Numbers, etc, so that you can distinguish specific files. Using this parameter, you can add new custom header/user metadata to existing S3 objects, edit default S3 metadata on a bucket or store/upload new objects with custom header or metadata.

Property	Type	Required	Default	Markup	Description
Name	Text	No	(Empty)	HEADER NAME="myHeader"	Specifies the "key" in a key-value pair. This is the handle that you assign to an object. In Amazon S3, details about each file and folder are stored in key value pairs called metadata or headers. System metadata is used and processed by Amazon S3, however, user metadata or custom headers can be specified by you. This adds more flexibility and enables you to better distinguish specific files by adding or editing custom headers on existing S3 objects or assigning custom headers to new objects. Press Click here to add new row... to add a key-value pair. Press the red X to remove an existing key-value pair.
Value	Text	No	(Empty)	VALUE="theValue"	Specifies the "value" in a key-value pair. This is the content that you are storing for an object. In Amazon S3, details about each file and folder are stored in key value pairs called metadata or headers. System metadata is used and processed by Amazon S3, however, user metadata or custom headers can be specified by you. This adds more flexibility and enables you to better distinguish specific files by adding or editing custom headers on existing S3 objects or assigning custom headers to new objects. Press Click here to add new row... to add a key-value pair. Press the red X to remove an existing key-value pair.

Description tab - A custom description can be provided on the Description tab to convey additional information or share special notes about a task step.

Error Causes tab - Specify how this step should behave upon the occurrence of an error. (Refer to Task Builder > Error Causes Tab for details.)

On Error tab - Specify what AWE should do if this step encounters an error as defined on the Error Causes tab. (Refer to Task Builder > On Error Tab for details.)

S3 Endpoints and Regions

This table contains a complete list of Amazon Simple Storage Service endpoints, along with their corresponding regions, supported protocols and location constraints.

Endpoint	Region	Protocol	Location Constraints
s3.amazonaws.com	US Standard *	HTTP and HTTPS	(none required)
s3.us-west-2.amazonaws.com	US West (Oregon) Region	HTTP and HTTPS	us-west-2
s3.us-west-1.amazonaws.com	US West (Northern California) Region	HTTP and HTTPS	us-west-1
s3.eu-west-1.amazonaws.com	EU (Ireland) Region	HTTP and HTTPS	EU
s3.ap-southeast-1.amazonaws.com	Asia Pacific (Singapore) Region	HTTP and HTTPS	ap-southeast-1
s3.ap-southeast-2.amazonaws.com	Asia Pacific (Sydney) Region	HTTP and HTTPS	ap-southeast-2
s3.ap-northeast-1.amazonaws.com	Asia Pacific (Tokyo) Region	HTTP and HTTPS	ap-northeast-1
s3.sa-east-1.amazonaws.com	South America (Sao Paulo) Region	HTTP and HTTPS	sa-east-1

* The US Standard region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps.

Datasets

A dataset is a multiple column, multiple row container object. This activity creates and populates a dataset containing a specific set of fields in addition to the standard dataset fields. The table below describes these fields (assuming the dataset name assigned was theDataset).

Name	Type	Return Value
theDataset.BucketName	Text	Returns the name of the bucket.
theDataset.CreationDate	Text	Returns the creation date/time of the bucket.

Example

The sample AML code below can be copied and pasted directly into the Steps panel of the Task Builder.

Description: Create and populate dataset "myDataset" with bucket(s) information. Use "Session1" S3 session.

<AMAWSS3 ACTIVITY="list_buckets" RESULTDATASET="myDataset" 
SESSION="Session1" />

Copyright © Fortra, LLC and its group of companies.
All trademarks and registered trademarks are the property of their respective owners.
v10 | 202212080228 | December 2022