HTTP (Hypertext Transfer Protocol) Action

HTTP stands for Hypertext Transfer Protocol. It's the network protocol used to transmit files and other data (collectively called resources) on the World Wide Web, whether they're HTML files, images, query results, or anything else. Usually, HTTP takes place through TCP/IP sockets. AWE's HTTP action contains activities that perform basic HTTP related operations, such as GET, PUT, POST, and DELETE. It supports Hypertext Transfer Protocol Secure (HTTPS) protocols, which is a combination of the Hypertext Transfer Protocol with the SSL/TLS protocol to provide encrypted communication and secure identification of a network web server.

See Also: FTP, Connect to Network, Execute Web Service, Get Email, Map Drive, MSMQ, Send E-mail, Twitter

Properties Tab

In the Activity to perform box, select an activity, then configure the properties for that activity. The available properties depend on the activity selected.

To set HTTP properties

  1. In the Available Actions pane, open the Network folder and double-click the HTTP action or drag it into the Steps pane.

  2. On the Properties tab, select an Activity to perform.

  3. Specify the properties that appear, depending on the activity selected.

  4. To set the Description tab properties, refer to Description Tab Options.

  5. To set the Error Causes and On Error tab properties, refer to Step Error Handling.

  6. When finished, click OK to save settings and close the properties dialog box.

Available Activities

The following table briefly describes the available activities for this action as specified in the Activity to perform box.

Activity

Description

Declaration

Examples

Delete

Deletes the specified resource and optionally populates a variable with the results and/or a dataset with information about request and response data.

NOTE: HTTP DELETE is not widely supported on public servers due to security risks. Before using the DELETE activity, it may be worth checking if the DELETE method is supported by the server using the OPTIONS activity.

Used by supplying a URL to delete the resource at and reading the response from the server.

<AMHTTP ACTIVITY="delete" URL="text" COOKIEFILE="text" OVERWRITECOOKIEFILE="YES/NO" RESULTVARIABLE="text" RESULTDATASET="text" AUTHENTICATION="text (options)" USERNAME="text" PASSWORD="text" VERSION="text (options)" CERTIFICATE="text" CERTIFICATEPASSPHRASE="text" IGNOREINVALIDCERTIFICATE="YES/NO"><HEADER NAME="text" VALUE="text" /><COOKIE NAME="text" VALUE="text" /></AMHTTP>

This sample code deletes url "www.sampleURL.com". It is set to block cookies and ignore invalid server certificates. It creates and populates dataset "theDataset" and stores result data in automate variable "theVar".

<AMHTTP ACTIVITY="delete" URL="www.sampleURL.com" BLOCKCOOKIES="YES" RESULTVARIABLE="theVar" RESULTDATASET="theDataset" IGNOREINVALIDCERTIFICATE="YES" />

Get

Requests a representation of the specified resource or downloads a file using the HTTP protocol from the specified URL. Optionally populates a variable with the results and/or a dataset request and response data as well as other relevant information about the action performed.

Useful when specific data or a file must be downloaded via the HTTP protocol and HTML does not need to be rendered inside an external browser. File data can be saved to a file or put into a variable for further processing.

<AMHTTP URL="text" COOKIEFILE="text" OVERWRITECOOKIEFILE="YES/NO" RESULTVARIABLE="text" RESULTDATASET="rwxt" AUTHENTICATION="text (options)" USERNAME="text" PASSWORD="text" VERSION="HTTP10" CERTIFICATE="text" CERTIFICATEPASSPHRASE="text" IGNOREINVALIDCERTIFICATE="YES/NO" FILE="text"><HEADER NAME="text" VALUE="text" /><COOKIE NAME="text" VALUE="text" /></AMHTTP>

<AMHTTP URL="http://www.networkautomation.com/index.htm" FILE="c:\temp\index.htm" RESULTVARIABLE="theResult" AUTHENTICATION="digest" USERNAME="text" PASSWORD="AM1bsEv2xlC2N9qWN7qDphOEw==aME" IGNOREINVALIDCERTIFICATE="YES" />

Head

Identical to the HTTP Get activity except that the server does not return a message-body in the response. The meta information contained in the HTTP headers in response to an HTTP Head request is identical to the information sent in response to a HTTP Get request. This activity can be used for obtaining information about the entity implied by the request without transferring the entity-body itself.

Often used for testing hypertext links for validity, accessibility, and recent modification.

 

 

Options

Returns the HTTP methods that the server supports for specified URL. This allows the client to determine the options and/or requirements associated with a resource, or the capabilities of a server, without implying a resource action or initiating a resource retrieval. This can be used to check the functionality of a web server by requesting '*' instead of a specific resource.

 

 

Post

Submits data to be processed (i.e. from an HTML form) to the identified resource. The data is included in the body of the request. This may result in the creation of a new resource or the updates of existing resources or both. Optionally populates a variable with result data and/or populates a dataset with request and response information.

The difference between POST and PUT is reflected in the different meaning of the request URL. The URL in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URL in a PUT request identifies the entity enclosed with the request.

Useful when data must be posted or uploaded via HTTP protocol without using an external browser.

<AMHTTP ACTIVITY="post" URL="text" DATA="text" ACTION="text (options)" BLOCKCOOKIES="YES/NO" CLEARCOOKIES="YES/NO" RESULTVARIABLE="text" AUTHENTICATION="text (options)" USERNAME="text" PASSWORD="text (encrypted) TIMEOUT="number" CONTENTTYPE="text" USERAGENT="text" CERTIFICATE="text" CERTIFICATEPASSPHRASE="text (encrypted)" IGNOREINVALIDCERTIFICATE="YES/NO" PROXYTYPE="text )options)" PROXYSERVER="text" PROXYPORT="number" PROXYUSERNAME="text" PROXYPASSWORD="text (encrypted)" />

 

Put

Uploads a representation of the specified resource. The difference between POST and PUT is reflected in the different meaning of the request URL. The URL in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URL in a PUT request identifies the entity enclosed with the request.

NOTE: HTTP Put is not widely supported on public servers due to security concerns.

<AMHTTP ACTIVITY="put" URL="text" COOKIE="text (options)" COOKIEFILE="text" RESULTVARIABLE="text" RESULTDATASET="text" AUTHENTICATION="digest" USERNAME="text" PASSWORD="AM2T0ncdJ4XqLI7Sbl05hfcsg==aME" TIMEOUT="number" USERAGENT="text" CERTIFICATE="text" CERTIFICATEPASSPHRASE="text (options)" TYPE="text (options)" DATA="text" CONTENTTYPE="text" />

You can designate a remote URL and a filename (path) to offload; however, you MUST also specify the remote PATH to which you want to offload the file. On the Properties tab, under General, in the drop-down box click  File, then in the text box designate the source file. In the URL box, append the destination path + filename to the remote host URL. For example:
http://192.168.101.117/path/destinationname.ext.

Trace

Echoes back the received request, so that a client can see what (if any) changes or additions have been made by intermediate servers.

Can be used as a debug tool by resounding input back to the user.

<AMHTTP ACTIVITY="trace" URL="text" COOKIEFILE="text" OVERWRITECOOKIEFILE="YES/NO" RESULTVARIABLE="text" RESULTDATASET="text" AUTHENTICATION="text (options)" USERNAME="text" PASSWORD="text (encrypted)" TIMEOUT="number" USERAGENT="text" CERTIFICATE="text" CERTIFICATEPASSPHRASE="text (encrypted)" IGNOREINVALIDCERTIFICATE="YES/NO" HEADER NAME="text" VALUE="text" /><COOKIE NAME="text" VALUE="text" /></AMHTTP>

 

 

General Properties

Property

Type

Required

Default

Markup

Description

URL

Text

Yes

(Empty)

URL="www.google.com"

The URL (Uniform Resource Locator) to the resource that should be deleted.

Local File

Text

Yes

(Empty)

FILE="c:\downloadfile.htm"

A valid path and filename in which to save the file that is downloaded. If the file does not exist, it will be created during runtime.

Overwrite

Yes/No

No

Yes

OVERWRITE="YES"

If set to YES (default), specifies that, if files already exist locally, they should be overwritten.

Type

Text (options)

No

Text

TYPE="file"

Indicates the type of data to post. The available options are:

  • Text: Data type to post is a text string. Requires entry of the text string to post in the provided data field.

  • File: Data type to post is a specific file. Requires entry of the path and file name in the provided data field.

  • Field: Data type to post is a field. Typically used to submit form data to an HTTP server.

Data (text box)

Text

Yes

(Empty)

DATA="The text string"

If the data type selected in the Type parameter is Text, this parameter specifies the string of data to post. If the data type selected is File, this parameter specifies the path and file name to post. This parameter is active only if Type is set to Text or File. If Field is selected, this parameter is ignored.

File field

Text

Yes

(Empty)

FILEFIELD="

The field (i.e. HTML form data) to upload. This parameter is available only of the Type parameter is set to Field.

File

Text

Yes

(Empty)

DATA="a"

The file that contains the field or form data to upload. This parameter is available only of the Type parameter is set to Field.

Request fields

Text

No

(Empty)

NAME="theName"

VALUE="theValue"

Allows you to add custom fields to the request. Required parameters are:

  • Name - String that specifies the field/form name.

  • Value - String that specifies the field/form value.

Populate variable with result data

Text

No

(Empty)

RESULTVARIABLE="myVar"

The name of an existing variable to populate with the header information.

Dataset to populate

Text

No

(Empty)

RESULTDATASET="theDataset"

The name of the dataset to populate with request and response data as well as other relevant information about the action performed. More details regarding the fields that this dataset creates can be found below under Datasets.

Block cookies

Yes/No

No

No

BLOCKCOOKIES="YES"

If set to YES, HTTP cookies are automatically blocked. If set to NO (default) cookies are allowed.

NOTE: Blocking cookies can help protect your privacy but it might limit your experience on some websites/servers.

Cookie file

Text

No

(Empty)

COOKIEFILE="myCookie.Text

The cookie file that this activity should use to identify and/or authenticate a server-based session. Cookies can be stored in different locations, depending on the version of explorer and Windows you are using. This parameter is available only if the Block Cookies parameter is set to NO.

Overwrite cookie file

Yes/No

No

No

OVERWRITECOOKIEFILE="YES"

HTTP requests and responses are written to the same cookie file. By default, new request/response data is added to the end of the file as a new line. If this parameter is set to YES, new request/response data will overwrite existing data. Set to NO by default.

Credentials Properties

Property

Type

Required

Default

Markup

Description

Authentication Type

Text (options)

No

None

AUTHENTICATION="digest"

The type of authentication that this activity should use. The available options are:

  • None (default)

  • Basic - Based on a username and password, this is the authentication mechanism defined in the HTTP/1.0.

  • Digest - Like Basic, Digest Authentication authenticates a user based on a username and a password. However the authentication is performed by transmitting the password in an ENCRYPTED form which is much MORE SECURE than the simple base64 encoding used by Basic Authentication.

  • NTLM - A challenge-response scheme, consisting of three messages, commonly referred to as Type 1 (negotiation), Type 2 (challenge) and Type 3 (authentication).

Username

Text

No

(Empty)

USERNAME="Coolguy"

A valid user name to be used during the authentication. If the Authentication Type parameter is set to None, this parameter is ignored.

Password

Text

No

(Empty)

PASSWORD="encrypted"

The password to be used during the authentication. If created in visual mode, this value is automatically encrypted. If the Authentication Type parameter is set to None, this parameter is ignored.

Advanced Properties

Property

Type

Required

Default

Markup

Description

Protocol Version

Text (options)

No

HTTP 1.1

VERSION="HTTP10"

The protocol version to use. The available options are:

  • HTTP/1.0 - Uses a separate connection to the same server for every request-response transaction.

  • HTTP/1.1 - Can reuse a connection multiple times allowing less latency.

User Agent

Text (options)

Yes

AutoMate

USERAGENT="text"

Sometimes it's necessary for a website to understand how it is being viewed, so most user agents identify themselves by sending a User Agent String to the website. The User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations (the default User Agent entered is AutoMate).

Timeout

Number

No

10

TIMEOUT="50"

The maximum time that the step should wait for a response from the HTTP server before failing with a time out error. It may be useful to increase this value when posting extremely large amounts of data or if waiting for a long transaction to complete. (default value is 10 seconds).

Measure

Text (options)

No

seconds

Measure="minutes"

The time measure that should be used in combination with the value entered in the Timeout parameter. The available options are:

  • Milliseconds

  • Seconds (default)

  • Minutes

  • Hours

Certificate File

Text

No

(Empty)

CERTIFICATE="certfile"

The path and file name of the certificate file used to validate connection. AWE supports the following certificate file types:

  • Internet Security Certificate (.cer) - Security file provided by a third party such as VeriSign or Thwate that confirms the authenticity of a website; installed on a Web server (such as Apache or Windows Advanced Server) to authenticate the validity of a certain website hosted on the server.

  • Privacy Enhanced Mail Certificate (.pem) - Base64-encoded certificate file, such as a PEM-encoded X509 certificate; used to authenticate a secure website; typically imported from a Unix-based Apache Web server and compatible with OpenSSL applications. PEM certificate files are generated automatically and are not meant to be opened or edited manually.

  • PKCS #12 Certificate File (.pfx) - Encrypted security file that stores secure certificates used to authenticate a person or device, such as a computer or Web server; requires a password to be opened; can be installed by right-clicking the file and selecting "Install PFX."

  • Personal Information Exchange (.p12) - SSL certificate used to authenticate a secure website on a Microsoft IIS Web server.

Passphrase

Text

No

(Empty)

PASSPHRASE="encrypted"

The passphrase used to authenticate connection. A passphrase is a password that may comprise of a whole phrase.

Ignore invalid server certificate

Yes/No

No

No

IGNOREINVALIDCERTIFICATE="yes"

If set to YES, specifies that this step will ignore invalid certificates when connecting to a server using SSL. This parameter is set to NO by default.

Request Headers

Text

No

(Empty)

NAME="theName"

VALUE="theValue"

Allows you to add custom HTTP headers to the request. Required parameters are:

  • Name - String that specifies the header name.

  • Value - String that specifies the header value.

Request Cookies

Text

No

(Empty)

NAME="theName"

VALUE="theValue"

Allows you to add custom HTTP cookies to the request. Required parameters are:

  • Name - String that specifies the cookie name.

  • Value - String that specifies the cookie value.

Proxy Properties

Property

Type

Required

Default

Markup

Description

Proxy Type

Text (options)

No

System Default

PROXYTYPE="Socks5"

The proxy protocol that should be used. If you are unsure of the value to use in this parameter, contact your network administrator. The available options are:

  • System Default (default, recommended): The settings specified globally in the system preferences are used.

  • None: Explicitly instructs the action not to use a proxy server regardless of the system default

  • Socks 4: Instructs the action to go through a Socks 4 compliant server

  • Socks 4a: Instructs the action to go through a Socks 4a compliant server

  • Socks 5: Instructs the action to go through a Socks 5 compliant server

  • HTTP: Instructs the action to go through an HTTP compliant server

NOTE: To create the most portable tasks, it is best to specify System Default and specify proxy settings globally in the system settings By proceeding in this manner, tasks created in one environment that may have a proxy server will be portable to others that do not - and vice-versa.

Proxy Host

Text

No

(Empty)

a)PROXYSERVER="proxy.host.com"

b)PROXYSERVER="123.456.789.123"

The host name (server.domain.com) or IP address (xxx.xxx.xxx.xxx) of the proxy server.

Proxy Port

Number

No

1028

PROXYPORT="3000"

The port that should be used to connect to the proxy server. Most proxy servers operate on port 1028 (the default) however the proxy server can be customized to operate on other ports.

Proxy Username

Text

No

(Empty)

PROXYUSERNAME="HulkHogan"

The user name that should be used to authenticate when connecting through the proxy server. This option is only valid when Socks 5 or HTTP is specified in the Proxy type parameter as they are the only versions that support authentication.

Proxy Password

Text

No

(Empty)

PROXYPASSWORD="encrypted"

The password that should be used to authenticate when connecting through the proxy server. When the step is created in visual mode using the Task Builder it is written to the task encrypted. This option is only valid when Socks 5 or HTTP is specified in the Proxy type parameter.

Description Tab

The Description tab allows you to customize the text description of any step as it appears in the Task Builder's Steps Pane.

More on setting custom step description

Error Causes Tab

The Error Causes tab allows you to select/omit specific errors that should cause a particular step to fail.

More about Error Causes properties

On Error Tab

The On Error tab allows you to determine what the task should do if a particular step encounters an error.

More about On Error properties

Datasets

A dataset is a multiple column, multiple row container object. This activity creates and populates a dataset containing a specific set of fields in addition to the standard dataset fields. The table below describes these fields (assuming the dataset name assigned was theDataset).

Name

Type

Return Value

theDataset.RequestHeader

Text

Returns the request header. The header fields define various characteristics of the data transfer that is requested or the data that is provided in the message body. HTTP request header fields contain the parameters of an HTTP request operation.

theDataset.RequestCookies

Text

Gets a collection of cookies sent by the client.

theDataset.Data

Text

The progress of the snapshot, in percentage.

theDataset.ResponseHeader

Text

Returns the response header. HTTP response header fields contain the parameters of an HTTP response operation.

theDataset.ResponseCookies

Text

Gets the response cookie collection.

Variables and Expressions

All text fields allow the use of expressions, which can be entered by surrounding the expression in percentage signs (example: %MYVARIABLE%, % Left('Text',2)%). To help construct these expressions, you can open Expression Builder from these fields by pressing F2.

More on variables

More on expressions

More on the expression builder