Retrieve Node
From Moving Pictures
The retrieve node is used for web request. Please review the attributes associated with this node.
| Attribute | Description |
|---|---|
| name | This is a required attribute that defines the return variable for this node. This will be the variable that holds the web response. |
| url | Either this attribute or the 'file' attribute is a required attribute that defines the URL that you want to request. |
| file | The name of a local file to use as input. Either this or the 'url' attribute is required. |
| useragent | This is an optional attribute where you can define the useragent header to use in the web request. |
| allow_unsafe_header | This is an optional attribute that will allow a web request accept responses that have non-compliant headers. (This is disabled by default. If you want this enabled, add this attribute to the retrieve node with a value of 'true'.) |
| use_caching | An optional attribute that specifies whether cached results should be used when the same URL has been retrieved multiple times. Valid values are "true" and "false". This is enabled by default. |
| encoding | This is an optional attribute that allows you to define the encoding type to use for the web request. |
| retries | This is an optional attribute that sets the number of retries a web request will attempt before moving on. This defaults to 5. |
| timeout | This is an optional attribute that sets the amount of time a web request will wait for a response before moving on. This defaults to 5000 (5 seconds). |
| timeout_increment | This is the amount of time to add to the timeout duration per retry. Default value is 2000 (2 seconds). So under default settings you have a timeout of 5000, 5 retries, and a timeout_increment of 2000 (5000 + (5 * 2000) = 10000). This yields a total time of 10 seconds before the web request officially times out. |
| cookies | An optional attribute that allows a cookie to be artificially injected into a web request. A cookie only needs to be supplied once and will automatically be used in all future requests to the same domain. A name value pair should be seperated by an equals sign and multiple cookies can be separated by a comma. For example: "user=johndoe,password=movpics" |
| method | An optional attribute that allows you to specify the method used to execute a request. The most common values are "GET" (default) and "POST". If using the POST method it is not currently possible to supply a payload to be sent. |
Example:
<retrieve name="search_page" url="http://www.filmweb.pl/szukaj?c=film&q=${search.title:safe}" allow_unsafe_header="true" />
This example does a web request for the page located at the URL and assigns the return on this request to a variable named 'search_page'. I can call this variable in later nodes with ${search_page}.