Web Scans

A scan will scan a client's network (layers 2-6) for vulnerabilities while the Web Scan specifically focuses on vulnerabilities found on web applications (layer 7). The Web Scan includes tests for things like cross site scripting and SQL injection.

There are a few different ways web scans are created, but an important note is that any FQDN or IP that a user wants scanned needs to be on a regular scan. You cannot perform a Web Scan on a hostname that is not associated with a scan.

Scans allow for multiple hosts to be on one scan while Web Scans are created per hostname. This is because the crawler must crawl the site to find the dynamic pages and then beSECURE scans the dynamic pages.

It is also important that the scanner IPs are whitelisted to allow access for our system to be able to perform the scan. Click Here for a list of IPs to whitelist.

To create a Web Scan

There are three ways to create a web scan, for all 3 opinions, first after logging into beSECURE, navigate into DevOps mode. DevOps mode is for scanning users meaning the user can view, edit and create new scans and web scans.

Option 1: Using the Create New Scan wizard

The “Create New Scan” dialog box will pop up. Fill out all the mandatory fields. Make sure the option “Create Web Scan” is checked (it should be checked by default).

NOTE: To enter multiple host names or IP addresses in the Range box, separate each with a comma (for example, 192.168.0.100,192.168.0.200).

Option 2: Adding a Web Scan to an existing scan

If a scan is already created without having included a Web Scan, the Web Scan can be enabled later by editing the existing scan.

  1. From the left navigation pane, click Scans > Scans List.

  2. Select the desire scan from the list.

  3. In the Web Scans section, select the desired hostname and/or IP address box(es) to create a corresponding Web Scan for each.

Option 3: Create a Web Scan from the Web Scan page

The Web Scan can be created directly on the Web Scan page as long as the hostname is already on a Scan.

NOTE: This option is considered a “quick add” meaning the crawler starts immediately after the scan is created.

  1. From the left navigation pane, click Scans > Scans List.

  2. Click the plus sign button (+) in the lower right corner to create a new scan.

  3. On the Settings > Main tab, complete the following parameters:

    1. Enter a name for the scan in the Scan Name box.

    2. Select an LSS (Local Scanning Server).

    3. Select an Organization .

    4. Enter the desired hostnames and IP addresses in the Hostname / IP Address Range box.

  4. Click the Reporting tab.

  5. Select a Contact Person.

  6. Click Create. The Settings > Main tab is displayed again.

  7. Under Web Scan, select the desired hostnames and IP addresses to create a corresponding Web Scan for each.

Overview of Settings Tabs for Web Scans

The Web Scan settings mirror the scan settings options:

Settings

  • Main - Shows required Web Scan details including Web Scan Name, LSS, Organization, Scan name the hostname is associated with, and hostname being scanned.

  • Authentication - The option to add authentication for authenticated Web Scans is an option but is not required. There are six options: Basic, NTLM, Web Login, Webtest, SSL Client- certifications, and Javascript (only relevant if using an On Premise scanner).

  • Tests - Shows the types of vulnerabilities that are being scanned for. There is an option to deselect any type of tests.

  • Crawler - The website is crawled to find the dynamic pages and then beSECURE will only scans the dynamic pages. There are two crawler options. The scrapper is the default crawler and is the only option on cloud 2/3. If using an on premise scanner, there is an option for the DOM crawler. DOM is the Javascript crawler.

    If there is a specific starting point that needs to be crawled that is not connected to the default URL, this can be added as a new starting point or only that single page can be crawled.

    Instead of having the website crawled for dynamic pages, if available, a list of dynamic pages can be imported as a CSV.

  • Configuration - beSECURE limits to 500 URLS crawled and scanned per website. Other settings on this tab include automatic start scanning (after the site is done with the crawling process), recrawl before starting to scan and turn off duplicate scripts detection.

Permissions

The permissions tab allows the Web Scan to be “Owned by” other users. Whoever owns the Web Scan can edit the scan and delete the scan. To give a user ownership of the scan, click on their users from the left side under “Available”. The user will now show up on the right side under “Assigned”.

Reporting

The reporting tab allows for the Web Scan notifications to be set up. A contact must be chosen (either an individual or a group of contact) and check the boxes for the notifications that need to be sent (Web Scan starts, Web Scan finishes, Web Scan results change, crawler done).

Scheduling

There is an option to set up a customized web scan schedules. Most commonly, web scans are scheduled to run at the same time as the scan. The scheduling tab will also show when the scan was mostly recently run and when the next schedule scan is scheduled.

NOTE: The schedule changes will not take effect until the “Modify Schedule” button is clicked and then the “Modify” button is clicked. The scan and web scan must be configured to use the same time.

Status

The Status tab shows if the scan is running or not. If the scan is running, a progress bar will be visible - There is an option in tab to stop the scan. The scan could be paused and restarted again another time. If the scan is not running, there is an option to disable the scan, meaning the scan will not run when it is scheduled but it is not deleted so the scan can be re-enabled and used again.

Other

Shows a comment area to display any important notes about the Web Scan. Example: Automatically created by the Scan Settings interface.

Other Key Options

From any tab from the Web Scan List, there are 5 important options located in the top right corner.

  • Immediate Scan - Run a scan right away out of schedule (if scheduled).

  • Modify - Save any changes that have been made to the scan.

  • Delete - Deletes the Scan.

  • View Scan Settings - View the scan settings that the Web Scan is associated with.

  • View Report - View the results for the scan and Web Scan (only available if a scan has ran already).

Using the Javascript Aware Crawler

When creating a web scan, a crawling job will be started immediately using the Scrapper Crawler Type (default). You will need to manually stop the crawler and switch it to the JavaScript Aware Crawler.

To select the JavaScript Aware Crawler Type, do the following:

  1. On the Web Scans Details page, select the Settings > Main tab.

  2. Scroll down to the Crawler Starting Points section.

  3. Select Stop for each active web scan.

  4. In the Crawler Type box for each web scan, select DOM.

  5. Select Crawl for each web scan.

  6. Once crawling is complete (or has been stopped), any crawled links will be added to the Dynamic or Static/Duplicate sections.

Starting a Web Scan

NOTE: Crawling and scanning a website are different actions.

Crawling and scanning a website are two different actions. Crawling a website doesn't always trigger a web scan as it depends on how the web scan was created and configured.

To automatically start a web scan at the end of a crawl job, do the following:

  1. On the Web Scans Details page, select the Settings > Configuration tab.

  2. Verify the Automatically start scanning checkbox is selected. The website can be recrawled, which will start a web scan after the initial crawling has been completed.

You can also select Immediate Scan from the upper-right corner of the Web Scan Details page to scan immediately.

Authentication by way of JavaScript Guide

Setting up JavaScript website authentication

  1. Log in to beSECURE.

  2. In the upper-left corner of the Home page, select DevOps.

  3. Select Scans > Web Scans List.

  4. On the Web Scan List page, select the web scan to authenticate.

  5. Under the Settings tab. select the Authentication tab.

  6. In the Web site requires authentication box, select via Javascript.

  7. In the Javascript Automation box, enter the desired JavaScript information (to generate a script, see Additional information and How to use Katalon Chrome extension).

  8. Select Modify to save your changes.

  9. Under Crawler Starting Points, set each site's Crawler Type to DOM, and then select Crawl.

Additional information

JavaScript login supports the following syntax: command | target | value |

NOTE:

  • A space is present after the last "|"

  • The Katalon Recorder Chrome extension can be used to record JavaScript sessions, export them using 'Sample for new formatters' as the output, and then provide it here (see below).

One of the following commands:

  • open - Access a site. Receives a target name in the form of a URL (for example, http://.. or https://).

  • click - Click on an element. Receives a target name which is one of the following

    • value of the 'name' attribute

    • value of the 'id' attribute, link

    • value for a href elements

    • a full xpath

      Copy
      Example
      click | name=username | |
      click | id=password | |
      click | link=Forgot password | |
      click | xpath=xpath=(.//*[normalize-space(text()) and normalize-space(.)=concat('Monitor
      your network', "'", 's readiness for attacks')])[1]/following::button[1] | |

  • type - Fill an input field with data. Received a target value of the id attribute or target value of the name attribute and fills it with the provided value. For example: 

    type | id=username | some_username
    Copy
    Complete example
    open | https://loginSSO.beyondsecurity.com/example |
    click | //a/b |
    click | link=Login here |
    click | //div[@id='root']/div/div/div/div/div/div/div/div[2]/div/div[2]/button/div[2] |
    type | id=identifierId | Yourname@example.com
    click | //div[@id='identifierNext']/span/span |
    type | name=password | ...Enter Password...
    click | //div[@id='passwordNext']/span/span |
    type | id=idvPin | ...Enter Pin...
    click | //div[@id='idvPreregisteredPhoneNext']/span |

How to use Katalon Chrome extension

To find the information required for the JavaScript Automation, downloand and install the Katalon Chrome extension, then do the following:

  1. Select the Katalon's icon in the top-right corner of the Chrome browser window.

  2. Select Record.

  3. Open a new tab in the Chrome browser window.

  4. Go to the URL of a site with a log in option (for example, https://accounts.google.com/).

  5. Enter the site's required credentials and log in.

  6. Return to the Katalon controls and select Stop.

  7. Select Export.

  8. In the Format box, select Sample for new formatters.

  9. Select Copy to Clipboard to save the script.

  10. In beSECURE, paste the script into the Javascript Automation box.

  11. Select Modify to save your changes.