The link scanner function will allow you to automatically check some/all of the links in your database to make sure the URLs are still working and that they are following your rules for blacklist items and reciprocal links. This document will describe how to setup and run the link scanner through the LinkX control panel.
Viewing Scanner Configurations
To view the scanner configurations that have been created, access the Links > Scan Links function from the control panel menu. This will display a page where you will be able to see all of the scanner configurations that are currently defined.
In the table that displays the scanner configurations, you will find several functions that you can run on each configuration. Each of those functions are described below.
Click this icon to view the results of the last scan completed using this configuration | |
Click this icon to start the link scanner with this configuration | |
Click this icon to stop the link scanner with this configuration | |
Click this icon to edit this scanner configuration | |
Click this icon to delete this scanner configuration |
Starting/Stopping the Scanner
The link scanner requires that your server have the CLI (command line interface) version of PHP installed in one of the common binary directories. If LinkX is unable to find the CLI version of PHP
on your server, you won't be able to use the link scanner. In this case you should contact your server administrator and ask them to install the CLI version of PHP for you and/or make sure it is placed
in one of the common binary directories on the server (/bin, /usr/bin, /usr/local/bin).
On some servers it may not be possible to start the link scanner through the LinkX control panel. If your server's PHP installation is running in safe mode or has the exec() function disabled you will not be able to start the link scanner through the LinkX control panel. In that case, please see the alternate scanner starting methods for possible workarounds.
The link scanner will run as a background process on your server, so you may need to confirm with your server administrator that you are allowed to run background processes.
On some servers it may not be possible to start the link scanner through the LinkX control panel. If your server's PHP installation is running in safe mode or has the exec() function disabled you will not be able to start the link scanner through the LinkX control panel. In that case, please see the alternate scanner starting methods for possible workarounds.
The link scanner will run as a background process on your server, so you may need to confirm with your server administrator that you are allowed to run background processes.
To start the link scanner, first locate the configuration you want to run and make sure the status shows that it is Not Running. You can then click on the icon to start the link scanner. You will be asked to confirm that you want to start the scanner, and if you select ok the process of starting the scanner will continue. A confirmation box will appear to let you know that the process has been completed, and you should now see the Status value updated with the current progress of the scanner.
To stop the link scanner, first locate the configuration you want to stop and make sure the status shows that it is scanning links. You can then click on the icon to halt the scanning process. It may take a few minutes for the scanner to stop, so during that time it is normal for the Status to still show that the scanner is working.
Adding a Scanner Configuration
To add a new scanner configuration access the Links > Scan Links function from the control panel menu. This will display your scanner configurations that have been created. Near the top of the page in the header bar that reads "Link Scanner Configurations" click on the icon. This will display a popup window where you will be able to enter the settings for the scanner configuration that you want to add. Each of the form fields are described below.
Identifier | Enter a short identifier so you can |
Link Status | Select the link statuses you want to scan (or leave all unchecked to scan all) |
Link Type | Select the link types you want to scan (or leave all unchecked to scan all) |
Link Added | Enter a date range of when the links were added that you want to scan (leave blank to scan all) |
Link Modified | Enter a date range of when the links were last modified that you want to scan (leave blank to scan all) |
Link Scanned | Enter a date range of when the links were last scanned that you want to scan (leave blank to scan all) |
Categories | Select the categories that you want to scan, or leave blank to scan all categories. If you do select a category, it will also scan all of the links in that category's sub-categories. Click the icon to select categories or the icon to clear the current selected categories. You will use the Category Selector interface to select categories. |
Extract value from the site's <title> tag and use for the Title value If this option is checked and the site does not yet have a title set, the scanner will extract the title from the page referenced by the Site URL and assign that to the Title value for this site in your database. |
|
Extract value from the site's <meta> description tag and use for the Description value If this option is checked and the site does not yet have a description set, the scanner will extract the meta description from the page referenced by the Site URL and assign that to the Description value for this site in your database. |
|
Extract value from the site's <meta> keywords tag and use for the Keywords value If this option is checked and the site does not yet have keywords set, the scanner will extract the meta keywords from the page referenced by the Site URL and assign that to the Keywords value for this site in your database. |
Ignore | No action will be taken |
Display in report only | The link will be displayed in the scanner report |
Change link status to disabled | The link status will be changed to disabled and will be displayed in the scanner report |
Delete link from database | The link will be deleted from the database and will be displayed in the scanner report |
Delete link and blacklist | The link will be deleted from the database, the link data will be blacklisted, and the link will appear in the scanner report. The site URL, e-mail address, and submit IP will be added to the blacklist. |
Links with connection errors | Select the action to take for links that the scanner cannot connect to. This could be the result of a DNS lookup failure, the server being offline, or internet connection issues. For this issue type it is not recommended that you delete or blacklist links because this can simply indicate that the server is temporarily offline. |
Links that are broken URLs | Select the action to take for links that the scanner has found are no longer working. Broken URLs are those that return a HTTP status code of 400 or greater. For example, a 404 Not Found status code for pages that no longer exist, or a 500 Internal Server Error for pages that are causing a server error. |
Links that forward | Select the action to take for links that the scanner has found are forwarding to a different URL. Forwarding URLs are those that return a HTTP status code in the 300 range, for example 302 Found. The scanner will take into account the 'Allow URL redirection' setting for each link. If the link is allowed to redirect, it will not be flagged for this issue type if it produces a 300 level HTTP status code. |
Links with blacklisted data | Select the action to take for links that the scanner has found to contain one or more items on your blacklist. |
Links with no reciprocal link | Select the action to take for links that the scanner has found to not contain a link back to your site. Both the Site URL and the Recip URL pages will be checked for a reciprocal link. The scanner will take into account the 'Require a reciprocal link' setting for each link. If the link is not required to have a reciprocal link, it will not be flagged for this issue type. |
Editing a Scanner Configuration
To edit a scanner configuration you will fist need to locate it through the Link Scanner Configurations interface. When you locate the configuration you want to edit, click on the icon to bring up the editing interface. This interface is nearly identical to the scanner configuration creation interface, and has the same form fields with the same properties. You can make the changes that you want, and then press the Update Scanner Configuration button to save your changes.
Deleting a Configuration
To edit a scanner configuration you will fist need to locate it through the Link Scanner Configurations interface. When you locate the configuration you want to delete, click on the icon, which will ask you to confirm the deletion.
You can also delete several configurations at once by putting a check in the box next to the configurations you want to delete, then pressing the Delete button at the bottom of the page.
Alternate Methods for Starting/Stopping the Scanner
If you are not able to start the link scanner through the control panel, there is currently one alternate method to run the scanner. This requires that you have shell access to your server and are comfortable working with the server's shell environment. You can login to your server using your favorite shell access software. Once logged in, change into the admin directory of your LinkX installation and issue the following command: