Issues with the Broken Links Synthetic

I’m trying to use the built in Broken Links Synthetic, but running into a couple of issues.
First, it appears that it fails to access the instagram and linkedin links on my home page 100% of the time. I’ve confirmed the links are working and that this is due to linkedin and instagram rejecting the crawler.
This would be fine, as I can confirm the availability of those sites through their APIs; however, I cannot determine a way to exclude those links from the monitor, so that the synthetic doesn’t just fail 100% of the time.
Is there a way to tune the synthetic or alerts on the synthetic to account for the fact these links will always fail the built in Broken Links synthetic?

Hi, @epinero: Unfortunately, it is not possible to customize the behavior of the Broken Links monitor (and the fact that LinkedIn returns a status code of 999 is well-known, so we really should be accounting for it; I will pass this along to the Synthetics team). The alternative is to use a Scripted Browser monitor to check the links on your page:

const assert = require('assert');
const $http = require('request');
const parse = require('url-parse');
const By = $driver.By;

// Replace with the URL you want to check for broken links
const urlToCheck = 'https://www.example.com/';

const domainsToIgnore = [
  'www.instagram.com',
  'www.linkedin.com'
];

$browser.get(urlToCheck)
  .then(function() {
    // find all 'a' elements
    return $browser.findElements(By.tagName('a'));
  })
  .then(function(links) {
    // for each link...
    return links.forEach(function(link) {
      // get link target
      return link.getAttribute('href')
        .then(function(href) {
          // if href is not missing or empty...
          if (href != null && href != '') {
            // skip mailto and tel links
            if (!href.startsWith('mailto') && !href.startsWith('tel')) {
              var domain = parse(href).hostname;
              // if target is not in list of excluded domains...
              if (!domainsToIgnore.includes(domain)) {
                // send HEAD request and check response
                return $http.head(href, callback);
              }
            }
          }
        });
    });
  });

  function callback(error, response, body) {
    if (response != null) {
      assert.ok(response.statusCode != 404, 'Broken link found: ' + response.request.uri.href);
    }
  }
1 Like