Scripted browser not working

In our company we are trying to use Scripted browser feature in New Relic and it is not working for me to find a link on the page. For Example, if you go to www.dayrize.com , at the bottom of the page, you will find “The Dayrize Score” link. I am trying to handle it from scripted browser script but it is giving error. Infact it not working for any of the link in the page

For example, here is the script I am using to run
var assert = require(‘assert’);

$browser.get('http://dayrize.com’).then(function(){
// Check the H1 title matches “Example Domain”
return $browser.waitForAndFindElement($driver.By.className(‘cookieConsent-closeButton-2Fe’), 7000).click()
return $browser.takeScreenshot()
}).
then(function(){
//Call the wait function to wait until the FAQ button appears.
return $browser.waitForAndFindElement($driver.By.linkText(“The Dayrize Score”), 7000).click()
});

=====

The script log error I could see is
Script execution failed: WebDriverError: unknown error: Element is not clickable at point (743, 986)
(Session info: chrome=72.0.3626.121)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 5.4.0-1041-aws x86_64)
WebDriverError: unknown error: Element is not clickable at point (743, 986)
(Session info: chrome=72.0.3626.121)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 5.4.0-1041-aws x86_64)
at Object.checkLegacyResponse (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/error.js:546:15)
at parseHttpResponse (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/http.js:509:13)
at doSend.then.response (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/http.js:441:30)
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task: WebElement.click()
at Driver.schedule (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:807:17)
at WebElementPromise.schedule
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:2010:25)
at WebElementPromise.click (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:2092:17)
at eval (eval at JobResource.getScriptFn (/opt/runtimes/4.0.0/modules/synthetics-runner/lib/job-resource/index.js:79:19), :12:87)
at ManagedPromise.invokeCallback_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:1376:14)
at TaskQueue.execute_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2927:27)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process._tickCallback (internal/process/next_tick.js:68:7)

This tends to indicate that there is an instruction in the script to click the element once found, although this appears to be missing from the script you posted. The post below contains some sample script you can modify to ensure an element is visible in the window prior to clicking it.

1 Like

I have changed the script as below , but it is still not taking me to Home Page - Dayrize page.

var assert = require('assert');
var currentElement;

$browser.get('http://dayrize.com').then(function(){
// Check the H1 title matches "Example Domain"
return $browser.waitForAndFindElement($driver.By.className('cookieConsent-closeButton-2Fe'), 7000).click()
return $browser.takeScreenshot()
}).
then(function(){
//Call the wait function to wait until the FAQ button appears.

  return $browser.findElement($driver.By.linkText("Contact us"), 7000)
    .then(function(el){
    currentElement = el;
    el.getLocation()
    .then(function(location) {
      $browser.manage().window().setPosition(0, location.y);
      currentElement.click();
    });
  });

})

[/quote]

I’ve found that working with the scripted browser checks the functions for FindElement and linkText are super temperamental and rely a lot on the CSS and IDs to be very specific and distinct in naming.

I had to change any link I had with those types of references to use the XPath equivalent.

As an example, using your your “The Dayrize Score” link, you could change the reference to be this:

//Call the wait function to wait until the FAQ button appears.
return $browser.waitForAndFindElement(By.xpath("//*[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a"), 7000).click()
});

Definitely not as pretty and easy to find, but see if that works for this one.

Thanks for sharing that workaround @jfry! Great stuff!

The solution for doing with XPATH is nether working and below is the error

Script execution failed: InvalidSelectorError: invalid selector: Unable to locate an element with the xpath expression //[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a because of the following error:
SyntaxError: Failed to execute ‘evaluate’ on ‘Document’: The string '//
[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a’ is not a valid XPath expression.
(Session info: chrome=72.0.3626.121)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 5.4.0-1041-aws x86_64)
InvalidSelectorError: invalid selector: Unable to locate an element with the xpath expression //[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a because of the following error:
SyntaxError: Failed to execute ‘evaluate’ on ‘Document’: The string '//
[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a’ is not a valid XPath expression.
(Session info: chrome=72.0.3626.121)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 5.4.0-1041-aws x86_64)
at Object.checkLegacyResponse (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/error.js:546:15)
at parseHttpResponse (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/http.js:509:13)
at doSend.then.response (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/http.js:441:30)
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task: WebDriver.findElements(By(xpath, //*[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a))
at Driver.schedule (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:807:17)
at Driver.findElements (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:1048:19)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/until.js:243:23
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:938:14
at TaskQueue.execute
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2974:25)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task:
at pollCondition (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2195:19)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2191:7
at new ManagedPromise (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:1077:7)
at ControlFlow.promise (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2505:12)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2190:22
at TaskQueue.execute
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2974:25)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task: Timed-out waiting for element to be located using: By(xpath, //*[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a)
at scheduleWait (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2188:20)
at ControlFlow.wait (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2517:12)
at Driver.wait (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:934:29)
at Driver. (/opt/runtimes/4.0.0/modules/synthetics-runner/lib/webdriver/index.js:878:23)
at ManagedPromise.invokeCallback
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:1376:14)
at TaskQueue.execute_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2927:27)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task: WebElement.click()
at Driver.schedule (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:807:17)
at WebElementPromise.schedule
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:2010:25)
at WebElementPromise.click (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:2092:17)
at eval (eval at JobResource.getScriptFn (/opt/runtimes/4.0.0/modules/synthetics-runner/lib/job-resource/index.js:79:19), :13:135)
at ManagedPromise.invokeCallback_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:1376:14)
at TaskQueue.execute_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2927:27)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process._tickCallback (internal/process/next_tick.js:68:7)

Below is the code I am using:

var assert = require(‘assert’);

var currentElement;

$browser.get(‘http://dayrize.com’).then(function(){

// Check the H1 title matches “Example Domain”

return $browser.waitForAndFindElement($driver.By.className(‘cookieConsent-closeButton-2Fe’), 7000).click()

return $browser.takeScreenshot()

}).

then(function(){

return $browser.waitForAndFindElement($driver.By.xpath("//*[@id=‘root’]/main/footer/div/div/div/div/div[1]/div[4]/ul/li[2]/a"), 7000).click()

});

I just ran a test on this script and Synthetics says it worked:

/** CONFIGURATIONS **/

// Theshold for duration of entire script - fails test if script lasts longer than X (in ms)
var ScriptTimeout = 180000;
// Script-wide timeout for all wait and waitAndFind functions (in ms)
var DefaultTimeout = 30000;
// Change to any User Agent you want to use.
// Leave as “default” or empty to use the Synthetics default.
var UserAgent = “default”;

/** HELPER VARIABLES AND FUNCTIONS **/

const assert = require(‘assert’),
By = $driver.By,
browser = $browser.manage()
/** BEGINNING OF SCRIPT **/

console.log(‘Starting synthetics script: {DR}’);
console.log(‘Default timeout is set to ’ + (DefaultTimeout/1000) + ’ seconds’);

// Setting User Agent is not then-able, so we do this first (if defined and not default)
if (UserAgent && (0 !== UserAgent.trim().length) && (UserAgent != ‘default’)) {
$browser.addHeader(‘User-Agent’, UserAgent);
console.log('Setting User-Agent to ’ + UserAgent);
}

// Get browser capabilities and do nothing with it, so that we start with a then-able command
$browser.getCapabilities().then(function () { })
.then(() => {
logger.log(1, “Home Page - Dayrize”);
return $browser.get(“Home Page - Dayrize”);
})
.then(() => {
logger.log(2, “clickElement //a[contains(text(),‘The Dayrize Score’)]”);
return $browser.waitForAndFindElement(By.xpath("//a[contains(text(),‘The Dayrize Score’)]"), DefaultTimeout)
.then(function (el) {
el.click();
})
})
.then(function() {
logger.end();
console.log(‘Browser script execution SUCCEEDED.’);
}, function(err) {
logger.end();
console.log (‘Browser script execution FAILED.’);
throw(err);
});

//** Export Functions
const logger=(function (timeout=3000, mode=‘production’) {

var startTime = Date.now(),
    stepStartTime = Date.now(),
    prevMsg = '',
    prevStep = 0;


if (typeof $util == 'undefined'  ){
    $util = {
        insights: {
            set: (msg) => {
                console.log(`dryRun: sending to Insights using ${msg}`)
            }
        }
    }

}

function log(thisStep, thisMsg) {

    if (thisStep > prevStep && prevStep != 0) {
        end()
    }

    stepStartTime = Date.now() - startTime;

    if (mode != "production") {
        stepStartTime = 0

    }

    console.log(`Step ${thisStep}: ${thisMsg} STARTED at ${stepStartTime}ms.`);

    prevMsg = thisMsg;
    prevStep = thisStep;

}

function end() {
    var totalTimeElapsed = Date.now() - startTime;
    var prevStepTimeElapsed = totalTimeElapsed - stepStartTime;

    if (mode != 'production') {
        prevStepTimeElapsed = 0
        totalTimeElapsed = 0
    }

    console.log(`Step ${prevStep}: ${prevMsg} FINISHED. It took ${prevStepTimeElapsed}ms to complete.`);

    $util.insights.set(`Step ${prevStep}: ${prevMsg}`, prevStepTimeElapsed);
    if (timeout > 0 && totalTimeElapsed > timeout) {
        throw new Error('Script timed out. ' + totalTimeElapsed + 'ms is longer than script timeout threshold of ' + timeout + 'ms.');
    }
}

return {
    log,
    end
}

})(ScriptTimeout)

See if you can just grab the elements you need from it (or the whole thing, your call) and make it work in your script.

It’s still using XPath, but just came up with a different variation than I posted before.

:-1: :frowning: It still didn’t work for me. Have you tested this from NewRelic Scripted Browser?

Below is the error:

Starting synthetics script: {DR}
Default timeout is set to ’ + (DefaultTimeout/1000) + ’ seconds
Step 1: Home Page - Dayrize STARTED at 1ms.
Step 1: Home Page - Dayrize FINISHED. It took 78ms to complete.
Step 2: clickElement //a[contains(text(),‘The Dayrize Score’)] STARTED at 80ms.
Step 2: clickElement //a[contains(text(),‘The Dayrize Score’)] FINISHED. It took 44ms to complete.
Browser script execution FAILED.
Script execution failed: InvalidSelectorError: invalid selector: Unable to locate an element with the xpath expression //a[contains(text(),‘The Dayrize Score’)] because of the following error:
SyntaxError: Failed to execute ‘evaluate’ on ‘Document’: The string ‘//a[contains(text(),‘The Dayrize Score’)]’ is not a valid XPath expression.
(Session info: chrome=72.0.3626.121)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 5.4.0-1041-aws x86_64)
InvalidSelectorError: invalid selector: Unable to locate an element with the xpath expression //a[contains(text(),‘The Dayrize Score’)] because of the following error:
SyntaxError: Failed to execute ‘evaluate’ on ‘Document’: The string ‘//a[contains(text(),‘The Dayrize Score’)]’ is not a valid XPath expression.
(Session info: chrome=72.0.3626.121)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 5.4.0-1041-aws x86_64)
at Object.checkLegacyResponse (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/error.js:546:15)
at parseHttpResponse (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/http.js:509:13)
at doSend.then.response (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/http.js:441:30)
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task: WebDriver.findElements(By(xpath, //a[contains(text(),‘The Dayrize Score’)]))
at Driver.schedule (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:807:17)
at Driver.findElements (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:1048:19)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/until.js:243:23
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:938:14
at TaskQueue.execute
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2974:25)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task:
at pollCondition (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2195:19)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2191:7
at new ManagedPromise (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:1077:7)
at ControlFlow.promise (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2505:12)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2190:22
at TaskQueue.execute
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2974:25)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process.tickCallback (internal/process/next_tick.js:68:7)
From: Task: Timed-out waiting for element to be located using: By(xpath, //a[contains(text(),‘The Dayrize Score’)])
at scheduleWait (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2188:20)
at ControlFlow.wait (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2517:12)
at Driver.wait (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/webdriver.js:934:29)
at Driver. (/opt/runtimes/4.0.0/modules/synthetics-runner/lib/webdriver/index.js:878:23)
at ManagedPromise.invokeCallback
(/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:1376:14)
at TaskQueue.execute_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3084:14)
at TaskQueue.executeNext_ (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:3067:27)
at asyncRun (/opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:2927:27)
at /opt/runtimes/4.0.0/node_modules/selenium-webdriver/lib/promise.js:668:7
at process._tickCallback (internal/process/next_tick.js:68:7)

Yes, I tested in Synthetics against San Francisco before posting it.

I wonder if you’re running into some issues with copying and pasting quotes. You may have to manually retype all the single and double quotes (may be one or the other, but better safe than sorry).

I copy and pasted it into a new Synth check and it failed too. Then I realized the ScriptTimeout was outside the quoted scroll area, so it didn’t like my quote block…

I tried my original text and it worked again:

Attached it as a text file if that works better.
Dayrize_Synth_Check.txt (3.3 KB)

I really appreciate for the kind of patient response but somehow I am having each time I think it is solved. Now when I run the code you shared, it ran without errors. But if you see the screenshots,they are empty.
Secondly if I want to write other usecase and , If I replace for example, the text " The Dayrize Score" with “Partner Login” , it wont work