Is it possible to have different signal loss threshold for different times?

Hey,

Is it possible to have different signal loss threshold for different times? Like, 20 mins for days and 1h for nights.

2 Likes

I’d been looking for a similar purpose as well, to have different alerting thresholds depending on the time of day - and the day itself (i.e., workdays vs weekend).

Though it’s not ideal, the workaround I’ve found is using hourOf(attr) - view docs. A usage example is shared by @AndyC in this thread.

  FROM Transaction
SELECT average(duration)
 FACET appName
 WHERE hourOf(timestamp) IN ('8:00', ... '18:00')
   AND weekdayOf(timestamp) NOT IN ('Saturday', 'Sunday')

It’s not ideal because the alert needs to be duplicated with a different set of hours for day, night or other time ranges. At least it enables greater control over various alert attributes, such as signal loss.

Hey Rishav, thanks. I’ve a question about your solution. Assume that we’ve prepared a “day” condition like you suggest. Then, doesn’t it mean that we’ll have “signal loss” alert for all night because of this?

That’s a great point I hadn’t considered. I suppose it would result in consistent false-positives for signal loss alerting, huh. I’ve only used this workaround for regular static/threshold alerts.

To dig in further, I guess we could add an alert-condition-specific muting rule to suppress false-positive alerts during the day/night (and night/day) transition period. Though, at that point, it seems a new feature request is better-suited to handle this use-case.

Maybe there’re other workarounds the NR Support folks might be able to share.

Indeed. Do you know how can we suggest a feature request? I mean, maybe there’s an official way to do this?

Should be able to create a ticket for NR Support for features requests, or any other queries.

https://support.newrelic.com/

Often times, the Support folks can also convert threads like these into feature requests as well. Though, I’m not sure if there’s an public/official list of requests we can see. It’d be really useful to have, especially to vote on them in order to help the teams prioritise individual impact.

1 Like

Hi @sedat.sevgili - You might be able to accomplish this by by setting up 2 different alert conditions in a policy and naming them something like Days and Nights. You can then use a NRQL alert condition and scope them to the hours that you would like them to alert by using WITH hourOf(attr) in your query as documented here. The example below would include data that came in from 1AM until 3AM.

hourOf(timestamp) IN ('1:00', '2:00', '3:00')

This would then allow to to create different thresholds (20mins and 60 minutes) for day and night respectively.

Note that the returned value does not include a prepended 0 for hours between 1am and 9am which is different than the SINCE clause.

I hope that helps!

1 Like

@dkoyano Thanks for your help!

But, with that, I’m afraid we’ll get “Signal Loss” alerts for Days conditions at nights and vice-versa, am I wrong?

Hello @sedat.sevgili

This is a great follow up question, but Im this is a little out of scope.

However fear not we have you covered, Im going to loop in an Alerts engineer here. They will be better equipped than I to support this. Please note they will reply here with their findings.

Should you have any updates regarding this or any further questions please do reach out! We are here to help!

Hi @dcody

Thanks for your help. We have not any update for now unfortunately.

Hi @sedat.sevgili

Thanks for reaching out.

I can confirm the engineer is working on this, however they are currently not in office, due to timezone difference.

However I will reach out to them on your behalf.

@dcody Great! Thanks!

Hi @sedat.sevgili I’m currently looking into this and will get back to you with a deeper answer tomorrow! I would like to touch base with a colleague on this!

Best,

1 Like

Hi @sarnce thank you!

Hi @sedat.sevgili is there a particular condition that you have set up right now that you are trying these things on or is this theoretical at this point? I might try to create some of the examples above by Derrik and start to update them as we find them not working ideally the way we want to. I believe your thoughts above are correct if you use Count this could also depend on your threshold in the condition if you are looking for above or below so there are some dependent thoughts that changes how you approach this I think. If you used Average instead of Count I believe that you wouldn’t get the loss of signals that you mentioned in your last post. Or something like a Filter Count If you have like a test condition with this set up, we might be able to see a bit more clearer.

Best,

Hi @sarnce ,

We have several conditions atm like this, so, it’s real. Here is an example:

SELECT count(*) FROM Segment WHERE event = 'Signed Up' AND request_source != 'webWoo' FACET request_source

This query counts the Signed Up events (request_source means iOS, Android, Web). And I want to have something like If our signing up process is broken for any request_source then we should be alerted. However we need to be alerted if we can not see any Signed Up for any request_source for 20 mins at days and 1h at nights. This is the case basically.

@sedat.sevgili - When it comes to ensuring reliability of a Web resource for something like a Web page, a Synthetics check might work better. You can create a scripted check to validate that the signup process works. You can also run this check from a specific geographic location to catch issues where the web page or functionality might not be available to actual users in Asia when it might still be available to users in Europe. Please see the documentation below for more information.

Get started with synthetic monitoring

2 Likes

Hi @dkoyano ,

Sure, let me check. Thank you!

@dkoyano I’ve checked the Synthetics, it’s nice and we can use it but for other needs. Because some of things that we want to monitor are not open to the public. So, that’s why, it’d be good to have different signal loss thresholds for different time periods.

@sedat.sevgili - For monitoring of resources not publicly available or on a private network you can always use private minions.

Private locations overview: Monitor internal sites and add new locations