Logs not showing up when using Log API

Same issue as here: Logs not available in NewRelic even after Logs Api returns request ID with 202 response

We need to log from a python app where we do not have access to the underlying infrastructure, but we do have the “requests” plugin available. We’ve worked our way through creating the appropriate headers and json request. We get a 202 response. However our logs do not show up anywhere in the UI.

When viewing NrIntegrationErrors you can see our previous failures, but since fixing our json formatting and having started receiving 202 responses we no longer receive failures with each request. The requests now just seem to go into the ether.

Thanks!

Hi @eburnside - looks like you had some success in getting some log data in on the 23rd, did you manage to get this resolved then?

I see we somehow got 11 entries through while we were trying different JSON formats and trying CURL vs Python CLI vs Python in a notebook. We probably made 3x-4x that number of requests.

Unfortunately we cannot seem to re-create it. We have tried several JSON formats, eg:

{"logtype": "accesslogs", "message": "hihi", "hostname": "desktop", "service": "processing-service", "timestamp": 1611696057}

[{"common": {"attributes": {"logtype": "accesslogs", "hostname": "desktop", "service": "processing-service"}}, "logs": [{"timestamp": 1611696377, "message": "yoyo!"}]}]

I sent both several times and we’re not seeing them show up today, either by tailing the logs in realtime or by searching for the message content.

I did run my CURL test again and it did result in a new NrIntegrationError so it would seem our Python calls are “working” (disappearing?) and our CURL calls are just bad JSON.

Here’s the output from one of our requests:

$ python ./newrelic_log_test.py  
[{"common": {"attributes": {"logtype": "accesslogs", "hostname": "desktop", "service": "processing-service"}}, "logs": [{"timestamp": 1611697614, "message": "yoyo!"}]}]
<PreparedRequest [POST]>
"[{\"common\": {\"attributes\": {\"logtype\": \"accesslogs\", \"hostname\": \"desktop\", \"service\": \"processing-service\"}}, \"logs\": [{\"timestamp\": 1611697614, \"message\": \"yoyo!\"}]}]"
202
{'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'Content-Length': '52', 'CF-Cache-Status': 'DYNAMIC', 'Vary': 'Accept-Encoding', 'cf-request-id': '07e2430577000005138bbce000000001', 'Server': 'cloud
flare', 'Connection': 'keep-alive', 'Access-Control-Allow-Credentials': 'true', 'Date': 'Tue, 26 Jan 2021 21:46:54 GMT', 'CF-Ray': '617d6de8bb2f0513-LAX', 'Access-Control-Allow-Methods': 'GET, POST, PUT, HEAD, OPTIONS', 'Content-Type': 'application/json;
charset=UTF-8', 'Access-Control-Allow-Origin': '*'}
{"requestId":"ae1fb610-001f-b000-0000-017740a9edfb"}

It is showing, in order:

• The CLI command
• The JSON string being given to the request
• The request object
• The request body from the request object
• The response code
• The response headers
• The response body

Are you able to query by that requestId and see where it may have gone?

Here is our test script, you can see the other structure we tried in there as well.

$ cat newrelic_log_test.py 
import requests
import time
import json

url = 'https://log-api.newrelic.com/log/v1'
headers = {'X-License-Key':'xxYYzz', 'Content-Type':'application/json'}

# data = {'message':'hihi','logtype':'accesslogs','service':'processing-service','hostname':'desktop'}
# data['timestamp'] = int(time.time())
# print json.dumps(data)
# r = requests.post(url, headers=headers, json=json.dumps(data))

message = {'message':'yoyo!'}
message['timestamp'] = int(time.time())
data = {
  "common": {
    "attributes": {
      "logtype": "accesslogs",
      "service": "processing-service",
      "hostname": "desktop"
    }
  },
}
data['logs'] = [message]
print json.dumps([data])
r = requests.post(url, headers=headers, json=json.dumps([data]))

print r.request
print r.request.body
print r.status_code
print r.headers
print r.content

Thanks!

@RyanVeitch - Unless you think there’s something else that can be done, I think we’re about to give up on this and try something else. Would be a shame - I had used NewRelic extensively on past projects like Bingo Blitz and recommended NewRelic for this project.

I have the same issue and it is frustrating.

Hi, @eburnside: You might check this post and see if it helps:

Thanks @philweber

In our python code (provided above) we do set the current timestamp.

I went back and tried making entries without setting it at all. Still not coming through.

Cheers