Summary:
- Both NR Infra and APM (PHP agent) runs in the same host
- When NR infra is running it makes
/tmp/.newrelic.sockroot owned which causes an issue for the APM agent as the Agent is embedded w PHP-FPM
Steps to reproduce:
Check status of New Relic Infra agent
|
1 2 3 4 5 6 7 8 9 10 11 |
[root@web1 tmp]# systemctl status newrelic-infra ● newrelic-infra.service - New Relic Infrastructure Agent Loaded: loaded (/etc/systemd/system/newrelic-infra.service; enabled; vendor preset: disabled) Active: inactive (dead) since Tue 2019-09-03 19:21:48 AEST; 15h ago Process: 1681 ExecStart=/usr/bin/newrelic-infra (code=exited, status=0/SUCCESS) Main PID: 1681 (code=exited, status=0/SUCCESS) Sep 03 18:43:40 web1 systemd[1]: Started New Relic Infrastructure Agent. Sep 03 19:21:48 web1 systemd[1]: Stopping New Relic Infrastructure Agent... Sep 03 19:21:48 web1 newrelic-infra[1681]: 2019/09/03 19:21:48 Orderly shutdown commenced Sep 03 19:21:48 web1 systemd[1]: Stopped New Relic Infrastructure Agent. |
Start up New Relic Infra
|
1 |
[root@web1 tmp]# systemctl start newrelic-infra |
Tail php_agent log
|
1 2 3 4 5 6 7 8 9 |
[root@web1 newrelic]# tail -f php_agent.log 2019-09-04 09:57:05.213 +1000 (6200 6200) info: New Relic 9.0.2.245 ("heliotrope" - "440ec18f237b") [daemon='/tmp/.newrelic.sock' php='7.3.7' zts=no sapi='fpm-fcgi' pid=6200 ppid=1 uid=0 euid=0 gid=0 egid=0 backtrace=yes startup=agent os='Linux' rel='3.10.0-957.27.2.el7.x86_64' mach='x86_64' ver='#1 SMP Mon Jul 29 17' node='web1'] 2019-09-04 09:57:05.214 +1000 (6200 6200) info: spawned daemon child pid=6201 2019-09-04 10:51:36.745 +1000 (9593 9593) info: New Relic 9.0.2.245 ("heliotrope" - "440ec18f237b") [daemon='/tmp/.newrelic.sock' php='7.3.7' zts=no sapi='cli' pid=9593 ppid=9592 uid=0 euid=0 gid=0 egid=0 backtrace=yes startup=agent os='Linux' rel='3.10.0-957.27.2.el7.x86_64' mach='x86_64' ver='#1 SMP Mon Jul 29 17' node='web1'] 2019-09-04 10:51:36.746 +1000 (9593 9593) info: spawned daemon child pid=9594 2019-09-04 10:51:36.769 +1000 (9593 9593) warning: daemon connect(fd=4 uds=/tmp/.newrelic.sock) returned -1 errno=ENOENT. Failed to connect to the newrelic-daemon. Please make sure that there is a properly configured newrelic-daemon running. For additional assistance, please see: https://newrelic.com/docs/php/newrelic-daemon-startup-modes 2019-09-04 10:51:40.520 +1000 (9728 9728) info: New Relic 9.0.2.245 ("heliotrope" - "440ec18f237b") [daemon='/tmp/.newrelic.sock' php='7.3.7' zts=no sapi='cli' pid=9728 ppid=9727 uid=0 euid=0 gid=0 egid=0 backtrace=yes startup=agent os='Linux' rel='3.10.0-957.27.2.el7.x86_64' mach='x86_64' ver='#1 SMP Mon Jul 29 17' node='web1'] 2019-09-04 10:51:40.521 +1000 (9728 9728) info: spawned daemon child pid=9729 2019-09-04 10:51:40.554 +1000 (9728 9728) warning: daemon connect(fd=4 uds=/tmp/.newrelic.sock) returned -1 errno=ENOENT. Failed to connect to the newrelic-daemon. Please make sure that there is a properly configured newrelic-daemon running. For additional assistance, please see: https://newrelic.com/docs/php/newrelic-daemon-startup-modes |
Possible resolution
This can be fixed by configuring New Relic daemon to use an abstract socket @newrelic-daemon
|
1 2 3 |
[root@web1 newrelic]# grep newrelic.daemon.port /etc/php.d/newrelic.ini ; Setting: newrelic.daemon.port newrelic.daemon.port = "@newrelic-daemon" |
Kill “hung” newrelic-daemon processes with signal 9 (SIGKILL)
|
1 2 |
[root@web1 newrelic]# ps aux | grep -i [n]ewrelic-daemon | awk '{print $2}' | xargs -n1 kill -9 |
Restart NR Infra. If New Relic Daemon is started with Agent-mode, then you do not need to start up New Relic Daemon manually.
|
1 |
[root@web1 newrelic]# systemctl start newrelic-infra |
Tail logs again and make sure no socket issues present.
In my case, newrelic.daemon.port was set in www.conf of PHP FPM.
|
1 2 3 |
[root@web1 newrelic]# grep newrelic.daemon.port /etc/php-fpm.d/www.conf php_value[newrelic.daemon.port] = @newrelic-daemon |
This would have been fine if only NR APM agent was running in the host. If NR infra agent runs in the same host then the newrelic.daemon.port option should be configured in the New Relic newrelic.ini file.
Thanks Josh (New Relic) for pointing out newrelic.ini