Credentials and Access
SSH Access
Please use your
GEANT AD account(GEANT\dante.xxx or GEANT\Firsname.Lastname)to havessh access and sudoer rights on the required servers.
Other Access
- Required credentials are in the "GÉANT Dashboard v3" LastPass folder
Problems
New alarms are not appearing in the gui
Possible Cause: Traps not being processed by RabbitMQ
Analysis
- Open one or more of the following RabbitMQ management consoles. (Credentials are in the "GÉANT Dashboard v3" LastPass folder)
- Scroll down to the "Nodes" section
- There should be 3 rows in the table and all status icons should be green. The expected node hostnames are:
- prod-noc-alarms01.geant.org
- prod-noc-alarms02.geant.org
- prod-dashboard-storage01.geant.org
Solution
- If one of the 3 nodes is failing or missing from the list, log into the failing server via ssh and restart the RabbitMQ service:
sudo systemctl restart rabbitmq-server
- After a minute or two the management consoles should show the cluster is restored.
Collectors have stopped working
Analysis
- Open https://net-alarms-monitoring.geant.org/d/hESYQotZz/correlation-services?orgId=1
- Scroll down to the "Collectors" panel
- Check that the graph shows a nonzero rate of traps being processes
Solution
- On each of the following servers:
- net-alarms01.geant.org
- net-alarms02.geant.org
- Log in via ssh and execute the following command:
sudo systemctl restart trap_collector
Possible Cause: Correlators have stopped working
Analysis
- Open https://net-alarms-monitoring.geant.org/d/hESYQotZz/correlation-services?orgId=1
- Scroll down to the "Collectors" panel
- Check that the graph shows one of the collector processes processing a non-zero rate of traps.
- note that it is normal for only one of the collectors to be processing traps, the other line should remain at zero.
Solution
- On each of the following servers:
- Log in via ssh and execute the following command:
sudo systemctl restart trap_correlator
Related articles