If SSH Connection takes long time to connect then

If you are using CentOS 7 or later then if your SSH Connection is taking long time then, just restart following service. It resolves the issue.

# systemctl restart systemd-logind