Hello All,
I have a 3020 running 7.2.4 that is failing to respond to SSH commands when they're sent one at a time. I can ssh in as root and get a shell via putty or ssh. But if I try to send an "uptime" command I get either "FATAL ERROR: Server unexpectedly closed network connection" (from plink) or "Connection to carbon-b closed by remote host." from ssh.
Also, when I'm logged in interactively, I see lots of this blowing across the console:
carbon-b> receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
We have lots of processes (monitoring, alerting, scripted processes) that touch this box via non-interactive SSH, is each of these errors some failed attempt to connect?
If I understand the inner workings correctly, pretty much all admin traffic on a filer channels into an internal RSH pipe. Are there different pipes for interactive and non-interactive SSH? I have tried stopping and restarting SSH (options ssh.enable off and on). Also RSH (options rsh.enable off and on). No love.
Anyone have any ideas?
Randy
Seattle
Sounds like a SSH server-side bug(s). I bet if the NetApp were rebooted, the problem would go away...
at least for a while. I Faintly recall some SSH bugs in earlier releases (like 7.x) that sounded exactly like what you are seeing.
Might be worthwhile to investigate updating the code to something more recent.
--tmac
*Tim McCarthy* *Principal Consultant*
Clustered ONTAP Clustered ONTAP NCDA ID: XK7R3GEKC1QQ2LVD RHCE5 805007643429572 NCSIE ID: C14QPHE21FR4YWD4 Expires: 08 November 2014 Expires w/release of RHEL7 Expires: 08 November 2014
On Wed, Feb 20, 2013 at 1:27 PM, Randy Rue rrue@fhcrc.org wrote:
Hello All,****
I have a 3020 running 7.2.4 that is failing to respond to SSH commands when they’re sent one at a time. I can ssh in as root and get a shell via putty or ssh. But if I try to send an “uptime” command I get either “FATAL ERROR: Server unexpectedly closed network connection” (from plink) or “Connection to carbon-b closed by remote host.” from ssh.****
Also, when I’m logged in interactively, I see lots of this blowing across the console:****
carbon-b> receive signals: rsh protocol failure in circuit setup. 300 received****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
receive signals: rsh protocol failure in circuit setup. 300 received****
receive signals: rsh protocol failure in circuit setup. 300 received****
receive signals: rsh protocol failure in circuit setup. 300 received****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
Try again.****
receive signals: rsh protocol failure in circuit setup. 300 received****
We have lots of processes (monitoring, alerting, scripted processes) that touch this box via non-interactive SSH, is each of these errors some failed attempt to connect?****
If I understand the inner workings correctly, pretty much all admin traffic on a filer channels into an internal RSH pipe. Are there different pipes for interactive and non-interactive SSH? I have tried stopping and restarting SSH (options ssh.enable off and on). Also RSH (options rsh.enable off and on). No love.****
Anyone have any ideas?****
Randy****
Seattle****
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Turns out there is a limit of 1 interactive SSH session (knew that) and 20 non-interactive sessions (did not know that). And there were hung sessions (monitoring, perfstats, etc) from several hosts. Rebooted two hosts and got the connection count down below 20 and all is well.
Thanks!
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Randy Rue Sent: Wednesday, February 20, 2013 10:28 AM To: toasters@teaparty.net Subject: can't shoot SSH commands to a filer
Hello All,
I have a 3020 running 7.2.4 that is failing to respond to SSH commands when they're sent one at a time. I can ssh in as root and get a shell via putty or ssh. But if I try to send an "uptime" command I get either "FATAL ERROR: Server unexpectedly closed network connection" (from plink) or "Connection to carbon-b closed by remote host." from ssh.
Also, when I'm logged in interactively, I see lots of this blowing across the console:
carbon-b> receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
Try again.
Try again.
Try again.
Try again.
Try again.
Try again.
receive signals: rsh protocol failure in circuit setup. 300 received
We have lots of processes (monitoring, alerting, scripted processes) that touch this box via non-interactive SSH, is each of these errors some failed attempt to connect?
If I understand the inner workings correctly, pretty much all admin traffic on a filer channels into an internal RSH pipe. Are there different pipes for interactive and non-interactive SSH? I have tried stopping and restarting SSH (options ssh.enable off and on). Also RSH (options rsh.enable off and on). No love.
Anyone have any ideas?
Randy
Seattle
-
Randy Rue -
tmac