On Wed 21 Apr, 1999, beepy@netapp.com (Brian Pawlowski) wrote:
I truly apologize for this.
This is a weird one. This is not a "feature" of our box, it was something I put in that exploits capabilities of the PROM (OpenFirmware) to allow floppy boot like processing at boot time.
You see, in development engineering, we used to have a special build kernels that allow booting off hard disk, but still going through the "floppy boot menu", and profiled kernels for performance analysis...
I, being extremely bored watching linking kernels, made it dynamically configurable... I was playing many games quickly when I hung out alone in a small perf lab.
It was not meant for field use... But obviously you guys are using it...
Strikes me that if you're finding it useful then it's not a million-to-one shot that it'll be useful to customers too, albeit in a slightly different way.
Let me check, and see if there is not yet an open RFE on this. I will submit one, asking for enable/disable of PROM environment variables from running system...
Ouch.
In one sense, I regret adding these features - since it is truly a break from the design of a simple box (the original design was that a floppy was required to access maintenance menu - I did an incomplete end run - but the remote console operation precludes floppy loading, so what is the real requirement here?)
Make it possible, to competent (read: curious, brave and support-contract- holding) customers to do floppy-boot tasks with remote consoles. Document it in a place that you can only get to if you hold a support contract.
I have to say that I've been using NetApp's for 3 years now, and have learnt a few tricks and tips but I'm not going to get to go on the training course as a direct result of that experience (it's not deemed necessary for my job).
I'm learning things here, and from support calls that apparently get taught to customers who also buy the courses - but which are built into these boxes from the get-go. I think they'd be useful to me, I want that knowledge, and all that's needed is for a bit of documentation to be placed somewhere on NOW.
Courses shouldn't teach 'secrets' IMHO, they should be about bringing people up to speed from zero-knowledge, or extending their knowledge into advanced domains in limited-time frames. Getting people away from their workplace and giving them the space and framework to learn.
The knowledge should also be provided as part of the basic package for others to pick up as and when, if they've already surmounted the learning curve to a large degree. Ie in full manuals and other documents - perhaps limited to people with support contracts to keep customers honest and to keep the feedback loop closed.
I have to say I'm a little peeved when courses are products in and of themselves (ie where they teach things over and above the manuals) because it's just another way for vendors to screw customers for a few more bucks IMHO.
Previously I never thought NetApp was that kind of vendor. The Appliance philosphy seems to preclude this kind of thing...
Someone could restore my faith by pointing me at some hidden docs on NOW that explain it all, which I've just never noticed before! 8)
No promises on a solution here...
Natch, but even the vague hint of one has some of us slavering in anticipation!
Again, my apologies...
For what? Nothing to apologise for IMHO.
-- End of excerpt from Brian Pawlowski
+-- mds@gbnet.net (mark) once said: | Make it possible, to competent (read: curious, brave and support-contract- | holding) customers to do floppy-boot tasks with remote consoles. Document | it in a place that you can only get to if you hold a support contract.
Everyone has been harping on how this is useful if you only have a console connection. Mainly, I'm just happy about it because I don't have to hunt down boot floppies and go run in the other room to switch them out. :)
Oz
Ozzie Sabina wrote:
+-- mds@gbnet.net (mark) once said: | Make it possible, to competent (read: curious, brave and support-contract- | holding) customers to do floppy-boot tasks with remote consoles. Document | it in a place that you can only get to if you hold a support contract.
Everyone has been harping on how this is useful if you only have a console connection. Mainly, I'm just happy about it because I don't have to hunt down boot floppies and go run in the other room to switch them out. :)
While this is nice, it comes with a certain amount of danger. I think it is dangerous to make a eeprom change without testing it immediately. Otherwise the next time you test to make sure you did it right will be months later when the machine reboots for some external reason (such as a power outage). You won't be around to rememeber "Oh yes, I changed the eeprom!". Or in my case (my memory is bad) I would be standing there *and* not remembering the change that I made.
--tal
P.S. Here's a RFE we thought of today when we found a F330 had halted. The "ok" prompt should be changed to "Hey, I'm not ok!" or "boot level:" :)
On Fri, 23 Apr 1999, Tom Limoncelli wrote:
P.S. Here's a RFE we thought of today when we found a F330 had halted. The "ok" prompt should be changed to "Hey, I'm not ok!" or "boot level:" :)
The "ok" prompt has historical reasons. It is the prompt of a FORTH compiler/interpreter. I don't know if you're aware that you can write yourself some very nice diagnostic routines you can store in NVRAM and run tem from the "ok" prompt. Changing the prompt to something else would be confusing, as it would readily indicate that it is a FORTH machine.
Tom
tkaczma@gryf.net wrote:
On Fri, 23 Apr 1999, Tom Limoncelli wrote:
P.S. Here's a RFE we thought of today when we found a F330 had halted. The "ok" prompt should be changed to "Hey, I'm not ok!" or "boot level:" :)
The "ok" prompt has historical reasons. It is the prompt of a FORTH compiler/interpreter. I don't know if you're aware that you can write yourself some very nice diagnostic routines you can store in NVRAM and run tem from the "ok" prompt. Changing the prompt to something else would be confusing, as it would readily indicate that it is a FORTH machine.
I'm aware of FORTH history (I used FORTH when it was still new, hip, and the future of programming languages! :) ). We were just joking the other day that an operator might be confused when they see a console that says "ok", which really tells them that the machine is not ok. Not our operators, of course...
--tal
Courses shouldn't teach 'secrets'
...
The knowledge should also be provided as part of the basic package for others to pick up as and when, if they've already surmounted the learning curve to a large degree. Ie in full manuals and other documents
I agree 100%.
As we grow, we learn more about the practice of "appliance philosophy." Secret commands is an area where we got confused.
Our initial idea was that, like many simple electronic items, NetApp appliances should have "no user serviceable parts inside." Why burden the customer with lots of information about situations that should "never" happen. For many of our customers, especially for customers with just a couple of boxes, I believe that this approach has actually worked well.
But it doesn't always work, and especially not for for "enterprise customers". (One might define an enterprise customer as someone who has so many boxes that it is no longer a question of whether they will fail, but when.)
Big computer users understand that all products fail *occasionally*. But when a product does fail, they want well documented procedures and commands to get them on their way as quickly as possible. And they want to understand what is happening in their environment to their equipment. When a technical support engineer asks them to run a command, they want to be able to check out the documentation to learn exactly what it does.
I believe that these desires are all sensible and reasonable, even for an appliance. But I confess (I might as well confess, since you already know) that this is an area where we have more work to do. We have "secret" commands that are not documented, and -- worse yet -- since we designed them with the mistaken notion that we would be the only ones to use them, the user interface is not always as clean as it should be.
These are things we need to fix.
(There will always be some secret commands that are used within engineering as part of our design and test scaffolding. We just need to take the position that when we learn a given command is useful externally, we should give it a face-lift and make it non-secret.)
I have to say I'm a little peeved when courses are products in and of themselves (ie where they teach things over and above the manuals) because it's just another way for vendors to screw customers for a few more bucks IMHO.
Previously I never thought NetApp was that kind of vendor. The Appliance philosophy seems to preclude this kind of thing...
That is not our intention, even if it looks that way! NetApp's mission is to make money by developing hardware and software that people want to buy.
I don't mean to be anti-social, but my fantasy customer is one who buys stuff that never breaks, and if it does break, he/she can easily read the documentation (or visit NOW) to figure out how to fix the problem without calling us. We are growing quickly, and in the long run, if we don't strive to help our customers help themselves, we will have big problems. I want to grow by selling more products -- not by building more classrooms!
Don't get me wrong: training classes will always be a very important part of "helping customers help themselves", and technical support engineers will always be necessary to solve those extra tricky problems. And of course, we can't give away these services for free. But our real focus, in terms of making money, is on selling more hardware and software.
So why all the secret stuff in the classes?
The answer is simple. It's much easier to teach secret commands in classes than it is to write production quality documentation. And it's even worse when some of the interfaces really ought to be redesigned before being documented. The "22/7" trick for floppy boot seemed really clever when the goal was to keep that stuff secret. It looks pretty silly as something to document.
So while we're trying to get our arms around this problem in engineering, and figure out the best way to solve it going forward, the folks teaching classes are doing the right thing by sharing what they know, even if it does make you feel left out for not having taken a class.
Sorry.
Dave
Dave Hitz writes:
I don't mean to be anti-social, but my fantasy customer is one who buys stuff that never breaks, and if it does break, he/she can easily read the documentation (or visit NOW) to figure out how to fix the problem without calling us.
My fantasy is to *be* that customer.
[I came really close for a couple of years. NetApp had a Canadian office for quite a while before they realized that I (and my F330) even existed. Hugh Morris wanted to give me an award for the least-effort customer :-) ]
-bmw
On Thu, 22 Apr 1999, Dave Hitz wrote:
These are things we need to fix.
(There will always be some secret commands that are used within engineering as part of our design and test scaffolding. We just need to take the position that when we learn a given command is useful externally, we should give it a face-lift and make it non-secret.)
I disagree, I think all commands (that perhaps don't give competitors an upper hand, although they're probably reverse engineering this stuff anyway) should be documented and released to the customers under a service contract or not. The world would be soo much better and technology would progress so much quicker if everyone documented and released their interfaces to the public.
That is not our intention, even if it looks that way!
I think NTAP 202 looks exactly this way.
The answer is simple. It's much easier to teach secret commands in classes than it is to write production quality documentation.
Easier, no. More profitable, certainly.
So while we're trying to get our arms around this problem in engineering, and figure out the best way to solve it going forward, the folks teaching classes are doing the right thing by sharing what they know, even if it does make you feel left out for not having taken a class.
With all due respect, the hidden commands were the only thing that I learned from NTAP 101 and NTAP 202 class material. The equipment we worked on was decrepit. We did not get to play with cluster failover, Fibre Channel, or trunking. Three things that are really important to me and I'm sure many of the people that were in class with me.
Tom
(There will always be some secret commands that are used within engineering as part of our design and test scaffolding.
At one point, I made most of the commands that were weird engineering testing stuff "development-only" commands, which aren't available in production releases; I suspect few people have ever needed to do "dereference_null" in the field....
We just need to take the position that when we learn a given command is useful externally, we should give it a face-lift and make it non-secret.)
Yes. That's what happened with "ifstat", for example; there are probably plenty of others (e.g., I think there's a command to get the number given by the temperature sensor on some of the boxes).
Long term, the set of "rc_toggle_basic" commands should be very small; we may want to do as some switch or router I dealt with at one point did, and have a set of "basic" and "expert" commands, and provide a *documented* command to enter "expert" mode, and document all the "expert" commands as well.
"rc_toggle_basic" stuff should, perhaps, only be stuff that *only* NetApp personnel should use, and it may be that in large sophisticated sites (no, Dave, I'm *NOT* going to use the damn E-word; it continues to grate on my ears, and will probably continue to do so indefinitely....), there *aren't* any such commands.
I am sure there is a Super Secret Ninja Mode for NetApp personnel :)
I'm looking in a 202 course book right now, and the 202 manual only lists one line descriptions of the commands... How hard would it be to add "man page" like documentation to the online documentation that comes with the filer?
List Issue: It would be great if the subject lines actually had something to do with the message....
Guy Harris wrote:
"rc_toggle_basic" stuff should, perhaps, only be stuff that *only* NetApp personnel should use, and it may be that in large sophisticated sites (no, Dave, I'm *NOT* going to use the damn E-word; it continues to grate on my ears, and will probably continue to do so indefinitely....), there *aren't* any such commands.
Whoops!! I just found the more elaborate documentation in the back of the book..
I, like a moron wrote:
I am sure there is a Super Secret Ninja Mode for NetApp personnel :)
I'm looking in a 202 course book right now, and the 202 manual only lists one line descriptions of the commands... How hard would it be to add "man page" like documentation to the online documentation that comes with the filer?
List Issue: It would be great if the subject lines actually had something to do with the message....
Hi,
Did you know that NetAppCache code has info in the binary mentioning the default admin password in clear text. This is definately a prob for security ??? This is part of the pdf docs bundled into the binary distrib. Obviously default passwords are changed to more secure passwords but it gives a intruder the first guess at trying to cause problems.
I understand the need to distribute pdf docs as there are useful for administration but password info in clear text could be overcome by encrypting the pdf's stored online(and in the binary distrib) and doing decryption of pdf info if required.
comments ??
Colin Johnston SA PSINET UK
On Fri, 23 Apr 1999, Colin Johnston wrote:
I understand the need to distribute pdf docs as there are useful for administration but password info in clear text could be overcome by encrypting the pdf's stored online(and in the binary distrib) and doing decryption of pdf info if required.
comments ??
I think you should get in the habit of changing default passwords, especially since they are documented. I think that one should provide as much open information about their product as possible. That includes default passwords. I don't want to have to remember one password just to find out another. I guess no password should be the default just to make sure that the administrator sets some password before he lauches the thing on a network.
Tom
-
Bruce Walker -
Colin Johnston -
guy＠netapp.com -
hitz＠netapp.com -
Marc A Levy -
mds＠gbnet.net -
Ozzie Sabina -
Raymond Brennan -
tkaczma＠gryf.net -
Tom Limoncelli