On Wed 7 Jan, 1930, Darrell Fuhriman darrell@grumblesmurf.net wrote:

mds@gbnet.net (mark) writes:

...

What I'm thinking of is that AFAICR tcp_xmit_hiwat and tcp_recv_hiwat are set to 8192 (bytes) and the equivalent *lowat's to 2048. This is

We're using this on our 2x400 MHz E450's and we see ~60MB/s using switched gigabit.

That's pretty respectable! Could you go through how some of these settings help - I'm not 100% clear on some of these meself:

These three pin the duplicity of the line, fair enough.

ndd -set /dev/ge adv_1000fdx_cap 1 ndd -set /dev/ge adv_1000hdx_cap 0 ndd -set /dev/ge adv_1000autoneg_cap 0

These two do what?

ndd -set /dev/ge adv_pauseTX 0 ndd -set /dev/ge adv_pauseRX 0

Much bigger buffers than I got for 100Mb/s networks, makes sense. Have you found these value by experiment or otherwise?

ndd -set /dev/tcp tcp_recv_hiwat 262144 ndd -set /dev/tcp tcp_xmit_hiwat 262144

How come this is set so low?

ndd -set /dev/tcp tcp_xmit_lowat 2048

How do each of these three things affect things?

ndd -set /dev/tcp tcp_wscale_always 1 ndd -set /dev/tcp tcp_tstamp_always 1 ndd -set /dev/tcp tcp_rexmit_interval_max 60000

Note these are dedicated backup servers, so the very high tcp_X_hiwait may not be appropriate if you're not doing lots of bulk data transfer.

The tcp_tstamp_always is a big win.

Tantalising - I'd like to know more.

Darrell

-- End of excerpt from Darrell Fuhriman