Gig Ethernet and Above
#1
Original Poster
FlyerTalk Evangelist



Join Date: Nov 2002
Location: ORD
Posts: 14,773
Gig Ethernet and Above
I have moved this discussion from http://www.flyertalk.com/forum/trave...ustrating.html to here, since it veered onto the a networking topic from the OP's NAS questions.
Last edited by gfunkdave; Mar 11, 2016 at 7:30 am
#2
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784
18 3TB and 6 2TB at present, and you miss redundancy -- it's 3 RAID 6s, and drive manufacturers and OSes measure TB differently, so:
/dev/md123 is the 6x 2TB RAID that's being retired (errr, will probably move into a gaming desktop ... no sense in wasting good drives, even small dated ones.)
As for the relative content of pr0n vs. other videos vs. non-video content, I'll simply say that there is plenty
A modern very-low-end CPU like the J1900 can get full bandwidth out of Gb ethernet as long as you've got reasonable disk for it to come off of.
"Ordinary" 100Mbps Ethernet? What decade are we in now?
(I'll have 10GbE up and running at home as soon as my addition is done.)
There are 3 of us (and 4 when my son is a little older) using the server as a mega-DVR at home, not to mention a few remote friends, and it's running among other things my own main cloud storage (albeit totally manual via SCP/SFTP)
IF I let it sleep, I'd need another server (although it could be something tiny like an old router) to be able to do the wake-on-LAN remotely, and I'd have to explain wake-on-lan to my wife and the babysitter (any day now, to the daughter as well.)
Also, starting and stopping does a lot more wear on time than continuous running without spinning down. The oldest drive in my server has 45,382 hours (> 5 years) of run time, and under 200 start-stop cycles.
Code:
/dev/md125 197G 57G 131G 31% / /dev/mapper/vg_claudius-lv_home 22T 20T 1.4T 94% /home /dev/mapper/vg_claudius-lv_video 17T 16T 171G 99% /video /dev/md126 526G 207G 317G 40% /data
Code:
Disk /dev/md123: 7.3 TiB, 8001054310400 bytes, 15627059200 sectors
If all you needed was ordinary 100 mbs ethernet, pretty much anything can saturate that channel.
(I'll have 10GbE up and running at home as soon as my addition is done.)
Re power: Since I'm the only one using the server it spends a lot of time just waiting for me. I have it go to sleep after 20 minutes and it responds to wake-on-lan nicely, so that's how I roll with it. Depending on the rates in your area, and how much you actually use it, you could easily save yourself a hundred bucks a year in electricity costs and quite a lot of wear and tear on the drives.
IF I let it sleep, I'd need another server (although it could be something tiny like an old router) to be able to do the wake-on-LAN remotely, and I'd have to explain wake-on-lan to my wife and the babysitter (any day now, to the daughter as well.)
Also, starting and stopping does a lot more wear on time than continuous running without spinning down. The oldest drive in my server has 45,382 hours (> 5 years) of run time, and under 200 start-stop cycles.
#3
FlyerTalk Evangelist

Join Date: Nov 2002
Location: Freeload Univ. Where are you sitting?
Posts: 14,818
GigE and Above
The point is, there are a lot more things going on than just a fast ethernet connection and powerful cpu.
Then again, there are a lot of folks out there with routers having 4x10/100 ports and a lot of "fast" ethernet switches. You can get by nicely with Cat5E cable, but it's got to be Gb all the way. If you're not in a situation where you require the absolute latest and highest speed (not all of us live in Silicon Valley
), 10/100 works fine and it's faster and more reliable than wireless G.Out of curiosity: what sort of pipes do you need for 10Gb? Does Cat6 even support that?
#4
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784
I just finished an install/upgrade with a brand new Lenovo desktop re-imaging from a slightly older Dell. It was computer-to-computer and I watched it run. It seemed to be taking a while, so I brought up the Task Manager and watched. For much of the time. the rate was in the 3-6 megabytes/sec range with occasional spurts up to 15-20. Then for a while it was 70-80 and I even saw one burst of 100+.

The point is, there are a lot more things going on than just a fast ethernet connection and powerful cpu.
Yehah - I've been surprised at finding relatively new machines with 10/100 connections. By this time, I would assume they're all Gb.
Then again, there are a lot of folks out there with routers having 4x10/100 ports and a lot of "fast" ethernet switches.
10/100 works fine and it's faster and more reliable than wireless G.
OTOH, the cost difference on new equipment of gigabit vs. faster ethernet if you can get it at all is trivial.
(I wish that were true for 10GbE, but it's not nearly as expensive as it used to be; the cost of the 2 switches with 2x 10GbE uplinks a piece plus the transceivers was about $350 more than two good-quality switches without the 10GbE uplinks. Which, if you look at that as $87.50 per 10GbE port is really cheap compared to the $400/port we were paying at work a couple of years ago, but still horrifically expensive compared to the $8-10/port for a good basic managed GigE port, let alone the $3-5 a port for unmanaged.)
[* 2x 45Mbps connections; bonded. Per the modem, I'm so close to the DSLAM I could be doing 120Mbps per connection, or 240Mbps total. The cr*p AT&T modem can barely keep up with 90Mbsps now...]
Out of curiosity: what sort of pipes do you need for 10Gb? Does Cat6 even support that?
10GbE can run on three different types of connection:
* TwinAx (which uses a direct connection to the SFP+ port otherwise used for Fiber)
* Fiber (several kinds; you match the SFP+ transceiver module to the type of fiber and the distance - the standard distances range from about 26M to about 40km, or even longer with manufacturer-specific extensions.)
* Twisted pair (10GBase-T).
I've only used 10GBase-T on very short runs; plain old Cat 6 works fine at the 25' to 10M range. I think Cat 6A (Cat 6 w/ shielding on the individual pairs, I think) is recommended to get the full 55M range that 1000Base-T is supposed to be good to.
On short enough runs, if the cables are in good condition, you can run all the way down to plain old Cat5 for 10GBase-T -- although I've only tried it by accident by using the wrong ~6' patch cable.
#5
FlyerTalk Evangelist
Join Date: Mar 2004
Location: Newport Beach, California, USA
Posts: 36,062
I was looking at 10GB NICs today. They're inexpensive enough, but I couldn't find any 10GB switches at a reasonable price (I assume there's no reason to have a 10GB router).
#6
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784

How many 10Gb ports do you need? I know of two fairly reasonable switches with a 10GbE uplink (2 ports, plus a bunch of 1 gig) -- I'm using the cheaper of the two, the DGS-1510. There are also plentiful similar used ones with 2 or 4 uplink ports.
Those do well if you need to link one server (with a 10GbE port) to a bunch of workstations or set top boxes with 1GbE.
There are also some reasonably inexpensive used 10GbE-only switches, and a non-exorbitant but still expensive (but 10Gbase-T-only, no fiber) couple of 10GbE models from Netgear (XS708E)
There's a very inexpensive router/switch with 2 10GbE uplinks that some coworkers at my last company liked from Miktrotik. Compared to the D-Link, it sounded very non-user-friendly unless you really need the routing features.
If you just have one client needing 10GbE and a server, you can also just direct wire them with two nics, no switch needed.
#7
FlyerTalk Evangelist
Join Date: Mar 2004
Location: Newport Beach, California, USA
Posts: 36,062
How many 10Gb ports do you need? I know of two fairly reasonable switches with a 10GbE uplink (2 ports, plus a bunch of 1 gig) -- I'm using the cheaper of the two, the DGS-1510. There are also plentiful similar used ones with 2 or 4 uplink ports.
Those do well if you need to link one server (with a 10GbE port) to a bunch of workstations or set top boxes with 1GbE.
There are also some reasonably inexpensive used 10GbE-only switches, and a non-exorbitant but still expensive (but 10Gbase-T-only, no fiber) couple of 10GbE models from Netgear (XS708E)
There's a very inexpensive router/switch with 2 10GbE uplinks that some coworkers at my last company liked from Miktrotik. Compared to the D-Link, it sounded very non-user-friendly unless you really need the routing features.
If you just have one client needing 10GbE and a server, you can also just direct wire them with two nics, no switch needed.[/QUOTE]
#8
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784
Not sure if they'll be inexpensive enough for you (and the 10GbE uplink modules are separate), but the Dell 6224 or 6248 might be a good choice if you can find one and a pair of 10GbE uplinks (which are 2 ports each) in budget.
#9
Join Date: Feb 2015
Posts: 67
#10
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784
We used dozens of those (the 6248s, their blade-installed M6220, and the related M8024 10GbE-only blade switch) at my job before last. Inexpensive new, and quite reliable and fast. They worked quite well, and while they're quite dated at this point, for home or test-lab use, if they are available at a reasonable price they're a great choice.
I can't speak to any other Dell-branded switches beyond those models -- and prior to buying Force10, they resold various OEM switches under their own branding.
I don't know Cisco nearly as well, but I think there are some models in the 3750/3850 series with uplinks; it's just somewhat hard to see finding those as cheaply as the Dells. The uplink-less ones sell very cheaply.
There are much better options with a real budget, but given that he needs more than the 2 ports on the $250-$350 D-Link or Miktrotik option, going up to 4 ports at that budget is going to be seriously used stuff.
I can't speak to any other Dell-branded switches beyond those models -- and prior to buying Force10, they resold various OEM switches under their own branding.
I don't know Cisco nearly as well, but I think there are some models in the 3750/3850 series with uplinks; it's just somewhat hard to see finding those as cheaply as the Dells. The uplink-less ones sell very cheaply.
There are much better options with a real budget, but given that he needs more than the 2 ports on the $250-$350 D-Link or Miktrotik option, going up to 4 ports at that budget is going to be seriously used stuff.
#11
FlyerTalk Evangelist
Join Date: Mar 2004
Location: Newport Beach, California, USA
Posts: 36,062
). Now I'm confused about cabling. I see there's a whole variety of different connectors, apparently depending at least partly on the physical cable (fiber optic, copper, etc.) selected. I was hoping for something that would simply be backwards compatible and use RJ45 and Cat6 (or Cat6a) but, apparently, it's going to be more complicated than that.10GB is appealing to me because I often move lots of data around my system and it would be nice to accomplish in seconds what currently takes minutes. I may have to wait, though, until the various standards shake out and a "preferred" one emerges. Then again, my first LAN was 10 Mbs, so I shouldn't complain about my 1 Gbs setup.
#12
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784
I saw a couple of surprisingly inexpensive 10GB switches on ebay last night (though I'm too broke to get one this month
). Now I'm confused about cabling. I see there's a whole variety of different connectors, apparently depending at least partly on the physical cable (fiber optic, copper, etc.) selected. I was hoping for something that would simply be backwards compatible and use RJ45 and Cat6 (or Cat6a) but, apparently, it's going to be more complicated than that.
10GB is appealing to me because I often move lots of data around my system and it would be nice to accomplish in seconds what currently takes minutes. I may have to wait, though, until the various standards shake out and a "preferred" one emerges. Then again, my first LAN was 10 Mbs, so I shouldn't complain about my 1 Gbs setup.
). Now I'm confused about cabling. I see there's a whole variety of different connectors, apparently depending at least partly on the physical cable (fiber optic, copper, etc.) selected. I was hoping for something that would simply be backwards compatible and use RJ45 and Cat6 (or Cat6a) but, apparently, it's going to be more complicated than that.10GB is appealing to me because I often move lots of data around my system and it would be nice to accomplish in seconds what currently takes minutes. I may have to wait, though, until the various standards shake out and a "preferred" one emerges. Then again, my first LAN was 10 Mbs, so I shouldn't complain about my 1 Gbs setup.

(There are also two obsolete ones: CX4 aka "infiniband" cabling, which is only useful over very short runs, and XFP which is an obsolete standard for fiber transceivers.)
Fiber is more common on the used market, and until recently was much cheaper. This has changed; the cheap Netgear switches are copper rather than fiber. The other thing is that with fiber, you need transceiver modules or special cables, which may or may not have ended up saving money (a lot depends on whether the manufacturer allows "passive" cables or not, and whether they lock to approved -- and much more expensive -- transceivers. Name-brand Intel transceivers were $250/each last I checked; non-brand from China were $99 for a pack of 4.) One thing I liked about the Dell-branded switches is they worked with absolutely anything cabling and transceiver-wise.
If you were doing especially latency-sensitive storage or high performance computing application, fiber also has lower latency, but the difference is an irrelevance in any normal application and is mostly relevant when comparing to like Myrinet or Infiniband.
Copper/10GBase-T will definitely be easiest for you, and if you were buying new, wouldn't be too bad of a premium. I see a lot less of it used.
#13
FlyerTalk Evangelist
Join Date: Mar 2004
Location: Newport Beach, California, USA
Posts: 36,062
There are only two connectors for 10GbE in production and worth considering -- fiber (SFP+) and copper (RJ45). Both are backwards compatible with gigabit ethernet, but unless you have fiber (SFP) gigabit, the former being backwards compatible is unhelpful.
(There are also two obsolete ones: CX4 aka "infiniband" cabling, which is only useful over very short runs, and XFP which is an obsolete standard for fiber transceivers.)
Fiber is more common on the used market, and until recently was much cheaper. This has changed; the cheap Netgear switches are copper rather than fiber. The other thing is that with fiber, you need transceiver modules or special cables, which may or may not have ended up saving money (a lot depends on whether the manufacturer allows "passive" cables or not, and whether they lock to approved -- and much more expensive -- transceivers. Name-brand Intel transceivers were $250/each last I checked; non-brand from China were $99 for a pack of 4.) One thing I liked about the Dell-branded switches is they worked with absolutely anything cabling and transceiver-wise.
If you were doing especially latency-sensitive storage or high performance computing application, fiber also has lower latency, but the difference is an irrelevance in any normal application and is mostly relevant when comparing to like Myrinet or Infiniband.
Copper/10GBase-T will definitely be easiest for you, and if you were buying new, wouldn't be too bad of a premium. I see a lot less of it used.
(There are also two obsolete ones: CX4 aka "infiniband" cabling, which is only useful over very short runs, and XFP which is an obsolete standard for fiber transceivers.)
Fiber is more common on the used market, and until recently was much cheaper. This has changed; the cheap Netgear switches are copper rather than fiber. The other thing is that with fiber, you need transceiver modules or special cables, which may or may not have ended up saving money (a lot depends on whether the manufacturer allows "passive" cables or not, and whether they lock to approved -- and much more expensive -- transceivers. Name-brand Intel transceivers were $250/each last I checked; non-brand from China were $99 for a pack of 4.) One thing I liked about the Dell-branded switches is they worked with absolutely anything cabling and transceiver-wise.
If you were doing especially latency-sensitive storage or high performance computing application, fiber also has lower latency, but the difference is an irrelevance in any normal application and is mostly relevant when comparing to like Myrinet or Infiniband.
Copper/10GBase-T will definitely be easiest for you, and if you were buying new, wouldn't be too bad of a premium. I see a lot less of it used.
#14
FlyerTalk Evangelist




Join Date: Jul 2000
Location: in the vicinity of SFO
Programs: AA 2MM (LT-PLT, PPro for this year)
Posts: 19,784
#15
FlyerTalk Evangelist
Join Date: Mar 2004
Location: Newport Beach, California, USA
Posts: 36,062

