Jump to content

My new file server..


Stu

Recommended Posts

Moving house is always a good opportunity to build something new, so i'm in the process of building up a new file server for home and thought i'd share it with anyone who may be interested. Feel free to ask any questions about the build, technical or whatever.

 

 

The purpose of the server will be primarily to store data, however I may consider using it for virtualisation (xVM) but I do have another box for that (running Win2k8+VMWare Server 2.0). The new box will be running OpenSolaris and making use of the neat-o ZFS file system (click here for more info). This will provide me with a few different storage pools for different groups of data, each with different sizes, redundancy and performance aspects.

 

 

So here's a list of the hardware:

 

 

http://www.madshrimps.be/vbulletin/attachments/f22/1602d1199805039-sharkoon-releases-rebel12-full-atx-tower-clipboard01.jpg Sharkoon Rebel 12 Case

 

http://www.apitcomp.ru/catalog/motherboards_intel/53098.jpg Intel S3210SHLX Server Board

 

http://www.hiteksystems.co.nz/images/Core2Duo%20Box.jpg Intel E7200 Dual Core CPU

 

http://static.scan.co.uk/Images/Products/1161216-a.jpg 4 x 2GB (8GB) Hynix ECC DDR2-800 Server RAM

 

http://static.scan.co.uk/Images/Products/684734-a.jpg Corsair 650W PSU

 

http://static.scan.co.uk/Images/Products/869581-a.jpg 8 x 1.5TB Seagate SATA HDD

 

http://www.samsung.com/us/system/consumer/product/2006/11/06/hd501lj/247720HDD_HD501LJ_medium.a.jpg 4 x Samsung Spinpoint 500GB SATA HDD

 

http://www.pacificgeek.com/productimages/xl/ST340014A-NDW-R.jpg 4 x 40GB Seagate Barracuda SATA HDD

 

http://static.scan.co.uk/Images/Products/1139027-a.jpg 2 x Intel 40GB X25-V Solid State Disk

 

http://www.acmemicro.com/estore/prodimg/AS-SM-SATA2MV8.gif 2 x SuperMicro 8 Port SATA 2 Card (PCI/PCI-X) AOC-SAT2-MV8

 

http://static.scan.co.uk/Images/Products/1115534-a.jpg Scythe Nesteq FanMax 8-Channel Black Fan Controller

 

http://static.scan.co.uk/Images/Products/701585-a.jpg 3 x Coolermaster Full Alloy 4 in 3 Devices Module

 

 

+ a bunch of random fans, cables, adapters etc. (not worth listing here)

 

I will be doing lots of custom, tidy wiring inside for power and fans to make it as neat as possible. This makes it better for airflow and easier to swap parts out if necessary. Some of the disks will be at the front of the case in the 4in3 carriers, and others will be in the slots at the side of the case (like the SSD's and boot disks).

 

Okay, so how will it be configured. As above it will be running the OpenSolaris operating system. If you're not familiar with Solaris, it is a unix-like o/s originally made by Sun Microsystems, who have recently been bought out by Oracle. Sun make/made a lot of enterprise quality of gear which most large companies make use of. If you'd like to learn more about OpenSolaris, click here.

 

The key part of this build is the ZFS file system. This is a super reliable file system which does many cool things to protect against data loss. If you've heard of 'RAID' then you may think it's super resilient (even RAID 6, or higher) however there is still possibility of data loss (see RAID 5 write hole). ZFS in theory removes these risks and also provides a simple manageable storage platform.

 

I will configure a few different ZFS storage pools, as follows:

 

Pool: rpool

Purpose: Operating System

Redundancy: 2-way Mirror

Disks: 4 x 40 GB (may end up 2 x 40, or perhaps a triple-way mirror with a hot spare)

Useable Size: 40 GB

 

Pool: cpool

Purpose: Critical Stuff (documents, things I never want to lose (again))

Redundancy: 2-way Mirror

Disks: 2 x 500 GB

Useable Size: 500 GB

 

Pool: vpool

Purpose: VMWare Storage (remote vmware host will use this storage for virtual machines over iSCSI)

Redundancy: 2-way Mirror

Disks: 2 x 500 GB

Useable Size: 500 GB

 

Pool: mpool

Purpose: Media Dump (Recorded TV, personal music/film/tv rips)

Redundancy: RaidZ2 (allows the loss of two disks before data loss)

Disks: 8 x 1.5 TB

Useable Size: 9 TB

 

 

So these are the storage pools. I can create mount points within each pool for different reasons, and apply things like quotas to them. Each mount point can also be presented by different methods, like CIFS (windows file sharing), iSCSI (block level SCSI over ethernet), NFS etc. I'll probably be making the most of all of these. I've not yet decided exactly how i'll split out each pool, but I won't bore you with the super low-level techie stuff here.

 

The two 40GB SSD's were chosen due to them being a) cheap and b) having great read performance (for the price). Write performance isn't great, but it isn't critical for their application.. which is to be used as ZFS l2arc (cache) and ZIL (ZFS intent log) devices. These two disks will be members of their own pool, with two different vdevs configured. One vdev will be a mirror (approx 4gb) for ZIL, the remaining space on each disk will be striped for l2arc caching purposes. Ideally these would be on seperate SSD's, but this is only for home and as they are it will provide a huge performance boost to the server.

 

And that's pretty much it really. I know it may seem a bit hardcore - i've done small servers, big servers, all kinds of servers over the years, but ultimately this should end up a nice package.

 

 

Here's some I made earlier (for home!):

 

 

http://di.cx/gallery/d/5278-2/batcave_005.jpg

 

http://di.cx/gallery/d/5308-2/IMAGE_052.jpg

 

http://di.cx/gallery/d/5401-2/stu_d50_160307.jpg

 

http://di.cx/gallery/d/17973-2/mini-me%20003.JPG

 

 

And if you'd like to see the kind of stuff I was working with ten years ago:

 

http://di.cx/stuff/phone/dc/Picture(14).jpg http://di.cx/stuff/phone/dc/Picture(15).jpg

 

 

Enjoy :)

Link to comment
Share on other sites

  • Replies 61
  • Created
  • Last Reply

Top Posters In This Topic

Have you had to recovery data from any solid state drives previously?

 

No, I have no experience of that. I haven't worked in an engineering type role for some time now so don't tend to get involved with such matters. These are the first SSD's I have purchased for home use, however I am not sure at this time if they will be suitable for the ZIL data due to their poor write performance but I will be doing some comparative testing between them and a normal HDD.

Link to comment
Share on other sites

Beautiful. Can i ask though why you built such a high spec machine thats being used as a server? As far as i can tell, its not going to have to handle many high demand processes?

 

Have you thought about chucking a few TV cards in and using it as a TV server over LAN as well? That way you can stream TV (and pause/record etc) to any other clients in the house.

 

Heres my effort from a few years back. Nothing special, but it did the job (until my RAID setup died on me through my utter stupidity of using a RAID0 array):

 

http://www.evo-host.co.uk/getimg/23412.jpg

 

  • Gigabyte mobo
  • 1Gb Corsair DDR667 RAM
  • AMD X2 Dual Core processor
  • 1x250Gb Seagate Sata HDD (for O/S)
  • 4x1Tb WD Sata HDD using RAID0 (media storage)
  • Hauppauge NovaT-500 TV card

 

 

 

...and 9 TERRABITES of films and TV programs?

 

youd be surprised how quickly media adds up, especially these days with the average bluray rip/shrink being in the region of 25Gb. i have 4Tb worth of films alone on my server.

 

i suspect stu is just future proofing.

Link to comment
Share on other sites

Can i ask though why you built such a high spec machine thats being used as a server?

It's a relatively low spec machine. Cheap CPU, and some of the bits I had laying around.

 

Have you thought about chucking a few TV cards in and using it as a TV server over LAN as well? That way you can stream TV (and pause/record etc) to any other clients in the house.

No, haven't considered this. As above it'll be running OpenSolaris and i'm not sure if such things like MythTV are available for Solaris. I have a seperate HTPC for that kind of stuff. This is just a flie server, I have other servers for other things.

 

until my RAID setup died on me through my utter stupidity of using a RAID0 array

I'm sure I said something about that at the time ;)

 

i suspect stu is just future proofing.

Kind of. It's more of a case that 1.5TB disks were the cheapest per gigabyte and I needed the number of spindles for performance reasons.

Link to comment
Share on other sites

RAID is not a replacement for backup :)

 

No. i dont bother with it any more as i couldnt afford a hardware RAID setup, and software is just too unreliable. I have physical duplicate backups of all my drives these days, and spend a day once every 6 months or so updating it.

 

Id be interested to read more about this ZFS file system youll be using. Does it use some kind of redundancy like RAID for keeping data safe?

Link to comment
Share on other sites

Id be interested to read more about this ZFS file system youll be using. Does it use some kind of redundancy like RAID for keeping data safe?

 

It has different levels of redundancy that can be loosely compared to typical RAID levels, it's worth checking the ZFS link above.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...