My quest to build a personal RAID server...
After stockpiling a terabyte of pictures, music, ebooks, journal articles, personal documents, movies, TV shows, and other good stuff over the past decade, I started getting very, very nervous..
The problem was that if I had one little hard drive failure, all that good stuff would be gone. All that time scavenging on the net. All those photos with my friends, family, and travels. All those e-books I'll never read. Et cetera. Not to mention all that good stuff was locked away on my desktop PC. I wanted a way to access all that good stuff remotely. I needed a backup / file server.
I would rather get stuff done than constantly have to copy files around, so I researched an automated solution. Here's a handy breakdown of what's currently available:
- Software - Depends a program running to periodically back up your important stuff. Copying that stuff around will tax the computer, and when the program runs there will be a noticeable decline in performance.
- Periodic Backup / Syncing - Copies your important data to a backup location periodically. This is an adequate solution for things of low importance, or things that don't get updated frequently.
- Software RAID - This is similar to a periodic backup or syncing program, but more technically advanced and mimics RAID systems. RAID systems have several advantages, especially when 3 or more hard drives are used.
- Hardware- Hardward backup systems are generally RAID controllers.
- Onboard RAID - Check your motherboard - it may have RAID capability. The first motherboard I was planning on using for my RAID server had this (see below)
- Dedicated RAID Controller - PCI or PCI Express controller card specifically bought for setting up RAID arrays.
I was going to add another hard drive to my desktop, and every few days or so run a backup application to backup the desired folders. But I didn't want to make my desktop a file server. Finally, this type of solution wouldn't do me any good if I forgot to run the backup program. It just didn't feel like the best solution.
The next thing I considered was running a hardware RAID configuration. a motherboard from my brother-in-law from an old system he built and upon closer inspection, discovered it had a RAID chip onboard! Score!
So why didn't I go with this solution? Well, to implement the use of the RAID chip, I would set up a RAID 1 mirror. Then I would have to install MS Windows on it, taking up precious space, and try to figure out how to FTP into it. Also, it just seems wasteful to have to install Windows on a computer that isn't meant to be used as a desktop. So it's close, but not a perfect solution to what I was looking for.
Design and Specifications
In short: I am going to show you a way to safely store all your data, and have access to it in multiple ways simultaneously. Granted, you could probably just buy a Home Media Server online. But, you pay $500, you get a HP computer with Windows Server pre-installed. No FTP out of the box, and only 750GB of storage on one hard drive. It's a start.
But just say you have some old computer parts laying around, collecting dust, or you know where to get some inexpensively. Well here's another idea:
First of all, let me be clear about what I am making:
- Hardware or RAID or RAID. That means 2+ disks continuously backing each other up
- Network access via a Samba share
- FTP access
I wanted the backups to be network-accessible, so that I can access any of the media from my laptop, desktop, or any computers on my network. I need the bandwidth to play a Blueray (~24 Mbps), or a couple DVDs (~6 Mbps) or audio files (~256Kbps for high quality) [ref] at once. Additionally, I'd like to be able to FTP in and access my data from anywhere in the world.
I found UnRAID through a fair amount of googling and reading message forums. It turns out that UnRAID is software RAID made by Lime Technology, and especially designed to run on top of a super-lightweight custom Slackware Linux distribution, syslinux. Don't worry, you don't even have to know Linux command line; they provide an HTML interface that you can access with your browser to set the program up. Lime Technology allows you to run a free version of their software for up to 3 hard drives, and have paid options available if you want more.
What really sold me on UnRAID is the way they do their backup. They use only 1 of your drives as a parity drive (it must be equally large or larger than any of the rest of the hard drives in the array). The rest of your drives are used to store data. This is a distinct advantage over RAID 1 mirroring. With RAID 1 mirroring, you can only use 2 drives (or multiples of 2 for more complicated setups). One drive contains data, the other mirrors. If you want to add another backup drive, you have to add a second mirroring drive as well. Basically, this means that with RAID 1 you get 1/2 the space of the physical drives. With UnRAID, you get the space of all the drives, minus the parity drive. This means for three 1 TB hard drives, you get 2 TB of backed-up space vs 1.5 for RAID 1.
Another thing I like is that with UnRAID, data is not striped across multiple drives. It . This means that you can just swap in drives without rebuilding the array. Essentially, you can shut down the UnRAID array, pull out a hard drive, and stick it in another computer supporting ReiserFS, and it will just work. Finally, unRAID runs off a USB stick, meaning that the drives you use in your computer are 100% used to store data. This made a difference to me because I love efficiency, and I love the "cool factor" of running an entire OS on a USB stick.
That reminds me, another great thing is that you can backup your entire UnRAID installation just by copying the contents of the USB stick onto another USB stick. Then, if the USB stick melts, just plug in the other one and you're back up and running. How could it be simpler?
What you'll need:
- Some cheap old computer. By computer I mean:
- Motherboard. I am currently using an MSI 7058G that I bought ca. 2004. Prior to that, I used a Biostar P4M80 or something. By the way, I was able to change motherboards in the built system and UnRAID booted up right away the first time with no issues and my array was back online! The motherboard basically just needs to have Ethernet, the ability to boot from USB, and some type of hard drive controller (generally SATA or IDE). Note that RAID is not required whatsoever. As a matter of fact, if you have onboard RAID, I'd say disable it!
- Processor: Cheap, small, low power is preferable. I'm using an Intel Pentium Prescott 2.8GHz Hyperthreading processor, not exactly low power, but it's all I had laying around.
- RAM - 512 MB minimum. I'm using 1 GB of 333MHz DDR.
- Some old computer case. ATX preferably. Technically you could get away with a case-less design, assuming you like your electronic components being free to the environment.
- Power supply.
- At least two hard drives. I'm using two 1 TB SATA drives and will be adding another as soon as I find a ridiculous deal on one.
- You'll need a monitor and keyboard for the initial setup, but you can run the system "headless" afterward.
- A USB stick, at least 128MB. I'm actually using a 1GB MMC card in a USB adapter!
- A copy of the UnRAID software. You can get it here: http://www.lime-technology.com/.
- Download UnRAID.
- Install it to the USB stick by following the directions here. If you want your backup server to have a static IP, manually edit the config file on the USB stick.
- Assemble the computer. Stick the USB drive into a motherboard USB slot on the back of the computer case.
- Plug in the Ethernet.
- Boot and during the POST, push DEL or F6 or whatever it is to access your motherboard's BIOS, setting it to boot from USB. Reboot.
- Go to the web browser another computer on your network and type "//tower" into the address bar. This is the UnRAID server's default name.
- All of the options are contained in a easy to use HTML web interface. Note that this webpage is not accessible from the "outside world" - only computers on your network.
- Build the array. UnRAID server will begin the parity check, essentially backing up the data drive(s) to the parity drives.
A couple gotchas:
- UnRAID uses ReiserFS as its file system. This only matters if you try to take the hard drive out of the server and put it into a Windows desktop without formatting it. This is an uncommon Linux file system, and is not supported at all by Windows. Currently, this means you have to download a (free) 3rd party driver, and that driver only makes the hard drive readable, not writable. This is not an issue if the hard drive is in the UnRAID system, because you are accessing it remotely over the network via a Samba share. Samba shares work great with Windows. The problem comes if you want to put the hard drive from the UnRAID computer in your desktop. The desktop Windows computer won't be able to read it unless you install the driver for ReiserFS. With a Linux desktop it may be less of an issue.
- Likewise, if you want to put a hard drive from your Windows desktop containing important data directly into the UnRAID server, you can't. You have to put the hard drive(s) in the UnRAID computer, format them in ReiserFS (erasing all data on the hard drives), then transfer the data to your UnRAID server over the network. This may mean that you have to shuffle data around until you get everthing on to the UnRAID server.
That's it. You now have all your data protected! You can FTP to your backup server remotely by opening and redirecting the appropriate ports in your router, and logging in with "root" and whatever password you chose!
Can't connect to the Samba share - I've noticed that in Windows Vista, the unRAID computer, called "TOWER" doesn't show up under Network Places. That's ok - the share is still available on the network. Just type "\\tower" into the Windows Explorer address bar (press Windows-E to bring up Windows Explorer, or just click on My Computer)
- Accessing files VIA FTP - Ok, so you opened port 21 on your router and forwarded it to your UnRAID server's local IP. You can log in via FTP, but no files are displayed. This is because you have to set "read" permission on the files you want to access. This means you have to telnet into the UnRAID server, using "root" and your password, and set the permissions via the command line. It's not too hard, but people unfamiliar with a command line may not be able to do this. For those who are, go into the /mnt directory and chmod +r FileName to allow read access via FTP. Use chmod -R +r DirName to enable read access for all the files in a directory.