Since I can’t reveal a lot of the projects I’m working on due to nondisclosure agreements, security, and privacy concerns, I thought I’d post some information about my homelab.
A “homelab” is a testing environment where I can simulate the infrastructure of different clients in order to create new systems or improve upon old ones — without risking the client’s information in a *production* environment. In other words, it’s a safe place to break things where the breaking of the things doesn’t disrupt business operations.
Instead of describing it, it will probably be helpful to see some photos.
Everything here was selected carefully for the available features and performance. I opted for CAT6 cable instead of CAT5e so that it would be relatively “future proof” without going overboard with fiber. I colour coded the patch cables to simplify visual inspection. Black for core network (Router Gateway/Firewall, Wireless AP), green for my Virtualization server, the four purple cables for the four NIC interfaces on the Synology DS1815+ in order to take advantage of the Link Aggregation (LACP) bonding for a 4-Gbps connection (made possible by the Netgear GS724Tv4), and the other patch cables I chose pink to please my daughter who loves pink.
Synology DiskStation DS1815+
I’ve been a Synology reseller for a while now, but until now didn’t have the budget to invest in a unit of my own. For the last several years I’ve relied heavily on a tried & true SuperMicro rack mount server (not shown) that I believe originally ran Windows Home Server before being upgraded to every Windows Server iteration since. Microsoft Windows was the path of least resistance since most software was available for it and I mostly had Windows workstations anyway. The more recent operating systems also had the benefit of coming with Hyper-V as a virtualization platform. Though it doesn’t have some of the features I’d grown to love from VMware (like the ability to forward physical USB devices directly to a guest OS), it did the job. However since there were only four drive bays, and because Windows software RAID doesn’t expand gracefully when swapping in a larger disk, it was quickly showing its age with a meagre 1.2 TB storage spread across four 500GB drives. This was not sufficient capacity for my family’s digital photo collection, let alone our music collection, videos, or any other projects I might intend.
Long story short, I took the plunge and invested in a Synology DS1815+, a relatively new model with an 8-disk capacity, quad-core processor, 6GB of RAM (upgraded from 2GB) auto-expanding RAID file system and the option of two additional 5-disk expansion devices for a potential total of 18 disks. At this point the largest drives available are 6TB which means a theoretical maximum of 108TB of data.
I knew the features Synology offered but somehow it wasn’t until I began to use it at home and started looking for opportunities to use it more that I really started seeing the potential.
First Up, Photos.
Photos were the top priority for us. My wife takes a lot of pictures. Currently we are storing a little over 380,000 photos, taken over the course of the last 15 years plus some older ones that were scanned in from analog photographs. That said, if I were ever to lose the photos in a drive crash or other disaster, she might murder me. Until we had the Synology, out of paranoia she would store primary copies on her desktop, backup to the old server, and then copy again to a couple of NAS devices I had lying around. The problem was that no single location had enough capacity to store the entire photo library, so it had to be split among devices. It was becoming increasingly difficult to know what was stored where, or even where the most up-to-date repository was. This simply could not stand for a trained IT professional, but as they say “A builder’s home is never finished.”.
Anyway long story short, I loaded the whole photo library to the Synology and it spent a few days generating thumbnails in the background in order to more rapidly show images through the Photo Station interface. In the end it looks and performs great! I can even access all the photos and videos remotely from my phone, tablet or laptop. The iPhone utility also enables me to auto-upload photos, which is great for someone like me who never takes the time to copy the photos off of their phone. Photo Station can also act as a full-fledged website, complete with photo sharing, tagging, and commenting. And not everything needs to be shared. Folder availability will depend on user permissions and all folders and albums can be locked with a password.
Then Music
Like most people, my wife and I have amassed a collection of music — not quite as extensive as the photo collection, but significant enough that it would be beneficial to store it somewhere we could access easily. Synology has a solution for that. More than one, in fact. One is Media Server, which is a DLNA/UPnP server that streams both audio and video to compatible devices. Then there’s the iTunes server, which is pretty much exactly what it sounds like — it makes the entire music library available and accessible from inside iTunes on your desktop. Then, of course, there is the Audio Station.
Again the music is accessible from anywhere (with the proper configuration). I sometimes stream music to my phone while I work, since somehow I’ve grown tired of the hundreds of songs on my iPhone. Audio Station can also stream music directly to AirPlay devices which is perfect since there are a handful of AppleTVs scattered throughout my home. Plugins are available to automatically search and display lyrics to the library.
Central Video Hub
Synology also has a streaming video solution — Media Server as mentioned above, and also the simply named Video Station. I spent a couple of weeks importing video from a large data set of old home DV tapes, over 120 Hours worth, and put them into the Synology Video folder. The system automatically scanned all the videos and generated thumbnails for easy identification. Lately I’ve been trying to introduce my kids to some classic Disney movies that I used to love at their age, but I don’t want them to continue to abuse discs the way kids do, so I imported a bunch and we’ve been enjoying streaming directly. You can see from the photo below that the system automatically recognizes films, then searches and displays the cover art and other information. Video Station also supports certain USB TV tuner cards which can effectively turn the unit into a DVR.
All of this software and more runs on DiskStation Manager or DSM. Other packages include a built-in AntiVirus, Download Station (which searches and downloads from a variety of Torrent and eMule sources), and hundreds more.
General File Storage
Of course on top of entertainment, the Synology can be used as a regular file server. Boring old documents and such. One of my favourite features is an app called Storage Analyzer, which you can schedule to run as often as you like. Not only does it give you a visual representation of the storage on the unit, but also what types of media, what users stored them and more.
Sometimes when I’m copying data from an old drive I risk storing the same file twice in separate locations. Or in the case of our photos and videos, something gets mis-filed in the wrong month. The same Storage Analyzer automatically checks for duplicate files. From the window I can compare the files and delete the mis-filed one if necessary and saves me from wasting precious storage space.
What About Backups?
Synology has a great backup system already which utilizes Amazon’s Glacier service — with possibly the most intuitive interface I’ve encountered . If you can manage to navigate your way through Amazon’s sometimes labyrinthine AWS console, and sign up for Glacier, you can store backups there for around 1¢ per GB. So for 500GB you’d be charged $5 per month.
However, since my photos alone account for around 1.2TB of data, I had to get creative. Google, as always, never fails to disappoint, and I ran into Scott Hanselman’s blog which outlined for me how to deploy CrashPlan to the Synology for unlimited storage at around $6 USD a month.
NAS>DAS
I love the fact that the Synology DS1815+ is mostly autonomous, but that I can get my hands dirty when necessary. The fact that it has it’s own “brain” means that it doesn’t depend on any other device to work. Not so with Direct Attached Storage — if I wanted to run a full AV scan, I would have to wait until it finishes, and could not let my computer sleep, reboot or update for fear of having to start the scan over again. In the past I’ve enjoyed using Drobo DAS and NAS devices, but I was often frustrated by the lack of administrative control I had over it. Drobo has designed their hardware and software to be dead simple, and for that they’re incredibly great. But I’ve grown to need a higher level of control. Another problem I encountered with the Drobo units was a Volume size limit of 16TB — no such limitation exists on Synology.
So that’s the Synology DS1815+ at the core of my home office/homelab. Stay tuned for part 2 where I highlight the network equipment, and part 3 for the various client systems.
Speedo says
Hi,
I found your article very interesting I have the same device. Would be great to have some more details about hardware setup and performances. For example what HDD did you select and what is the transfer rate (iops, mb/s…) you get with such setup ? Which protocol are you using for bridging vm to the storage ( iSCSI… ). Is your switch fully PoE ? Do you have 10GB port SFP on the switch available ? How many VM ? What server specs did you attach to the synology ?
Thank you for your blog,
Andrew McLean says
Thanks for visiting, Speedo!
I wasn’t too picky about the HDDs I picked for the DS1815+. I mostly used whatever I had lying around. I wouldn’t necessarily suggest that anyone follow my example here but it was a hodgepodge of Seagate and Western Digital disks – the larger ones were mostly Seagate Enterprise drives that were warranty replacements, one old 2TB Seagate NVR disk from an old security recorder and some smaller WD Reds. I filled up 7 bays with hard disks and for the 8th bay I put in a 250GB SSD. Actually originally I put in a 120GB SSD but I found it wasn’t adequate for my needs, or at least the DSM didn’t seem to think so.
I haven’t done a comprehensive performance test since I haven’t really needed to yet. Plus I’m slowly phasing out the lower-performing drives and replacing with larger WD Reds so the performance wouldn’t be truly indicative until I do. I may actually pick up an expansion bay and use it to start over by creating a 6-disk array instead of a 7-disk one, and use the other two for read/write caching on a set of SSDs. Another project down the road.
My switch isn’t PoE at all. At some point I may upgrade, probably a Ubiquiti Edge Switch since that seems to be what I’m specializing in these days. At the time I build this, I was looking for the best bang for my buck while aiming for 802.3LACP, which the Netgear switch had. If I could do it again, I’d probably opt for the Ubiquiti for the PoE and 10G SFP. Not that there’s anything wrong with the Netgear, it’s just that I’m already nearing the limit of what it can do and I’m only getting started. The Netgear only has 1G SFP.
Right now I’m using a separate server – an older rack-mount Xeon that I’ve had for a few years – for virtualization purposes. It’s a quad-core 2.5 GHz Xeon. The data is stored on the server itself – I only briefly dabbled in iSCSI and NFS for virtualization/remote mounting. I don’t generally leave it running anymore unless using it to develop a project – my wife can’t stand the white noise of the fans at night.
I do have a Raspberry Pi 2 running Emulation Station that is mounting my console game library straight off an NFS share. That took some doing, but I learned some valuable lessons in the process.
The number of VMs I run varies, especially now that I’m using Docker on the Synology. I did have a handful of Raspberry Pis running various monitoring tasks for me, but eventually I migrated the ones I intended to keep to Docker. The performance difference is incredible. Like night and day. Docker works great.
Mostly I use the Hyper-V server to test out new software and network/firewall designs without sullying my own workstations or servers. I just finished designing a DMZ for a local financial institution. Fun stuff.
Union Alarm says
I tried setting up a home lab with the help of my friend, but we were not successful. So I just called in an expert, and he was able to do it for us in less than an hour with perfect organization.