The A-lister for dedicated servers is Rackspace, a managed hosting provider I'm always happy to recommend. Great support & service. The one "downside" is that their price reflects their quality.

For cheaper dedicated boxes, you have to sift through hundreds of hosting companies and try to find ones who meet your needs, aren't going to screw up, and support you to the level needed. Sometimes you don't need a lot of support or uptime, and so you can get away with a cheaper box. I read some good things about ServerBeach and LayeredTech -- they seem to be a popular choice for inexpensive dedicated machines.

VPS Hunt

I've been looking into VPS hosting a bit more and found a few well-recommended providers:

• ServInt
• LiquidWeb
• Zone.net
• PowerVPS
• Hostmaven
• MediaTemple's Grid-Service and Dedicated Virtual services.
• RimuHosting

ServInt especially seems to get a lot of kudos, but I know MediaTemple is very popular with the designer crowd. And by looking at my site you know I'm a developer not a designer. ;) A designer friend of mine uses and likes RimuHosting a lot.

GoDaddy Virtual Dedicated Server

Personally, I have months of hands-on experience with GoDaddy's VPS offering, which is pretty inexpensive, and clients have heard of them (or at least seen their SuperBowl commercials). Price is good, performance is ok (not great), but it's a very no-frills offering, and you have to be prepared to administer the server and manage backups yourself. Most of their VPSes are Linux, so you'll be using the shell a lot, but they also offer a Windows 2003 VPS with 10GB of space starting around $40/mo, so you can Remote Desktop into that.

You can pay for better control panels, managed backup, etc., but then the price starts to get into the $60-$100+/mo range, at which point you may want to look at one of the above VPS providers instead.

I also don't know whether they offer any kind of quick upgrade path, in case your site suddenly gets popular and you need to move your VPS to a big dedicated box pronto. I kinda doubt it.

Rolling Stones Gather No ... Mosso?

Other than VPSes, I was also intrigued by Mosso's offering -- hybrid hosting allowing you to run PHP and ASP/ASP.NET sites in the same set of files. However, I don't really care so much about running PHP & ASP side-by-side, since if I really have that kind of blend I might just get two VPSes (one LAMP, one Windows). I was more interested in their focus on making it easier for development companies to provide turnkey hosting and 24x7 tech support services for their clients.

However, they don't support ASP.NET 2.0, which for me is a dealbreaker -- I'd be doing my clients a disservice if I didn't give them the opportunity to upgrade ASP.NET 1.1 sites to 2.0 -- development is faster & easier, features are better, and performance is greater. Plus I have several .NET 2.0 sites already.

A question came up recently on a mailing list I'm on about virtual machines for production applications. I've used virtual servers for web & database apps before, but only for dev/qa purposes. It made it much easier to go through deployment scenarios, try out new builds & roll back if the install failed, & generally avoid the time & money sink of dealing with setting up & maintaining actual physical machines. Plus virtual machines are easier to throw onto more powerful hosts as your needs grow.

However, if your organization is considering virtual machines for their production apps, they're in growing company. Most of my experience with production apps has been to start with traditional shared hosting plans, then move up to dedicated machines. That approach works especially well if you only have a few web apps that can all run on the same machines, or if your CPU needs are high.

However, using virtual machines (aka virtual servers or virtual private servers) can be a good way to save money if you have a number of heterogenous and/or legacy applications that need their own "server" (for security, configuration, manageability, specific OS requirements, or other reasons) but don't have very high horsepower needs. In that scenario, having a separate physical machine for each app can be costly overkill, and require you to answer repeated questions from your boss like "why did we need to buy and maintain 5 new servers when they're all under 10% utilization?" 

Instead, you can have each app on its own virtual server, and put them all on one or more physical machines -- however many are needed to run the apps effectively. Costs are reduced since you have fewer machines to power, watch, store, and maintain. Scaling applications can be simplified since you can allocate more or fewer resources to specific vitual servers, and you can always move them onto other, more powerful physical machines as your needs grow. Plus, disaster recovery can be easier, since if you have a hardware failure, you can just load the virtual server onto a different machine and keep on truckin'.

VMWare has some good info on their site: http://www.vmware.com/solutions/home.html

So does XenSource (Xen is open source virtualization software) http://www.xensource.com/solutions/ 

And I might as well link to MS Virtual Server, which apparently is free(?) :) http://www.microsoft.com/windowsserversystem/virtualserver/evaluation/vsoverview.mspx

Plus there are some other links around the web if you search on "virtual server" or "virtual private server."

...or, why should I get RAID?

Like I said before, if you're getting a dedicated server for web hosting, you should get RAID. Just get it. But if you really want to know why, keep reading.

How likely is my hard drive to fail?

There are a lot of factors that contribute to hard drive failures.

Usage

If you run your hard drive 24x7, and run applications that read & write the hard drive a lot, that causes more wear on the hard drive than using it infrequently.

Environment

If your hard drive is inside a hot computer case and a hot room, it's more likely to fail.

Expected lifetime: MTBF and Service Life

The mean time between failures (MTBF) for modern hard drives is 50 years or more. MTBF is measured by running a bunch of hard drives for days or weeks and seeing how many fail during that time. That data is then extrapolated out to an MTBF number. You can search on Google or read this tutorial for more info.

However, a MTBF of 50 years does not mean your hard drive will last for 50 years. Instead, it's an average. So some hard drives will last longer, while others will last shorter. Sometimes much shorter.

That MTBF number also assumes you only use your hard drive within its specified service life (usually 3-5 years). So a 50 year MTBF for a hard drive with a 5 year service life means that, if you replaced the hard drive with a new one every 5 years, it should last 50 years on average before failing.

A rule of thumb

Ok, so we really don't know exactly when your hard drive will fail. We do know that you shouldn't use it past the end of its service life, so I tell clients that a rule of thumb for a hard drive with a 5 year service life is that you have a 2% chance that the hard drive will fail in the first year, a 4% chance it will fail in the second year, 8% in the third, 16% in the fourth, 32% in the fifth, and just keep doubling it.

What happens if I lose a hard drive?

Hard drives are near the top of the list of things to fail, and a server with a single failed hard drive can be down for 4, 8, even 24 hours as you and/or the hosting company support staff

• notice the site is down
• figure out the hard drive is dead
• find a replacement hard drive & install it
• restore your machine by restoring from backup and/or reinstalling & reconfiguring things
• test to make sure that everything is working correctly and fix any issues

So if you lose a hard drive and don't have a backup web server your site could be down for 8-24 hours depending on your backup strategy & the quality of your support team.

Compare that to the loss of a fan or power supply which can be swapped out in under an hour. Some fans you can even swap out while the server is still running.

You need to think about what downtime means to your company. Will you lose money if your site is down for a day, either through lost sales, departing customers, or uptime fees paid to your clients? Will it be an embarrassment? Remember that sites can often fail at the worst time. Think about what 8-24 hours of downtime will "cost" your company. It could be a dollar amount or something non-monetary.

Then you can multiply this cost by the chance of your hard drive to fail (see the rule of thumb below) to come up with the expected cost to your company due to non-RAID hard drive failure.

So how do I decide to buy RAID or not?

So based on the above rule of thumb, plus the potential downtime, you can estimate whether it's worth paying for RAID or not.

You might decide that up to 24 hours down isn't acceptable at all, in which case you should go for RAID. Otherwise, if it's a potentially acceptable loss, you can think of RAID as insurance. Is the extra cost of RAID more or less than the expected downtime cost?

But what if I have two web servers?

If you're running multiple web servers and load-balancing across them, you might not need RAID, since you can simply direct all traffic to the "good" web server if one fails. Remember, though, that to be able to fail over successfully, you need to be capable of supporting all your traffic on a single web server. Since servers seem to enjoy failing at the worst times (which is often when your traffic is highest), you should expect that it'll be a good amount of traffic on that single server.

If you have two load-balanced web servers and both are at 80% capacity, what happens if one server fails and you redirect all traffic onto the good server? That's right -- your good server will get overloaded & some percentage of your users will get errors and long load times, or even worse your good server will redline and crash, and then you're totally down.

So...thats all for now.

I was reading about NewsHutch, yet another web-based RSS aggregator. Although it looks nice.

Anyhow, their blog post about looking for a new services yielded some suggestions on hosting plans. You can skim through the comments and see.

I've personally worked with Hostway and Rackspace in terms of hosting. Rackspace is expensive but has IMO great support and service. Hostway has a decent array of plans but I've had issues with their responsiveness.

Railsmachine seems interesting for RoR shops. SuperbServers seems like it has great prices (2gb Conroe with RAID for $200/mo) -- the question would be how good their service is. I guess at those prices maybe you just buy two web servers & load balance them. There were also a few kudos for ThePlanet, whose prices also seemed decent. 

If you're in the market for a dedicated server, I would highly suggest getting a RAID setup.

So I have a little rant ... I was typing up a blog article on ASP.NET Membership Providers and how to move that information from SQL Server Express to SQL Server 2005 Standard. And I accidentally (as I sometimes do) clicked the thumb button on my Logitech MX510. By default that button executes the browser's "back" button, but in the case of blogging, instead executes the "undo" feature of the textbox you're typing into. Which means my pages of text disappeared, and of course there's no way to "re-do" the text other than typing it all in again.

So I'm a bit annoyed. And in order to not have that happen, I have to go to logitech.com, download the software, install it, allow the inevitable reboot, then open the software, disable that button, then have the software run in my tray all the time.

So...I'll be back later.

The problem

When the number of computers in your house starts to grow, a file server starts looking more attractive. A place to back up all your important files, as well as store central files (e.g. virtual machines, tools, photos, mp3s) that are used by multiple machines. Sure, you can burn em to CD or DVD, but that's so slow, manual, and old school. Where exactly can you back up those files?

Option 1: Get a DAS (Direct Attached Storage)

AKA an external USB drive. USB enclosures are easy to get, and you can definitely store a lot of files on them. I have a few lying around, and you don't even need an external power brick for the 2.5" drive enclosures. I decided against them, though, since I needed to share files between multiple computers. 

Advantages:

• Cheap
• Very quick & easy to use
• High transfer speed

Disadvantages:

• Can't share files to multiple computers at the same time
• Connecting to a different client involves unplugging, lugging, & plugging.

Option 2: Build a File Server

You could also build a full-fledged file server. Get a no-frills computer, load Linux or Windows on it, stick a bunch of drives in there, and share them across the LAN. I decided against this option for the time being since I wanted something cheap, easy, small, & quiet.

Advantages:

• A file server can be used for other things, like a print server, mail server, web server, etc. Which can be fun, a source of geek pride, and educational.
• Administration is via the server GUI, which you probably already know very well. And if you don't, it helps you build your resume. "Familiar with Windows Server administration" looks better than "Familiar with ECS EZ-NAS 2000."
• Very good hardware support, including gigabit LAN
• Support for RAID (software or hardware), so you can mirror two drives in case one fails
• Advanced features like disk quotas, permissions, etc. You could also set up a WebDAV or FTP server to be able to access your files over the internet.

Disadvantages:

• Cost: running a full OS (especially Windows 2003 or a fancy Linux GUI) can require more powerful hardware, meaning more money. More powerful machines also use more power.
• Size: a normal server can be kinda bulky.
• Time: installing & configuring an OS can take a while. It's not awful, but time is money!
• Noise: a full server can be noisy, too, unless you spend more money making it quiet.

Option 3: Get a SAN (Storage Area Network)

I'm not really considering this due to the cost, although you can create a SAN using a file server (above) that implements iSCSI. There is a good (albiet a bit older) walkthrough for Linux and Windows here and of course Wikipedia has plenty of blurbage on it.

Option 4: Store Files Online

You could FTP all your important files to your ISP's server (or your server), or use your Gmail account like a hard drive, or use an online storage provider like Web 2.0 darling Box.net, collaboration tool Basecamp, or others. I use this method for sharing files with others, as offsite backup, and before travelling, but not as my main file storage.

Advantages:

• No local equipment needed. You don't need to deal with hardware at all, which is attractive.
• Cheap, even free for smaller amounts of files. Like if you're just backing up source code, documents, etc.
• Access from anywhere, even on the road. If you have a net connection, you've got your file.
• Reliable: in theory your ISP, Gmail, Basecamp, etc will have better backup & power systems than you do. So you don't need to worry (as much) about your files getting destroyed due to fire, hardware failure, etc.

Disavantages:

• Slow, oh how slow. Basically as slow as your internet connection, which is probably 99% slower than your LAN. Which rules out videos and large files.
• Not a lot of storage. This is relative, of course. 1-5gb might be plenty for some people, but not for others. Upgrading to more storage might either be expensive or not an option.
• Out of your control. If the provider goes south, your files have, too.

Option 5: Get a NAS (Network Attached Storage)

This is the option I'm trying out. A NAS is basically external storage that you connect to your LAN. Your computers then access it over the network. You can always buy an enclosure for about $100 from NewEgg, etc (search on NAS), but I recently stumbled across NASLite and FreeNAS, embedded Linux kernels that turn an old PC (e.g. 200mhz!) into a NAS. Here is a very good and recent walkthough on FreeNAS, and here is a review on NASLite (plus a tip on making a 1.7mb floppy in order to boot NASLite). I'll post an update once I have either FreeNAS or NASLite running.

Advantages:

• Inexpensive, especially if you already have an old PC lying around. If you don't, you can always buy one from the Salvation Army, Craigslist, or vendors like RetroBox.
• Quiet: Old PCs don't need a lot of fans, and can often get away with passive cooling. The NAS units you buy are also pretty quiet.
• Size: Old PCs aren't small, but the dedicated devices are. And some of them are even cute, leading to a better WAF.
• Easy: Easy enough to get going.
• Stability: A simpler device means fewer moving parts, as well as very little (if any) risk of that server getting infected with trojans and viruses. That doesn't mean that an infected client can't stick a virus-laden file onto the NAS, though, so make sure all your client PCs have antivirus (hint: Google Pack has a free 6-month Norton Antivirus).

Disadvantages:

• Transfer speed: you're limited by the speed of your network and the amount of memory in the server. If you're running gigabit LAN, then it's decent, but if you're 100mbit or 10mbit, you'll start to notice transfer times for larger files compared to USB or Firewire. You can still stream mp3s & copy files around, but streaming video might be an issue for slower networks.
• Size: Old PCs can be big.
• Features: Cheaper NAS solutions don't offer RAID, which means if you lose the drive, your files are gone. FreeNAS does offer RAID, though, which is why I'm trying it first.

Conclusion

Hopefully the above helps you when considering storage solutions for your home network and important files. Keep an eye on DevelopmentNow for an update after my adventure with FreeNAS on a celeron 600.

So I bought a nice quiet 200mb Segate drive and stuck it into an external USB drive. I'm using it to store my Virtual PC images (boosts performance) and other shared files. But whenever I want to turn it off, Windows XP really doesn't want to let me stop it using the "Safely Remove Hardware" function, instead serving up a "The device 'Generic volume' cannot be stopped right now. Try stopping the device again later."

After digging around a bit I found the solution on a post on Geekly Rambling -- Norton Protected Recycle Bin (aka Unerase Wizard) kept holding onto the drive. Once I told Norton to stop protecting that drive, I was able to Stop it easily.

To configure Norton Protected Recycle Bin (if you installed Norton SystemWorks, you probably have it) to leave your USB drive alone:

• Connect your USB drive to your computer & power it up. Wait until Windows recognizes it.
• Right-click the Norton Protected Recycle Bin wastebasket on your desktop and click Properties.
• Go to the Norton Protection tab.
• Choose the drive letter of your USB drive from the dropdown, and uncheck the Enable Protection checkbox. Click OK.

That's it! Now, you might be wondering why I bother to Stop the drive at all...why not just unplug it from the PC? Well, that's because I configured the drive for higher performance using the "Optimize for Performance" setting in the Device Manager. The downside of that is you should use "Safely Remove Hardware" before unplugging it.