AnyHosting

One key to managing a small business that you do not spend full-time on is to spend the little time you have automating the processes that you know you will need, such as:

• billing
• account management (create/remove/update/delete)
• monitoring
• backups

It can be tough to know when you should do a particular task (say, moving records from one billing system to another) by hand, or take the time to automate it. My advice would be to do something by hand the first few times, and if you expect to have to do it again you’ll have the hands-on knowledge you’ll need to automate.

When I have a persistent task that I’ve had to do a few times, I first look for a system to suit my needs, before trying to write it on my own. Since I’ve done a bit of programming in the past, my priority is usually:

1. find an external service that can do this for me
2. find a suitable, well-supported, open-source application
3. create a custom application, using as many open-source modules as possible

#1 is a fairly new addition there, for me I prefer not to used closed-source software with my own business, because it means I can directly modify (or hire someone to directly modify) the processes that run my business. I’m less concerned with software and processes that other businesses use, as long as I can easily integrate and replace their services as necessary.

Google Apps is one example of #1 for email/calendaring services. On #2, for instance managing server rollout, I’d look to things like kickstart and cfengine (or maybe puppet) to keep my servers in sync and up to date.

Account creation and possibly billing can be an example of #3, depending on how your business works, exactly how you configure customer’s accounts, etc. For billing, some find that a spreadsheet or other “offline” software is enough; personally I need a system that reminds me.

The most important things, to me, when writing custom software are:

1. don’t do it if you don’t have to
2. do the simplest thing that could possibly work

Both of these will save a lot of time, energy and money down the road, both your own and in contractor fees.

A simple, custom system is easier for someone to get started with, understand, fix and modify than a complicated, custom system; a system that other people actually know and use will ideally save you more time still, and make people who already understand the implementation easier to find.

After finishing my post on customer focus, I found this blog entry by fellow planateer Isabel Wang. I wish I had found it before I wrote my previous post, as it’s a very insightful piece and I would’ve hit more on the customer focus versus “traditional customer service” aspect. Here’s an excerpt:

Which is why I don’t buy it when folks in web hosting say that they’re not worried about Amazon or Microsoft or Google - because they have “better customer service”. It’s also why I disagree with this Texan’s view that in order to build a better hosting company, you’ve got to make sure your employees understand the importance of customer service.

In my mind, there are two types of “customer service”. It’s great if you pick up the phone before the second ring, answer how-to questions with super human patience and respond to billing disputes with courtesy and generosity. But sooner rather than later, the need for this kind of generic support will be eliminated through innovation and automation.

On the other hand, there will be increasing demand for Rackspace’s kind of support, for helping the Slideshares and YouTubes of the world (aka the successors to today’s shared hosting resellers) plan and manage and scale their infrastructure. In fact, support might be the wrong word for their kind of requirement. Expert advice would be a better way to describe it. In depth knowledge from someone who understands your technology infrastructure - AND your business - at least as well as you do.

I totally agree, except that I think there is a further distinction - your customers (no matter who they are) should never need to contact your company for support, in the general. It’s not a user experience anyone wants, in almost all circumstances.

But no matter how redundant and automated and innovative your setup is, there are always bugs (or perceived bugs), network outages, and the like. Having appropriate communications channels with your customer is absolutely key, and one of those channels should always be a real human being (and they should have a “real” job as well, because they should almost never get calls ).

Hooking customers up with a knowledgeable person who has the authority to solve their problem is expensive, and should be treated as such. Their experience should be quick and easy if they are contacting you directly, because they have already run into something that is not working as they expect, and they are already frustrated.

In the end, it’s really all about the user experience. Everything should work seamlessly and easily as much as possible, and the last resort should  be getting a human to manually intervene for you. Customers really don’t want to be *forced* to interact with you (nothing personal!), so put effort into making it so they don’t have to.

In this excellent article on remarkable customer service, Joel Spolsky lays out seven things he learned from doing customer service himself for his “bootstrapped” software company.

I’ve done tech support and customer service in the past, and a lot of these points resonate with me, but particularly #4 (”Take the blame”) and #6 (”Practice puppetry”). I think the title on #6 is pretty self-deprecating, and I wouldn’t put it just the way he does, although his description certainly brings vivid imagery to mind, and packs a more visceral punch.

I’d emphasize more that, when doing customer service, you are on the customer’s side, representing them to the rest of your company (and not the other way around).

“Practice puppetry” is about how to handle abusive or otherwise offensive customers, and not take it personally. This is absolutely key to running a worthwhile business, in my opinion. There’s nothing to be gained by winning an argument with your customer (that link is a poignant response to this post showing someone taking their customer’s frustration personally).

#7 (”Greed will get you nowhere”) really got me thinking. AnyHosting is purposely a very small operation, and we tend to be flexible about billing (e.g. if someone signed up for a year and decides to move, there’s no contract lock-in), but the idea of having an explicit no-questions-asked money-back guarantee sounds like a really great idea. We have a lot of policies about putting the customer’s needs first, that we need to be more explicit about.

(EDIT - removed “over the phone” from “customer service” )

One thing I didn’t cover in my post about secure web hosting is how to do virtual hosting when you’re running different instances of the web server (be they in a virtualized environment, a chroot, or just different users on the same host, or even a standalone server like Tomcat, JBoss, Weblogic, Ruby on Rails, etc.).

Note that you can’t host secure (SSL) sites via any kind of name-based virtual hosting, because SSL happens before HTTP.  However, you can do IP-based virtual hosting, meaning that you need an IP address for each secure site.

A simple approach is to set up Apache to proxy requests from your primary server to the actual client server. IBM Developerworks has an excellent article on this, complete with an image that underscores the compartmentalization aspect I was hitting on before:

In this illustration, the “backend servers” can be on separate hosts, virtual hosts, or the same host (as long as each web server instance uses a different HTTP port).

We’ve done both managed hosting as well as virtual hosting at AnyHosting, and there are many challenges to doing it securely. As Bruce Schneier is fond of saying, “Security is a process, not a product.“. All of the techniques described in this post aim to mitigate the amount of damage a successful attack can do, but this must be part of an overall security strategy, which is beyond the scope of this post (but I’ll say more about in the future).

There are many aspects to security; what I’m talking about today is compartmentalization -  that is, either physically or virtually separating customers from each other (into separate “compartments”), so that a break-in to one service does not result in a break in to all of them.

When you’re doing managed hosting of separate physical servers (or even virtual private servers), there’s generally a lot less to worry about - a malicious user getting access to one customer’s server does not generally lead to a break-in of all your customer’s servers, unless you’ve been lax with password or network security (e.g. not using encryption inside your network, using the same passwords everywhere, etc.).

However, virtual hosting presents much more of a challenge to secure properly, as the whole point is to use the same server instance for multiple sites. The primary problem is that most servers operating systems that use Apache (the most popular web server on the net right now) run all websites as the same user, which means that if you allow dynamic scripts to be run (CGI, PHP, etc.) then all users have read/write access to all files on that server.

The halfway point between VPS and virtual hosting is operating system-level virtualization. You’ve got different choices depending on OS: Solaris Containers, FreeBSD Jails, and Linux VServers  are all popular choices. These each give the customer a private filesystem and process list, as well as allowing the web host to limit resources like CPU, memory and I/O.

A solution of last resort is the Unix command “chroot“, which at least separates the filesystem from separate users, but does not allow the host to control resources and care must be taken to prevent users from breaking out. Tools like jailkit can be helpful, if you decide that you need to go this route.

The safest and easiest way to host separate sites is of course separate physical servers for each client; it’s also of course the most expensive. The cost/effort ratio continues in the same direction as you look at VPS, OS-level virtualization, chroot and finally no separation (this is only applicable if you either manage everything yourself and do not allow user uploads, or don’t allow dynamic scripting of any kind - however, you still run the risk of a break-in to one site affecting all of your customers).

(EDIT: fixed jailkit link)

I’ve been doing some research to prepare for writing a new article on hosting reliability in general. In particular I’ve been brushing up on geographical failover, as well as routing users to the server that’s going to be most responsive for them (these two problems have a lot of solutions in common).

I am really surprised that there seems to be a lot of general information out there on Global Server Load Balancing (aka GSLB), and not a whole lot about multihomed setup using BGP (aka Border Gateway Protocol; this really just means announcing the same IP from multiple places).

Read the rest of this entry »

Much is written about reliability for computing services, web services in particular. “Five nines” uptime became a popular catchphrase a few years back (referring to 99.999% uptime, which turns out to be about 5.2 minutes downtime per year).

Reliability is obviously a good thing; you want your website to be up whenever your customers might need it. However this often leads people to spend way more time, energy, and money focusing on building out a large infrastructure (and excess capacity to handle peak load) than is really necessary or useful, and can in fact have a negative effect on your reliability if you’re centralizing too much.

Assuming you have the same amount of resources (human, financial, etc.) to spend on a centralized location with a small number of large servers, or on several locations with a large number of small servers, I’d almost always go for the latter. Also, having redundant machines and a way to remotely restart and diagnose your equipment should be a given; it doesn’t matter where in the world your servers physically reside.

Read the rest of this entry »

Isabel Wang’s blog feed is now carried on planet.anyhosting.com. I’ve been enjoying following her posts for some time.

Isabel has a ton of industry experience and I am very pleased to carry her posts on Planet.

Entry-level web hosts like AnyHosting tend to have fairly simple price structures designed to fit within their upstream provider’s price structure. Typically, there is a buffer such that the virtual hosting customer or the provider can go over limit temporarily, although some fees may be incurred (especially in the case of bandwidth charges, usually measured at the 95th percentile to allow for this kind of thing).

Provisioning more storage space or bandwidth for a particular customer is generally a manual process, and makes the pricing structure more complicated, so many providers tend to avoid it if possible.

But what if you could access (virtually) unlimited storage on-the-fly, or boot up a few more (or a few hundred more) servers to handle an unusually high load?

Amazon has launched a few services (under their Amazon Web Services moniker) that do just this - Simple Storage Service (S3) and the Elastic Compute Cloud (EC2). Both seem to use fairly straightforward web service APIs (EC2 looks like it supports SOAP and also simple HTTP queries).

S3 gives you (virtually) unlimited online storage, and EC2 lets you create a Linux disk image and boot up a (virtually) unlimited number of virtual servers.

EC2 is still in beta, and I haven’t my own account to start playing with it yet, but I have checked out the storage piece (S3). You can read/write files using the web services APIs that Amazon provides, and also link users directly to the files.

It looks like you can tie this all together with Amazon’s Simple Queue Service (SQS), the idea being that you could put the computing jobs you need done into a queue, which would send messages to your EC2 virtual servers, which could then read and write into S3. There’s a good illustration in this tutorial on doing mass video conversion using AWS:

The cost of all of these services is all per unit used (storage, CPU, messages sent, etc) as well as data transfered. These are all currently in the range of 0.10 to 0.20 cents, so if you have a good idea of your projected usage (or have infrequent need of large amounts of resources) it could be a really good deal.

This has been out for a while, and there are a ton of reviews and opinions out there, so I am going to hold off on saying much more until I’ve had a chance to try it for myself. That said, I am very much looking forward to this kind of service taking hold; it has the potential to shake up the industry and bring the cost way down on the website lifecycle idea that I’ve written about before.

(EDIT - “seem use” -> “seem to use”)

I’ve just thrown the switch on planet.anyhosting.com, a planetplanet-style blog aggregator. This gives you one place to go (or pull a feed from) to see a variety of web-related blogs.

Please send an email to admin@anyhosting.com if you have a blog and would like to be considered for inclusion. You don’t have to do any special setup, just include a link to your blog’s RSS or Atom feed in the email.