Brad Fitzpatrick (bradfitz) wrote in lj_biz,
Brad Fitzpatrick

Artificial Latency

Intro Story:
I worked at a place once with a pop machine that was free. You just pushed the button and your can fell out. It was great. One day there was a company meeting and they told us they were spending over $100/day just on Coke & Dr. Pepper. And this was a small company.. just a house full. In the end, they had to change their policy and set the cost at 25 cents. They said their rationale was that they wanted it "somewhere between cheap and free". A quarter wasn't bad.

Actual Post:
In this post I'm going to discuss a new change as far as how the free servers vs. paid servers work. But it's not so much of a change as much as it is an adjustment to make up for the speed that clustering brought.

Let me explain....

In the past we had paid user web servers and paid user database servers. All were fast. Then there were the normal web servers and the normal database servers (about 3 times as many of them), which were always heavily loaded and slower.

With the new clustered database setup, it's imperative that all database servers are always fast, which they are. The database loads are all incredibly low now, which is wonderful. It actually turned out way better than I'd even hoped.

The result, though, is that free user access became as fast as paid users. Is this bad? Not if you're a free user. But it is if you're paying over $10,000 per month on bandwidth. Since the clustering, our bandwidth usage has shot through the roof. I fear our next bandwidth bill.

So, to help out, I put mod_gzip everywhere, which compresses page before they're sent to your browser. This is all automatic. It puts a slightly higher load on the web servers, but it saves about $2,000/month.

The end result for users, though, was that the site was even faster, since they were downloading less. (modem users especially)

Today's bandwidth usage was over 20 Mbps for a number of hours. It's never that high, and never for that long.

So, I had to resort to more aggresive measures to slow down the free service. I wrote an Apache module (mod_latency, now in CVS) that adds an artificial delay to each page request for free users.

My target is to make the site fast (and reliable) for free users, but not quite free. (see story above for parallel). Making it infinitely fast just costs me too much. I want to make everybody happy, but that has to include LJ's bank account.

Please consider that I could've hid this all from everybody. I didn't have to subject myself to the potential flames that I'm sure are coming. But I wanted to be honest with what's up.

You can observe the difference now here:

Hope you all understand. Hell, it can't get any worse than it was a few weeks ago. :-)

  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

← Ctrl ← Alt
Ctrl → Alt →
← Ctrl ← Alt
Ctrl → Alt →