Monday, February 27, 2006

Server Load and Scalability

A while back, I wrote up an article on server load and scalability. I thought it's high time that I linked to it: http://mike.dewolfe.bc.ca/programming/scaleandload.asp

tags: server load, scalability, load balancing

Tuesday, February 21, 2006

Application Triage

It is said that people will have several careers in their lifetime. For me, a past career was in first aid. Part of first aid is triage: a system of delegated and prioritizing injuries. While I did shelve this concept for a while, it has resurfaced in my current profession: IT. Using an analogue of the medical profession, triage, look at some IT problems and sort out what takes priority.

Note the title of this piece: "Application Triage." There is a wholly separate protocol for large scale hardware failures. Let's focus on what happens when applications fail and there are more problems than developers/sysadmins.

The idea is START: Simple Triage And Rapid Treatment. A triage system that can be performed by lightly-trained lay people and IT staff alike. The first rule: triage is only done when there are two or more problems to deal with.

Triage separates crises into four groups. Let's lift these directly from the medical rules for triage. The DECEASED are beyond help, the injured who can be helped by IMMEDIATE attention, the issues that can be DELAYED, and those with MINOR problems—the walking wounded who need help less urgently.

If the application in question won't run: it's DECEASED. In medical triage, an apparently deceased patient is put to the bottom of the triage order. The good things about a DECEASED application: it won't write errors or give users improper imformation. The bad thing: a dead application is still required and so it must be eventually repaired. In application triage: we revisit deceased applications after those that required IMMEDIATE attention.
If the application runs but writes errors (e.g. inserts empty records), it needs IMMEDIATE attention. As long as it runs, it will make additional errors. That's a big deal.
If the application runs but does not write errors (e.g. produces erroneous output that is not stored or used in a critical process), it can be DELAYED.
If the application runs and has errors that are minor (spelling and display errors), it can be considered MINOR. If something isn't right, it never gets below MINOR unless it's been REPAIRED.
REPAIRED: you've dealt with it and all is well with this particular issue.

To further divide these four categories, there are three more considerations: reach, impact and repair.

Reach. Does the problem affect one user or every user? If it affects more users, it's obviously a higher priority.
Impact. How does it relate to the key goals of your organization? For example: the payment processing part of an e-commerce site is critical while the "contact us" section is less so. When the impact is great, push the issue to the top of the list for that category of problems.
Proximity. If you have two related problems: deal with them both at the same time. Why open up a script, fix it. Close it, move on only to return to same part of the same script later on? For example: if you have two DELAYED problems and one MINOR problem; and one DELAYED problem happens in the same script as the MINOR problem: repair the DELAYED and the MINOR problem while the script is open.
Parts. You never think this one is a factor: what if you can't repair something because you are missing something essential to your repair? What if you know something is missing, but you do not know what to replace it with? For example: you are told that a web link is linking to the wrong place. You ask what the right location is but do not get an answer. You don't have the parts required to carry out the repair. If that's the case, leaving it means it always next in the queue for repair. If you put in protem information, you haven't repaired it. If you cannot get the information, downgrade the item (e.g. from IMMEDIATE to DELAYED). If the information is available by the time you get drop the triage list: great. If not, downgrade it again.

Interested in medical triage : http://www.absoluteastronomy.com/reference/triage

tags: triage, application development, debugging

Monday, February 06, 2006

Email Stamps On Their Way

Subtitle: "How to Monetize Users.. More"

AOL and Yahoo are preparing to charge senders 1/4 to 1 cent per message to bypass the junk mail filters and go directly into the mailboxes of their respective clients.

This stupid idea has been circulating the web for years: tax email, charge for stamps, etc.. The problem with this concept is two fold. First, anyone with a copy of Apache and P233 can process mail. It's not like the US Postal Service: on the web, there are tens of millions of mail servers. Second, who pays for a message? The sender? Sure, but I wouldn't pay if I could get away with it. So the recipient? Great, what would be worse than getting 200 messages per day, but having to pay for it.

What AOL is pitching isn't an end to free email. All mail will get through, both paid and unpaid. But, AOL will be conduiting unpaid email through their junk mail filters, which we have seen issues with before. If a mail sender pays, the mail goes on a express path for the mailboxes and it's flagged as legitimate. Paid emails will no doubt be much faster, travelling the sparse and orderly highway amongst the pinkish muck of spam.

Yahoo and AOL also say that the messages have to be requested by the recipient. Yeah, right. When was the last time that you asked for an ad? There has to be a financial incentive for the users to accept an ad. AOL could charge some users more for ad free service but if they raise their prices to allow for this model, people will go elsewhere. Will ISPs share some of their ad money with users? That's like being paid to surf and you know how well that's working out these days.

Email certification in general is an interesting idea, but leaving one company (Goodmail) in charge of it is bad and promotes a monopoly. Here's a link to the L-Soft discussion about Goodmail. Goodmail is going to either be the next Microsoft or the next New Coke. If you want to bet money on it being the former, I'll take that bet. I'll give you odds.

This will do a lot of things, few of them are good: I think this may ghettoize unpaid emails. I also think that spammers will figure out a way to forge the stamp of approval. Lastly, there is no reason that a spammer with the need for market penetration won't pay the money to hit AOL addresses unencumbered.

This also is another example of balkanizing-- of network neutrality fading away. The idea of neutrality: you allow it, because you benefit from getting this same privilege from others. A lot of big players in networking are trying to hand Google and similar big sites a bill for the use of their networks. Usually with a bill comes a promise of termination if you don't pay. We'll have to see how that will play out. One theory is that Google may transmit the data themselves. This Cringley article talks about Google's idea of landing ready-to-go data centers that could be a few hops away from any user. If they are that close, Google traffic could go through its own dedicated network and ignore all of the ISPs waving their bills.

Worst of all: this won't stop spam. Before, spam cost the ISP money to deliver; cost your time to read and cost the spammer very little. Now it's going to cost the spammer a little more (e.g . $100 for 40,000 emails); cost the ISP much less; and you're still stuck with spam-- spam and the frustration that your Aunt Millie's message is stuck in a queue clogged with other freebie emails while the Loan&Mortgage people can drop stuck in your mailbox in an instant.

How will senders be guaranteed to deliver to real users on AOL or Yahoo? If I were an advertiser, I'd feel pissed off if I paid $100 to send thousands of messages only to have jjwalker100@aol.com and jjwalker101@aol.com turn out to be non-existant recipients. How do you get around this? Well, the ISP would have to somehow verify the addresses before the fact. Does this mean that AOL will get into the mailing list game?

Viewed in an optimisitic light, the cost is too much for spammers. Instead it will be used for transactional mail-- if your bank or an online shop wants to get a message to you with a stamp of approval. With that said, I get 20 messages a week from eBay and PayPal and about 1/100 of those are legit. Will users know what to look for when verifying their mail?

Spammers will use the best, cheapest technology. I'm already receiving Skype spam. I get cellphone spam. I've deleted splog. I'm ready to take a mirror with me into the shower to make sure a spammer hasn't tattooed an ad on my ass tailored to appeal to my wife and annoying motorists.

This will not work at the end of the day. Since the decline of the gopher protocol, people on the web have figured out that there is always an alternative. If you can't download an image here, go there. Can't get an answer? Look elsewhere. Don't like your ISP, move. With hundreds of millions of post offices out there, the schmuck with stamp office may end up without much business.

tags: AOL, Yahoo, email, spam