Apparently, I've been a bad, bad boy....

Some of you might have heard me eluding to problems with atlasquest.org. What is atlasquest.org, you ask? It's a domain name I own but generally haven't had much use for. I bought it with the theory that I'd rather own it and not need it, than to want it and later find out I couldn't have it.

If you do go to the website atlasquest.org, you'll see that it is indeed a real, honest-to-goodness website. It's not very useful for anything, but it does serve one important use for me: backups. Each night, Atlas Quest dumps the database into files. Backups are extremely important for anything important because computers aren't perfect. Hard drives crash, things happen.

So I don't really want to store the backups ON the main Atlas Quest server. If something should happen to that server, not only would the website go down, but the backups for it would be lost as well! Definitely not a satisfactory solution.

So that's where atlasquest.org came into play. I picked a cheap web hosting provider. To make things work, they kind of like to have a domain name to work with. So I "sacrificed" atlasquest.org. That account is hosted on a completely different server, in a completely different state, by a completely different company. And each night, after Atlas Quest backups up the database for the day, it FTPs everything that needs to be backed up to atlasquest.org.

This includes the nightly backups of the database, along with photos that were uploaded during the day--including calendar submissions which get their own special place.

I'm now storing about 6 gigs of data at that location. Which isn't really a whole lot. When I signed up, I was given a disk quota of 10 gigs. I knew I'd outgrow the space eventually, but I figured I still had at least another year or two before I had to worry about it.

Until last week, when the company hosting my account sent me an e-mail saying that it appeared I was using the account to backup data, and accounts were not allowed to be used for that purpose. Actually, they said I could not back up data that was "personal and non-web-related." It came as rather a surprise--I don't remember seeing that clause in the terms of serice when I signed up a couple of years back.

I don't really consider the data "personal" and it's very much "web-related," so I tried talking them into letting it slide. But alas, no, they have standards or something and the terms of service must be complied with. I have to get rid of the data from their servers. I've been a bad boy, and they no longer want me around. =(

So, I'm in the process of setting up a new system for backing up data. Turns out, there's this interesting concept called "cloud computing," and it'll actually cut my backup expenses by about 80%. And there's no limit to the amount of data I can back up, so that won't be a concern. (They will charge more as I use more space, however, but the rates are quite reasonable.)

The new backup location doesn't require me to "use up" a domain name, so atlasquest.org will once again be free for me to use and abuse as I wish. I'll probably move it to the main AQ server and just have it redirect to atlasquest.com. At least until I find something better to do with the domain.....

You realize you folks have uploaded over 60,000 images to Atlas Quest? Some of them have been deleted over the years, of course, but there are a lot of images. Far more than I ever imagined I'd have to worry about less than five years after introducing Atlas Quest. Each image is stored three times--a thumbnail version, a "normal" version, and a "large" version. Most of the large versions I've been deleting from the main AQ server because of space concerns and because most of them are all that important, but the backups still had the original large versions. (For those clues that weren't properly marked as a "photo clue," that's how I was able to retrieve the large versions of the photo.) So with 60,000 images uploaded, times three for each of the three versions of it stored--that's 180,000 images.

I've at the library now (broadband connection--you don't deal with 180,000 images on a dial-up connection!)--and have been here for four hours. So far, I've retrieved 43,200 images. About 10,000 images per hour. At this rate, I'll need 18 hours here to get all of the images off of the backup site. But I'm still being productive. As I take data off the old backup site, I'm working on code to get Atlas Quest set up and working with the new backup location. FTP won't cut this time around. I'm learning how to get and set secure web pages and learning the API the new hosting provider has for storing and retieving files.

Which is all "back of the house" stuff that none of you will even notice. =) That's okay--I don't expect anyone to notice. In this case, if I do my job RIGHT, it means you WON'T notice. The one thing some observant members might notice is that atlasquest.org will no longer be an official stand-alone site like it is now. But it's not a site you'd normally visit anyhow.

But I do find the sheer number of photos I'm having to deal with somewhat astounding. It used to be I could copy them all to my personal computer in mere minutes. When I first started backing up images and the database, it filled perhaps 100M of space. Atlas Quest has come a long way since those humble orgins, and just another example of the tweaks and modifications necessary to keep AQ running smoothly despite the increase in size and usage. =)