Friday, June 24, 2011

Another Year Older.....

Seven years ago, Atlas Quest became a reality. I first started working on it earlier in the year--back in January, in fact--but it was in June when I finally decided that I was ready to take the website live, and it wasn't until June 24th that I committed to the name Atlas Quest when I purchased the domain name. It almost seemed astounding that the name was even available--Amanda and I considered all sorts of ideas for names but the domain name invariably wasn't available. I started uploading the website that night, but admittedly, it wasn't until the next day I finished getting it all up there. I was using a dial-up connection at the time, and uploading an entirely new website took some times. Especially the database that supported over half a million cities, towns and podunks from around the world.

That seems like a lot, but I actually narrowed it down from a list more than ten million locations I had information about. =) When I tried to run queries with that much data, however, the performance hit was dramatic. Maybe it was just on my development machine, but that much data had trouble, so I cut out a lot of it. And really, why did I need a list of a million locations in Russia? It's not like that country was a hotbed of letterboxing activity! So poof--I deleted all but the largest 1,000 cities or so. I could always add individual cities as needed later.

At the time, the only cities that AQ supported were ones that it already knew about. And being able to support addresses--specific points on the globe--wasn't even a twinkle in my eye. I was just happy I could support so many cities and towns from around the world. When I started my little project, I wasn't sure I'd be able to find the data I needed, or be able to make it run fast enough to be useful.

I was so proud of how many different cities that AQ could recognize, I even added that as a statistic on the AQ Statistics page. The number doesn't change very much, but occasionally, when a new city is added, the number might increase by one every now and then. As of this writing, it's up to 540,093.

Later, I learned about the Google geocoder--a wonderful little service that Google provides which allows me to send it a location (address or otherwise), and it'll return the latitude and longitude coordinates of that point. This was very cool, and I immediately set to work in incorporating it into Atlas Quest. It has its quirks, but without it, there would be no way to list boxes in locations more specific than being "somewhere in a city." It was a revolutionary development. I basically rewrote the core of the search engine around that Google geocoder.

But, oh, how quickly technology becomes obsolete.... Earlier this year, while working on a somewhat unrelated issue, I discovered that the version of the Google geocoder I was using had been deprecated. Which is a fancy word meaning it had been replaced with a bigger and better geocoder and that the old one could be discontinued at any time and should no longer be used. I needed to.... upgrade. *sigh*

I looked at the specs for the new Google geocoder version and didn't much like it. It seemed needlessly complicated, and I started poking around for other geocoders. By now, surely Yahoo or MapQuest or something had publicly available geocoders for use, so I poked around and discovered that I really liked Yahoo's newest entry into the world of geocoding.

I also decided that it would be a bad idea to rely on just one geocoder. What if it goes down, is changed, or discontinued unexpectedly? What if the Yahoo geocoder is depreciated to be replaced with something bigger and better later?

Without getting into the technical details, I wanted to create an interface that would allow me to switch out geocoders on the fly, almost like Lego pieces. This actually provides additional advantages--it means if I try using one geocoder and it cannot find the latitude and longitude coordinates of a location, I can try again with a different geocoder. And another. In the end, I now have about a dozen different geocoders working under the hood to try to interpret your locations.

The number of cities supported now is something of a misnomer. I created that statistic when the only cities supported were those that AQ already knew about. Once I added that first Google geocoder, it was capable of learning new cities automatically that it didn't know about before. I don't know how many cities the Google geocoder knows about, but it's certainly a heck of a lot more than AQ is familiar with, and there's not really any way for me to accurately count this number.

These new geocoders I'm adding support for, however, I'm designing from the ground up. The only locations the new database tables support are those that are actually being used by someone somewhere on Atlas Quest. So all those cities that AQ used to already know about in Lybia that never really got used will be gone. I'll depend on the geocoders to find that information for me as people need it.

As of right now--I'm still in the process of converting the old letterbox locations into new ones so this number will still go up (although probably not by much)--there are 19,207 different cities being used. A far cry from the 540,093 currently "supported," but that just goes to show that most of those locations just never got used. Not even half the cities and towns in the United States were being used.

In the Next Big Update, that "supported number of cities" in the statistics will be retired and replaced with a "number of cities used" statistic. I don't know how many cities are supported by the various geocoders, but I know it's even larger than ever before. Being able to look up specific coordinates for addresses in all sorts of countries has been added, and as companies like Yahoo and Google will continue to add more and better support for more and more countries throughout the world, AQ will also seemingly get smarter and smarter with each passing day.

AQ turns seven years old today. In that time, that original core search feature has been rewritten essentially from scratch twice. Which doesn't even include the countless minor tweaks I've had to make to it over the years. I'd like to think, "This time it's it. Once this is done, I'll never have to rewrite this stuff again." I want to think this, I see nothing on the horizon to think it's not true, but I also thought the same thing those last two times I wrote this part of AQ. I'd like to think I learned something from those first two coding attempts and that the third time's a charm--but I know I wouldn't make any bets on it. That's for sure. =)

On a completely unrelated note.... I wrote a small piece of code a couple of months ago capable of counting the number of various file types I've developed.

As of this minute, on my development machine (I didn't install the code on the live website), you'll find these numbers:

  • 982 HTML files: that's now many individual URLs that AQ handles and knows about. Some of these, admittedly, are test pages that I have no intention of ever uploading to the live website.
  • 3739 image files: each icon, logo, background image--there are 3739 images for use across the website. GIF files rule with 1896, and JPGs are runner up with 1278 files. PNG files make up the bulk of the rest.
  • 117 CSS files: as a general rule of thumb, each CSS is usually one theme, but there are a few that are not.
  • 10 MP3 files: these are mostly for the chat room sounds, but one was used for the last April Fools Day =)
  • 12 JavaScript files: almost every page on AQ uses JavaScript, but there are only 12 pages I created that have nothing but JavaScript in them. (These do not count JavaScript files that come from libraries such as JScript or Prototype--this number is about how many files I created and maintain.)
This website is amazing, but not really for any of the reasons I wrote above. These are just the behind-the-scenes stuff that I find interesting, but the real miracle is that Atlas Quest has thrived for the last seven years. When I started it, I didn't imagine this would become anything more than a hobby site, but you guys turned it into the community it is today. You've supported me, encouraged me, and I thank you. I might have built AQ, but it would still be an empty shell without all of you folks who populated it. =)

Anyhow.... enough reminiscing.... there's code I need to be working on! =)

Wednesday, June 15, 2011

And why was AQ down and there's no blog to tell about it?

If only Anthony Weiner twitted this photo instead....
The title of this post was inspired by Dizzy, who posted this query to the Billboard Widget on AQ early this morning. (If you aren't sure what a Billboard widget is or why you don't see one available, that's probably because it's a premium member widget.)

The reason there's no blog to tell about it is because there's not much to tell. It's like a lot of those updates I often do without taking Atlas Quest down at all and nobody ever notices.... Changes happen, but you either can't see them, or they're so minor, you'd often not notice them.

But when I actually take Atlas Quest down and people know I'm up to something, there's that curiosity--what is Ryan up to? What is he changing? Something is clearly happening.... but what?

This particular update is the next stepping stone on the long road to a new and improved geocoder experience which I've blogged about before. This time, I rolled out the geocoder updates to event listings. If you add or edit an event listing, you'll notice that the "location" page of the listing has changed quite a bit, and--IMHO--is a lot less fussy about supporting locations that cannot be geocoded or if multiple matches are found. It even displays a cute little Google map of the location that was geocoded so you can confirm that that's exactly the location you were talking about.

The update itself was relatively quick and painless.... the reason I took AQ down for the better part of an hour was to convert the "old locations" into the "new locations." The conversion process isn't perfect, and I set up AQ to automatically do the conversions as well as can be expected, then had it send me a list of conversions that weren't "perfect." Anything where the old location didn't quite match the new location. Such as an old location of "3036 SW 3rd Ave, Seattle, WA, US" suddenly turning into "3, 3036, Beijing, Beijing Shi, China." I see something like that and I think, "Hmm.... something bad happened in that translation....." Then I'd laugh at my unintended pun before figuring out what went wrong. =)

About 3% of the conversions had problems. There are about 2,000 events now listed on AQ, so about 75 had issues I had to fix by hand. Japan and China listings were unusually common among those!

In truth, I wasn't especially worried about locations that were in the past. So if that event from five years ago is now listed as having took place in China, who cares anymore? =) But still, I looked through all of those old entries and fixed them because they were test cases that failed! I needed to figure out why these entries had failed and figure out if there was some way I could fix things so when someone is trying to list a new event, that doesn't happen again.

It's amazing what you can learn from things that don't work properly. =) I found several tweaks I could make to my own code that helped matters along a bit better, and if I tried running the conversions today, there would still be some failures--but almost universally cosmetic ones.

A cosmetic "failure" would be something like the old location being "Montana de Oro SP" while the new location maps to "Montana de Oro State Park." Technically, the location is correct and spot on and "fixing" this problem wasn't necessary. But still, it wasn't an "exact" match and my code isn't smart enough to realize that they were two different ways of saying the same thing, so it would report to me the "non-perfect" conversion.

It's important I get this right, because eventually, I want to roll out these same changes to boxes. There are currently over 100,000 traditional letterboxes listed on Atlas Quest, and about 65,000 of them are listed as "active." When the locations for these boxes are updated, I need to make sure they don't end up in China! And while I could go through 2,000 event locations in one night correcting errors, going through 100,000+ letterbox listings is a mammoth undertaking that could take eons. I will need perfect conversions with a high success rate. Even with a 1% failure rate, I'd still have to manually fix over a thousand listings.

And while fixing the listings, it also allowed me to use the real code for editing locations on the live website. It let me check that the maps were displaying correctly, and it let me check that it handled things properly when no exact matches could be found, or when more than one matching location was found. I was able to do a lot of testing during the process of fixing problems! =)

It was a grand old time, let me tell you. ;o)

In related news, everything that comes with the new geocoders also works with events now. For instance, you can now run a linear search or rectangle search on events. Additionally, if you pull up an event listing, you can now tag it with a "custom location." Imagine an event that's a mystery location--once you've figured out the location, you can specify it in your custom location, and it'll show up in event searches at your specified location. (Nobody else can see your custom locations, though, so it won't be a spoiler for others.) And the new geocoders are much improved over the old one. Thousands of more parks and places are now supported than ever before.

Lots of good stuff going on! =) But unless you're an organizer for an event, you won't really notice most of them at the moment.....

Monday, June 06, 2011

My First Plant

The day was May 12, 2001. I carved several stamps including an oak tree, a spider web, a butterfly, and a worm. I was impressed with myself. These were my best stamp carvings ever! Well, the worm wasn't particularly great, but I had a long, skinny piece of the pink stuff, and I hated the idea of it going to waste, so I was determined to carved something out of it. Being so long and skinny, my options were limited to a snake or a worm. I'd already found two or three boxes that had snakes carved, however, and I wanted to be different, so I carved a worm. Nobody--so far as I knew--had ever carved a worm before. Come to think of it, I'm not sure anyone has ever carved a worm since. =)

Back then, I always stamped my signature stamp into my planted boxes--a practice I've largely stopped doing mostly out of laziness than any real reason. And this logbook is the only logbook I know of, still in the wild, that had my original signature stamp in it. I only used this signature stamp in maybe two or three dozen boxes--I soon replaced it with a purple spider stamp because I quickly realized that the signature stamp was much too large for many logbooks! I used this signature stamp for about a month before retiring it. And that was ten years ago--the logbooks I stamped it into have either gone missing or the logbooks damaged by water.
I based it on a photo I took of Mount Hood, near where I lived at the time, and carved my initials into the stamp to really "make it mine." I didn't have a trailname back then, I was just Ryan. I'm also amused that I ordered people to plant their own letterbox and let me find it. =) Back when I planted this series, there were 30-some letterboxes in the entire state of California, and none in San Luis Obispo County where I planted the series. Getting others to plant boxes was important!

I look back at the stamps today and shake my head. So primitive. So simple. I still like them, though. Of the four boxes I originally planted that fateful day, only that worm has survived the whole ten years. The oak tree went missing almost immediately, and I've carved a couple of replacements since. The butterfly lasted a couple of years, but it too met its demise and was replaced on a couple of occasions. The spiderweb survived for most of the decade, which was all the more surprising because it was found on a regular basis by muggles who'd sign in about how exciting it was to find the box by accident. They kept the logbook and stamp there for years. The last few years, however, have not been kind to that box. Seems that potheads found it and stopped leaving it in such a pristine condition. The box lingered on, but the logbook faced water damage when the ZipLocks failed. I replaced the logbook on a couple of occasions, but when I checked up on the series last Friday, the stamp was gone.

But the worm keep on ticking.... The box celebrated its tenth anniversary last month, and still had the original stamp and original logbook in the box. The original logbook really surprises me. About a hundred people have logged in, and it's still the original logbook! Stamps can survive water damage, but logbooks generally do poorly. Invariably, ZipLocks always fail. It's not a matter of if--it's a matter of when.

And this box is no exception--the ZipLocks have failed multiple times over the years, and I've replaced them, yet the logbook has always stayed dry. Or at least never more than a slight dampness about it, like dew on the grass in the morning. And you want to know the secret of longevity for this letterbox? A boardwalk.

I hid the letterbox under a boardwalk. I didn't realize it at the time, but it was the perfect hiding place. The boardwalk had about two inches of space under it--not enough for people or kids to play under and find the letterbox. The boardwalk had boards pressed right up against each other--no large gaps that rain would fall through. Oh, water could (and does) dribble through the slots between the boards, but most of the water sheds off the sides. The hiding place essentially is a roof for the letterbox to protect it from the elements, and that's why the logbook has survived for so long.

I also included a logbook with a lot of pages--the tiny logbooks common today were a rarity back then. People had more space to write messages (and did so!), so the logbook never filled up despite the hundred or so people who have found it.

That's an important lesson to remember. When you're out looking for a place to hide your letterbox, look for a location where it won't be found by accident, and look for a location that has a natural "roof" to shed rain. I've used fancy, seal-tight containers, but they can fail. Some are better than others, but even the mighty Lock-n-Lock can fail. ZipLocks will also fail. And when they both fail, your logbook is toast.
The best, idiot-proof way to keep those logbooks intact for years and years is a hiding place where water just won't drip on it in a rain storm. These kind of locations aren't easy to find, and in many places, it may be absolutely impossible. But look for them. Holes in a tree (as long as the hole doesn't open skyward), under large rocks that can shed rain water, and under man-made structures like boardwalks that can shed water to the sides. These are the kind of places where a logbook can survive for a decade or more, even when the container and the ZipLocks fail.

I also like this logbook because people have written lots of messages in it. Many logbooks today are too small to really write anything, but if you give people enough space, at least they have the option to do so. The larger the logbook, the more likely people will fill it up with stamps and words. Of course, the larger the logbook, the harder it is to find a good place to hide the letterbox. Give and take, ying and yang. =)

I've thought about replacing this logbook for years. Not because it was almost full, but because it had so many finders, so many stamps, and so many wonderful messages written in it. If it was ever lost or damaged, I'd be heartbroken. Most logbooks I've planted I don't have any sentimental attachment too, but this one is special. It was while visiting this logbook with Jess and Wes (AQ members #2) that I felt my first (and thus far, only) earthquake in California. I was sitting on the boardwalk when I felt it shake a little, and at first I thought one of them was shaking it. They thought I was, which was when I realized it was actually a small earthquake and had the pleasure to tell them they felt their first earthquake ever. =) And how lucky were they? I grew up my whole life in California and never felt an earthquake. They visit the state for two days and get to experience one! It was just a small quake, though. Later we learned it was 5-point-something. Enough to feel it, but not really enough to do any damage.

But I wanted to see if I could keep it out there for a full 10 years. And finally, it's been 10 years. A decade. I've switched out the logbook with a new one. The original stamp is still there to found.

When I started Atlas Quest, I needed to populate the site with letterboxes, and naturally I started by adding all of my plants, in the same order I planted the boxes, so the Los Osos Oaks Series officially become box #1. A historical moment in AQ's history, to be sure. =) As I type this today, the last letterbox listed on AQ is box #173076. (I'm waving at you, Canada!)

When I checked up on the boxes last Friday, one box was missing completely. The box I call the pothead box was still there, but the logbook was soaked through and the stamp missing, so I've removed it for now. The worm was still alive and well, but I replaced the logbook just to preserve this one forever. And the last box was severely water damaged, so I replaced that logbook as well.

Whenever I've replaced these boxes, I've always carved exact duplicates of the original stamps, even though my carving abilities have improved dramatically since then. After ten years, though, I think I'd like to do something more. I want to carve new and improved stamps the next time I replace the boxes. Trails have changed since I originally wrote the clues, and I want to fix that up a bit. The next time I'm down visiting San Luis and go out to this park, I'd like to expand the series. This series is special to me, but by golly, it needs a facelift, and I'm going to give it one. =)