Major Announcement! And totally not click-bait!

So.... since the Last Big Update, you might be wondering what I've been up to. Mostly still working on speeding up slow queries. That Last Big Update largely fixed the worst of the issues, but that didn't mean there wasn't still room for improvement and I've continued to improve.

I wish I could play the Jaws music here. The major announcement
is ominous, but good in the long run! Photo provided by
Silver Eagle while snorkeling with whale sharks in the Maldives.
(I am SO totally jealous too!)

The new and improved code I wrote for Atlas Quest has been simply awesome. I'm very, very happy with how it's been performing, and once AQ seemed to be running well, I copied it over to LbNA where I fixed up some other slow-ish queries. Those weren't as problematic since not as many people use that website and the queries are a lot simpler anyhow, but since all my websites run on the same server, keeping things running fast on other databases will also help the AQ database run faster and better as well.

Then I copied the new query code over to Walking 4 Fun and repeated the process, improving the slower queries that I identified. The busiest page on W4F, the "My Walk" page, I'm happy to report, used to take--on average--about 1 to 2 seconds to run all of the queries needed to generate that page, and it's now down to about 35 ms. I just opened the page for myself and all of the queries--combined--took precisely 38 ms to run. Or, in other words, 0.038 seconds. Massive improvements!

And like I said before, that does help the AQ database run better as well. If you happened to run a search just as someone else was trying to look up their daily steps on Walking 4 Fun, your query might have run a second or two slower if the database was bogged down handling these other queries. So fixing these types of things are really good for all the databases on the system.

And then I've been hopping on and off each of the three major websites that use the database each day, looking for additional other queries that can be optimized. Some of the optimizations might not even seem like they're worth the effort. One query I improved a couple of days ago on LbNA now runs ten times faster! Which sounds impressive, until you realize that it originally took 0.20 seconds to run and now it takes 0.02 seconds to run. It is ten times faster, but it was already pretty fast to begin with.

But still, I've been making these kinds of "modest" improvements for weeks--months even, if you include the tweaks I made before the Last Big Update, and collectively, it starts to add up. A query that runs 1/10th of a second faster 10,000 times per day means the database doesn't spend 1000 seconds (about 16 minutes) every day running that query anymore.

And occasionally, I still stumbled onto an elephant of an improvement. Someone runs a search with weird parameters that take 20 seconds to run and I can tweak the query to make it run in less than a second. For that person, they'll see a dramatic improvement if they ever run that search again, and for everyone else, their queries won't be hung for 20 seconds while dealing with that other one. I'm not finding many of these kinds of improvements anymore, but it's always satisfying when I do. =)

But the database is still suffering from one issue that all the speed improvements in the world won't fix which is... a lack of memory. The database runs on a server with a mere 10GB of memory and the databases it uses are quite a bit larger than that. There's sometimes trouble getting all of the indexes to fit into internal memory--which helps queries run fast--and there's not really any way for me to optimize that. Basically, it sometimes needs to look up an index from the disk, which is slow. It's kind of like the difference between looking up a term in an index of a book or having the index memorized and being able to skip the process of actually looking it up because it's already in your head. 

And the next big improvement for the database is.... a bigger server! Specifically, a server with more RAM. So I started looking into upgrading the server. Originally I thought I'd just upgrade to a bigger VPS which is relatively painless and easy, but it turns out if I upgrade to a dedicated server, I'd actually get even more RAM, more disk space and at a lower cost than keeping it as part of a VPS!

So.... within the next week or so, I plan to move AQ (and LbNA, and W4F, and all of my other websites) to a fancy, new dedicated server.

Unfortunately, this will require a relatively significant period of downtime. The IP address of the websites will be changing, so there's also the whole issue of IP address propagation as well. Even if the new server is up and running fine, you might have trouble accessing it until the new IP address gets to your computer. For most people, this usually updates within a few hours, but some people in the past have had systems that took a week or longer before picking up the new IP address. It's not something I have a lot of control over either.

Will it be worth it, asks Wise Wanderer? I think so. *nodding* =)

The support team where I host the server reports the migration will take "24 to 48 hours"--that's the period of time that AQ (and LbNA) will be off limits to everyone. Once the migration is complete, the websites should be available within a few hours after that. Technically, the old servers will keep running during the migration, but any new letterboxes that are listed, messages posted, AQ mail sent, etc. won't be copied to the new server. It'll just appear like they were deleted.

I'm going to try seeing if I can just prevent people from logging in. If you can't log in, you can't really add any data to the database that won't get copied to the new server. This way, at least you can still run searches and look up publicly available boxes and message boards, even if you can't see the restricted ones that you would otherwise qualify for. I'm not actually sure how simple this will be since I never really designed AQ to keep running with the login options turned off. If it starts becoming too complicated, I might just choose to put up the "Under Construction" theme and make the entire website unavailable until the migration is complete.

But in any case, this will put both AQ and LbNA (and W4F for those of you who use it) out of commission for at least a day or two. Looking at the stats on AQ, mid-week (Tuesday through Thursday) is the least busy time for the website, so I'll probably try doing the update starting Tuesday of next week, and it should be available again by Thursday.

That's my tentative plan. It could change. When AQ and LbNA are down, I'll post updates to the AQ Facebook page at https://www.facebook.com/AtlasQuest so make sure you're subscribed there if you want updates.

Even after the transfer is complete, there will likely be times when I need to restart the database server. With more memory, I'll have to update the database configuration options and I'll watch to see how it performs for a day or so, make some tweaks, restart the server, and repeat. So there will likely be some hiccups for the first week after the update as I fine-tune the configuration settings. Once it's all done, however, I hope AQ (and LbNA) run better than ever! =)

The new dedicated server I'm eying, btw, has 16GB of RAM--which almost doubles the available amount of memory for the database to use. It also comes with a lot more disk space as well, so I plan to stop AQ from deleting the high-resolution photos from the photo albums. At least until disk space becomes more problematic again. Do not expect that the high-resolution photos will always be stored forever, though. This new server has about 3-4 times more disk space available and I don't really have 3-4 times more data to store, so I'll just allow the high-resolution photos to linger longer than before.

And thanks to all of you premium members (and LbNA donors) who pay for all these expenses! I actually started the premium memberships when AQ was on a shared server, fearful that I'd have to move to a dedicated server and I didn't have the money to afford one. Turned out, I never did upgrade to a dedicated server once I learned about the existence of VPSes (Virtual Private Servers), which basically acts like a dedicated server but is actually hosted on a server that shares resources with other websites and accounts. (And theoretically, one of those running a particularly slow query could cause other sites like AQ to be more sluggish as well, but I have no idea what the other websites are or how well they run.) The extra money premium memberships generated beyond the cost of the VPS wound up being my income since a VPS was far cheaper to run than a dedicated server.

I've upgraded the VPS a number of times over the years, increasing the memory and disk space along the way, so it's not nearly as cheap as when I first moved to the VPS, but it feels like this website has really come full circle now that it's more economical to run on a dedicated server than a VPS! At long last, the original reason I started the premium membership--to pay for a dedicated server--is finally coming true. =)

In completely unrelated news.... you'll still find eclipse glasses in the AQ Marketplace. If you want to watch the eclipse on October 14th, be sure to order them while there's still time! At no point anywhere in will it be safe to directly view the eclipse without eye protection and everyone in the contiguous United States can see it (weather permitting). And every time there's an eclipse, there are always stories about price-gouging by desperate people in the days leading up to the eclipse. Be prepared! (I won't price-gouge you if you wait until the last minute--I just won't sell them at all because I can't be certain that they would arrive to you in time.)

Happy trails! And sorry in advance if the downtime next week causes you any inconvenience!

Make sure you have your eclipse glasses before October 14th!

Please accept an invasion into your privacy....

Big brother is watching! Or is he...?

The title of this post is this very tongue-in-cheek. You might have noticed that AQ will now "sometimes" display a pop-up window like every single other website on the Internet about your privacy settings. For the most part, you can blame Europe for this since those are mostly about following European laws. As an American business primarily doing business with North American consumers, I've generally ignored it. I'm not even entirely sure if it applies to all websites or just for companies of a certain size. (Seems kind of like a waste to go after individuals who happen to be running a small, personal website.)

However.... I use Google AdSense to display Google ads (at least for you non-premium members out there!), and Google is going to require that websites have these pop-ups to comply with European laws or they'll stop serving ads completely. Thus.... I had them activated through my Google AdSense account.

You can actually deny consent--that just means Google can't serve ads that they believe are tailored specifically to you. The website will still work, however, even though AQ uses cookies to remember who you are. The consent form is more about "third-party cookies" (or even "second-party cookies"?). In any case, none of it applies to cookies that AQ uses just to remember who you are from page to page.

If you are a premium member and are set to auto-login whenever you visit the site, I don't think you'll see the consent form at all. It's tied to the Google Ads and premium members don't see Google Ads, so the AdSense code doesn't go into pages for premium members. And if you've clicked the button to auto-login, there won't be a chance for Google Ads to show up before you log into your account. So premium members set to auto-login may never see the consent form. It's new to me too, though, so maybe I'm wrong about that. Google Is Everywhere. They're probably watching you read this post right now. (They own Blogger, which is what this blog uses.) Maybe using a Chrome browser (brought to you by Google) will cause that consent form to pop up? I don't really know.

It might even be possible that the consent form pops up after already clicking through several pages on AQ since there are not Google Ads on every page. If you go to the main AQ home page at https://www.atlasquest.com -- where there are no Google Ads -- you could run a search for letterboxes then get the pop-up consent because the search results have a Google Ad at the bottom of the page. It won't necessarily pop up on the first page you see on AQ because of that.

I've also enabled these consent forms for all my other websites that display Google Ads such as Walking 4 Fun and The Soda Can Stove . Letterboxing.org doesn't use Google Ads so it'll be spared the consent form annoyance for the time being. (I don't have any plans to add Google Ads to that website, but who knows how privacy laws in the future will develop and even though I don't plan to add Google Ads to letterboxing.org, I'm not making any promises that it'll never happen. That website does need to at least pay for itself.)

But in any case, nothing on AQ has really changed because of the consent form. Nothing in the privacy policy has changed, I'm not out selling your email address, or sharing private information in your AQ account to Google or anything. It's all about just making sure the Google Ads will continue to work for non-premium members. Gotta keep the lights on somehow! I'd just as soon prefer premium memberships than messing with advertisements, but for those who can't afford a premium membership or prefer not to for some reason, Google ads are the alternative way to help support Atlas Quest financially.

One of the unsung perks of premium membership.... more privacy for you! =)

Click bait here! You won't believe what happens next!

Photo taken by Thunderbird for
the 2012 annular eclipse

As some of you might know, there is a looming crisis. An solar eclipse is headed to America. It's expected to hit the shores United States on October 14th, and nobody will be spared from its wrath. Well, perhaps those in Alaska or Hawaii... but for everyone else.... Millions of people around the country will start looking at the sky--even looking directly at the sun. Without proper eye protection, this can lead to permanent blindness and even death. Well, okay, maybe not death, but I said this was click-bait, so I have to exaggerate and make false claims. But permanent blindness is certainly possible. You may have noticed--the sun is pretty bright.

This is not just any average eclipse, either. No..... This will be an annular eclipse. The proverbial Ring of Fire. The moon, alas, will be too small to completely cover the moon so this isn't technically a total solar eclipse--however  the moon will cross directly over the middle of the sun so at its peak, you'll see a ring of fire completely surrounding the moon.

While the eclipse will be visible (weather permitting) to everyone in the contiguous United States, not everyone will be able to see the ring of fire. You'll need to be on a thin line, maybe a 100 miles wide, that slashes across the western United States in order to see that particular phenomena. It'll run through Oregon, touch the corners of California and Idaho, dive deep into Nevada and Utah, clip the corners of Arizona and Colorado before cutting through the heart of New Mexico and deep into Texas. It'll continue onward, running through Central and South America, but I'm assuming that there won't be many people from those areas reading this blog.

Outside of that thin line, every location within the contiguous United States will see some sort of partial eclipse. Most of the country can see at least 50% of the sun covered.

With these stylish shades, you can safely watch the solar eclipse happening on October 14th.

 

Do you want to watch this marvelous dance of celestial worlds? Then buy your eclipse shades from the AQ Marketplace now! With these fashionable eclipse glasses, you can safely watch the eclipse directly, perhaps while out letterboxing at the same time. They filter out 100% of the harmful ultraviolet, 100% of the harmful infrared, and 99.999% of intense visible light. They're sold in sets of 3--perfect to share with a friend or two. Or maybe your family. It's up to you! Or sell any extras for a fat profit to your enemies and/or co-workers. 

Come the day of the eclipse, there will probably be plenty of people desperate to acquire these glasses after procrastinating until it was too late and price-gouging is rampant. Don't let this happen to you! Another reason not to procrastinate.... they'll only be in the AQ marketplace while supplies last. Once we run out, that'll probably be the last of them since it's unlikely I could reorder another batch before the eclipse. (I'll also take them out of the AQ Marketplace about a week before the eclipse if any are left so nobody can order them at the last minute. The post office takes time!)

Want to learn more about the upcoming eclipse? Check out NASA's page about October 14, 2023, Solar Eclipse. Or check out this eclipse simulator.

Photo of previous satisfied customers from the 2012 annular eclipse--even if they aren't actually wearing their eclipse glasses in the photo! But they have them! Photo by Makita.