The Update is Ready.....

...or at least it will be by tonight. =)

As you might have read from my last post, I woke this morning like I do most mornings, with the sun streaming through the window. I laid around for a half hour, thinking about all the things I had to do. Could I finish my updates to the message boards to upload the changes tonight?

And somewhere in my meandering thoughts, I had an idea. When I created the WikiBoxing feature, I decided to use the wiki-style formatting of text. It's simplier to learn and use than the html-style AQ has always used before, and in the back of my head it's always been a thought that I could update AQ use the wiki-style formatting exclusively. For writing posts, and writing AQ mail, and writing clues, and--well, you get the idea.

Except now that everyone's learned HTML, a lot of people probably would be resistant to learning wiki-style formatting as well. And all the pieces popped into my head at just the right moment: Set it up so people can choose which option to use.

Frankly, I'd rather use one or the other instead of confusing people with two different options, but the wiki-style is so much more flexible, but it seems criminal to rip out the rug from everyone using html-style too. I'm hoping people might start trying the wiki-style formatting, get used to it, and eventually I can retire the html-style option completely. New members on AQ will automatically default to the wiki-style formatting. Old members will automatically default to the html-style. And all members will be able to chose exactly which style of formatting should be applied to a given message board post. The default option can be set as part of your preferences, but you can override the default as necessary.

I'm already mucking around with the mesage board code, so now's as good as time as any to introduce the wiki-style formatting. It was elegantly simple to include, I might add, in the sleek new code I've spent the last week working on. =) That's what sleek new code gets me--the ability to modify and add features quickly and easily.

I did my rounds on Atlas Quest, then ate breakfast, then sat down and added the wiki-style formatting to message board posts. Works beautifully!

Then I start thinking--since I'm already adding it to the message board posts, why not allow it to work for AQ mail as well? Keep things consistant. *nodding* So I worked on allowing that as an option while composing an AQ mail. That wasn't so easy--the "write an AQ mail" code is pretty old and in bad shape, so that took a couple of hours to fix up enough to add the wiki-style formatting option. It's still not great code--I just cleaned it up enough to add what I wanted. Eventually, it'll likely need to be rewritten from scratch someday.

The one other major place where people can enter large quantities of text is the clue. I'd love to support wiki-style formatting for clues, but alas, that will have to wait for another day. The html-sytle formatting has two procedures that go with it: One to convert from html-style text to the display version, and one to convert from the display version back into html-style text. That's how you can edit clues. AQ converts your clues back into the text your originally typed then lets you update.

The wiki-style formatting only has one procedure that turns it into "display ready" text. There's nothing to convert it back into the original text you typed. Once it's converted, it can't be edited anymore. Which is fine for message boards and AQ mail--you can't edit those to begin with. Clues have to be edited.

But OH, you might think--you can edit entries in WikiBoxing. What's THAT about!

Clever thinking, I might add, but there's a trick to that. Interally, every time you edit a wiki entry, AQ stores TWO versions of it in the database. One is your original text and the other is your "display ready" version of the text. When it comes to editing a wiki entry, AQ pulls up the original text and lets you change that. The display version just gets copied over with your edits.

So that's how that works.

I could do that with clues, but that would require significant amounts of rewritting the code that I'm not ready to delve into at the moment. Perhaps someday, but I'd just prefer to create a procedure that allows AQ to convert from display-ready text into wiki-style formatting and use that. But until I create that, clues are stuck with html-style formatting only.

I also fixed-up the reporting of problem posts and set it up so when you try to contact someone to send them AQ mail, Atlas Quest will return you to the page you were last one before you contacted them. (It used to take you to your inbox after the message was sent.) Much more friendly to take people back to where they left off. Details! Details!

*** Technical speak ahead--you might want to skip the next two paragraphs if you don't want your head to explode ***

While testing the new features, I noticed that AQ was remarkably slow (a few seconds) when trying to display a list of messages, but only SOME of the time. I finally determined it was only slow when I wanted it to display messages from just one board. I visually checked the query, and it looked good. I double-checked the indices were correctly there, which also looked good. So I ran an explanation of how the database was processing the query, and it was completely ignoring the one index it should have been using. The database is usually very good about picking an optimal query, but alas, this one seemed to be stumping it. To test my theory that this was the problem, I tweaked the query by explicitly telling it what indices to use, and the query ran blazing fast. YES!

I added that change to the code and now the messages were turning up fast, but only if I looked at messages from a single board. If I tried to look at just a thread or all messages, the query was running slow. Exactly the opposite problem I had before. In a nutshell, I needed to tweak the code to only use my hand-picked indices if AQ was trying to display all messages from a single board, otherwise the database should use whatever it felt was best. This would have a terrible, terrible thing to do with the old message board code. But it was such a simple, elegent fix--just the addition of just one line of code. I commented it heavily--that line of code is VERY important but it's not obvious why. I don't want to inadvertantly delete it later while looking at the code and assuming it's not really important.

Why is this important? The queries run faster. There are nearly 90,000 posts on Atlas Quest now, growing by two or three hundred more every single day. Think about that for a minute. You click on a link that says something to the effect: "Show me every single post from the Yakking It Up board posted since 9:30 last night (the last time you read the board)." Atlas Quest has to search through nearly 90,000 posts, figure out which ones are on that specific board, which of those posts were posted after 9:30 last night, which of them were posted by people on your ignore list (and therefore should NOT be displayed), which of them were deleted (and therefore should NOT be displayed--except if you're an admin who needs to undelete it), and then pull up each of those messages. Then it needs to look up the name of each person who posted one of those messages, information about them like PFX counts that show up by their name. And it needs to do all that in about 1/10 of a second. (Faster, if I can, but alas, that's about as fast as I've been able to get it working.)

That's pretty darned amazing. How long do you think it would take YOU to do all that? =) The message boards are the single most often used feature on Atlas Quest, with tens of thousands of message board pages being displayed every single day. During the last few months, Atlas Quest been been bogging down a bit under the strain, and I think that one little line of code could help improve that particular query by a factor of 10. A full TEN times faster than without that line. Perhaps it's the one thing I needed to get in there to keep AQ running fast and efficient for another year without having to upgrade again? I don't know, but it'll certainly help! Upgrading my hosting account to the next 'level' would cost about $500/year. That single line of code may have saved me $500. Now you see why I'm so happy about it? =) And also why I want to make SURE I don't unwittingly delete it in the future thinking it's not important.

But you won't see a visual change because of that. AQ will do what it always has, just a little bit faster.

There will be other visual changes in this update, though. Mostly minor ones, but a whole lot of new code backing it up.

The one problem left (that I know about, at least) is the forwarding of AQ mail. I expect to get that working correctly soon, though, and once it is all these updates will be ready for the live site. With all this new code, there likely will be bugs in it I have not yet discovered. I'm sure you all will help me root them out and squash them as they come into the light, though. =)

As such, I've set AQ to close down at midnight tonight, Pacific time. I'll be uploading all these changes then.

I'll take a nap later this afternoon--the better to stay up until an ungodly hour of the morning updating the site when you are all asleep in bed. =)

As far as jobs go, working on AQ isn't half bad. It's the only time I've ever been able to justify a nap as "working." =) Otherwise, I'd have to take AQ down at more inconvenient hours when everyone is awake and online.

So there's my little speach for today, and some of the things you can expect in the Next Big Update.