Tinkering

I’ve spent much too much of this weekend wrestling with a series of thorny and utterly unnecessary technical problems related to various of my websites. And I’m having a hard time making myself stop and do the things I actually need to be doing this weekend. Like grading. This is in no small part because dealing with these technical problems looks like work without really feeling like it, allowing me to spend hours and hours goofing off while still maintaining the appearance of productivity.

This all began just over a week ago when my hosting provider sent me a message, out of the blue, informing me that my site (this one, in fact) was using up way more than its fair share of CPU resources — this despite being way under my limits for both storage and bandwidth — and that, unless I was able to reign in my site’s processing needs in short order, I would be moved to an “evaluation server,” which sounded to me like a secret eastern European outpost where the Powers That Be would be able to avoid all Geneva Convention regulations with respect to due process and data rights. The message steered me to a new daily report that would be dumped into my stats folder, letting me know where all those resources were going.

The first such report told me that 100% of my CPU seconds were going to the process php.cgi, a piece of information that is quite the opposite of useful, as just about everything here at plannedobsolescence.net — from this site, to the class blogs I’ve hosted here, to the MarxWiki — just about everything runs on a massive conglomeration of PHP. So, like, more specific information seemed in order. But after some back and forth with my hosting provider, whose support folks were at great pains to assure me that the problem was ExpressionEngine, I discovered a few ways to lighten the processing load for this site, which did in fact cut my resource usage by about two-thirds.

This decrease came, however, at the cost of one of the features of EE that I like best: embedded templates. Prior to this week, the header, left sidebar, and right sidebar that appear on this page (as well as many others) were contained in separate templates; apparently, however, a call to the template index, which resulted in many further calls to the header, leftbar, and rightbar templates, was vastly increasing the CPU time required to render the page. Moving the header and sidebars into the index template (and the comments template, and the many other templates that the site uses) cut that usage, but at the cost now that whenever I want to change something in the sidebars, I’ve got to change it in ten places rather than just one. Which is massively annoying.

What my hosting provider’s support folks failed to mention is the following: the reason why the only info I was getting was that php.cgi was getting a bajillion calls a day, rather than any info about the individual scripts calling the process php.cgi, was because my hosting provider defaults to running PHP as CGI, rather than as an Apache module. One can shut this off, I discovered, but only after being warned that boy, we sure hope you know what you’re doing, because you’re setting yourself up for a whole bunch of security problems, not to mention ease of use issues. The good folks at the ExpressionEngine support forums suggested tactfully that I stop being an idiot and shut off PHP as CGI, such that I get proper info. So I held my breath, made the switch, and waited to see if I’d triggered the apocalypse.

I hadn’t. Everything’s running fine. And today’s resource usage report shows that my CPU usage has dropped by another 80%.

Wankers. I want my damned embedded templates back.

Nonetheless, not knowing that such a reduction was coming and on the lookout for further ways to cut my CPU resources, I decided yesterday to move all of my course-related material out of my privately-hosted space here (where it had resided at classes.plannedobsolescence.net) and resituate it in my lovely expansive campus UNIX server space, which I came by a little over a year ago after throwing one fit too many over the college’s completely insane move to microsoftize absolutely everything (windows network, exchange email server, the whole nine yards). It turns out that the CS department here has maintained for some time a separate cluster of UNIX-based machines, that this cluster has expanded to include a bunch of machines operated by several of the science departments, and that the admin for the cluster is a former student of mine who generously agreed to hook me up. I went off the grid as quickly as I could, abandoning the exchange email system, moving as much as I could out of the windows network, and generally looking for ways that I could put this space — in which I could run scripts! in which I had access to perl! — to pedagogical use.

(You laugh. But I began hosting my class materials on this server precisely because several years ago I wanted to run a course blog, using MoveableType, and when I approached the campus computing folks to ask for help in getting that set up, I got nothing but blank, quizzical looks, and questions about why I would want to do such a thing and whether running cgi scripts — cgi scripts! — wouldn’t in fact put the entire campus network at risk.)

So the aggregator blog I’m running via WordPress for one of this semester’s classes is running in my campus space, as is my research wiki. And I figured that once I got around to it, I’d migrate the MarxWiki and the old course blogs to the campus space as well. And, facing a big pile of grading this weekend, I said what the heck and started tinkering instead, figuring that migrating the wiki would only take a couple of hours, given that it’s just a matter of dumping the MySQL database and importing it into a database on the new server.

It turned out to be a little more complicated than that.

First off, just in terms of MySQL, my hosting provider is running 4.1.14-Debian_3-log, and the campus server is running 4.0.24_Debian-10sarge1. I have no idea how many of the problems I faced are accounted for right there, and how many of them have to do with the fact that my hosting provider here is also running phpMyAdmin, and the campus server is Not. So the exporting part was easy-peasy, but importing was nightmarish, with hundreds of syntax errors and access errors and other pesky problems. The good news is that I got a crash course in running MySQL from the command line. The bad news is that I needed a crash course in running MySQL from the command line.

But, after hours — literally, like four of them — I got the database set up and ready to go. I downloaded and installed the latest release of MediaWiki, and set about configuring the thing. Only to discover that the database structure used by MediaWIki had changed radically from 1.3.x to 1.5.x, and that the changes broke the entire site.

So, backtrack: delete the MediaWiki software. Drop the database. Re-create and re-import the database. Install a 1.3.x version of MediaWiki, and configure. Tinker with the database some more to figure out why page updates aren’t properly incrementing, and fix the problem. Make sure everything’s working. Backup.

Download and install a 1.4.x version of MediaWiki, and upgrade. Tinker with the configuration to make sure that everything is working properly. Backup.

Reinstall the 1.5.x version of MediaWiki, and upgrade using the instructions for automatically upgrading via the browser. Get messages suggesting that the database has been updated to the new structure and that all is well. Make sure the configuration is correct, and proceed to the site — where there is no data in any page, at all. Contemplate throwing the computer out of the window, but decide against.

Drop the database. Re-create and re-import the database from the 1.4.x backup. Check the database to make sure that it reimported correctly. Attempt to upgrade the database to the 1.5.x structure using the command-line version of the instructions, but get an error message saying that the database user doesn’t have sufficient privileges to complete the queries. Re-grant all privileges to the user; flush privileges; try again. Upgrade the database to the 1.5.x structure using the command-line version of the instructions. Get messages suggesting that all is well. Run the configuration script via the browser, and proceed to the site — which is now running perfectly, like nothing had ever happened.

Create “we’ve moved!” pages for the old wiki location; update the link to the wiki in the sidebar of all ten ExpressionEngine templates.

That accounts for roughly ten hours of my day yesterday.

So this morning I got up, virtuously deciding that I was really going to work today — grading first; tinkering second! Except that I started poking around, and thought that maybe I could do a little more work on the migration before settling in. Like, maybe, I could redesign my index page on the UNIX server so that it wasn’t quite so appallingly 2001. I decided to appropriate the CSS and layout of my class’s aggregator blog (Machine), which I like quite a lot; so with a little bit of cutting and pasting, and a little bit of tinkering, got the page looking pretty much like this. And can’t for the life of my figure out why the background image for the rap div, which surrounds all of the page’s content, is only repeat-ying about half an inch down from the header before stopping.

I decided to write this ginormous post in no small part so that I would stop tinkering — I really need to get that grading done — hoping both that I’d distract myself from it with writing and that some wonderful reader, far more CSS-literate than I, would take a look and immediately see where the problem lies.

Anyhow, this has been my weekend — utterly pleasant, utterly pointless. On to the real work, now.

[UPDATE, 4.35 pm: Solved. It was a column-height problem, I think; I’m not exactly sure why the change I made worked, but it did. And yes, I did finish grading that batch of papers, thanks for asking.]

5 thoughts on “Tinkering

  1. Do tell how you got the aggregator blog set up and working… are you using something in the background, or is this just a feature of WordPress?

    I’ve been hoping to find an easy aggregator that streams several blogs into one feed, in the order that they’ve been posted (but without saving the whole durn thing (I don’t need double-posts in the database, know what I mean?).

    Tweaking the website is an insidiously effective procrastination method, isn’t it? Which is why my blog still has broken templates from an old version of MT.

  2. Oh, I completely forgot you were looking for this, Jason. I’m using a genius plugin for WordPress called, appropriately, FeedWordPress, which can be got from RadGeek here. The installation is relatively simple, and getting the aggregator blog set up with the appropriate subscriptions is a piece of cake. The only hitch for me was that all my students are using Blogger for their blogs, and Blogger won’t allow you to set up custom pings. So what I had to do was to set up a little PHP prepend that runs the aggregation script every time the index page is loaded. Causes a bit of slow loading, but it works. Assuming your folks are all using MT, though, you can just get everybody to ping the aggregator whenever they update, which will probably be easier…

  3. Does it then save each entry into its own archive, or just show the most recent 10 (or 20, etc)?

    This is me, asking questions first, and then going to read the documentation second. It’s just that sort of day 😉

    Thanks KF!

  4. I think you can set it up multiple ways, if I’m remembering correctly, depending on how you configure WordPress. I know you can page back pretty far through the entries, and there are monthly archives and category archives (useless as my students are not using categories *sigh*), but I’m not sure if it’s setting up individual entry archives or not, as the permalinks are set to redirect the reader back to the original blog that the post came from… Make sense?

  5. I think so… but there’s gotta be an easy way to implement this on a single install of MT, instead of using YAPOS (Yet Another Piece of Software).

    I wonder if Wordherders will give me a research assistant? 😉

    Thanks again KF.

Leave a Reply

Your email address will not be published. Required fields are marked *