AMCharts vote

So, we found some cool flash-based web chart utilities at amcharts.com. All the UCS programmers have been asked to vote for three of them to be templated and proceduralized for use in web apps. Here are my votes:

Stock: Multiple Data Sets
Pie and Donut: 3D donut
Column and Bar: Columns and line chart mix

Project Honeypot

A couple years ago I signed up for Project Honeypot, which is a distributed network of fake email domains set up to catch spam for research purposes. All I had to do was create a subdomain off of a domain I already had (I didn't use any WOU resources for this) and set it up to point to the Project Honeypot servers, and then forget about it. They don't even need access to my site or anything.

So anyway, I hadn't thought about this in a while, but this morning they sent me a notification that they'd caught their one billionth spam message (which happened to be an IRS phishing scam, in case you're curious.) They also included some statistics (Quoted from their email:)

• Monday is the busiest day of the week for email spam, Saturday is thequietest


• 12:00 (GMT) is the busiest hour of the day for spam, 23:00 (GMT) is the quietest


• Malicious bots have increased at a compound annual growth rate (CAGR) of 378% since Project Honey Pot started


• Over the last five years, you'd have been 9 times more likely to get a phishing message for Chase Bank than Bank of America, however Facebook is rapidly becoming the most phished organization online


• Finland has some of the best computer security in the world, China some of the worst


• It takes the average spammer 2 and a half weeks from when they first harvest your email address to when they send you your first spam message, but that's twice as fast as they were five years ago


• Every time your email address is harvested from a website, you can expect to receive more than 850 spam messages


• Spammers take holidays too: spam volumes drop nearly 21% on Christmas Day and 32% on New Year's Day

You can find lots more here.

So how does the Web really work, anyway?

I'm always fascinated to learn how things work, especially stuff we completely take for granted, like for instance how electricity gets from a power plant to your house. So in the hopes that there are others like me out there, I'm going to describe the inner workings of something most of us take for granted: the World-Wide Web.


Naturally this is going to take more than one post, since I'll try to start from fairly non-technical concepts, and use analogies. Those of you who already know most of this may find the explanations not quite accurate, because I'll leave out a lot of the nitpicky details, especially at first. I don't have an outline in mind, so I can't say exactly how this is going to go, but here's a basic idea of what I'll try to cover:

• Internet 101


• What is a protocol?


• Before the Web was born


• HTTP vs. HTML


• Why are there different browsers?


• What's a URL and how do I read it?


• What is actually happening when I click that link?


• How does the page get to me?


• What if there's a problem?


• How forms work


• Secure connections (HTTPS)


• E-commerce and shopping carts


• Web video


• More security concerns


• What's "Web 2.0"?

Hmmm, OK, just off the top of my head I came up with a lot more than I thought I would. And there's a lot more where that came from! So we'll see how far I get, and how many entries it takes.

Ubuntu Linux update

While I'm on the subject of system updates, here's another I want to talk about: Ubuntu Linux, which is what I have on my Dell Mini 9 netbook. At the end of October Ubuntu officially upgraded from 9.4 to 9.10. (Or, according to their whimsical naming scheme, from "Jaunty Jackalope" to "Karmic Koala". I wonder what will happen when they get to Q?)

Though I was warned against trusting the automatic update process from 9.04 to 9.10, it worked just fine for me. All my data, settings, and apps were still there. (I still backed up my data first, though. Only a fool doesn't back up before a major change.)

While the improvements aren't as dramatic as the Blackberry upgrade, there are some nice ones. Probably the biggest one from my point of view is that the faulty driver for the Mini-9's graphics chipset has been fixed. It's not as dramatic an improvement as I was hoping, but it does make video run more smoothly. Flash is still problematic, as it apparently is for all Linux flavors, but it's a bit better than it was.

Firefox 3.5 is now part of the default system. There are a lot of other apps with new versions as well, like Open Office. The login screen is improved, with some language and accessibility options you can set before login. You can now turn off Bluetooth from the menu bar instead of having to open an app and type in the superuser password. File windows have a slightly more compact arrangement, a new icon set, and multi-tabbing capability. On new installations, the much more efficient Ext4 filesystem is used (but not on upgrades, so I didn't get that.)

I've been using it for a week now, and I haven't seen any problems. I'll let you know if any show up.

Blackberry Storm system update, woo-hoo!

Back when the university cellphone policies were changed, I took advantage of it to get myself a Blackberry Storm. I was tempted to get an iPhone, but ultimately decided against it because of the hoops you have to jump through to install third-party apps. I got spoiled by my old Palm, which had a huge developer community and tons of great apps you could install without worrying about approval from Big Brother. I've given up on the Palm for other reasons, so the Blackberry platform seemed to be the best remaining choice.

I picked up the Storm because I wanted a nice big screen and I've never been a fan of the tiny physical keyboards and trackballs on the other Blackberry models; the Storm seemed like it would be the easiest conversion from the stylus-based operation of the Palm. The device seemed pretty cool from testing a co-worker's newly arrived one. So I plunked down the money and ordered one...



When I actually got the thing, after the initial excitement, I was kind of disappointed. Sure, it was miles ahead of my old Palm in things like web browsing and file storage, but it was also laggy and required frequent reboots because of memory leaks. The on-screen keyboard was really slow, and the camera almost unusable due to a three-second delay between clicking the button and the actual picture being taken. And for me the biggest deal was the poor text editing capabilities; I was so used to being able to quickly jot stuff down on the Palm, easily update notes when I needed to, edit large text documents, work with a nearly full-size add-on keyboard, and so on. In this respect, the Blackberry Storm was nearly crippled. At least it was better than the iPhone, which at the time didn't even have copy and paste!
Clearly, the Storm had been released before it was really ready.
Thus I joined the thousands of Blackberry fans eagerly waiting for an operating system update that would fix these problems. The company was working on one, and I saw various leaked versions and almost installed several of them, but ultimately decided to wait for the official release. When it came a few months ago, it helped, but not really enough. The phone was more usable, required fewer reboots, cut a second off the camera delay, and had slightly better text editing, but was still annoyingly laggy too much of the time.
By then I had my netbook, and just used that for all my documentation and editing needs when I was on the go. The Storm was pretty much just my phone and calendar, unlike my old Palm. I stopped monitoring the Web for more Storm updates, and resigned myself to the situation. After all, I was lucky to have as much as I did, right?
So then late last week I hooked up my storm and out of the blue was told there was a system update ready. I went ahead and did it without paying much attention; I figured it was just a minor bugfix for some program, like I'd seen before. Much to my surprise, it was a full update to the brand spankin' new Version Five OS. It's a huge improvement! The camera now works almost instantly. The keyboard is much faster. Selecting text is no longer a trial-and-error process. The much-improved predictive text function saves a lot of keystrokes. There's flick-scrolling to move quickly through long documents and lists. The new Files app makes it easy to navigate large document trees and preview your files. I'll still have to practice a lot to get my text entry speed up to what I could do with the Palm stylus, but now the effort actually seems worth it.
So I'm actually happy with this phone now. Imagine that.

Adobe Flash security hole

This is sort of scary.

For those not familiar with security terminology, this article states that websites which allow uploading of Flash files are vulnerable to a security hole that lets bad guys run code that has all the security accesses of the webserver combined with those of the unsuspecting person who runs that file.

For instance, an attacker could send a specially coded Flash attachment to their victim in a gmail message. When the victim loads the attachment, it gets to do anything the gmail server could do with the victim's account; reset the password, delete messages, send messages (spam!), etc.

The scariest part is that there's not really a fix without significantly changing the way Flash works behind the scenes. In the meantime, you should avoid flash that isn't directly provided by the website you're going to. For instance, the Flash slideshow on the WOU homepage is OK because we wrote it, but if you go to somebody's personal website like "http://www.wou.edu/~joeblow" then you should be careful unless you personally know that Joe Blow isn't the kind of person to play nasty tricks.

Actually that's not really the best example, because even if Joe Blow has one of these malicious Flash files on his webspace on our server, it wouldn't profit him much because there's nothing much our webserver can do other than show you web pages. The WOUPortal and the Sun Java Email system are on separate servers, so they wouldn't be vulnerable to Joe Blow's attack. Of course, Joe Blow could send you a Flash attachment in an email, and if you open it in the Java email system, it could do nasty things to your email account.

This security hole isn't easy to exploit, but it is theoretically possible. I recommend limiting the Flash files you run on the Web; there are browser extensions to help you do that. If you use Firefox, an extension called NoScript can block Flash files (and malicious javascript code as well) on all sites except those you designate as safe. If you use Internet Explorer, you can install Toggle Flash, a toolbar button that lets you turn Flash off and on whenever you want. Instructions for both are available in (ironically enough) a flash video on the page I linked at the top of this entry. Don't worry; Foreground Security is a reputable company, so the video is safe to watch.

OK, let's try this again.

So I got dinged on my performance review for not blogging enough. Justifiably; as you can tell from my archives I haven't hardly made any entries at all for a while.

Anyway, time to start getting more active again.

Change in wou_ldap.vnum_to_uid

Fair warning: this entry will make little or no sense to you unless you work in UCS and do PL/SQL programming.

I've made a change to wou_util.wou_ldap.vnum_to_uid, specifically to the way it deals with V-numbers that are attached to multiple user accounts. Before, if you passed a usertype as the optional second parameter, and it couldn't find a uid matching that type, it would still return a uid if it found one of another type that had the given V-number.

As of today, passing the second parameter will make the function behave more strictly; if a user account of the given type cannot be found, the function will return zero even if there is a user account or another type that has the given V-number.

In other words, passing a usertype to vnum_to_uid() means you want a matching uid only if it also matches the given usertype.

If you only pass a single parameter, the function will behave exactly as before; if multiple accounts are found, it will return the last one found. This is usually the most recently created account, but don't rely on that always being true.

Oh, and one other note: there is a new usertype, "Alumnus". All LDAP accounts of people who have graduated from WOU have this type. It is possible for someone to have both Student and Alumnus, for example if they graduated and then returned for a Masters program.

Air Conditioning FAIL

On Saturday all three air conditioning units in the server room shut down, and the place rapidly turned into an oven. Our servers put out a lot of heat, and have to be kept cool to prevent Bad Things from happening... and so when the air handlers stopped, Bad Things started to happen.

Luckily, only a couple of servers had actual hardware damage, and those didn't have anything critical on them. Several more servers shut down ungracefully or started behaving erratically. Luckily our two biggest servers, cougar and sundown, never actually crashed, but since our main network infrastructure server did, nobody could get to cougar or sundown.

Since I live so close to campus, I got called in, but it was Paul Lambert and Dave Diemer who did most of the heavy lifting. Once the major problems were cleared away, then I could do my thing. Dave was still working on three servers until the next morning, and I was up until really late babysitting the webserver, which seemed to go catatonic every few minutes for no apparent reason. We'll still be cleaning this up for a while.

Continuing the mini theme

My new keyboard got here yesterday and I installed it without much trouble. This mini is far easier to upgrade than any laptop I've ever worked with -- just unscrew two screws, lift the keyboard, pop a couple of little latches and unplug the cable and the old one's out, then reverse the process and the new one's in. The ribbon cable was a little hard to get lined up right, but I got it after a few tries.

The new keyboard is much, much better than the old one. The keys are offset like a standard keyboard, and the punctuation keys are in their normal places rather than shoehorned into odd corners or converted into function-key combinations. My typing speed is way up, even though the keys are slightly narrower. Here are pictures of the old and new keyboards together that someone posted to a forum; the topic includes instructions on how to get and install the keyboard.

I've also been delving more into Linux. Like I said a few posts ago, it's a lot easier than it was in the past -- however, all the geeky stuff is still there under the hood, ready to be poked and prodded and reconfigured. More on that later.

Mini memory

My 2GB memory module arrived today (thanks Joanie!) and I installed it in all of two minutes, one of which was spent finding the right screwdriver. This machine is incredibly easy to upgrade. The keyboard will be nearly as easy to replace as the memory, though there's a couple of persnickety little tabs I'm going to have to be careful with. The keyboard isn't going to arrive until after Memorial day, though.

Oh, and I got VirtualBox installed without any of the finagling Michael had to do on his mini; Ubuntu 9.04 seems to have almost all the prerequisites installed already. Now I just need to figure out how to get a legal Windows CD and a drive that connects via USB-- Dell makes good machines, but even they couldn't squeeze a CD drive into this tiny box. It would have filled half the insides, even without the bigger power supply they'd have to put in.

Did I mention this thing doesn't even have a hard drive? Well, technically it does; it's just a solid-state one, like a USB stick. That means the machine doesn't have to burn a lot of power spinning a stack of metal platters, which in turn means I get over four hours of battery life even with the dinky little four-cell 32WH battery Dell put into the machine. It also means there's no need for a built-in fan, though I'm a little worried about the machine overheating and killing my battery (you do not want to get Lithium-ion batteries hot; leaving one in a car on a summer day can permanently destroy most of its capacity. For more on this see Battery University.) Ive taken to popping out the battery and running on AC only when I have a plug available; probably a bit paranoid, but I like this thing and you won't be able to get batteries for it forever.

I sprung for the extra-big 16GB drive, which may sound small compared to normal drives, isn't even a quarter full even with a full operating system, Open office, and a metric boatload of other programs. Put that in your cache and smoke it, Windows. If I ever start running out of space, there's an SD card slot for more space, plus I can always use some of the metric boatload of USB sticks I've accumulated over the years.

And I guess I'm old, because I remember when it was totally awesome that you could get a hard drive with 20 whole megabytes on it! Like, you could never fill that up for years, man! It was the size of a brick, and weighed about the same as one too. Now a thousand times that much fits on a couple of chips, and seems like not very much room. The eighties were a long time ago, and we live in the future now.

Out of stock, ha!

You know that keyboard Dell told me was out of stock? Yeah, you guessed it, not really. I called their spare parts department (1-800-372-3355) and they found one in under five minutes. That is all.

More on the Mini

I mentioned that I didn't like the keyboard on my mini, and it turns out a lot of mini-9 owners share that feeling. I was looking around on the forums at mydellmini.com last night and found out about a different keyboard you can order from Dell for fifteen bucks. Apparently by shrinking the spacebar and backspace keys by a fair bit, and slightly narrowing the others, they've gotten a much more normal arrangement. I tried to order it, but apparently it's out of stock; they're going to email me when it gets back in.

I did find out about another deal, though; they were selling 2GB memory modules for thirty bucks. Oddly enough, had I ordered my mini originally with 2GB, it would have added $50 to the price, so I grabbed the chance. I want to run Windows XP in a virtual machine on the thing, and that takes a fair chunk of RAM.

Wait, you may say, aren't you running Windows already? Nope, though you can get the Dell Minis with Windows, it's more expensive that way. To get the best price you need to get them with Ubuntu Linux. In case you're not really up on the computer world, Linux is a free operating system (well, technically a group of free operating systems) very similar to Unix, which has been around since the 1970s and is still used on a lot of servers, including many here at WOU.

Linux has been around since the 1990s, but until fairly recently, you had to be a serious computer geek to get much use out of it. The Ubuntu project is one of several efforts to change that, and it's been very successful, combining the many open-source programs and systems to build a variant of Linux that's probably the easiest ever for non-geeks to get into.

It's so easy that when I decided I didn't like the somewhat idiot-proofed version of Ubuntu that came with my Mini, I was able to completely wipe and reinstall it with version 9.04, the latest and greatest, in just a couple of hours. I'm liking 9.04 (AKA "Jaunty Jackalope" in Ubuntu's naming scheme) a lot better than the version I started with, and I only had to fix one little problem for it to work perfectly on my Mini. There are a bunch of very useful instructions available at ubuntumini.com so I didn't have to spend hours hunting around for obscure snippets of information as I did when I tried installing other versions of Linux on other machines in the past.

Anyway, back to work. After a slow few months, I'm starting to feel like I'm getting some programming mojo back, and that feels pretty good. Hopefully things keep looking up, because I'm behind on some stuff that really needs to be finished soon.

Mini laptop

A couple weeks ago, Michael Ellis clued me in on a nice little deal from Dell; through their Faculty/Staff/Student purchase program, they have great prices on their mini laptops, also known as netbooks. (If that link doesn't work, go to www.dell.com/epp and choose Higher Education from the menu.)

Just for being part of WOU, you get a 7% discount at any time, though to take advantage of it you'll need to create a dell login and give them your V-number to prove you're really associated with WOU. They also have $50-off deals that come and go on various systems from week to week; if you don't see the deal on the model you want, wait a few days and look again, and repeat until you do see it. Make sure you're logged in with your dell account, or it might not show you the deals!

You can find their netbooks on this page. I got the Mini-9, and I'm happy with it except for the narrow keyboard which has several keys in odd places. The brand-new Mini-10v is almost the same price, but with a slightly wider screen and a more normal keyboard.

I'll probably be posting more about this thing as the days go by.

Incredibly exciting project update.

It turns out the Moodle course population thing isn't going to be used until summer, but that's OK since it'll be ready to go whenever they need it and meanwhile I can turn to other projects.

Right now the main ones are updating the user account deletion system, automating the resetting of guest user accounts, and setting up a system for automatic creation and changing of email aliases. All very exciting stuff, I know. I'll take a minute so you can stop jumping up and down and get back in your chair.

Seriously, though, even though these kind of things are dull as dishwater to talk about, they are important. Updating the user account deletion process will let us clean out thousands of unused accounts from our servers, and free up a lot of space. This means we can go longer before having to buy more disk space, and it means a time savings for those of us in UCS who administer these accounts; several processes go way too slow because of the sheer number of accounts. Every minute we don't spend waiting on pages to load is a minute we can spend solving problems for you. The guest account reset and email alias update automation will save significant time for some of us, who can then get on with other projects.

So, even though projects like this aren't sexy and cool, they are important. Saving time and money is always a good thing. but even more so right now.

Current projects

OK, I'm trying to get back into this blogging thing. This month my main project has been setting up a system to automatically populate Moodle courses with the students who have registered for them. Currently, if the course has an online component in Moodle, each student has to get an enrollment code from their professor and then sign on to Moodle and enter that code to enroll in the Moodle course.

To save this extra step, I've developed a system that can look at a CRN entered in Moodle, and go to Banner and get the roster of students registered in that course, then create an enrollment record for them in that Moodle course. This was a little hairy to figure out because the Moodle database doesn't just say "this student is enrolled in this course"; there's an abstraction layer I don't fully understand, but I did figure out how to use it to enroll a student.

I'm just waiting for the CRNs to be entered into Moodle, and then I can run the script. I'll set ut up to run every morning for the first few weeks of the term, to catch late adds. I don't yet have a good way to unenroll a student from a course, so the professors are going to have to handle drops on their own.

The ultimate goal is to automatically create a Moodle course component for every course listed in Banner for a given term, but that's still a ways off. I'll need to dissect the process by which Moodle course shells are created, and find a way to do that via a script.

The next major project is to clean up our user database before we move to the new LDAP, email and calendar servers sometime in the next few months. We have several thousand accounts ont he system for students who did not graduate but have not registered for any classes for two years; over the long term, we're going to delete accounts when they reach that point, but the first time we do it, we'll be getting rid of about five years' worth at once.

This will mean cleaning out a lot of disk space, too, at least hopefully. Students already lose access to their files after they leave or graduate, but a lot of that stuff is still on the system. Graduates will still keep their email addresses, as long as they log in every couple years or so, but they won't have an on-campus network login, or any file storage.

I feel like we've been running around putting out fires for so long, it'll be really nice to actually make some progress on something like this.

Oh, and I'm on Facebook now if anybody's into that. Just look for me by name; there are more Swartzendrubers than you might expect, but only one Ron who is listed as Western Oregon staff.

Felt like I got hit by a train

I've been fighting a low-level cold for a couple of weeks, and I thought I had it beaten, but last Thursday it came back and bought its friends. That pretty much shot my weekend, but I'm feeling better now and ready to get caught up on the stuff I got behind on from taking sick time.

Other than that, the main thing I'm working on now is a way to automatically create everybody a Moodle account, so we can link Moodle to the WOUPortal. In case you aren't familiar with Moodle, it's our main tool for online classes, or for adding online content to normal classes. Some of our online class stuff is still on the old WebCT server, but most of it has been moved to Moodle. To check out Moodle, go to http://online.wou.edu.

The next project is to automatically enroll students in the Moodle courses as soon as they register for the class in Wolf Web. I've figured out the basics of how to do this, but the tricky part will be to detect when people drop courses, and un-enroll them in Moodle.

Some progress on the blog server front!

OK, of the four problems I noted last time, I've made progress on three of them. Plus I got permission to delete all empty blogs on the system and stop automatically creating a blog with every new user account. That in itself is going to make a big difference.

Luckily, I have a semi-automated procedure to create new blogs, so if anybody is offended that their empty, unused blog was wiped without their permission, I can recreate it in under a minute. Well, OK, luck has nothing to do with the fact that this procedure exists. It's there because I created it. All these after-midnight workdays have to count for something, you know. (No I'm not whining... late nights mean I get to come in late in the afternoon. Yes, my schedule is weird. Yes, my boss is very generous and forgiving. And yes, it's late at night and I may later regret being so glib.)

Anyway, progress on the specific problems:

• The permissions issue was actually caused by a misconfiguration on the old server that gave it too many rights. The new one is set up correctly (and much more securely) but this means that some old blogs that were set up under the old, too-loose security rules won't work now that things are the way they were supposed to be all along. (No, I will not explain exactly what was wrong and how it's right now, sorry. We can't give out detailed security info.) Anyway, I still need to go in and fix some of the blogs, but the major ones have already been taken care of.


• The style problems happen because the upgrade didn't change the templates on the existing blogs. The company says "User data is sacred and we never change it", which is really just a nice way to spin "We couldn't possibly upgrade the actual contents of your blogs without messing them up really bad." Luckily, I found a way to upgrade the templates on an existing blog; it's been successfully tested on two blogs, and now I need to apply this fix to everybody's blog, except those which were so highly customized that the owner doesn't want their templates converted to the generic MT4 versions. Those people probably aren't going to be applying the canned styles anyway, so this problem won't affect them.


• The random logouts were caused by a subtle error in the code I added to the blog server to make it compatible with the WOUportal single sign-on system. I just found that and fixed it... or at least, it seems to be fixed, because I'm not getting logged out anymore. And, oh yeah, logging into the WOUportal automatically logs you into blog admin, too.


• Then there's the blog stats widget thing. I have no clue here, sorry. Of course, that widget didn't even exist on the old server, so I don't consider it a gigantic tragic loss.

Anyway, back to work....

Blog server update woes

Known problems with our upgrade:

• People with blogs outside their public_html folder may encounter permission errors when rebuilding (eh, they call it "publishing" now) their blog


• If you apply a style to your blog, it will completely mess things up and your blog will look like the computer puked. (That's the technical term, anyway)


• You get randomly logged out when administering your blog


• the blog stats widget doesn't show anything

I have yet to figure out why this is. I can fix the permission errors when they are reported to me, at least. And newly created blogs won't have any problem with styles. I just wish we didn't have 42 million blogs on our system (well, OK, I exaggerate. It;s really a bit over 13,000, of which fewer than 500 have even one entry.)

FeH. OK, I need to get back to working on this thing instead of complaining about it.