Geeky stuff about the blog server's database

Who knows how many of you will find this interesting, but I'll talk about it anyway. I've been delving into the database that holds all the blog server's data - all the blogs, their settings, entries, comments, all the users, security permissions, and all that.



The designers of Movable Type did a good job with their database; everything is efficiently linked together and easy to understand. If I want a count of all unapproved comments on the system? One simple SQL query, bam, it's done. Listing of all blogs ranked by the number of entries? slightly more complicated query, bam, done. Add myself to the permissions list for an old blog with an insane number of comments? Simple SQL insert, bam, I'm in. Hmmm, those comments are all spam except for ones from WOU IP addresses? Simple SQL delete statement, double-check it to make totally sure it's only going to wipe the ones that need it, bam, done. Say good bye to lots of spam that was cluttering the system; when I started this, there were over 25,000 comments in the database, but after checking them out, I found I could delete all but 3000 or so.
Oddly enough, all the spam was concentrated in about 15 blogs. I cleaned out all of them that didn't seem to currently be in use, but there are a couple that I didn't touch because people had been updating them recently.
I'm thinking about setting up some webpages to report blog use; now that I have the database figured, I can easily make a page to display links to the ten most recent blog entries, or list the most active blogs, or whatever. Now I need to write up this week's FAQ, which will talk about some of the new features on the blog server; since hardly anybody has asked me any actual questions, it'll probably be a short one. I'll intro the features I think are most interesting and/or useful and leave it at that.

Ahhh, junk filter!

Wow, in just the few days the new version has been up, I've already seen a major reduction in the amount of spam comments I get. It used to be a few hundred a week, but in the last couple of days I've gotten a grand total of thirteen, eleven of which were automatically sent to my junk list. Looks like I may need to tinker with my junk filter a little bit in order to catch ones like those last two, though.

I'm not sure what happened to the other eighty-some spam comments I was expecting, though. My junk filter isn't set to actually delete anything, so where did they go? It would be too much of a coincidence that a bunch of spammers just happened to cut down their blog spam right exactly when we upgraded, so it must have something to do with the upgrade itself.

I wonder if they can detect the version of a Movable Type server before spamming it, and don't send as many to recent versions that have the anti-spam features? Though that's probably doubtful, considering that all along we had the nofollow plugin that makes spam links useless for improving google rankings, yet the spammers kept on firehosing us. It's probably more effort for them to target their spam in any way than simply to spew it out it blindly.

But whatever the reason, the amount of spam on my blog is way, way down. Now if only I could do that to my email...

Blog server upgrade!

OK, our blog server is now up to the latest version of Movable Type. You'll notice a lot of new features, though luckily the basic operations of the admin system are still located where they've always been. There's just a lot of new stuff added: a better search function, improved handling of large lists, defaults for new posts, and my personal favorite: junk filters for comments and trackbacks!

I'll cover all this stuff in more detail as part of this Friday's FAQ. Now I have to go in and test some more stuff to make sure everything is working.

Contribute is fixed

OK, I take back everything I said about Adobe's tech support after the first time the Contribute server failed on us. What? You say you didn't hear what I said about them? That's probably a good thing, considering some of it was unprintable.

Anyway, this time I actually got a competent support engineer, and I didn't even have to battle my way up the hierarchy for more than a few hours (let alone three days like last time, grumble grumble.)

What impressed me the most about her was not just that she knew what she was talking about, but that she kept after the problem. We had one possible solution that would have taken a couple of days to implement, and instead of just sitting back and waiting, she kept working on our issue. That floored me; maybe I've just had bad luck with corporate tech support, but I've never yet dealt with someone who would do that.

Anyway, she came up with a simple solution that I hadn't tried because it didn't seem to make any sense... but it worked like a charm. You may have noticed that the login delay is now much shorter, too.

So, tech support saved the day!

FAQ: new WOUPortal!

Hello, everybody--

This week we're announcing the beta version of the WOUPortal! It's available for use by anyone in the WOU community.

What's the WOUPortal?

It is a web portal, which is a customizable page designed to make many web resources available in one place. The home pages of yahoo.com and msn.com are good examples of portals; they include lots of information, links and resources, and if you log in, you can customize them to show what you want, and hide what you don't want. The WOU Portal is available to anyone with a WOU email address, and just like other portals is customizable so you can see the links and information you find most useful. The goal is to make so many useful features available here that you'll want to use it as your browser's start page.

How do I get there?

The address is http://www.wou.edu/portal. It will take you to the login screen; use your email username and password to get in. If you can't log in, or you don't remember your password, a link to the account lookup system is provided. The first time you log in, you will see a short explanation of the different parts of the portal; on later logins, you won't see the page automatically but you can get back to it by clicking "WOUPortal Help" in the Programs for Everyone section. Click OK at the bottom of the help screen to go to the main portal page.

What are the different sections for?

On the main page there are three main sections: My Programs, My Connections, and Programs for Everyone. There is also a My Preferences tab, but I'll talk about that separately.

The My Programs section is for programs that you have special access to. If you're part of a group that uses a portal-enabled program that isn't available for everyone, you'll see a link to that program here. For now, most of you won't have anything in this section.

The My Connections section is for links; through the My Preferences tab (see below) you can add any link you want here. It also includes space for portal-enabled tools; right now we have a dictionary lookup, a weather report, a language translator, and a map search. There won't be anything here when you first log in, but you can add links and tools by customizing the WOUPortal.

The Programs for Everyone section shows options that are available to everyone, which currently is the WOUPortal help and the communication log. If you have enabled the advanced features (see below) you'll also see links to your email, calendar and other services.

What is the communication log?

The communications log is designed to make it easy to get information out to a group of people. Anybody can set up a group and give them messages, which they will see when they check the communication log. You will be able to see how many of the recipients have read each message. Currently, this is used by several sports groups, plus student workers and their supervisors, but anyone can start a group and use it for any purpose.

How do I customize the WOUPortal?

Click the My Preferences tab at the top of the page. In the top section, you can change your nickname and enable advanced features. There is a brief explanation of the advanced features here, but I'll cover them in more detail below.

In the next part of the preferences page, you can add and delete the links that show up in the My Connections section of the main WOUPortal page.

In the third part, you can hide or show items in the My programs section, and enable or disable any of the four tools currently available. Note that it says that "My Feeds" is coming soon; this feature will let you subscribe to RSS feeds, which let you view things like news and blogs directly in the portal without having to go to other pages.

When you're in the Preferences tab, you have an additional tab available; you can use it to give feedback about the portal. Praise, criticism, feature requests... whatever you want to say.

So what about these advanced features you keep mentioning?

If you have enabled the advanced features, the Programs for Everyone section will include links to your email and calendar pages in Communications Express, and to your Banner Web page (either Web for Students or Web for Employees, depending on which you are) and your blog. The WOU forums server will soon be connected as well. The really cool thing here is that once you have logged in to the WOUPortal, you can use these links without needing to log in to each service separately. Just don't forget to log out of the portal when you are done.

When I enabled advanced features I got a warning about not holding WOU responsible for data loss? What's up with that?

Because of the way the WOUPortal logs you into external services like Communications Express and the blog server, you need to specifically log out when you are done. Even if you quit the browser completely, someone else could still open it up again, go to Communications Express, and read your email.

This can be completely prevented by logging out of the portal, so it is crucial that you do that every time; that warning message is there to remind you that the advanced features really are optional, and that you agree not to hold WOU responsible if you forget to log out and somebody does something bad with your account. Personally, I do use the advanced features; I believe the extra convenience of not having to log in multiple times outweighs the (very small) risk involved.

Wasn't there another FAQ about a portal last year or something?

There was. But that portal was never widely used, and was not as flexible as we wanted, so we made our own. Michael Ellis did most of the heavy lifting; if you like the portal. he's the one to thank, and if you want changes, use the Give Feedback tab in the Preferences, and he'll get the message.

You said this is a beta version of the WOUPortal?

That's right; it's a work in progress. However, it is a very stable beta version. Some people, notably Housing and Dining staff and student workers as well as students in the residence halls, have been using it for months already. We expect to add new features and programs every few weeks. The visual design is also open to change.

That's all for this week; remember to use the "Give Feedback" tab in the WOUPortal preferences to send any kudos, comments, questions, or concerns about the portal. Feel free to contact me at faq@wou.edu as well.

Thanks as ever for your time and attention,

----Ron

Contribute woes

The Contribute Publishing Server is down again. This is really frustrating; it was running great for months and then bam, two major failures since the beginning of February. Nobody can log in to the system at all.

Last time I had to reinstall the whole system and recreate all the site settings by hand (a tedious process for thirty sites) because when I tried restoring the backed-up settings the problem came right back. This time, even that doesn't work; the problem occurs even on a totally fresh install with no sites defined.

I was able to demonstrate this to one of Adobe's senior tech support engineers, and she confirmed this was a previously unknown program bug and submitted it to the CPS developers. They are supposed to get back to me within one business day, but it's now been longer than that, so tomorrow morning it's time to start pestering them again.

It's frustrating because I still think this is a good product... at least it makes my job as an administrator much easier when it it working correctly. We'll see how things go, I guess.

FAQ: P2P and the RIAA

Hello, everyone--

Since last week's FAQ on peer-to-peer (P2P) file sharing, we've gotten some new information. The Recording Industry Association of America (RIAA) is beginning a crackdown on illegal file sharing at universities, and we'd like to pass that warning along to you, as well as tell you about an alternative source of free and legal music for students.

Who is the RIAA, anyway?

The Recording Industry Association of America is a group that includes all of the major music recording companies, representing well over a thousand recording labels. In recent years they have strongly focused on preventing illegal distribution of music, and one result of this is lawsuits against people found to be illegally sharing copyrighted music through peer-to-peer software. You can see their website at http://www.riaa.com/. Note that a similar organization, the Motion Picture Association of America, fills the same role for the movie industry, and has been suing people who illegally share movies.

So what exactly does this "crackdown" involve?

We have been warned by the state Attorney General's office that the RIAA is beginning an extensive new program of lawsuits targeting students. They intend to pursue hundreds of cases per month nationwide. You can read more about this program on the RIAA website, and we are also making available a copy of an explanatory letter sent to all university presidents; find it at http://www.wou.edu/ucs/faq/documents/RIAA_University_ltr.pdf. As part of this program, students found illegally sharing music will be offered a chance to pay a settlement fee to avoid the lawsuit.

How does that settlement work?

As mentioned in the previous P2P FAQ, the RIAA will not know exactly who has been sharing their copyrighted files, but they will be able to track the illegal activity to WOU's network and get the IP address of the computer from which the files are shared. Under their old policies, they would then send us a takedown notice, and we would act on it because WOU as a whole could be sued if we did not.

But now these warnings will also come with a pre-lawsuit letter; you can see an example at http://www.wou.edu/ucs/faq/documents/RIAA_pre_lawsuit_ltr.pdf. (Note that the example letter was received at another institution; WOU has not yet received any of these letters, but believe it is only a matter of time until we do.) We will forward this letter to the individual who owns the computer whose address is given in the takedown notice. If that person chooses to accept the settlement offer, they must contact the RIAA as stated in the letter, and will be given further instructions. We will not give any student's identity to the RIAA unless legally compelled, but this may well happen if a lawsuit is filed.

If someone gets one of these letters, should they accept the settlement offer?

We cannot make a recommendation on this matter, other than that the recipient of the letter should consult an attorney, or at least educate themselves as much as possible, before deciding. Please be aware that there is a lot of contradictory information about RIAA file sharing lawsuits, and many sources are biased one way or another, so it is difficult to find reliable advice online. Wikipedia's entry on the RIAA at http://en.wikipedia.org/wiki/RIAA is a good starting point. Whether or not a student accepts the settlement offer, WOU will continue to follow our current judicial policy, including temporarily disconnecting that student's residence hall network access.

You said something about a new source of free and legal music?

We are looking into a program called Ruckus, which offers a catalog of over two million songs free to any current student with a valid .edu email address. They make money though advertisements, and by charging fees for additional services and to non-student users. You can take a look at them yourself at http://www.ruckus.com. We are investigating the possibility of getting a local server for their catalog, which will make download times faster and grant access to some additional material such as movies, but any student can use their service even if WOU does not make such a deal with the company.

That's all for this week. Please feel free to send me any questions about this topic, or anything else you'd like to see addressed in an FAQ, at faq@wou.edu.

Thanks once again for your time and attention,

----Ron

FAQ: Peer-to-Peer (P2P) file sharing

Hello, everyone--

This week's topic is peer-to-peer (P2P) file sharing. It's been in the media a lot recently, and unfortunately there's a lot of misunderstanding about it, and about WOU's policies towards it.

What is peer-to-peer file sharing?

Every computer running a P2P file sharing program (such as LimeWire, BitTorrent, or iMesh) acts as a server so others using the same program can download files from it. It also lets the user search for and download files being shared by other computers running the same software. This is a benefit when legal files are being shared, since there doesn't have to be an expensive central server that everybody downloads from. But if the shared files are illegal, this is a drawback, because there is no single source to shut down.

Is P2P file sharing bad?

There has been a lot of hype in the media about P2P file sharing being bad. It's not so simple, though; it's only wrong when you share stuff that you don't have the rights to. We do not ban P2P software on campus, because there are legitimate uses for it; think of it like owning a gun. Guns are not banned simply because people might use them to commit crimes; only the crimes themselves are banned. The entertainment industry is very much against P2P file sharing, because as copyright holders, they are the main victims of illegally sharing files. However, musicians and actors are also hurt by illegal sharing; every album or movie that is illegally downloaded instead of bought means fewer royalties for them. On the other hand, many amateur musicians embrace P2P file sharing; by releasing their songs to be legally shared, they hope to gain popularity. Some software companies also make use of P2P to share free or trial versions of their products, so they don't have to maintain expensive centralized servers.

How do I know if I'm allowed to download or share something?

If you created the file yourself and have not sold the rights to it, it belongs to you and you can share it as much as you want. If a file is in the public domain, or the owner has given clear permission to share it, it is also OK. But if you cannot find any legitimate notice that a file is OK to share, then you should not download it or share it with others, because that would be stealing.

Why is UCS concerned about P2P file sharing?

We care about file sharing for two main reasons: bandwidth and legality. Since p2p programs are mostly used for music and video, which have large file sizes, they tend to take up more than their share of our Internet bandwidth. If people all over the world are downloading the latest blockbuster movie from a computer in our residence halls, that adds up to a lot of traffic, and other people on campus may have trouble doing legitimate academic work. This is true whether or not the files being shared are legal or illegal.

Why does UCS care about the legality of file sharing? Isn't that solely the responsibility of the person sharing the files?

When copyrighted files are shared from a computer on campus, the owner of the copyright usually can't identify who owns that computer. But they can easily discover that it is coming from someplace on our campus, so they often come to us with their complaints. According to the law, if we have been warned of illegal file sharing on our network, we are liable for criminal charges if we don't take action. These complaints average about once a month, but sometimes they come in groups; in one single day last year, we received complaints about ten different people illegally sharing files. The rate is increasing each year.

What does UCS do when you get a complaint?

Using our internal records, we are usually able to trace the owner of the computer where the files were being shared. If that person is in the Residence Halls, they will lose their Internet access for ninety days, and will be removed from the halls entirely on the second proven offense. If you have any questions or complaints about this policy, please contact Residential computing at (503)838-9201.

I was running LimeWire and Residential Computing uninstalled it when I took my computer in for help! Why is that?

LimeWire is one of the most popular P2P programs currently out there. Unfortunately, we have frequently seen it install spyware and adware. We have also seen it continue to run even after supposedly being uninstalled. When we see LimeWire on a computer, it will be deleted. Not all P2P file sharing programs include spyware, though; we won't automatically install them unless we get confirmation that they are infecting your computer with bad stuff.

That's all for this week! Thanks once again for your time and attention.

----Ron