I am what you would call a “professional student”; I have an Bachelors of Science in Computer Science, I am finishing a Masters of Science in Computer Science with a focus in Intelligent Systems, contemplating a PhD, and I have been a software engineer for 6 years now.

Recently, Joel Spolsky published yet another article about how he feels the universities of the world are churning out students incapable of doing the daily duties of software development. I’ve read other scathing articles about academia. I’ve even responded to many comments similar to “you have a degree but don’t know how to use ToolX or program in LanguageY.” These criticisms always irritate me (and strike me as originating from someone who begrudges those with degrees), so I want to set the record straight about academia. There are 2 simple points I want to get across:

1. The university’s primary concern is to teach you core knowledge and how to obtain new knowledge in any field.
2. Computer Science is a division of Applied Mathematics.

It’s that simple. At no point is it the university’s responsibility to teach students an arbitrary tool or language that the industry is consistant in its opinion. I read this really great response to his article that echos many of my complaints about this misconception.

Joel often comments that universities are trying to teach a particular language because it’s what the industry does or because MIT does it. That is wholly incorrect. The only reason why a university favors a particular language is so that the professors can focus on teaching towards and grading just one language, as that greatly simplifies their job. The choice of Java or Python is because you can express ideas simply and straight forward. The point isn’t to teach a language but to teach an idea expressed in a language.

If my undergraduate university had taught me specifically how to use CVS, that skill would essentially be wasted. Instead, they teach how versioning control systems work so that I may either implement one or just use one in my day to day job. Which sounds like a better idea in the long run?

Now keep in mind that CS is really just a division of Applied Mathematics. If you haven’t come to understand that, then you do not truly understand the field. In fact, the original “computers” were humans who computed mathematic equations.

Sure, most undergraduate assignments seem simple in comparison, but it’s because they don’t want to teach the peripheral tasks. Those tasks, such as testing, working in teams, and the latest “agile” techniques, are unrelated to the core understanding and vary widely within the industry. By understanding the core concepts, everything else is an extension of your existing knowledge.

If my university had taught me how to use FogBugz or how to write Perl TAP tests, I would have been looking for another university. My graduate school has yet to require I use a language or a tool and has yet teach a specific language and a tool. In the long run, that makes me a more adaptable developer and far more valuable to my employer.

So this semester I have been investigating and working on my thesis. Right now, my focus is in Statistical Natural Language Processing. I don’t want to discuss the specifics of the research just yet, but it has the potential of completely up-ending the entire search industry.

I have been investigating how to build a large corpus from the web. My advisor favors using Google directly since they already exist and provide their search for free.

The first thing I did was investigate the Google SOAP API only to find out that they deprecated it when they introduced the AJAX API. The new API only allows for about 60 results with no paging. Then I looked into the REST::Google API, but that only returns 10 results. Neither of those options seem feasible. I checked Yahoo’s Yahoo::Search interface and it only seemed to return 10 results (paging, if possible, was not obvious). I could write a direct scraper but that would take a good deal of effort and I am not sure it would be worth it.

Then I even started looking at writing my own spider using WWW::Robot. This is a fairly complex module that does a ton of grunt work for you. The downside is that it behaves and follows the robots.txt protocol; that’s a problem for someone who wants to scrape everything with no regard for such a protocol.

I spent maybe about 20-30 hours flipping over this in the last 6 weeks, I finally made the effort to meet with my advisor. Since he is no longer answering email or his phone, I met him after his late class and talked it over with him while he ate dinner in the campus restaurant. We talked and waffled back and forth about our approach. In the end, we decided to investigate Lucene’s capabilities.

Frustrated and lost, I went about my week until I talked with a PhD student currently being advised by my advisor as well. Her patience for our advisor has been continually declining. She missing a publication deadline because he failed to review a paper of hers. She also divulged that she intended on switching advisors because she is not making progress. I have been contemplaing this myself, so it was good to hear that I am not the only one at their wits’ end.

I am not making progress and I am not willing to sacrifice my graduation. If I change advisors, hopefully I will find an advisor that provides much more support and direction yet gives me the option to continue developing in perl. One of the professors I want to speak with runs a programming language lab.

Maybe I can merge my interest with Perl 6 with my thesis!

Earlier this week, I was hanging out in the #corehackers channel and I decided to voice my opinion on the flood of discussions concerning the problems plaguing Perl 5. David Golden challenged me to respond to his post concerning what I want from Perl 5 moving forward.

First, my suggestion to chromatic was that new features should receive first class citizenship over deprecations. Since I prescribe to the technology-over-politics philosophy, I think deprecated features should be removed after 2 releases. Once removed, they should be provided within a library. I don’t feel those aging applications depending on these features need to be intentionally broken, but at some point, features that have been deprecated for 10+ years need to be removed. Keeping them only makes the deprecation problem harder every release they remain.

So what would I like to see out of Perl 5 in the future? Perl 5. It’s simple, I just want to see Perl 5 continue to thrive until there is a complete Perl 6 interpreter (it’s been 9 years, may very well be another 9). Frequent releases have the effect of convincing people that there is an active developer community. When application developers feel there is an active community, they are more likely to consider the platform viable. Application developers do not seek dead platforms for The Next Big Thing.

There are many things Perl 5 pumpkings and developers can do to achieve this. Frequent releases are a must; it doesn’t matter if 1 or 1000 tickets were closed since the last release, getting it out the door is the most important thing. Worrying about 100% backwards compatibility in every release means releases don’t happen. Just look at 5.10.1. 18 months for a minor version upgrade? How many pumpkings has 5.10 had anyways? It’s time to move on and to stop worrying about breaking all of DarkPAN.

Features that extend the expressiveness and flexibility of Perl 5 need the highest priority. The grammars in Perl 6 mean that Perl 6 can eventually host Perl 7. That feature alone will keep Perl 6 alive years beyond any other language.

In the end, bickering about release schedules, deprecation policies, dependency problems, or any other issue is not going to get Perl out the door. At the end of the day, there are technical and social issues to work through. Social issues are hard; people don’t change their minds easily. Let’s just agree to solve the technical issues, which are significantly easier. Scheduling releases, fixing segfaults, and updating modules are all just technical problems that all have solutions. They may not be pretty or quick to fix but solutions exist, and in most cases, we already know what’s needed but are too afraid to move forward. Sure, breaking 10 year old Perl 5 scripts will be painful but no where near as painful as watching Perl 5 die a slow and miserable death where everyone involved bad-mouths it until it actually is dead.

Finding motivation lately has been more challenging than usual. My posts about perl have grown further apart. I have mostly slowed down working on my pet projects. I haven’t been reading my Perl related blogs anymore. Even writing this entry seems forced and unnatural. I hope this post is not picked up by the Ironman Perl planet due to the complete lack of any substantial content.

Don’t be mistaken, I am still interested in all of these things. I have a stack of interesting modules that I want to clean up and release. My feed reader is chock full Ironman Perl entries. I have 2 unfinished posts about Perl 5.

There are several reasons for my recent sloth. The summer is drawing to a near. I have a large list of honey-do items that need to be done around the house. Worst of all, I am working lots of overtime for some projects that are perpetually behind schedule and over-budget.

On the positive side, I have spent a majority of my summer learning about compilers, programming languages, and parsing grammars. I was really taken by the Topaz project, which has been a source of inspiration. I was also very impressed with the PGE presentation and discussion at the Parrot Virtual Machine Workshop.

I have exceeded my ability to maintain my attention about my own blog post. I’d say more about my next post but I just want to stop typing.

It’s no secret that many technology circles are boys’ clubs. There’s something about the industry that seems undesirable to women. For the longest time, it eluded me as to why anyone would not love technology.

Then Matt Aimonetti and Hoss Gifford proved why it’s a culture that is generally unwelcoming to women. I won’t bother linking to them or any of the stories; you’ll have to Google it yourself to find out.

These two cartoons presented at a developers’ convention and utterly degraded the entire profession in the span of an hour or less. First, Matt mad unclever and borderline offensive slides that urged developers to “perform like porn stars.” Not to be outdone, Hoss decided to put a most definitely not safe for work image directly in his slides in a recent conference, and even acted like a prepubescent boy during the presentation. Again, you’ll have to Google it for the video.

If software engineers ever hope to be taken serious by the traditional professions, such as doctors and lawyers, this kind of crap has to stop.

Much to my dismay, Kirrily Robert decided to include an image link directly in her feed that is aggregated on Planet Perl. The result is I am reading my daily dose of software related blogs and suddenly I see a resized image of something not even remotely professional in my feed reader. Google Reader dutifully downloads a dozen feed items in advance so I am not waiting to download them. No matter what I do at that point, it won’t look good.

Now I am in the uncomfortable position of explaining to my boss tomorrow why that was in my feed reader and what I intend to do about it. I am basically going to have to cut myself off of Planet Perl and try in the most professional manner to explain that I was not looking for that sort of content. Planet Perl will now join the likes of Gizmondo and XKCD as being inappropriate to read at work.

Wow, that title sounds pretty gloomy and not entirely meant as a news header. It’s true but it was never declared.

Let’s start out by looking at the b.g.o. 206455. Check out the header information on that bug. Let me reproduce the interesting bits:

Perl 5.10.0 was released about a month ago. I attach modified ebuilds and
patches that I used to install it successfully (?) on my system

Reproducible: Always

Opened: 2008-01-17 19:39 0000
Current Status: New

In case you didn’t catch that, check out the bolded text one more time. That’s right, it’s been about 18 months since perl-5.10 was released and Gentoo still does not support it. How could a source based distribution that used to pride itself on bleeding edge support possibly fall so far behind?

Simple: the herd maintainers, both of them, have basically abandoned Gentoo.

This is interested and saddening for many reasons. I’m a long time user and supporter of Gentoo and it pains me to see it fall. In my opinion, Gentoo was the only distribution to get package management correct. I loved being able to test bleeding edge software before everyone else, including Debian and RedHat.

This situation also shows 2 problems with open source projects that you would not typically exist. First, maintaining distribution supplied versions of Perl and CPAN modules is loser’s game. It’s nearly impossible to update all of those ebuilds as fast as the developers of the modules themselves. g-cpan was a terrible project that never really worked well and no one wants to take over (as you can see from some of the recent comments). What I’m taking from this is that CPAN authors themselves are the most likely candidates to keep their modules building and installing since they’re doing something very similar already.

It also points out that with any project, there needs to be some level of satisfaction. It’s apparent from the herd’s (not so) recent commits that they lacked the desire to continue. This could stem from the historically poisonous Gentoo developer community, the difficulty in maintaining the ebuilds, or real life interference.

What can we learn from this? How can we possibly improve the situation? These are difficult questions and it’s heartening to see that concerned parties are finally starting to ask them publically. I think the first thing that needs to happen is to stop asking “where is perl-5.10″. It won’t be released on Gentoo by the herd. I’ve had to live with that for 18 months and now everyone else needs to as well. What we can do is try to improve the tools we have available, fix their problems, and write new tools to fill in the gaps. We need tools to help distributors with the ever-growing nature of CPAN. No one knows what needs to happen for that yet. Time for asking “where is it?” is over; now is the time to roll up our sleeves and get to work.

perl-5.10 on Gentoo should become a case study on how open source projects can succeed and fail. Gentoo itself is a case study with developer relations, but that’s a talk for another day.

So today, I ended up starting a very long discussion with several members of the Perl6 community about how functions are declared. Basically, I made several mistakes in trying to call my class functions “sub” when I meant method. Perl6 happily lets me shoot myself in the foot and then complains when I try calling the sub against an object, which doesn’t exist. Here’s a few examples of things to illustrate the clunky OO model:

Snippet 1:

class A {
sub foo {
say "never gonna happen"
}
};
my A $a .= new;
$a.foo;


Snippet 2:

class A {
method foo {
say "nope"
};
method bar {
foo()
}
}


_{note: the code highlighter does not support perl6}

So basically, that first snippet is an error. ’sub’ apparently means something other than what we want it to mean in this case. That snippet will fail with an error about no such method. The second snippet will fail because the ’self’ invocant is not assumed in the bar function. jnth says it’s because they don’t want methods like push being overwritten if I define a push method in my class.

The really sad thing is that I still don’t get it. I can’t make heads or tails of this object model. In some cases, Perl6 chooses to reuse syntax correctly, and in others it fails. There are about a billion ways to define a function. Each function can be a sub, method or submethod. It can have modifiers like multi and my and attributes like does and a huge multitude of parameters. The list of function modifiers just keeps going. How anyone is going to be able to use all of this is beyond me; I doubt I’ll ever be able to make sense of it.

Okay, so my raw opinion is this: they are doing it wrong. The “many paradigms” argument is wrong because the language should do one thing and do it well. It can’t serve all masters and still do a good job. If the object model is to be a real first-class citizen, then it must place the object model above all else.

That being said, I am of the opinion that C++ is the only language to get it right (see this, this, and this). Well almost, it still can’t do runtime template binding (think variant or Boost::Any) and I loathe the ‘friend‘ keyword (breeds bad code). Namespaces and classes are different concepts, so they are treated differently. When I place a function inside of a class definition, it becomes a member function. If I prepend ‘static‘, it becomes a class method (not a Class member like Java) and I can call it without a ‘this’ reference. If I want a function outside any class, just put it at the file scope and external storage. Calling a function defined within the scope (file or member) that has been included (imported) implies the ‘this’ invocant. All parameters are required unless given default values (Perl6 has 3 different syntaxes for this: required “!”, optional “?”, default value “=”). This is simple and direct. Templates were added for an additional level of flexibility, which worked out pretty well. In fact, C++0x will improve on this, assuming it’s ever released this year (otherwise it’ll be C++1x).

In bringing this up in the channel, I was really hoping to make people think about how Perl6 is doing it in hopes that someone who knows better than I would stop and think whether Perl6’s design is ideal. I was wrong and I think I may have agitated some of the developers. The beauty of open source is that I can do things my way if I see fit and likely will once the only usable implementation has support for macros.