Just after I had been through a bit of a posting slump due to some fading tuits, it would seem as if they have magically returned. This week brings the start of the Fall 2009 semester and I have seemingly sprung back to life. With the new semester comes new opportunities to use perl!

First, I wanted to mention the one bit of news that made my summer, maybe even the last 2 years, seem as if I haven’t been running in circles. I noticed that my new department head at the university posted the final new rules regarding the qualification exams. The change is that there is a list of approved courses that will be offering the QE, passing any 3 is sufficient, and there is still no marginal grade. The last exam I took, Data / Text Mining in Bioinformatics, was not listed in the blessed course list. Turns out, the department would still accept the score, so I now only lack 1 remaining pass to be an official “phd candidate“.

This semester I am taking the Design & Analysis of Computer Algorithms course. The down side is this course tends to be theory-centric, so I won’t have many chances to flex my Perl muscles. There are tons of modules on CPAN though that might help understanding the basics. There are plenty of graph, tree, and dynamic programming solutions available.

Interestingly enough, while sifting though those modules, I discovered a module of personal interest. I stumbled across the Algorithm::Viterbi module. I have studied Markov models, Markov chains, and Hidden Markov models a bunch in the last 2 years. One algorithm that keeps showing up is the Viterbi algorithm. I’ll leave it as an exercise to the reader as to how this algorithm is used, but I will point out that the Wikipedia page has Python code. Ironically, “Python’s answer to CPAN” isn’t quite all it’s cracked up to be; it lacks any packages pertaining to “viterbi” and no generic Markov package.

Perl: Automatically tested, student approved.

I started a Perl 6 project some time ago from a few discussions during a Dallas.p6m (Perl 6 Mongers) meeting. The idea was to create a complete packaging system in Perl 6 for Perl 6 with Perl 6. That sounds confusing, so let me explain the project some more. In retrospect, this is a long post that may bore all but the most ardent reader.

First, you can find the epoxy project on github in my account and the epoxy-resin database in the perl6 account (since pmichaud gives out commit access liberally). Epoxy is the main system with the epoxy-resin project as the package database.

The idea is similar to Gentoo’s ebuild system. Instead of writing clever shell scripts and executing them in crafty ways, I decided to take another route. Perl developers clearly like doing 1 thing: writing perl code. What better way to build and release your perl code than by writing some more perl code? The key is most difficult and common tasks need to be automated and allow for a huge range of flexibility.

I chose to implement the project as a tool that uses ebuild-like files, called resin files, that provide all of the functionality necessary for a user or developer to find, build, install, and repackage a project. The goal was to fully replace Makefiles, Configure.pl scripts, ExtUtils::MakeMaker, Module::Build, Module::Install, and most importantly, CPAN and PAUSE. That sounds like a lofty goal, but it’s working out quite nicely so far.

As a sidenote, I have seen Mark Overmeer’s CPAN6 and PAUSE6. I have seen masak’s proto and dcarrera’s ppm. I have seen the PAR project and Software::Packager. I have written ebuilds and RPM spec files. I have worked with JAR packages and WAR packages. I have created JNLP files. I have worked with (but not necessarily created) packages for basically every mainstream Linux distribution. I decided that none of these solutions are interesting except for a few. I also concluded that CPAN and PAUSE as we know it need to be recycled. They’re old enough and Perl 6 is new enough that it’s time to reinvent the wheel. Rather than creating a crufty 1990’s wheel all over, let’s create a modern wheel capable of handing modern technology (for example, CPAN authors have migrated to github in droves).

A resin file is similar to the ebuild file. Rather than giving it a unique extension or artsy syntax, I decided to make the resin files classes. To provide a resin build and packager for your project, you inherit from the base resin file and override the functionality you need. The idea is you will have to implement the smallest subset of functionality required for epoxy to fetch, build, and package your project. That is, for a “hello world” project, you only need to subclass Epoxy::Resin and provide a BUILD submethod that sets your metadata.

The base class, Epoxy::Resin, provides base functions, known as targets, to do things such as fetch, build, dist (repack), install, test, clean, upgrade, and remove. Each module may either use the basic functionality or provide some custom actions for each target. A resin file declares the module’s metadata by setting public accessor values, such as author, website, and license.

There are several resin files already written, though they have to be constantly maintained as I improve the functionality of the shell and the base class. Those resin files are for masak’s projects right now since his are the most visible. I intend on writing more resin files once the current ones are smaller and some of their functionality resides in the base class.

There’s also a shell that does this work. I used Tene’s sweet dispatcher to handle the commands in the most sensible and shortest manner I could come up with. Currently, it only dispatches the shell commands out directly as targets, but I intend on grouping the targets into meta-commands so that common tasks are easier. Also, only a few targets are currently supported since I have to also write the targets for each of the resin files I currently maintain. This is a time management juggling act, so I decided what I have will suffice for now.

So once you look at it (or try it out for that matter), you’ll realize there is much to do. I won’t lie, it’s very early work. There is no test suite (I haven’t figured out the best way to integrate with rakudo’s Test.pm). There is duplicated code to save time solving a harder problem (such as the shell dispatch handlers and the build targets). Most of the targets are non-functional, such as upgrade, dist, and remove. It doesn’t currently attempt to follow the standard or even remotely care what the standard says on the matter. There is a lack of error handling, such as use-ing modules that don’t exist.

What I’m trying to say is that I understand how it is incomplete. Everything that it lacks is because I have not determined the best way to solve the problem yet. In some cases, I had to even work around known problems or a complete lack of an eco-system. The Epoxy::Fringe and Git modules are proof of that. I have the tuits and development will inch forward.

I haven’t decided on an issue tracker for the project yet. I tried to use Lighthouse but it seems unintuitive. In order to complete a milestone, you have to create tickets and mark them as complete. This is really just good for cyclic or maintenance projects but not for off-the-cuff new development. I’m currently thinking about github’s newly added Issue page, which looks like it will suffice for now.

I asked masak, a prominent member of the Perl 6 community to review my code. He had mostly positive things to say. He pointed out that he didn’t like the name of the module that builds Epoxy itself: Epoxy::Build. He told me about IO.prompt and how BUILD submethods are now supported. He also pointed out my lack of attention to the metadata attributes. These are all things on my TODO list which will be addressed eventually. He was surprised to find that I was replacing almost all of the Makefile, Makefile.in and Configure scripts to his projects.

This all started out as an exercise to learn Perl 6 better. I wanted to know the syntax and to start thinking of how things work. The best way to learn is to act. I am having fun working on this every other day or so. It’s a nice metal exercise that keeps me busy over the summer.

I have grand plans for this. I will take my time and focus on the code. I will not be distracted by a charged discussion without any working code. Concepts and ideals are great, but sometimes, code speaks for itself. I won’t argue about the future of CPAN or which way is the best to package a product. I won’t partake in the arguments because everyone seems to have an opinion but no solution. Those are arguments no one can win. Until you know that, you are useless.

I will however take feedback and constructive critisism. I don’t want to fan any flame wars or discuss what anyone thinks is the best idea for the future of Perl or Perl 6. This is my free time and the last thing I’m interested in is pointless heated discussions. I am always open to feedback, reviews, and assistance.

I finally took the plunge and requested a PAUSE user account. PAUSE, for those uninformed, is the Perl Authors Upload Server. Through this, users are able to upload modules, scripts, and source code to CPAN.

I’ve been writing perl programs for maybe 7 years now but have never found anything I wrote to be useful to anyone other than myself. Then I started reading Linus Torvalds’ book “Just for Fun: The Story of an Accidental Revolutionary“. I’m not done reading it yet, but I will post a review when I finish.

In the 10 years or so of using Linux, I’ve never really contributed much. I wrote a quick program over on SourceForge but found I never wanted to use it myself. I’ve contributed a handful of patches to various projects. I even wrote the arbitrary radix parsing code in Rakudo. Yet I’ve never contributed any significant amount of code. I’ve always felt guity about that; like a leech who used other people’s work for my own selfish ends.

Well, that book changed everything.

I’ve got a backlog of a ton of little programs that all solve some domain specific problem. Most have been for school; some are completely boring while some produce highly interesting results. It’s time I start contributing my code back to the world.

I’m not sure what I’ll contribute first. Most of them need to be cleaned up from their current state of “get it working” to publishable code. I’m going to start working on that over the summer. There are a few projects that I am working on, one of them being my thesis, which can all ultimately find their way on CPAN.