technology


OCaml first hit my radar in November 2013. I had just learnt SML, a similar but older language, in the excellent Programming Languages Coursera course. Dan Grossman is one of the best lecturers I’ve ever seen, I found his explanations hit all the right notes and made learning easy. The simplicity of the SML syntax, and the power of the language while still producing code that is readable with minimal training immediately appealed to me.

Over the last 3 years I have tried, and failed, to learn Haskell. The combination of minimalist syntax, pure functional programming style and lazy evaluation is like a 3-hit sucker punch that is very hard to grasp all at once. Having now learnt SML and OCaml, which like Haskell are based on the ML language, that has changed. I have yet to put any more effort into learning Haskell, but it is now clear to me that the syntax is only a small leap from ML and the pure functional style has similarities to SML.

I still don’t want to write production code in Haskell, but the fact that I find it less scary than I used to indicates I have made a significant jump in my knowledge and, arguably, career in the last 6 months.

Dynamic typing

Before I go any further, I need fans of dynamic typing to exit the room. My 12 years in the industry have set my camp firmly on the static typing side of the fence, and discussions about static vs dynamic will not be productive or welcome here.

So, why OCaml?

Smarter people than me have written about this, but I’ll give it a shot.

I have found OCaml to be a refreshing change of pace. Most of my favourite things are derived from the ML base language; variants, records, and pattern matching combine to create elegantly powerful code that is still easy to follow (unlike most Haskell code I’ve seen).

Ocaml takes the expression-based ML style and incorporates enough imperative features to make it comfortable for someone learning Functional Programming. Don’t know how to use recursion to solve a problem? Drop into a for loop in the middle of your expression. Need some debug output? Add it right there with a semicolon to sequence expressions.

Throw in almost perfect static type inference, a compiler that gives useful error messages and immutable-by-default variables and I just can’t get enough. I won’t sit here and list every feature of the language, but hopefully that piques your interest as much as it did mine ;)

Industry acceptance

There is always an element of “I have a hammer, everything looks like a nail” when learning a new language but the evidence that OCaml is becoming more widely accepted is not hard to find.

In the middle of February, Thomas Leonard’s OCaml: what you gain post made waves; the reddit and hackernews discussions are fascinating. A lot of people using OCaml in the industry came out of the woodwork for that one. I’m still working my way through the series of 11 posts Thomas made, dating back to June 2013, about his process of converting a large Python codebase to OCaml.

Facebook have a fairly extensive OCaml codebase (more details below).

It doesn’t take much googling to find presentations by Skydeck in 2010 (they wrote ocamljs, the first ocaml to JS compiler) or a 2006 talk describing why OCaml is worth learning after Haskell.

OCamlPro appear to be seeing good business out of OCaml, and they have an excellent browser-based OCaml tutorial (developed using, of course, js_of_ocaml).

No list of OCaml developers would be complete without mentioning the immense amount of code at Jane Street.

There are plenty of other success stories.

The elephant in the room

The first question I usually get when I tell a Functional Programming guru that I’m learning OCaml is “Why not Haskell?”. It’s a fair enough question. Haskell can do a ton more than OCaml can, and there are only one or two things OCaml can do that Haskell can’t (I don’t know the details exactly, I would think it was zero). I see a lot of references to OCaml being a gateway drug for Haskell.

The answer is JavaScript. As much as I hate the language, JS is the only realistic way to write web apps. Included in the many and varied AltJS languages, both OCaml and Haskell can be compiled to JavaScript but the Haskell compilers aren’t mature enough yet (and I’m not convinced lazy evaluation in JavaScript will have good performance).

In fact, some study has revealed OCaml may be the most mature AltJS compiler of all by virtue of support for existing OCaml libraries.

JavaScript

Late last year I started hearing about OCaml at Facebook. Their pfff tool, which is a serious OCaml codebase all by itself, is already open source – but there was talk of an even larger project using js_of_ocaml (the link seems to be offline, try the video). That presentation by Julien Verlaguet is almost identical to the one he gave at YOW! 2013 and it really grabbed my attention. (Hopefully the YOW! video is online soon, as it’ll be better quality).

To cut a long story short, Facebook created a new language (Hack, a statically typed PHP variant) and wrote the compiler in OCaml. They then use js_of_ocaml to compile their entire type checker into JavaScript, as the basis of a web IDE (@19 minutes in the video) along the lines of cloud9. Due to the use of OCaml for everything, this IDE has client-side code completion and error checking. It’s pretty amazing.

Maturity of tools and js_of_ocaml

The more I dive into OCaml, and specifically js_of_ocaml, the more it amazes me how the maturity of the tools and information reached suitability for production use just as I need them.

  • The package manager OPAM is now a little over 12 months old and every library I’ve looked at is available on it. Wide community acceptance of a good package manager is a huge plus.

  • The Real World OCaml book was released in November and is an excellent read. The book is so close to the cutting edge they had features added to September’s 4.01.0 compiler release for them :)

  • OCaml Labs has been around for 12 months, and they’re helping to move the OCaml community forward into practical applications (see the 2013 summary).

  • Ocsigen are investing heavily in js_of_ocaml (among other things) with the next release including an improved optimiser (I can attest to the fact that it’s awesome) and support for FRP through the React library.

Moving forward

Is it perfect? No. Software development is not a one-size-fits-all industry. There are as many articles cursing the limitations of OCaml as there are singing its praises. But in the current market, and with the size of JavaScript applications we are starting to generate, I believe OCaml has a bright future.

Close to 18 months ago, when I first started seriously using that old mac laptop, I decided I needed a way to easily transfer my speakers between the desktop games machine and my mac that I used for everything else. One of my mates at work had an Audigy 2 NX, and after borrowing it for a day to make sure it worked on macs I decided to get one. It wasn’t until I had it that I realised the mac was only giving me 2 channels instead of 5.1 :(

I shrugged and chalked this up to the built-in mac drivers, it was fine under windows with the official creative drivers.

And so it was that when I upgraded to the mac mini, and again with this second mini, that I was stuck with a sound card that wasn’t giving me surround. Most of the time this doesn’t concern me as I usually only listen to stereo sources, but I’d never even considered that it might work (the few references I could find to this device on the net were it only working in stereo on the mac).

Until tonight.

While doing some research for a friend who was interested in USB sound cards, I saw a product review stating that the Zalman USB card does work on macs in full 5.1 surround mode. This piqued my interest so I went searching and stumbled on a list of working sound cards forum post. Right there at the top is the Zalman card, but hang on, what’s that sitting at the bottom under supported 7.1 cards? Why it’s my damn Audigy 2 NX! WTF!

I immediately (and stupidly) installed the package attached to that post, but thankfully I read a bit further down the post before rebooting and realised I didn’t need to. This was a good idea because the package is from 10.4 somewhere and I would almost certainly have been left trying to do a restore from backup. I’ve reverted the kext files that the package installed, hopefully my mac doesn’t die when I reboot it after posting this.

In any case, the answer is Audio MIDI Setup! A program that had always sat in the Utilities folder looking summarily useless but turns out to be the hidden gem that Apple really needs to make more obvious. For those who will no doubt arrive here from google one day, here’s how to enable 5.1 surround sound on a USB sound card:

  1. Select your sound card under the Properties For: dropdown
  2. Select the number of channels under the audio output format
  3. Click Configure Speakers
  4. Select Multichannel
  5. Select the correct number of speakers from the dropdown (only the valid one should be enabled)
  6. You can now assign channels to each speaker, I’m pretty sure the numbers I used are correct although 3/4 and 5/6 might be in the wrong order

Here’s a couple of screenshots with number highlights to make it clear:
Audio Midi Setup
Audio Midi Speaker Setup

Maybe it’s just this sound card, but that’s a ridiculous requirement to get 5.1 surround sound working (and I haven’t actually tested if DVDs will play correctly, only some 6 channel test wavs I found). Wish me luck! ;)

On the plus side, if this does work I will no longer have to worry about surround sound output from my media centre when I buy proper home theatre speakers (the audigy has optical and spdif out). I had been concerned that I would be stuck with stereo output from my Mac forever!

The new Google mobile ActiveSync is working great for my calendars. Syncing iCal to google was pretty easy; I exported my 3 local calendars, cleared out the main Google calendar & created 2 new ones (naming my primary calendar “work” thanks to the stupid Outlook plugin), subscribed to all 3 via CalDav with Calaboration and then imported the data. No worries at all. I can create an event in iCal and 10 seconds later it appears on my phone :D

There was a bit of confusion and duplication after syncing my Outlook calendar at work to Google (Did I mention the plugin’s main-calendar-only restriction is REALLY annoying? How about it’s complete inability to detect duplicates?) but that was pretty easy to clear up.

What I haven’t done is turn on address book syncing with the phone. As I suspected and others have confirmed, turning on ActiveSync for contacts & calendar stops iTunes from doing any sync work with them for the iPhone. Which, since iTunes initiates the contact sync to Google, means that contacts are no longer synced to my desktop.

Both forum posts I’ve just linked to have suggested fixes (particularly if you expand them beyond the accepted answer), but I can see three options personally to sync my contacts:

  • Resurrect the fancy iSync scheduling that I haven’t used since switching to the iPhone (I still use the scheduler, just for some Address Book hackery instead of activating the iSync menu)
  • Don’t drop Plaxo completely as I had planned, but use it to sync between google’s contacts and Address Book
  • Leave over-the-air contact sync disabled and continue with iTunes to Google contact sync

So far option 3 sounds the easiest to me. I don’t need to sync my contacts more than once a day (which is how often I sync with iTunes), over-the-air sync wouldn’t give me all of my contact numbers on the phone anyway, and this way I can completely disconnect from Plaxo.

Not that Plaxo is bad – I’ve really enjoyed the service, including a far better Outlook calendar sync platform than Google’s Outlook plugin provides – but ever since I switched my email to Google I have only used it for the Outlook sync (hotmail contact sync is enabled but I don’t need it anymore). It just doesn’t make sense to continue using it in light of Google’s improved Mac/iPhone sync options.

I’m in the middle of another post and was distracted reading some RSS feeds when I came across this gem. Google now supports ActiveSync for contacts and calendar :D :D

I already sync my contacts and calendar to google but it’s only updated on the phone when I sync it with iTunes. Now I get to experience the joy that is push calendaring!

Six years ago, I built my first Linux server. It was my third year at uni and I had just bought a new computer; being a poor student I would usually have upgraded but a mate of mine gave me his old case for some reason I don’t remember.

At the time I was living on campus with plenty of geeks on the local irc channel to encourage me and get me started – my debian stable server was born. I named it rei, after the evangellion character (it was the first anime I ever watched, which happened around the same time).

The campus environment at uni was the ideal place to learn server administration. I had lots of people in the same building to get advice from, but each room was networked so if I messed up it only affected me (well, most of the time anyway ;) ).

And so it was that I learned how to set up a basic firewall, file server and eventually an IMAP server + fetchmail which I have been using ever since. In the years since it has run – in addition to the normal tasks – stuff like an irc server and a local network game server (can’t remember which game though). When I finished uni it became the house firewall (requiring a far more advanced ruleset).

After upgrading to debian testing almost immediately post-setup that little box has survived two motherboard failures, three case transplants, a lucky recovery from a storage drive failure and the incredibly lucky recovery from an accidental “rm -rf *” in the root directory (yay for reiserfsck!).

I did eventually wipe the machine last year, after getting sick of kernel compiles and various other things caused by the age of the install that meant the server took more administration time than I wanted it to. By then I had a mythtv server as well (running fedora core 3), to make life easier I put ubuntu on both. And life was good.
 

Then with the recent possibility of using a small low powered mythtv frontend instead of the combined front/backend I have now, it got me thinking about why I need two servers. And as much as I hate to admit it, I don’t. Rei’s firewall has been replaced by a decent modem/router that doesn’t need any maintenance; the file server has been mostly replaced by my flatmate who is such a hoarder that he decided to build his own server. The files it serves now can easily be moved to the mythtv box.

The only thing I have needed that server for recently is mail, and I’ve been less than happy with that since I had to punch a hole in the firewall to my imap server in order to check mail on my phone. Even though we have adsl2 at home it isn’t as fast as I’d like. Not to mention that the fans in the case are so old and noisy that they’ve been unplugged so the drives are running hot. I had been holding off switching to gmail, but really the only excuse left was because I didn’t want to lose my first server. With the migration now done, it’s time to pull the plug.

The server name will live on; I already have plans to replace the mythtv backend with my dual core desktop machine when I either buy a new gaming PC or the still-rumored mac mini upgrade (if it has a decent graphics card that can replace the PC). That will be quite an upgrade from the 8-year old CPU (Athlon 1200c) it has been using since the first mobo failure.

But rei is dead, long live rei :) (that’s a bit of a strange way to put it if you’ve seen the anime!)

So that was quick; the gmail migration is complete. 18400 emails (and that’s actual count, gmail reports 9400 conversations), a tad under 600mb. It turns out that gmail is quite happy to let OS X mail open a bunch of connections, so all I had to do was make a few temp labels and suddenly I’m copying six lots of 1000 messages instead of one. In fact, it would’ve been done sooner but my modem shut down in the middle of a move last night and OS X mail sat there with the copied mail counters rising far too fast. Needless to say that was quite disturbing.

Half an hour of concerned investigation later it turns out I that not only does gmail merge if you copy an email twice, Mail was actually caching the move. It attempted to complete the process after the internet returned, until I accidentally cancelled it (this was quite surprising, considering I was moving between IMAP accounts).

Oh well, it didn’t waste that much time.

The cool part is that I don’t need to worry about sending via gmail with my normal address. I found the easiest way to set up SMTP on my iPhone was gmail, so I have been using it to send email for a few months now and nobody has been the wiser. All it took was updating gmail to acknowledge I owned my home email address and setting that as the default.

My gmail account has looked a little strange (it had copies of every sent email since July, with none of the received) but it has all worked out in the end.

Migrating to gmail is hard when you’re a packrat. Especially one that spent years using Outlook Express :)

I have basically every non-spam email sent to me in the last 8 years; I even have a few earlier than that. The oldest email I have is dated June 1999, from one of my first year uni subjects. I won’t know exactly how many emails I have until I finish moving it all to gmail, but rough estimates from the folder counts put the number around 12000. My Maildir folder is 620mb ;)

I had a crazy number of folders on my IMAP server. My setup consisted of the usual collection of folders for sorting, but Outlook Express can get annoying with large folders so I wanted to keep the per-folder message count down. My solution, 8 years ago, was an “archive” folder which mirrored the major folders and I created a new set of them for every year. I’ve spent most of this evening combining them into 1000-mail chunks, I don’t want to babysit this process but I also don’t have enough confidence in the universe to try moving all emails in a single hit!
 

So why am I just now moving to gmail? That story will have to wait for another day. Don’t worry, the post is already written so I will actually follow up this time ;) but certain events are still in progress and I want to wait until they’re done before posting it.

I have to say though I’m not sure keeping all of this email was the best idea. I’ve glanced at a few old emails while sorting this evening and… well put it this way, would you want a detailed account of your uni years? :)

Next Page »

Follow

Get every new post delivered to your Inbox.

Join 90 other followers