Announcing ical: A pessimistic iCalendar (RFC 5545) library

This post announces the new ical Haskell library for implementing with RFC 5545: Internet Calendaring and Scheduling.

RFC 5545 is a widely used specification for calendar data exchange on the internet. It defines a data format for representing calendar events, such as appointments, meetings, and reminders.

Historical perspective

In order to build Smos' smos-calendar-import tool to view your calendar from within smos, I had to integrate with the iCalendar specification.

I found the iCalendar Haskell library and found that it was good enough for its purpose at the time. This library does not implement recurrence, so I had to implement it myself. It took a very deep reading of the specification (and errata!), a few iterations, and lots of tests, but I managed to implement recurrence for smos-calendar-import.

A little while later it turned out that the iCalendar Haskell library parses more strictly than smos-calendar-import can afford to, and has no controls for that. Indeed, all of Google, Apple, and Microsoft sometimes output invalid .ics files (according to the spec). The iCalendar library failed to read those entirely. While this is a valid approach(!), it was not good enough for smos-calendar-import because it meant that once an invalid event got into my calendar somehow, I could not longer import anything.

I had tried contributing to the iCalendar Haskell library directly, but never got much of a response to that. Given the GitHub activity for this library, it looks like the library has been unmaintained for a while. Even benign version bumps have not been reviewed or merged. There's nothing wrong with that, but it meant I had no choice but to fork it indefinitely or write my own.

Goals

Fit for purpose for Smos: Consuming iCalendar files
Fit for purpose for Social Dance Today: Producing iCalendar files
Fit for integration with faulty producers (just about every one of them, including itself probably)
Flexible strictness with respect to the specification. I.e. Parse as strictly as you want: Parse your own files strictly and others' leniently.

Cursed transformer

The core parser uses a piece of necessary-evil code:

-- | A conforming monad transformer to compute a result according to a spec.
--
-- RFC 2119 describes these terms:
--
-- 1. MUST and MUST NOT:
--    These describe absolute requirements or absolute prohibitions.
--    However, some implementations still do not adhere to these.
--    Some of those situations are fixable, and some are not.
--   
--    If the situation is fixable, we error with an error of type @ue@.
--    
--    If the situation is fixable, we can either error out (a strict implementation) with an error of type @fe@ or apply the fix.
--    The @fe@ parameter represents fixable errors, which can either be emitted as warnings, or errored on.
--    A predicate @(fe -> Bool)@ decides whether to fix the error. (The predicate returns True if the fixable error is to be fixed.)     
-- 2. SHOULD and SHOULD NOT:
--    These describe weaker requirements or prohibitions.
--    The @w@ parameter represents warnings to represent cases where requirements or prohibitions were violated.
newtype ConformT ue fe w m a = ConformT
  { unConformT ::
      ReaderT
        (fe -> Bool)
        (WriterT ([fe], [w]) (ExceptT (Either ue fe) m)) a
  }

This transformer lets us write a single parser, but still parse more or less strictly depending on the use-case. We can then run it using functions like these:

runConformTStrict ::
  Monad m =>
  ConformT ue fe w m a ->
  m (Either (Either ue (([fe], [w]))) a)

runConformT :: 
  Monad m =>
  ConformT ue fe w m a ->
  m (Either (Either ue fe) (a, [w]))

runConformTLenient ::
  Monad m =>
  ConformT ue fe w m a ->
  m (Either ue (a, ([fe], [w])))

runConformTFlexible ::
  (fe -> Bool) ->
  ConformT ue fe w m a ->
  m (Either (Either ue fe) (a, ([fe], [w])))

For example:

> let parser = emitFixableError "hi"
> runConformStrict parser
Left (Right ((["hi"], [])))
> runConform parser
Left (Right "hi")
> runConformLenient parser
(Right ((), (["hi"], [])))


> let parser = emitFixableError "hi"
> runConformStrict parser
Left (Right (([], ["ho"])))
> runConform parser
Right ((), ["ho"])
> runConformLenient parser
Right ((), ([], ["ho"]))

Note that this should not be necessary. All .ics files should be valid according to the iCalendar specification(s). The problem is: What do you when you find invalid output anyway?

You could reject it entirely, but that comes at a cost for users.
You could read it partially, but that alters the semantics of the file.
You could fix the issues upstream, but good luck getting Google, Apple, or Microsoft to fix their ICal implementation.
You could guess what was meant, but you may be wrong.

Really there are no good options, so we let the users decide which they prefer.

Testing

I really wanted to make this library and be done with it. RFC 5545 is too complicated and finicky to shotgun, so I wrote a boatload of tests. Indeed, the libraries have great test coverage:

The greatest insight for testing this library was that the combination of the following three types of tests worked very well to find faults;

Parsing <-> rendering roundtrip property test
Golden rendering test
Unit tests for parsing

The library uses parsing and rendering in multiple stages, that are each tested independently as well as together:

Text <-> Unfolded lines <-> Content Lines <-> General Components <-> Typed components

In order to ensure that the library can deal with other producers, it also has regression tests for previously received (invalid) ical from other producers, and roundtrip tests with libraries in other languages.

References

The ical libraries are available on GitHub. They are in use in production at Social Dance Today and in Smos.