20081118: Agile Open France 2009

Ça y'est, c'est officiel, les inscriptions à la conférence Agile Open France 2009 sont désormais ouvertes. Bien évidemment, je me suis précipité sur le formulaire d'inscription dés réception de l'annonce. J'ai eu la chance de participer à l'édition 2008 et je dois dire que j'en suis revenu enthousiasmé par le format Forum Ouvert (ou Open Space) qui, à mon sens, est la meilleure manière de produire un tel événement.  

J'aurais voulu utiliser ce format pour Agile Tour Lille 2008 mais celui-ci suppose que les participants aient déjà une idée de ce qu'ils viennent trouver et apporter, ce qui n'était pas nécessairement le cas du public de l'Agile Tour.  

La deuxième journée des Valtech Days était aussi organisée selon ce modèle et je dois dire que c'est la partie que j'ai le plus appréciée.  

J'attends avec impatience le mois de janvier pour être de nouveau surpris.  

20081114: Paying back technical debt

I am struggling to improve my Muse parsing software, a project that I have been working on for couple of years now and which I use to publish this blog and the whole OQube's site. It is based on the Emacs Muse wiki format which means that I can operate within the context of Emacs with all the useful tools and keyboard shortcuts I find there, and be able to serve muse content over the web in various backend formats, HTML and RSS feeds being the most useful.

As all software, especially software that is used and maintained by a single person, it is made of layers of code injected haphazardly from urgent needs or nigthly ideas.  

There are some basic flaws in the design of the key elements of this software that are now impediments for further evolutions:  

This means that the time spent implementing new features is raising. I am trying to implement a simple slide formatter, something that I could use to produces slide-shows for talks, including code samples for online execution and examples in Javascript, something along the line of what John Resig published on his site

To implement this, I need one of the following but both are hards to implement using the current state of my muse parser:

I have already had 3 false starts trying to implement this, which is a sure sign that something gets wrong: either me or the software is becoming very reluctant to change.  

20081113: Dojo de Lille

Aujourd'hui au dojo, nous avons fait en randori du refactoring d'un bout de code Java (voir cette page). Nous avons eu le plaisir d'accueillir de nouveaux venus dont certains que je connaissais par ailleurs (Michael et Vincent, que j'ai rencontré dans le groupe PYLULE), d'autres que je ne connaissais pas. Il semblerait que l'Agile Tour ait porté ses fruits et intéressé des gens à la pratique de l'agilité.  

Le code de départ est le suivant: 


 /**
   * Split a signature string into its constituent parts and fills the list
   * argument.
   * 
   * @param sig
   *          java standard signature
   * @param ret
   *          a List where parsed arguments will be stored. May be null.
   * @return a List<String> of arguments' types plus return type. The strings
   *         correspond to the standard internal encoding of the JVM.
   */
  public static List parseSignature(String sig, List ret) {
    if (ret == null)
      ret = new ArrayList();
    int i = 0;
    int j = 0;
    int ln = sig.length();
    StringBuffer arr = new StringBuffer();
    parse: while (i < ln) {
      switch (sig.charAt(i)) {
      case '(':
        i++;
        break;
      case 'L':
        j = sig.indexOf(';', i);
        arr.append(sig.substring(i, j + 1));
        ret.add(arr.toString());
        arr.delete(0, arr.length());
        i = j + 1;
        break;
      case ')':
        i++;
        break;
      case '[':
        i++;
        arr.append("[");
        break;
      default:
        arr.append(sig.charAt(i));
        ret.add(arr.toString());
        arr.delete(0, arr.length());
        i++;
      }
    }
    return ret;
  }

Nous n'avons pas achevé le remaniement du code, l'objectif initialement fixé étant de restructer le code pour le rendre plus lisible et maintenable tout en gardant le comportement identique. La motivation est d'envisager une évolution future de code, par exemple une modification de signature pour rendre explicite des conditions d'exception, ou une suppression de paramètres du fait du non-respect du Principe de Responsabilité Unique

La stratégie adoptée a consisté:  

  1. à fixer le comportement global de la fonction dans des tests "de recette" à partir des informations contenues dans la Javadoc, c'est—à-dire de rendre la Javadoc exécutable
  2. à explorer certains comportements non prévus, comme par exemple le comportement d'une signature nulle (rien n'en est dit dans la doc, mais le cas est prévu pour le second paramètre donc on peut supposer que l'auteur du code a envisagé la question et qu'on ne s'attend pas à avoir une signature nulle). Dans ce dernier cas, on va rendre explicite le comportement dans le code à l'aide d'une excepition qui pour l'instant sera NullPointerException mais qui pourra évoluer par la suite;
  3. une fois le comportement "bordé", on peut s'attaquer au corps de la méthode, d'abord en renommant correctement les variables;
  4. puis en détaillant par des tests les différents cas prévus par l'instruction switch interne, ce qui nous permettra d'extraire ces différents cas particuliers dans des fonctions/objets connexes et donc de simplifier le code.

Dans le cas de cette fonction, il est clair que l'on est en présence d'un parseur donc que l'utilisation d'expressions régulières ou d'autres mécanismes d'analyse syntaxique serait pertinents et rendrait le code beaucoup plus explicite.  

Aucune vérification de correction n'est faite sur les constituants de la signature, par exemple sur le nom d'une signature représentant une classe ou sur les différents codes possibles pour les types primitifs. Cette fonctionnalité pourrait être facilement ajoutée une fois l'analyse syntaxique correctement réalisée.  

20081110: Type layering pattern

This is an interesting idiom found in the paper on reactive programming in Haskell, that constructs new types using layers of type constructors, each layer providing additional properties through instances of type classes:

It is used to define an improved time type with some useful properties to work on timed-streams:

previous

type FTime = Max (AddBounds (Improving Time))
newtype Max a = Max a deriving (Eq, Ord , Bounded )
instance (Ord a, Bounded a) => Monoid (Max a) where
  mempty              = Max minBound
  Max a `mplus` Max b = Max (a `max` b)


data AddBounds a =
  MinBound | NoBound a | MaxBound deriving Eq


instance Bounded (AddBounds a) where
  minBound = MinBound
  maxBound = MaxBound


-- An improving value. Invariant:
-- compare I iv compare (exact iv )
data Improving a =  Imp{exact :: a, compare I :: a -> Ordering }


compare :: Ord a => Improving a -> a -> Ordering
exactly :: Ord a => a -> Improving a
exactly a = Imp a (compare a)

20081109: Functional Programming IS hype

20081022: Valtech Days - Day 2

Dojo refactoring legacy code

Dojo randori sudoku

Enterprise Agile (Jeff McKenna)

list of topics:

voting: -> Arguments to convincing

Arguments for convincing

What's worked ?

cf Guitart, "l'envie de s'y mettre"

Agile team inside non-agile organization

Programming languages

Paul Graham in Hackers and Painters: "Language matters, and LISP is the best one because of macros/meta-programming and its sheer rarity."

Programmation Postmoderne

20081021: Session sur les langages

20081021: Valtech Days - première journée

En vrac

Programmation concurrente

Session présentée par Julien Delhomme, consultant chez Valtech, sur les différents modèles de concurrence et l'avenir de ce type de programmation pour le commun des mortels. Plusieurs modèles ont été présentés:

  1. processus multiples et mémoire partagée (eg. Java), le modèle standard, le plus bas niveau et le plus compliqué à programmer
  2. vectorisation du code (eg. OpenMP), plutôt que parallélisation. On produit des calculs concurrents isolés  
  3. mémoire transactionnelle (Software Transactional Memory, eg. Haskell)  
  4. passage de messages (eg. Erlang)

Dojo - Kata Mastermind en Ruby

Kata présenté par Emmanuel Gaillot et Etienne Charignon, sur le théme du Mastermind. Excellente préparation, avec un bon déroulé du problème et un gros effort d'explication de la part du copilote et du pilote. La solution obtenue est effectivement assez élégante, encore que l'on aurait pût - dût ? - remanier le code plus tôt de manière à introduire les abstractions objets plus facilement (eg. le comptage des jetons bien et mal placés).

Questions intéressantes sur:

20081021: Web application - Part 4 - Thinking

I have (nearly) the skeleton of a working web application in Haskell, using FastCGI interface, storing session in a map keyed by some integer value, and allowing users to login, register, logout... The UI is pure HTML with javascript based on jQuery and its rich API: data is requested from the server that is supposed to return javascript or JSON objects, which will be used by the client side to update the UI. This last part is still missing, but I am feeling I have reached, once again, some dead end.

The structure of the application is embedded within the services it exposed, meaning that, for example, that register updates the session to allow login action, which in turns update the session to allow logout  and unregister, etc. As those functions work on with a database, this implies that to both work with I/O and with the state of the application, they must work within a StateT monad transformer parameterized with IO monad. We can foresee that composing additional behavior will entail more and more monad stacking for eg. logging, error tracking.

What we really is, as usual, decouple the various components and layers we are working with so that each function serves one and only one purpose and that the top-level application orchestrates them. At this stage, without thinking about additional monadic behaviours, we would like to distinguish at least two layers:

Being a very flexible language, Haskell should allow us to express composition of fucntions and behaviours externally, without the individual elements ever noticing they are part of some composite application. What is done with XML configuration files in Spring-like frameworks should be done in Haskell within the Haskell framework.

This idea I have already implemented in Java while developing the M-Commerce framework, which used an underlying finite-state machine to orchestrate access to service and the Pico container framework for dependency injection. DI within Haskell can be done:

20081020: Dojo Paris

Sujet

20081020: Pattern-matching with functions only

While researching a way to represent pattern-matching easily within the My Language is Bigger than Yours boardgame, I came upon a book chapter on google-books that precisely defines a program transformation for eliminating pattern-matching and recursive data-types definitions from Haskell progras, thus making them composed only of functions and primitive constants.

The key to this transformation is to observe that any datatype can be equivalently transformed, using what is called Church-encoding into a set of constructor and desctructor functions. This I have already shown in a preceding article (see this article ). Given the following data-type for binary trees:

previous

data Tree a = Leaf a
            | Node a (Tree a) (Tree a)

we can define the constructor functions: previous


type Tree a = ((a -> c) -> (a -> t -> t -> c) -> c)
Leaf a      = (\l n -> l a)
Node a t t' = (\l n -> n a t t')

If we want to know the value stored at a tree, we can either write, using pattern-matching: previous


value Node x     = x
value Leaf x _ _ = x
or, using constructor functions: previous

value t = t (\a -> a) (\a  _ _ -> a)

The key insight to this transformation is to notice that:

  1. each term of the sum of an algebraic datatype is basically a function that produces a type given some arguments, and
  2. an algebraic data type can be viewed as a selector function that discrimates between various possible behaviours according to its internal structure.

So we can hopefully generalize this transformation to any datatype: given a type defined as a sum of products as: previous


data Type = Ctor1 p11 p12 ... p1n
          | Ctor2 p21 ... p2m
          | ...
          | Ctork pk1 ... pkl

we can define the type Type and a set of constructor functions ctor1, ..., ctorl defined as: previous


type Type = (t11 -> ... -> t1n -> c)
         -> (t21 -> ... -> t2m -> c)
         -> ...
         -> (tk1 -> ... -> tkl -> c)
         -> c


ctor1 v11 ... v1n  = (\ f1 f2 .... fk -> f1 v11 ... v1n)
ctor2 v21 ... v2m  = (\ f1 f2 .... fk -> f2 v21 ... v2m)
...
ctork vk1 ... vkl  = (\ f1 f2 .... fk -> f2 vk1 ... vkl)
and then each function operating on Type that wants to pattern-match against its instances should be implemented by passing the specialized function to the given instance of Type that it is passed.

This can be generalized to n-ary pattern matching in the obvious way, ie. by currying.

20081018: Filtering sections from Muse RSS Feed

It took me quite a few hours to implement a feature that, at first sight, appears quite simple: Extract and display a single entry from a log file as an HTML page. This feature is meant to be used as the target of links from RSS feeds. Given the unexpected difficulty of the task, and the somewhat contorted solution I found, I felt it would be worthwhile to provide details of the problem, the various paths I took and the (partial) solution I found.

The context

I am developing a set of Java components based on the Emacs Muse publishing format, a simple wiki-like text format that I discovered a few years ago while using EmacsPlanner tool. Among other things, this blog and the whole OQube's site are written in Emacs Muse and published on the web using a dedicated web application.

This system is composed of, among others, the following components:

The problem

The Feed backend produces a RSS 2.0 formatted news feed from a muse file: Each level one title is considered a news item if the title contains a date formatted as YYYYMMDD. In the RSS 2.0 format, each entry is identified with a supposedly unique URI, which is formed by transforming the date to YYYY/MM/DD format and suffixing it with the title from which all non-ascii characters have been escaped (I should use the standard urlEscape() function).

For example, this news item is transformed to: 

<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/"
     xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
     xmlns:dc="http://purl.org/dc/elements/1.1/"
     xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>OQube's Blog</title>
    <link />
    <description>Random things I am working on...</description>
    <dc:creator>Arnaud Bailly</dc:creator>
    <item>
      <title>Filtering sections from Muse RSS Feed</title>
      <description>20081018: Filtering sections from Muse RSS Feed 
&lt;h2&gt;The context &lt;/h2&gt;
&lt;p&gt;I am developing a set of &lt;em&gt;Java&lt;/em&gt; 
      <pubDate>Fri, 17 Oct 2008 22:00:00 GMT</pubDate>
      <guid isPermaLink="false">http://localhost:4444/journal/journal/2008/10/18/Filtering-sections-from-Muse-RSS-Feed</guid>
      <dc:date>2008-10-17T22:00:00Z</dc:date>
    </item>
...

The problem lies in the <guid> tag: Using the given URI in a browser should provide the given news in XHTML format. The news item should be extracted from the muse file which contains all the news. The question is: how to extract the selected news (or any other subset of the file) from the muse file and format it ?

Abstracting away from the details, this problem can be narrowed to the following: Given a stream of objects, how to select a subset of this stream using simple rules ? For example, if we consider a stream of a-z letters, we want to select the substream that is contained between the letters ab and the letters yz. Applying this rule to the word jlkjlkabcdeffdsfyzflsfdkj, we obtain the word abcdeffdsfyz

It is obvious that, in the case of letters, this problem boils down to regular languages matching: extract some rational pattern from a stream (ie. a word). But in the case of events, we are facing a somewhat different problem: the alphabet is infinite which implies the language is not regular. Furthermore, writing a  finite state machine ie. a regular language acceptor, while not an extremely complex task, is not immediate and quite error-prone.

The (path to the) solution

The idea I explored is based on event filtering and the Decorator pattern:  

The problem I faced was managing the state of the underlying FSA so that parsing events can be passed in a timely fashion:

Neither of this solutions is satisfying:

The choosen solution

The idea is to define the filter as a SinkEvent transformer, a function from SinkEvent to SinkEvent that returns the event to be passed to the real sink. A special event is defined, NullEvent that is a non-event and, when asked to activate a sink, does nothing.  By defining filters as transformers, they can be composed.  

previous

public interface SinkFilter {
  SinkEvent filter(SinkEvent e);
}

Filters are then made into combinators that operate on stream of sink events and, together with filter operators, can form a whole algebra of stream transformers and selectors.

For this to works, I also needed to create a CompoundEvent object that implements the SinkEvent interface and that passes all accumulated events to its underlying sink. This is needed because, when the filter needs more than one event lookahead, it buffers the events passed so far and it cannot produce the correct event before having matched the whole pattern.  

previous

public interface SinkEvent {
  SinkEvent NULL_EVENT = new SinkEvent() {...};


  void passTo(MuseSink sink);
}

The interesting thing about a combinator-based implementation is that, being in essence purely functional, it has a number of interesting properties:

The current implementation suffers however from some defects and is not fully functional

Actually, one would like filters to be much like a boolean algebra, closed under the usual operators: negation, intersection, union, difference, much like lists are in most functional languages, and like rational languages are.

20081015: Agile Tour 2008

Ça y'est, l'Agile Tour 2008 est passé par Lille ! En voici un bref résumé, plus quelques idées pour une prochaine manifestation.  

Les sessions

Nous avons eu pas mal de désistements, mais il faut dire que nous n'avons vraiment pas assuré côté relance: un mail tardif et une relance téléphonique la veille pour le lendemain. Seuls les gens que François et moi connaissions personnellement sont finalement venus.  

Les sessions proposées étaient donc les suivantes:

  1. Introduction à l'agilité, par François Wauquier
  2. Les projets Agile - de la réponse aux appels d'offres à la maintenance - retour d'expérience, par Oana Juncu, et  
  3. La démarche Agile: contexte favorable/défavorable
  4. Pratiques d'ingéniérie incrémentale, par Eric Mignot et Laurent Cobos, ainsi que  
  5. Spécifications exécutables avec GreenPepper, et
  6. Dojo Randori
  7. L'art du jonglage avec les pratiques XP pour le développement web / PHP, par Perrick Penet
  8. Opentime et ses 10000 tests automatisés, toujour par Perrick,
  9. Intégration continue & Offshore, par Thomas Recloux,
  10. Eclipse + Mylyn : Se concentrer sur une tâche, par Dimitri Baeli,
  11. et un atelier de Planification Agile par Patrice Petit.

Les sessions étaient découpées en créneaux de 30', plus 5' de temps élastique permettant de laisser aux gens de bouger.

Comme nous n'avions pas plus de sessions que de créneaux (18 au total), les participants ont "voté" en arrivant avec des gommettes pour nous permettre de répartir au plus juste les salles. La répartition des votes semblait indiqué en grand besoin d'information, plutôt que du pratiques, probablement parce que beaucoup de participantsn n'étaient pas des experts de l'agilité et venaient là pour découvrir, apprendre et surtout discuter.  

J'ai fait le planning après avoir introduit Agile Tour et les grandes lignes de l'organisation, tandis que les orateurs se présentaient, eux et leurs sessions.  

Les lieux

Nous avions donc trois salles:  

Les locaux de TELECOM Lille 1 sont très bien, mais la disposition des salles, très éclatées les unes des autres n'étaient pas idéale. Heureusement, le fléchage était bien fait et les salles quand même situées dans des endroits évidents, pas au fin fond d'une aile. Personne ne semble s'être perdu, ou du moins pas suffisamment longtemps.  

Les participants

Il y avait 85 inscrits, et il me reste 22 badges, donc j'en déduis qu'il y avait au moins 63 participants. EN réalité, il devait y en avoir légèrement plus puisque des gens non inscrits se sont présentés spontanèment.  

Les profils étaient très variés. Je n'ai pas de statistiques précises, dont le sens sur un échantillon de 60 personnes serait de toute façon douteux, mais j'ai perçu les grandes lignes suivantes:

Le déroulement

Tout le monde a salué l'organisation de l'événement, ce qui ne laisse pas de me surprendre car j'ai vraiment eu l'impression qu'on a fait ça à l'arrache et que c'était un peu... chaotique. Je pense, cela dit, que beaucoup ont apprécié le côté informel. Le groupe était de petite taille, la manifestation de courte durée et le lieu compact, donc l'improvisation pouvait suppléer à la préparation.

Je n'ai eu que de bons échos, aussi bien de la part des participants que des orateurs. Il faudrait avoir dépouiller les questionnaires de satisfaction pour en savoir plus. J'ai le sentiment que tout le monde a trouvé ce qu'il venait chercher ou a pu dire ce qu'il avait à dire.  

Je regrette un peu le manque d'ateliers et la faible participation au dojo présenté par Eric Mignot et Laurent Cobos. Il me semble pourtant de plus en plus que le développement de l'agilisme passe par là, que c'est un outil extraordinaire pour progresser et faire progresser. Comme si beaucoup de gens avaient un peu peur de se mettre en danger en codant collectivement, comme si le code était toujours quelque chose que l'on faisait solitairement.

Des idées pour plus tard

J'ai vraiment envie de trouver d'autres formes que la classique conférence où l'on vient écouter la bonne parole, confortablement assis dans un amphithèâtre. Il me paraît essentiel, partie intégrante des valeurs et des principes de l'agilité, que les participants contribuent activement au contenu de la conférence.  

Les ateliers sont un bon moyen de parvenir à ce résultat, et en discutant avec Marie-Aimée, nous avons imaginé d'autres types d'organisation:

20081015: On code quality and TDD

An old post by Michael Feather that talks about things I would like to present at SPA2009: different approach to code quality than TDD. Here it talks about the Clean Room method that, back in the 80's, enforced quality through formal expression of code behavior with logical predicates (presumably Hoare's logic predicates).

20081009: Web application - Part 3

Things are getting more and more complicated as I am closing in on the client-side of the web app. Thanks to some help from the haskell-cafe mailing list denizens,  I managed to got a running FastCGI web application working, with the shared state passed around, although this feature is untested.

The basic idea is to initialize the World once in a shared variable reference, preferably a thread-safe one, then to pass that value to the worker functions that could well be some IO threads.  

previous

main = do world <- newTVarIO initialState  -- initialize shared variable containing state
          runFastCGI  $ work world         -- service requests 


work :: TVar World -> CGI CGIResult
work world = do sid    <- fmap (maybe 0 id) $ readInput "sessionId"      -- extract session identifier
                sname  <- fmap (maybe "index" id) $ getInput "service"  -- extract service name
                world' <- liftIO $ atomically $ readTVar world                     -- extract state of the world
                setHeader "content-type" "text/plain"
                output $ service sname world'

I am presently struggling with the client side, trying to use jQuery and its Form plugin. I found the hard way that although they are somewhat equivalent, and sometimes use equivalently in HTML, id and name are not the same. jQuery selectors use id while form submission uses the name attribute. So one usually define both or use [name='toto'] selector to extract the needed  field.  

I am still stuck with the intricacy of submitting forms, between the subtleties of submit(), ajaxForm() and ajaxSubmit(), but I can now pass around values from the client to the server, and most notably the requested service and probably the session id if it is defined.  

I plan to make the application in client/server architecture, with the Haskell code providing services and JSON data, and the client code using jQuery to update the UI.

20081007: State Transformers again

I managed to refactor the code to use explicitly State and StateT monad, thanks to this thread on Haskell-café and this blog post. In the process I got to understand a few things about those dreaded monads and the way to combine them, although this obviously only scratch the surface of things.

The code of authenticate now looks like: previous


getUser :: String -> String -> IOWorld (Maybe User)
getUser login password = lift $ readResource (User login password)


authenticate :: String -> String -> IOWorld (Maybe Session)
authenticate login pwd = do u <- getUser login pwd
                            liftState (createSession [("unregister", unregister),("logout", logout)] 
                                       (checkPassword u))

All top-level actions are done in the IOWorld context which is a State monad encapsulating IO monad. We need IO because we are dealing with the persistence layer.

The tests look a lot nicer, although some duplication obviously persists: previous


aLoggedInUserCanUnregisterWithKey =
    "Logged in user can access unregister action with key" ~:
          (withDB "test.db" 
               (evalStateT (do register "nono" "password"
                               Just session <- authenticate "nono" "password"
                               let Just action = lookup "unregister" (actions session)
                               action session
                               authenticate "nono" "password") initialState >>=
                   assertEqual "unexpected user" Nothing))    

The sequencing of actions that the monad allows - and imposes - is not cluttered anymore by explicit arguments passing. The extraction of the action could encapsulated too in a nice monadic action.  

The important lesson about monads is: you can't mix apples and oranges. Monad sequencing is just a special kind of composition:

  1. ````simple functions f: a \rightarrow b and  g: b \rightarrow c are composed using .`` operator, yielding function  g . f: a \rightarrow c,  
  2. ````functions with effects f: a \rightarrow m b and  g: b \rightarrow m c (ie. functions within monads) are composed  using >> and <code>>>=</code> (ie. bind``) operators, yielding function f join g: a \rightarrow m c

The important thing to note is that both encapsulated type and monad type have to be compatible (ie. the same) for this to work. One cannot compose actions occuring in different monads without some kind of transformation which is just why monad transformers are useful: they allow one to unify different kind of actions/effects within a single layered monads.  

One important function in this context is the lift operator that allows, as it name implies, lifting functions operating within one monad to another layer of monad. This means that given a  IO a action, you can lift it to a StateT s IO a action by  lift, but given a State s a action, how do you lift it to StateT s IO a ? You need to apply the IO monad to the result of the  State monad before returning it.  

One solution given by the cited posts is to define a liftState function: previous


liftState :: Monad m => State s a -> StateT s m a
liftState s = do state1 <- get
                 let (result, state) = evalState (do result <- s
                                                     state <- get
                                                     return (result, state)) state1 
                 put state
                 return result

But there is another, more elegant, solution: use the interface MonadState to constrain the context of the createSession function and all other functions that operate at the state level and could need to be lifted at some other level. We can thus now write: previous


createSession :: MonadState World m => [(String,Action)] -> Maybe User -> m (Maybe Session)


authenticate login pwd = getUser login pwd >>=
                         (createSession [("unregister", unregister),("logout", logout)] . checkPassword)

I can understand this barely but will try to explain it:

20081006: StateT monad transformers and IO

While working on Haskell implementation for Colivri, I got stuck within the complexity of monad transformers, most specifically with the wiring of the State monad with the IO monad. The code uses state to store session informations when users log in and out of the system. For example, the authenticate function looks like:

previous

authenticate :: String -> String -> World -> IO (World, Maybe Session)
authenticate login pwd s = readResource (User login "") >>= 
                           return . (createSession s [("unregister", unregister),
                                                      ("logout", logout)] . checkPassword) 
     where
      checkPassword Nothing = Nothing
      checkPassword u@(Just (User uid pwd')) | pwd' == pwd = u
                                             | otherwise   = Nothing

The StateT is defined as

previous

newtype StateT s m a = StateT (\s -> m (a,s))

so I am tempted to rewrite authenticate as:

previous

type IOWorld a = StateT World IO a
authenticate :: String -> String -> IOWorld (Maybe Session)

The problem is that I am not able to write the code that would make this to typecheck or compile !

Otherwise, I managed to write a simple command-line interface through which I can login/logout and register/unregister users within the database. Next step is to wire that logic through FastCGI. But how do we maintain state on a FastCGI server ?

20081003: jsUnit and Callbacks

In a recent entry of Testing in the Toilets, a technique is described for testing asynchronous handlers tied to setTimeout et al. family of functions, available at the global scope. This is something I tried to do in Benefit for the scheduling code of test executors, but much better as here the passing of time is controllable.  

20081002: Web Application in Haskell  - Part 2

Database

I spent part of the morning switching from flat file storage to DB storage for the tiny little embryonic web app in Haskell I am working on. I managed to use Sqlite3 bindings for HDBC which is quite low-level but at least works smoothly once you recall how to code in SQL.  

Installation was easy once I removed the .cabal and .ghc directories from my $HOME

sudo aptitude install libghc6-hdbc-sqlite3-dev

I then made User type an instance of IResource and implemented the CRUD operations there: previous


import Database.HDBC
import Database.HDBC.Sqlite3


...
handleError str err = error $ str ++ ":" ++ ( seErrorMsg err) ++ ", code:" ++ show (seNativeError err)


instance IResource User where
    keyResource User {login = uid} = uid
    writeResource User {login = uid, password = pwd} = 
        handleSql (handleError "in write resource") 
                  (do con <- connectSqlite3 "test.db"
                      run con "insert into auth_user (username, password) values (?,?);" [SqlString uid, SqlString pwd ]
                      commit con
                      disconnect con)

The rest of the code is unchanged as I have abstracted operations on users using IResource interface.  

Unit tests are unchanged too, except for the setup and tearDown methods which now respectively create the necessary table and drop the table.

previous

createtableuser = "CREATE TABLE \"auth_user\" ("++
                  "\"username\" varchar(30) NOT NULL PRIMARY KEY,"++
                  "\"password\" varchar(128) NOT NULL"++
                  ");"


setupDB fn = 
    (do con <- catchError "connectinig" (connectSqlite3 fn)
        catchError "creating tables" (withTransaction con (\con -> run con createtableuser []))
        catchError "disconnectinig" (disconnect con))


tearDownDB fn =
    (do con <- catchError "connectinig" (connectSqlite3 fn)
        catchError "dropping" (withTransaction con (\con -> run con "drop table auth_user;" []))
        catchError "committing" (commit con)
        catchError "disconnecting" (disconnect con))

The reason I am catching errors for each statement rather than encapsulating everythnig at a higher-level is that I got strange error messages at first which I spent an hour tracing. The message was something like unknown error in sqlite3... and conducted to the base being locked. It was actually caused by incorrect parameters for statements: you don't need to put quotes around the ? when the expected parameter is a string.

So I am now ready to continue working on the business domain and implement the book manipulation logic.

Session handling

I refactored the code to add session handling: a user is tied to a session that is supposed to restrict the actions the user can do, and the sessions are all part of some state object that is updated and read by the various interface functions (ie. functions that corresponds to actions in the application).  

The code is not very nice, with lot of redundancy as I pass along the state object everywhere in the code instead of using it implicitly through the State monad

This code should be replace by something less ugly: previous


-- session
differentUserHaveDifferentSessionKeys =
    "Log two users and check for session key" ~:
          (withDB "test.db" 
               (do st               <- register "nono" "password" initialState
                   st'              <- register "tutu" "pass" (fst st)
                   (st'', Just s1)  <- authenticate "nono" "password" (fst st' )
                   (st''', Just s2) <- authenticate "tutu" "pass" st''
                   assertBool "session key not different" ((key s1) /= (key s2))))

The idea is that the Session object should hold a  list of allowable functions in the present state of the application. Later on, the Session shall have a time limit attached which will be checked periodically by some cleanup thread code.

In short, each session is a continuation for the current user, that when applied some action yield another continuation until termination of the interaction. The Back button problem is solved by defining the set state space independently of the pages so that whenever a user crosses some important threshold (eg. does some action that has a side-effect in the world and is not idempotent), this action is simply removed from the continuation. Getting back on the page and reactivating the link yields to an error.  

OTOH, it may be possible that back is not a problem, if the state includes several views embodied as pages.  

20081002: From data to functions and back

See this article, cited from Universe of Discourse  for an in-depth explanation of how data types and functions are related in FP. A data type is just a function type listing all constructors for the type.

previous

type ChurchPair a b = (forall c . a -> b -> (a -> b -> c) -> c)


churchPair x y = (\operation -> operation x y)
churchFst  p   = p (\x y -> x)
churchSnd  p   = p (\x y -> y)

A pair is constructed (by churchPair) from two elements (a -> b) and is a function that can apply a function of two arguments (a -> b -> c) to its two constituents. Here churchFst and churchSnd are destructors that can observe the values stored in the pair by decapsulating it.  

A robot whose coordinates are defined as (x,y) and which has a direction can thus be modelled as the type: previous


type Robot x y d = (forall r . x -> y -> d -> (x -> y -> d -> r) -> r)


makeRobot x y d = (\obs -> obs x y d)
posX r          = r (\ x y d -> x)
posY r          = r (\ x y d -> y)
direction r     = r (\ x y d -> d)

20081001: First web application in Haskell

For project COlivri, I am trying to create my first web application in Haskell, using HApps as a web application server and framework.

Installation

Getting HApps built and installed is not straightforward, at least when your GHC installation seems a bit broken as is my case (Ubuntu Hardy Heron, GHC 6.8.2). It took me approximatively 1 hour to got the first basic example compiled and running. Assuming a ghc compiler available (in my case 6.8.2), the right way seems to be:

  1. Install cabal, following instructions at Hackage's site
  2. Don't forget to update cabal's list of packages: 
    cabal update
  3. download HApps distribution following instructions on the site: 
    darcs get --partial --tag=0.9.2 http://happs.org/repos/HAppS-HTTP
cd HApps-HTTP
cabal install

This scenario implies that all packages got installed into $HOME/.cabal directory. There should be a way of installing everything system-wide, probably using some prefix command.  

I could not manage to start working with the HApps-Begin sample blog, which actually seems outdated and is not referenced anymore in the HApps main page. Looking at AllIn example is not really convincing that HApps is, as advertized on the web page:

A web framework for developers to prototype quickly, deploy painlessly, scale massively, operate reliably, and change easily.

Using FastCGI

I am turning towards a more barebone approach using CGI scripts. The idea anyway would be for the application to be structured as:

  1. static HTML pages providing basic layout and style
  2. Javascript for loading data and jQuery for manipulating the DOM and formatting everything
  3. Haskell CGI scripts as web services  

I decided to give a try to the approaches described here and here, which is certainly cruder than using a web application framework but at least everything in there I understand.

FastCGI is a replacement for Common Gateway Interface protocol where, instead of starting one new process for every invocation of desired script, the scripts are kept running within the context of a server so that there is no loading and linking time wasted.  

Installing FastCGI to run on apache 2.2 is easy: 

sudo aptitude install libapache2-mod-fastcgi
sudo a2enmod fastcgi
sed -i -e '/<\/VirtualHost>/ i\
\
    Alias /hcgi/ /var/soft/hcgi/\
\
    <Directory /var/soft/hcgi/>\
        SetHandler fastcgi-script\
        Options +ExecCGI\
    </Directory>' /etc/apache2/sites-available/default
/etc/init.d/apache2 restart

This makes directory /var/soft/hcgi/ a container for executable FastCGI programs (they should have extension .fcgi).

Then I copied a simple echo program that outputs process and PID id for each request: previous


import Control.Concurrent
import System.Posix.Process (getProcessID)


import Network.FastCGI


test :: CGI CGIResult
test = do setHeader "Content-type" "text/plain"
          setStatus 200 "OK" 
          pid <- liftIO getProcessID
          threadId <- liftIO myThreadId
          let tid = concat $ drop 1 $ words $ show threadId
          output $ unlines [ "Process ID: " ++ show pid,
                             "Thread ID:  " ++ tid]


main = runFastCGIConcurrent' forkIO 10 test

Compilation is: 

ghc -threaded -package fastcgi --make -o info.fcgi info.hs

Application development

Basic

I start by writing an authentication module. This module lets a user authenticate itself on a form providing a login/password combination, then lookup the user in some database. If the user is regristered, then it receives main page of the application with the ability to logout, otherwise it is show the login page again with an error message.

I want to work using continuations-based forms, without relying on cookies for user identification. All the information is stored in the server using a map from tokens to functions. Tokens have an expiry time which makes functions unavailable after a while.  

Storing resources

I found this blog that shows how to implement persistence layer for data objects à la Hibernate in Haskell. It describes a Transaction Cache that stores allows CRUD operations for abstract resources mapped to underlying persistence layer.

I try to implement a persistence layer using sqlite and colivri DB.

2008.10.01 18:24:28

I managed to implement login and registration logic, with unit tests, using simple file system storing as I could not manage to find a working HDBC sqlite3 driver to work. When trying to install the modules from cabal, they would not compile, complaining about missing identifiers which obviously is a problem. I may try to install them from sources directly.

The code so far looks like: previous


module Colivri where
import TCache
import DefaultResource


data User = User { login :: String 
                 , password :: String }
            deriving (Eq, Show, Read)


instance DefaultPersistResource User where
    defKeyResource User {login = uid} = uid
    defPath User {}  = "users/"


authenticate :: String -> String -> IO (Maybe User)
authenticate login pwd = readResource (User login "") >>= 
                         return . checkPassword 
    where
      checkPassword Nothing = Nothing
      checkPassword u@(Just (User uid pwd')) | pwd' == pwd = u
                                             | otherwise   = Nothing




register :: String -> String -> IO (Maybe User)
register login pwd = readResource (User login pwd) >>= registerIfNotPresent 
    where
      registerIfNotPresent Nothing = do let u = (User login pwd)
                                        writeResource u
                                        return (Just u)
      registerIfNotPresent _       =  return Nothing

The TCache module provides a simple CRUD layer where objects are stored with Show and retrieved with Read. This is actually a limitation of the DefaultPersistResource which stores data in flat files named by primary key. It should not be too complicated to implement something else as IResource interface does not rely on particular structure of data.

More interesting are the unit tests, partially shomn here: previous


import Control.Exception(bracket_)
import System.Directory(createDirectoryIfMissing,removeDirectoryRecursive)


setupDir :: FilePath -> IO ()
setupDir fn =
    createDirectoryIfMissing True fn 


tearDownDir :: FilePath -> IO ()
tearDownDir fn =
    removeDirectoryRecursive fn


withDirectory fn = bracket_ (setupDir fn) (tearDownDir fn)


unregisteredUserCannotBeAuthenticated =
    "Empty user tries to authenticate and fail" ~:
              (withDirectory "users" 
               (authenticate "" "" >>= assertEqual "user should not exist" Nothing))




registeredShouldBeAuthenticated =
    "Existing user tries to authenticate and succeeds" ~:
              (withDirectory "users" 
               (register "nono" "password" >>
                authenticate "nono" "password" >>= assertEqual "user exist" (Just $ User "nono" "password")))


registeredUserWithBadPasswordIsRejected =
    "Existing user tries to authenticate with wrong password and fails" ~:
              (withDirectory "users" 
               (register "nono" "password" >>
                authenticate "nono" "pass" >>= assertEqual "user exist" Nothing))

I found this article very useful. it provided me the necessary features to create setup and teardown methods. The key point is to notice that Assertion actually produces an IO value, and thus can be injected into IO monad code easily.

Here is a short list of resources I used today:

20081001: The long tail

Some mathematical musing about the illusions of normal distribution and how it affects our perception of average, even for that fraction of a population which is being selected from the above average  

http://blog.plover.com/2008/09/30/#right-skewed

It has some interesting consquences as far as programming and software development is concerned, and I particularly liked this sentence:

That's an important thing to know about the sport, and about team sports in general: you don't need great players to completely clobber the opposition; it suffices to have players that are merely above average. But if you're the coach, you'd better learn to make do with a bunch of players who are below average, because that's what you have, and that's what the other team will beat you with.

Do not confuse the average with the median !

20080930: More work on site feed

Working on displaying links from the site feed. The idea is that a link like /journal/2008/08/08/some-title should display only the selected entry if it exists, respecting the overall page style  of the site. The problem is: how to extract part of the file for markup ?  

The solution I came up with is to wrap the destination sink (usually HTHL) so that it receives only events within the start and end of the selected section. One obvious problem with this scheme is that sectioning is not hierarchical. Titles are just like hX header tags in HTML, they only denote some formatting change but not distinct levels. This could be changed easily by embedding sectionning within the muse parser, so that section state would last to next sectioning or end of document, whichever comes first.  

It is then easy to select only the events that pertains to some section as we can filter on the start event and then decide whether or not to pass all events to wrapped sink.

Another solution is to accumulate the markup events and pass them to downstream formatter only if selector's pattern is triggered.

20080929: Testing OQube web site

Started to write tests for OQube's site that could be run under CI to ensure proper operation of the site. First test written is a spider that checks all links collected from the web pages traversed by the site. This test uses code from Jeff Heaton web sites for a simple spider. For the moment, it is sophisticated enough for my needs.

I ran into a problem reported here while checking URL from Ubuntu web site using HTTPS. Looks like Sun's default cacerts file is missing some useful certificates.

I also want to setup some load and performance testing. The problem is how to setup a meaningful test for such a thing ? I looked at various tools for web application testing:

Might have a look at http://webflange.sourceforge.net/ too.

20080917: Interesting variation around Form validation problem

On a recent post, Ricky  CLarkson implements constraints on data fields using a builder pattern: fields are declared using a combination of filter (Parameter) objects that apply certain constraints. Then an object is constructed with Builder pattern that checks appropriate constraints and throws unchecked exception upon error.  

20080909: Dojo de lundi

Un dojo inspiré de Tony Morris: comment analyser une chaîne de caractères composée de paires de parenthèses et crochets équilibrés ?

20080909: Builder Tester agent

Started working on the tester agent, first laying out basic infrastructure for executing JUnit tests: wrapping JUnitCore executor so that test cases get detected by the framewok, while handling class loading issues.  

Test executions is OK, both for failures and successes, but I suspect reporting is rather lame. This shall be improved later on. My goal is to reach rapidly a state where:

20080904: Stateful closures in functional java

I read and compiled the code from http://lucdup.blogspot.com/2008/09/closures-statefulness.html using the latest java compiler integrating closures from http://javac.info.  

20080903: Handling CSV files for accounting purpose

I finally wrote a Haskell program for managing my ledger files and producing raw data for my accountant. The problem is simple: I use ledger program for bookkeeping, which is basically a flat text file containing all basic accounting transactions in double-entry format (a transaction is valid iff all atomic writings to its accounts sum up to 0, see http://en.wikipedia.org/wiki/Double-entry_bookkeeping_system).

Ledger program can produces various outputs, most notably the balance format, that outputs the net debit and credit value of each account taking into account all transactions; and the csv format which is all transactions as CSV, for easy inclusion in spreadsheets for example. There is no debit or credit notion in ledger, only positive and negative numbers.  

What I need is somewhat similar to CSV output, but with additional requirements:

So the basic steps for this is the following:

  1. parse the CSV input and create a list of transactions items, one for each line of the input. Each transaction item should contain the date, the account number and title, the debit or  credit amount ;
  2. sort this list by account number and date;
  3. group by account number;
  4. accumulate value for each group;
  5. output the items as CSV lines.

In short, we have the following structure for the main function:

previous

makeGeneralLedger :: [String] -> [String]
makeGeneralLedger =
          (concatMap displayCsv) .
          (map (accumulate amountZero)) .
          groupByAccount .
          sort .
          (map parseRawCsvLine)


groupByAccount :: [TxItem] -> [[TxItem]]
groupByAccount = groupBy (\ t t' -> accountNo t == accountNo t')


accumulate :: Integer -> [TxItem] -> [TxItem]
accumulate acc [] = []
accumulate acc ((TxItem d c p l an al am _) : ts) = (TxItem d c p l an al am (addAmount am acc)) : accumulate (addAmount am  acc) ts




displayCsv :: [TxItem] -> [String]
displayCsv ts = (map displayTx ts) ++ [""]


displayTx :: TxItem -> String
displayTx (TxItem d c p l an al am ac) = concatMap (++ ";") [(show d)
                                                            ,an
                                                            ,al
                                                            ,"(" ++ p ++ ")"
                                                            ,l
                                                            ,(debcred am ";")
                                                            ,(debcred ac ";")]

TxItem is a record data type that contains all the necessary information about a single transaction.

Version 1: Parsec

The first version I wrote uses parsec library for parsing the input. This is where I spent a lot of time, understanding how parsing with combinators worked and making lot of mistakes, out of ignorance and out of bad practice.

I tried to use TDD to do the parsing, which helped me a bit but did not produce a lot of tests. Tests are painful to write, like:

previous

canParseCSVTxItem =
    TestList [
              txItemParser txItem "\"2006/06/12\",\"Facture Norsys SAS\",\"706100:Produits:Services\",\"EUR-900,000000\",\"EUR-900,000000\",\"*\",\"F20060601\",\"\""
              ~?=
              Right (TxItem (toGregorian 2006 06 12) True "F20060601" "Facture Norsys SAS" "706100" "Produits:Services" (TxAmount Credit "EUR" 900000000) (TxAmount Credit "EUR" 900000000))
             ]

Main problems I had were:

Version 2 : Correct version 1

Developping version 1 I cut corners and it failed: The program computed incorrect values. Two things that I did not take into account where:

The interesting thing is that both errors are due to lack of verification of those parts of the code that just do this parsing, or in other words, I did not test enough my program before shipping, or worse I did not developed it using TDD, which would have hopefully revealed this problem before.

These problems are actually the two sides of the same coin, and what I really need is asserting that:

  1. given any correctly formatted decimal string, I can parse it then show it again and it will be the same string plus maybe some added zeros at decimal place for padding,
  2. given any Integer, I can show it then parse it again and it will be the same value.

The real problem, of course, lies in the any: These properties should be proved for all possible correct strings and all possible Integer values, while my TDD tests cover only a fraction of the possible values, selected for their representativity or their usefulness at the time of writing the test. The devils lurks in the details, so I must be careful while TDDing in choosing those values.

Quickcheck may help in solving those problems. I can express the two above properties using Quickcheck: previous


roundtripOnInteger = forAll generateDecimal $ \d -> 
                          (right . parseDecimal . showDecimal) d == d


roundtripOnStrings = forAll generateCorrectStrings $ \s ->
                        let s' = trimStrings s
                        in  collect (lengthOfDecimal s') $
                            (showDecimal . right . parseDecimal) s' == s

The second one is a little more complex as we are verifying the property for all strings where trailing zeros in the decimal part have been omitted, which means we first generate a correct version of the string, then trim it, then compare the roundtrip transformation to the initial input.

The collect (lengthOfDecimal s') statement allows us to collect some statistics about the distribution of the lenght of generated strings, which unsurprisingly shows that lenght 7 (including the leading comma) are vastly outnumbering other lengths, but those are not null: 

90% 7.
8% 6.
0% 5.
0% 4.
0% 3.

The 0% figures are due to some formatting issues of percentages in Quickcheck.  

Given that we generated 10000 test cases without failures, we can now be pretty confident that this part of the program is safe. We can have some information about what part of the code we exercised using hpc: 

ghc --make -fhpc -i/home/nono/soft/Calendar Accounting.hs QCTests.hs
hpc report QCTests
hpc markup QCTests

and here are the figures: HPC Html report We can see that the coverage of Accounting module which contains the main code is rather poor, which is normal as the quickcheck tests only exercise a small part of the reading and parsing of numbers.

20080903: Projets au forfait

La traduction d'un article de Scott Ambler sur les projets au forfait dans lesquels toutes les variables sont fixées: prix, délais, étendue.  

20080902: Maven reactor & dependency management

While working on a dependency management tool for Courtanet, I am trying to understand how maven resolves things. Given a POM file for some project, here are the following steps I could identify:

20080901: Parsec experiments

Tried to do some handling of ledger's CSV output using Haskell's parsec library. It took me a whole afternoon to manage to construct a signle transaction item as I had lot of problems:

The resulting code is ugly, following closely my experiments and progress in understanding Parsec's combinators. I suspect the time to implement loading into database will be similar...

20080827: GTTSE 09 announced

Maybe I could propose a short or informal tutorial software engineering practices that relates to generative programming:

20080827: SPA Conference proposal

This workshop could easily be a 300' full tutorial for the Sunday afternoon, or day-long workshop.

Title: More Bells and Whistles: Validating Code Beyond Unit Tests with Functional Programming

One-line description: Exploring validation and verification of code from an agile perspective with more than unit tests, or how can we improve quality, simplicity, legibility using tools from the functional programming paradigm ?

Session format:  150 Minutes Workshop/Tutorial

Abstract: Test Driven Development is one cornerstone of eXtreme Programming and its advantages are well-known: it guides developers, builds confidence in the system and provides secure scaffolding to support change. There are however some drawbacks in using this technique: it is quite verbose, it places to much emphasis on details, on the 'how' instead of the 'what', it produces more code that will need to be maintained and is not very useful to non technical people.

In this session, we want to explore techniques that keep the spirit of TDD while being more concise, provides a better abstract view of what we want to achieve, and may sometimes be more legible. These techniques are drawn from the functional programming languages tools and techniques. Functional languages are getting more and more attention and some of their characteristics may be useful in increasing software quality.

This workshop will be a hands-on guided coding party using the pure, lazy, statically typed language Haskell and providing introduction to some tools and techniques. Examples in other languages (Java, Scala, Javascript) will also be provided.

Its goal is to study and assess each of the following techniques and tools within the framework of XP practices in order to go beyond standard code quality improvement practices:

Audience background:

Benefits of participating:

Materials you'll provide:

Process:

  1. a "simple" yet realistic problem is presented: how to manage form/input validations in a uniform and legible way ?
  2. a solution is sketched in Haskell and needed infrastructure and process are set up,
  3. various kind of development techniques made possible with Haskell are introduced, explained and tried on this example, using part of the larger problem as motivation:
    1. TDD using property-based test case generation with QuickCheck,
    2. Proper composition of validations and definition of an expressive embedded DSL using (and abusing) the type system,
    3. proof of simple properties and complexity measures using formal reasoning on equations,
  4. applicability of each of this technique is assessed wrt to mainstream agile development practices

Detailed timetable:

Outputs:

History: I designed with Christophe Thibaut an introductory session on functional programming languages which has been presented at XP Day Paris and Agile 2008, and will be presented at XP Days Benelux 2008. This session is a creation that draws upon our experience on this theme, exploring more advanced topics.

Themes: Technology Practice

Session Leaders Arnaud Bailly

20080825: Reviving Builder project

While writing script for extracting commit stats from Hg such that one can produce a graph of volume change in a code base, I needed to upgrade my builder tool which contains some classes for handling programmatically in Java Mercurial repositories. The code did not work anmyore, if it has ever worked, a couple of tests failing. While digging through JCI API I finally concluded that it would be better to use standard tools provided by javax.tools.* package in Java 6.

As a side-effect, I was finally able to use closure proposal for Java and compile (mostly) functional java samples. This will allow me to start studying Reductio and how it applies to some standard code base.

I plan to revive builder and quickly adding the two most needed tools:

20080825: Presentation of Spec#

20080825: Phantom types

A type parameter used in a datatype declaration that does not appear in the rhs (ie. in the constructors) but serve as a way to separate various instances of the same types at compile time. Can be used to:

20080825: SPA 2009 Session Proposal

Test Driven Development is one cornerstone of eXtreme Programming. The red-green-refactor cycle is the smallest form of feedback the developer can receive. The advantages of using TDD are well-known and may be summarized as: first, it gives the developers a strong guidance on what to do that together with the heartbeat nature of cycles provide confidence and ultimately good code; second, it incrementally builds a kind of scaffolding for software that gives strong assurance when one needs to change anything in the system.

There are however some drawbacks in using this technique: it is quite verbose, it places to much emphasis on details, on the 'how' instead of the 'what', it produces more code that will need to be maintained and is not very useful to non technical people (this problem arises most often when trying to introduce the TDD practice as lot of managers, and all customers, cannot easily appreciate the value it provides without delving into code and may dismiss it as some mundane developer practice that we can easily get rid of when in a hurry, while actually it stands at the heart of XP).

In this session, we want to explore techniques that keep the spirit of TDD: build incrementally a software's safety net, while being more concise, provides a better abstract view of what we want to achieve, and may sometimes be more legible. These techniques, drawn from the vast array of formal methods and programming languages research and development, are not new, but most often are confined in some particular contexts, eg. embedded and human-life critical software, research groups.  

We believe that two parallel movements we see are occuring in the software development business may be signs that the time is ripe for introducing more formal techniques in the mundane world of IT: the widespread adoption of some core agile practices such as unit and functional automated testing, driven by the need for better quality software; and the rise in interest for functional programming languages such as Scala, F#, Haskell or functional aspects of OO languages: closures, value-objects, side-effect free computations.

We are thus proposing this workshop as an exploration into languages and tools for better quality software, applied to some short yet representative problems. This workshop will be a hands-on guided coding party using Scala or Haskell language and providing introduction to some tools:

The goal of the workshop is to study each of these techniques and tools within the framework of XP practices, in order to assert which, if any, could supplement, complement or replace standard TDD practices.

This workshop could introduce some of the following features:

20080724: Connectique disque mobile

Attention à la connectique des disques transportables, elle doit résister aux épreuves d'un transport en sac à dos.

20080724: Corrections format trac pour muse

20080721: Dojo on Systems Thinking

System thinking is about modeling systems as variables (observables) with constraints between them. More formally, a dynamic system is  a set of differential equations defining the behavior of a set of variables over time.

This kind of system can easily be represented as labelled digraphs where each node is a variable and each edge is labelled with some coefficient, the value of the variable at any point in time being the sum of all its incoming edges coefficient time the opposite's vertex value.

Equivalently, the system can be represented as a square matrix where each row is a variable, with value in cell defining the coefficient for the variable at this column.

We want to simulate the evolution of such a system, with a discrete time: state of the system progresses in discrete equal time increments. While the former representation is easier to understand, the latter is better suited for easy calculation of the state of the system, which is just matrix multiplication.

20080718: Haskell networking

Networking in Haskell is just as magic as actors: one it compiles, chances are high that you end up with a running program.

previous

-- a server that listen for connection on some socket then
-- dispatches the content to actors according to message id
main = do sock <- listenOn (PortNumber $ fromInteger 6789)
      forever $ answer sock
          where
          answer sock = do (hdl, host, port)  <- accept sock
                   forkIO $ server hdl

20080717: First actors

Some experiments on actors and concurrency with Haskell. My first actor program : previous


module Main where
import Control.Concurrent
import Control.Concurrent.Chan
import System.IO


-- simple messages
type Msg = String


-- an actor has an idea, an associated channel and some action
-- performed when receiving messages
data Actor = MkActor Int (Actor -> Msg -> IO ()) 


run :: Chan Msg -> Actor -> IO ()
run chan a@(MkActor id hdl)  = do msg <- readChan chan
                  hdl a msg
                  threadDelay 1000




handler :: Actor -> Msg -> IO ()
handler (MkActor id hdl) msg =  putStrLn $ "actor " ++ show id ++ ": handling " ++ msg


main = let
       channels = [ newChan | id <- [ 1 .. 10]]
       actors   = [ (MkActor id handler) | id <- [1 .. 10] ]
       -- read a command from stding
       -- xx some string
       command  = do line <- getLine
             return $ splitL line
       splitL :: String -> (Int, String)
       splitL (x : ' ' : rest) = (read (x : ""), rest)
       splitL _                = (0,"")
       loop chans = do (i,s) <- command
               case i of
                  0 -> return ()
                  x -> do writeChan (chans !! (x - 1)) s
                      loop chans
       in
       do -- create 10 channels
          chans <- sequence channels
      -- create 10 runners
      runs  <- mapM (forkIO . uncurry run) (zip chans actors)
      -- loop over input line
      loop chans

This program is definitely ugly, however, and does not do justice to the terse style usually available in haskell. I still have some things to understand when working in a monad. Channels and actors creation is especially tricky:

There should be a simpler way to assemble those things.

Something really extraordinary is that, once I managed to compile that program, iet just worked as expected ! I had no previous exposure to doing I/O within Haskell apart from whaat I have read, neither did I have any clue about Control.Concurrent.Chan, but it was quite easy to assemble things up.

This is a tribute to Haskell's type systems (and a bit of my own experience) that prevents you from writing totally silly expressions. At the very least, you have the assurance that your program will run, may be not in the expected way, but it won't crash haphazardly.

20080715: Comptabilité pour le code

20080715: Réinstallation serveur

Nom   IP Utilisation Port naté
public 3 serveur mail entrant et sortant 25,993
web 4 serveur web public 80
dev 2 serveur de développement 443

Public

TODO

Web

Dev

20080821: Example of bad coding

An example method that constructs some UI object. The intent is expressed as comments in the code: previous


    /**
     * Construction de l'etape(initialisation de la vue)
     * 
     * @param styleStep
     * @param styleContent
     */
    public FlexTable initialiseView(String styleStep, String styleContent) {
        // on la construit
        view = new FlexTable();
        view.setStyleName(styleStep);
        // 1. header
        view.setWidget(0, 0, buildHeader());
        // 2. contenu
        if (null == form) {
            form = initForm();
            form.setStyleName(styleContent);
        }
        view.setWidget(1, 0, form);
        // 3. footer
        if (hasFooter()) {
            displayFooter();
        }
        return view;
    }

We can rephrase it, hopefully in much more legible way thus:

previous

    public FlexTable initialiseView(String styleStep, String styleContent) {
        FlexTable view = new FlexTable();
        view.setStyleName(styleStep);
        buildTableHeader(view,0);
        buildTableBody(view,1);
        if(hasFooter()) displayFooter(view);
        return view;
    }


    private void buildTableHeader(FlexTable view, int pos) {
        view.setWidget(pos, 0, buildHeader());
    }


    private void buildTableBody(FlexTable view, int pos) {
        if (null == form) {
            form = initForm();
            form.setStyleName(styleContent);
        }
        view.setWidget(pos, 0, form);
    }


    private void maybeDisplayFooter(FlexTable view,int pos) {
            view.setWidget(pos, 0, buildFooter());
            view.getCellFormatter().setStyleName(pos, 0, AbstractStyles.FOOTER);
    }

This is not perfect as we can see that, for example, buildTableBody method initializes some form in the enclosing context while at the same time modifying the passed FlexTable view.

20080820: Jeux de langages et test

La tour de Babel

Ce court trajet nous a permis enfin de mettre en évidence l'importance de la question du discours et son irréductibilité: on ne peut pas tout dire, et sur ce dont on ne peut rien dire, il faut se résigner à ce taire. Cette position centrale du discours, des niveaux de discours, des métalangages est un leg du postmodernisme. Le sens n'est pas un produit univoque consubstantiel à un discours et qui serait produit de bout en bout par un locuteur, c'est une relation, un processus en constante évolution.  

Le processus de développement d'un logiciel doit donc être envisagé

20080820: Get Feedback

http://www.io.com/~wazmo/blog/archives/2008_08.html#000285

20080818: Agile 2008 - Compte-rendu

Chansons Françaises

Autres sessions

Dojos

20080807: Options in Java

A cool hack to have Maybe (or Option) wrapper in Java: http://stephan.reposita.org/archives/2008/08/06/for-hack-with-option-monad-in-java/

20080804: Agile 2008 - Day Zero

First real day at Agile 2008, starting with the AA-FTT workshop. Workshop was in OpenSpace format (sort of). I like this format: much more dynamic, gives plenty of rooms for interaction, surprises, works great with small rooms. I was both a bit tired and a lot impressed by the attendance, comprised of highly renowned and respected people in the agile testing community like Brian Marick, Elizabeth Hendrickson and many others, so I did not contribute a lot.

Some topics I picked up through the workshop:

20080723: Le héros post-moderne

J'ai commencé à m'intéresser à l'informatique à peu près au moment où le cyberpunk, comme genre littéraire, naissait.  

Profondément enfoui dans mon subconscient se trouve donc la conviction que l'informaticien est LE héros post-moderne, et conséquemment que l'informatique, ou plus précisèment l'activité de programmer des ordinateurs et aussi celle d'assembler des machines qui lui est souvent associée, sont des activités, des processus, éminemment post-modernes.  

Le héros post-moderne se méfie des grands récits, il est paranoïaque.  

Il est même parfois un peu schizophrène, quand il lui arrive de détourner le système à ses propres fins, car qui sait si ce n'est pas le système qui le détourne ? La matrice n'est-elle pas que l'espace utilisateur d'un gigantesque OS, et en déchirer le voile ne serait-il pas accéder à l'espace noyau ? Auquel cas, l'envers du décor, c'est toujours un autre décor, et non pas la réalité.  

Le héros post-moderne est profondément libertaire. Il n'aime pas le pouvoir et n'apprécie les règles et les lois que lorsqu'elles sont révocables à merci. Il peut adopter un mode de vie strict, voire ascétique, dans un but de dépassement de soi, jamais par dévotion envers une religion.  

Le héros post-moderne est Agile. Il est pragmatique dans sa pratique, et idéaliste dans ses objectifs. Il évolue dans des organisations qui sont idéalistes dans la pratique qu'elles imposent, et pragmatiques dans les buts qu'elles poursuivent.  

Le héros post-moderne sait que la chair est triste, et il a lu un peu trop de livres. Il n'en aime pas moins passionnément la vie et la connaissance.  

Il sait que la révolution doit être souhaitée, mais qu'elles finissent toutes dans le bain de sang et la dictature. Il institue donc des micro-révolutions là où c'est possible, trace des lignes de fuite. Sa stratégie est celle du rhizome. Son horizon la totalité du réel.

Le héros post-moderne aime XP car XP a été conçu par un héros post-moderne, pour d'autres héros post-modernes. Il n'est pas un solitaire et n'a aucune appétence pour l'érémitisme. Au contraire, il recherche la compagnie de ses semblables, car il sait que le sens est dans la relation aux autres, pas dans le solipsisme.  

Le héros post-moderne ne se désole pas de la récupération de ses fétiches par la société spectaculaire, car il sait que c'est le lot de toute chose, et la seule chose que sait faire le spectacle. Il est lui-même un grand récupérateur, assembleur, bidouilleur, sampleur, mashupeur... Nomade et pragmatique, il change de terrain de chasse quand l'herbe a été trop remâchée.

Je ne suis pas ce héros, mais il ne me déplairait pas que je le fusse.  

20080714: Installation serveur

Vserver script

Created vserver creation script:

DNS

Created zone files for oqube.net

cat >> /etc/bind/db.oqube.net << EOF
$TTL 3h
oqube.net.  IN SOA  master.oqube.net. adminsys.oqube.net. (
        1   ; serial
        3h  ; refresh
        1h  ; retry 
        1w  ; expire
        1h ); negative cache TTL


oqube.net. IN NS master.oqube.net.


master.oqube.net. IN A 192.168.50.1
dev.oqube.net.    IN A 192.168.50.2
EOF
cat >> /etc/bind/db.192.168.50 << EOF
$TTL 3h
50.168.192.in-addr.arpa.  IN SOA  master.oqube.net. adminsys.oqube.net. (
        1   ; serial
        3h  ; refresh
        1h  ; retry 
        1w  ; expire
        1h ); negative cache TTL


50.168.192.in-addr.arpa. IN NS master.oqube.net.


1.50.168.192.in-addr.arpa. IN PTR master.oqube.net.
2.50.168.192.in-addr.arpa. IN PTR dev.oqube.net.
EOF
cat >> /etc/bind/named.conf << EOF
// zone files for oqube.net. virtual network
zone "oqube.net" {
        type master;
        file "/etc/bind/db.oqube.net";
};


zone "50.168.192.in-addr.arpa" {
        type master;
        file "/etc/bind/db.192.168.50";
};
EOF

20080712: Haskell

2008.07.12 19:42:58

Tried using hpc. Works nicely and generates clean if somewhat unappealing coverage reports. Very simple to use, is integrated in ghc6.8.2.

Tried installing HaRE tool for Haskell Refactoring on Ubunte 7.10 with ghc 6.6.1. Two libraries are needed for proper compilation of the snapshot:  

#provide networking
sudo aptitude install libghc6-network-dev
# providing Control.Monad.State
sudo aptitude install libghc6-mtl-dev

Tried to use the tool on Controles.hs but failed to do anything:

Unit tests does not work either:

20080703: Sur les jeux de langage

../services/agile/postmodern un article en cours