Thursday, June 5, 2014

Haskell FFI Example

There are a few examples and tutorials that are out there on the Foreign Function Interface (FFI) in Haskell but I personally feel that you can never have too many examples. This past week I was working on calling C functions from Haskell, and this post will walk through a few examples and will hopefully warn you of some pitfalls that I ran into.

Why Use FFI? 

Here are a few reasons why you would want to use FFI:
  • You are looking for a library to solve a particular problem and you can't find any packages in your current language that would solve it well, but you find a tested library in a different language.
  • The implementation of a library in your current language is inefficient when compared with another language's implementation
  •  You really love a particular language and have to use it in any project that you touch because let's face it, that language is awesome
  • You want to learn a FFI...
I haven't played around with calling Haskell from C much so I'll leave that part for another day. Without further ado, here is an example of using Haskell to call C functions:

Example Background 

I was working on a project that used Xen's Hypervisor. Within Xen there is a communication mechanism called XenVChan that allows communication between two virtual machines on the same hypervisor. The implementation of this mechanism is all in C. With the communication we chose to use Haskell in order to give us the ability to have typed packets in our communication as well as all the other fun things Haskell can do. After glancing through the C source code I quickly realized that we were not going to want to re-implement VChan in Haskell. That leads us to using FFI. 

Here is an example of a C function that I would like to call from Haskell:

int readClientMessage(xentoollog_logger * xc_logger, struct libxenvchan * ctrl,
                        char * msg, int * sz);

We have a few things to figure out:
  1. How do we call a C function from Haskell? - Our original general question
  2. How can we create variables in Haskell that have types that C will accept when calling the function? 
  3. How do we deal with pointers?

Baby Steps - Calling a function in C from Haskell

Let's work on a more trivial C function first:

# Functions.c
int add( int x, int y);

This function simply adds x and y together and returns the result. To set up plumbing between Haskell and C Here's what we do:

#Haskell_Functions.hs
import Foreign
import Foreign.C.Types
import Foreign.C.String

foreign import ccall unsafe "functions.h add" c_add :: CInt -> CInt -> CInt

We just created a new haskell function call c_add that has the type signature CInt->CInt->CInt. Here is what the syntax is:
foreign import ccall  [safe/unsafe] [HeaderFile funcName] [HaskellFunc]

safe - the C function may call back into the Haskell program before returning. This is basically a flag to tell Haskell that it needs to do the extra book-keeping necessary to accept a returning call from this function.
unsafe - the C function will not call back into the Haskell Program before returning

If unsafe/safe is not specified it will default to safe.

The next parameter is a string which specifies the location of the C Header file and the C function name that you are wanting to call. This can be an absolute path or a relative path ("../include/functions.h" would be valid).

The last parameter is the name and type signature of the function that your Haskell code will call. So in this example my Haskell code can call the C function by calling c_add.

Now to call c_add we just need to figure out how to make a CInt

C Types

By importing  Foreign.C.Types we can create a large number of C primitives, both signed and unsigned. The full list can be seen in the documentation: Foreign.C.Types.

In order to call c_add we need to create some CInt variables in haskell. we can do so in the following way:

fromIntegral 5 :: CInt 

My personal preference is to create another Haskell function that abstracts away all the marshalling needed between Haskell types and C types:

#Haskell_Functions.hs
myAdd x y = c_add (fromIntegral x :: CInt) (fromIntegral y :: CInt)

And that's it! A few other helpful functions:

fromRational  5.5 :: CDouble
castCharToCChar 'c'      -- == fromIntegral (ord 'c') :: CChar 
newCString "Hello World" -- String -> IO CString

Handling C Pointers in Haskell

Our original function needed an int * and a char * To allocate space for a CInt we have the following function:
alloca :: Storable a => (Ptr a -> IO b) -> IO b
Let's pretend that our C_add function takes a third parameter that is an int * to store the result: # Functions.c
 int add (int x, int y, int *res);

and that we did the corresponding  foreign setup calls: 

#Haskell_Functions.hs
foreign import ccall unsafe "functions.h add" c_addWithPtr :: CInt -> CInt-> Ptr CInt -> CInt

Here is how we would allocate the pointer and then look at the result

#Haskell_Functions.hs
myAddPtr :: Int -> Int -> IO Int
myAddPtr x y= alloca $ \ res -> do 
                          c_addWithPtr (fromIntegral x ::CInt) (fromIntegral y::CInt) res
                          return $ fromIntegral (peek res):: Int

We allocate a ptr called res we then call the function and afterwords we look at the result by calling peek to dereference the pointer and then return the Integer. If you needed to allocate more than one pointer then you would just nest the alloca calls.

Pointers to Data Structures

Let's look again at the opening C Function:
# Functions.h
int readClientMessage(xentoollog_logger * xc_logger, struct libxenvchan * ctrl,
                        char * msg, int * sz);
In this function we are passing around pointers to data structures instead of just pointers to primitives. In my case I never needed to look at the contents of the data structures I just needed a handle to pass these to different functions. I also had some functions that would return a pointer to the structure.  If you need to look at the values of the structures inside your Haskell program then I would recommend that you look at the "Working with C Structures" section here
If you don't need to look at the data structure in Haskell then I would recommend doing the following in your Haskell Code:
#Haskell_Functions.hs
{-# LANGUAGE ForeignFunctionInterface, EmptyDataDecls #-}

data XenToolLogger
data LibXenVChan


You can then pass around Ptr XenToolLogger and have then bound to a variable. Here are some functions that return pointers to data structures :

# Functions.c
xentoollog_logger * createDebugLogger(void);
struct libxenvchan * createReceiveChan (xentoollog_logger * xc_logger, int id);

The corresponding Haskell code to call these would be:

#Haskell_Functions.hs
foreign import ccall unsafe "../include/exp1Common.h createDebugLogger"
    c_createDebugLogger:: IO (Ptr XenToolLogger)

foreign import ccall unsafe "../include/exp1Common.h createReceiveChan"
    c_createReceiveChan:: Ptr XenToolLogger -> CInt-> IO (Ptr LibXenVChan)

createSrvCtrl :: Ptr XenToolLogger -> Int-> IO (Ptr LibXenVChan)
createSrvCtrl logger clientId = c_createreceivechan logger clientId

Now we just need to implement that initial function and we are done:

#Haskell_Functions.hs
foreign import ccall unsafe "../include/exp1Common.h readClientMessage"
    c_readClientMessage:: (Ptr XenToolLogger)->(Ptr LibXenVChan)-> CString -> Ptr CInt->IO (CInt)

readClientMessage :: (Ptr XenToolLogger)->(Ptr LibXenVChan)->Int->IO (Int)
readClientMessage logger chan dataSize = allocaArray0 dataSize $ \( ptr) ->
                    alloca $ \(size) -> do
                                    poke size (fromIntegral dataSize:: CInt)
                                    c_readClientMessage logger vchan ptr size
                                    sz<- peek size
                                    response <- peekCStringLen (ptr, fromIntegral sz:: Int)
                                    return $ response


There is a lot happening here and quite a few new things, let's walk through it:
  • We allocate an array using allocaArray0
    • allocaArray0 is just like alloca but it takes an Int as and allocates an array of that size +1 (the +1 is to make room for a \NUL character, if you don't want it use allocaArray instead)
  • We then allocate a pointer to a CInt
    • poke is a function that takes a pointer and a value and puts the value in the pointer
  • We then call our c function with the marshalled values and the string
  • Using peek we can dereference the pointers and look at the values
  • Using peekCStringLen we can look at the String  of a certain size
    • peekCStringLen will allow null characters in the string, if you want to get up until the null you can just use peekCString and it will give you everything until the first null character
  • We then return the value

Side Notes:

When calling the foreign import ccall  method and giving the Haskell type if the C function is pure you can give the Haskell a type that isn't IO (see the initial c_add function) but if it isn't a pure function you can include IO.

If a c function is asking for a char * as an argument you need to check about how it is used. If the function is allocating memory and then just setting your pointer to the front of it, then you can just do an alloca call instead of an allocaArray call.

I mentioned it in passing when talking about peekCStringLen, but don't forget that strings in C are nul terminated, whereas Haskell strings are not. This can lead to some "interesting" bugs if you are dealing with ByteStrings or other strings which have null characters in the string.




Thursday, May 8, 2014

Tab Complete Is Awesome!

Tab Complete is AMAZING and will literally save your life!

Ok maybe not literally but still this is something that anyone working on the command line should know!

When you are on a command line and you start to type a command you can hit the tab key once and if there is only one match it will complete what you are typing. If you hit the tab key once and nothing happens you can hit it one more time and it will show you the current matching possibilities. This is a built in feature! For example lets say we have a directory that has the following contents:  2 directories: 

dir1 whatWasIThinkingWhenIPickedSuchALongNameForADirectory 

and a couple of files:

file1  file2 file3

if you were wanting to change directory into the second directory, you would normally have to type out the following:

cd whatWasIThinkingWhenIPickedSuchALongNameForADirectory

Using Tab completion you can type:

cd  w

And then hit the Tab key and it will fill out the rest of the directory for you!
if you just type cd and then hit tab, nothing will happen. If you hit the tab key twice it will show:

dir whatWasIThinkingWhenIPickedSuchALongNameForADirectory 

Notice that it doesn't show the files with it, just the directories. When you hit tab after cd it knows that the command cd takes a directory so it filters out the files. If your current directory had only 1 directory you can type cd then hit Tab and it will complete the directory.

Tab complete works with all commands! If there were multiple matches then Tab complete will complete the item that you are typing until there is a conflict. If you look at our first environment with 3 files and two directories as an example, you can type:

cat f

Then hit tab. This is equivalent to saying, "I want to see the contents of a file that starts with an 'f' but I can't remember what I called it". Since we have 3 files that start with an f and the only difference in the name is the number at the end (file1 file2 file3), hitting tab will then fill in as much as it can.

cat file

And then you can add the number. When is tab complete most useful? All the time! Especially when you are moving around deep directories. Let's say I have a bunch of directories dir1 dir2 .... And one of them has a directory called secretTreasure the reset are empty. I want to move into the secretTreasure directory, because let's face it treasure is awesome. Without Tab complete you will have to cd dir1 and then ls to see if the right directory is there. Then go to the second directory and continue. Or with Tab complete you can type cd dir1 and then hit tab once if it doesn't show up then change the 1 to a 2 and hit tab again, rinse and repeat until you find it, saving tons of key strokes.

I use this all of the time especially if I am going through multiple levels of directories to do something:

cp Downloads/folder/fileThatIWant.txt Documents/classes/eecs101/lectureNotes/

Hitting Tab at each level ensures that I am at a place that exists and that I haven't misspelled the directory name, or messed up the capitalization. If I start to type a directory and hit tab and nothing happens I can hit tab again, if nothing shows up then I know that I am not where I should be, so I remove what I was typing and hit tab twice and see what my options are.

Saturday, May 7, 2011

Sat Work

So today I was able to get FarmAide to stop working when you are out of gold, it will also stop plowing when you aren't able to pay for planting the plowed plots, and then plant as much as you can, that way you can still have an income. Also changed sleeping message to not have the minutes and seconds, we are already saying the time and day that we will be waking up, why does the user need to know that it happens to be 239.1398501293 minutes? Also continued testing with the sleeping and waking up.


Wednesday, May 4, 2011

Team TANIS

Stood in front of the slides for most of the presentation


Good demo but it was hard to see with the projector setup, slanted on the wall, with the whiteboard.

Awesome to answer the question with the next slide.


Monday, May 2, 2011

Class Match

Um/uh count: first presenter about 5, second presenter: 26

First presenter kept walking in front of the projector,

Second presenter relied a lot on the slides, was always looking out of the corner of his eyes to see the slides

TaskPoet

I'm counting um's/uh's in the presentations : taskpoet had about 9 during the presentation, most were during the beginning, understandable for a nervous presenter.

They took 3 minutes to setup.

Some feedback:

Presenter said "Here's a quote" - we can see that it is a quote it has quotation marks and an author of the quote.

After reading the quote the presenter downplayed the whole reason for having the quote.

the quote was to point out that chrysler spends more money on RMI than on steel.

He then says that it is probably because they don't use much steel anymore.

In the question don't try to make something up or dance around the question if the answer is no.

I'm not sure that there was a slide about future work

Thursday, April 28, 2011

Project Beautify

So today we had a meeting and we did a couple of dry runs of the presentation. No one seems to be jumping up to volunteer for the presentation so basically everyone is going to try and present and we will choose who we want to present to the class and to the angels. Today Jacob and I presented, Ryan did our first iteration presentation, and Amr did the second iteration. John is going to present tomorrow to our professor.

One of the things that we noticed at the meeting is that the previous sprint we really didn't have cut and dried requirements, or tasks to go after because we were focused on the presentation and getting that through. Because of this we saw a dramatic drop in our effectiveness as a team. We did get some things done, but I think we could have been better. But since we slacked off on this assignment I finally had time to work on some assignments for other classes.

Today I was able to get the stats to update and give us a delta of what the automation software accomplished in the session. I also worked on making the GUI look a little more attractive. This was mainly by putting our FarmAide logo and a background. It took a while to figure out how to get the font color for the labels and to load an image in, but I am pretty happy with the way it turned out. I wasn't able to get the groupBox titles and the checkboxes to change font the same way that I did the labels, so I will have to see if I can figure out how to do that. I also changed the axis of the gold graph to not include the year, to remove some clutter and to allow the Time axis label to be shown.

That's about it for today, we will see how the meeting goes with Joe.