📦 Intro

Posted on May 24, 2018

I was fortunate enough to be accepted for the Haskell foundation for this year's Summer of Code, and the project I will be working on is with the Haskell IDE Engine. From the repository description, the Haskell IDE Engine (hie) is the engine for Haskell IDE integration. Most interestingly, it acts as a server for the Language Server Protocol so it can provide rich Haskell support for any IDE or text editor that supports the protocol. It can give diagnostics, refactor code, search document symbols and tons of other neat stuff. I'll be working on a test framework that can simulate a session interacting with a client. You can read more about it here.

haskell-ide-engine

The project kind of acts as the glue between LSP and the various coding tools in the Haskell ecosystem. It has a plugin API that's used for integrating fan favourites such as ghc-mod, hlint and HaRE.

It's spread across multiple repositories:

  1. haskell-ide-engine provides the I/O and communicates between the client and ghc-mod, as well as any plugins such as HaRE or brittany
  2. haskell-lsp (and haskell-lsp-types) contain definitions for functions and types according to the LSP specification
  3. ghc-mod is a separate tool that provides most of the analysis and diagnostics, but its tightly coupled with HIE.
  4. haskell-lsp-client is a library for LSP clients that my mentor Alan pointed out, we plan to use it as a starting point for #5.
  5. haskell-lsp-test will soon be the testing framework!

IRC

I've mostly been communicating in the #haskell-ide-engine room on freenode. It's been a while since I've used IRC, but I got round to setting up and running an IRC bouncer on my server (ZNC). Launching Colloquy was a blast from the past, complete with pre-retina icons. But as it turns out it's still actively developed on GitHub!. It's no longer distributed on the website, so I cloned the repository, created an archive with Xcode and then moved the .app to /Applications.

Blog

At the moment I'm using this makeshift bash script to blog:

cat header.html > index.html
for md in $(ls -tr *.md); do
  markdown $md >> index.html
done
cat footer.html >> index.html

Setup

It took me a while to get the development environment set up.

I started off by using VSCode and vscode-hie-server for the client. There was an issue on master that caused the LSP parser to fail with VSCode, so I created a pull request for it.

But I'm mainly a Vim user, so I tried out vim-lsc and then eventually settled for LanguageClient-neovim which seems to support more LSP features. Here's my current ~/.vimrc bindings for it

let g:LanguageClient_serverCommands = {
      'haskell': ['hie', '--lsp', '--debug', '-l', '/tmp/hie.log', '--vomit']
      \ }

nnoremap <silent> K :call LanguageClient#textDocument_hover()<CR>
nnoremap <silent> gd :call LanguageClient#textDocument_definition()<CR>
nnoremap <silent> gr :call LanguageClient#textDocument_rename()<CR>
nnoremap <silent> ga :call LanguageClient#textDocument_codeAction()<CR>
nnoremap <silent> gs :call LanguageClient#textDocument_documentSymbol()<CR>
set completefunc=LanguageClient#complete

At one point I ended up getting strange errors from HIE when running it on haskell-lsp and haskell-ide-engine (very meta):

hie: <command line>: cannot satisfy -package-id HaRe-0.8.4.1-inplace: 
    HaRe-0.8.4.1-inplace is unusable due to shadowed dependencies:
      base-4.11.1.0 Strfnsk-StrtgyLb-5.0.1.0-e162e946 cabal-helper-0.8.0.3-inplace containers-0.5.11.0 directory-1.3.1.5 ghc-8.4.2 ghc-xctprnt-0.5.6.1-3f0c080b ghc-mod-core-5.9.0.0-inplace hslggr-1.2.10-e253fcf2 mnd-cntrl-1.0.2.3-90183ebd syb-0.7-652252ef syz-0.2.0.0-ae4391c5
    (use -v for more information)

After two days of repeated rm -r ~/.stack .stack-work, I learnt two important lessons when working with stack projects:

  1. Double check with ls -a to make sure there are no dist, dist-newstyle, .cabal.project.local or .ghc-environments lying around. These will fool ghc-mod into thinking into using cabal instead of stack.
  2. You need to use a hie that was compiled with the same GHC version as your stack resolver. Turning on the hie wrapper in VSCode can automatically find it for you, otherwise you will need to specify it yourself.

But once I got hie working, it was amazing. Being able to rename variables with two keystrokes, apply quick-fixes straight fresh from hlint and jump to symbols with fuzzy finding all from vim was a glorious feeling.

Unfortunately this did not last for long as the second time I launched vim I got this delightful bug. It's an issue that lies all the way within ghc and only affects macOS on 8.4.2. It looks like it won't get fixed till the next 8.6 release either, which is due around August. It only affects modules that use the PatternSynonyms language extension, which haskell-ide-engine uses. I'm still trying various linker flags to see if there is a workaround, but for the meantime it means that I can't use hie on hie without swapping out the ghc-8.4.2 resolver for ghc-8.2.

Starter PRs

Here are some PRs so far:

The last one is still a work in progress, but here's a summary of how it came about and what's going down.

  • When trying out haskell-lsp-client, the document symbols request returned an empty list when run immediately after starting hie, unless a delay was added so that hie had time to load the module.
  • I tried submitting a PR to move the symbol request from the IdeM monad to the IdeGhcM monad
  • These two monads determine what thread they run on: IdeM is for internal requests and stuff, but IdeGhcM is for anything that goes through ghc-mod, which may take some time to run
  • This turned out to be a bad ideaâ„¢ since putting it on IdeGhcM would cause the request to block whenever a large module was being compiled in the background. We want it to only wait for the module to load whenever there was no cache available, but serve the cache whenever possible.

I owe all of my thanks to wz1000 for noticing this and pointing me in the right direction. The agreed plan is this:

Change the function that returns cached modules, getCachedModule to return not just a Maybe but a more descriptive ADT:

data CachedModuleResult = ModuleLoading
                        | ModuleFailed String
                        | ModuleCached CachedModule IsStale

Add a queue of actions that get executed whenever a module is finished loading:

data IdeState = IdeState
  { moduleCache :: GhcModuleCache
  -- | A queue of actions to be performed once a module is loaded
  , actionQueue :: Map.Map FilePath [CachedModule -> IdeM ()]
  ...
  }

Change IdeResponse to be a full blown ADT instead of a pattern, and add a new deferred type that the dispatcher can distinguish between and handle the queueing for:

-- | The IDE response, with the type of response it contains
data IdeResponse a = IdeResponseOk a
                   | IdeResponseDeferred FilePath (CachedModule -> IdeGhcM (IdeResponse a))
                   | IdeResponseFail IdeError

This is where I'm currently at. My head hurts and this is turning out to be a huge undertaking for such a small edge case.

But as wz1000 said:

there really is no better way to get to know a codebase than to tear it up

Tomorrow the community bonding period ends, and the real coding begins. I've a lot to learn about pracitcal Haskell: I've lived a very sheltered life in university, far away from the IO monad. But I'm looking forward to working with my mentor Alan and the other lovely members of the Haskell community, and hopefully making the future of Haskell tooling a little better.