How to migrate from selectSource to selectCursor?

[dnikolovv]

That may be a dumb question, but I've struggled with it for an embarrassingly long time now.

We have a Pool SqlBackend in the environment and queries such as

query :: ConduitT () Thing (ReaderT SqlBackend m) ()

that work fine using selectSource.

When I switch from selectSource to selectCursor, I get the following error

Could not deduce (BackendCompatible (RawPostgresql SqlBackend) SqlBackend)

I assume there's a very straightforward way of fixing this since no one has asked. It's definitely not one of the 40 ways I tried today though.

[ivb-supercede]

Hey! Firstly, thanks for taking a look at the library.

Unfortunately, the problem you have is simple to describe but not simple to fix. This library needs your backend to be of the RawPostgresql SqlBackend type so that it can access the underlying Postgres Connection to retrieve cursor-ed query results with. This is unfortunately not possible with a normal SqlBackend, because it doesn't expose its Connection.

This means that we can't supply an instance for BackendCompatible (RawPostgresql SqlBackend) SqlBackend. However, there is an instance of BackendCompatible SqlBackend (RawPostgresql SqlBackend).

Since you can't go from a SqlBackend to a RawPostgresql SqlBackend, but you can go the other way, the solution is to replace your Pool SqlBackend with a Pool (RawPostgresql SqlBackend), using one of the pool creation functions in Persistent.

This will sadly probably mean lots of changes in the rest of your codebase, because Persistent and Esqueleto functions often have SqlBackend hard-coded in their type signatures. The only suggestion I can make is to use withCompatibleBackend to tell Persistent to use the underlying SqlBackend wherever it doesn't need the raw Postgres Connection.

[dnikolovv]

Thank you for the quick reply!

I had a hunch we'd need a Pool (RawPostgresql SqlBackend) 😅 . I'll try it out tomorrow and see how that goes.

[dnikolovv]

Alright I tried this in isolation and it seems to work fine!

The only thing that I had to do was add this orphan instance (maybe we should add this as an orphan in this library, or even better in persistent as instance BackendCompatible backend backend?)

instance BackendCompatible (RawPostgresql SqlBackend) (RawPostgresql SqlBackend) where
  projectBackend = id

and for selectSource queries to continue to work I had to transPipe them using withCompatibleBackend

theQuery ::
  MonadResource m =>
  ConduitT () Thing (ReaderT (RawPostgresql SqlBackend) m) ()
theQuery = transPipe (withCompatibleBackend @SqlBackend)
  (E.selectSource (E.from $ \thing -> return thing)
    .| CL.map fromRow)