I'm new to Slick. I'm creating a test suite for a Java application with Scala, ScalaTest and Slick. I'm using slick to prepare data before the test and to do assertions on the data after the test. The database used has some tables with more than 22 columns. I use slick-codegen to generate my schema code.
For tables with more than 22 columns, slick-codegen does not generate a case class, but a HList-based custom type and a companion ‘constructor’ method. As I understand it, this is because the limitation that tuples and case classes can only have 22 fields. The way the code is generated, the fields of a Row-object can only be accessed by index.
I have a couple of questions about this:
- For what I understand, the 22 fields restriction for case classes is already fixed in Scala 2.11, right?
- If that's the case, would it be possible to customize slick-codegen to generate case classes for all tables? I looked into this: I managed to set
override def hlistEnabled = false
in an overriddenSourceCodeGenerator
. But this results inCannot generate tuple for > 22 columns, please set hlistEnable=true or override compound.
So I don’t get the point of being able to disbale HList. May be the catch is in the ‘or override compound’ part, but I don't understand what that means. - Searching the internet on slick and 22 columns, I came across some solutions based on nested tuples. Would it be possible to customize the codegen to use this approach?
- If generating code with case classes with > 22 fields is not a viable option, I think it would be possible to generate an ordinary class, which has an ‘accessor’ function for each column, thus providing a ‘mapping’ from index-based access to name-based access. I’d be happy to implement the generation for this myself, but I think I need some pointers where to start. I think it should be able to override the standard codegen for this. I already use an overridden
SourceCodeGenerator
for some custom data types. But apart from this use case, the documentation of the code generator does not help me that much.
I would really appreciate some help here. Thanks in advance!
I ended up further customizing slick-codegen. First, I'll answer my own questions, then I'll post my solution.
Answers to questions
Solution: the generated code
So, I ended up generating "ordinary" classes for tables with more than 22 columns. Let me give an example of what I generate now. (Generator code follows below.) (This example has less than 22 columns, for brevity and readability reasons.)
The hardest part was finding out how the
*
mapping works in Slick. There's not much documentation, but I found this Stackoverflow answer rather enlightening.I created the
BigAssTableRow
object
to make the use ofHList
transparent for the client code. Note that theapply
function in the object overloads theapply
from the case class. So I can still create entities by callingBigAssTableRow(id: 1L, name: "Foo")
, while the*
projection can still use theapply
function that takes anHList
.So, I can now do things like this:
For this code it's totally transparent wether tuples or HLists are used under the hood.
Solution: how this is generated
I'll just post my entire generator code here. It's not perfect; please let me know if you have suggestions for improvement! Huge parts are just copied from the
slick.codegen.AbstractSourceCodeGenerator
and related classes and then slightly changed. There are also some things that are not directly related to this question, such as the addition of thejava.time.*
data types and the filtering of specific tables. I left them in, because they might be of use. Also note that this example is for a Postgres database.This issue is solved in Slick 3.3: https://github.com/slick/slick/pull/1889/
This solution provides
def *
anddef ?
and also supports plain SQL.There are few options available as you have already found out - nested tuples, conversion from Slick HList to Shapeless HList and then to case classes and so on.
I found all those options too complicated for the task and went with customised Slick Codegen to generate simple wrapper class with accessors.
Have a look at this gist.
}
As of Slick 3.2.0, the simplest solution for >22 param case class is to define the default projection in the * method using
mapTo
instead of the <> operator (per documented unit test):EDIT
So, if you then want the slick-codegen to produce huge tables using the
mapTo
method described above, you override the relevant parts to the code generator and add in amapTo
statement:You then structure the codegen code in a separate project as documented so that it generates the source at compile time. Unlike what's documented, you don't need to write your own main method. Instead, you can pass your classname as an argument to the
SourceCodeGenerator
you're extending: