jOOQ et jOOλ :

How to write clean code for your Data Access Objects

Mathieu Gandin / Alexandre Dubreuil

LesFurets.com

  • 1 website with 5 insurance products: car, health, home, bike, loan
  • 1 codebase: 450k lines of code, 60k unit tests, 150 selenium tests
  • 22 Developers, 2 DevOps, 4 Architects
  • 19 production servers: including load balancers, frontend, backend, databases, BI
  • 1 release per day, with 8 years of code history
  • 3M quotes/year, 40% of market share, 4M of customers

General Problems

  • Object-relational mapping in Java is still a combination of two programming languages
  • In 2017, SQL is still present in our DAOs code
  • How to prevent boiler-plate code in DAO?

Context at LesFurets.com

  • MySQL (MariaDB cluster) runtime and backoffice
  • Cassandra cluster for analytics and storage
  • In-house frameworks with JDBC
  • Code in DAO needs to stay close to SQL
  • Refactor DAO for expressiveness (less boilerplate)

Java doesn't lack solutions to the persistence problem

  • JDBC
  • JPA / Hibernate
  • Spring JDBC Template
  • MyBatis, in-house framework, ...
  • Let's try Java Object Oriented Query a.k.a jOOQ

About ORM...

jOOQ blog : jOOQ vs Hibernate

"Hibernate shares JPA’s vision of ORM. jOOQ shares SQL’s vision of powerful querying, so for the sake of the argument, let’s use Hibernate / JPA / ORM interchangeably, much like jOOQ / JDBC / SQL."

Hibernate have functionality that jOOQ will never have: optimistic locking for complex writes, cache, etc.

Live demo

            
public FicheAuto selectRecentFiche(Integer statut) throws Exception {
    return DSL.using(connection)
            .select(...)
            .from(FICHES_AUTO)
            .where(statut == null ? DSL.trueCondition() 
                                  : FICHES_AUTO.STATUT.gt(statut))
            .orderBy(FICHES_AUTO.STATUT.desc())
            .limit(1)
            .stream()
            .map(this::mapFicheAuto)
            .findFirst()
            .orElseThrow(() -> new Exception("No fiche for " + statut));
}
            
            

SQL mapping choices

  • SQL mapping with JDBC
  • SQL mapping simplified with Spring JDBC Template
  • Object relational mapping with JPA
  • jOOQ Record with Java 8 method reference

SQL mapping with JDBC

                
try {
  preparedStatement = connection.prepareStatement("select departement, manager,
    netprofit, operatingexpense,year,turnover from result");
  resultSet = preparedStatement.executeQuery();
  while(resultSet.next()) {
    Result result = new Result();
    result.setDepartement(resultSet.getString("departement"));
    result.setManager(resultSet.getString("manager"));
    result.setNetProfit(resultSet.getDouble("netprofit"));
    result.setOperatingExpense(resultSet.getDouble("operatingexpense"));
    result.setYear(resultSet.getInt("year"));
    results.add(result);
  }
  return results;
} catch (SQLException e) {
...
                
            

SQL mapping with JDBC

pros

  • Simple, fast, no special restriction
  • Based on manipulation of String type

cons

  • No syntax validation before executing your code
  • It's not always simple to verify your code with unit tests
  • Lot of boiler-plate code to handle errors, resources and mapping

Let's try to simplify the mapping with Spring JDBC Templates.

            
@Autowired
public InsuranceParamRepository(DataSource dataSource) {
  this.jdbcTemplate = new JdbcTemplate(dataSource);
}

public List<Provider> findProvidersBy(String agencyId) {
  return jdbcTemplate.query(FIND_PROVIDER_BY_AGENCY_ID,
    new Object[] { agencyId }, (resultSet, i) ->
      new Provider(resultSet.getString("CODE"),
                   resultSet.getString("NAME"),
                   resultSet.getString("COUNTRY_CODE")));
}
            
            

Simplified SQL Mapping with Spring JDBC Template

  • JDBC is less painful
  • But Object relational mapping is still problematic

What about ORM with JPA ?

            
@Entity
@Table(name = "marques")
public class Marque {
@Id
private String code;
private String name;
private String logoUrl;
@Transient
private byte[] logo;
@Transient
private String logoName;
@OneToMany(mappedBy = "marque", fetch = FetchType.EAGER)
@Fetch(FetchMode.SELECT)
private List<Brochure> brochures;
            
            

What about ORM with JPA ?

            
public class MarqueDao {
    @PersistenceContext
    private EntityManager entityManager;

    public Marque findByCode(String code) {
        try {
            Query query = entityManager
                .createQuery("from Marque to where to.code=:code")
                .setParameter("code", code);
            return (Marque) query.getSingleResult();
        } catch (Exception e) {
            throw new ResourceNotFoundException(e);
        }
    }
}
            
            

ORM with JPA

pros

  • POJO approach, more object oriented
  • Navigation by objects, not data tables
  • CriteriaQuery

cons (for our codebase)

  • Lots of configuration to make a first step
  • We have complex SQL requests, it's difficult to make it work with JQL/HQL (more limited than SQL)

How do you handle the mapping part with the jOOQ DSL?

            
public FicheAuto selectRecentFiche(int statut) throws Exception {
    return DSL.using(connection)
            .select(...)
            .from(FICHES_AUTO)
            .where(FICHES_AUTO.STATUT.gt(statut))
            .orderBy(FICHES_AUTO.STATUT.desc())
            .stream()
            .map(record -> new FicheAuto(
                    FICHES_AUTO.ID.get(record),
                    FICHES_AUTO.OFFRE_UID.get(record),
                    FICHES_AUTO.CREATION_DATE.get(record),
                    FICHES_AUTO.CODE_POSTAL.get(record),
                    FICHES_AUTO.EMAIL.get(record),
                    FICHES_AUTO.CODE_SRA.get(record),
                    FICHES_AUTO.STATUT.get(record)))
            .findFirst()
            .orElseThrow(() -> new Exception("No fiche for " + statut));
}
            
            

Use Java 8 stream operations

            
public FicheAuto selectRecentFiche(int statut) throws Exception {
    return DSL
            .using(connection)
            .select(...)
            .from(FICHES_AUTO)
            .where(FICHES_AUTO.STATUT.gt(statut))
            .orderBy(FICHES_AUTO.STATUT.desc())
            .stream()
            .map(this::mapFicheAuto)
            .findFirst()
            .orElseThrow(() -> new Exception("No fiche for " + statut));
}
            
            

Keep the mapping part clean and use method references

            
private FicheAuto mapFicheAuto(Record7 record) {
    return new FicheAuto(
            FICHES_AUTO.ID.get(record),
            FICHES_AUTO.OFFRE_UID.get(record),
            FICHES_AUTO.CREATION_DATE.get(record),
            FICHES_AUTO.CODE_POSTAL.get(record),
            FICHES_AUTO.EMAIL.get(record),
            FICHES_AUTO.CODE_SRA.get(record),
            FICHES_AUTO.STATUT.get(record));
}
            
            

To jOOQ and beyond...

What about dates ?

            
Date creationDate = resultSet.getDate("creation_date");
LocalDateTime date = Instant.ofEpochMilli(creationDate.getTime())
                            .atZone(ZoneId.systemDefault())
                            .toLocalDateTime();
            
            

What about dates ?

            
public List<Tuple2<LocalDateTime, String>> findPlate(Connection connection)
    throws SQLException {
  return DSL
      .using(connection)
      .fetch(FIND_PLATE_AND_CREATION_DATE)
      .stream()
      .map(r -> {
        String plate = r.getValue(PLATE, String.class);
        LocalDateTime date = r.getValue(CREATION_DATE, LocalDateTime.class);
        return new Tuple2<>(date, plate);
      }).collect(toList());
}
            
            

Tuple2 ? Wait ? What ?

            
public List<Tuple2<LocalDateTime, String>> findPlate(Connection connection)
    throws SQLException {
  return DSL
      .using(connection)
      .fetch(FIND_PLATE_AND_CREATION_DATE)
      .stream()
      .map(r -> {
        String plate = r.getValue(PLATE, String.class);
        LocalDateTime date = r.getValue(CREATION_DATE, LocalDateTime.class);
        return new Tuple2<>(date, plate);
      }).collect(toList());
}
            
            

We use jOOλ with jOOQ

  • Helper classes extending lambda from Java 8
  • Your code tends to be more functional, declarative
  • From Function1 to Function16
  • From Tuple1 to Tuple16
  • Sequences like Seq
  • Unchecked for streams, supplier and consumer

From Tuple1 to Tuple16

            
Tuple2<LocalDate, String> tuple = Tuple.tuple(now(), "Yolo");
            
            

Sequences

            
Seq.of(1, 2, 3).concat(Seq.of(4, 5, 6)); // Seq.of(1, 2, 3, 4, 5, 6);
            
            

Function, Supplier and Consumer in Java 8

            
return abScenarios
    .stream()
    .map(abScenario -> {
        try {
            return uuidPairsWithAbTesting(...);
        } catch (SQLException e) {
            e.printStackTrace();
        }
    })
    .collect(toList());
            
            

Unchecked Function, Supplier and Consumer

            
return abScenarios
    .stream()
    .map(Unchecked.function(abScenario -> uuidPairsWithAbTesting(...)))
    .collect(toList());
            
            

jOOλ, conditions of usage

  • Don't use Tuple to replace POJOS
  • You may need to do some refactoring if you're planning to use Function16
  • Remember, Java is not a functional language

Code generation with jOOQ

  • Works with Ant, Maven and Gradle

Database supports

  • Oracle dialect support specific datatypes, stored procedures, ...
  • SQL-Server Dialect support SQL-Server 2012 and 2008
  • Support of MariaDB, PostgreSQL, SQLite, ...
            
<configuration>
    <jdbc>
      <driver>${db.driver}</driver>
      <url>${db.url}</url>
      <user>${db.username}</user>
      <password>${db.password}</password>
    </jdbc>
    <generator>
      <database>
          <name>org.jooq.util.h2.H2Database</name>
          <includes>.*</includes>
          ...
        </database>
        <generate>
          <instanceFields>true</instanceFields>
          <javaTimeTypes>true</javaTimeTypes>
          <deprecated>false</deprecated>
          <pojos>false</pojos>
          <records>false</records>
        </generate>
        <target>
          <packageName>com.lesfurets.db</packageName>
          <directory>target/generated-sources</directory>
        </target>
      </generator>
</configuration>
            
            

Our usage of codegen

  • javaTimeTypes: true to use java.time instead of java.sql for time types
  • pojos: false to remove generation of FicheAuto we already have
  • records: false to remove generation of FicheAutoRecords, we are using Recordn directly

Pitfalls

  • jOOQ does not throw checked exception, be careful when migrating old code
  • DSL.insert().returning() (equivalent of JDBC Statement.RETURN_GENERATED_KEYS) do not work unless you use code generation
  • DSL.fetchOne returns an empty Record (not null), using DSL.stream().findFirst() is better, since you get back an Optional

Falling back to JDBC

Sometimes jOOQ does not support something you need (e.g. insert returning for batch inserts), but falling back to JDBC is easy by using the Query object

            
List<String> queries = leads.stream()
                .map(lead -> insertLead(conn, refTimeByHour, insert, lead))
                .map(query -> query.getSQL(INLINED))
                .collect(toList());
            
            

Then use the SQL in a normal JDBC statement

NoSQL databases

jOOQ blog : DSL for cassandra

"NoSQL databases will not profit from jOOQ directly any time soon – unless they implement actual SQL"

No unified language between NoSQL databases

Java Object Oriented Query

  • SQL mapping with a DSL
  • Typed mapping valid in SQL
  • Code generation
  • Java 8 stream support
  • Java 8 java.time API support
  • You can use jOOλ to simplify your code
  • Supports MariaDB, MySQL, Oracle, Oracle, PostgreSQL, SQLite, SQL Server ...

Resources

THANK YOU!