Accessing CouchDB in Vertx with LightCouch

Hello again! This time, I would like to move deeper into the subject of data access with Vertx framework. Our current candidate is CouchDB – NoSQL database, that comes with some handy features, like high performance, native JSON structures and easy to understand query language. Compare to SQL databases, the Vertx ecosystem lacks of “Vertx-native” clients, so we can actually use any client we want. An interesting point about CouchDB is that we actually don’t require to do so, as it provides HTTP API, that we can access with just a HTTP client.

In this post, we would talk about CouchDB features and then we would build a sample API with Vertx framework and LightCouch client for CouchDB.

Table of Contents

What is CouchDB?

CouchDB is NoSQL database, that means it provides schema-free data storage. As Mongo, Couch utilizes a concept of documents. In its regard, docs can have any fields and also Couch add metadata to docs that helps database to maintain data. Any document has unique id in the database, and Couch exposes a REST HTTP API for reading and updating database documents. From a technical point of view each document is nothing than JSON object, therefore it supports the same basic types as supported by JavaScript: array, boolean, number, object, string. When client updates a document, it rather is saved successfully, rather failed at all. Couch never contains partially saved or edited documents.

Couch is supplied with Fauxton GUI, that is a web-based admin tool (access it on port 5984 by default), that helps you to manage databases, documents, users etc. Click on Create database button and create a new database (I call it humans). This database will be used in this post. What is cool, that we actually don’t need any client library to work with Couch, as it has built-in REST API. Before we will move to Vertx, let see how to use this API from Postman.

Accessing CouchDB using HTTP API

The interesting feature of CouchDB is that it explores a REST HTTP API that you can use directly, without any specific client library. To demonstrate, let me provide some examples of accessing CouchDB instance with nothing but Postman.

As I said before, documents is technically JSON object. Therefore, we dont need any magic middleware that converts incoming/outcoming JSON payload to/from internal data (like table rows).

Let first test that everything is up and running. A GET request to localhost:5984/humans endpoint will return database metadata:

Hooray! It works. Let explore basic CRUD functionality using Couch REST API: add, deleting, finding, updating a document. As we don’t have any documents yet, let create a human. It is super easy: you have to send POST request localhost:5984/humans with JSON body, like this one:

{
    "firstName":"John",
    "lastName":"Doe",
    "address":"Istanbul",
    "age":27,
    "phoneNumber":"435 134 124"
}

At response, Couch returns us metadata of created object: status, id, revision id:

CouchDB generates UUID id for created documents. John Doe received 76f53376784d7df81a4a3567f6000382. Now, we can manipulate it. Let check that it is actually in database (well, we know it, but I have to write complete tutorial, so…). You can get a single document by sending a GET request to localhost:5984/humans/76f53376784d7df81a4a3567f6000382:

John Doe is a digital nomad and he travels around the globe. He lived in Istanbul, but recently moved to Copenhagen and changed his phone number as well. This means, we need to update his record to keep our data up to date. Updating of documents in Couch API is performed by PUT request to the endpoint localhost:5984/humans/76f53376784d7df81a4a3567f6000382. NB you need to include both id and rev id into payload, otherwise document would not be updated:

Well, John now resides in Copenhagen, therefore GDPR applies to his data. He claimed us to revoke his data from database, and we have no other choice, then remove his record. NB to delete a document we use a bit tricky route: database/[_id]?rev=[_rev]. In our case we send a DELETE request to localhost:5984/humans/76f53376784d7df81a4a3567f6000382?rev=2-60568059ddaf9cd4d2c87b24fe02ec0b:

Basically, as it was shown, you don’t need a client library and you can use HTTP client, like Java HTTP client to manipulate data via REST API. Another approach is to use one of CouchDB clients. In this post we would use LightCouch. Next section we explore how to use it with Vertx web apps.

Accessing CouchDB with LightCouch

This section describes steps of the creation of sample app – CouchHumans – a very simple REST API that have CRUD functionality for Human objects (I selected this name because I want to distinguish it from Person or People examples). Before we start, check that you have these dependencies in your classpath:

  • Vert.x Core
  • Vert.x Web
  • Vertx. Config
  • LightCouch

You can configure this with your preferred build system (my choise is Maven). Also I recommend you to use the latest Java version (JDK 8+ is required by Vertx anyway).

Step 1. Create new app

First, we need to create core elements of a new app – Apllication class and Verticle class. I’d like also to separate routing logic in separate class to make a code less coupled. Start with creation of CouchHumanApp.java – this an entry point of app:

public class CouchHumanApp {

    private static ConfigRetrieverOptions getConfigOptions(){
        ConfigStoreOptions options = new ConfigStoreOptions();
        options.setType("file");
        options.setFormat("properties");
        options.setConfig(new JsonObject().put("path", "test-config.properties"));
        return new ConfigRetrieverOptions().addStore(options);
    }


    public static void main(String[] args) {
        Vertx vertx = Vertx.vertx();
        vertx.deployVerticle(new CouchHumanVerticle(new CouchDbHumansDAOImpl(), getConfigOptions()));
    }
}

You can note, that this code consits of two methods – main() that starts our app and deploy a verticle and getConfigOptions. I utilize here Vertx-Config to retrieve configuration. This is an abstraction that permits us to inject almost any type of configuration to our app – from properties and environment variables to HashiCorp Vault secrets. Inside a verticle, ConfigRetriever instance determine a source using a provided config store and load necessary data as JSON object. Let move to CouchHumanVerticle.java, where we place core application logic:

public class CouchHumanVerticle extends AbstractVerticle {

    private ConfigRetrieverOptions configRetrieverOptions;
    private IHumansDAO dao;

    public CouchHumanVerticle(IHumansDAO dao, ConfigRetrieverOptions configRetrieverOptions){
        this.configRetrieverOptions = configRetrieverOptions;
        this.dao = dao;
    }

    @Override
    public void start(Promise<Void> promise) throws Exception {
        ConfigRetriever configRetriever = ConfigRetriever.create(vertx, configRetrieverOptions);
        configRetriever.getConfig(conf->{
            if (conf.succeeded()){
                JsonObject config = conf.result();
                dao.initialize(config);
                HttpServer server = vertx.createHttpServer();
                HumanRouter router = new HumanRouter(vertx, dao);
                server.requestHandler(router.getRouter());
                int port = config.getInteger("app_port");
                server.listen(port, res->{
                    if (res.succeeded()){
                        promise.complete();
                    } else {
                        promise.fail(res.cause());
                    }
                });
            } else {
                promise.fail(conf.cause());
            }
        });
    }

}

Inside the Verticle we do following tings:

  1. Load a configuration with ConfigRetriever. NB, that Verticle has no idea about source (or Store), so we can interchange it easily as we need. This is a very cool abstraction.
  2. We initialize DAO.
  3. We create a HTTP server
  4. We create a router. NB: we place a router in a separate class, that we will explore a bit later in this section.

This code contains some drawbacks, let discover them. I agree, that the approach with DAO initialization (step 2) may looks like controversial for some developers and for reasons. However this is dictaded by logic of Vertx Config library. We don’t have configuration before start of verticle. So we provide an instance of DAO in constructor, but it is not yet initialized. We need first to obtain configs. The second thing to improve is an idea with router in separate class. As you will discover later, it is not a router, but a container, that moves router to another class. This is done in order to simplify testing and separation, also it is not a final solution. You can think in separation routing handlers as separate classes.

We need to define a DAO interface, that would abstract data access logic from main code:

public interface IHumansDAO {

    void initialize (JsonObject config);

    Optional<Human> findById (String id);

    Human add (Human human);

    void remove (String id, String revId);

    void update (Human human);

    List<Human> findAll();
}

order to avoid null checking. These components finalize our preparation step. Next, we are ready to move into data source creation and later – to a DAO implementation.

Step 2. Prepare data source

This section consists of two logical sub-sections. First, we define a data model in Java. Second, we create a CouchDB instance and add mock data there.

Define a data model

The first point is a definition of data model. We create a POJO class, that keep a structure of data. Create a Human.java as folloiwng:

public class Human {

    private String id;
    private String lastName;
    private String firstName;
    private String address;
    private int age;

    // getters and setters

}

Note, I escaped getters and setters to save space. You can generate them easily with your IDE (or use Lombok).

Prepare a database

After you defined a data model, you need to obtain required credentials as well populate our database with some mock data. In order to access CouchDB from LightCouch we need to have username and password. Open Fauxton and navigate to Create Admin tab (last in the column):

Now, Couch asks you to login with new credentials. After you log in, go to database and add some mock data:

After we prepared a database, it is a time to go and implement DAO.

Step 3. Implement DAO

This is the crucial section of the post – a DAO implementation. Let go step by step for each method.

Initialization

As we agreed in the first step, we have to explicitly initialize DAO after we will get a configuration from ConfigRetriever. All data access operation are performed by CouchDbClient, that have to initialize with configuration. There are several ways to do it – pass it as a constructor as hard-coded values, to add couchdb.properites file in a classpath or use CouchDbProperties and create config programmaticaly. As we receive configs as a JSON object from Vertx, the last approach seems to be preferred:

@Override 
public void initialize(JsonObject config) {
    CouchDbProperties properties = new CouchDbProperties();
    properties.setDbName(config.getString("db_name"));
    properties.setProtocol(config.getString("db_protocol"));
    properties.setCreateDbIfNotExist(false);
    properties.setHost(config.getString("db_host"));
    properties.setPort(config.getInteger("db_port"));
    properties.setUsername(config.getString("db_user"));
    properties.setPassword(config.getString("db_secret"));
    couchDbClient = new CouchDbClient(properties);
}

Now, let move to data access methods themselves.

Add

Our first candidate is an adding method add(). It takes a Human object as an argument and should return to a caller a Human instance with generated ID. Let see how it works with LightCouch:

@Override
public Human add(Human human) {
    Response response = couchDbClient.save(human);
    String id = response.getId();
    human.setId(id);
    return human;
}

There are two players here – client, that performes an operation and Response. It is returned by database and it typically contains an id and rev values. Additional data might be returned such as error from Bulk request. We can do an operation without getting response as we will see in other operations, however here we need to obtain an id generated by database.

Remove

To remove a document with LightCouch, as with HTTP API we need to supply two parameters: id and revision id:

@Override
public void remove(String id, String rev){
    couchDbClient.remove(id, rev);
}

Update

Here comes an update functionality. We can do it as easy as:

@Override
public void update(Human human) {
    couchDbClient.update(human);
}

Note, that CouchDB does update as a PUT HTTP request. So, we need to provide an id in order to update an existing entity. Otherwise, database creates a new record and generate UUID id for it (and this will be add method, that we discussed already).

Find one

Another handy method provided by LightCouch library is find one record functionality. It is extremely straightforward – we just need to pass an id:

@Override
public Optional<Human> findById(String id) {
    try {
    	return Optional.of(couchDbClient.find(Human.class, id));
    } catch (NoDocumentException ex){
    	return Optional.empty();
    }
}

There are several overloaded methods that we can choose from. For example, in this code snippet we supply a record’s id. Alternatively we can pass query params as Params object. In this example, we use optional to avoid null checking. In case the requested record is not presented in a database, LightCouch throws NoDocumentException. In this case we return an empty Optional. Although, this is better to write with Vavr’s Try.

Query

Finally, let do query functionality. Our API provides a way to query all documents from database. Couch has a concept of views, that is reasonably broad to this post. All what is important to know here is that to get a collection of documents you need to provide a view (like query). There are some special views, for instance to get all documents – the one that we use:

@Override
public List<Human> findAll() {
    couchDbClient.view("_all_docs").query(Human.class);
}

Views are a very powerful tool, and if you are serious about using CouchDB in your architecture, I suggest you to learn about it in a more detailed way. Go to Learn more section – I provided some useful links there.

Step 4. Build API

To finish our sample app, we have to build an API. We separated it inside a dedicated router class. I performed it in order to simplify testing and avoid coupling. This is not an ideal solution, for example you can also attempt to put handlers as separate classes. Take a look on the code, that represents HumanRouter.java implementation:

public class HumanRouter {

    private Router router;
    private IHumansDAO dao;
    private Vertx vertx;

    public HumanRouter(Vertx vertx, IHumansDAO dao){
        this.router = Router.router(vertx);
        this.vertx = vertx;
        this.dao = dao;
        attachRoutes();
    }

    private void attachRoutes(){
        router.route("/*").handler(BodyHandler.create());
        router.post("/").handler(this::addHuman);
        router.put("/:id").handler(this::updateHuman);
        router.delete("/:id/:rev").handler(this::removeHuman);
        router.get("/one/:id").handler(this::findOne);
        router.get("/all").handler(this::query);
    }

    public Router getRouter(){
        return router;
    }

    private void addHuman (RoutingContext context){
        JsonObject body = context.getBodyAsJson();
        Human payload = Json.decodeValue(body.encode(), Human.class);
        vertx.executeBlocking(promise -> {
            Human human = dao.add(payload);
            context.response().setStatusCode(200).end(JsonObject.mapFrom(human).encode());
            promise.complete();
        });
    }

    private void removeHuman(RoutingContext context){
        String id = context.pathParam("id");
        String rev = context.pathParam("rev");
        vertx.executeBlocking(promise -> {
            dao.remove(id, rev);
            context.response().setStatusCode(200).end();
            promise.complete();
        });
    }

    private void updateHuman(RoutingContext context){
        JsonObject body = context.getBodyAsJson();
        Human payload = Json.decodeValue(body.encode(), Human.class);
        vertx.executeBlocking(promise -> {
            dao.update(payload);
            context.response().setStatusCode(200).end();
            promise.complete();
        });
    }

    private void findOne(RoutingContext context){
        String id = context.pathParam("id");
        vertx.executeBlocking(promise -> {
            dao.findById(id).ifPresentOrElse(human->{
                JsonObject response = JsonObject.mapFrom(human);
                context.response().setStatusCode(200).end(response.encode());
            }, ()-> context.response().setStatusCode(404).end());
            promise.complete();
        });
    }

    private void query(RoutingContext context){
        vertx.executeBlocking(promise -> {
            List<Human> results = dao.findAll();
            context.response().setStatusCode(200).end(JsonObject.mapFrom(results).encode());
            promise.complete();
        });
    }
}

Some things to note in this code. First, in order to keep reactivity, we place DAO methods inside executeBlocking blocks. This permits us to move potentially blocking code from main thread to worker threads. NB: we use latest Vertx, and there some Core API was changed, so keep it mind, when you’re working with old versions. Second, we utilize Optinal class inside the find one method. This allows us to use ifPresentOrElse and have logic for both cases – if there is a record in a database and if there is no record in a database. NB this method was added in Java 11, so it would not work in older versions. If you want to do this trick in Java 8, you have to use Vavr Option instead.

That’s all folks! You can runt this app and play with it around.

Conclusion

CouchDB is a powerful, however easy to use, NoSQL database. You can use it even without any client library – straight with REST HTTP API. In this post we explored why and how to use this database in a context of Vertx web apps development. We also wrote a sample API, that use LightCouch library in order to access CouchDB. If you have any questions, feel free to contact me, or leave a comment below. Have a nice day!

Learn more

If you are keen to learn more about CouchDB Views, I suggest you to browse these links:

  • Guide to Views CouchDB Documentation, read here
  • Betty Tran A Recipe for Creating CouchDB Views (2016) read here

Leave a Reply

Your email address will not be published. Required fields are marked *