Using Microservices via RestExpress

Main Idea

In this article, we want to develop an application based on RestExpress. This application will allow the user to upload Java sources (including Maven projects) as ZIP archives and obtain the result of the compilation. The application will be based on four Web services, as shown in Figure 1 (the Upload, Unzip, and Compile services will authenticate the user based on the Authentication service):

Figure 1: The application structure

We will have four Web services, as follows:

• Authenticator Service: Manage API clients and JWT authentication.
• Upload Service: Manage the file upload.
• Unzip Service: Manage the unzipping process.
• Compile Service: Compile Java application (including Maven-based applications).

We choose to implement this application via a microservices architecture and promote developing and deploying applications composed of autonomous, self-contained units. I’ll go with the very lightweight framework, RestExpress. This is  an open-source tool that enables quickly creating RESTful microservices that embrace industry best practices. It is a container-less, extremely lightweight, fast, REST engine and API for Java. RestExpress is a thin wrapper on the Netty IO HTTP handling. RestExpress lets you create performant, stand-alone REST APIs rapidly. An important aspect and differentiator from SOA-based APIs, RESTful APIs do not require XML-based Web service protocols (SOAP and WSDL) to support their interfaces.

So, we have a simple, performant, very lightweight, reliable, and scalable solution for providing flexible functionality to an end user, in a language-platform-agnostic approach. In addition, RestExpress provides several Maven archetypes in NetBeans, which is very helpful in such conditions. Basically, the archetype will provide us a structure, as shown below:

• Base Package: (org.*)
• Configuration class: Parses the environment properties (/config/dev/) and instantiates controllers (for production, other environment properties should be placed in /config/prod/)
• Constants class: Contains project constants
• Relationships class: Defines linking resource that aids service discoverability (HATEOAS)
• Main executable class
• Routes class: Defines the routes (endpoints) exposed by the service and the corresponding controller class
• Model/Controllers Packages: (org.objectid and .uuid)
• Entity class: Defines the data entity; for example, a Client.java
• Controller class: Contains the methods executed when the route (endpoint) is called
• Repository class: Defines the connection details to MongoDB
• Service class: Contains the calls to the persistence layer, validation, and business logic
• Serialization Package: (org.serialization)
• XML and JSON serialization classes
• Object ID and UUID serialization and de-serialization classes: We don’t need this

Also included in the default RestExpress MongoDB Maven project is a Java properties file (environment.properties). Probably, the most important setting here is the host and port for each Web service and for MongoDB. For the development stage, we have set the host to localhost and the port as follows:

• Authentication service: http://localhost:8587
• Upload service: http://localhost:8081
• Unzip service: http://localhost:8084
• Compile service: http://localhost:8086
• MongoDB URI: mongodb://localhost:27017/authentication

These values can be altered in production by adding a new environment.properties in /config/prod and running the service by using the -Dexec.args="prod" option.

RestExpress uses Java EE, Maven, MongoDB, and Netty. Each service can have its own MongoDB instance. But, we will use only one for the authentication service. Of course, we can easily add new instances for the remaining services (this can be done any time).

Figure 2: The application layers

So, we are talking about a fine-grained segmentation of services functionality. This means that we are moving away from monolithic and n-tier applications, and SOA based applications. We are talking about a client-centric application based around business capabilities, and not about application-centric functionality dependent on organization structure. In front of our services, we can put anything from a free open-source API gateway to an expensive one. Or, we even rely on SO load balancing, (reverse) proxying, and so forth (for example, for Windows Server 200x).

Technology Used

From the significant number of technologies used to write RESTful services, we can choose for this project the following:

• RestExpress: Create performant, stand-alone microservices-based REST APIs
• Java EE: Primary development language
• Netty: Async, event-driven Java network application framework
• Apache Maven: Dependency management and more
• MongoDB: Data source
• JSON Web Tokens (JWT): Claims-based authentication
• Java EE Logging: Logback (based on SLF4J API)
• JUnit and HttpClient: Unit testing
• HAProxy: Service gateway and load-balancing for Linux. For Windows, we should choose another solution.
• NetBeans IDE: Using the RestExpress MongoDB Maven Archetyp.
• ARC (Chrome): REST API testing (or Postman)

MongoDB Notes

We will go with a DDM architecture, meaning that each microservice will have its own MongoDB instance. In our case, only the authentication service will have a MongoDB instance. All you have to do is install and start MongoDB! No schemas, no SQL, no DDL, and the like. A client will be a MongoDB document that looks like Figure 3:

Figure 3: MongoDB document sample

The client data is stored in the authentication database, under the clients collection. From the command line, you can check the data like this:

UML

Let’s have some UML. First, the use case diagram (our actor can be a human or a process):

Figure 4: UML use case diagram

• The client can authenticate in the service (or require a JWT)
• An authenticated client can upload files via the upload service
• An authenticated client can unzip files via the unzip service
• An authenticated client can compile Java applications via the compile service
• The admin has deeper access in the authentication service (he can perform administration tasks)

Personally, I prefer swim-lane diagrams instead of use case and prototypes, but for such a small project, it would look ridiculous.

Authentication Service

For authentication, we will use JWT (JSON Web Tokens). The authentication service relies on the following UMLs.

Figure 5: The “road” to API key

Figure 6: The “road” to JWT

Validating JWT implies: Check the user secret against MongoDB via API key; confirm algorithm from JWT header (SHA-256); use the secret to decode JWT (HMAC-256); confirm JWT is not expired;

Basically, we follow these steps:

1. Create a new client by supplying the application name and a shared secret (one time only).
2. Receive an API key in response from the authentication Web service (one time only).
3. Use the API key and the shared secret to obtain an JWT (each user session or renew when the previous JWT expires).
4. Include the JWT in each API call.

Upload Service

For upload, we need to pass the JWT and the file to upload.

Figure 7: Upload service flow

Unzip Service

For unzip, we need to pass the JWT and the name of the folder where the zip was uploaded (in production this should be changed, but of course there are many details to take into account).

Figure 8: Unzip service flow

Compile Service

For compilation, we need to pass the JWT and the name of the folder where the zip file was unzipped.

Figure 9: Compile service flow

Communication Between Services

There a few approaches to ensure the service-to-service communication (you may want to go with a messaging approach via a message broker to obtain lose-coupling (for example, RabbitMQ, ActiveMQ, and the like). In this example, the Upload, Unzip, and Compile services must communicate with the Authentication service. For this, we can use HTTP-based REST calls via the HttpURLConnection API. Based on the same API, we can write JUnit tests, as the TestAuthenticationService example.

Figure 10: Communication between Web services

Or, we can use the HttpClient API. For example, this API is used in the case of UploadTest.

Services API Documentation

The RESTful services don’t provide WSDL (or WADL) documents. This means that API documentation is an issue. But, this is not so true, because we have access to free tools capable of creating API documentation for RESTful services, as in the RESTful Analyzer. Or, we can simply provide documents as shown below. For example:

AuthenticationService (URL ROOT: http://localhost:8587)

NEW CLIENT

{
   "application":"Leonard",
   "secret":"java797b"
}

{
   "_links": {
      "self": {
         "href": "localhost/clients/56f1abd6b2fe0109fe1eecc4"
      }
      "up": {
         "href": "localhost/clients"
      }
   }
   "application": "Leonard"
   "secret": "java797b"
   "apiKey": "CDwQxserXm5GkuOca0xfDzqn"
   "id": "56f1abd6b2fe0109fe1eecc4"
   "createdAt": "2016-03-24T10:32:22.474Z"
   "updatedAt": "2016-03-24T10:32:22.474Z"
}

CompileService (URL ROOT: http://localhost:8086)

COMPILE

UnzipService (URL ROOT: http://localhost:8084)

UNZIP

UploadService (URL ROOT: http://localhost:8081)

UPLOAD NEW FILE

These documents can be provided to the client as separate documentation (one document/service).

Running and Testing the Application

To run the services, you need to accomplish several steps:

1. Install and start MongoDB on the default port (follow instructions from here).
2. From CLI (I suppose that JDK 8 and Maven are installed and configured in the classpath), simply start each service via command (the order of starting is not important):

Authentication and upload services:

mvn clean package exec:java -Dmaven.test.skip=true

Unzip and compile services:

mvn clean package exec:java

For more details, please consult the README.txt file for each service.

After the services are running, you can test them in several ways. Use browser extensions (for example, ARC client of Google Chrome), JUnit (see test samples TestAuthenticationService and UploadTest (basically, the rest of tests are fundamentally based on these)), the simple Web application provided for testing (it doesn’t require an application server; simply open index.html in your browser), Postman app, and so forth.

I wrote two JUnit tests, but obviously it is just a matter of time to write more. You can run the tests like this:

Authentication service test: Check the creation of a client by verifying the existence of API key in the response. From the authentication service folder:

mvn -Dtest=TestAuthenticationService test

Upload service test: Check the upload process without needed authentication. From the upload service folder:

mvn -Dtest=UploadTest test

These instructions are the same for Windows and Linux OS.

What Else Can We Do?

There are many aspects that we can improve on this project, like adding a microservices failover system, SSL support (RestExpress can enable SSL support via SSL context, but this can be enabled on the API gateway also), better exception handling and messaging standardization (bad error/exception handling will lead to high maintenance costs), better upload handling and upload progress bar, another MongoDB instances for tracking calls, and so on, a batch service for cleaning the uploaded files (now the files are uploaded and processed in the temporary folder), a friendly and real-time output via JMS or WebSockets merged with a common topic, add configuration parameters for compilation time (for example, javac- and mvn-specific parameters), service discovery support (for example, Apache ZooKeeper), and so on.