Vedran Kolka
DATA ENGINEER
Quarkus is a Kubernetes-native Java framework tailored for GraalVM and HotSpot, crafted from the best-of-breed Java libraries and standards. Its goal is to make Java the leading platform in Kubernetes and serverless environments while offering developers a framework to address a wider range of distributed application architectures.
Quarkus is a Reactive framework. Reactive represents a set of principles used to build robust, efficient, and concurrent applications and systems. These principles let you handle larger computational loads than traditional approaches, all the while being able to use the resources (CPU and memory) more efficiently and gracefully react to failures.
Essentially, Quarkus feels like a young Spring Boot, oriented towards Reactive, natively compiled Java applications.
Some of the main selling points of Quarkus:
-
Container oriented – minimal memory footprint, fast startup
-
Reactive style communication
-
Kubernetes native – generation of Kubernetes resources based on defaults and user-supplied configuration
-
Ease of development – CLI, dev services, etc.
This page will cover some of the mentioned selling points but the main focus will be the reactive messaging features of Quarkus.
0-Getting started with Quarkus
-
Installing the Quarkus CLI guide.
-
Creating a Quarkus project using the CLI:
quarkus create app net.syntio.: \
--extension=, \
1-Dev Services
When running a Quarkus app in development mode, by default, Quarkus will run services on which the application depends on, by analyzing the projects dependencies. These services are run using Docker.
This means that when starting a Kafka Streams application, it will spin up a Kafka broker (actually RedPanda) before starting the application. If the application relies on AMQP, it will spin up an Artemis broker.
A list of available dev services can be found here.
It is possible to list running services and get their logs. To do so:
- Start the app in development mode:
quarkus dev
. - Enter
:
to enter the quarkus console. - Once inside the console, run
devservices list
to list all the running services. - Run
devservices logs <service_name>
to print logs of a service.
2-SmallRye Reactive Messaging
SmallRye Reactive Messaging is an implementation of the Eclipse MicroProfile Reactive Messaging specification 2.0.1. Quarkus utilizes this framework to offer a broker agnostic way of writing producer/consumer/processor applications.
Here is a class defining an HTTP endpoint which sends the received Order
to an orders Channel
.
@Path("/orders")
public class OrderResource {
@Channel("outgoing-orders")
MutinyEmitter orderEmitter;
@POST
@Produces(MediaType.APPLICATION_JSON)
public Uni createOrder(Order order) {
return orderEmitter.send(order);
}
}
Configuring the channel to use a Kafka connector and providing the topic name is done through the application.properties
(or application.yml
) file:
# configure outgoing channel to use the kafka connector
mp.messaging.outgoing.outgoing-orders.connector=smallrye-kafka
# configure the topic name, by default it is the same as channel name
mp.messaging.outgoing.outgoing-orders.topic=orders
# optionally, configure port of kafka broker for development mode
%dev.quarkus.kafka.devservices.port=9092
Here is a processor which consumes from the orders
channel, filters out invalid Orders and sends valid ones to the valid-orders
channel:
public class OrderFilteringResource {
/** Filter invalid orders */
@Incoming("incoming-orders")
@Outgoing("outgoing-valid-orders")
public Multi filterInvalidOrders(Multi orders) {
return orders.filter(o -> o.quantity > 0 && o.customerId != null &&
o.productId != null);
}
}
While it is possible to define multiple @Incoming
channels, it is not possible to define multiple @Outgoing
channels. Fanning out is still possible by using multiple Emitter
objects binded to different outgoing channels.
Binding the channels to Kafka topics through application.properties
:
mp.messaging.incoming.incoming-orders.connector=smallrye-kafka
mp.messaging.incoming.incoming-orders.topic=orders
# the incoming channel must have a deserializer set for kafka
mp.messaging.incoming.incoming-orders.value.deserializer=net.syntio.labs.model.OrderDeserializer
mp.messaging.outgoing.outgoing-valid-orders.connector=smallrye-kafka
mp.messaging.outgoing.outgoing-valid-orders.topic=valid-orders
Finally, a consumer which consumes valid orders:
public class OrderConsumerResource {
private Random random = new Random();
@Incoming("incoming-valid-orders")
@Blocking(ordered = false)
public void saveOrder(Order order) throws InterruptedException {
// simulate blocking call
Thread.sleep(3000 + random.nextLong(500));
System.out.printf("%s ordered %d %ss.\n",
order.customerId, order.quantity, order.productId);
}
}
A @Blocking
annotation should be added for a method which makes a blocking call so it isn’t executed on the called thread but on an IO thread (check reactive architecture).
By default, the property ordered
is set to true
so the messages wouldn’t be processed concurrently.
To start the application in development with Quarkus CLI run quarkus dev
.
To start with the maven wrapper, run ./mvnw quarkus:dev
.
Switching brokers
Changing the underlying broker of a channel to another supported broker (e.g. AMQP broker) is simple:
1. Add dependency either by runningquarkus extension add quarkus-smallrye-reactive-messaging-amqp
or by adding the dependency to the pom.xml
:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-reactive-messaging-amqp</artifactId>
</dependency>
2. Modify the application.properties:
# change the connector from 'smallrye-kafka' to 'smallrye-amqp'
mp.messaging.outgoing.outgoing-valid-orders.connector=smallrye-amqp
# change property name from 'topic' to 'address'
mp.messaging.outgoing.outgoing-valid-orders.address=valid-orders
# Configure the incoming queue the same way
mp.messaging.outgoing.incoming-valid-orders.connector=smallrye-amqp
mp.messaging.outgoing.incoming-valid-orders.address=valid-orders
3. Manually deserialize as the AMQP connector will return a io.vertx.core.json.JsonObject
for objects:
@Incoming("incoming-valid-orders")
@Blocking(ordered = false)
public void saveOrder(JsonObject json) throws InterruptedException {
// deserialze manually
Order order = json.mapTo(Order.class);
// simulate blocking call
Thread.sleep(3000 + random.nextLong(500));
System.out.printf("%s ordered %d %ss.\n", order.customerId, order.quantity, order.productId);
}
The connector returns a io.vertx.core.json.JsonObject
, not a javax.json.JsonObject
.
That’s it! Quarkus will now run both Kafka and Artemis (an AMQP broker) as dev services when running in development mode.
List of SmallRye connectors supported by Quarkus:
-
Kafka
-
AMQP
-
RabbitMQ
-
Apache Camel
-
JMS
-
MQTT
3-Kafka Streams with Quarkus
To use Kafka Streams in a Quarkus application, the only two things you need to do is add the dependency quarkus-kafka-streams
and define a bean which has a method producing a Topology
:
@ApplicationScoped
public class TopologyProducer {
@Produces
public Topology buildTopology() {
return new StreamsBuilder()
.stream(...)
// perform transformations...
.to(...)
.build();
}
}
Quarkus will then detect the bean and start the processing application.
Accessing objects from the Kafka Streams application can be done through dependency injection, like so:
@ApplicationScoped
public class InteractiveQueries {
@Inject
KafkaStreams streams;
...
}
When a processing application is doing some aggregation, the application has its local state, which represents the current state of the aggregation. In a WordCount application, this would be the current count of all the encoutered words. This state would typically be stored in a KeyValueStore.
WordCount application
If we wanted to query the current count for a particular word and not read the counts from the output topic, until we find the latest record for the requested word, we could query the Local State of the application.
Here is how it would be done using Quarkus:
@Inject
KafkaStreams streams;
@GET
@Path("/counts/{word}")
@Produces(MediaType.TEXT_PLAIN)
public int getWordCount(@PathParam("word") String word) {
ReadOnlyKeyValueStore<string, integer=""> store =
streams.store(StoreQueryParameters.fromNameAndType(
"word-counts-store",
QueryableStoreTypes.keyValueStore()
));
return store.get(word);
}
</string,>
More about these queries, called interactive queries can be found here.
The official Quarkus with Kafka Streams code example can be found here.
4-Building a Docker image with GraalVM and Quarkus
GraalVM enables compiling Java applications so they can be run natively, without the JVM.
Quarkus employs GraalVM to compile Java applications natively for the container on which the application will be run. To accomplish this, the first thing needed to do is to install and configure GraalVM (guide).
Then:
1. Add the extension: quarkus extension add container-image-docker
2. Run the following command:
quarkus build --native \
-Dquarkus.native.container-build=true \
-Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-native-image:22.2-java17
-
The
--native
flag indicates that the program should be compiled. -
The second option specifies it should be compiled for a Linux container, not the host OS.
-
The third option specifies the image on which the application is built, the default is the same as this one but with Java 11.
3. Package the created runnable into a Docker image:
docker build -f src/main/docker/Dockerfile.native -t <repo>/<image>:<tag> .
Here is also a full guide for building a native executable.
5-Deploying to Kubernetes with Quarkus
Quarkus also comes with support for deploying applications to Kubernetes. Apart from deploying to a regular cluster, there is additional support to ease deployment to Minikube, for development purposes.
To easily deploy a Quarkus application to a Kubernetes cluster:
1. Add the Kubernetes extension: quarkus extension add kubernetes
.
2. Configure a registry to which the image will be pushed, as well as image name:
quarkus.container-image.registry=registry.syntio.net #optional, not pushed by default
quarkus.container-image.group=quarkus #optional, defaults to the system username
quarkus.container-image.name=demo-app #optional, defaults to the application name
quarkus.container-image.tag=1.0 #optional, defaults to the application version
3. Run quarkus build
, which builds the application, packages it into an image and pushes it to the registry, as well as creates a kubernetes.json
and a kubernetes.yml
file in the target/kubernetes/
directory.
Generation of these files can be configured through application.properties
using this guide.
4. Run kubectl apply -f target/kubernetes/kubernetes.yml
.
Deploying to Minikube with Quarkus
1. Add the Minikube extension: quarkus extension add quarkus-minikube
.
2. Set the environment to use Docker from inside Minikube:eval $(minikube -p minikube docker-env)
3. Run quarkus build
.
4. Build a Docker image (let’s say an image that uses JVM, not a compiled one):
docker build -f src/main/docker/Dockerfile.jvm -t <repo>/<image>:<tag> .
5. Deploy to Minikube:
kubectl apply -f target/kubernetes/minikube.yml
Of course, now Quarkus isn’t running any dev services, so a Kafka and an AMQP broker should be set up and the application should be configured to use them.
6-Comparing Mandrel (GraalVM) and JVM images
Mandrel is a downstream, open-source version of the GraalVM Community Edition, focused on the native-image
component in order to provide an easy way for Quarkus users to generate native images for their applications. It shares the same license as the GraalVM Community Edition.
In this test, the image size and startup times of a simple Kafka Streams application were compared.
Startup time of the container is measured as the Quarkus guide suggests. Two durations are measured:
- Time until the Quarkus application sends a
StartupEvent
(startup time). - Time until the application is ready to respond to an HTTP request (first response time).
The values are measured as an average of 5 separate runs.
application type | startup time [s] | first response time [s] | size [MB] |
JAR application |
0.961 |
1.633 | – |
JVM image | 2.154 | 2.835 | 602 |
native executable | 0.037 | 0.055 |
72 |
Mandrel UBI 8 image | 1.054 | 1.345 | 177 |
Mandrel distroless image | 1.057 | 1.169 | 97.6 |
The five different application types used for the test are as follows (from top to bottom):
- JAR packaged Java application, using
java -jar ...
. - Classic Docker image with a Java application, running on the JVM inside the container.
- Executable compiled by Mandrel on Linux (no JVM).
- Docker image with the executable (no JVM), built from an UBI 8 image.
- Docker image with the executable but on a distroless image.
Conclusion
Quarkus is a rich framework for developing applications meant to be containerized. The CLI eases creating a Quarkus project, dependency management and starting the application. The dev services are of great help, as setting up these services manually can take up a lot of time in the development process.
Building the application for native execution takes up a lot of resources and time but offers great benefits when building a production-grade application, since the final artifact is much smaller memory-wise which greatly improves startup time.