Saddle up: A Camel tutorial for beginners

Updated 12 December 2021

Saddle up, choose your Camel and watch this video first.

What is Apache Camel?

So what is Apache Camel, and what is it used for?

Apache Camel is an integration library for Java. It’s a language, set of Java APIs and a whole heap of components that let you integrate and process data between different computer systems.

In other words, Camel is like glue for Java, that connects different applications together.

The Apache Camel toolkit — Riding with the Camel

Camel comes with over 320 components for working with lots of different applications. Camel can work with almost everything, from web services to reading and writing files on disk. Camel can even talk to apps like Twitter, Facebook and Salesforce.

You can think of Apache Camel like a plumbing toolkit for Java. Just like real plumbing pipes, Camel takes data from one point, and pipes it to another. Along the way, the data can be changed, transformed, or sent through other pipes.

Included in your plumbing toolkit is a range of adaptors that fit all sorts of different apps and software systems. With a full range of tools at your disposal, it’s then up to you how you choose to build the plumbing.

Where did the idea for Camel come from?

Camel was inspired by the book Enterprise Integration Patterns, which is an academic textbook about integrating software systems. The authors of the book (which is now considered a ‘classic’!) took lots of common integration use cases, distilled them into reusable patterns and documented them. In the book, each pattern is described, and in some cases, there is also some example code.

The Camel developers thought it would be a great idea to build a Java framework that represented the ideals of the book. And so Apache Camel borrows heavily from this book.

Camel facts

Hopefully you’re beginning to understand what Camel is. Here are some facts about Camel to help you learn more about the project itself:

It’s built in Java – this might seem an obvious point, but once you understand this you’ll see that you have the full power of Java at your disposal
The entire code is completely open source – check it out at Github - nothing is hiding behind expensive closed-source products. You can contribute to the project and give back to the community. (You don’t have to be a coder to contribute)
It’s not just for web services – it’s a general integration framework. This means you can choose to use Camel to build something to do with web services, but you don’t have to.
It comes with a huge library of components – if you can think of a system you’d like to interact with, somebody has probably already written a component for it; you can do everything from pushing files to AWS, to sending a tweet.
It’s mature – Apache Camel is at the foundation of some commercial integration products, like Red Hat Fuse and Talend ESB. Engineers working at these companies contribute code back into the open source Camel project to make it better for everyone.

Common use cases for Camel

Almost any time you need to move data from A to B, you can probably use Camel. Any of the following scenarios could be implemented using Camel:

Picking up invoices from an FTP server and emailing them to your Accounts department
Taking files from a folder and pushing them into Google Drive
Taking messages from an ActiveMQ queue and sending them to a web service
Making a web service that allows users to retrieve customer details from a database

These are just a few examples. With the wide range of components available in Camel, the sky’s the limit.

Camel concepts explained

How do you think like a Camel? The diagram below shows some of the core concepts in Camel.

In this section I’m going to look at each of these in turn:

A Route is like a pipeline

The most basic concept in Camel is the route. Routes are objects which you configure in Camel, which move your data from A to B.

To use the plumbing example from earlier, a route is a pipe that moves data from one place to another. It moves data between things called endpoints.

You can create routes in Camel either using a Java syntax, or using an XML syntax. Here’s a very simple route written in the Java syntax. This moves a file from one folder to another:

                // This is a complete Camel route definition!
// Camel will move files from one folder to another
from("file:home/customers/new")
    .to("file:home/customers/old");

              

Endpoints

In Camel, an endpoint is an interface, through which Camel exchanges a message with another system. Camel can receive a message from an endpoint, or send a message to an endpoint.

Endpoints are the steps along the way in your routes. You can declare them in a couple of different ways, but the most common way is to declare them using a syntax that looks like a URI, like this:

prefix:mainpart?option1=xxx&option2=xxx...

…where prefix is the Component that the endpoint refers to, and mainpart is some bit of configuration that the endpoint needs.

For example: at the start of a route, Camel receives a message from an endpoint. If this endpoint is file:/myfolder, then Camel will use the File component to read files in /myfolder.

Components

To allow Camel to create an endpoint, it comes with a library of components. A component is just like a plug that allows you to connect to an external system, such as a file on disk, a mailbox, or an app like Dropbox or Twitter.

You can think of a Component as a factory for creating Endpoints.

Whenever you need to put data into or out of an application, you’ll probably find that a Camel component already exists to do the job for you. This means you don’t need to waste time writing your own code to read a file, or invoke a web service. You just find the component you need, and use it.

Camel components are reusable, open source, and you can even contribute your own. Here are some of the most common components, and how you might reference them in an endpoint:

Component	Purpose	Endpoint URI
HTTP	for creating or consuming web sites	`http:`
File	for reading and writing files	`file:`
JMS	for reading and writing to JMS message queues	`jms:`
Direct	for joining your Camel routes together	`direct:`
Salesforce	for getting data in and out of Salesforce	`salesforce:`

You can find a list of all the Camel components here.

As you can see, each component can usually read and write:

A component that is configured to write something is called a producer – for example, writing to a file on disk, or writing to a message queue.
A component that is configured to read something is called a consumer – for example, reading a file from disk, or receiving a REST request.

Between each endpoint, the data can also be transformed or modified, either by passing the data through another endpoint, or by using an EIP.

Enterprise Integration Patterns (EIPs)

EIPs are another important part of Camel. They do special processing on messages according to the patterns defined in the book, Enterprise Integration Patterns, that I mentioned earlier.

When you want to perform some common activities on a message, such as transformation, splitting and logging, you’ll use an EIP. Here are some common EIPs in Camel:

EIP name	What it does	Java syntax
Splitter	Splits a message into multiple parts	`.split()`
Aggregator	Combines several messages into one message	`.aggregate()`
Log	Writes a simple log message	`.log()`
Marshal	Converts an object into a text or binary format	`.marshal()`
From*	Receives a message from an endpoint	`.from()`
To*	Sends a message to an endpoint	`.to()`

(Yes, from and to are EIPs too!)

Camel Context

Finally, to run and manage your routes, Camel has a container called the Camel Context. Your routes run inside this engine. You could think of it almost like a mini application server.

When Camel starts, it reads your route definitions (in Java or XML), creates the routes, adds them to a Camel Context, and starts the Camel Context.

When Camel terminates, it shuts down your routes, and closes the Camel Context.

What does a Route look like?

So. Now you know that when you develop in Camel, you create routes that move data between endpoints, using components.

It’s probably easiest to understand all of this by looking at some code, right?

Although Camel is a library for Java, it can be configured using one of two languages - either Java or XML. In Camel-speak, these are known as DSLs (Domain Specific Languages).

Each route starts with a from, configured with a URI, that defines the endpoint where the data is coming from.

A route can consist of multiple steps – such as transforming the data, or logging it. But a route usually ends with a to step, which describes where the data will be delivered to.

A really simple route in Camel’s Java DSL could look something like this:

                from("file:home/customers/new")
    .to("file:home/customers/old");

In this example, we use the File component (identified by the file: prefix) to move all incoming files in the customers/new folder, to thecustomers/old folder.

That same route above could be expressed in Camel’s XML DSL like this:

                <route>
    <from uri="file:home/customers/new"/>
    <to uri="file:home/customers/old"/>
</route>

              

But… what if we wanted to add another step in our route? Let’s say we want to log a message when we’ve received a file. Then we simply need to add our new step in between the existing steps. Like this:

                from("file:home/customers/new")
    .log("Received a new customer!")
    .to("file:home/customers/old");

              

In the code above, a message is moved from the new folder to the old folder. In the middle, we use the Enterprise Integration Pattern (EIP) called log, which writes a simple Log message to the console.

And in the XML DSL, it would look something like this:

                <route>
    <from uri="file:home/customers/new"/>
    <log message="Received a new customer!"/>
    <to uri="file:home/customers/old"/>
</route>

              

Now you know what a route looks like, let’s quickly look at how data flows through your routes.

What does data look like in Camel?

Camel treats data as individual messages – like letters being sorted through a post office.

Each message is an individual object. A message can be huge, or it can be very small. Camel has an object to represent messages, and helpfully it’s called Message.

A Message has a body, where the message content lives. It also has headers, which can be used to hold values associated with the message. The Message object is then passed along a route.

A Message is part of another Camel object called an Exchange. You’ll often see the term Exchange mentioned in Camel documentation. An Exchange is simply a message or interaction currently taking place inside your Camel route.

The important thing to understand about Camel’s message model is that the message body can contain almost any kind of Java object, such as a List, Map or String. The body doesn’t have to be a String, like JSON or XML.

The Exchange body can contain any type of Java object — Examples of different body types in Camel

The real power of Camel becomes clear when you start using these different types of objects. Camel has great built-in support for converting between common object types.

In fact, for many common file types, you might barely even have to write any conversion code. (Less time writing boilerplate code? Sounds good, doesn’t it?)

You’ll learn more about Camel’s message model as you get more experienced with Camel!

Example: Creating a Camel project

In this part of the tutorial, I’m going to show you how to create a new Apache Camel project using Maven, even if you’ve never worked with Camel before. By the end of this section, you will have already created your first project, run it, and begun to understand the power of Camel.

We’re going to do this using a Maven archetype. Maven archetypes are like templates for new Java projects. Camel provides quite a few of them with each release. This make it easy for you to start new projects.

For this tutorial, you will need:

Creating a Camel project using Maven

From your desktop, drop to a Terminal or Command Prompt.

Type this command (all on one line):

                     mvn archetype:generate
 -DarchetypeGroupId=org.apache.camel.archetypes
 -DarchetypeArtifactId=camel-archetype-spring-boot
 -DarchetypeVersion=3.4.0

                  

Then answer the questions when prompted:
- Define value for property ‘groupId’: com.example
- Define value for property ‘artifactId’: my-camel-app
- Define value for property ‘version’ 1.0-SNAPSHOT: (type Enter)
- Define value for property ‘package’ com.example: (type Enter)
- Finally, you’ll be prompted to confirm, type Y and press Enter.
Maven will now create your new Camel project. When it’s finished, you will find it inside the new directory my-camel-app!

Looking inside the project

A Camel project created from the camel-archetype-spring-boot archetype (v3.4.0) will have a folder structure that looks like this:

                my-camel-app
├── pom.xml
└── src
    ├── main
    │   ├── java
    │   │   └── com
    │   │       └── example
    │   │           ├── MySpringBean.java
    │   │           ├── MySpringBootApplication.java
    │   │           └── MySpringBootRouter.java
    │   └── resources
    │       ├── application.properties
    │       └── META-INF
    │           ├── LICENSE.txt
    │           └── NOTICE.txt
    └── test
        ├── java
        │   └── com
        │       └── example
        └── resources

              

The RouteBuilder class

The interesting code is in MySpringBootRouter.java. This is a RouteBuilder class, and this is where you define your Camel routes. The Maven archetype includes a sample route to get you started:

                package com.example;

import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;

@Component
public class MySpringBootRouter extends RouteBuilder {

    @Override
    public void configure() {
        from("timer:hello?period=").routeId("hello")
            .transform().method("myBean", "saySomething")
            .filter(simple("${body} contains 'foo'"))
                .to("log:foo")
            .end()
            .to("stream:out");
    }

}

              

Here, you add route definitions to the configure() method that describe each route you want Camel to create.

If you look at the code, you’ll see that the MyRouteBuilder class comes with a route defined inside it already.

We’ll look at the route shortly.

The utility class

The Camel route depends on another class. In the Camel route, references a bean (a Java object) using the identifier "myBean". This is actually resolved by looking in the Spring context for any beans with the ID myBean. Which means it actually means this class, MySpringBean:

                @Component("myBean")
public class MySpringBean {

    @Value("${greeting}")
    private String say;

    public String saySomething() {
        return say;
    }

}

              

This utility class is included to show you how you can invoke Java code from a Camel route. The class has a method saySomething(), which the Camel route calls. We’ll see this shortly.

The bootstrap class

For your convenience, the project also includes another Java class, MySpringBootApplication, which is used to bootstrap and run the application. This is a standard Spring Boot main class:

                package com.example;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class MySpringBootApplication {

    /**
     * A main method to start this application.
     */
    public static void main(String[] args) {
        SpringApplication.run(MySpringBootApplication.class, args);
    }

}

              

Now let’s look at the code and see what this project does.

What does the project do?

The demo project defines a route using the code below. The code is given in a special Camel syntax, or DSL (Domain-specific Language):

                from("timer:hello?period=").routeId("hello")
    .transform().method("myBean", "saySomething")
    .filter(simple("${body} contains 'foo'"))
        .to("log:foo")
    .end()
    .to("stream:out");

              

This code looks complicated, but it really just generates messages and prints them.

Visual illustration of the Camel route — Riding our first Camel route

Let’s look at each part of the route in detail:

It uses a timer (timer:...) to kick off the route. Camel’s Timer component can be used to fire a route at certain interval. Here, the interval is timer.period, which is a property defined in the file application.properties:
```
 from("timer:hello?period=")
```
This shows how you can use Spring Boot properties in Camel routes.
The transform() EIP states that we want to change the content of the Message. In this example, we use the bean() method, which invokes a method on a Java bean (a Java class).
- Camel looks for a bean in the registry named myBean. Because we’re using Spring Boot, it searches the Spring context for the bean, and it finds it, because the MySpringBean class is annotated with @Component("myBean").
- Camel invokes the method saySomething on that bean.
```
 .transform().method("myBean", "saySomething")
```
The filter() EIP tells Camel to filter the message, based on some expression. In this case, we use Camel’s simple expression language to check whether the message Body contains “foo”. If so, we dump the current exchange to the log.
```
 .filter(simple("${body} contains 'foo'"))
 .to("log:foo")
 .end()
```
Finally, the content of the message body is written to the output stream (that is, standard out or STDOUT):
```
 .to("stream:out")
```

Now let’s run the route and see what happens.

Running the Camel application

Let’s run the application. We’ll run it with Maven.

From your Terminal or Command Prompt, run Maven with the spring-boot:run goal:

mvn clean spring-boot:run

This will compile your application and use the Spring Boot Maven Plugin to run your application.

When the application starts, you’ll see the logs will look something like this:

Screenshot of log output from Camel — Logs from our Camel application

The logs above tell us that Apache Camel started correctly.

Then, every few seconds, you should see the text Hello World in the logs. This is because Camel is triggering the route to be executed every few seconds.

When the route executes, the message body is transformed (set to Hello world), and then written to the standard output stream, or console.

To end the app you can just press Ctrl+C.

What next?

This is a very basic demo of Camel, but you can start to see what it’s capable of. Using components and EIPs you can build almost any kind of data flow that you can think of.

Now check out my other articles to help you learn Camel:

Learn how to create a REST service in Apache Camel when you want to create your own REST API, using Camel’s REST DSL
Learn how to use ActiveMQ with Apache Camel for reading and writing to a JMS message broker
Learn about Camel’s content-based routing to choose how to process messages based on their content

And now, it’s over to you. What do you want to learn how to do in Camel?

Questions about this tutorial? Feedback? Share your thoughts and your Camel plans in the comments below!