Apache Camel Tutorial

Updated:

Apache Camel Tutorial

Wanting to get started with the Apache Camel framework? Then check out this free tutorial on this popular integration toolset for Java.

I’ve written this Apache Camel tutorial for complete beginners. It introduces the core Camel concepts to you, and shows you how to use Maven to build and run your first ever Camel project.

What you’ll learn

By the end of this tutorial, you will:

  • Understand what Apache Camel is, and what it can do
  • Create a basic Apache Camel project using Maven
  • See how Camel works with Spring Boot
  • See how logging is used
  • Run your application and test it out

This tutorial is suitable for you if:

  • You want to learn Apache Camel

  • You want to know how to create a Camel project

  • You’re just getting started with Red Hat Fuse and want to know how to build Camel routes

  • You’re just getting started with Talend ESB and want to know the underlying concepts so that you can understand Mediation routes

  • You’re a Java developer and you want to level-up your skills with a powerful framework

You should have a little bit of Java experience, and be comfortable with using the command line.

Camel in 4 minutes

Let’s start at the top. Want a quick intro to Apache Camel? Check out my 4-minute video, which covers the main concepts. Then, keep reading to find out more:

Introducing Camel

So what is Apache Camel and what is it used for?

Apache Camel is an integration library for Java. It’s a set of Java APIs that help you integrate and process data between different computer systems.

In other words, Camel is like glue between different applications.

Camel_toolkit

Camel comes with components that let you integrate lots of different applications — everything from web services, to reading and writing files on disk. You can even connect Camel with web apps like Twitter and Salesforce.

You can think of Camel as a plumbing toolkit for Java. Just like real plumbing pipes, Camel takes data from one point, and pipes it to another. Along the way, the data can be changed, transformed, or sent through other pipes.

Included in your plumbing toolkit is a range of adaptors that fit all sorts of different apps and software systems. With a full range of tools at your disposal, it’s then up to you how you choose to build the plumbing.

Camel facts

Hopefully you’re beginning to understand what Camel is. Here are some facts about Camel to help you learn more about the project itself:

  • It’s built in Java – this might seem an obvious point to make, but once you understand this you’ll see that you have the full power of Java at your disposal

  • The entire code is completely open sourcecheck it out at Github - nothing is hiding behind expensive closed-source products. You can contribute to the project and give back to the community. (You don’t have to be a coder to contribute)

  • It’s not just for web services – it’s a general integration framework. This means you can choose to use Camel to build something to do with web services, but you don’t have to.

  • It comes with a huge library of components – if you can think of a system you’d like to interact with, somebody has probably already written a component for it; you can do everything from pushing files to AWS, to sending a tweet.

  • It’s mature – Apache Camel is at the foundation of some commercial integration products, like Red Hat Fuse and Talend ESB. Engineers working at these companies contribute code back into the open source Camel project to make it better for everyone.

What is Camel used for?

Almost any time you need to move data from A to B, you can probably use Camel. Any of the following scenarios could be implemented using Camel:

  • Picking up invoices from an FTP server and emailing them to your Accounts department

  • Taking files from a folder and pushing them into Google Drive

  • Taking messages from a JMS queue and using them to invoke a web service

  • Making a web service that allows users to retrieve customer details from a database

These are just a few examples. With the wide range of components available in Camel, the sky’s the limit.

Camel concepts explained

How do you think like a Camel? The diagram below shows some of the core concepts in Camel.

Visualising a Camel route
Visualising a Camel route

In this section I’m going to look at each of these in turn:

Route

The basic concept in Camel is the route. Routes are objects which you configure in Camel, which move your data from A to B.

To use the plumbing example from earlier, a route is a pipe that moves data from one place to another. It moves data between things called endpoints.

You can create routes in Camel either using a Java syntax, or using an XML syntax. Here’s a very simple Java route:

// This is a complete Camel route definition!
from("file:home/customers/new")
    .to("file:home/customers/old");

Endpoints

In Camel, an endpoint represents any other external system to Camel. For an example, at the start of a route, Camel receives a message from an endpoint.

Then, the message might be processed in some way — perhaps by an EIP — before being sent to another destination endpoint.

Components

To allow Camel to connect to an endpoint, it comes with a library of components. A component is simply like a plug that allows you to connect to an external system (such as a file on disk, a mailbox, or an app like Dropbox, Twitter, etc). You can also think of it like an adaptor.

A huge library of components… Any time that you need to put data into or out of an application, you’ll probably find that a Camel component already exists to do the job for you. This means you don’t need to waste time writing your own code to read a file, or invoke a web service. You just find the component you need, and use it.

Camel components are reusable, open source, and you can even contribute your own. Here are some of the most common components, and how you might reference them in an endpoint:

Component Purpose Endpoint URI
HTTP for creating or consuming web sites http:
File for reading and writing files file:
JMS for reading and writing to messaging queues jms:
Direct for joining your Camel routes together direct:
Salesforce for getting data in and out of Salesforce salesforce:

You can find a list of all the Camel components here.

As you can see, each component can usually read and write:

  • A component that is configured to write something is called a producer — for example, writing to a file on disk, or writing to a message queue.
  • A component that is configured to read something is called a consumer — for example, reading a file from disk, or receiving a REST request.

Between each endpoint, the data can also be transformed or modified, either by passing the data through another endpoint, or by using an EIP.

Enterprise Integration Patterns (EIPs)

EIPs are another important part of Camel. They do special processing on messages according to the patterns defined in the book, Enterprise Integration Patterns, that I mentioned earlier.

When you want to perform some common activities on a message, such as transformation, splitting and logging, you’ll use an EIP. Here are some common EIPs in Camel:

EIP name What it does Java syntax
Splitter Splits a message into multiple parts .split()
Aggregator Combines several messages into one message .aggregate()
Log Writes a simple log message .log()
Marshal Converts an object into a text or binary format .marshal()
From* Receives a message from an endpoint .from()
To* Sends a message to an endpoint .to()

(Yes, from and to are EIPs too!)

Camel Context

Finally, to run and manage your routes, Camel has a container called the Camel Context. Your routes run inside this engine. You could think of it almost like a mini application server.

When Camel starts, it reads your route definitions (in Java or XML), creates the routes, adds them to a Camel Context, and starts the Camel Context.

When Camel terminates, it shuts down your routes, and closes the Camel Context.

What does a Route look like?

So. Now you know that when you develop in Camel, you create routes that move data between endpoints, using components.

It’s probably easiest to understand all of this by looking at some code, right?

Although Camel is a library for Java, it can be configured using one of two languages - either Java or XML. In Camel-speak, these are known as DSLs (Domain Specific Languages).

Each route starts with a from, configured with a URI, that defines the endpoint where the data is coming from.

A route can consist of multiple steps — such as transforming the data, or logging it. But a route usually ends with a to step, which describes where the data will be delivered to.

A really simple route in Camel’s Java DSL could look something like this:

from("file:home/customers/new")
    .to("file:home/customers/old");

In this example, we use the File component (identified by the file: prefix) to move all incoming files in the customers/new folder, to the customers/old folder.

That same route above could be expressed in Camel’s XML DSL like this:

<route>
    <from uri="file:home/customers/new"/>
    <to uri="file:home/customers/old"/>
</route>

But… what if we wanted to add another step in our route? Let’s say we want to log a message when we’ve received a file. Then we simply need to add our new step in between the existing steps. Like this:

from("file:home/customers/new")
    .log("Received a new customer!")
    .to("file:home/customers/old");

In the code above, a message is moved from the new folder to the old folder. In the middle, we use the Enterprise Integration Pattern (EIP) called log, which writes a simple Log message to the console.

And in the XML DSL, it would look something like this:

<route>
    <from uri="file:home/customers/new"/>
    <log message="Received a new customer!"/>
    <to uri="file:home/customers/old"/>
</route>

Now you know what a route looks like, let’s quickly look at how data flows through your routes.

What does data look like in Camel?

A nice envelope

Camel treats data as individual messages – like letters flowing through a post office.

Each message is an individual object. A message can be huge, or it can be very small. Camel has an object to represent messages, and helpfully it’s called Message.

A Message has a body, where the message content lives. It also has headers, which can be used to hold values associated with the message. The Message object is then passed along a route.

A Message is part of a Camel object called an Exchange. You’ll often see the term Exchange mentioned in Camel documentation. An Exchange is simply a message or interaction currently taking place inside your Camel route.

The important thing to understand about Camel’s message model is that the message body can contain almost any kind of Java object, such as a List, Map or String. The body doesn’t have to be a String, like JSON or XML.

The Exchange body can contain any type of Java object
Examples of different body types in Camel

The real power of Camel becomes clear when you start using these different types of objects. Camel has good built-in support for converting between different object types.

In fact, for many common file types, you might barely even have to write any conversion code. (Less time writing boilerplate code? Sounds good, doesn’t it?)

You’ll learn more about Camel’s message model as you get more experienced with Camel!


Your first Camel project

In this part of the tutorial, I’m going to show you how to create a new Apache Camel project using Maven, even if you’ve never worked with Camel before. By the end of this section, you will have already created your first project, run it, and begun to understand the power of Camel.

We’re going to do this using a Maven archetype. Maven archetypes are like templates for new Java projects. Camel provides quite a few of them with each release. This make it easy for you to start new projects.

Why bother with archetypes? The archetypes for Camel are an opinionated way of starting a new Camel project. In other words, they promote best practices. So I always use the archetypes where possible.

For this tutorial, you will need:

Note for Mac users: If you’re using a Mac, I recommend using the Homebrew package manager to help you install Maven. Follow the instructions at http://brew.sh to install. Then, once Homebrew is installed, install Maven by typing brew install maven from a Terminal window.

Creating a Camel project using Maven

  1. From your desktop, drop to a Terminal or Command Prompt.

  2. Type this command (all on one line):

    mvn archetype:generate
        -DarchetypeGroupId=org.apache.camel.archetypes
        -DarchetypeArtifactId=camel-archetype-spring-boot
        -DarchetypeVersion=2.24.0
    
  3. Then answer the questions when prompted:

    • Define value for property ‘groupId’: com.example

    • Define value for property ‘artifactId’: my-camel-app

    • Define value for property ‘version’ 1.0-SNAPSHOT: (type Enter)

    • Define value for property ‘package’ com.example: (type Enter)

    • Finally, you’ll be prompted to confirm, type Y and press Enter.

  4. Maven will now create your new Camel project. When it’s finished, you will find it inside the new directory my-camel-app!

Looking inside the project

A Camel project created from the camel-archetype-spring-boot archetype (v2.24.0) will have a folder structure that looks like this:

my-camel-app
├── pom.xml
└── src
    ├── main
    │   ├── java
    │   │   └── com
    │   │       └── example
    │   │           ├── MySpringBean.java
    │   │           ├── MySpringBootApplication.java
    │   │           └── MySpringBootRouter.java
    │   └── resources
    │       ├── application.properties
    │       └── META-INF
    │           ├── LICENSE.txt
    │           └── NOTICE.txt
    └── test
        ├── java
        │   └── com
        │       └── example
        └── resources

The RouteBuilder class

The interesting code resides in MySpringBootRouter.java. This is a RouteBuilder class, where your routes should go. The Maven archetype includes a sample route to get you started:

package com.example;

import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;

@Component
public class MySpringBootRouter extends RouteBuilder {

    @Override
    public void configure() {
        // The route is defined here - keep reading to find out more
    }

}

Here, you add route definitions to the configure() method that describe each route you want Camel to create.

If you look at the code, you’ll see that the MyRouteBuilder class comes with a route defined inside it already.

The bootstrap class

For your convenience, the project also includes another Java class, MySpringBootApplication, which is used to bootstrap and run the application. This is a standard Spring Boot main class:

package com.example;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class MySpringBootApplication {

    /**
     * A main method to start this application.
     */
    public static void main(String[] args) {
        SpringApplication.run(MySpringBootApplication.class, args);
    }

}

Now let’s look at the code and see what this project does.

What does the project do?

The demo project defines a route using the code below. The code is given in a special Camel syntax, or DSL (Domain-specific Language):

from("timer:hello?period=").routeId("hello")
    .transform().method("myBean", "saySomething")
    .filter(simple("${body} contains 'foo'"))
        .to("log:foo")
    .end()
    .to("stream:out");

This code looks complicated, but it really just generates messages and prints them.

Illustration of the route in the Spring Boot archetype

Let’s look at each part of the route in detail:

  1. It uses a timer (timer:...) to kick off the route. Camel’s Timer component can be used to fire a route at certain interval. Here, the interval is timer.period, which is a property defined in the file application.properties:

     from("timer:hello?period=")
    

    This shows how you can use Spring Boot properties in Camel routes.

  2. The transform() EIP states that we want to change the content of the Message. In this example, we use the bean() method, which invokes a method on a Java bean (a Java class).

    • Camel looks for a bean in the registry named myBean. Because we’re using Spring Boot, it searches the Spring context for the bean, and it finds it, because the MySpringBean class is annotated with @Component("myBean").
    • Camel invokes the method saySomething on that bean.
     .transform().method("myBean", "saySomething")
    
  3. The filter() EIP tells Camel to filter the message, based on some expression. In this case, we use Camel’s simple expression language to check whether the message Body contains “foo”. If so, we dump the current exchange to the log.

     .filter(simple("${body} contains 'foo'"))
         .to("log:foo")
     .end()
    
  4. Finally, the content of the message body is written to the output stream (that is, standard out or STDOUT):

     .to("stream:out")
    

Now let’s run the route and see what happens.

Testing the Camel application

Let’s test the application by running it with Maven.

From your Terminal or Command Prompt, run Maven with the spring-boot:run goal:

mvn clean spring-boot:run

This will compile your application and use the Spring Boot Maven Plugin to run your application.

If you want to run this project from inside Eclipse or IntelliJ, just create a new Maven Run Configuration for the project, and configure it to execute the goal: spring-boot:run.

When the application starts, you’ll see the logs will look something like this:

Logs from the Spring Boot Camel archetype application

The logs above tell us that Apache Camel started correctly.

Then, every few seconds, you should see the text Hello World in the logs. This is because Camel is triggering the route to be executed every few seconds.

When the route executes, the message body is transformed (set to Hello world), and then written to the standard output stream, or console.

To end the app you can just press Ctrl+C.

What next?

This is a very basic demo of Camel, but you can start to see what it’s capable of. Using components and EIPs you can build almost any kind of data flow that you can think of.

Now check out my other articles to help you learn Camel:

And now, it’s over to you. What do you want to build in Camel?

Questions about this tutorial? Feedback? Share your thoughts and your Camel plans in the comments below!

Leave a Comment

You can use Markdown in your comment. To write code, indent lines by 4 spaces.