Transformation in Camel: How to do it

Updated 27 November 2021

Tags: Apache Camel

Comments

How do you transform data in Apache Camel?

If you’re just getting started with Apache Camel, it can seem like there is a bewildering choice of options.

A processor? One of the EIP thingies? A message translator?

Transforming data in Camel is usually done in one of the following ways:

• Mapping with Java code and Type Converters

• Using a specialised Camel component, like XSLT, Bindy or Atlasmap

• Marshalling/unmarshalling with data formats, like CSV and JSON

• Using a templating engine, like Velocity, Mustache or Freemarker

Transformation approaches in Camel: marshalling, type conversion, and templating!

So how do you know which one to choose? After all, they seem to all do the same thing, don’t they?

Let’s have a look at each of them.

I’m not going to recommend one approach over another here. The intent of this article is to give you a broad overview of the options so that you can dig into the Camel documentation and examples and make a choice.

So let’s take a look!

Transforming using Java code

Also known as… The Java architect’s dream

In this approach, your input and output data need to both be Java objects, and you use Java statements to map between them. When you need to transform from your source type to your target type, you write a method that creates a new target object, and populates it with the relevant fields from the source object. Then, you tell Camel to invoke that method (e.g. using bean, or perhaps a Processor).

For example, you might convert a Lead object to a Customer. So you write a transformation method public static Customer convertToCustomer(Lead lead):

public static Customer toCustomer(Lead lead) {
    Customer customer = new Customer();

    customer.setName(lead.getName());
    customer.setCompany(lead.getCompany());
    customer.setCity(lead.getCity());

    return customer;
}

If your source data comes into Camel as a text-based format (like 🖹 XML 🖹), then firstly you need to unmarshal from that text-based format, into a Java object, like a POJO.

This approach is proper, full-on Java. It often requires writing lots of Java code. However, it’s also strongly typed. In my opinion this is a good thing. The great thing about Java’s type system is that you get built-in “type safety” – so, for example, you can’t put a String value into a slot that’s designed for an Integer.

There are some tools that can make this process a little easier. If you’re working with XML, then you can use Java’s JAXB (implemented in tools like Apache CXF) to create Java classes from your XML schema and then convert that XML object into Java.

Even better if…. you register a TypeConverter

Once you’ve written your transformation code, you can call it from Camel, just like any other regular Java method.

But you can make it more… Camelly… by registering it as a TypeConverter. This adds your code to Camel’s library of adaptors, that can convert Java objects from TypeA to TypeB. When you write your own transformation code, you can register it as a TypeConverter, so that Camel knows how it can convert from your Lead to a Customer.

To do this you just add the @Converter annotation to your class:

import org.apache.camel.Converter;

@Converter
public class CustomerConverter { ... }

You also add your full class name into the file resources/META-INF/services/org/apache/camel/TypeConverter:

com.cleverbuilder.cameldemos.typeconverters.CustomerConverter

Now you can use .convertBodyTo(Customer.class) in your Camel routes, and Camel will execute the typeconverter.

Transforming using a specialised Camel component (Endpoint)

Also known as…. Handing the job to a specialist

Writing Java code is “fine”. Although I wouldn’t call it “pleasant”, unless you really enjoy writing reams and reams of boilerplate code. So you might wonder if there are tools that can help you transform from one format to another – without writing all this boilerplate code yourself.

That’s where the specialist transformation components come in. They’re a bunch of workers who you can call on to get a specific job done. And unlike some workers, they generally turn up on time, and don’t say “I can’t do this for you, because I’m not qualified mate. But I will still be charging you for this visit.”

Sorry. Rant.

When you initialise one of these components – either explicitly in code, or perhaps automatically in a framework like Spring Boot – you get an Endpoint that you can push your message through, as part of a route. When the message goes through the Endpoint, the component kicks in, transforms your message, and returns the output.

This makes your Camel route look something like this:

from("file:somewhere...")
    .to("some-component:blah?someConfig=1&anotherConfig=true")
    .to("file.....")

As you can see, your Camel code looks rather clean. Now you’re using Camel as an orchestrator, handing off to other components to do the work.

So, pray, what are these 🎠 magical components 🎠 which you can use? Some good examples of these transformation components in Camel are:

Component	What it does	Based on
XSLT	Transforms XML to XML directly, using an XSLT transformation file. You’ll know one of these when you see it.
XSLT Saxon	Use this component if you want to use the Saxon library. Gives you a few fancier transforms and perhaps better performance.	Saxon-HE
Dozer	Dozer is for Java-to-Java mapping. It replaces all of that boilerplate code above, with an XML mapping file. (Whether that’s a good thing is debatable :-))	Dozer
Atlasmap	This is the new kid on the block. It can map between JSON, XML and Java objects. Just like Dozer, it works with a mapping file. You can create mapping files in your dev environment using the Atlasmap extension for Visual Studio Code	Atlasmap

Using data formats

Also known as… The marshallers and unmarshallers.

This is another piece of the puzzle. Data formats in Camel are utility layers for dealing with things like Zip files, Avro, Base64 encoded files, HL7 (Healthcare) files, JSON, Protobuf and much more.

Once you’ve configured a data format, you can plug it into a marshal or unmarshal step, like this:

// import org.apache.camel.dataformat.csv.CsvDataFormat;
// import org.apache.camel.model.dataformat.JsonDataFormat;

// Set up a CSV data format
CsvDataFormat csv = new CsvDataFormat();

// Set up a simple JSON output format
JsonDataFormat json = new JsonDataFormat(JsonLibrary.Jackson);

// Convert an incoming CSV into JSON
from("direct:start")
        .unmarshal(csv) // unmarshal from CSV
        .marshal(json) // marshal out to JSON
        .to("direct:end");

Bindy is a customisable data format that can handle fixed-width (e.g. FIX) and variable-width (e.g. CSV) files. Bindy is a damn cool library that will help you out of a hole, especially if you’re working with some rather unusual or old file formats.

With Bindy, as long as you can establish the rules of the file format – like how many characters the first column is, what the separators are, etc – you can handle it with Camel.

See a demo of Bindy in action.

Using a template engine

Also known as… I need to produce a report for The Boss

The final option I’m going to introduce in this article is the templating engines. These are fantastic if you need to build any sort of text output that’s going to be seen by people.

Think things like emails, documents, web pages …. Just like mail merge, template engines can take a template, and some source data and produce a document.

If you’re still not sure what I mean, then I’m talking about support for things like Velocity and Mustache { 👨.

In Camel, the templating engine reads values from your Exchange. So it can access things like the message Body and Headers.

Camel has components for a few templating engines. I’ve not worked with all of these so I don’t know the differences, but if you need something specific, check with the documentation of each of them first:

• Chunk

• Freemarker

• Mustache

• MVEL

• Velocity

That’s transformation in Camel, in a nutshell

And that’s an overview of transformation in Apache Camel. You’ve got a lot of different options, depending on exactly what you need Camel to do.