4 ways you're not (but should be) using Talend tMap Component
By Talend Community
Ever wondered what our most-used components are? Here at Talend, one of my “Shadow IT” jobs is to report on component usage. If you've ever used Talend Studio (either the open source Talend Open Studio or the commercial version) you most likely already know and love the component tMap.
The component tMap is the most-used component by a long shot. Why? Simply put, it’s extremely versatile and useful. When I was first asked to write this article I thought, “Why not, this will be easy.” But as I started actually picking out use cases I quickly realized that there are so many more than just four to choose from! I actually challenge everyone to respond and tell me below what you think the top features are of the tMap component. I would love to hear from you.
I will start out by listing the most obvious but most needed and then get into some more advanced uses. (I will try and sneak a couple in together so I can have more than 4 - hopefully, our editor doesn't catch it.)
Editors note: Caught - but proceed.
1. Mapping of course!
The tMap's most basic use is to map inputs to outputs. This can be as simple as Source fields to Target fields of the data integration job. It can also be from some other input components like aggregators, matching, or data quality components.
With the tMap you can also limit the fields mapped from left to right, basically filtering unneeded columns. You can create new columns coming out of the tMap - for example adding sequence keys, or concatenating multiple input columns into a new column (like address fields into one column to make a single mailing data field).
2. Expression Builder
Within the tMap on any column or variable you can open the expression builder wizard where you get access to hundreds of Talend functions, and if you can't find a Talend function that meets yours needs then you can fall back on a native Java function (don't fear, if you don't know Java just Google it).
If you happen to know some Java, you can easily build custom Java routines which will then be available within the expression builder. Also, expression builder allows you do complex math on multiple fields if needed. You can extract parts of dates, do data conversions, even case statements to build in “what if” logic. The same conditional statements can be used to determine if a row should pass through the tMap at all, acting as a filter. With tMap Expression Builder you get great transformation powers at your fingertips.
3. Lookups
The tMap is where you can do what many Data Integration specialists refer to as “lookups” on data. For those unfamiliar, this is basically joining data from one source to another source. The tMap has a lot of functionality on lookups, like inner join or outer join, reject if join is not found, cache the lookup data and much more. Lookups are critical to data transformation process as you often need to pull in reference data or get expanded views of records.
With Talend the lookup source can be anything that can be sourced into a Talend job. This has almost endless possibilities. To illustrate, let’s imagine a multi-cloud scenario quickly. Let’s say you have customer data in AWS on S3 and some other critical data on Azure Blob storage. In a single Talend job using tMap and lookups you can easily join the two sources together and write your data anywhere you need!
4. Route Multiple Outputs
The tMap can only have one input (not counting Lookups) but you can have multiple outputs with any number of columns as outputs on each stream.
This becomes a fast and powerful way to route errors down a different flow or to just duplicate the data flows to different streams. One flow can go to an aggregation component while another output could go direct to your target outputs and a third have a conditional statement looking for errors. All this becomes extremely useful as your data flows become complex with multiple outputs, error processing, and conditional outputs.
There you have it, a quick and (hopefully) helpful introduction to our most popular component in Talend. If you want some more tMap knowledge, let me know in the comments below and I’ll spin up my next article around some more advanced mapping functionalities. Happy connecting!