Use Transformations in Cloud, Device, and Hybrid Mode
Use transformations with destinations connected in cloud, device, and device mode.
10 minute read
This guide explains how to use transformations with destinations connected in different RudderStack connection modes.
Transform a single event
While using a transformation, RudderStack applies the transformEvent function on each event that takes the following two arguments:
Argument
Description
event
The input event.
metadata
The JavaScript function to access the event’s metadata. See the Access event metadata section below for more information.
After the transformation is complete, the transformEvent function returns the final event to be sent to the destination.
Access event metadata
RudderStack injects a function metadata(event) into your transformations as an argument. This lets you access the event metadata variables to customize your transformations.
metadata() takes the event as the input and returns the metadata of the event.
The following properties, if available, are present in the metadata response:
Property
Description
sourceId
The source ID in the Settings tab of your source configured in the RudderStack dashboard.
sourceName
The source name set in the RudderStack dashboard. For example, JavaScript-QA, TestJSSource, etc.
destinationId
The destination ID in the Settings tab of your destination configured in the RudderStack dashboard.
messageId
The unique ID for each event.
sourceType
The source type, for example, Android, iOS.
destinationType
The destination type where RudderStack sends the transformed event, for example, Snowflake.
Example transformations using the metadata data method are shown below:
You can make any number of external API requests in your transformation functions and use the fetched responses to enrich your events.
JavaScript
RudderStack injects an asynchronous fetch function in your transformations. It makes an API call to the given URL and returns the response in the JSON format.
You can use the fetch function in your transformations:
exportasyncfunctiontransformEvent(event,metadata){constres=awaitfetch("post_url",{method:"POST",// POST, PUT, DELETE, GET, etc.
headers:{"Content-Type":"application/json;charset=UTF-8",Authorization:"Bearer <authorization_token>"},body:JSON.stringify(event)});event.response=JSON.stringify(res);returnevent;}
To see the fetch function in action, refer to the Clearbit enrichment example.
For improved performance, it is highly recommended to use the batch API requests wherever possible instead of a separate API request for each event.
Fetch v2
fetchv2 is a wrapper for the fetch call. It enables you to fetch the response properties more efficiently while making the external API calls.
The fetchv2 response contains the following properties:
Property
Description
status
Status code of fetch response, for example, 200.
url
The URL of the Fetch API.
headers
The response headers
body
The response body in JSON or TEXT. By default, it is JSON.
The following example highlights the use of the fetchV2 function in a transformation to capture failure due to a timeout:
To ensure event ordering when using the transformBatch function, make sure you pass the messageId from the input event to the output event. Without the messageId, RudderStack does not guarantee event ordering.
It is highly recommended to use transformEvent as much as possible, as it ensures event ordering.
Cloud mode
When you add a transformation and connect it to a destination in cloud mode, RudderStack does the following:
Tracks and collects events at the source.
Applies the user transformation logic to your events.
Transforms the events in the destination-specific format. This is done internally and requires no user intervention.
Forwards the transformed events to your destination.
Connect cloud mode destination
You can connect only one transformation to a destination. However, one transformation can be used by multiple destinations.
There are two ways to connect a transformation to a destination:
From transformation
Click the Connections tab of your transformation and click Connect Destination. You will see a list of all the destinations and the transformations connected to them.
Scroll to the destination you want to connect to the transformation and click Connect.
Enable the Connect to transformation toggle and click Save.
From destination
Go to the destination in the dashboard. Click the Transformation tab and click Add a transformation:
Select the transformation to connect to the destination and click Choose.
Device mode
Device mode transformations is a beta feature and available for the Enterprise plan users only.
You can use transformations with destinations supporting device mode, like Firebase, Hotjar, and so on.
Device mode transformations leverage our Client Transformation service to transform events, then send the transformed events directly to the device mode destination through their native SDK.
Set up your device mode destination in RudderStack.
In the RudderStack dashboard, go to Collect > Transformations in the left sidebar. Then, select the transformation to connect to this destination.
Click the Connections tab . If you don’t have any destinations connected to it, click Connect Destination. If the transformation already has connected destinations, click Manage destinations.
Depending on whether the destination you set up (in Step 1) is already connected to some transformation, there are two ways to add a device mode transformation to it:
If your destination is not connected to any transformation click Connect. You will see a fly window with these options:
If your destination is already connected to some other transformation, click the Edit connections button to switch your transformation. See Switch transformation for more information. Once you switch the transformation, you will see the above options to enable the device mode transformation.
Configure the following settings under Device Mode:
Enable the Connect to transformation toggle.
Enable the Propagate errors toggle depending on your requirement.
Note that:
Under Cloud Mode, the Connect to transformation toggle is enabled by default. Do not disable this toggle as it will not allow you to configure any device mode transformation settings.
If Propagate errors is enabled, RudderStack sends the events to the destination without transforming the events, for any transformation errors. This is helpful in preventing data loss in case of transformation code or runtime errors.
If your transformation involves hashing PII, enabling this setting will send the untransformed event data in case of any transformation errors.
Click Save to confirm the settings and for the changes to take effect.
To confirm that the device mode transformation is connected, go to the Connections tab of your transformation and check the Connection Mode column:
Follow the below steps for setting up the required SDK:
The detailed steps to set up the JavaScript SDK are mentioned in the Quickstart guide.
Install the Android SDK following the detailed steps mentioned in Installing the Android SDK section. Additionally, use the following dependency in the Android app level build.gradle:
The iOS SDK v2 (beta) does not support device mode transformations. Use the iOS SDK v1 (stable version) instead.
Install the iOS SDK following the detailed steps mentioned in Installing the iOS SDK section. Additionally, use the following dependency in Podfile:
pod'Rudder','1.17.0'
Tokenization
RudderStack provides the tokenization feature in device mode transformations to add an extra layer of security. The token is available as a metadata in the transformation and you can validate it before attaching with the RudderStack SDK.
A sample code to access the token in the transformation:
exportfunctiontransformEvent(event,metadata){constdmtToken=metadata(event)["Custom-Authorization"];//verify the token
...returnevent;}
You can use the following methods to attach the token (in the string format) to the SDK:
rudderanalytics.setAuthToken("my-token");
RudderClient.putAuthToken("my-token")
[RSClientputAuthToken:@"my-token"];
You can call these methods multiple times.
RudderStack clears the token when the user logs out, that is, when the reset() API is called.
Client Transformation service architecture
RudderStack provides the Client Transformation service to facilitate transformations for device mode destinations. It ensures that the event ingestion is unaffected, ensuring minimum response time from the RudderStack backend.
Workflow
When you add a transformation and connect it to a device mode destination supporting the device mode:
RudderStack SDK sends the event to the Client Transformation service. It multiplexes the event for different destinations and connects with the transformations attached to these device mode destinations.
Client Transformation service sends the event to the RudderStack Transformation service to transform the event according to the functions defined in the attached transformations.
After the transformation is applied, the transformed event is returned to the Client Transformation service.
Client Transformation service responds to the SDK with the transformed events for all the destinations.
Finally, the SDK parses the transformed events and forwards them to the specified device-mode destinations.
If the request to the Client Transformation service fails, the RudderStack SDK makes three retry attempts. If it fails, the SDK forwards the untransformed events to the destination or drops them - based on the Propagate errors toggle while configuring the device mode transformation (see Step 6).
Debugging
Check the Network tab in the browser’s developer tools for any transform network requests to debug errors in your device mode transformations. These requests can help you determine if:
The transformation has run as expected.
The responses are as expected, based on the sent batch or otherwise.
Performance considerations
RudderStack makes a network call for each event (in case of JavaScript SDK) or a batch of events (in case of mobile SDKs) when you use transformations with device mode destinations. Hence, there can be an added performance cost.
Events sent in both cloud and device mode: By default, the transformation is applied to both the cloud and device mode events.
Only device mode events: Access the mode field in your transformation’s event metadata and check if its value is deviceMode. Then, specify the transformation logic to be applied to the device mode events:
Only cloud mode events: Return the device mode events as it is using the mode field and specify the transformation logic for the rest of the events (cloud mode events), as shown:
A transformation connected in either cloud mode or device mode must follow the below memory and time limits. It fails if these limits are exceeded:
Parameter
Limit
Memory limit
128 MB
Execution time limit
4 seconds
The following limitations are applicable when invoking a transformation in device mode:
The native destination SDKs might persist some captured data - mainly identifiers and traits. So, in device/hybrid mode, you cannot transform these fields using device mode transformations.
Network unavailability can lead to higher latency than expected while sending the events to the destinations. RudderStack doesn’t lose data due to the network loss and stores the events in the client device until they are successfully delivered to the destination.
The iOS SDK does not support the background processing of an event when the app is closed. However, it sends the pending events the next time the app is opened.
This site uses cookies to improve your experience while you navigate through the website. Out of
these
cookies, the cookies that are categorized as necessary are stored on your browser as they are as
essential
for the working of basic functionalities of the website. We also use third-party cookies that
help
us
analyze and understand how you use this website. These cookies will be stored in your browser
only
with
your
consent. You also have the option to opt-out of these cookies. But opting out of some of these
cookies
may
have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This
category only includes cookies that ensures basic functionalities and security
features of the website. These cookies do not store any personal information.
This site uses cookies to improve your experience. If you want to
learn more about cookies and why we use them, visit our cookie
policy. We'll assume you're ok with this, but you can opt-out if you wish Cookie Settings.