Set Transformation

---

🎯 Purpose

The Set Transformation node reshapes data flowing through your workflow—cleaning, filtering, joining, grouping, flattening nested objects, and running custom logic—so downstream nodes receive exactly the structure they expect.

📥 Inputs

• Items (Array<Object>, Required): Records from previous nodes to transform.
• (Optional) Secondary Items (Array<Object>): Used by join/stack operations.

📤 Outputs

• Transformed Items (Array<Object>): Records after the selected transformation (structure depends on operation and settings).

⚙️ Parameters

Name	Type	Required	Default	Description
Transformation	Dropdown	No	none	Operation to apply. Options: none, replace-in-elements-a-by-b, inner-join-a-on-b, group-by, filter-by, join-array-to-string, stack-a-b-props, redirect-to, foreach-js, filter-by-js.
Specific Properties	String (CSV)	No	(empty)	For replace-in-elements-a-by-b: limit replacements to these properties.
Wrapping Property	String	No	(empty)	For replace-in-elements-a-by-b: property containing replacement map/object.
Join A	String	No	(empty)	For inner-join-a-on-b: key in dataset A.
On B (Join)	String	No	(empty)	For inner-join-a-on-b: key in dataset B.
On property names (comma separated)	String (CSV)	No	(empty)	For group-by: property names to group by.
Column name for ‘count’	String	No	(empty)	For group-by: name of the count column to add.
Group by method	Dropdown	No	List of groups	For group-by: List of groups or Keep top one item (cut the rest).
Filter Expression	String	No	(empty)	For filter-by: condition (e.g., status == "active" && total > 100).
Value Getter Expression (Array-to-string)	String	No	(empty)	For join-array-to-string: expression to pick value from each element.
Separator	String	No	(empty)	For join-array-to-string: separator between values (e.g., ", " or " | ").
Output property name	String	No	joined_string	For join-array-to-string: output property name.
Value Getter Expression (Redirect-to)	String	No	(empty)	For redirect-to: path/expression to nested data to lift/flatten.
jsCode	Code (JS)	No	(empty)	For foreach-js: function (item, idx) => item returning a modified item.
jsFilter	Code (JS)	No	(empty)	For filter-by-js: predicate (item, idx) => boolean.

ℹ️ Parameters appear contextually based on the selected transformation.

🧭 Operation Details

replace-in-elements-a-by-b

Replace values in specified properties using a map from Wrapping Property.

• Example: Normalize status using { active: "Active", ACTIVE: "Active" }.

inner-join-a-on-b

Join dataset A and B on matching keys (Join A vs On B (Join)). Produces merged records.

group-by

Group items by On property names (comma separated). Adds Column name for 'count'.

• Method:
  • List of groups: returns groups with items and counts.
  • Keep top one: returns the first item per group with count.

filter-by

Keep items where Filter Expression evaluates truthy. Supports boolean logic and comparisons.

join-array-to-string

From an array property, pick a value via Value Getter Expression (Array-to-string), join with Separator, store in Output property name.

stack-a-b-props

Zip/stack properties from two sources into paired records (useful for aligning arrays or parallel datasets).

redirect-to

Lift nested data (e.g., order.customer_details) to the top-level stream (flattening).

foreach-js

Run custom JS per item: (item, idx) => ({ ...item, ...changes }).

filter-by-js

Custom JS predicate to include/exclude items: (item, idx) => boolean.

💡 Example Usage

Customer Segmentation

• Transformation: group-by
• On property names (comma separated): purchase_frequency,total_spent
• Column name for ‘count’: customer_count
• Method: List of groups
• Result: Segmented cohorts with counts.

Status Standardization

• Transformation: replace-in-elements-a-by-b
• Specific Properties: status,priority
• Wrapping Property: standardized_values
• Result: Unified values across sources.

Product Names Summary

• Transformation: join-array-to-string
• Value Getter Expression (Array-to-string) : product_name
• Separator: ", "
• Output property name: ordered_products
• Result: "Laptop, Mouse, Keyboard".

📘 Best Practices

• Validate Keys: For joins, confirm both sides use the same type and normalization (e.g., trim/case).
• Filter Early: Reduce dataset size before heavy operations (join/group).
• Name Clearly: Use descriptive output names like ordered_products, customer_count.
• Test Small: Dry-run with a small sample before scaling.

🧪 Test Cases

• Join Happy Path
  Given: A: [{id:1,name:"A"}], B: [{user_id:1, tier:"pro"}], Join A=id, On B (Join)=user_id →
  Expected: [{id:1,name:"A",tier:"pro"}]
• Group With Counts
  Given: [{cat:"A"},{cat:"A"},{cat:"B"}], On property names (comma separated)=cat, Column name for 'count'=count →
  Expected: Groups A(count=2), B(count=1)
• Filter Expression
  Given: [{status:"active"},{status:"inactive"}], Filter Expression=status=="active" →
  Expected: Only [{status:"active"}]
• Join Array To String
  Given: items:[{name:"A"},{name:"B"}], Value Getter Expression (Array-to-string) =name, ", " →
  Expected: ordered_products="A, B"
• Redirect-To
  Given: [{order:{customer:{id:1}}}], Value Getter Expression (Redirect-to)=order.customer →
  Expected: [{id:1}]

---