Purpose: Verifies data integrity, checks if incoming data meets the required business rules before saving, and ensures data quality standards are met before final processing. If validation fails, it stops processing with an error.

Key Characteristics:

• Executes last in the pipeline (Pass 3)
• Validates processed data after all transformations
• Can check against external data sources (databases, APIs)
• Implements business rule validation
• Generates validation errors when criteria are not met

Common scenarios include:

• Ensuring a field is not empty, not null, value must present on required fields during creation.
• Data existence validation (checking if records exist in database)
• Verifying number/date ranges
• Data completeness checks
• Duplicate checks
• Field length check
• and many more

Configuration Fields

• Name – Validator name.
• Type – Set to Validator (Pass 3).
• In Fields – Fields to validate.
• Order – Execution order.
• Comments – Optional notes.
• Script – Code defining validation logic.

Script Development

• Always handle null values appropriately
• Use the createError function for meaningful error messages
• Implement proper error handling
• Test with various data scenarios
• Document complex logic within the script

Example Implementation (The value(actualCost) must be > 0)

Actual cost must be > 0

Set up:

Script for it:

return async function verify(sourceData, columnNames, rowObject, connectionData, createError) {
/**
 * PositiveCostValidator
 *
 * WHAT
 * ----
 * Ensures a numeric column is present and positive.
 * Validators cannot *fix* data; they only raise a clear error so the
 * user can correct the source file.
 *
 * Example: Verify that a numeric field is greater than 0.
 *
 * Why? – Some fields (like actualCost, quantities, etc.) must not
 * be zero or negative. This validation blocks bad imports early.
 * In my below example I use ActualCost field of WO-OtherCost.
 */
	const rawValue = sourceData?.[0]; // safely get first item
        const inputData = Number(rawValue);
 
         const displayColumnName = columnNames[0].replace("Change.", "");
        if (isNaN(inputData) || inputData <= 0) {
          createError(
              `The '${displayColumnName}'must be greater than 0. You entered: "${sourceData[0] ?? ''}".`
          );
      }
      return;
 
}

Source data in the source file:

Output: