# Workflow data handling Serverless Workflow data is represented in [JSON](https://www.json.org/json-en.html) format. Data flow and execution logic go hand in hand, meaning as workflow execution follows the workflow definition logic, so does the workflow data:

The initial Workflow data input is passed to the workflow starting state as its data input. When a state finishes its execution, its data output is passed as data input to the next state that should be executed. When workflow execution ends, the last executed workflow state's data output becomes the final Workflow data output. States can filter their data inputs and outputs using State Data filters. States can also consume events as well as invoke services. These event payloads and service invocation results can be filtered using Event data filters and Action data filters. Data filters use workflow expressions for selecting and manipulating state data input and output, action inputs and results, and event payloads. Multiple filters can be combined to gain high level of control of your workflow state data. You can find an example of that in this section. Data from consumed events,and action execution results are added/merged to state data. Reference the data merging section to learn about the merging rules that should be applied. #### **Workflow Data Input** The initial data input into a workflow instance. Must be a valid [JSON object](https://tools.ietf.org/html/rfc7159#section-4). If no input is provided, the default data input should be an empty JSON object: ```json { } ``` Workflow data input is passed to the workflow starting state as its data input.

#### **Information Passing Between States** States in a workflow can receive data (data input) and produce a data result (data output). The state's data input is typically the previous state's data output. When a state completes its execution, its data output is passed to the state's data input it transitions to. There are two rules to consider here: * If the state is the workflow starting state, its data input is the workflow data input. * When workflow execution ends, the data output of the last executed state becomes the workflow data output.

#### **Workflow data output** Each workflow execution should produce a data output. The workflow data output is the data output of the last executed workflow state. #### **State data filters** | Parameter | Description | Type | Required | | --------- | ------------------------------------------------------- | ------ | -------- | | input | Workflow expression to filter the states data input | string | no | | output | Workflow expression that filters the states data output | string | no | Example:

JSON	YAML
`{ "stateDataFilter": { "input": "${ .orders }", "output": "${ .provisionedOrders }" } }`	`stateDataFilter: input: "${ .orders }" output: "${ .provisionedOrders }"`

State data filters can be used to filter the state's data input and output. The state data filters `input` property expression is applied when the workflow transitions to the current state and receives its data input. It can be used to select only data that is needed and disregard what is not needed. If `input` is not defined or does not select any parts of the state's data input, its data input is not filtered. The state data filter `output` property expression is applied right before the state transitions to the next state defined. It filters the state's data output to be passed as data input to the transitioning state. If the current state is the workflow end state, the filtered state's data output becomes the workflow data output. If `output` is not defined or does not select any parts of the state's data output, its data output is not filtered. Results of the `input` expression should become the state data input. Results of the `output` expression should become the state data output. For more information on this you can reference the data merging section. Let's take a look at some examples of state filters. For our examples let's say the data input to our state is as follows: {% code lineNumbers="true" %} ```json { "fruits": [ "apple", "orange", "pear" ], "vegetables": [ { "veggieName": "potato", "veggieLike": true }, { "veggieName": "broccoli", "veggieLike": false } ] } ``` {% endcode %} For the first example, our state only cares about fruits data, and we want to disregard the vegetables. To do this we can define a state filter: {% code lineNumbers="true" %} ```json { "stateDataFilter": { "input": "${ {fruits: .fruits} }" } } ``` {% endcode %} The state data output then would include only the fruits data: {% code lineNumbers="true" %} ```json { "fruits": [ "apple", "orange", "pear"] } ``` {% endcode %}

For our second example, let's say that we are interested in the only vegetable "veggie-like". Here we have two ways of filtering our data, depending on if actions within our state need access to all vegetables, or only the ones that are "veggie-like". The first way would be to use both "input", and "output": {% code lineNumbers="true" %} ```json { "stateDataFilter": { "input": "${ {vegetables: .vegetables} }", "output": "${ {vegetables: [.vegetables[] | select(.veggieLike == true)]} }" } } ``` {% endcode %} The states data input filter selects all the vegetables from the main data input. Once all actions have performed, before the state transition or workflow execution completion (if this is an end state), the "output" of the state filter selects only the vegetables which are "veggie like".

The second way would be to directly filter only the "veggie like" vegetables with just the data input path: {% code lineNumbers="true" %} ```json { "stateDataFilter": { "input": "${ {vegetables: [.vegetables[] | select(.veggieLike == true)]} }" } } ``` {% endcode %} #### **Action data filters** | Parameter | Description | Type | Required | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | -------- | | fromStateData | Workflow expression that filters state data that can be used by the action | string | no | | useResults | If set to `false`, action data results are not added/merged to state data. In this case 'results' and 'toStateData' should be ignored. Default is `true`. | boolean | no | | results | Workflow expression that filters the actions data results | string | no | | toStateData | Workflow expression that selects a state data element to which the action results should be added/merged into. If not specified denotes the top-level state data element | string | no | **Example:**

JSON YAML

JSON	YAML
`{ "actionDataFilter": { "fromStateData": "${ .language }", "results": "${ .results.greeting }", "toStateData": "${ .finalgreeting }" } }`	`actionDataFilter: fromStateData: "${ .language }" results: "${ .results.greeting }" toStateData: "${ .finalgreeting }"`

{
  "actionDataFilter": {
    "fromStateData": "${ .language }",
    "results": "${ .results.greeting }",
    "toStateData": "${ .finalgreeting }"
  }
}

actionDataFilter:
  fromStateData: "${ .language }"
  results: "${ .results.greeting }"
  toStateData: "${ .finalgreeting }"

Action data filters can be used inside Action definitions. Each action can define this filter which can: * Filter the state data to select only the data that can be used within function definition arguments using its `fromStateData` property. * Filter the action results to select only the result data that should be added/merged back into the state data using its `results` property. * Select the part of state data which the action data results should be added/merged to using the `toStateData` property. To give an example, let's say we have an action which returns a list of breads and pasta types. For our workflow, we are only interested into breads and not the pasta. Action results: {% code lineNumbers="true" %} ```json { "breads": ["baguette", "brioche", "rye"], "pasta": [ "penne", "spaghetti", "ravioli"] } ``` {% endcode %} We can use an action data filter to filter only the breads data: {% code lineNumbers="true" %} ```json { "actions":[ { "functionRef": "breadAndPastaTypesFunction", "actionDataFilter": { "results": "${ {breads: .breads} }" } } ] } ``` {% endcode %} The `results` will filter the action results, which would then be: {% code lineNumbers="true" %} ```json { "breads": [ "baguette", "brioche", "rye" ] } ``` {% endcode %} Now let's take a look at a similar example (same expected action results) and assume our current state data is: {% code lineNumbers="true" %} ```json { "itemsToBuyAtStore": [ ] } ``` {% endcode %} and have the following action definition: {% code lineNumbers="true" %} ```json { "actions":[ { "name": "fetch-items-to-buy", "functionRef": "breadAndPastaTypesFunction", "actionDataFilter": { "results": "${ [ .breads[0], .pasta[1] ] }", "toStateData": "${ .itemsToBuyAtStore }" } } ] } ``` {% endcode %} In this case, our `results` select the first bread and the second element of the pasta array. The `toStateData` expression then selects the `itemsToBuyAtStore` array of the state data to add/merge these results into. With this, after our action executes the state data would be: {% code lineNumbers="true" %} ```json { "itemsToBuyAtStore": [ "baguette", "spaghetti" ] } ``` {% endcode %} In the case action results should not be added/merged to state data, we can set the `useResults` property to `false`. In this case, the `results` and `toStateData` properties should be ignored, and nothing is added/merged to state data. If `useResults` is not specified (or it's value set to `true`), action results, if available, should be added/merged to state data. #### **Event data filters** | Parameter | Description | Type | Required | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | -------- | | useData | If set to `false`, event payload is not added/merged to state data. In this case 'data' and 'toStateData' should be ignored. Default is `true`. | boolean | no | | data | Workflow expression that filters the event data (payload) | string | no | | toStateData | Workflow expression that selects a state data element to which the action results should be added/merged into. If not specified denotes the top-level state data element | string | no | Example:

JSON	YAML
`{ "eventDataFilter": { "data": "${ .data.results }" } }`	`eventDataFilter: data: "${ .data.results }"`

Event data filters can be used to filter consumed event payloads. They can be used to: * Filter the event payload to select only the data that should be added/merged into the state data using its `data` property. * Select the part of state data into which the event payload should be added/merged into using the `toStateData` property. Allows event data to be filtered and added to or merged with the state data. All events have to be in the CloudEvents format and event data filters can filter both context attributes and the event payload (data) using the `data` property. Here is an example using an event filter:

Note that the data input to the Event data filters depends on the `dataOnly` property of the associated Event definition. If this property is not defined (has default value of `true`), Event data filter expressions are evaluated against the event payload (the CloudEvents `data` attribute only). If it is set to `false`, the expressions should be evaluated against the entire CloudEvent (including its context attributes). In the case event data/payload should not be added/merged to state data, we can set the `useData` property to `false`. In this case, the `data` and `toStateData` properties should be ignored, and nothing is added/merged to state data. If `useData` is not specified (or it's value set to `true`), event payload, if available, should be added/merged to state data. #### **Using multiple data filters** As Event states can take advantage of all defined data filters. In the example below, we define a workflow with a single event state and show how data filters can be combined. {% code lineNumbers="true" %} ```json { "name": "greet-customers-workflow", "description": "Greet Customers when they arrive", "version": "1.0.0", "specVersion": "0.8", "start": "WaitForCustomerToArrive", "states":[ { "name": "wait-for-customer-to-arrive", "type": "event", "onEvents": [{ "eventRefs": ["customer-arrives-event"], "eventDataFilter": { "data": "${ .customer }", "toStateData": "${ .customerInfo }" }, "actions":[ { "name": "greet-customer", "functionRef": { "refName": "greeting-function", "arguments": { "greeting": "${ .hello.spanish } ", "customerName": "${ .customerInfo.name } " } }, "actionDataFilter": { "fromStateData": "${ { hello, customerInfo } }", "results": "${ .greetingMessageResult }", "toStateData": "${ .finalCustomerGreeting }" } } ] }], "stateDataFilter": { "input": "${ .greetings } ", "output": "${ { finalCustomerGreeting } }" }, "end": true } ], "events": [{ "name": "customer-arrives-event", "type": "customer-arrival-type", "source": "customer-arrival-event-source" }], "functions": [{ "name": "greeting-function", "operation": "http://my.api.org/myapi.json#greeting" }] } ``` {% endcode %} The workflow data input when starting workflow execution is assumed to include greetings in different languages: {% code lineNumbers="true" %} ```json { "greetings": { "hello": { "english": "Hello", "spanish": "Hola", "german": "Hallo", "russian": "Здравствуйте" }, "goodbye": { "english": "Goodbye", "spanish": "Adiós", "german": "Auf Wiedersehen", "russian": "Прощай" } } } ``` {% endcode %} The workflow data input then becomes the data input of the starting workflow state. We also assume for this example that the CloudEvent that our event state consumes include the data (payload): {% code lineNumbers="true" %} ```json { "customer": { "name": "John Michaels", "address": "111 Some Street, SomeCity, SomeCountry", "age": 40 } } ``` {% endcode %} Here is a sample diagram showing our workflow, each numbered step on this diagram shows a certain defined point during workflow execution at which data filters are invoked and correspond to the numbered items below.

**(1) Workflow execution starts**: Workflow data is passed to our "WaitForCustomerToArrive" event state as data input. Workflow executes its starting state, namely the "WaitForCustomerToArrive" event state. The event state **stateDataFilter** is invoked to filter its data input. The filters "input" expression is evaluated and selects only the "greetings" data. The rest of the state data input should be disregarded. At this point our state data should be: {% code lineNumbers="true" %} ```json { "hello": { "english": "Hello", "spanish": "Hola", "german": "Hallo", "russian": "Здравствуйте" }, "goodbye": { "english": "Goodbye", "spanish": "Adiós", "german": "Auf Wiedersehen", "russian": "Прощай" } } ``` {% endcode %} **(2) CloudEvent of type "customer-arrival-type" is consumed**: Once the event is consumed, the "eventDataFilter" is triggered. Its "data" expression selects the "customer" object from the events data. The "toStateData" expression says that we should add/merge this selected event data to the state data in its "customerInfo" property. If this property exists it should be merged, if it does not exist, one should be created. At this point our state data contains: {% code lineNumbers="true" %} ```json { "hello": { "english": "Hello", "spanish": "Hola", "german": "Hallo", "russian": "Здравствуйте" }, "goodbye": { "english": "Goodbye", "spanish": "Adiós", "german": "Auf Wiedersehen", "russian": "Прощай" }, "customerInfo": { "name": "John Michaels", "address": "111 Some Street, SomeCity, SomeCountry", "age": 40 } } ``` {% endcode %} **(3) Event state performs its actions**: Before the first action is executed, its actionDataFilter is invoked. Its "fromStateData" expression filters the current state data to select from its data that should be available to action arguments. In this example it selects the "hello" and "customerInfo" properties from the current state data. At this point the action is executed. We assume that for this example "greetingFunction" returns: {% code lineNumbers="true" %} ```json { "execInfo": { "execTime": "10ms", "failures": false }, "greetingMessageResult": "Hola John Michaels!" } ``` {% endcode %} After the action is executed, the actionDataFilter "results" expression is evaluated to filter the results returned from the action execution. In this case, we select only the "greetingMessageResult" element from the results. The action filters "toStateData" expression then defines that we want to add/merge this action result to state data under the "finalCustomerGreeting" element. At this point, our state data contains: {% code lineNumbers="true" %} ```json { "hello": { "english": "Hello", "spanish": "Hola", "german": "Hallo", "russian": "Здравствуйте" }, "goodbye": { "english": "Goodbye", "spanish": "Adiós", "german": "Auf Wiedersehen", "russian": "Прощай" }, "customerInfo": { "name": "John Michaels", "address": "111 Some Street, SomeCity, SomeCountry", "age": 40 }, "finalCustomerGreeting": "Hola John Michaels!" } ``` {% endcode %} **(4) Event State Completes Execution**: When our event state finishes its execution, the states "stateDataFilter" "output" filter expression is executed to filter the state data to create the final state data output. Because our event state is also an end state, its data output becomes the final workflow data output. Namely: {% code lineNumbers="true" %} ```json { "finalCustomerGreeting": "Hola John Michaels!" } ``` {% endcode %} #### **Data Merging** Consumed event data (payload) and action execution results should be merged into the state data. Event and action data filters can be used to give more details about this operation. By default, with no data filters specified, when an event is consumed, its entire data section (payload) should be merged to the state data. Merging should be applied to the entire state data JSON element. In case of event and action filters, their "toStateData" property can be defined to select a specific element of the state data with which merging should be done against. If this element does not exist, a new one should be created first. When merging, the state data element and the data (payload)/action result should have the same type, meaning that you should not merge arrays with objects or objects with arrays etc. When merging elements of type object should be done by inserting all the key-value pairs from both objects into a single combined object. If both objects contain a value for the same key, the object of the event data/action results should "win". To give an example, let's say we have the following state data: {% code lineNumbers="true" %} ```json { "customer": { "name": "John", "address": "1234 street", "zip": "12345" } } ``` {% endcode %} and we have the following event payload that needs to be merged into the state data: {% code lineNumbers="true" %} ```json { "customer": { "name": "John", "zip": "54321" } } ``` {% endcode %} After merging the state data should be: {% code lineNumbers="true" %} ```json { "customer": { "name": "John", "address": "1234 street", "zip": "54321" } } ``` {% endcode %} Merging array types should be done by concatenating them into a larger array including unique elements of both arrays.\ To give an example, merging: {% code lineNumbers="true" %} ```json { "customers": [ { "name": "John", "address": "1234 street", "zip": "12345" }, { "name": "Jane", "address": "4321 street", "zip": "54321" } ] } ``` {% endcode %} into state data: {% code lineNumbers="true" %} ```json { "customers": [ { "name": "Michael", "address": "6789 street", "zip": "6789" } ] } ``` {% endcode %} should produce state data: {% code lineNumbers="true" %} ```json { "customers": [ { "name": "Michael", "address": "6789 street", "zip": "6789" }, { "name": "John", "address": "1234 street", "zip": "12345" }, { "name": "Jane", "address": "4321 street", "zip": "54321" } ] } ``` {% endcode %} Merging number types should be done by overwriting the data from events data/action results into the merging element of the state data. For example merging action results: {% code lineNumbers="true" %} ```json { "age": 30 } ``` {% endcode %} into state data: {% code lineNumbers="true" %} ```json { "age": 20 } ``` {% endcode %} would produce state data: {% code lineNumbers="true" %} ```json { "age": 30 } ``` {% endcode %} Merging string types should be done by overwriting the data from events data/action results into the merging element of the state data.