# Background
At work we are trying using [Amazon AppFlow](https://aws.amazon.com/appflow/) to pull data from Salesforce, update our internal systems, and update back to Salesforce. We are looking to replace some scenarios that we have running in Integromat. The Integromat scenarios have some issues because the Salesforce posts kind of large files to Integromat which Integromat processes synchronously. When Integromat takes too long then Salesforce times out and tries again...even though Integromat is still running.
The Integromat scenario is designed with a series of steps. Seeing the steps made it seem like an obvious AWS implementation would be using [AWS Step Functions](https://aws.amazon.com/step-functions/) state machines. Initially I implemented each step in the state machine as an individual Lambda to do the work.
While working on another portion of the project, I started making use of more of the native Step Functions steps. It occurred to me that I might be able to eliminate all of the Lambdas used by the state machine and replace them with native steps.
# Thoughts
I ran into a bunch of challenges when passing data from step to step. Those challenges were a result of my ignorance about [Amazon States Language (ASL)](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html). I worked through those issues. The process felt more like trial-and-error because there isn't step-through debugging. Step Functions has thorough logging showing input to, and output from, each step. I ended up swapping between two tabs, one showing the state machine and the other showing the output of the latest execution. With step-through, It's nice to be able to update code while you are looking at live data. Maybe I could have been updating code while looking at data (from previous execution), but I was working on an iPad. By the way, you can work on Step Functions in a browser on an iPad!
Once the state machine was configured it just felt kind of weird. I've spent years writing software and deploying a full application somewhere. A running server, even if just spun up temporarily, seems natural. A full application would, of course, have its own functions inside of the app. There would be some orchestration logic in the app. The orchestration would be like:
```python
list_of_files = get_list_of_files()
for file in list_of_files:
file_lines = file.read()
for line in file_lines:
account = line["account"]
account_config = get_config(account)
for provider in account_config["providers"]:
pause_settings = configure_pause_request(provider)
update_config(account, pause_settings)
```
An initial Step Functions implementation of the same orchestration is to make a Lambda for each function (e.g., `get_list_of_files()`, `get_config()`, `configure_pause_request()`, and `update_config()` ) and configure a state machine to do the looping. That feels like a kind of hybrid approach though. Some of the logic is in the state machine and some of the logic is scatters in four different Lambda functions. Each Lambda, though, is very tight and focused on doing its one task. That makes each Lambda simple to reason about. However, I found I was bouncing around a lot in the web browser so look at the various resources.
A revised Step Functions implementation is to include more of the processing in the state machine itself by using native Step Functions steps. The revision eliminates the Lambdas entirely. There are calls using steps. Within the step configurations you can transform incoming and outgoing data by providing some mappings like:
```json
{
"app_id.
quot;: "$.app_id",
"payload": {
"name": "Salesforce Integration",
"service_paused": true,
"pause_until.quot;: "$.Pause_Until_Date",
"providers.quot;: "States.Array($.provider_config.provider)",
"reason": {
"property1.quot;: "$.Primary_Reason_Lost__c",
}
}
}
```
_You just setup the rules._ Step Functions does all of the execution. All of the looping. All of the function calling. You just setup the rules to transform data between the function calls.
It feels sort of gratifying and safe. I feel like I setup a rules-based system and it feels like it's going to work nicely. I feel like there is much less for me to think about. I'm not worried that the system will crash. I'm not worried that it will be unavailable. I don't worry that I'll screw up the looping.
```ad-quote
title: Tom
The code _is_ the documentation!
```
# Initial Implementation with Lambda Steps
![[Pasted image 20221119104221.png]]
# Revised Implementation with Native Steps
![[Pasted image 20221119104416.png]]