Wim Arys on Unsplash

How to run serverless step functions locally

In the following article we will be looking at how to set up serverless step functions locally. As well as look at some state machine flow examples.

TLDR; code examples can be found here— https://github.com/richlloydmiles/serverless-step-functions-local-example

I’ll assume that you went ahead and clicked on the TLDR link — however, if this still doesn’t help give you all of the information about how step functions work locally, continue reading.

Getting Setup

I’m going to assume that you have cloned the repo above and run the commands to get yourself up and running, namely: npm install and npm run start

You should see something like this in your terminal:

By default the serverless framework will look at our serverless.yml file to give us some instructions about how the serverless app should run.

The default serverless.yml file has some basic serverless configuration entries including plugin and function definitions, but where it really gets interesting is the stepFunctions: section.

Here you will see a couple of things:

• stateMachines: — this is the key where we define a list of state machines (step functions are run in the context of a state machine). In our example we have defined a single stateMachine called WaitMachine . Which in itself will have its own set of definitions and will specify step functions to run.
• The Comment entry is pretty self explanatory.
• The StartAt is similarly self-explanatory with the exception that the value of this needs to reference a state found in the States: definition, (if not blindingly obvious, this is the first state (or step function) that will be executed in our state machine by default).
• States: Are a list of states that have key / value pairs based on their functions (these are what could be considered our “step functions”).

There are various types of states.

Task State

• When referencing a Task type, the state needs to contain an ARN for that resource in the Resource definition (usually a lambda function task). By default, serverless version 2.5 and up supports the Fn::GetAtt shorthand references for resources within your serverless configuration.

Side note: when running your state machine in the real world (e.g. not locally), this is all you should need, however, the severless-step-functions-local plugin requires that your specify these Task resource arn’s in the custom configuration, so that it knows what to reference when executing your state machine locally.

You will see this in the TaskResourceMapping definition in the stepFunctionsLocal custom configuration section. e.g.

TaskResourceMapping:
  FirstState: arn:aws:lambda:us-east-1:101010101010:function:hello
  FinalState: arn:aws:lambda:us-east-1:101010101010:function:world

The region (us-east-1) and AWS account ID (101010101010) are local values that are irrelevant when using this in the real world.

As we continue to go through our States: definition in the stepFunctions: section of the serverless.yml you will notice a Next: wait_using_secondor End: true. This tells our state machine whether or not to continue to the next step (usually based on some condition), or to end the execution of the state machine.

Wait State

The next thing you’ll notice is that the wait_using_seconds state does not specify a resource arn, this is because it is a baked in state whose purpose is to wait for a specified amount of time (in this case 10 seconds), before moving onto the next state (FinalState) or ending. When you execute this particular example locally you should see ‘hello’ being logged to the console, followed by ‘world’ logged 10 seconds later.

Parallel State

If you load up (or replace the serverless.yml file with) the serverless-parallel.yml file you will notice the addition of the Type: Parallel state (FirstState). This state also does not specify a resource arn(much like the wait state), but does have a definition for Branches: . This allows us to run multiple independant states (or step functions) at the same time, that can have completely independent flows that branch off at this point. In our case they are just running two parallel task states second_task and FinalState.

Choice State

Finally, if you look at the serverless-choice.yml file you will see a new type of state called Choice , this allows us to conditionally run step functions based on a set of conditions, which in our case is an object key returned by the response in the FirstState lambda function calledfoo,which conditionally runs the next step based off its value (whether it is equal to 1 or 2 ) This can be found in the hello function contained in handler.js .

module.exports.hello = async event => {
  console.log('hello')
  return { foo: 1 }
}

There are a few more Types of state functions that can be used found in the offical aws documentation— https://docs.aws.amazon.com/step-functions/latest/dg/concepts-states.html

with an infite amount of combinitions in which they can be used.

Executing our state machine

The final piece in the puzzle would be how we go about actually triggering the execution of our state machine.

In our case the startSF function in handler.js is where this happens:

const AWS = require('aws-sdk')
module.exports.startSF = (event, context, callback) => {
  const stepFunctions = new AWS.StepFunctions({ endpoint: 'http://localhost:8083' })
  const stateMachineArn =     process.env.OFFLINE_STEP_FUNCTIONS_ARN_WaitMachine
  const params = {
    stateMachineArn
  }
  return stepFunctions.startExecution(params).promise().then(() => {
    callback(null, `Your state machine ${stateMachineArn} executed   successfully`)
  }).catch(error => {
    callback(error.message);
  })
}

Let’s go through the main parts of this.

const stepFunctions = new AWS.StepFunctions({ endpoint: 'http://localhost:8083' })

http://localhost:8083 is where the underlying state machine will be run locally. If you were running this in the real world you would obviously run this in a different way (no using your local port)

Next, the environment variable

OFFLINE_STEP_FUNCTIONS_ARN_WaitMachine

is automatically added to the context of our local serverless execution. With a base value of:

OFFLINE_STEP_FUNCTIONS_ARN_

and the suffix specifying the name of the state machine that we want to start executing:

WaitMachine

we then execute our state machine, calling it with:

stepFunctions.startExecution

In order to actually call this lambda function

startSF()

we need to invoke it in some way, in our case we have chosen to invoke it via an http endpoint within our existing serverless context:

startSF:
  handler: handler.startSF
  events:
  - http:
    path: hello
    method: GET

You should be able to fire this by hitting the following in your web browser or postman:

http://localhost:3000/local-dev/hello

Error handling

If you get the following error when trying to start your offline server (which I suspect you will be after making any changes to the serverless state machine definitions):

StateMachineAlreadyExists: State Machine Already Exists: 'arn:aws:states:us-east-1:101010101010:stateMachine:WaitMachine'

You can delete your state machine manually by running the following after stopping the serverless offline server.:

npm run delete

You should be able to run npm run start successfully afterwards.

Conclusion

In my opinion step functions are the future of AWS serverless infrustucture, especially with the rising popularity of the low-code no code approaches - (https://aws.amazon.com/blogs/aws/new-aws-step-functions-workflow-studio-a-low-code-visual-tool-for-building-state-machines/. Having a low overhead way to test this is almost a nessesity, making the use of local step functions an integral part of the serverless journey.

That’s all folks!