I’m back to life and here’s some tasty article on something I’ve recently learnt. Unfortunately, this is again on 2 very specific commercial solutions instead of general purpose programming. And another one on AWS. Maybe I’ll come up with an article on signal handling in operating systems. Anyway, here are some tips and trick on how to set up PyCharm to work with AWS Lambda functions.

What are AWS Lambdas?

First, we need a brief introduction to the lambda functions. A lambda is a piece of code that can be called from within the AWS infrastructure. It can be invoked on certain occasions, like file upload events, pushing messages to queues, HTTP requests and so on. While a lot can be achieved with infrastructure-agnostic tools like Celery for Python (which I utilized in my side project), Lambdas have certain advantages, like:

• cost – you only pay for what you use. Beware that even this cheap service may get out of control and cost you a fortune if you don’t set up a budget monitoring;
• Scaling – AWS takes care of providing resources for you to run the code;
• The knowledge of what is called, when it starts and end – and believe me or not this turned out to be quite critical for me recently. The fact that the runner knows whether a code has been triggered so that a deploy does not interrupt running process can be surprisingly important, especially for long-running tasks that are not atomic.

Of course all this comes with a cost, like binding yourself with the infrastructure provider. An attempt to make a change can be painful.

Getting started with a Lambda

Let’s imagine that we want to call some piece of code every time someone sends a message to a queue. We’ll use another AWS solution – SQS for this. If you’re beginning to wonder, whether this is a paid article – unfortunately not. I’m just describing what I’ve learnt recently.

First, we need to create the function. I’m going to author it from scratch and use Python 3.8 for this.

After doing this, we need to add a trigger from an existing SQS:

Since we’re creating a solution from scratch, the Lambda does not have permissions to be called. We need to assign proper role:

The code is pretty straightforward. We just want to print the information that we’ve received an event (we’ll look at it in the logs later).

Now we’ll create and call a test event from the Lambda console itself. The code is a JSON that passes “Hello from SQS!” string as body.

Finally, we can find it in the CloudWatch logs.

Configuring AWS Lambda in PyCharm

The problem is that the IDE available is very… Minimalist. Not to say ascetic. What I missed was some serious solution that would allow me to work on the code in a civilized manner. The least acceptable features being the ability to use debugger with the test events. At the beginning, we need to install the AWS Toolkit plugin from the PyCharm’s marketplace.

Next, after finishing this step, and setting up proper credentials (remember not to use the root account), we’ll be able to find the explorer bar. As we can see, there’s an option to update the function code.

The cool thing about the solution is that we’re able to create the same test events in the Run Configuration.

Of course we won’t be able to use debugger. In order to do this, we need a local, not remote configuration. This requires a few additional steps, like installing SAM CLI.

Once we’ve carefully followed the instructions and done like a ton of stuff not covered in the official docs, we can move on to setting up the local environment. Watch out: by saying “a ton” I really mean it. Some issues were fairly easy, like creating empty requirements file when SAM complained about the lack of it. Other cases required a little bit of googling. You’d need to do some manual symlinking for instance. The positive side, however, is that there are people, who already had those problems, so it’s enough to copy the error message into Google. For instance, when you encounter a complaint about SAM not being able to locate Python 3.8 in the PATH, you’ll need to run

ln -sf /usr/local/opt/python@3.8/bin/python3.8 /usr/local/bin/

Another example is the need to share /Applications directory with Docker. Nothing scary.

How to run AWS Lambda in Pycharm?

Now, let’s move on to the fun part. When we have everything up and running, we can configure the test event to run locally:

Take a look at the possibility to set up timeouts or maximum memory, just like you’d normally do in the AWS console. This allows us to modify the code easily. Let’s assume for a moment, that we only want to extract the body from the event. Now we can do it easily and debug the running code.

Deploying Lambda to AWS through PyCharm

As you’ve might noticed before, we can directly update the lambda code. For this we need an S3 bucket and one additional configuration step to point to this bucket and handler.

The AWS console should welcome us with a message informing about the changes that took place in the background.

Here you can see that the updated code actually landed in the console.

Finally, we can test a real event. Let’s send something from the SQS console.

Lambda has received and processed the event correctly logging the contents to the CloudWatch service.

Final thoughts

To sum up – we’ve created a Lambda function in the AWS and we’ve learnt how to use some external tools to make our work with it smooth. Of course this solution has drawbacks, the biggest one being the need to stay close to the AWS, but on the other hand, it opens up some interesting possibilities. Lambdas are fairly cheap, so I encourage you to play around with it yourself. Have fun and happy hacking. And subscribe to the newsletter if you want to be notified about my infrequent posts.