Does the AWS SDK for Python reuse HTTPS connections?

Originally posted on 2021-07-16

I was asked this question today and I assumed the answer was yes. However, being the kind of person I am I just had to find out if it was true and provide proof.

How did I do this? I wrote a few lines of Python code that published a message to AWS IoT Core, using the iot-data client in boto3, and turned on DEBUG logging for urllib3.connectionpool.

Many people don't know this but iot-data is a data plane API that uses HTTPS, not MQTT. It is great for cloud-based IoT components since it doesn't require a lot of management of MQTT connections. The drawback is that it only allows you to publish messages and not receive them. If you want to receive messages in cloud-based IoT components on AWS you should look into having the rules engine either invoke your ingest code as a Lambda function or publish to an HTTPS endpoint in your application.

Anyway, here's the code:

#!/usr/bin/env python3

import boto3
import logging

control_plane = boto3.client(service_name='iot')
endpoint_address = control_plane.describe_endpoint(endpointType="iot:Data-ATS")['endpointAddress']
data_plane = boto3.client(service_name='iot-data', endpoint_url='https://' + endpoint_address)
boto3.set_stream_logger('urllib3.connectionpool', logging.DEBUG)

for i in range(100):
  response = data_plane.publish(
    topic='topic',
    qos=0,
    payload=b'payload'
  )

This will publish 100 messages to IoT Core in your account if you have AWS credentials set up and you have access to publish to IoT Core in your current region.

You should see a message like this only once:

2021-07-16 12:08:58,006 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): axxxxxxxxxxxxxd-ats.iot.us-east-1.amazonaws.com:443

Followed by 100 messages like this:

2021-07-16 12:09:40,692 urllib3.connectionpool [DEBUG] https://axxxxxxxxxxxxd-ats.iot.us-east-1.amazonaws.com:443 "POST /topics/topic?qos=0 HTTP/1.1" 200 65

That verifies that the Python SDK will reuse HTTPS connections when it is possible. Enjoy!