AWS HTTP(S) Authentication

The AWS service connector exposes an HTTP proxy that brokers connection requests between AWS clients and AWS endpoints, without ever revealing the access keys to the client. By default, AWS clients communicate with AWS endpoints over HTTPS. The Secretless AWS connector is an HTTP proxy, which means clients can connect to it with an HTTP-only connection.

The purpose of the AWS service connector is to authenticate a request by signing it with the credentials associated with the connector. The AWS client doesn't know that Secretless exists and must have all of the configuration settings it typically expects. The example below shows that fake credentials are provided to the client. This is to satisfy AWS client configuration setting requirements and the fake credentials are overwritten with real credentials by the connector.

All other AWS client configuration setting requirements (such as the AWS region) must be met, and will be honored by the request. For more information, see AWS client configuration settings.

In order to use the Secretless connector, AWS clients need to be specially configured so they can make use of the Secretless HTTP proxy. This requires two steps:

In order for AWS clients to use the Secretless connector, they need to be reconfigured so they can make use of the Secretless HTTP proxy. This requires two steps:

  1. Configure the HTTP proxy for the client to use. In a majority of cases, this is achieved by setting an environment variable called http_proxy.The variable value should be set to the Secretless service endpoint.

  2. Configure the client to communicate with the AWS endpoint via HTTP. This is needed so the client can use the HTTP proxy configured in Step 1 above. The way we achieve this is to set the AWS endpoint that the client will use. Note that when you set the AWS endpoint, you lose the automatic endpoint detection capability, which is offered by AWS clients by default.

    • If you want to retrieve the automatic endpoint detection while using the HTTP proxy from Secretless, set the endpoint URL to a specific dummy value (http://secretless.empty for example: aws --endpoint-url http://secretless.empty. Secretless detects the dummy value and performs the same automatic endpoint detection used by the AWS client to route the request.

    • If you were already using a custom endpoint, make sure that the scheme of the endpoint URL is HTTP.

 

You may need to provide your client with dummy AWS authentication configuration. This is so the client doesn't error due to missing authentication. This information is not actually consumed by the client since it uses a connection authenticated via Secretless.

Example client configuration

Assuming that you've configured a Secretless AWS service endpoint that is listening on port 8080, here is how you might configure the AWS CLI to consume Secretless:

 

Although we show the AWS CLI in the following example, the same approach would apply to other AWS clients.

  1. Set the environment variable so that the AWS CLI uses the Secretless HTTP proxy HTTP_PROXY=http://localhost:8080.

  2. Set an HTTP endpoint URL so the client can make use of the HTTP proxy configuration from Step 1 above.

     

    Notice that we are setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables to dummy values. This is so the client doesn't error due to missing authentication. This information is not actually consumed by the client since it uses a connection authenticated via Secretless.
      
    alias aws="AWS_ACCESS_KEY_ID=x AWS_SECRET_ACCESS_KEY=x HTTP_PROXY=http://localhost:8080 aws --endpoint-url http://secretless.empty

    In the above example, the HTTP endpoint URL is set to http://secretless.empty, which allows Secretless to leverage automatic endpoint discovery. Optionally, you could set the HTTP endpoint URL to a fixed value, for example: http://secretsmanager.us-east-1.amazonaws.com.

Now that the AWS CLI is set up, use it to seamlessly connect to AWS via Secretless.

  
aws sts get-caller-identity
{
    "UserId": "ABC123",
    "Account": "000000",
    "Arn": "arn:aws:iam::xxx:xxx/xxx"
}

Service parameter

  • authenticateURLsMatching

    Required

    An array of regex patterns which match a request URI, either partially or fully. Requests which are matched by a regex in this array will be authenticated by this connector.

Credentials

  • accessKeyID

    Required

    AWS access key ID

  • secretAccessKey
    Optional
    AWS secret access key

  • accessToken
    Required
    AWS session token

Examples

Authenticate all requests

  
                            
version: "2"
services:
  aws-proxy:
    connector: aws
    listenOn: tcp://0.0.0.0:8080
    credentials:
      accessKeyId:
        from: env
        get: AWS_ACCESS_KEY_ID
      secretAccessKey:
        from: env
        get: AWS_SECRET_ACCESS_KEY
    config:
      authenticateURLsMatching:
        - .*

Only authenticate requests to Amazon EC2

  
                            
version: "2"
services:
  aws-proxy:
    connector: aws
    listenOn: tcp://0.0.0.0:8080
    credentials:
      accessKeyId:
        from: env
        get: AWS_ACCESS_KEY_ID
      secretAccessKey:
        from: env
        get: AWS_SECRET_ACCESS_KEY
    config:
      authenticateURLsMatching:
        - ^https\:\/\/ec2\..*\.amazonaws.com