How to track usage of Amazon Bedrock models?

100% Human-written

Amazon bedrock is an API based service in the AWS ecosystem which helps customers use foundation models from different providers in their own applications. Tagging resources is one of the ways in which B2Bs track their resource utilisation. But how do you track calls to a model which is used in a microservice which is in-turn invoked by many tenants.

You can do this by creating “application inference profiles”. This is an abstraction layer built over the foundation model. If you look at the response of create_inference_profile function, you will be able to see the extra layers added to the base-foundation model.

bedrock_client = boto3.client('bedrock', region_name='us-east-1')

import json
def create_application_inference_profile(profile_name: str, model_arn: str, description: str, tags: list):
    response =  bedrock_client.create_inference_profile(
        inferenceProfileName=profile_name,
        description=description,
        modelSource={'copyFrom': model_arn},
         tags=tags # create tags when creating a custom application inference profile
    )
    
    print("CreateInferenceProfile Response:", response['ResponseMetadata']['HTTPStatusCode']),
    print(json.dumps(response, indent=4, sort_keys=True, default=str))
    return response

tags = [{'key': 'dept'…