Lab week 4: Lambda function for streaming pipeline keeps erroring

Course Q&A Data Engineering Introduction to Data Engineering
,
1

I followed exactly the steps in the course, however the lambda function for streaming data transformations keeps erroring, cf Cloudwatch log in image below. It seems the error is linked to user_emb in call below but URL_LAMBDA_INFERENCE seems properly set in AWS UI interface (cf image below):

        # Getting the embedding of the user
        usr_emb = get_user_embedding(url=URL_LAMBDA_INFERENCE, data=usr_dict)

        # Recommending item for the user
        recommended_items = get_item_from_user(
            url=URL_LAMBDA_INFERENCE,
            data=usr_emb[0],
            item_limit=int(ITEM_LIMIT),
        )

image
image1285×454 25.6 KB

image
image868×182 15 KB

2

After some digging and adding additional logging information in the lambda function it seems like a permission problem from the inference function:

Code added in lambda function

logger.info(f"user_embeddings status={response.status}, body={response.data}")

New message in the log after code modification

[INFO] 2026-03-27T15:02:32.536Z e8a69855-dc3f-47ef-9729-e8893d545061 get_user_embedding url=https://ljjp52ike2gmd2bykhihn2ubei0lgwsd.lambda-url.us-east-1.on.aws/user_embeddings status=403 body= { "Message": "Forbidden. For troubleshooting Function URL authorization issues, see: https://docs.aws.amazon.com/lambda/latest/dg/urls-auth.html" }

This problem also seems hinted at by Amazon message (cf message in blue below).

image
image1423×1345 161 KB

I tried solving it by adding the proper permission but i didn’t have the right to do so (cf image below).

image
image1411×874 78.5 KB

3

@fab42 don’t over-complicate things. You have just enough permissions to complete this lab. Try to reboot it and start again. Usually works.