PROBLEM => Practice Lab 3: Interacting With Amazon S3 Object Storage

  1. Week 1
  2. Practice Lab 3: Interacting With Amazon S3 Object Storage

Within the JupyterLab C2_W1_Lab_3_S3.ipynb → select_object_content() always produces the same error.

Please see!
While executing the following command:

file_s3_key = ‘csv/ratings_ml_training_dataset.csv’
kwargs = {‘ExpressionType’: ‘SQL’,
‘Expression’: “”“SELECT * FROM s3object AS s WHERE s."productline" = ‘Trains’ LIMIT 20"”",
‘InputSerialization’: {‘CSV’: {“FileHeaderInfo”: “Use”}, ‘CompressionType’: ‘NONE’},
‘OutputSerialization’: {‘CSV’: {}},
}

response = s3_select_object_content(bucket_name=BUCKET_NAME, object_key=file_s3_key, **kwargs)

It ALWAYS produces the same error like:

ClientError Traceback (most recent call last)
Cell In[15], line 9
1 file_s3_key = ‘csv/ratings_ml_training_dataset.csv’
3 kwargs = {‘ExpressionType’: ‘SQL’,
4 ‘Expression’: “”“SELECT * FROM s3object AS s WHERE s."productline" = ‘Trains’ LIMIT 20"”",
5 ‘InputSerialization’: {‘CSV’: {“FileHeaderInfo”: “Use”}, ‘CompressionType’: ‘NONE’},
6 ‘OutputSerialization’: {‘CSV’: {}},
7 }
----> 9 response = s3_select_object_content(bucket_name=BUCKET_NAME, object_key=file_s3_key, **kwargs)

Cell In[10], line 14, in s3_select_object_content(bucket_name, object_key, **kwargs)
2 “”“Uses the s3 select utility to get the content of a file.
3
4 Args:
(…)
9 Dict[Any, Any]: Response from S3 select
10 “””
12 s3_client = boto3.client(‘s3’)
—> 14 response = s3_client.select_object_content(
15 Bucket=bucket_name,
16 Key=object_key,
17 **kwargs
18 )
20 return response

File ~/environment/jupyterlab-venv/lib64/python3.9/site-packages/botocore/client.py:553, in ClientCreator._create_api_method.._api_call(self, *args, **kwargs)
549 raise TypeError(
550 f"{py_operation_name}() only accepts keyword arguments."
551 )
552 # The “self” in this scope is referring to the BaseClient.
→ 553 return self._make_api_call(operation_name, kwargs)

File ~/environment/jupyterlab-venv/lib64/python3.9/site-packages/botocore/client.py:1009, in BaseClient._make_api_call(self, operation_name, api_params)
1005 error_code = error_info.get(“QueryErrorCode”) or error_info.get(
1006 “Code”
1007 )
1008 error_class = self.exceptions.from_code(error_code)
→ 1009 raise error_class(parsed_response, operation_name)
1010 else:
1011 return parsed_response

ClientError: An error occurred (MethodNotAllowed) when calling the SelectObjectContent operation: The specified method is not allowed against this resource.

I checked the solution notebook (C2_W1_Lab_3_S3_Solution.ipynb), but No Luck! Could you kindly have a look at this and provide a solution!? The sooner the better!

Many Thanks!

Hello @Utpoul_Kumar_MONDAL , this issue has been reported by several others as well, first post and here.
Short answer is to skip the two cells after s3_select_object_content function and continue with the lab for now, maybe come back later if fixed. Thank you

3 Likes

Thanks Georgios!

I think, I have found out the reason. In the LAB Setup, within the IAM there’s NO policy for the Resources - which should look like (just an example for my Lab) the following:

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“s3:GetObject”,
“s3:SelectObjectContent”
],
“Resource”: “arn:aws:s3:::de-c2w1lab3-339712852169/csv/ratings_ml_training_dataset.csv”
}
]
}

Without the permission select_object_content cannot pass!
The irony is : we are not authorized to create the policy!

I’ve even tried a simple query like:
kwargs = {
‘ExpressionType’: ‘SQL’,
‘Expression’: “SELECT * FROM s3object LIMIT 10”, # Simple query
‘InputSerialization’: {‘CSV’: {“FileHeaderInfo”: “USE”}},
‘OutputSerialization’: {‘CSV’: {}},
}

response = s3_select_object_content(bucket_name=“de-c2w1lab3-339712852169”, object_key=file_s3_key, **kwargs)

And Yet…
It says:
An error occurred (MethodNotAllowed) when calling the SelectObjectContent operation: The specified method is not allowed against this resource.

So, I think, we need the resource (IAM) permission.

Well, let’s complete the others as you’ve suggested!
Thanks again!

2 Likes

Thank you @Georgios .Looks like the issue hasn’t been fixed yet. I think we are not granted the IAM permission

Having the same issue over here. Any more suggestions ?