Course 2, week 1 lab3 - Error

Course Q&A Data Engineering Source Systems, Data Ingestion, and Pipelines
1

Getting an error in the code cell that is querying the S3 bucket. Here is the code:
file_s3_key = ‘csv/ratings_ml_training_dataset.csv’
kwargs = {‘ExpressionType’: ‘SQL’,
‘Expression’: “”“SELECT * FROM s3object AS s WHERE s."productline" = ‘Trains’ LIMIT 20"”",
‘InputSerialization’: {‘CSV’: {“FileHeaderInfo”: “Use”}, ‘CompressionType’: ‘NONE’},
‘OutputSerialization’: {‘CSV’: {}},
}

response = s3_select_object_content(bucket_name=BUCKET_NAME, object_key=file_s3_key, **kwargs)

Here is the error:
ClientError: An error occurred (MethodNotAllowed) when calling the SelectObjectContent operation: The specified method is not allowed against this resource.

Everything seems to be set up properly. I get the same error running the provided solution code. Is there an issue in the lab?

Thank you for any help,
Lee

3 Likes
2

I get the same issue

3

according to S3 documentation, this capability is not available to new customers. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/select_object_content.html

not sure if that’s the reason

4

Dealing with the same issue right now. Glad someone else ran into it :sweat_smile:

Since that method isn’t available to new users, did you find a workaround for it? Or are you still stuck?

5

I skipped that step and the next one. Did the rest of the lab and got 100% on it. I guess it isn’t required. Hope this helps.

2 Likes
6

Thank you everyone for reporting this issue. We have informed our engineering team and they will look into it. In the meantime, please skip over the last two code cells in section 4.1 and continue on with the rest of the lab.

5 Likes
7

I got the same error. There is an alternative way to get the data using Athena.
(How to optimize querying your data in Amazon S3 | AWS Storage Blog)
But I got a permission error. I am guessing we do not have permission for Athena on the lab.

8

I tried this on my personal account today and it worked fine.

image
image954×601 37 KB

9

hi @jessica-dlai any update on this issue?

10

Hi @arunvsuresh , no updates yet, sorry about this. The engineers are working through a backlog of troubleshooting issues, hopefully they can resolve this issue next week. Thank you so much for you patience. Are you able to complete the rest of the lab by skipping the two cells?

11

Yup! And no worries, thanks for the update.

12

Hey @jessica-dlai, I also encounter the same issue. Wondering if the issue has been resolved?

Error code happend when I execuded following:

file_s3_key = 'csv/ratings_ml_training_dataset.csv'
kwargs = {'ExpressionType': 'SQL',
          'Expression': """SELECT * FROM s3object AS s WHERE s.\"productline\" = 'Trains' LIMIT 20""",
          'InputSerialization': {'CSV': {"FileHeaderInfo": "Use"}, 'CompressionType': 'NONE'},
          'OutputSerialization': {'CSV': {}},
}

response = s3_select_object_content(bucket_name=BUCKET_NAME, object_key=file_s3_key, **kwargs)

Below is the error code:

---------------------------------------------------------------------------
ClientError                               Traceback (most recent call last)
Cell In[22], line 8
      1 file_s3_key = 'csv/ratings_ml_training_dataset.csv'
      2 kwargs = {'ExpressionType': 'SQL',
      3           'Expression': """SELECT * FROM s3object AS s WHERE s.\"productline\" = 'Trains' LIMIT 20""",
      4           'InputSerialization': {'CSV': {"FileHeaderInfo": "Use"}, 'CompressionType': 'NONE'},
      5           'OutputSerialization': {'CSV': {}},
      6 }
----> 8 response = s3_select_object_content(bucket_name=BUCKET_NAME, object_key=file_s3_key, **kwargs)

Cell In[20], line 14, in s3_select_object_content(bucket_name, object_key, **kwargs)
      2 """Uses the s3 select utility to get the content of a file.
      3 
      4 Args:
   (...)
      9     Dict[Any, Any]: Response from S3 select
     10 """
     12 s3_client = boto3.client('s3')
---> 14 response = s3_client.select_object_content(
     15     Bucket=bucket_name,
     16     Key=object_key,
     17     **kwargs
     18 )
     20 return response

File ~/environment/jupyterlab-venv/lib64/python3.9/site-packages/botocore/client.py:553, in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    549     raise TypeError(
    550         f"{py_operation_name}() only accepts keyword arguments."
    551     )
    552 # The "self" in this scope is referring to the BaseClient.
--> 553 return self._make_api_call(operation_name, kwargs)

File ~/environment/jupyterlab-venv/lib64/python3.9/site-packages/botocore/client.py:1009, in BaseClient._make_api_call(self, operation_name, api_params)
   1005     error_code = error_info.get("QueryErrorCode") or error_info.get(
   1006         "Code"
   1007     )
   1008     error_class = self.exceptions.from_code(error_code)
-> 1009     raise error_class(parsed_response, operation_name)
   1010 else:
   1011     return parsed_response

ClientError: An error occurred (MethodNotAllowed) when calling the SelectObjectContent operation: The specified method is not allowed against this resource.
13

Hi @minlin ,

Thanks for raising this again. Our engineers looked into this. The issue is due to AWS deprecating S3 select. We are working on a solution. In the meantime, please skip over this and complete the rest of the lab. Thank you so much for your patience!

Jess

14

I am too getting the same error .
Kind of stuck here