Executing the first cell which installs the required packages:
# please ignore warning messages during the installation
!pip install --disable-pip-version-check -q sagemaker==2.35.0
!conda install -q -y pytorch==1.6.0 -c pytorch
!pip install --disable-pip-version-check -q transformers==3.5.1
returns these warnings and messages:
DEPRECATION: pyodbc 4.0.0-unsupported has a non-standard version number. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pyodbc or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at https://github.com/pypa/pip/issues/12063
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
sparkmagic 0.20.4 requires nest-asyncio==1.5.5, but you have nest-asyncio 1.5.7 which is incompatible.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working...
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- defaults/linux-64::anaconda-client==1.7.2=py37_0
- defaults/noarch::anaconda-project==0.8.4=py_0
- defaults/linux-64::bokeh==1.4.0=py37_0
- defaults/noarch::dask==2.11.0=py_0
- defaults/linux-64::distributed==2.11.0=py37_0
- defaults/linux-64::spyder==4.0.1=py37_0
- defaults/linux-64::watchdog==0.10.2=py37_0
done
## Package Plan ##
environment location: /opt/conda
added / updated specs:
- pytorch==1.6.0
The following packages will be downloaded:
package | build
---------------------------|-----------------
cudatoolkit-10.2.89 | hfd86e86_1 365.1 MB
ninja-1.10.2 | h06a4308_5 8 KB
ninja-base-1.10.2 | hd09550d_5 109 KB
openssl-1.1.1v | h7f8727e_0 3.7 MB
pytorch-1.6.0 |py3.7_cuda10.2.89_cudnn7.6.5_0 537.7 MB pytorch
pyyaml-5.3.1 | py37h7b6447c_0 181 KB
------------------------------------------------------------
Total: 906.8 MB
The following NEW packages will be INSTALLED:
cudatoolkit pkgs/main/linux-64::cudatoolkit-10.2.89-hfd86e86_1 None
ninja pkgs/main/linux-64::ninja-1.10.2-h06a4308_5 None
ninja-base pkgs/main/linux-64::ninja-base-1.10.2-hd09550d_5 None
pytorch pytorch/linux-64::pytorch-1.6.0-py3.7_cuda10.2.89_cudnn7.6.5_0 None
pyyaml pkgs/main/linux-64::pyyaml-5.3.1-py37h7b6447c_0 None
The following packages will be UPDATED:
openssl 1.1.1u-h7f8727e_0 --> 1.1.1v-h7f8727e_0 None
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Retrieving notices: ...working... done
DEPRECATION: pyodbc 4.0.0-unsupported has a non-standard version number. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of pyodbc or contact the author to suggest that they release a version with a conforming version number. Discussion can be found at https://github.com/pypa/pip/issues/12063
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Then executing the 2nd cell:
import boto3
import sagemaker
import pandas as pd
import numpy as np
import botocore
config = botocore.config.Config(user_agent_extra='dlai-pds/c2/w2')
# low-level service client of the boto3 session
sm = boto3.client(service_name='sagemaker',
config=config)
sm_runtime = boto3.client('sagemaker-runtime',
config=config)
sess = sagemaker.Session(sagemaker_client=sm,
sagemaker_runtime_client=sm_runtime)
bucket = sess.default_bucket()
role = sagemaker.get_execution_role()
region = sess.boto_region_name
returns error as follow:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-0fae4f2af982> in <module>
1 import boto3
----> 2 import sagemaker
3 import pandas as pd
4 import numpy as np
5 import botocore
/opt/conda/lib/python3.7/site-packages/sagemaker/__init__.py in <module>
16 import importlib_metadata
17
---> 18 from sagemaker import estimator, parameter, tuner # noqa: F401
19 from sagemaker.amazon.kmeans import KMeans, KMeansModel, KMeansPredictor # noqa: F401
20 from sagemaker.amazon.pca import PCA, PCAModel, PCAPredictor # noqa: F401
/opt/conda/lib/python3.7/site-packages/sagemaker/estimator.py in <module>
26 from six.moves.urllib.parse import urlparse
27 import sagemaker
---> 28 from sagemaker import git_utils, image_uris
29 from sagemaker.analytics import TrainingJobAnalytics
30 from sagemaker.debugger import TensorBoardOutputConfig # noqa: F401 # pylint: disable=unused-import
/opt/conda/lib/python3.7/site-packages/sagemaker/image_uris.py in <module>
20
21 from sagemaker import utils
---> 22 from sagemaker.spark import defaults
23
24 logger = logging.getLogger(__name__)
/opt/conda/lib/python3.7/site-packages/sagemaker/spark/__init__.py in <module>
14 from __future__ import absolute_import
15
---> 16 from sagemaker.spark.processing import PySparkProcessor, SparkJarProcessor # noqa: F401
/opt/conda/lib/python3.7/site-packages/sagemaker/spark/processing.py in <module>
33
34 from sagemaker import image_uris
---> 35 from sagemaker.local.image import _ecr_login_if_needed, _pull_image
36 from sagemaker.processing import ProcessingInput, ProcessingOutput, ScriptProcessor
37 from sagemaker.s3 import S3Uploader
/opt/conda/lib/python3.7/site-packages/sagemaker/local/__init__.py in <module>
14 from __future__ import absolute_import
15
---> 16 from .local_session import ( # noqa: F401
17 file_input,
18 LocalSagemakerClient,
/opt/conda/lib/python3.7/site-packages/sagemaker/local/local_session.py in <module>
21 from botocore.exceptions import ClientError
22
---> 23 from sagemaker.local.image import _SageMakerContainer
24 from sagemaker.local.entities import (
25 _LocalEndpointConfig,
/opt/conda/lib/python3.7/site-packages/sagemaker/local/image.py in <module>
36
37 import sagemaker
---> 38 import sagemaker.local.data
39 import sagemaker.local.utils
40 import sagemaker.utils
/opt/conda/lib/python3.7/site-packages/sagemaker/local/data.py in <module>
24 from six.moves.urllib.parse import urlparse
25
---> 26 import sagemaker.amazon.common
27 import sagemaker.local.utils
28 import sagemaker.utils
/opt/conda/lib/python3.7/site-packages/sagemaker/amazon/common.py in <module>
21 import numpy as np
22
---> 23 from sagemaker.amazon.record_pb2 import Record
24 from sagemaker.deprecations import deprecated_class
25 from sagemaker.deserializers import SimpleBaseDeserializer
/opt/conda/lib/python3.7/site-packages/sagemaker/amazon/record_pb2.py in <module>
50 extension_scope=None,
51 options=_descriptor._ParseOptions(descriptor_pb2.FieldOptions(), _b("\020\001")),
---> 52 file=DESCRIPTOR,
53 ),
54 _descriptor.FieldDescriptor(
/opt/conda/lib/python3.7/site-packages/google/protobuf/descriptor.py in __new__(cls, name, full_name, index, number, type, cpp_type, label, default_value, message_type, enum_type, containing_type, is_extension, extension_scope, options, serialized_options, has_default_value, containing_oneof, json_name, file, create_key)
559 has_default_value=True, containing_oneof=None, json_name=None,
560 file=None, create_key=None): # pylint: disable=redefined-builtin
--> 561 _message.Message._CheckCalledFromGeneratedFile()
562 if is_extension:
563 return _message.default_pool.FindExtensionByName(full_name)
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
These packages cause dependency issues to complete the assignment!
How could we fix the errors?
Cheers,