This line in the final cell:
statistics = tfdv.generate_statistics_from_tfrecord(TRAIN_DIR, stats_options=stats_options)
Creates this repeating warning that eventually results in the cell failing:
WARNING:apache_beam.utils.retry:Retry with exponential backoff: waiting for 4.2796533589988215 seconds before retrying list_prefix because we caught exception: TypeError: ‘>=’ not supported between instances of ‘NoneType’ and ‘int’
Traceback for above exception (most recent call last):
File “/usr/local/lib/python3.7/dist-packages/apache_beam/utils/retry.py”, line 253, in wrapper
return fun(*args, **kwargs)
File “/usr/local/lib/python3.7/dist-packages/apache_beam/io/gcp/gcsio.py”, line 561, in list_prefix
response = self.client.objects.List(request)
File “/usr/local/lib/python3.7/dist-packages/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py”, line 1180, in List
config, request, global_params=global_params)
File “/usr/local/lib/python3.7/dist-packages/apitools/base/py/base_api.py”, line 729, in _RunMethod
http, http_request, **opts)
File “/usr/local/lib/python3.7/dist-packages/apitools/base/py/http_wrapper.py”, line 360, in MakeRequest
max_retry_wait, total_wait_sec))
File “/usr/local/lib/python3.7/dist-packages/apache_beam/io/gcp/gcsio_overrides.py”, line 45, in retry_func
return http_wrapper.HandleExceptionsAndRebuildHttpConnections(retry_args)
File “/usr/local/lib/python3.7/dist-packages/apitools/base/py/http_wrapper.py”, line 290, in HandleExceptionsAndRebuildHttpConnections
Any ideas on the cause and the fix?