Has anyone gotten cudf.read_csv to work?

Attempting to open any csv file using cudf.read_csv results in a crash. For example,
taxi2016 = cudf.read_csv("taxi2016.csv")

results in:

RuntimeError: cuFile error at: [/var/home/fraser/anaconda3/envs/rapids-23.12/include/kvikio/file_handle.hpp:178](https://file+.vscode-resource.vscode-cdn.net/var/home/fraser/anaconda3/envs/rapids-23.12/include/kvikio/file_handle.hpp:178): internal error

I have never been able to read a file using cudf 23.12.01.

Polars and Pandas read_csv opens the same file. Does anyone use cudf?

Source: Installation Guide - RAPIDS Docs

Docker command:

docker run --gpus all --pull always --rm -it \
    --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \
    -p 8888:8888 -p 8787:8787 -p 8786:8786 \


Thanks. I have rapids up and running in conda (and distrobox can pull the docker image). Every example works from the cudf notebook examples except the read_csv function. I will try either pandas dataframe to cudf or polars dataframe to cudf as it is only the read_csv function that is crashing. I could be having issues because my Nvidia A1000 is so small but the helpful error message is, well, not telling me anything.

Yup, loading from pandas and then converting to cudf works as a workaround: cudf.DataFrame.from_pandas(taxi2016pl).

For Polars, monstrously, one must convert the polars dataframe to pandas before converting to cudf!

# Assuming `df_polars` is your Polars DataFrame
df_pandas = df_polars.to_pandas()
df_cudf = cudf.from_pandas(df_pandas)

I will have to wait for native polars support in cudf as polars is noticeably faster than pandas.

unfortunately, using dask results in the same issue as with cudf (dask may be using cudf to read csv).

Jumping across dask / pandas & polars is not necessary when there’s a read_csv method on cudf. Docker is pretty much the last thing to do when we’re running on different environments and the logs shared are not informative.

Now is a good time for you to file a github issue / join the rapids discussion forum.
Please reply on this thread once you find a solution. Good luck.