IndexError: list index out of range in using fetch_pandas_all() on Snowflake

Ah, my favorite type of error: sporadic!

A day days we started getting failures when using fetch_pandas_all() against Snowflake. The query was a basic ‘SELECT *’ against an analytical data set. Here is the trace:

Traceback (most recent call last):
  File "/Users/david/path/test/main.py", line 31, in <module>
    df = cs.fetch_pandas_all()
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/cursor.py", line 978, in fetch_pandas_all
    return self._result_set._fetch_pandas_all(**kwargs)
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_set.py", line 185, in _fetch_pandas_all
    dataframes = list(self._fetch_pandas_batches())
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_set.py", line 181, in _fetch_pandas_batches
    return self._create_iter(iter_unit=IterUnit.TABLE_UNIT, structure="pandas")
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_set.py", line 223, in _create_iter
    first_batch_iter = self.batches[0].create_iter(**kwargs)
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 717, in create_iter
    return self._get_pandas_iter(connection=connection, **kwargs)
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 699, in _get_pandas_iter
    dataframe = self.to_pandas(connection=connection, **kwargs)
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 691, in to_pandas
    table = self.to_arrow(connection=connection)
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 684, in to_arrow
    return self._create_empty_table()
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 674, in _create_empty_table
    fields = [
  File "/Users/david/path/.scriptenv/lib/python3.8/site-packages/snowflake/connector/result_batch.py", line 675, in <listcomp>
    field(s.name, FIELD_TYPE_TO_PA_TYPE[s.type_code]) for s in self.schema
IndexError: list index out of range

This error happens… sometimes. Restarting the Python kernel may fix it for a run and adding a limit statement to the select (larger than the number of rows) would sometimes fix it.

The culprit… the Geography columns (points and lines) that we recently added to the set. I’m still not sure why it would sometimes work, but removing all of those columns got everything back to reliably working.