Fixture not found error when using pytest on a cluster

Downgrade pytest to version 8.3.2 or upgrade Databricks Runtime to 16.1.

Written by kaushal.vachhani

Last published at: January 7th, 2025

Problem

When you spin up your cluster, Databricks Runtime updates pytest to version 8.3.3. You notice Fixtures do not function as expected, and when you attempt to execute a test you receive an error message. Fixtures worked correctly in pytest version 8.3.2. 

 

Example of test execution

import ipytest 
import pytest

def add(a, b):
  return a+b

@pytest.fixture
def input_data():
  return (2, 3)

def test_add (input_data):
  a, b = input_data
  result = add(a, b)
  assert result == 5

ipytest.run('-vv')

 

Example of returned error

____________________________________ ERROR at setup of test_add ____________________________________
file /home/spark-3e47400d-17a6-444e-ac05-ed/.ipykernel/56909/command-4400526906963776-1085653689, line 12
  def test_add (input_data):
E       fixture 'input_data' not found
>       available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory

 

Cause

Databricks uses specialized IPython functionality that conflicts with pytest's execution environment in version 8.3.3. 

 

Solution

Update Databricks Runtime to 16.1 for a built-in fix for this issue. 

If you can’t or don’t want to update your Databricks Runtime, you can safely downgrade pytest to version 8.3.2 to continue using the library in your Databricks environment.

 

Note

As with any third party library, downgrading from the most current version of the library may mean you miss library patches or important updates. Read the library release notes as needed.