Problem
When you spin up your cluster, Databricks Runtime updates pytest to version 8.3.3. You notice Fixtures do not function as expected, and when you attempt to execute a test you receive an error message. Fixtures worked correctly in pytest version 8.3.2.
Example of test execution
import ipytest
import pytest
def add(a, b):
return a+b
@pytest.fixture
def input_data():
return (2, 3)
def test_add (input_data):
a, b = input_data
result = add(a, b)
assert result == 5
ipytest.run('-vv')
Example of returned error
____________________________________ ERROR at setup of test_add ____________________________________
file /home/spark-3e47400d-17a6-444e-ac05-ed/.ipykernel/56909/command-4400526906963776-1085653689, line 12
def test_add (input_data):
E fixture 'input_data' not found
> available fixtures: anyio_backend, anyio_backend_name, anyio_backend_options, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
Cause
Databricks uses specialized IPython functionality that conflicts with pytest's execution environment in version 8.3.3.
Solution
Update Databricks Runtime to 16.1 for a built-in fix for this issue.
If you can’t or don’t want to update your Databricks Runtime, you can safely downgrade pytest to version 8.3.2 to continue using the library in your Databricks environment.
Note
As with any third party library, downgrading from the most current version of the library may mean you miss library patches or important updates. Read the library release notes as needed.