Introduction
You need to find details of an SQL query run from a notebook on an all-purpose cluster but only see records for queries run using SQL warehouses or serverless compute in the query history table.
SQL queries executed from a notebook using an all-purpose cluster run as Apache Spark jobs. Both SQL commands and commands executed in notebooks are processed using Spark behind the scenes. You can use verbose audit logs instead to get the query run details.
Instructions
- Enable verbose audit logs. For instructions, refer to the Enable verbose audit logs (AWS | Azure | GCP) documentation.
- Check the
system.access.audittable for your query details. You can use the following example query to fetch the SQL query record.
select * from system.access.audit
where service_name = "notebook"
and request_params.notebookId = "<notebook-id>"
and event_date = "<yyyy-mm–dd>"