Problem

After creating jobs using the runs submit API, you don’t see them in the Job run and Job tabs within the Databricks UI. 

Cause

Jobs created using the runs submit API are not indexed and do not create a job object in the jobs list. (The runs submit API runs a one-time job without generating a persistent job object.) 

However, the Job run and Job tab search functionality in the UI relies on the jobs list. As a result, jobs created through runs submit are not visible in the UI. You can confirm this in the Create and trigger a one-time run API documentation.

Solution

To retrieve job details or ensure jobs appear in the UI, either retrieve the job information also using the jobs/runs/get API, or use the Jobs API to create searchable jobs.

Retrieve job information using the API

Use the jobs/runs/get API. Call the Get a single job run API endpoint to get details about a specific job run. 

Use the Databricks Jobs API to create searchable jobs

To ensure jobs appear in the UI, use the Databricks Jobs API instead. This creates a job object in the jobs list, making it searchable.To create and trigger a one-time job run, refer to the Create and trigger a one-time run API documentation.