如何获取Azure Databricks笔记本运行详细信息

Question

我正在使用Azure Data Factory运行我的databricks笔记本，它在运行时创建了作业集群，现在我想知道这些作业的状态，我的意思是它们是成功还是失败。那么我可以知道，我如何使用作业id或运行id来获得运行状态。

注意:我没有在我的databricks工作区中创建任何作业，我正在使用Azure Data Factory运行我的笔记本，它在运行时创建作业集群，并在该集群上运行该笔记本，然后终止该集群

Answer 1

https://forums.databricks.com/questions/12134/possible-to-get-job-run-id-of-notebook-run-by-dbut.html

dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson() 

Answer 2

你必须去Azure Data Factory的monitor页面。您可以在这里通过runId进行过滤。

https://learn.microsoft.com/en-us/azure/data-factory/transform-data-using-databricks-notebook#monitor-管道运行

Answer 3

import json
import requests

gethooks= "https://" + databricks_instance_name + "/api/2.0/jobs/runs/list"     #add your databricks workspace instance name over here
headers={"Authorization": "Bearer ********************"}        # Add your databricks access token
response = requests.get(gethooks, headers=headers)

print(response.json())      # you will get all cluster and job related info over here in json format

# traversing through response.json
for element in response.json()['runs']:
    job_id = element['job_id']
    status = element['state']['result_state']
    job_path = element['task']['notebook_task']['notebook_path']
    job_name = job_path.split('/')