与 Jira 和 LLM 的互动项目报告

对于我从事的所有项目，我使用了某种项目管理系统，其中项目范围被定义为任务列表（票证），并通过更改任务状态来报告进度。

虽然此类项目管理系统提供各种仪表板和报告，但解释工程师创建的具有神秘标题的一长串任务并非易事。为了给项目发起人和客户提供透明度，我必须手动创建项目报告，然后回答相关问题。

如果我们向法学硕士寻求帮助怎么办？这个想法很简单：获取所有项目任务并将它们提供给法学硕士索取报告。然后开始聊天以提出进一步的问题。

收集数据

我们将使用 Jira 进行此实验，因为它是一个流行的工具，具有易于使用的 REST API。我们将为其创建报​​告的示例项目非常技术性 - 它是关于创建一个构建脚本，该脚本可以检测代码使用的内容并生成构建系统所需的指令。这样的项目肯定会有一些标题晦涩难懂的技术任务。

让我们从获取任务开始。示例设置中的项目表示为带有子票证（任务）列表的单父票证（史诗）。对于每个任务，我们将获取完整的历史记录，以查看工单状态如何随时间变化。使用Python的Jira客户端实现很简单。请注意，在 Jira 命名法中，使用术语 issue 代替 ticket，这反映在代码中。

jira = JIRA(server=os.environ["JIRA_SERVER"], basic_auth=(os.environ["JIRA_USER"], os.environ["JIRA_TOKEN"]))

def fetch_issues(epic_key):
    issues = []
    print("Loading epic data...", end="", flush=True)
    issues.append(jira.issue(epic_key))
    print("done")

    print("Loading tasks...", end="", flush=True)
    child_issues = jira.search_issues(f"parent = {epic_key}")
    for issue in child_issues:
        issues.append(jira.issue(issue.key, expand="changelog"))
    print("done")

    return issues

由于获取所有具有历史记录的票证需要一段时间，因此可以方便地将这些数据存储在本地以进行进一步的实验。在使用实现时，我使用以下函数将任务保存到文件并从文件加载它们：

def save_issues(filename, issues):  
    with open(filename, "x") as file:
        file.write("[")
        file.write(",".join(
            json.dumps(issue.raw) for issue in issues))
        file.write("]")

def load_issues(filename):
    with open(filename, "r") as file:
        data = json.load(file)
        return [Issue(jira._options, jira._session, raw=raw_issue)
            for raw_issue in data]

准备数据

下一步是为LLM准备数据。 JSON 格式的原始 Jira 数据非常冗长，我们不需要所有这些额外的字段。让我们提取基本信息：主题、描述、类型、状态和创建日期。从历史记录中，我们只会提取工单状态更改及其日期和作者，忽略其他字段的更改。

所有这些信息都将以纯文本形式存储。我见过有人使用 JSON 或 XML 作为 LLM 输入，但我的观察是 LLM 非常擅长解释纯文本数据。另外，通过这种方法，我无需担心将文本字段格式化为 JSON 或 XML 兼容。我所做的唯一处理就是从描述中去掉空行，主要原因是为了让我更容易查看结果。

def strip_empty_lines(s):
    return "".join(line for line in (s or "").splitlines() if line.strip())

def issue_to_str(issue):
    return f"""
{issue.fields.issuetype}: {issue.key}
Summary: {issue.fields.summary}
Description: {strip_empty_lines(issue.fields.description)}
Type: {issue.fields.issuetype}
Status: {issue.fields.status}
Created: {issue.fields.created}
Priority: {issue.fields.priority}
"""

def changelog_to_str(changelog, changeitem):
    return f"""
Author: {changelog.author.displayName}
Date: {changelog.created}
Status change from: {changeitem.fromString} to: {changeitem.toString}
"""


def history_to_str(issue):
    if issue.changelog is None or issue.changelog.total == 0:
        return ""
    history_description = ""
    for changelog in issue.changelog.histories:
        try:
            statuschange = next(filter(lambda i: i.field == "status", changelog.items))
            history_description += changelog_to_str(changelog, statuschange)
        except StopIteration:
            pass
    return history_description

#this function assumes the first issue is an epic followed by tasks.
def describe_issues(issues):
    description = "Project details:"
    description += issue_to_str(issues[0])
    description += "\nProject tasks:"
    for issue in issues[1:]:
        description += "\n" + issue_to_str(issue)
        description += f"History of changes for task {issue.key}:"
        description += history_to_str(issue)
    return description

我用于此实验的史诗有 30 个任务，这些任务的历史记录中有 1 到 15 次状态更改。我不会引用describe_issues函数的完整输出，但为了让您了解它的外观，这里有一个简短的摘录：

Project details:
Epic: TKT-642
Summary: Create universal build script
Description: 
Type: Epic
Status: In Development
Created: 2024-05-24T10:48:33.050+0200
Priority: P4 - Low

Project tasks:

Task: TKT-805
Summary: add test reporting for e2e tests
Description: 
Type: Task
Status: In Progress
Created: 2024-09-06T09:56:33.919+0200
Priority: P4 - Low
History of changes for task TKT-805:
Author: Jane Doe
Date: 2024-09-06T10:04:15.325+0200
Status change from: To Do to: In Progress

Task: TKT-801
Summary: Sonar detection
Description: * add sonar config file detection *
Type: Task
Status: In Progress
Created: 2024-08-30T13:57:44.364+0200
Priority: P4 - Low
History of changes for task TKT-801:
Author: Jane Doe
Date: 2024-08-30T13:57:58.450+0200
Status change from: To Do to: In Progress

Task: TKT-799
Summary: Add check_tests step
Description: 
Type: Task
Status: Review
Created: 2024-08-29T18:33:52.268+0200
Priority: P4 - Low
History of changes for task TKT-799:
Author: Jane Doe
Date: 2024-08-29T18:40:35.305+0200
Status change from: In Progress to: Review
Author: Jane Doe
Date: 2024-08-29T18:33:57.095+0200
Status change from: To Do to: In Progress

提示

我们将使用的提示由两部分组成。首先，它给出了创建报告的说明，详细说明了我们想要在报告中包含哪些具体信息。然后我们插入前面段落中准备的门票信息。法学硕士往往会给出冗长的答复，因此我们特别要求不要添加任何额外的解释。由于实验是在终端中进行的，我们还会要求响应终端友好。

def create_prompt(isses_description):
    return f"""
    Using below information from jira generate executive report that focuses on what was done and what is remaining.
    Provide project progress information. Point out problematic tasks and what may be wrong.
    Use status change history to point out tasks that were worked on for too long or have been reopened many times. 
    Check if you can detect other problems from tasks or their history.
    Provide only text content that can be displayed on terminal, without any additional explanation or markdown.
    {isses_description}
    """

我用于此实验的法学硕士是 Anthropic 的 Claude 3.5 Sonnet。除了发送提示消息之外，我们还将指定身份。可以将身份添加为提示开头的另一行，但由于计划是跟进问题（创建交互式聊天），因此克劳德的文档建议在这种情况下将身份设置为单独的系统参数。

def send_messages(messages):
    client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_KEY"])
    identity = "You are a seasoned IT program manager. You have very good communication skills, your answers are concise and focus on key insights."
    message = client.messages.create(
        system=identity,
        max_tokens=2048,
        messages=messages,
        model="claude-3-5-sonnet-20240620"
    )
    return message.content[0].text

def generate_report(prompt):
    messages=[
        {
            "role": "user",
            "content": prompt 
        }]
    return send_messages(messages)

让我们把它放在一起并运行：

if __name__=="__main__":
    issues = fetch_issues('TKT-642')
    descrption = describe_issues(issues)
    prompt = create_prompt(description)
    report = generate_report(prompt)
    print(report)

以下是我从克劳德那里收到的完整报告：

执行报告：通用构建脚本项目 (TKT-642)

项目进展：

• 24 项任务中的 11 项已完成 (45.8%)
• 5 项任务正在进行或正在审核 (20.8%)
• 8 项任务尚未启动 (33.3%)

主要成就：

1. 为构建器创建了原型 jinja 模板
2. 实现了配置扩展和验证
3. 添加了单元测试执行步骤
4. 集成声纳扫描支持
5. 创建了用于构建配置和技术脚本的图像

进行中：

• 添加 e2e 测试的测试报告
• 实现环境和声纳检测的过滤逻辑
• 开发开发者文档

有问题的领域：

1. Task TKT-734 (Add config extension) went through multiple status changes, indicating potential issues or complexity.
2. Task TKT-697 (Add support for sonar scanning) has been reopened multiple times, suggesting implementation challenges.
3. Task TKT-715 (Create developer documentation) has been in progress for an extended period without completion.

Upcoming Work:

• Integration of the build system into various services (authentication, database-api, shopping-cart, reporting, localization)
• Adding support for monorepos and custom jobs
• Creating user documentation

Recommendations:

1. Review the implementation of config extension and sonar scanning to address recurring issues.
2. Prioritize the completion of developer documentation to support team adoption.
3. Begin planning for the integration of the build system into remaining services.
4. Consider reassessing the priority of tasks that have remained in the "To Do" state for an extended period.

Overall, the project is making steady progress, with nearly half of the tasks completed. However, attention should be given to resolving recurring issues and completing in-progress tasks to maintain momentum.

The result is quite good. It did miscount tickets - in various runs I did it gave me a different number, but never a correct number of 30. Well, LLMs are known for not being able to count r's in the word strawberry. But the summary is correct, recommendations are also reasonable.

Asking questions

Now let's create an interactive session and ask some questions. We'll start with a pre-defined question which is the original prompt we used to generate the report. Then we will read the user's input and attach it to messages. Each response, along with the report is also recorded in the messages chain.

def interactive_session(prompt):
    messages = []
    user_input = prompt
    while user_input != "exit":
        messages.append({
            "role" : "user",
            "content" : user_input
        })
        response = send_messages(messages)
        print(response)
        messages.append({
            "role" : "assistant",
            "content" : response
        })
        user_input = input("QUESTION: ")
    return messages

To run this, just replace generate_report in the main section of the script with interactive_session. Below is a recording of an example session I did. I omitted from the quote the first response which just prints out the report.

QUESTION: how many engineers the team counts?

Based on the information provided in the task histories, I can identify at least three distinct engineers working on this project:

1. John Smith
2. Jane Doe
3. Mike Carpenter

These three names appear frequently in the status change histories for various tasks. However, it's important to note that this may not represent the full team size. There could be other team members who haven't made changes recorded in these specific task histories, or who are involved in the project in other capacities not reflected in this data.

To get an accurate count of the team size, you would need additional information beyond what's provided in these task histories.

QUESTION: is unit test execution task done yet?

Yes, the unit test execution task is completed. Specifically, task TKT-700 with the summary "Add unit tests execution step" is marked as Done. Here's the relevant information:

Task: TKT-700
Summary: Add unit tests execution step
Status: Done

The history of changes for this task shows that it was moved to "Done" status on 2024-08-02, after going through the typical workflow stages (To Do -> In Progress -> Review -> PREPROD VERIFICATION -> READY FOR PRODUCTION -> Done).

QUESTION: does the scope include running some sort of static code analysis tool?

Yes, the project scope includes running static code analysis. Specifically, there's a task for implementing Sonar scanning, which is a popular static code analysis tool. Here are the details:

Task: TKT-697
Summary: Add support for sonar scanning
Status: Done

This task was completed and moved to "Done" status on 2024-09-02. The implementation of Sonar scanning indicates that static code analysis is part of the universal build script project.

Additionally, there's another related task that's currently in progress:

Task: TKT-801
Summary: Sonar detection
Status: In Progress

This task mentions "sonar detection" in its summary, which further confirms that Sonar (a static analysis tool) is being integrated into the build process.

These tasks demonstrate that static code analysis, particularly using Sonar, is included in the project scope and has been largely implemented.

QUESTION: what are the 3 biggest risks of the project?

Based on the provided information, the three biggest risks for this project appear to be:

1. 整合延迟：
   将构建系统集成到各种服务（本地化、报告、购物车、身份验证、数据库 API）的多项任务仍处于“待办事项”状态。这带来了跨不同服务广泛采用通用构建脚本的延迟风险。未能集成所有预期服务可能会限制项目的整体影响和价值。

2. 文档滞后：
   创建开发人员文档 (TKT-715) 的任务在长时间处于“待办事项”状态后，最近才转到“进行中”。此外，用户文档任务 (TKT-714) 仍处于“待办事项”状态。文档创建方面的滞后可能会阻碍新构建系统的采用、正确使用和维护。由于缺乏明确的指导，它可能会导致滥用、效率低下或拒绝采用。

3. 实施的复杂性：
   多项任务，特别是与声纳扫描 (TKT-697) 和配置扩展 (TKT-734) 相关的任务，已显示“进行中”和“审核”之间的多个状态变化。这种模式表明实施过程中存在潜在的技术挑战或不确定性。如果不妥善解决，这些复杂性可能会在未来导致错误、性能问题或维护困难。

如果管理和缓解不当，这些风险可能会影响项目的时间表、采用率和整体成功。

这真是出奇的好。我确实喜欢法学硕士除了提示数据之外还使用通用知识，因此它能够回答有关“静态代码分析工具”的问题，即使没有票证数据提到这些术语。答案有点过于冗长，需要对提示进行更多实验 - 可能向每个用户的查询附加说明将有助于形成更好的答案。
诸如误算门票之类的问题应该很容易解决，我们可以计算基本统计数据并将其包含在提示本身中。

以上是与 Jira 和 LLM 的互动项目报告的详细内容。更多信息请关注PHP中文网其他相关文章！