Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor grep functionality and enhance documentation tools across codebase #757

Merged
merged 22 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
9daa2b2
Refactor grep functionality and enhance documentation tools across co…
pelikhan Oct 7, 2024
d52641f
Refactor documentation querying and tool functionality, enhance loggi…
pelikhan Oct 7, 2024
e0893e3
Update description of system.agent_docs to clarify its querying capab…
pelikhan Oct 7, 2024
11bfc09
Add option to truncate text from the end in truncateTextToTokens func…
pelikhan Oct 7, 2024
e8c217b
Enhance token handling with new utilities for counting and truncating…
pelikhan Oct 7, 2024
cee33a6
Update script paths and descriptions, and revise git diff handling logic
pelikhan Oct 7, 2024
28346b7
Add maxTokens option and enhance tool usage for pull request reviewer
pelikhan Oct 7, 2024
8d7ad22
Update script references and improve error handling in tool calls
pelikhan Oct 7, 2024
8a8b7cc
Fix typos and add note on pull requests in GitHub agent documentation
pelikhan Oct 7, 2024
8228e4a
Fix typo by changing "globs" to "glob" in workspace.grep call
pelikhan Oct 7, 2024
7a00aa2
Remove excludedPaths in git diff config and update commit prompt with…
pelikhan Oct 7, 2024
a95d3ff
Add note about using the description for pull requests 📝
pelikhan Oct 7, 2024
bef76d7
Refactor error handling, update image model, and enhance logging 📦🛠️
pelikhan Oct 7, 2024
20a33c8
Update model references from gpt-4-turbo-v to gpt-4o throughout the p…
pelikhan Oct 7, 2024
63ed371
Remove debugger statement from cli.ts 🚀
pelikhan Oct 7, 2024
7ab8dcf
Remove `workspace grep` command and related functionality 🗑️
pelikhan Oct 7, 2024
877ea80
Add PR descriptor script for pull request summaries 📝✨
pelikhan Oct 7, 2024
6642f85
Add git branch default tool and enhance PR descriptor tools 🚀
pelikhan Oct 7, 2024
ea54423
Improve logging, add options handling, and fix typos in code and docs 📄🔧
pelikhan Oct 8, 2024
ab0d5ed
Update PR description generation: add GPT-4o model and increase token…
pelikhan Oct 8, 2024
9f925d3
Remove workspace grep tests from cli.test.ts 🧹
pelikhan Oct 8, 2024
0b2b74d
Update fs_diff_files description for clarity 📄✨
pelikhan Oct 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore.genai
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
.vscode/*
**/genaiscript.d.ts
**/yarn.lock
THIRD_PARTY_LICENSES.md
15 changes: 0 additions & 15 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,21 +32,6 @@
"showReuseMessage": false,
"clear": true
}
},
{
"label": "rv",
"type": "shell",
"command": "node packages/cli/built/genaiscript.cjs run rv",
"detail": "genai reviewer",
"problemMatcher": "$tsc",
"presentation": {
"echo": true,
"reveal": "always",
"focus": false,
"panel": "shared",
"showReuseMessage": false,
"clear": true
}
}
]
}
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ The quick brown fox jumps over the lazy dog.
Grep or fuzz search [files](https://microsoft.github.io/genaiscript/reference/scripts/files).

```js
const { files } = await workspace.grep(/[a-z][a-z0-9]+/, "**/*.md")
const { files } = await workspace.grep(/[a-z][a-z0-9]+/, { globs: "*.md" })
```

### LLM Tools
Expand Down
71 changes: 62 additions & 9 deletions docs/genaisrc/genaiscript.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/genaisrc/image-alt-text.genai.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
script({
title: "Image Alt Text generator",
description: "Generate alt text for images",
model: "openai:gpt-4-turbo-v",
model: "openai:gpt-4o",
group: "docs",
maxTokens: 4000,
temperature: 0,
Expand Down
1 change: 1 addition & 0 deletions docs/src/components/BuiltinAgents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { LinkCard } from '@astrojs/starlight/components';

### Builtin Agents

<LinkCard title="agent docs" description="query the documentation" href="/genaiscript/reference/scripts/system#systemagent_docs" />
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link text should be descriptive and unique for accessibility. "agent docs" is not descriptive enough.

generated by pr-docs-review-commit link_text

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link provided is incorrect; it should point to 'system.agent_docs' instead of 'system#systemagent_docs'.

generated by pr-docs-review-commit broken_link

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link provided for "agent docs" is incorrect or outdated.

generated by pr-docs-review-commit link_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new LinkCard was added without a corresponding description in the documentation.

generated by pr-docs-review-commit link_card_missing

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link for "agent docs" is missing a title attribute in the frontmatter.

generated by pr-docs-review-commit missing_link

<LinkCard title="agent fs" description="query files to accomplish tasks" href="/genaiscript/reference/scripts/system#systemagent_fs" />
<LinkCard title="agent git" description="query a repository using Git to accomplish tasks. Provide all the context information available to execute git queries." href="/genaiscript/reference/scripts/system#systemagent_git" />
<LinkCard title="agent github" description="query GitHub to accomplish tasks" href="/genaiscript/reference/scripts/system#systemagent_github" />
Expand Down
2 changes: 2 additions & 0 deletions docs/src/components/BuiltinTools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { LinkCard } from '@astrojs/starlight/components';
<LinkCard title="fs_diff_files" description="Computes a diff between two files." href="/genaiscript/reference/scripts/system#systemfs_diff_files" />
<LinkCard title="fs_find_files" description="Finds file matching a glob pattern. Use pattern to specify a regular expression to search for in the file content." href="/genaiscript/reference/scripts/system#systemfs_find_files" />
<LinkCard title="fs_read_file" description="Reads a file as text from the file system. Returns undefined if the file does not exist." href="/genaiscript/reference/scripts/system#systemfs_read_file" />
<LinkCard title="git_branch_default" description="Gets the default branch using git." href="/genaiscript/reference/scripts/system#systemgit" />
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new LinkCard was added without a corresponding description in the documentation.

generated by pr-docs-review-commit link_card_missing

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link for "git_branch_default" is missing a title attribute in the frontmatter.

generated by pr-docs-review-commit missing_link

<LinkCard title="git_branch_current" description="Gets the current branch using git." href="/genaiscript/reference/scripts/system#systemgit" />
<LinkCard title="git_branch_list" description="List all branches using git." href="/genaiscript/reference/scripts/system#systemgit" />
<LinkCard title="git_diff" description="Computes file diffs using the git diff command. If the diff is too large, it returns the list of modified/added files." href="/genaiscript/reference/scripts/system#systemgit" />
Expand All @@ -28,6 +29,7 @@ import { LinkCard } from '@astrojs/starlight/components';
<LinkCard title="github_pulls_get" description="Get a single pull request by number." href="/genaiscript/reference/scripts/system#systemgithub_pulls" />
<LinkCard title="github_pulls_review_comments_list" description="Get review comments for a pull request." href="/genaiscript/reference/scripts/system#systemgithub_pulls" />
<LinkCard title="math_eval" description="Evaluates a math expression" href="/genaiscript/reference/scripts/system#systemmath" />
<LinkCard title="md_find_files" description="Get the file structure of the documentation markdown/MDX files. Retursn filename, title, description for each match. Use pattern to specify a regular expression to search for in the file content." href="/genaiscript/reference/scripts/system#systemmd_find_files" />
pelikhan marked this conversation as resolved.
Show resolved Hide resolved
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new LinkCard was added without a corresponding description in the documentation.

generated by pr-docs-review-commit link_card_missing

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link for "md_find_files" is missing a title attribute in the frontmatter.

generated by pr-docs-review-commit missing_link

<LinkCard title="md_read_frontmatter" description="Reads the frontmatter of a markdown or MDX file." href="/genaiscript/reference/scripts/system#systemmd_frontmatter" />
<LinkCard title="python_code_interpreter_run" description="Executes python 3.12 code for Data Analysis tasks in a docker container. The process output is returned. Do not generate visualizations. The only packages available are numpy, pandas, scipy. There is NO network connectivity. Do not attempt to install other packages or make web requests." href="/genaiscript/reference/scripts/system#systempython_code_interpreter" />
<LinkCard title="python_code_interpreter_copy_files" description="Copy files from the host file system to the container file system" href="/genaiscript/reference/scripts/system#systempython_code_interpreter" />
Expand Down
6 changes: 3 additions & 3 deletions docs/src/components/CreateScript.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@ description: Learn how to create a new script using Visual Studio Code or other
keywords: create script, VSCode, command palette, CLI, script generation
---

import { Tabs, TabItem } from '@astrojs/starlight/components';
import { Tabs, TabItem } from "@astrojs/starlight/components"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect use of double quotes for import statement, should use single quotes.

generated by pr-docs-review-commit syntax_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mismatched quotes in import statement.

generated by pr-docs-review-commit quote_mismatch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing semicolon at the end of the import statement.

generated by pr-docs-review-commit missing_semi

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The import statement for Tabs and TabItem has been changed, which may affect the rendering of components.

generated by pr-docs-review-commit import_statement_changed


<Tabs>
<TabItem label="Visual Studio Code, Cursor">

Use the `> GenAiScript: Create new script...` command in the [command palette](https://code.visualstudio.com/docs/getstarted/userinterface#_command-palette)
Use the `> GenAiScript: Create new script...` command in the [command palette](https://code.visualstudio.com/docs/getstarted/userinterface#_command-palette)
(`Ctrl+Shift+P` on Windows/Linux, `⇧⌘P` on Mac)
to create a new script.

![The command palette and the create script command](../assets/vscode-create-new-script.png)
![A command palette with a search bar showing ">createn" and a suggestion below for "GenAIScript: Create new script..." highlighted in blue.](../assets/vscode-create-new-script.png)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alt text for the image should be descriptive. The current text is too verbose and not succinct.

generated by pr-docs-review-commit alt_text


</TabItem>
<TabItem label="Other Editors">
Expand Down
5 changes: 2 additions & 3 deletions docs/src/content/docs/blog/drafts/code-review-with-genai.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,12 @@ tags: ["code review", "development", "programming", "automation"]
authors: genaiscript
canonical_url: https://microsoft.github.io/genaiscript/blog/code-review-with-genai
description: Dive into automated code review processes with GenAI, enhancing
developer efficiency and code quality.

developer efficiency and code quality.
---

## Introducing "Code Review with GenAI" 🧐

Have you ever wished for an extra set of eyes while coding? Well, GenAI has got your back! Let's delve into the "Reviewer" script, which automates the code review process, making it a breeze for developers. This powerful script for the GenAIScript platform can be found [here on GitHub](https://github.com/microsoft/genaiscript/blob/main/packages/vscode/genaisrc/rv.genai.mts).
Have you ever wished for an extra set of eyes while coding? Well, GenAI has got your back! Let's delve into the "Reviewer" script, which automates the code review process, making it a breeze for developers. This powerful script for the GenAIScript platform can be found [here on GitHub](https://github.com/microsoft/genaiscript/blob/main/packages/vscode/genaisrc/prr.genai.mts).
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link to the "Reviewer" script is incorrect or outdated; it should point to "prr.genai.mts" instead of "rv.genai.mts".

generated by pr-docs-review-commit link_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link text "here on GitHub" points to an incorrect URL; it should be updated to match the new script name "prr.genai.mts" instead of "rv.genai.mts".

generated by pr-docs-review-commit incorrect_link

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link provided for the "Reviewer" script is incorrect; it should point to "prr.genai.mts" instead of "rv.genai.mts".

generated by pr-docs-review-commit broken_link


### What is the Script About?

Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/docs/guides/ask-my-image.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The quick-start guide illustrates how to write a GenAIScript that takes input fr
```js
script({
title: "Apply a script to an image",
model: "openai:gpt-4-turbo-v",
model: "openai:gpt-4o",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name 'openai:gpt-4o' is incorrect; it should be 'openai:gpt-4-turbo-v'.

generated by pr-docs-review-commit model_name_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model ID has been updated, ensure that the documentation reflects the correct model ID.

generated by pr-docs-review-commit model_id

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model in the script header has been updated, which may affect the script's behavior with images.

generated by pr-docs-review-commit model_update

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model ID "openai:gpt-4o" should be consistent with the rest of the documentation. It appears to be incorrect or outdated.

generated by pr-docs-review-commit model_id_mismatch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name has been updated and should be reflected in the documentation.

generated by pr-docs-review-commit model_name_update

})
```
4. Use [defImages](/genaiscript/reference/scripts/images/) to ingest the image file into the model context:
Expand Down
4 changes: 2 additions & 2 deletions docs/src/content/docs/guides/search-and-transform.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,9 @@ that allows to efficiently search for a pattern in files (this is the same searc
that powers the Visual Studio Code search).

```js "workspace.grep"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable name 'globs' should be 'glob' to match the property name used in the 'workspace.grep' method.

generated by pr-docs-review-commit variable_name_mismatch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable 'globs' is incorrect; it should be 'glob' as used in the workspace.grep function.

generated by pr-docs-review-commit incorrect_variable

const { pattern, glob } = env.vars
const { pattern, globs } = env.vars
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable 'glob' has been renamed to 'globs', which may affect the script's functionality.

generated by pr-docs-review-commit variable_renamed

const patternRx = new RegExp(pattern, "g")
const { files } = await workspace.grep(patternRx, glob)
const { files } = await workspace.grep(patternRx, { globs })
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect API usage, 'workspace.grep' should be called with 'glob' instead of 'globs'.

generated by pr-docs-review-commit api_usage

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace.grep' function is incorrectly called with an object instead of separate arguments.

generated by pr-docs-review-commit incorrect_function_usage

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace.grep' function is called with an object containing 'globs' instead of a string 'glob' parameter.

generated by pr-docs-review-commit incorrect_argument

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace.grep' function is incorrectly called with an object instead of separate arguments.

generated by pr-docs-review-commit api_usage_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage of the 'workspace.grep' method has changed, which may affect the script's functionality.

generated by pr-docs-review-commit method_usage_change

```
pelikhan marked this conversation as resolved.
Show resolved Hide resolved

## Compute Transforms
Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ The quick brown fox jumps over the lazy dog.
Grep or fuzz search [files](/genaiscript/referen/script/files)

```js wrap
const { files } = await workspace.grep(/[a-z][a-z0-9]+/, "**/*.md")
const { files } = await workspace.grep(/[a-z][a-z0-9]+/, { globs: "*.md" })
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect API usage, 'workspace.grep' should be called with 'glob' instead of 'globs'.

generated by pr-docs-review-commit api_usage

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace.grep' function is incorrectly called with an object instead of separate arguments.

generated by pr-docs-review-commit incorrect_function_usage

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter for 'workspace.grep' should be 'glob' instead of 'globs'.

generated by pr-docs-review-commit parameter_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace.grep' function is called with an object containing 'globs' instead of a string 'glob' parameter.

generated by pr-docs-review-commit incorrect_argument

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace.grep' function is incorrectly called with an object instead of separate arguments.

generated by pr-docs-review-commit api_usage_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage of the 'workspace.grep' method has changed, which may affect the script's functionality.

generated by pr-docs-review-commit method_usage_change

```
pelikhan marked this conversation as resolved.
Show resolved Hide resolved
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The structure of the parameter passed to 'workspace.grep' has changed from a string to an object with a 'globs' property. This change needs to be reflected in all relevant code snippets.

generated by pr-docs-review-commit parameter_structure_change


</Card>
Expand Down
24 changes: 0 additions & 24 deletions docs/src/content/docs/reference/cli/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,30 +407,6 @@ Options:
-h, --help display help for command
```

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace' command and its subcommands have been removed from the documentation, which suggests they are no longer supported.

generated by pr-docs-review-commit removed_feature

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire section for the workspace command has been removed. This change should be documented properly to inform users about the removal.

generated by pr-docs-review-commit removed_section

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace' command and its subcommands have been removed from the CLI documentation.

generated by pr-docs-review-commit command_removed

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "workspace" command section is deprecated and should be removed from the documentation.

generated by pr-docs-review-commit deprecated_section

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'workspace' command section is deprecated and should be removed from the documentation.

generated by pr-docs-review-commit deprecated_section

## `workspace`

```
Usage: genaiscript workspace [options] [command]

Workspace tasks

Options:
-h, --help display help for command

Commands:
grep <pattern> [files...]
help [command] display help for command
```

### `workspace grep`

```
Usage: genaiscript workspace grep [options] <pattern> [files...]

Options:
-h, --help display help for command
```

## `info`

```
Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/docs/reference/scripts/images.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ sidebar:
order: 10
---

Images can be added to the prompt for models that support this feature (like `gpt-4-turbo-v`).
Images can be added to the prompt for models that support this feature (like `gpt-4o`).
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name 'gpt-4o' is incorrect; it should be 'gpt-4-turbo-v'.

generated by pr-docs-review-commit model_name_error

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model ID has been updated, ensure that the documentation reflects the correct model ID.

generated by pr-docs-review-commit model_id

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model in the documentation has been updated, which may affect the script's behavior with images.

generated by pr-docs-review-commit model_update

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model ID "openai:gpt-4o" should be consistent with the rest of the documentation. It appears to be incorrect or outdated.

generated by pr-docs-review-commit model_id_mismatch

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name has been updated and should be reflected in the documentation.

generated by pr-docs-review-commit model_name_update

Use the `defImages` function to declare the images. Supported images will vary
with models but typically include `PNG`, `JPEG`, `WEBP`, and `GIF`. Both local files and URLs are supported.

Expand Down
Loading
Loading