DIQ (Document Indexer and Querier)

diq is a CLI tool for searching, indexing, and querying metadata within documents, based on the specification in DESIGN.md.

Current Status

This project is in the initial setup phase. The repository structure and basic tooling have been configured.

Development Notes

• DAL Parsing: For the initial implementation, we will use regular expressions for parsing the DIQ Annotation Language (DAL). The use of a more robust parsing library like lark is deferred to a future date.
• Database: The index database will be managed with direct SQL queries using Python's built-in sqlite3 module. An ORM like SQLAlchemy is not being used at this stage.

Setup

To set up the development environment:

# Create a virtual environment & install dependencies
uv sync

# Activate the virtual environment
source .venv/bin/activate

Code Quality

To ensure code quality, run the following checks before committing:

# Lint and format the code
uv run ruff check . --fix
uv run ruff format .

# Run static type checking
uv run mypy .

Usage

Parse YAML frontmatter from a Markdown file and output JSON:

uv run diq frontmatter tests/data/frontmatter/test1.md

List Markdown files with frontmatter (file or directory path, default is .):

uv run diq list tests/data/frontmatter

Include Markdown files that do not have frontmatter:

uv run diq list tests/data/frontmatter --include-no-frontmatter

Attribute search mode (key=value semantics, optional PATH):

uv run diq search tests/data/frontmatter name="Main Requirements Document"

Tag similarity mode (optional --take):

uv run diq search tests/data/frontmatter --tags-similar tag1,tag2 --take 1

CLI changes

• When making changes to the cli, add or update the README.md with succinct usage examples.