diff --git a/.gitignore b/.gitignore index 8c9c1fcf9f253046c2067756e320566e85c3f8f0..753f3431d1698c58226483087102f4a343adf283 100644 --- a/.gitignore +++ b/.gitignore @@ -14,6 +14,7 @@ __pycache__ # Build venv/* +site/* # ACMC phenotype build files diff --git a/docs/index.md b/docs/index.md index 85db380914afbfd25c9795dda9145392a03cad68..ae1a3478131d698e6083c46fa053bbe8f7c83bca 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,18 +1,19 @@ -## Overview +# ACMC Documentation -### Workflow +## Contents + - [Installation](./installation.md) + - [Usage](./usage.md) + - Tutorials + - [Example 1 - Basic local phenotype](./tutorials/example1.md) + - [Example 2 - More complex local phenotype](./tutorials/example2.md) + - [Example 3 - Using a remote git repository](./tutorials/example3.md) + - [API Reference](./api.md) + - [Contributing](./contributing.md) + - [Change Log](./changelog.md) + - [Troubleshooting](./troubleshooting.md) -The high level steps to use the tools are outlined below: - - -**1. Define concept sets:** A domain expert defines a list of [concept sets](#defining-concept-sets) for each observable characteristic of the phenotype using CSV file format (e.g., `PHEN_concept_sets.csv`). - -**2. Define concept code lists for concept sets:** A domain expert defines [code lists](#defining-concept-codes) for each concept set within the phenotype using supported coding list formats and stores them in the `/src` directory. - -**3. Define mapping from code lists to concept sets:** A domain expert defines a [phenotype mapping](#mapping-codes-to-concept-sets) that maps code lists to concept sets. - -**4. Generate versioned phenotype coding lists and translations:** A domain expert or data engineer processes the phenotype mappings [using the command line tool](#usage) to validate against NHS TRUD-registered codes and mapping and to generate versioned concept set code lists with translations between coding standards. +### Overview ### Supported Medical Coding Standards @@ -40,347 +41,3 @@ The tool supports verification and mapping across diagnostic coding formats belo -## Usage - -The `acmc` command-line tool provides various commands to interact with TRUD, OMOP, and Phenotype data. Below are the usage details for each command. - -### General Syntax - -```bash -acmc [OPTIONS] COMMAND [SUBCOMMAND] [ARGUMENTS] -``` - -Where: -- `[OPTIONS]` are global options that apply to all commands (e.g., `--debug`, `--version`). -- `[COMMAND]` is the top-level command (e.g., `trud`, `omop`, `phen`). -- `[SUBCOMMAND]` refers to the specific operation within the command (e.g., `install`, `validate`). - -### Global Options - -- `--version`: Display the acmc tool version number -- `--debug`: Enable debug mode for more verbose logging. - -### Commands - -#### TRUD Command - -The `trud` command is used for installing NHS TRUD vocabularies. - -- **Install TRUD** - - Install clinically assurred TRUD medical code mappings: - - ```bash - acmc trud install - ``` - -#### OMOP Command - -The `omop` command is used for installing OMOP vocabularies. - -- **Install OMOP** - - Install vocabularies in a local OMOP database: - - ```bash - acmc omop install -d <OMOP_DIRECTORY_PATH> -v <OMOP_VERSION> - ``` - - - `-d`, `--omop-dir`: (Optional) Directory path to extracted OMOP downloads, default is `./build/omop` - - `-v`, `--version`: OMOP vocabularies release version. - -- **Clear OMOP** - - Clear data from the local OMOP database: - - ```bash - acmc omop clear - ``` - -- **Delete OMOP** - - Delete the local OMOP database: - - ```bash - acmc omop delete - ``` - -#### PHEN Command - -The `phen` command is used phenotype-related operations. - -- **Initialize Phenotype** - - Initialize a phenotype directory locally or from a remote git repository: - - ```bash - acmc phen init -d <PHENOTYPE_DIRECTORY> -r <REMOTE_URL> - ``` - - - `-d`, `--phen-dir`: (Optional) Directory to write phenotype configuration (the default is ./build/phen). - - `-r`, `--remote_url`: (Optional) URL to a remote git repository. - -- **Validate Phenotype** - - Validate the phenotype configuration: - - ```bash - acmc phen validate -d <PHENOTYPE_DIRECTORY> - ``` - - - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen). - -- **Map Phenotype** - - Process phenotype mapping and specify the target coding and output format: - - ```bash - acmc phen map -d <PHENOTYPE_DIRECTORY> -t <TARGET_CODING> -o <OUTPUT_FORMAT> - ``` - - - `-t`, `--target-coding`: Specify the target coding (e.g., `read2`, `read3`, `icd10`, `snomed`, `opcs4`). - - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen). - - `-o`, `--output`: Output format(s) (`csv`, `omop`, or both), default is 'csv'. - -- **Publish Phenotype Configuration** - - Publish a phenotype configuration, committing all changes and tagging with a new version number. If the phenotype has been initialised from a remote git URL, then the commit and new version tag will be pushed to the remote repo: - - ```bash - acmc phen publish -d <PHENOTYPE_DIRECTORY> - ``` - - - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen). - -- **Copy Phenotype Configuration** - - Copy a phenotype configuration from a source directory to a target directory at a specific version. This is used when wanting to compare versions of phenotypes using the `acmc phen diff` command: - - ```bash - acmc phen copy -d <PHENOTYPE_DIRECTORY> -td <TARGET_DIRECTORY> -v <PHENOTYPE_VERSION> - ``` - - - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen). - - `-td`, `--target-dir`: (Optional) Directory to copy the phenotype configuration to, (the default is ./build). - - `-v`, `--version`: The phenotype version to copy. - -- **Compare Phenotype Configurations** - - Compare a a new phenotype version with pervious version of a phenotype: - - ```bash - acmc phen diff -d <NEW_PHENOTYPE_DIRECTORY> -old <OLD_PHENOTYPE_DIRECTORY> - ``` - - - `-d`, `--phen-dir`: (Optional) Directory of current phenotype configuration (the default is ./build/phen). - - `-old`, `--phen-dir-old`: (Required) Directory of old phenotype version) - -# Development Guide - -## Prerequisites -To install the ACMC package for development, ensure you have Python 3.9 or later installed. - -### Clone the Repository -```sh -git clone https://git.soton.ac.uk/meldb/concepts-processing.git -cd concepts-processing -``` - -### Set Up a Virtual Environment -It is recommended to use a virtual environment to manage dependencies. - -```sh -python -m venv venv -source venv/bin/activate # On Windows use `venv\Scripts\activate` -``` - -### Install Dependencies -Using `hatch`, you can install the necessary dependencies easily. - -```sh -pip install --upgrade pip -pip install hatch -``` -This will create a virtual environment with all required dependencies. - -## Development Environment - -We have two separate environments to ensure that development dependencies (such as testing tools, linters, or other utilities) don't interfere the production environment. - -- default environment: includes the core dependencies to run acmc (e.g., requests, etc.). -- dev environment: includes additional tools for testing, code formatting, linting, and other development workflows (e.g., pytest, black, mypy, etc.). - -The development toos used include: - -- pytest: testing -- mypy: type checking -- black: code formatting - -### Activate the Development Environment -To enter the (dev) development environment, use: - -```sh -hatch shell dev -``` - -### Activate Production Environment -To enter the (acmc) production environment, use: - -```sh -hatch shell -``` - -### Deactivate Environment -To exit an environment from hatch, use: - -```sh -exit -``` - -### Running Tests -To run tests using `pytest`, use: - -```sh -hatch run pytest -``` - -### All code checks -The project run all type and formatting checking - -```sh -hatch run check -``` - -### Code Formatting -The project uses `black` for code formatting. Ensure your code is properly formatted before committing - -To check if any of the code needs to be formatted, run black with the `--check` option - -```sh -hatch run black --check . -``` - -To format the coode and modify the files, use - -```sh -hatch run black . -``` - -### Type Checking -The project uses `mypy` for type checking: - -```sh -hatch run mypy . -``` - -## Building the Package -To build the package, use: - -```sh -hatch build -``` - -## Running the CLI -The package provides a command-line interface (CLI) entry point. - -```sh -acmc --help -``` - -## Contributing - -### GitLab Basic Workflow Overview - -1. Create an new issue in the [Issue Tracker](https://git.soton.ac.uk/meldb/concepts-processing/-/issues) - -2. Create a new branch for your feature - -You can do this from the issue page in GitLab by selecting "Create Branch", then checkout using: - - ```sh - git pull origin - git checkout -b feature-branch origin/feature-branch - ``` - -This works if someone has already created the branch and you want to work on it. - -You can also create a new branch in your local repo if it does not exist and it will be created when you push - - ```sh - git checkout -b feature-branch - ``` - -4. Make changes and commit them: - ```sh - git add . - git commit -m "Description of changes" - ``` -5. Push your branch: - ```sh - git push origin feature-branch - ``` -6. Open a merge request in GitLab. - -If you make changes, ensure all tests pass and code formatting is correct before submitting a merge request. - -## GitLab Workflow for Managing Releases - -### 1. Main Branch (`main`) -- Represents the stable production-ready code. -- Contains only release versions. -- Direct commits are not allowed; changes must go through a pull request (PR) from `dev`. -- Merges into `main` require authorization from a repository maintainer or admin. -- Protected branch settings should enforce code reviews and approvals before merging. - -### 2. Development Branch (`dev`) -- Used for integration and testing. -- All feature branches must be merged here first before reaching `main`. -- Regularly synced with `main` to keep it up to date. - -### 3. Feature Branches (`feature-xxx`) -- Created for each issue or feature. -- Always branch from `dev` based on an issue. -- Named according to the issue number and short description (e.g., `feature-123-user-auth`). -- Can be created directly from the issue page in GitLab by selecting "Create Branch". -- Merged into `dev` through a pull request. - -## Pull Request Sequence - -1. **Create a Feature Branch** - - Branch from `dev`: `git checkout -b feature-xxx dev` - - Alternatively, create the branch from the issue itself in GitLab. - - Implement feature and commit changes following commit message guidelines. - -2. **Push the Branch** - - `git push origin feature-xxx` - -3. **Create a Merge Request (MR) to `dev`** - - Ensure code review and approvals before merging. - - Resolve conflicts if any. - -4. **Merge to `dev`** - - Squash and merge when approved. - - Delete feature branch after successful merge. - -5. **Release to `main`** - - Create a merge request from `dev` to `main`. - - Perform final testing. - - Tag the release with versioning (e.g., `v1.0.0`). - - Deploy to production after approval. - - Requires authorization from a repository maintainer or admin before merging. - -### Commit Message Guidelines -- Use clear and descriptive commit messages. -- Format: `<type>(<scope>): <description>` -- Example: `docs: git workflow documentation (#issue)` -- Types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`. - -## Additional Guidelines -- Always pull the latest changes before starting a new feature: `git pull origin dev` -- Write unit tests for new features. -- Keep feature branches short-lived and regularly updated with `dev`. - -## Additional Resources -- [Repository](https://git.soton.ac.uk/meldb/concepts-processing) -- [Documentation](https://git.soton.ac.uk/meldb/concepts-processing/docs) -- [Issue Tracker](https://git.soton.ac.uk/meldb/concepts-processing/-/issues) \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index 4e861658f7a7638d8aad78304f561f217b88eb3a..04e94938abd58743b2cbb1c7a1c8e5616a9f937d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -65,11 +65,17 @@ dependencies = [ "pydocstyle", "pytest", "black", - "mypy" + "mypy", + "mkdocs", + "mkdocs-material", + "mkdocstrings" ] -[tool.hatch.envs.default.scripts] +[tool.hatch.envs.dev.scripts] check = "black . && mypy ." +docs = "sphinx-build -b html docs/source docs/build" +mkdocs = "mkdocs build" +pdoc = "pdoc --html acmc -o docs --force" [tool.hatch.build] include = ["acmc/**"] # Ensure only the acmc package is included