diff --git a/.gitignore b/.gitignore
index 753f3431d1698c58226483087102f4a343adf283..e385af1d1f474f6dec80537cd2136e03b6bf3c68 100644
--- a/.gitignore
+++ b/.gitignore
@@ -23,3 +23,4 @@ workspace/*
v[0-9]*.[0-9]*.[0-9]*/
*output*
*.log
+.acmc/*
diff --git a/README.md b/README.md
index 5dfb6cce5e92e99ced354cc462488fb9bfd6934d..432e728a3cae2d779273648df78716792d9bbf86 100644
--- a/README.md
+++ b/README.md
@@ -18,7 +18,7 @@
## Introduction
-This tool automates the verification, translation and organisation of medical coding lists defining phenotypes for inclusion criteria in cohort analysis. By processing externally sourced clinical inclusion criteria into actionable code lists, this tool ensures consistent and efficient curation of cohort definitions. These code lists can be subsequently used by data providers to construct study cohorts.
+ACMC is a tool that automates the verification, translation and organisation of medical coding lists defining phenotypes for inclusion criteria in cohort analysis. By processing externally sourced clinical inclusion criteria into actionable code lists, this tool ensures consistent and efficient curation of cohort definitions. These code lists can be subsequently used by data providers to construct study cohorts.
## Requirements
@@ -178,7 +178,7 @@ Expected Output:
From the command prompt, copy medical code lists `/examples/codes`to the phenotype code directory:
```bash
-cp -r ./examples/codes/* ./workspace/phen/codes
+cp -r ./examples/concepts/* ./workspace/phen/concepts
```
- You can view the source code list here [`res176-abdominal-pain.csv`](.//examples/codes/clinical-codes-org/Symptom%20code%20lists/Abdominal%20pain/res176-abdominal-pain.csv)
@@ -224,9 +224,10 @@ Expected Output:
[INFO] - Processing phenotype: <path>/concepts-processing/workspace/phen
[INFO] - Validating phenotype: <path>/concepts-processing/workspace/phen
[INFO] - Phenotype validated successfully
-[INFO] - Processing read2 codes...
+[INFO] - Processing read2 codes for <path>/concepts-processing/workspace/phen/concepts/clinical-codes-org/Symptom code lists/Abdominal pain/res176-abdominal-pain.csv
[INFO] - Converting to target code type read2
[INFO] - Saved mapped concepts to <path>/concepts-processing/workspace/phen/map/read2.csv
+[INFO] - Phenotype processed target code type read2
[INFO] - Phenotype processed successfully
```
@@ -243,18 +244,18 @@ acmc phen publish
Expected Output:
```bash
-[INFO] - Validating phenotype: /home/mjbonifa/datahdd/brcbat/derived_datasets/mjbonifa/concepts-processing/workspace/phen
+[INFO] - Validating phenotype: <path>/concepts-processing/workspace/phen
[INFO] - Phenotype validated successfully
-[INFO] - New version: v1.0.3
+[INFO] - New version: 0.0.1
[INFO] - Phenotype published successfully
```
-7. **Generate phenotype in SNOWMED code format**
+7. **Generate phenotype in Read3 code format**
-Generate the phenotype in `snomed` format:
+Generate the phenotype in `read3` format:
```bash
-acmc phen map -t snomed
+acmc phen map -t read3
```
Expected Output:
@@ -263,40 +264,21 @@ Expected Output:
[INFO] - Processing phenotype: <path>/concepts-processing/workspace/phen
[INFO] - Validating phenotype: <path>/concepts-processing/workspace/phen
[INFO] - Phenotype validated successfully
-[INFO] - Processing read2 codes...
-[INFO] - Converting to target code type snomed
-[INFO] - Saved mapped concepts to <path>/concepts-processing/workspace/phen/map/snomed.csv
+[INFO] - Processing read2 codes for <path>/concepts-processing/workspace/phen/concepts/clinical-codes-org/Symptom code lists/Abdominal pain/res176-abdominal-pain.csv
+[INFO] - Converting to target code type read3
+[INFO] - Saved mapped concepts to <path>/concepts-processing/workspace/phen/map/read3.csv
+[INFO] - Phenotype processed target code type read3
[INFO] - Phenotype processed successfully
```
The concept sets translating read2 to snomed will be stored in acmc CSV format in `./workspace/phen/concept-set/snomed/`, e.g. `./workspace/phen/concept-set/snomed/ABDO_PAIN.csv`
-8. **Get a copy of the previous version from the repo**
-
- Use the following `acmc` command to retrieve a copy of the previous version (`v1.0.3`) from the repository:
-
-```bash
-acmc phen copy -v v1.0.3
-```
-
-Expected Output:
-
-```bash
-[INFO] - Validating phenotype: <path>/concepts-processing/workspace/phen
-[INFO] - Phenotype validated successfully
-[INFO] - Copying repo <path>/concepts-processing/workspace/phen to <path>/concepts-processing/workspace/v1.0.3
-[INFO] - Checking out version v1.0.3...
-[INFO] - Phenotype copied successfully
-```
-
-A copy of the phenotype will be created in the directory `./workspace/v1.0.3`
+8. **Compare the previous version `0.0.1` with the latest version**
-9. **Compare the previous version `v1.0.3` with the latest version**
-
- Use the following `acmc` command to compare the previous version `v1.0.3` with the latest version in the repository:
+ Use the following `acmc` command to compare the previous version `0.0.1` with the latest version in the workspace phen directory:
```bash
-acmc phen diff -old ./workspace/v1.0.3/
+acmc phen diff -ov 0.0.1
```
Expected Output:
@@ -309,14 +291,14 @@ Expected Output:
[INFO] - Phenotypes diff'd successfully
```
-A report comparing the phenotype versions will be created in the workspace called './workspace/phen/v1.0.3_diff.md'
+A report comparing the phenotype versions will be created in the workspace called './workspace/phen/latest_0.0.1_diff.md'
-10. **Publish the phenotype at the next version**
+9. **Publish the phenotype at a major version**
- Use the following `acmc` command to publish the phenotype at the next version:
+ Use the following `acmc` command to publish the phenotype at a major version:
```bash
-acmc phen publish
+acmc phen publish -i major
```
Expected Output:
@@ -324,7 +306,7 @@ Expected Output:
```bash
[INFO] - Validating phenotype: /home/mjbonifa/datahdd/brcbat/derived_datasets/mjbonifa/concepts-processing/workspace/phen
[INFO] - Phenotype validated successfully
-[INFO] - New version: v1.0.4
+[INFO] - New version: 1.0.0
[INFO] - Phenotype published successfully
```
diff --git a/acmc/logging_config.py b/acmc/logging_config.py
index 735365a38a5ac3d653f2ea04d48c93a94d7815ba..7257d9e0d8fe828a93bed536567979c118f76fb4 100644
--- a/acmc/logging_config.py
+++ b/acmc/logging_config.py
@@ -1,24 +1,20 @@
+"""
+logging_config.py
+
+This module defines functions to setup logging for acmc across all module.
+
+"""
+
import pandas as pd
import logging
DEFAULT_LOG_FILE = "acmc.log"
+"""The default acmc application log filename."""
-# TODO: Determine if bcolours is still needed considering use of logging not print
-class bcolors: # for printing coloured text
- HEADER = "\033[95m"
- OKBLUE = "\033[94m"
- OKCYAN = "\033[96m"
- OKGREEN = "\033[92m"
- WARNING = "\033[93m"
- FAIL = "\033[91m"
- ENDC = "\033[0m"
- BOLD = "\033[1m"
- UNDERLINE = "\033[4m"
-
+def setup_logger(log_level: int = logging.INFO):
+ """Sets up acmc logger as a singleton outputing to file and sysout syserr."""
-def setup_logger(log_level=logging.INFO):
- """Sets up logger as a singleton outputing to file and sysout syserr"""
# Create a logger
logger = logging.getLogger("acmc_logger")
logger.setLevel(logging.INFO)
@@ -33,7 +29,6 @@ def setup_logger(log_level=logging.INFO):
stream_handler.setLevel(logging.INFO)
# Create a formatter for how the log messages should look
-
# Add the formatter to both handlers
file_formatter = logging.Formatter(
"%(asctime)s - - %(levelname)s - %(message)s"
@@ -49,10 +44,16 @@ def setup_logger(log_level=logging.INFO):
return logger
-def set_log_level(log_level):
- """Sets the log level for the acmc logger"""
+def set_log_level(log_level: int):
+ """Sets the log level for the acmc logger.
+
+ Args:
+ log_level (int): log level from the python logging libraru
+
+ """
+
logger = logging.getLogger("acmc_logger")
- logger.setLevel(log_level) # Set logger level
+ logger.setLevel(log_level)
# Also update handlers to match the new level
for handler in logger.handlers:
diff --git a/acmc/main.py b/acmc/main.py
index 89df65dda2d1da11eef38260f2d760a6bbf9846c..4b7ac6ca7afa07cb5331ec9fe2628afcbd968ffe 100644
--- a/acmc/main.py
+++ b/acmc/main.py
@@ -1,69 +1,84 @@
+"""
+main.py module
+
+This module provides the functionality for the acmc command line interface
+"""
+
import argparse
import logging
from pathlib import Path
import acmc
-from acmc import trud, omop, phen, logging_config as lc
+from acmc import trud, omop, phen, parse, logging_config as lc
-# setup logging
-logger = lc.setup_logger()
-DEFAULT_WORKING_PATH = Path("./workspace")
+DEFAULT_WORKSPACE_PATH = Path("./workspace")
+"""Default phenotype workspace path"""
-def trud_install(args):
+def _trud_install(args: argparse.Namespace):
"""Handle the `trud install` command."""
trud.install()
-def omop_install(args):
+def _omop_install(args: argparse.Namespace):
"""Handle the `omop install` command."""
omop.install(args.omop_zip_file, args.version)
-def omop_clear(args):
+def _omop_clear(args: argparse.Namespace):
"""Handle the `omop clear` command."""
omop.clear(omop.DB_PATH)
-def omop_delete(args):
+def _omop_delete(args: argparse.Namespace):
"""Handle the `omop delete` command."""
omop.delete(omop.DB_PATH)
-def phen_init(args):
+def _phen_init(args: argparse.Namespace):
"""Handle the `phen init` command."""
phen.init(args.phen_dir, args.remote_url)
-def phen_validate(args):
+def _phen_fork(args: argparse.Namespace):
+ """Handle the `phen fork` command."""
+ phen.fork(
+ args.phen_dir,
+ args.upstream_url,
+ args.upstream_version,
+ new_origin_url=args.remote_url,
+ )
+
+
+def _phen_validate(args: argparse.Namespace):
"""Handle the `phen validate` command."""
phen.validate(args.phen_dir)
-def phen_map(args):
+def _phen_map(args: argparse.Namespace):
"""Handle the `phen map` command."""
phen.map(args.phen_dir, args.target_coding)
-def phen_export(args):
+def _phen_export(args: argparse.Namespace):
"""Handle the `phen copy` command."""
phen.export(args.phen_dir, args.version)
-def phen_publish(args):
+def _phen_publish(args: argparse.Namespace):
"""Handle the `phen publish` command."""
- phen.publish(args.phen_dir)
+ phen.publish(args.phen_dir, args.msg, args.remote_url, args.increment)
-def phen_copy(args):
+def _phen_copy(args: argparse.Namespace):
"""Handle the `phen copy` command."""
phen.copy(args.phen_dir, args.target_dir, args.version)
-def phen_diff(args):
+def _phen_diff(args: argparse.Namespace):
"""Handle the `phen diff` command."""
- phen.diff(args.phen_dir, args.phen_dir_old)
+ phen.diff(args.phen_dir, args.version, args.old_phen_dir, args.old_version)
def main():
@@ -88,7 +103,7 @@ def main():
trud_install_parser = trud_subparsers.add_parser(
"install", help="Install TRUD components"
)
- trud_install_parser.set_defaults(func=trud_install)
+ trud_install_parser.set_defaults(func=_trud_install)
### OMOP Command ###
omop_parser = subparsers.add_parser("omop", help="OMOP commands")
@@ -106,19 +121,19 @@ def main():
omop_install_parser.add_argument(
"-v", "--version", required=True, help="OMOP vocabularies release version"
)
- omop_install_parser.set_defaults(func=omop_install)
+ omop_install_parser.set_defaults(func=_omop_install)
# omop clear
omop_clear_parser = omop_subparsers.add_parser(
"clear", help="Clear OMOP data from database"
)
- omop_clear_parser.set_defaults(func=omop_clear)
+ omop_clear_parser.set_defaults(func=_omop_clear)
# omop delete
omop_delete_parser = omop_subparsers.add_parser(
"delete", help="Delete OMOP database"
)
- omop_delete_parser.set_defaults(func=omop_delete)
+ omop_delete_parser.set_defaults(func=_omop_delete)
### PHEN Command ###
phen_parser = subparsers.add_parser("phen", help="Phen commands")
@@ -135,12 +150,44 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Phenotype workspace directory",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
)
phen_init_parser.add_argument(
- "-r", "--remote_url", help="URL to remote git repository"
+ "-r",
+ "--remote_url",
+ help="(Optional) URL to repository where the phenotype will be published.",
+ )
+ phen_init_parser.set_defaults(func=_phen_init)
+
+ # phen fork
+ phen_fork_parser = phen_subparsers.add_parser(
+ "fork", help="Fork an existing phenotype"
+ )
+ phen_fork_parser.add_argument(
+ "-d",
+ "--phen-dir",
+ type=str,
+ default=str(phen.DEFAULT_PHEN_PATH.resolve()),
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
+ )
+ phen_fork_parser.add_argument(
+ "-r",
+ "--remote_url",
+ help="(Optional) URL to repository where the forked phenotype will be published.",
+ )
+ phen_fork_parser.add_argument(
+ "-u",
+ "--upstream-url",
+ required=True,
+ help="(Required) URL to the phenotype repository to fork.",
)
- phen_init_parser.set_defaults(func=phen_init)
+ phen_fork_parser.add_argument(
+ "-v",
+ "--upstream-version",
+ required=True,
+ help="(Required) Phenotype version to fork.",
+ )
+ phen_fork_parser.set_defaults(func=_phen_fork)
# phen validate
phen_validate_parser = phen_subparsers.add_parser(
@@ -151,9 +198,9 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Phenotype workspace directory",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
)
- phen_validate_parser.set_defaults(func=phen_validate)
+ phen_validate_parser.set_defaults(func=_phen_validate)
# phen map
phen_map_parser = phen_subparsers.add_parser("map", help="Process phen mapping")
@@ -162,24 +209,15 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Phenotype workspace directory",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
)
phen_map_parser.add_argument(
"-t",
"--target-coding",
- required=True,
- choices=["read2", "read3", "icd10", "snomed", "opcs4"],
- help="Specify the target coding (read2, read3, icd10, snomed, opcs4)",
+ choices=parse.SUPPORTED_CODE_TYPES,
+ help=f"Specify the target coding {parse.SUPPORTED_CODE_TYPES}",
)
- phen_map_parser.add_argument(
- "-o",
- "--output",
- choices=["csv", "omop"],
- nargs="+", # allows one or more values
- default=["csv"], # default to CSV if not specified
- help="Specify output format(s): 'csv', 'omop', or both (default: csv)",
- )
- phen_map_parser.set_defaults(func=phen_map)
+ phen_map_parser.set_defaults(func=_phen_map)
# phen export
phen_export_parser = phen_subparsers.add_parser(
@@ -190,7 +228,7 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Phenotype workspace directory",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
)
phen_export_parser.add_argument(
"-v",
@@ -199,7 +237,7 @@ def main():
default="latest",
help="Phenotype version to export, defaults to the latest version",
)
- phen_export_parser.set_defaults(func=phen_export)
+ phen_export_parser.set_defaults(func=_phen_export)
# phen publish
phen_publish_parser = phen_subparsers.add_parser(
@@ -210,9 +248,23 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Phenotype workspace directory",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
+ )
+ phen_publish_parser.add_argument(
+ "-i",
+ "--increment",
+ type=str,
+ default=phen.DEFAULT_VERSION_INC,
+ choices=phen.SEMANTIC_VERSION_TYPES,
+ help=f"Version increment: {phen.SEMANTIC_VERSION_TYPES}, default is {phen.DEFAULT_VERSION_INC} increment",
+ )
+ phen_publish_parser.add_argument(
+ "-m", "--msg", help="Message to include with the published version"
+ )
+ phen_publish_parser.add_argument(
+ "-r", "--remote_url", help="URL to remote git repository"
)
- phen_publish_parser.set_defaults(func=phen_publish)
+ phen_publish_parser.set_defaults(func=_phen_publish)
# phen copy
phen_copy_parser = phen_subparsers.add_parser(
@@ -223,13 +275,13 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Phenotype workspace directory",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
)
phen_copy_parser.add_argument(
"-td",
"--target-dir",
type=str,
- default=str(DEFAULT_WORKING_PATH.resolve()),
+ default=str(DEFAULT_WORKSPACE_PATH.resolve()),
help="Target directory for the copy",
)
phen_copy_parser.add_argument(
@@ -239,7 +291,7 @@ def main():
default="latest",
help="Phenotype version to copy, defaults to the latest version",
)
- phen_copy_parser.set_defaults(func=phen_copy)
+ phen_copy_parser.set_defaults(func=_phen_copy)
# phen diff
phen_diff_parser = phen_subparsers.add_parser(
@@ -250,15 +302,28 @@ def main():
"--phen-dir",
type=str,
default=str(phen.DEFAULT_PHEN_PATH.resolve()),
- help="Directory for the new phenotype version",
+ help="(Optional) Local phenotype workspace directory (default is ./workspace/phen).",
+ )
+ phen_diff_parser.add_argument(
+ "-v",
+ "--version",
+ default="latest",
+ help="Phenotype version to compare with an old version, defaults to the HEAD of the workspace directory",
+ )
+ phen_diff_parser.add_argument(
+ "-od",
+ "--old-phen-dir",
+ type=str,
+ default=str(phen.DEFAULT_PHEN_PATH.resolve()),
+ help="Directory for the old phenotype version, defaults to workspace directory",
)
phen_diff_parser.add_argument(
- "-old",
- "--phen-dir-old",
+ "-ov",
+ "--old-version",
required=True,
- help="Directory of the old phenotype version that is compared to the new one",
+ help="Old phenotype version to compare with the changed version",
)
- phen_diff_parser.set_defaults(func=phen_diff)
+ phen_diff_parser.set_defaults(func=_phen_diff)
# Parse arguments
args = parser.parse_args()
diff --git a/acmc/omop.py b/acmc/omop.py
index 90b621b1d028d6f51a327286c0a1574363e129bf..6ce36ad63ba0244615fbf73ddc70b8f4e3d97467 100644
--- a/acmc/omop.py
+++ b/acmc/omop.py
@@ -1,3 +1,9 @@
+"""
+omop.py module
+
+This module provides functionality to manage OMOP vocabularies.
+"""
+
import os
import argparse
import sqlite3
@@ -8,38 +14,53 @@ import shutil
import json
import yaml
from pathlib import Path
-
-from acmc import logging_config
+from acmc import util, logging_config
# setup logging
-logger = logging_config.setup_logger()
+_logger = logging_config.setup_logger()
-# constants
VOCAB_PATH = Path("./vocab/omop")
-DB_PATH = VOCAB_PATH / "omop_54.sqlite"
-VERSION_FILE = "omop_version.yaml"
+"""Default OMOP vocabulary path"""
+
+OMOP_CDM_Version = "54"
+"""Supported OMOP CDM version number"""
+
+OMOP_DB_FILENAME = f"omop_{OMOP_CDM_Version}.sqlite"
+"""Default OMOP sqllite database filename"""
+
+DB_PATH = VOCAB_PATH / OMOP_DB_FILENAME
+"""Default OMOP sqllite database path"""
+
+VERSION_FILE = "omop_version.yml"
+"""Default OMOP version file"""
+
VERSION_PATH = VOCAB_PATH / VERSION_FILE
-EXPORT_FILE = "omop_export.db"
+"""Default OMOP version path"""
+
+EXPORT_FILE = f"omop_{OMOP_CDM_Version}_export.sqlite"
+"""Default OMOP export database filename"""
vocabularies = {
"source": "OHDSI Athena",
"url": "https://athena.ohdsi.org/vocabulary/list",
+ "cdm_version": OMOP_CDM_Version,
"version": "",
"vocabularies": [
- {"id": 1, "name": "SNOMED"},
- {"id": 2, "name": "ICD9CM"},
- {"id": 17, "name": "Readv2"},
- {"id": 21, "name": "ATC"},
- {"id": 55, "name": "OPCS4"},
- {"id": 57, "name": "HES Specialty"},
- {"id": 70, "name": "ICD10CM"},
- {"id": 75, "name": "dm+d"},
- {"id": 144, "name": "UK Biobank"},
- {"id": 154, "name": "NHS Ethnic Category"},
- {"id": 155, "name": "NHS Place of Service"},
+ {"id": 1, "name": "SNOMED"}, # No license required
+ {"id": 2, "name": "ICD9CM"}, # No license required
+ {"id": 17, "name": "Readv2"}, # No license required
+ {"id": 21, "name": "ATC"}, # No license required
+ {"id": 55, "name": "OPCS4"}, # No license required
+ {"id": 57, "name": "HES Specialty"}, # No license required
+ {"id": 70, "name": "ICD10CM"}, # No license required
+ {"id": 75, "name": "dm+d"}, # No license required
+ {"id": 144, "name": "UK Biobank"}, # No license required
+ {"id": 154, "name": "NHS Ethnic Category"}, # No license required
+ {"id": 155, "name": "NHS Place of Service"}, # No license required
],
"tables": [],
}
+"""Required OMOP vocabularies definition"""
omop_vocab_types = {
"read2": "Read",
@@ -51,49 +72,61 @@ omop_vocab_types = {
"med": None,
"cprd": None,
}
+"""Type mappings from acmc medical coding types to OMOP vocabulary types"""
-# Populate SQLite3 Database with default OMOP CONCEPTS
-def install(omop_zip_file, version):
- """Installs the OMOP release csv files in a file-based sql database"""
- logger.info(f"Installing OMOP from zip file: {omop_zip_file}")
+def install(omop_zip_file: str, version: str):
+ """ "Installs the OMOP release csv files in a file-based sql database
+
+ Args:
+ omop_zip_file (str): vocabularies zip file distributed by OHDSI Athena
+ version (str): version of the vocabularies distributed by OHDSI Athena
+
+ Raises:
+ ValueError: if the zip file does not exist
+ ValueError: if the file is not a zip file
+ Exception: if error reading omop csv files
+ """
+ _logger.info(f"Installing OMOP from zip file: {omop_zip_file}")
omop_zip_path = Path(omop_zip_file)
# Check if the file exists and is a ZIP file
if not omop_zip_path.exists():
msg = f"{omop_zip_path} does not exist."
- logger.error(msg)
+ _logger.error(msg)
raise ValueError(msg)
+
if not zipfile.is_zipfile(omop_zip_path):
msg = f"Error: {omop_zip_path} is not a valid ZIP file."
- logger.error(msg)
+ _logger.error(msg)
raise ValueError(msg)
# check codes directory exists and if not create it
if not VOCAB_PATH.exists():
VOCAB_PATH.mkdir(parents=True)
- logger.debug(f"OMOP directory '{VOCAB_PATH}' created.")
+ _logger.debug(f"OMOP directory '{VOCAB_PATH}' created.")
else:
# removing existing OMOP files
csv_files = list(VOCAB_PATH.glob("*.csv"))
for file in csv_files:
file.unlink()
- logger.debug(f"Deleted OMOP csv file: {file}")
+ _logger.debug(f"Deleted OMOP csv file: {file}")
# Extract ZIP contents
with zipfile.ZipFile(omop_zip_path, "r") as zip_ref:
zip_ref.extractall(VOCAB_PATH)
- logger.info(f"Extracted OMOP zip file {omop_zip_path} to {VOCAB_PATH}/")
+ _logger.info(f"Extracted OMOP zip file {omop_zip_path} to {VOCAB_PATH}/")
# connect to database, if it does not exist it will be created
conn = sqlite3.connect(DB_PATH)
+
# Iterate through files in the folder
csv_files = list(VOCAB_PATH.glob("*.csv"))
total_tables_count = len(csv_files)
table_count = 1
for filename in csv_files:
try:
- logger.info(
+ _logger.info(
f"Processing {table_count} of {total_tables_count} tables: {filename}"
)
# read the CSV file with the specified delimiter
@@ -103,7 +136,7 @@ def install(omop_zip_file, version):
df.to_sql(filename.stem, conn, if_exists="replace", index=False)
# add to the metadata
- vocabularies["tables"].append(filename.stem)
+ list(vocabularies["tables"]).append(filename.stem)
table_count = table_count + 1
except Exception as e:
raise Exception(f"Error reading file {filename}: {e}")
@@ -113,50 +146,83 @@ def install(omop_zip_file, version):
# write version file
write_version_file(version)
- logger.info(f"OMOP installation completed")
+ _logger.info(f"OMOP installation completed")
-def write_version_file(version):
- """Writes the OMOP vocaburaries and version to a file"""
+def write_version_file(version: str):
+ """Writes the OMOP vocaburaries and version to a file
+
+ Args:
+ version (str): version of the vocabularies distributed by OHDSI Athena
+ """
vocabularies["version"] = version
with open(VERSION_PATH, "w") as file:
- yaml.dump(vocabularies, file, default_flow_style=False, sort_keys=False)
-
-
-def clear(db_path):
- """Clears the OMOP sql database"""
- logger.info(f"Clearing OMOP data from database")
- omop_db_path = Path(db_path)
- if not omop_db_path.is_file():
- raise FileNotFoundError(f"Error: OMOP DB file '{omop_db_path}' does not exist.")
+ yaml.dump(
+ vocabularies,
+ file,
+ Dumper=util.QuotedDumper,
+ default_flow_style=False,
+ sort_keys=False,
+ default_style='"',
+ )
+
+
+def clear(db_path: Path):
+ """Clears the OMOP sql database
+
+ Args:
+ db_path (Path): the path to the omop sqllite database
+
+ Raises:
+ FileNotFoundError: if the omop sqllite database does not exist
+ """
+ _logger.info(f"Clearing OMOP data from database")
+ if not db_path.is_file():
+ raise FileNotFoundError(f"Error: OMOP DB file '{db_path}' does not exist.")
conn = sqlite3.connect(db_path)
cur = conn.cursor()
cur.execute("SELECT name FROM sqlite_master WHERE type='table';")
# Fetch and print table names
tables = cur.fetchall()
- logger.debug("Tables in database:", [table[0] for table in tables])
+ _logger.debug("Tables in database:", [table[0] for table in tables])
# cur.execute("DROP TABLE CONCEPT_SET;")
# cur.execute("DROP TABLE CONCEPT_SET_ITEM;")
conn.close()
- logger.info(f"OMOP database cleared")
+ _logger.info(f"OMOP database cleared")
+
+def delete(db_path: Path):
+ """Deletes the OMOP sql database
-def delete(db_path):
- """Deletes the OMOP sql database"""
- logger.info(f"Deleting OMOP database")
- omop_db_path = Path(db_path)
- if not omop_db_path.is_file():
- raise FileNotFoundError(f"Error: OMOP DB file '{omop_db_path}' does not exist.")
+ Args:
+ db_path (Path): the path to the omop sqllite database
- omop_db_path.unlink()
- logger.info(f"OMOP database deleted")
+ Raises:
+ FileNotFoundError: if the omop sqllite database does not exist
+ """
+ _logger.info(f"Deleting OMOP database")
+ if not db_path.is_file():
+ raise FileNotFoundError(f"Error: OMOP DB file '{db_path}' does not exist.")
+
+ db_path.unlink()
+ _logger.info(f"OMOP database deleted")
+
+
+def table_exists(cursor: sqlite3.Cursor, table_name: str) -> bool:
+ """Query to check if the table exists
+
+ Args:
+ cursor (sqlite3.Cursor): a sqllite database cursor
+ table_name (str): the table name to check
+
+ Returns:
+ bool: true if table exists
+ """
-def table_exists(cursor, table_name):
- # Query to check if the table exists
cursor.execute(
"""
SELECT name
@@ -172,8 +238,17 @@ def table_exists(cursor, table_name):
return result is not None
-def vocab_exists(cursor, vocab_id):
- # Query to check if the table exists
+def vocab_exists(cursor: sqlite3.Cursor, vocab_id: str) -> bool:
+ """Query to check if the vocabulary exists
+
+ Args:
+ cursor (sqlite3.Cursor): a sqllite database cursor
+ vocab_id (str): the vocabulary id to check
+
+ Returns:
+ bool: true if vocabulary id exists
+ """
+
cursor.execute(
"""
SELECT vocabulary_id
@@ -189,7 +264,16 @@ def vocab_exists(cursor, vocab_id):
return result is not None
-def concept_set_exist(cursor, concept_set_name):
+def concept_set_exist(cursor: sqlite3.Cursor, concept_set_name: str) -> bool:
+ """Query to check if the concept set exists
+
+ Args:
+ cursor (sqlite3.Cursor): a sqllite database cursor
+ concept_set_name (str): the concept set name to check
+
+ Returns:
+ bool: true if concept set exists
+ """
query = f"SELECT EXISTS (SELECT 1 FROM CONCEPT_SET WHERE concept_set_name = ?)"
cursor.execute(query, (concept_set_name,))
@@ -198,8 +282,22 @@ def concept_set_exist(cursor, concept_set_name):
return cursor.fetchone()[0] == 1
-def export(map_path, export_path, version, omop_metadata):
- logger.debug(f"exporting with metadata {omop_metadata} at version {version}")
+def export(
+ map_path: Path, export_path: Path, version: str, omop_metadata: dict
+) -> Path:
+ """Export concept sets to omop database in csv format
+
+ Args:
+ map_path (Path): path to the acmc map directory containing concept sets in csv format
+ export_path (Path): path to the directory where the omop database csv files are to be written
+ version (str): phenotype version for omop vocabulary version
+ omop_metadata (dict): phenotype omop metadata for omop vocabulary metadata
+
+ Returns:
+ Path: path to the exported sqllite database
+ """
+
+ _logger.debug(f"exporting with metadata {omop_metadata} at version {version}")
# copy the baseline omop database
export_db_path = export_path / EXPORT_FILE
@@ -251,20 +349,19 @@ def export(map_path, export_path, version, omop_metadata):
# read map files
map_files = list(map_path.glob("*.csv"))
total = len(map_files)
- logger.info(f"Exporting {total} map files")
+ _logger.info(f"Exporting {total} map files")
for index, map_file in enumerate(map_files):
- logger.info(f"Processing {index+1} of {total}: {map_file}")
+ _logger.info(f"Processing {index+1} of {total}: {map_file}")
df = pd.read_csv(map_file)
for concept_set_name, grp in df.groupby("CONCEPT_SET"):
-
# create Concept_Set
- if not concept_set_exist(cur, concept_set_name):
+ if not concept_set_exist(cur, str(concept_set_name)):
cur.execute(
f"INSERT INTO CONCEPT_SET (concept_set_name, vocabulary_id) VALUES ('{concept_set_name}', '{omop_metadata['vocabulary_id']}');"
)
else:
- logger.debug(f"Concept_set {concept_set_name} already exists")
+ _logger.debug(f"Concept_set {concept_set_name} already exists")
# TODO: ask to remove old concept_set?
# get Concept_set_Id
@@ -278,10 +375,10 @@ def export(map_path, export_path, version, omop_metadata):
),
)
# FAILS HERE WITH NONE REUR
- logger.debug(f"target code type {target_code_type}")
- logger.debug(f"omop code type {omop_vocab_types[target_code_type]}")
+ _logger.debug(f"target code type {target_code_type}")
+ _logger.debug(f"omop code type {omop_vocab_types[target_code_type]}")
concept_set_id = cur.fetchone()[0]
- logger.debug(f"concept set id {concept_set_id}")
+ _logger.debug(f"concept set id {concept_set_id}")
# get corresponing Concept_id (OMOP) for each Concept_code (e.g. SNOMED)
concept_codes = "'" + "', '".join(list(grp["CONCEPT"].astype(str))) + "'"
@@ -290,7 +387,7 @@ def export(map_path, export_path, version, omop_metadata):
df_out = pd.DataFrame(cur.fetchall(), columns=["concept_id"])
if not len(grp) == len(df_out):
- logger.error(
+ _logger.error(
f"ERROR: Some {omop_vocab_types[target_code_type]} Codes do not exist in OMOP Database"
)
@@ -305,17 +402,17 @@ def export(map_path, export_path, version, omop_metadata):
# Export each table to a separate CSV file
for table in tables:
- table_name = table[0] # Extract table name
- df = pd.read_sql_query(f"SELECT * FROM {table_name}", conn)
- output_file = f"{table_name}.csv"
- output_path = export_path / output_file
- df.to_csv(output_path, index=False) # Save as CSV
- logger.info(f"Exported {table_name} to {table_name}.csv")
+ table_name = table[0]
+ # ignore SQLite's internal system table
+ if table_name != "sqlite_sequence":
+ df = pd.read_sql_query(f"SELECT * FROM {table_name}", conn)
+ output_file = f"{table_name}.csv"
+ output_path = export_path / output_file
+ df.to_csv(output_path, index=False) # Save as CSV
+ _logger.info(f"Exported {table_name} to {table_name}.csv")
conn.close()
- logger.debug(f"Created export db successfully")
-
- return export_db_path
+ _logger.debug(f"Created export db successfully")
return export_db_path
diff --git a/acmc/parse.py b/acmc/parse.py
index ca5b3e1dc847994cab714142d1bfc946eb15bc64..45a9839a6533b8a742091d3ac6dba343a02e3ca5 100644
--- a/acmc/parse.py
+++ b/acmc/parse.py
@@ -1,12 +1,22 @@
+"""
+parse.py module
+
+This module provides functionality to set up medical code translation classes
+
+"""
+
import pandas as pd
import numpy as np
import os
-from typing import Callable
-
+from typing import Callable, Optional, Tuple
+from pathlib import Path
from acmc import trud, logging_config as lc
# setup logging
-logger = lc.setup_logger()
+_logger = lc.setup_logger()
+
+SUPPORTED_CODE_TYPES = {"read2", "read3", "icd10", "snomed", "opcs4", "atc"}
+"""List of support medical coding types"""
class CodesError:
@@ -39,36 +49,40 @@ class Proto:
tuple[
str, # The description, e.g., "Not Empty"
Callable[
- [list],
+ [pd.DataFrame],
pd.Series,
], # The first lambda function: takes a list and returns a pd.Series of booleans
Callable[
- [list, str],
- None,
+ [pd.DataFrame, Path],
+ pd.DataFrame,
], # The second lambda function: takes a list and a string, and returns nothing
]
]
- def __init__(self, name, trud_codes_path=None):
+ def __init__(self, name: str, trud_codes_path: Optional[Path] = None):
if trud_codes_path is not None:
if trud_codes_path.is_file():
- self.trud_codes_path = trud_codes_path
- self.db = pd.read_parquet(self.trud_codes_path)
+ self.trud_codes_path: Path = trud_codes_path
+ self.db: pd.DataFrame = pd.read_parquet(self.trud_codes_path)
else:
raise FileNotFoundError(
f"Error: Read2 code file '{trud_codes_path}' does not exist. Please ensure you have installed TRUD correctly"
)
- self.name = name
+ self.name: str = name
- def raise_exception(self, ex):
+ def raise_exception(self, ex: Exception):
"""Raises an exception inside a lambda function. Python does not allow using raise statement inside lambda because lambda can only contain expressions, not statements. Using raise_exception not raise_ as it's more explict"""
raise ex
- def in_database(self, codes, db, col):
+ def in_database(
+ self, codes: pd.DataFrame, db: pd.DataFrame, col: str
+ ) -> pd.DataFrame:
return codes.isin(db[col])
- def process(self, codes, codes_file):
+ def process(
+ self, codes: pd.DataFrame, codes_file: Path
+ ) -> Tuple[pd.DataFrame, list]:
"""identify issues that do not pass and fix them with define/d process"""
errors = []
# Iter through each item in check.
@@ -76,21 +90,21 @@ class Proto:
# Check if any codes fail the check to False
if not cond(codes).all():
# Log the number of codes that failed
- logger.debug(
+ _logger.debug(
f"Check: {msg} {(~cond(codes)).sum()} failed, trying to fix"
)
# try fix errors by running lamba "process" function
try:
codes = fix(codes, codes_file)
- logger.debug(f"Check: Fixed")
+ _logger.debug(f"Check: Fixed")
except InvalidCodesException as ex:
errors.append(ex.error)
else:
- logger.debug(f"Check: passed")
+ _logger.debug(f"Check: passed")
return codes, errors
- def verify(self, codes, codes_file):
+ def verify(self, codes: pd.DataFrame, codes_file: Path):
"""verify codes in codes file"""
conds = np.array([])
@@ -330,10 +344,6 @@ class Icd10(Proto):
# )
]
- def trim_icd10(codes):
- codes = codes.str[:4]
- return codes
-
class Snomed(Proto):
def __init__(self):
@@ -535,8 +545,7 @@ class Cprd(Proto):
class CodeTypeParser:
"""A class used in InvalidCodesException to report an error if a code parser check fails"""
- def __init__(self, trud_processed_dir=trud.PROCESSED_PATH):
-
+ def __init__(self, trud_processed_dir: Path = trud.PROCESSED_PATH):
if not trud_processed_dir.exists() or not trud_processed_dir.is_dir():
raise FileNotFoundError(
f"Cannot initialise parsers as the TRUD processed directory {trud_processed_dir} does not exist, please check that TRUD has been installed: acmc trud install"
diff --git a/acmc/phen.py b/acmc/phen.py
index 866966863f61854779836ead5335e498cb3f3ba6..ec675c2eee0d0683c7d772aad2dc0667d0035647 100644
--- a/acmc/phen.py
+++ b/acmc/phen.py
@@ -1,3 +1,9 @@
+"""
+phenotype.py module
+
+This module provides functionality for managing phenotypes.
+"""
+
import argparse
import pandas as pd
import numpy as np
@@ -6,43 +12,144 @@ import os
import sqlite3
import sys
import shutil
+import time
import git
import re
import logging
import requests
import yaml
+import semver
+from git import Repo
+from cerberus import Validator # type: ignore
+from deepdiff import DeepDiff
from pathlib import Path
from urllib.parse import urlparse, urlunparse
-
+from typing import Tuple, Set, Any
import acmc
-from acmc import trud, omop, parse
-
-# setup logging
-import acmc.logging_config as lc
+from acmc import trud, omop, parse, util, logging_config as lc
-logger = lc.setup_logger()
+# set up logging
+_logger = lc.setup_logger()
pd.set_option("mode.chained_assignment", None)
PHEN_DIR = "phen"
+"""Default phenotype directory name"""
+
DEFAULT_PHEN_PATH = Path("./workspace") / PHEN_DIR
+"""Default phenotype directory path"""
+
+CONCEPTS_DIR = "concepts"
+"""Default concepts directory name"""
-CODES_DIR = "codes"
MAP_DIR = "map"
-CONCEPT_SET_DIR = "concept-set"
-OMOP_DIR = "omop"
-DEFAULT_PHEN_DIR_LIST = [CODES_DIR, MAP_DIR, CONCEPT_SET_DIR, OMOP_DIR]
-CONFIG_FILE = "config.yaml"
-VOCAB_VERSION_FILE = "vocab_version.yaml"
+"""Default map directory name"""
+
+CONCEPT_SET_DIR = "concept-sets"
+"""Default concept set directory name"""
+
+CSV_PATH = Path(CONCEPT_SET_DIR) / "csv"
+"""Default CSV concept set directory path"""
+
+OMOP_PATH = Path(CONCEPT_SET_DIR) / "omop"
+"""Default OMOP concept set directory path"""
+
+DEFAULT_PHEN_DIR_LIST = [CONCEPTS_DIR, MAP_DIR, CONCEPT_SET_DIR]
+"""List of default phenotype directories"""
+
+CONFIG_FILE = "config.yml"
+"""Default configuration filename"""
+
+VOCAB_VERSION_FILE = "vocab_version.yml"
+"""Default vocabulary version filename"""
+
+SEMANTIC_VERSION_TYPES = ["major", "minor", "patch"]
+"""List of semantic version increment types"""
+
+DEFAULT_VERSION_INC = "patch"
+"""Default semantic version increment type"""
DEFAULT_GIT_BRANCH = "main"
+"""Default phenotype repo branch name"""
SPLIT_COL_ACTION = "split_col"
+"""Split column preprocessing action type"""
+
CODES_COL_ACTION = "codes_col"
+"""Codes column preprocessing action type"""
+
DIVIDE_COL_ACTION = "divide_col"
+"""Divide column preprocessing action type"""
+
COL_ACTIONS = [SPLIT_COL_ACTION, CODES_COL_ACTION, DIVIDE_COL_ACTION]
+"""List of column preprocessing action types"""
CODE_FILE_TYPES = [".xlsx", ".xls", ".csv"]
+"""List of supported source concept coding list file types"""
+
+# config.yaml schema
+CONFIG_SCHEMA = {
+ "phenotype": {
+ "type": "dict",
+ "required": True,
+ "schema": {
+ "version": {
+ "type": "string",
+ "required": True,
+ "regex": r"^\d+\.\d+\.\d+$", # Enforces 'vN.N.N' format
+ },
+ "omop": {
+ "type": "dict",
+ "required": True,
+ "schema": {
+ "vocabulary_id": {"type": "string", "required": True},
+ "vocabulary_name": {"type": "string", "required": True},
+ "vocabulary_reference": {
+ "type": "string",
+ "required": True,
+ "regex": r"^https?://.*", # Ensures it's a URL
+ },
+ },
+ },
+ "map": {
+ "type": "list",
+ "schema": {
+ "type": "string",
+ "allowed": list(
+ parse.SUPPORTED_CODE_TYPES
+ ), # Ensure only predefined values are allowed
+ },
+ },
+ "concept_sets": {
+ "type": "list",
+ "required": True,
+ "schema": {
+ "type": "dict",
+ "schema": {
+ "name": {"type": "string", "required": True},
+ "file": {
+ "type": "dict",
+ "required": False,
+ "schema": {
+ "path": {"type": "string", "required": True},
+ "columns": {"type": "dict", "required": True},
+ "category": {
+ "type": "string"
+ }, # Optional but must be string if present
+ "actions": {
+ "type": "dict",
+ "schema": {"divide_col": {"type": "string"}},
+ },
+ },
+ },
+ "metadata": {"type": "dict", "required": True},
+ },
+ },
+ },
+ },
+ }
+}
+"""Phenotype config.yml schema definition"""
class PhenValidationException(Exception):
@@ -53,7 +160,7 @@ class PhenValidationException(Exception):
self.validation_errors = validation_errors
-def construct_git_url(remote_url):
+def _construct_git_url(remote_url: str):
"""Constructs a git url for github or gitlab including a PAT token environment variable"""
# check the url
parsed_url = urlparse(remote_url)
@@ -90,71 +197,87 @@ def construct_git_url(remote_url):
)
-def create_empty_git_dir(path):
+def _create_empty_git_dir(path: Path):
"""Creates a directory with a .gitkeep file so that it's tracked in git"""
path.mkdir(exist_ok=True)
keep_path = path / ".gitkeep"
keep_path.touch(exist_ok=True)
-def init(phen_dir, remote_url):
+def _check_delete_dir(path: Path, msg: str) -> bool:
+ """Checks on the command line if a user wants to delete a directory
+
+ Args:
+ path (Path): path of the directory to be deleted
+ msg (str): message to be displayed to the user
+
+ Returns:
+ Boolean: True if deleted
+ """
+ deleted = False
+
+ user_input = input(f"{msg}").strip().lower()
+ if user_input in ["yes", "y"]:
+ shutil.rmtree(path)
+ deleted = True
+ else:
+ _logger.info("Directory was not deleted.")
+
+ return deleted
+
+
+def init(phen_dir: str, remote_url: str):
"""Initial phenotype directory as git repo with standard structure"""
- logger.info(f"Initialising Phenotype in directory: {phen_dir}")
+ _logger.info(f"Initialising Phenotype in directory: {phen_dir}")
phen_path = Path(phen_dir)
# check if directory already exists and ask user if they want to recreate it
- configure = False
if (
phen_path.exists() and phen_path.is_dir()
): # Check if it exists and is a directory
- user_input = (
- input(
- f"The phen directory already exists. Do you want to reinitialise? (yes/no): "
- )
- .strip()
- .lower()
+ configure = _check_delete_dir(
+ phen_path,
+ f"The phen directory already exists. Do you want to reinitialise? (yes/no): ",
)
- if user_input in ["yes", "y"]:
- shutil.rmtree(phen_path)
- configure = True
- else:
- logger.info("Phen directory was not recreated.")
else:
configure = True
if not configure:
- logger.info(f"Exiting, phenotype not initiatised")
+ _logger.info(f"Exiting, phenotype not initiatised")
return
# Initialise repo from local or remote
- repo = None
+ repo: Repo
+
# if remote then clone the repo otherwise init a local repo
if remote_url != None:
# add PAT token to the URL
- git_url = construct_git_url(remote_url)
+ git_url = _construct_git_url(remote_url)
# clone the repo
- repo = git.cmd.Git()
- repo.clone(git_url, phen_path)
+ git_cmd = git.cmd.Git()
+ git_cmd.clone(git_url, phen_path)
+
# open repo
- repo = git.Repo(phen_path)
+ repo = Repo(phen_path)
# check if there are any commits (new repo has no commits)
if (
len(repo.branches) == 0 or repo.head.is_detached
): # Handle detached HEAD (e.g., after init)
- logger.debug("The phen repository has no commits yet.")
+ _logger.debug("The phen repository has no commits yet.")
commit_count = 0
else:
# Get the total number of commits in the default branch
commit_count = sum(1 for _ in repo.iter_commits())
- logger.debug(f"Repo has previous commits: {commit_count}")
+ _logger.debug(f"Repo has previous commits: {commit_count}")
else:
# local repo, create the directories and init
phen_path.mkdir(parents=True, exist_ok=True)
- logger.debug(f"Phen directory '{phen_path}' has been created.")
+ _logger.debug(f"Phen directory '{phen_path}' has been created.")
repo = git.Repo.init(phen_path)
commit_count = 0
+ phen_path = phen_path.resolve()
# initialise empty repos
if commit_count == 0:
# create initial commit
@@ -178,42 +301,48 @@ def init(phen_dir, remote_url):
# if the phen path does not contain the config file then initialise the phen type
config_path = phen_path / CONFIG_FILE
if config_path.exists():
- logger.debug(f"Phenotype configuration files already exist")
+ _logger.debug(f"Phenotype configuration files already exist")
return
- logger.info("Creating phen directory structure and config files")
+ _logger.info("Creating phen directory structure and config files")
for d in DEFAULT_PHEN_DIR_LIST:
- create_empty_git_dir(phen_path / d)
-
- # set initial version based on the number of commits in the repo, depending on how the repo was created
- # e.g., with a README.md, then there will be some initial commits before the phen config is added
- next_commit_count = commit_count + 1
- initial_version = f"v1.0.{next_commit_count}"
+ _create_empty_git_dir(phen_path / d)
# create empty phen config file
config = {
"phenotype": {
- "version": initial_version,
+ "version": "0.0.0",
"omop": {
"vocabulary_id": "",
"vocabulary_name": "",
"vocabulary_reference": "",
},
+ "translate": [],
"concept_sets": [],
}
}
with open(phen_path / CONFIG_FILE, "w") as file:
- yaml.dump(config, file, default_flow_style=False, sort_keys=False)
+ yaml.dump(
+ config,
+ file,
+ Dumper=util.QuotedDumper,
+ default_flow_style=False,
+ sort_keys=False,
+ default_style='"',
+ )
# add git ignore
ignore_content = """# Ignore SQLite database files
- *.db
- *.sqlite3
+*.db
+*.sqlite3
- # Ignore SQLite journal and metadata files
- *.db-journal
- *.sqlite3-journal
+# Ignore SQLite journal and metadata files
+*.db-journal
+*.sqlite3-journal
+
+# python
+.ipynb_checkpoints
"""
ignore_path = phen_path / ".gitignore"
with open(ignore_path, "w") as file:
@@ -225,12 +354,104 @@ def init(phen_dir, remote_url):
repo.git.add(all=True)
repo.index.commit("initialised the phen git repo.")
- logger.info(f"Phenotype initialised successfully")
+ _logger.info(f"Phenotype initialised successfully")
+
+
+def fork(phen_dir: str, upstream_url: str, upstream_version: str, new_origin_url: str):
+ """Forks an upstream phenotype in a remote repo at a specific version to a local director, and optionally sets to a new remote origin"
+
+ Args:
+ phen_dir (str): local directory path where the upstream repo is to be cloned
+ upstream_url (str): url to the upstream repo
+ upstream_version (str): version in the upstream repo to clone
+ new_origin_url (str, optional): url of the remote repo to set as the new origin. Defaults to None.
+
+ Raises:
+ ValueError: if the specified version is not in the upstream repo
+ ValueError: if the upstream repo is not a valid phenotype repo
+ ValueError: if there's any other problems with Git
+ """
+ _logger.info(
+ f"Forking upstream repo {upstream_url} {upstream_version} into directory: {phen_dir}"
+ )
+
+ phen_path = Path(phen_dir)
+ # check if directory already exists and ask user if they want to recreate it
+ if (
+ phen_path.exists() and phen_path.is_dir()
+ ): # Check if it exists and is a directory
+ configure = _check_delete_dir(
+ phen_path,
+ f"The phen directory already exists. Do you want to reinitialise? (yes/no): ",
+ )
+ else:
+ configure = True
+
+ if not configure:
+ _logger.info(f"Exiting, phenotype not initiatised")
+ return
+
+ try:
+ # Clone repo
+ git_url = _construct_git_url(upstream_url)
+ repo = git.Repo.clone_from(git_url, phen_path)
+
+ # Fetch all branches and tags
+ repo.remotes.origin.fetch()
+
+ # Check if the version exists
+ available_refs = [ref.name.split("/")[-1] for ref in repo.references]
+ if upstream_version not in available_refs:
+ raise ValueError(
+ f"Version '{upstream_version}' not found in the repository: {upstream_url}."
+ )
+
+ # Checkout the specified version
+ repo.git.checkout(upstream_version)
+ main_branch = repo.heads[DEFAULT_GIT_BRANCH]
+ main_branch.checkout()
+
+ # Check if 'config.yml' exists in the root directory
+ config_path = phen_path / "config.yml"
+ if not os.path.isfile(config_path):
+ raise ValueError(
+ f"The forked repository is not a valid ACMC repo because 'config.yml' is missing in the root directory."
+ )
+
+ # Validate the phenotype is compatible with the acmc tool
+ validate(str(phen_path.resolve()))
+
+ # Delete each tag locally
+ tags = repo.tags
+ for tag in tags:
+ repo.delete_tag(tag)
+ _logger.debug(f"Deleted tags from forked repo: {tag}")
+
+ # Add upstream remote
+ repo.create_remote("upstream", upstream_url)
+ remote = repo.remotes["origin"]
+ repo.delete_remote(remote) # Remove existing origin
+
+ # Optionally set a new origin remote
+ if new_origin_url:
+ git_url = _construct_git_url(new_origin_url)
+ repo.create_remote("origin", git_url)
+ repo.git.push("--set-upstream", "origin", "main")
+
+ _logger.info(f"Repository forked successfully at {phen_path}")
+ _logger.info(f"Upstream set to {upstream_url}")
+ if new_origin_url:
+ _logger.info(f"Origin set to {new_origin_url}")
+
+ except Exception as e:
+ if phen_path.exists():
+ shutil.rmtree(phen_path)
+ raise ValueError(f"Error occurred during repository fork: {str(e)}")
-def validate(phen_dir):
+def validate(phen_dir: str):
"""Validates the phenotype directory is a git repo with standard structure"""
- logger.info(f"Validating phenotype: {phen_dir}")
+ _logger.info(f"Validating phenotype: {phen_dir}")
phen_path = Path(phen_dir)
if not phen_path.is_dir():
raise NotADirectoryError(
@@ -243,10 +464,10 @@ def validate(phen_dir):
f"Error: phen configuration file '{config_path}' does not exist."
)
- codes_path = phen_path / CODES_DIR
- if not codes_path.is_dir():
+ concepts_path = phen_path / CONCEPTS_DIR
+ if not concepts_path.is_dir():
raise FileNotFoundError(
- f"Error: source codes directory {source_codes_dir} does not exist."
+ f"Error: source concepts directory {concepts_path} does not exist."
)
# Calidate the directory is a git repo
@@ -256,9 +477,20 @@ def validate(phen_dir):
raise Exception(f"Phen directory {phen_path} is not a git repo")
# Load configuration File
- if config_path.suffix == ".yaml":
- with config_path.open("r") as file:
- phenotype = yaml.safe_load(file)
+ if config_path.suffix == ".yml":
+ try:
+ with config_path.open("r") as file:
+ phenotype = yaml.safe_load(file)
+
+ validator = Validator(CONFIG_SCHEMA)
+ if validator.validate(phenotype):
+ _logger.debug("YAML structure is valid.")
+ else:
+ _logger.error(f"YAML structure validation failed: {validator.errors}")
+ raise Exception(f"YAML structure validation failed: {validator.errors}")
+ except yaml.YAMLError as e:
+ _logger.error(f"YAML syntax error: {e}")
+ raise e
else:
raise Exception(
f"Unsupported configuration filetype: {str(config_path.resolve())}"
@@ -270,7 +502,7 @@ def validate(phen_dir):
code_types = parse.CodeTypeParser().code_types
# check the version number is of the format vn.n.n
- match = re.match(r"v(\d+\.\d+\.\d+)", phenotype["version"])
+ match = re.match(r"(\d+\.\d+\.\d+)", phenotype["version"])
if not match:
validation_errors.append(
f"Invalid version format in configuration file: {phenotype['version']}"
@@ -286,62 +518,51 @@ def validate(phen_dir):
else:
concept_set_names.append(item["name"])
- # TODO: change this to some sort of yaml schema validation
- required_keys = {"name", "file", "metadata"}
-
# check codes definition
for item in phenotype["concept_sets"]:
+ # check concepte code file exists
+ concept_code_file_path = concepts_path / item["file"]["path"]
+ if not concept_code_file_path.exists():
+ validation_errors.append(
+ f"Coding file {str(concept_code_file_path.resolve())} does not exist"
+ )
- if required_keys.issubset(item.keys()):
+ # check concepte code file is not empty
+ if concept_code_file_path.stat().st_size == 0:
+ validation_errors.append(
+ f"Coding file {str(concept_code_file_path.resolve())} is an empty file"
+ )
- # check concepte code file exists
- concept_code_file_path = codes_path / item["file"]["path"]
- if not concept_code_file_path.exists():
- validation_errors.append(
- f"Coding file {str(concept_code_file_path.resolve())} does not exist"
- )
+ # check code file type is supported
+ if concept_code_file_path.suffix not in CODE_FILE_TYPES:
+ raise ValueError(
+ f"Unsupported filetype {concept_code_file_path.suffix}, only support csv, xlsx, xls code file types"
+ )
- # check concepte code file is not empty
- if concept_code_file_path.stat().st_size == 0:
+ # check columns specified are a supported medical coding type
+ for column in item["file"]["columns"]:
+ if column not in code_types:
validation_errors.append(
- f"Coding file {str(concept_code_file_path.resolve())} is an empty file"
+ f"Column type {column} for file {concept_code_file_path} is not supported"
)
- # check code file type is supported
- if concept_code_file_path.suffix not in CODE_FILE_TYPES:
- raise ValueError(
- f"Unsupported filetype {concept_code_file_path.suffix}, only support csv, xlsx, xls code file types"
- )
-
- # check columns specified are a supported medical coding type
- for column in item["file"]["columns"]:
- if column not in code_types:
- validation_errors.append(
- f"Column type {column} for file {concept_code_file_path} is not supported"
- )
-
- # check the actions are supported
- if "actions" in item["file"]:
- for action in item["file"]["actions"]:
- if action not in COL_ACTIONS:
- validation_errors.append(f"Action {action} is not supported")
-
- else:
- validation_errors.append(
- f"Missing required elements {required_keys} in concept set {item}"
- )
+ # check the actions are supported
+ if "actions" in item["file"]:
+ for action in item["file"]["actions"]:
+ if action not in COL_ACTIONS:
+ validation_errors.append(f"Action {action} is not supported")
if len(validation_errors) > 0:
- logger.error(validation_errors)
+ _logger.error(validation_errors)
raise PhenValidationException(
f"Configuration file {str(config_path.resolve())} failed validation",
validation_errors,
)
- logger.info(f"Phenotype validated successfully")
+ _logger.info(f"Phenotype validated successfully")
-def read_table_file(path, excel_sheet=None):
+def _read_table_file(path: Path, excel_sheet: str = ""):
"""
Load Code List File
"""
@@ -350,23 +571,23 @@ def read_table_file(path, excel_sheet=None):
if path.suffix == ".csv":
df = pd.read_csv(path, dtype=str)
elif path.suffix == ".xlsx" or path.suffix == ".xls":
- if excel_sheet:
+ if excel_sheet != "":
df = pd.read_excel(path, sheet_name=excel_sheet, dtype=str)
else:
df = pd.read_excel(path, dtype=str)
elif path.suffix == ".dta":
- df = pd.read_stata(path, dtype=str)
+ df = pd.read_stata(path)
else:
raise ValueError(
- f"Unsupported filetype {codes_file_path.suffix}, only support{CODE_FILE_TYPES} code file types"
+ f"Unsupported filetype {path.suffix}, only support{CODE_FILE_TYPES} code file types"
)
return df
-def process_actions(df, concept_set):
+def _process_actions(df: pd.DataFrame, concept_set: dict) -> pd.DataFrame:
# Perform Structural Changes to file before preprocessing
- logger.debug("Processing file structural actions")
+ _logger.debug("Processing file structural actions")
if (
"actions" in concept_set["file"]
and "split_col" in concept_set["file"]["actions"]
@@ -374,7 +595,7 @@ def process_actions(df, concept_set):
):
split_col = concept_set["file"]["actions"]["split_col"]
codes_col = concept_set["file"]["actions"]["codes_col"]
- logger.debug(
+ _logger.debug(
"Action: Splitting",
split_col,
"column into:",
@@ -389,30 +610,25 @@ def process_actions(df, concept_set):
return df
-# Perform QA Checks on columns individually and append to df
-def preprocess_codes(df, concept_set, code_file_path, target_code_type=None):
- """Parses each column individually - Order and length will not be preserved!"""
+def _preprocess_source_concepts(
+ df: pd.DataFrame, concept_set: dict, code_file_path: Path
+) -> Tuple[pd.DataFrame, list]:
+ """Perform QA Checks on columns individually and append to df"""
out = pd.DataFrame([]) # create output df to append to
code_errors = [] # list of errors from processing
- # TODO: Is there a better way of processing this action as it's distributed across
- # different parts of the programme.
- if (
- "actions" in concept_set["file"]
- and "divide_col" in concept_set["file"]["actions"]
- ):
- divide_col_df = df[concept_set["file"]["actions"]["divide_col"]]
- else:
- divide_col_df = pd.DataFrame()
+ # remove unnamed columns due to extra commas, missing headers, or incorrect parsing
+ df = df.drop(columns=[col for col in df.columns if "Unnamed" in col])
# Preprocess codes
code_types = parse.CodeTypeParser().code_types
for code_type in concept_set["file"]["columns"]:
parser = code_types[code_type]
- logger.info(f"Processing {code_type} codes...")
+ _logger.info(f"Processing {code_type} codes for {code_file_path}")
- # get code types
- codes = df[concept_set["file"]["columns"][code_type]].dropna()
+ # get codes by column name
+ source_col_name = concept_set["file"]["columns"][code_type]
+ codes = df[source_col_name].dropna()
codes = codes.astype(str) # convert to string
codes = codes.str.strip() # remove excess spaces
@@ -420,77 +636,88 @@ def preprocess_codes(df, concept_set, code_file_path, target_code_type=None):
codes, errors = parser.process(codes, code_file_path)
if len(errors) > 0:
code_errors.extend(errors)
- logger.warning(f"Codes validation failed with {len(errors)} errors")
+ _logger.warning(f"Codes validation failed with {len(errors)} errors")
- # append to output dataframe
+ # add processed codes to df
+ new_col_name = f"{source_col_name}_SOURCE"
+ df = df.rename(columns={source_col_name: new_col_name})
+ process_codes = pd.DataFrame({code_type: codes}).join(df)
out = pd.concat(
- [out, pd.DataFrame({code_type: codes}).join(divide_col_df)],
+ [out, process_codes],
ignore_index=True,
)
+ _logger.debug(out.head())
+
return out, code_errors
# Translate Df with multiple codes into single code type Series
-def translate_codes(df, target_code_type):
- codes = pd.Series([], dtype=str)
-
+def translate_codes(
+ source_df: pd.DataFrame, target_code_type: str, concept_name: str
+) -> pd.DataFrame:
+ """Translates each source code type the source coding list into a target type and returns all conversions as a concept set"""
+
+ # codes = pd.DataFrame([], dtype=str)
+ codes = pd.DataFrame(
+ columns=["SOURCE_CONCEPT", "SOURCE_CONCEPT_TYPE", "CONCEPT"], dtype="string"
+ )
# Convert codes to target type
- logger.info(f"Converting to target code type {target_code_type}")
- for col_name in df.columns:
+ _logger.info(f"Converting to target code type {target_code_type}")
+
+ for source_code_type in source_df.columns:
# if target code type is the same as thet source code type, no translation, just appending source as target
- if col_name == target_code_type:
- logger.debug(
- f"Target code type {target_code_type} has source code types {len(df)}, copying rather than translating"
+ if source_code_type == target_code_type:
+ copy_df = pd.DataFrame(
+ {
+ "SOURCE_CONCEPT": source_df[source_code_type],
+ "SOURCE_CONCEPT_TYPE": source_code_type,
+ "CONCEPT": source_df[source_code_type],
+ }
+ )
+ codes = pd.concat([codes, copy_df])
+ _logger.debug(
+ f"Target code type {target_code_type} is the same as source code type {len(source_df)}, copying codes rather than translating"
)
- codes = pd.concat([codes, df[target_code_type]])
else:
- filename = f"{col_name}_to_{target_code_type}.parquet"
+ # get the translation filename using source to target code types
+ filename = f"{source_code_type}_to_{target_code_type}.parquet"
map_path = trud.PROCESSED_PATH / filename
+
+ # do the mapping if it exists
if map_path.exists():
- col = df[col_name]
+ # get mapping
df_map = pd.read_parquet(map_path)
- # merge on corresponding codes and take target column
- translated = pd.merge(col, df_map, how="left")[target_code_type]
- # TODO: BUG mask does not match column
- codes = pd.concat([codes, translated]) # merge to output
- else:
- logger.warning(
- f"No mapping from {col_name} to {target_code_type}, file {str(map_path.resolve())} does not exist"
+
+ # do mapping
+ translated_df = pd.merge(
+ source_df[source_code_type], df_map, how="left"
)
- return codes
+ # normalise the output
+ translated_df.columns = pd.Index(["SOURCE_CONCEPT", "CONCEPT"])
+ translated_df["SOURCE_CONCEPT_TYPE"] = source_code_type
+ # add to list of codes
+ codes = pd.concat([codes, translated_df])
-# Append file's codes to output Df with concept
-def map_file(df, target_code_type, out, concept_name):
+ else:
+ _logger.warning(
+ f"No mapping from {source_code_type} to {target_code_type}, file {str(map_path.resolve())} does not exist"
+ )
- # translate codes
- codes = translate_codes(df, target_code_type)
codes = codes.dropna() # delete NaNs
- # Append to output if translated
- if len(codes) > 0:
- codes = pd.DataFrame({"CONCEPT": codes})
- codes["CONCEPT_SET"] = np.repeat(concept_name.strip(), len(codes))
- out = pd.concat([out, codes])
+ # added concept set type to output if any translations
+ if len(codes.index) > 0:
+ codes["CONCEPT_SET"] = concept_name
else:
- logger.debug(f"No codes converted with target code type {target_code_type}")
-
- return out
-
+ _logger.debug(f"No codes converted with target code type {target_code_type}")
-def sql_row_exist(conn, table, column, value):
- # Execute and check if a result exists
- cur = conn.cursor()
- query = f"SELECT 1 FROM {table} WHERE {column} = ? LIMIT 1;"
- cur.execute(query, (value,))
- exists = cur.fetchone() is not None
-
- return exists
+ return codes
-def write_code_errors(code_errors, code_errors_path):
+def _write_code_errors(code_errors: list, code_errors_path: Path):
err_df = pd.DataFrame(
[
{
@@ -508,7 +735,7 @@ def write_code_errors(code_errors, code_errors_path):
err_df.to_csv(code_errors_path, index=False, mode="w")
-def write_vocab_version(phen_path):
+def write_vocab_version(phen_path: Path):
# write the vocab version files
if not trud.VERSION_PATH.exists():
@@ -537,12 +764,18 @@ def write_vocab_version(phen_path):
}
with open(phen_path / VOCAB_VERSION_FILE, "w") as file:
- yaml.dump(version_data, file, default_flow_style=False, sort_keys=False)
+ yaml.dump(
+ version_data,
+ file,
+ Dumper=util.QuotedDumper,
+ default_flow_style=False,
+ sort_keys=False,
+ default_style='"',
+ )
-def map(phen_dir, target_code_type):
- logger.info(f"Processing phenotype: {phen_dir}")
- logger.debug(f"Target coding format: {target_code_type}")
+def map(phen_dir: str, target_code_type: str):
+ _logger.info(f"Processing phenotype: {phen_dir}")
# Validate configuration
validate(phen_dir)
@@ -550,43 +783,70 @@ def map(phen_dir, target_code_type):
# initialise paths
phen_path = Path(phen_dir)
config_path = phen_path / CONFIG_FILE
- codes_path = phen_path / CODES_DIR
# load configuration
with config_path.open("r") as file:
config = yaml.safe_load(file)
phenotype = config["phenotype"]
+ if len(phenotype["map"]) == 0:
+ raise ValueError(f"No map codes defined in the phenotype configuration")
+
+ if target_code_type is not None and target_code_type not in phenotype["map"]:
+ raise ValueError(
+ f"Target code type {target_code_type} not in phenotype configuration map {phenotype['map']}"
+ )
+
+ if target_code_type is not None:
+ _map_target_code_type(phen_path, phenotype, target_code_type)
+ else:
+ for t in phenotype["map"]:
+ _map_target_code_type(phen_path, phenotype, t)
+
+ _logger.info(f"Phenotype processed successfully")
+
+
+def _map_target_code_type(phen_path: Path, phenotype: dict, target_code_type: str):
+ _logger.debug(f"Target coding format: {target_code_type}")
+ concepts_path = phen_path / CONCEPTS_DIR
# Create output dataframe
out = pd.DataFrame([])
code_errors = []
# Process each folder in codes section
for concept_set in phenotype["concept_sets"]:
- logger.debug(f"--- {concept_set['file']} ---")
+ _logger.debug(f"--- {concept_set['file']} ---")
# Load code file
- codes_file_path = Path(codes_path / concept_set["file"]["path"])
- df = read_table_file(codes_file_path)
+ codes_file_path = Path(concepts_path / concept_set["file"]["path"])
+ df = _read_table_file(codes_file_path)
# process structural actions
- df = process_actions(df, concept_set)
+ df = _process_actions(df, concept_set)
- # Preprocessing & Validation Checks
- logger.debug("Processing and validating code formats")
- df, errors = preprocess_codes(
+ # preprocessing and validate of source concepts
+ _logger.debug("Processing and validating source concept codes")
+ df, errors = _preprocess_source_concepts(
df,
concept_set,
codes_file_path,
- target_code_type=target_code_type,
)
- logger.debug(f"Length of errors from preprocess {len(errors)}")
+ # create df with just the source code columns
+ source_column_names = list(concept_set["file"]["columns"].keys())
+ source_df = df[source_column_names]
+
+ _logger.debug(source_df.columns)
+ _logger.debug(source_df.head())
+
+ _logger.debug(
+ f"Length of errors from _preprocess_source_concepts {len(errors)}"
+ )
if len(errors) > 0:
code_errors.extend(errors)
- logger.debug(f" Length of code_errors {len(code_errors)}")
+ _logger.debug(f" Length of code_errors {len(code_errors)}")
- # Map
+ # Map source concepts codes to target codes
# if processing a source coding list with categorical data
if (
"actions" in concept_set["file"]
@@ -594,57 +854,104 @@ def map(phen_dir, target_code_type):
and len(df) > 0
):
divide_col = concept_set["file"]["actions"]["divide_col"]
- logger.debug(f"Action: Dividing Table by {divide_col}")
- logger.debug(f"column into: {df[divide_col].unique()}")
+ _logger.debug(f"Action: Dividing Table by {divide_col}")
+ _logger.debug(f"column into: {df[divide_col].unique()}")
df_grp = df.groupby(divide_col)
for cat, grp in df_grp:
if cat == concept_set["file"]["category"]:
grp = grp.drop(columns=[divide_col]) # delete categorical column
- out = map_file(
- grp, target_code_type, out, concept_name=concept_set["name"]
+ source_df = grp[source_column_names]
+ trans_out = translate_codes(
+ source_df,
+ target_code_type=target_code_type,
+ concept_name=concept_set["name"],
)
+ out = pd.concat([out, trans_out])
else:
- out = map_file(df, target_code_type, out, concept_name=concept_set["name"])
+ source_df = df[source_column_names]
+ trans_out = translate_codes(
+ source_df,
+ target_code_type=target_code_type,
+ concept_name=concept_set["name"],
+ )
+ out = pd.concat([out, trans_out])
if len(code_errors) > 0:
- logger.error(f"The map processing has {len(code_errors)} errors")
+ _logger.error(f"The map processing has {len(code_errors)} errors")
error_path = phen_path / MAP_DIR / "errors"
error_path.mkdir(parents=True, exist_ok=True)
error_filename = f"{target_code_type}-code-errors.csv"
- write_code_errors(code_errors, error_path / error_filename)
+ _write_code_errors(code_errors, error_path / error_filename)
# Check there is output from processing
if len(out.index) == 0:
- logger.error(f"No output after map processing")
+ _logger.error(f"No output after map processing")
raise Exception(
- f"No output after map processing, check config {str(config_path.resolve())}"
+ f"No output after map processing, check config {str(phen_path.resolve())}"
)
- # Final processing
+ # final processing
out = out.reset_index(drop=True)
out = out.drop_duplicates(subset=["CONCEPT_SET", "CONCEPT"])
out = out.sort_values(by=["CONCEPT_SET", "CONCEPT"])
+ out_count = len(out.index)
+ # added metadata
+ # Loop over each source_concept_type and perform the left join on all columns apart from source code columns
+ result_list = []
+ source_column_names = list(concept_set["file"]["columns"].keys())
+ for source_concept_type in source_column_names:
+ # Filter output based on the current source_concept_type
+ out_filtered_df = out[out["SOURCE_CONCEPT_TYPE"] == source_concept_type]
+ filtered_count = len(out_filtered_df.index)
+
+ # Remove the source type columns except the current type will leave the metadata and the join
+ remove_types = [
+ type for type in source_column_names if type != source_concept_type
+ ]
+ metadata_df = df.drop(columns=remove_types)
+ metadata_df = metadata_df.rename(
+ columns={source_concept_type: "SOURCE_CONCEPT"}
+ )
+ metadata_df_count = len(metadata_df.index)
+
+ # Perform the left join with df2 on SOURCE_CONCEPT to add the metadata
+ result = pd.merge(out_filtered_df, metadata_df, how="left", on="SOURCE_CONCEPT")
+ result_count = len(result.index)
+
+ _logger.debug(
+ f"Adding metadata for {source_concept_type}: out_count {out_count}, filtered_count {filtered_count}, metadata_df_count {metadata_df_count}, result_count {result_count}"
+ )
+
+ # Append the result to the result_list
+ result_list.append(result)
+
+ # Concatenate all the results into a single DataFrame
+ final_out = pd.concat(result_list, ignore_index=True)
+ final_out = final_out.drop_duplicates(subset=["CONCEPT_SET", "CONCEPT"])
+ _logger.debug(
+ f"Check metadata processing counts: before {len(out.index)} : after {len(final_out.index)}"
+ )
+
# Save output to map directory
output_filename = target_code_type + ".csv"
map_path = phen_path / MAP_DIR / output_filename
- out.to_csv(map_path, index=False)
- logger.info(f"Saved mapped concepts to {str(map_path.resolve())}")
+ final_out.to_csv(map_path, index=False)
+ _logger.info(f"Saved mapped concepts to {str(map_path.resolve())}")
# save concept sets as separate files
- concept_set_path = phen_path / CONCEPT_SET_DIR / target_code_type
+ concept_set_path = phen_path / CSV_PATH / target_code_type
- # empty the concept-set directory if it exists but keep the .git file
- git_items = [".git", ".gitkeep"]
+ # empty the concept-set directory except for hiddle files, e.g. .git
if concept_set_path.exists():
for item in concept_set_path.iterdir():
- if item not in git_items:
+ if not item.name.startswith("."):
item.unlink()
else:
concept_set_path.mkdir(parents=True, exist_ok=True)
# write each concept as a separate file
- for name, concept in out.groupby("CONCEPT_SET"):
+ for name, concept in final_out.groupby("CONCEPT_SET"):
concept = concept.sort_values(by="CONCEPT") # sort rows
concept = concept.dropna(how="all", axis=1) # remove empty cols
concept = concept.reindex(
@@ -656,10 +963,42 @@ def map(phen_dir, target_code_type):
write_vocab_version(phen_path)
- logger.info(f"Phenotype processed successfully")
+ _logger.info(f"Phenotype processed target code type {target_code_type}")
+
+
+def _generate_version_tag(
+ repo: git.Repo, increment: str = DEFAULT_VERSION_INC, use_v_prefix: bool = False
+) -> str:
+ # Get all valid semantic version tags
+ versions = []
+ for tag in repo.tags:
+ tag_name = (
+ tag.name.lstrip("v") if use_v_prefix else tag.name
+ ) # Remove 'v' if needed
+ if semver.Version.is_valid(tag_name):
+ versions.append(semver.Version.parse(tag_name))
+
+ # Determine the next version
+ if not versions:
+ new_version = semver.Version(0, 0, 1)
+ else:
+ latest_version = max(versions)
+ if increment == "major":
+ new_version = latest_version.bump_major()
+ elif increment == "minor":
+ new_version = latest_version.bump_minor()
+ else:
+ new_version = latest_version.bump_patch()
+ # Create the new tag
+ new_version_str = f"v{new_version}" if use_v_prefix else str(new_version)
-def publish(phen_dir):
+ return new_version_str
+
+
+def publish(
+ phen_dir: str, msg: str, remote_url: str, increment: str = DEFAULT_VERSION_INC
+):
"""Publishes updates to the phenotype by commiting all changes to the repo directory"""
# Validate config
@@ -678,53 +1017,68 @@ def publish(phen_dir):
# check if any changes to publish
if not repo.is_dirty() and not repo.untracked_files:
- logger.info("Nothing to publish, no changes to the repo")
- return
+ if remote_url is not None and "origin" not in repo.remotes:
+ _logger.info(f"First publish to remote url {remote_url}")
+ else:
+ _logger.info("Nothing to publish, no changes to the repo")
+ return
+
+ # get next version
+ new_version_str = _generate_version_tag(repo, increment)
+ _logger.info(f"New version: {new_version_str}")
- # get major version from configuration file
+ # Write version in configuration file
config_path = phen_path / CONFIG_FILE
with config_path.open("r") as file:
config = yaml.safe_load(file)
- match = re.match(r"v(\d+\.\d+)", config["phenotype"]["version"])
- major_version = match.group(1)
-
- # get latest minor version from git commit count
- commit_count = len(list(repo.iter_commits("HEAD")))
- # set version and write to config file so consistent with repo version
- next_minor_version = commit_count + 1
- version = f"v{major_version}.{next_minor_version}"
- logger.debug(f"New version: {version}")
- config["phenotype"]["version"] = version
+ config["phenotype"]["version"] = new_version_str
with open(config_path, "w") as file:
- yaml.dump(config, file, default_flow_style=False, sort_keys=False)
+ yaml.dump(
+ config,
+ file,
+ Dumper=util.QuotedDumper,
+ default_flow_style=False,
+ sort_keys=False,
+ default_style='"',
+ )
- # Add and commit changes to repo
+ # Add and commit changes to repo including version updates
commit_message = f"Committing updates to phenotype {phen_path}"
repo.git.add("--all")
repo.index.commit(commit_message)
- # Create and push the tag
- if version in repo.tags:
- raise Exception(f"Tag {version} already exists in repo {phen_path}")
- repo.create_tag(version, message=f"Release {version}")
- logger.info(f"New version: {version}")
+ # Add tag to the repo
+ repo.create_tag(new_version_str)
# push to origin if a remote repo
+ if remote_url is not None and "origin" not in repo.remotes:
+ git_url = _construct_git_url(remote_url)
+ repo.create_remote("origin", git_url)
+
try:
- origin = repo.remotes.origin
- origin.push("main")
- origin.push(tags=True)
- logger.debug("Changes pushed to 'origin'.")
- except AttributeError:
- logger.debug("No remote named 'origin' found, local repo.")
+ if "origin" in repo.remotes:
+ _logger.debug(f"Remote 'origin' is set {repo.remotes.origin.url}")
+ origin = repo.remotes.origin
+ _logger.info(f"Pushing main branch to remote repo")
+ repo.git.push("--set-upstream", "origin", "main")
+ _logger.info(f"Pushing version tags to remote git repo")
+ origin.push(tags=True)
+ _logger.debug("Changes pushed to 'origin'")
+ else:
+ _logger.debug("Remote 'origin' is not set")
+ except Exception as e:
+ tag_ref = repo.tags[new_version_str]
+ repo.delete_tag(tag_ref)
+ repo.git.reset("--soft", "HEAD~1")
+ raise e
- logger.info(f"Phenotype published successfully")
+ _logger.info(f"Phenotype published successfully")
-def export(phen_dir, version):
+def export(phen_dir: str, version: str):
"""Exports a phen repo at a specific tagged version into a target directory"""
- logger.info(f"Exporting phenotype {phen_dir} at version {version}")
+ _logger.info(f"Exporting phenotype {phen_dir} at version {version}")
# validate configuration
validate(phen_dir)
@@ -737,13 +1091,13 @@ def export(phen_dir, version):
map_path = phen_path / MAP_DIR
if not map_path.exists():
- logger.warning(f"Map path does not exist '{map_path}'")
+ _logger.warning(f"Map path does not exist '{map_path}'")
- export_path = phen_path / OMOP_DIR
+ export_path = phen_path / OMOP_PATH
# check export directory exists and if not create it
if not export_path.exists():
export_path.mkdir(parents=True)
- logger.debug(f"OMOP export directory '{export_path}' created.")
+ _logger.debug(f"OMOP export directory '{export_path}' created.")
# omop export db
export_db_path = omop.export(
@@ -755,10 +1109,10 @@ def export(phen_dir, version):
# write to tables
# export as csv
- logger.info(f"Phenotype exported successfully")
+ _logger.info(f"Phenotype exported successfully")
-def copy(phen_dir, target_dir, version):
+def copy(phen_dir: str, target_dir: str, version: str):
"""Copys a phen repo at a specific tagged version into a target directory"""
# Validate
@@ -772,55 +1126,138 @@ def copy(phen_dir, target_dir, version):
# Set copy directory
copy_path = target_path / version
- logger.info(f"Copying repo {phen_path} to {copy_path}")
+ _logger.info(f"Copying repo {phen_path} to {copy_path}")
- if not copy_path.exists():
- # If copy directory doesn't exist, clone the repo
- logger.debug(f"Cloning repo from {phen_path} into {copy_path}...")
- repo = git.Repo.clone_from(phen_path, copy_path)
- else:
- # If copy directory exists, open the repo
- logger.debug(
- f"Copy of repository already exists in {copy_path}. Opening the repo..."
+ if (
+ copy_path.exists() and copy_path.is_dir()
+ ): # Check if it exists and is a directory
+ copy = _check_delete_dir(
+ copy_path,
+ f"The directory {str(copy_path.resolve())} already exists. Do you want to overwrite? (yes/no): ",
)
- repo = git.Repo(copy_path)
+ else:
+ copy = True
+
+ if not copy:
+ _logger.info(f"Not copying the version {version}")
+ return
+
+ _logger.debug(f"Cloning repo from {phen_path} into {copy_path}...")
+ repo = git.Repo.clone_from(phen_path, copy_path)
# Check out the latest commit or specified version
if version:
# Checkout a specific version (e.g., branch, tag, or commit hash)
- logger.info(f"Checking out version {version}...")
+ _logger.info(f"Checking out version {version}...")
repo.git.checkout(version)
else:
# Checkout the latest commit (HEAD)
- logger.info(f"Checking out the latest commit...")
+ _logger.info(f"Checking out the latest commit...")
repo.git.checkout("HEAD")
- logger.debug(f"Copied {phen_path} {repo.head.commit.hexsha[:7]} in {copy_path}")
+ _logger.debug(f"Copied {phen_path} {repo.head.commit.hexsha[:7]} in {copy_path}")
- logger.info(f"Phenotype copied successfully")
+ _logger.info(f"Phenotype copied successfully")
-def diff(phen_dir, phen_old_dir):
- """Compare the differences between two versions of a phenotype"""
+# Convert concept_sets list into dictionaries
+def extract_concepts(config_data: dict) -> Tuple[dict, Set[str]]:
+ """Extracts concepts as {name: file_path} dictionary and a name set."""
+ concepts_dict = {
+ item["name"]: item["file"]["path"]
+ for item in config_data["phenotype"]["concept_sets"]
+ }
+ name_set = set(concepts_dict.keys())
+ return concepts_dict, name_set
- # validate phenotype directories
- validate(phen_old_dir)
- validate(phen_dir)
- old_phen_path = Path(phen_old_dir)
- new_phen_path = Path(phen_dir)
+def _extract_clean_deepdiff_keys(diff: dict, key_type: str) -> Set[Any]:
+ """
+ Extracts clean keys from a DeepDiff dictionary.
- # Load report (FOR SOME REASON THIS WAS APPEND SO SET TO w for NOW)
- report_file_name = old_phen_path.name + "_diff.md"
- report_path = new_phen_path / report_file_name
- report = open(report_path, "w")
- logger.debug(f"Writing to report file {str(report_path.resolve())}")
+ :param diff: DeepDiff result dictionary
+ :param key_type: The type of change to extract (e.g., "dictionary_item_added", "dictionary_item_removed")
+ :return: A set of clean key names
+ """
+ return {key.split("root['")[1].split("']")[0] for key in diff.get(key_type, [])}
- # Get maps files from phenotype
- old_map_path = old_phen_path / MAP_DIR
- new_map_path = new_phen_path / MAP_DIR
- # List files from output directories
+def diff_config(old_config: dict, new_config: dict) -> str:
+ report = f"\n# Changes to phenotype configuration\n"
+ report += f"This compares changes in the phenotype configuration including added, removed and renamed concept sets and changes to concept set source concept code file paths\n\n"
+
+ old_concepts, old_names = extract_concepts(old_config)
+ new_concepts, new_names = extract_concepts(new_config)
+
+ # Check added and removed names
+ added_names = new_names - old_names # Names that appear in new but not in old
+ removed_names = old_names - new_names # Names that were in old but not in new
+
+ # find file path changes for unchanged names
+ unchanged_names = old_names & new_names # Names that exist in both
+ file_diff = DeepDiff(
+ {name: old_concepts[name] for name in unchanged_names},
+ {name: new_concepts[name] for name in unchanged_names},
+ )
+
+ # Find renamed concepts (same file, different name)
+ renamed_concepts = []
+ for removed in removed_names:
+ old_path = old_concepts[removed]
+ for added in added_names:
+ new_path = new_concepts[added]
+ if old_path == new_path:
+ renamed_concepts.append((removed, added))
+
+ # Remove renamed concepts from added and removed sets
+ for old_name, new_name in renamed_concepts:
+ added_names.discard(new_name)
+ removed_names.discard(old_name)
+
+ # generate config report
+ if added_names:
+ report += "## Added Concepts\n"
+ for name in added_names:
+ report += f"- `{name}` (File: `{new_concepts[name]}`)\n"
+ report += "\n"
+
+ if removed_names:
+ report += "## Removed Concepts\n"
+ for name in removed_names:
+ report += f"- `{name}` (File: `{old_concepts[name]}`)\n"
+ report += "\n"
+
+ if renamed_concepts:
+ report += "## Renamed Concepts\n"
+ for old_name, new_name in renamed_concepts:
+ report += (
+ f"- `{old_name}` ➝ `{new_name}` (File: `{old_concepts[old_name]}`)\n"
+ )
+ report += "\n"
+
+ if "values_changed" in file_diff:
+ report += "## Updated File Paths\n"
+ for name, change in file_diff["values_changed"].items():
+ old_file = change["old_value"]
+ new_file = change["new_value"]
+ clean_name = name.split("root['")[1].split("']")[0]
+ report += (
+ f"- `{clean_name}` changed file from `{old_file}` ➝ `{new_file}`\n"
+ )
+ report += "\n"
+
+ if not (
+ added_names
+ or removed_names
+ or renamed_concepts
+ or file_diff.get("values_changed")
+ ):
+ report += "No changes in concept sets.\n"
+
+ return report
+
+
+def diff_map_files(old_map_path: Path, new_map_path: Path) -> str:
old_output_files = [
file.name
for file in old_map_path.iterdir()
@@ -843,25 +1280,21 @@ def diff(phen_dir, phen_old_dir):
# Outputs that are the intersection of old_output_set and new_output_set
common_outputs = old_output_set & new_output_set
- # Write outputs report
- new_config = new_phen_path / CONFIG_FILE
- with new_config.open("r") as file:
- new_config = yaml.safe_load(file)
- report.write(f"\n\n# Report for version {new_config['phenotype']['version']}\n\n")
- report.write(f"- Removed outputs: {list(removed_outputs)}\n")
- report.write(f"- Added outputs: {list(added_outputs)}\n")
- report.write(f"- Common outputs: {list(common_outputs)}\n")
+ report = f"\n# Changes to available translations\n"
+ report += f"This compares the coding translations files available.\n\n"
+ report += f"- Removed outputs: {sorted(list(removed_outputs))}\n"
+ report += f"- Added outputs: {sorted(list(added_outputs))}\n"
+ report += f"- Common outputs: {sorted(list(common_outputs))}\n\n"
- report.write(
- f"\n\n## Compare concepts {str(old_phen_path.resolve())} to {str(new_phen_path.resolve())}\n\n"
- )
- # Compare common outputs between versions
+ # Step N: Compare common outputs between versions
+ report += f"# Changes to concepts in translation files\n\n"
+ report += f"This compares the added and removed concepts in each of the coding translation files. Note that this might be different to the config.yaml if the translations have not been run for the current config.\n\n"
for file in common_outputs:
old_output = old_map_path / file
new_output = new_map_path / file
- logger.debug(f"Old ouptput: {str(old_output.resolve())}")
- logger.debug(f"New ouptput: {str(new_output.resolve())}")
+ _logger.debug(f"Old ouptput: {str(old_output.resolve())}")
+ _logger.debug(f"New ouptput: {str(new_output.resolve())}")
df1 = pd.read_csv(old_output)
df1 = df1[["CONCEPT", "CONCEPT_SET"]].groupby("CONCEPT_SET").count()
@@ -869,12 +1302,11 @@ def diff(phen_dir, phen_old_dir):
df2 = df2[["CONCEPT", "CONCEPT_SET"]].groupby("CONCEPT_SET").count()
# Check for added and removed concepts
- report.write(
- "- Removed concepts {}\n".format(list(set(df1.index) - set(df2.index)))
- )
- report.write(
- "- Added concepts {}\n".format(list(set(df2.index) - set(df1.index)))
- )
+ report += f"- File {file}\n"
+ sorted_list = sorted(list(set(df1.index) - set(df2.index)))
+ report += f"- Removed concepts {sorted_list}\n"
+ sorted_list = sorted(list(set(df2.index) - set(df1.index)))
+ report += f"- Added concepts {sorted_list}\n"
# Check for changed concepts
diff = df2 - df1 # diff in counts
@@ -885,8 +1317,127 @@ def diff(phen_dir, phen_old_dir):
if len(diff.index) > 0:
for concept, row in diff.iterrows():
s += "\t - {} {}\n".format(concept, row["CONCEPT"])
- report.write(f"- Changed concepts {s}\n\n")
+ report += f"- Changed concepts {s}\n\n"
+ else:
+ report += f"- Changed concepts []\n\n"
+
+ return report
+
+
+def diff_phen(
+ new_phen_path: Path,
+ new_version: str,
+ old_phen_path: Path,
+ old_version: str,
+ report_path: Path,
+):
+ """Compare the differences between two versions of a phenotype"""
+
+ # validate phenotypes
+ _logger.debug(f"Validating for diff old path: {str(old_phen_path.resolve())}")
+ validate(str(old_phen_path.resolve()))
+ _logger.debug(f"Validating for diff new path: {str(new_phen_path.resolve())}")
+ validate(str(new_phen_path.resolve()))
+
+ # get old and new config
+ old_config_path = old_phen_path / CONFIG_FILE
+ with old_config_path.open("r") as file:
+ old_config = yaml.safe_load(file)
+ new_config_path = new_phen_path / CONFIG_FILE
+ with new_config_path.open("r") as file:
+ new_config = yaml.safe_load(file)
+
+ # write report heading
+ report = f"# Phenotype Comparison Report\n"
+ report += f"## Original phenotype\n"
+ report += f" - {old_config['phenotype']['omop']['vocabulary_id']}\n"
+ report += f" - {old_version}\n"
+ report += f" - {str(old_phen_path.resolve())}\n"
+ report += f"## Changed phenotype:\n"
+ report += f" - {new_config['phenotype']['omop']['vocabulary_id']}\n"
+ report += f" - {new_version}\n"
+ report += f" - {str(new_phen_path.resolve())}\n"
+
+ # Step 1: check differences configuration files
+ # Convert list of dicts into a dict: {name: file}
+ report += diff_config(old_config, new_config)
+
+ # Step 2: check differences between map files
+ # List files from output directories
+ old_map_path = old_phen_path / MAP_DIR
+ new_map_path = new_phen_path / MAP_DIR
+ report += diff_map_files(old_map_path, new_map_path)
+
+ # initialise report file
+ _logger.debug(f"Writing to report file {str(report_path.resolve())}")
+ report_file = open(report_path, "w")
+ report_file.write(report)
+ report_file.close()
+
+ _logger.info(f"Phenotypes diff'd successfully")
+
+
+def diff(phen_dir: str, version: str, old_phen_dir: str, old_version: str):
+ # make tmp directory .acmc
+ timestamp = time.strftime("%Y%m%d_%H%M%S")
+ temp_dir = Path(f".acmc/diff_{timestamp}")
+
+ changed_phen_path = Path(phen_dir)
+ if not changed_phen_path.exists():
+ raise ValueError(
+ f"Changed phenotype directory does not exist: {str(changed_phen_path.resolve())}"
+ )
+
+ old_phen_path = Path(old_phen_dir)
+ if not old_phen_path.exists():
+ raise ValueError(
+ f"Old phenotype directory does not exist: {str(old_phen_path.resolve())}"
+ )
+
+ try:
+ # Create the directory
+ temp_dir.mkdir(parents=True, exist_ok=True)
+ _logger.debug(f"Temporary directory created: {temp_dir}")
+
+ # Create temporary directories
+ changed_path = temp_dir / "changed"
+ changed_path.mkdir(parents=True, exist_ok=True)
+ old_path = temp_dir / "old"
+ old_path.mkdir(parents=True, exist_ok=True)
+
+ # checkout changed
+ if version == "latest":
+ _logger.debug(
+ f"Copying changed repo from {phen_dir} into {changed_path} at version {version}..."
+ )
+ shutil.copytree(changed_phen_path, changed_path, dirs_exist_ok=True)
else:
- report.write(f"- Changed concepts []\n\n")
+ _logger.debug(
+ f"Cloning changed repo from {phen_dir} into {changed_path} at version {version}..."
+ )
+ changed_repo = git.Repo.clone_from(changed_phen_path, changed_path)
+ changed_repo.git.checkout(version)
- logger.info(f"Phenotypes diff'd successfully")
+ # checkout old
+ if old_version == "latest":
+ _logger.debug(
+ f"Copying old repo from {old_phen_dir} into {old_path} at version {old_version}..."
+ )
+ shutil.copytree(old_phen_path, old_path, dirs_exist_ok=True)
+ else:
+ _logger.debug(
+ f"Cloning old repo from {old_phen_dir} into {old_path} at version {old_version}..."
+ )
+ old_repo = git.Repo.clone_from(old_phen_dir, old_path)
+ old_repo.git.checkout(old_version)
+
+ report_filename = f"{version}_{old_version}_diff.md"
+ report_path = changed_phen_path / report_filename
+ # diff old with new
+ diff_phen(changed_path, version, old_path, old_version, report_path)
+
+ finally:
+ # clean up tmp directory
+ if temp_dir.exists():
+ shutil.rmtree(temp_dir)
+ print(f"Temporary directory removed: {temp_dir}")
diff --git a/acmc/trud.py b/acmc/trud.py
index d8852cc5b424c20e7271bd6162e849671f5ba840..926efffad7e3e018cc8039a4329e3a05705e704d 100644
--- a/acmc/trud.py
+++ b/acmc/trud.py
@@ -1,3 +1,10 @@
+"""
+trud.py module
+
+This module provides functionality to manage installation of the NHS TRUD vocabularies.
+
+"""
+
import os
import sys
import requests
@@ -6,30 +13,34 @@ import shutil
import hashlib
import zipfile
import pandas as pd
-import simpledbf # type: ignore
+import simpledbf # type: ignore
import yaml
from pathlib import Path
+from acmc import util, logging_config as lc
# setup logging
-import acmc.logging_config as lc
+_logger = lc.setup_logger()
-logger = lc.setup_logger()
-
-# Constants
FQDN = "isd.digital.nhs.uk"
+"""Fully Qualified Domain Name of NHS digital TRUD service API"""
+
VOCAB_PATH = Path("./vocab/trud")
-VERSION_FILE = "trud_version.yaml"
+"""Default path to the TRUD vocabulary directory relative to the the acmc execution directory"""
+
+VERSION_FILE = "trud_version.yml"
+"""TRUD version file"""
+
VERSION_PATH = VOCAB_PATH / VERSION_FILE
-DOWNLOADS_PATH = VOCAB_PATH / "downloads"
-PROCESSED_PATH = VOCAB_PATH / "processed"
+"""Default path to the TRUD version file"""
+DOWNLOADS_PATH = VOCAB_PATH / "downloads"
+"""Default path to the TRUD vocabulary downloads directory"""
-def error_exit(message):
- logger.error(message, "error")
- sys.exit(1)
+PROCESSED_PATH = VOCAB_PATH / "processed"
+""" Default path to the processed TRUD mappings directory"""
-def get_releases(item_id, API_KEY, latest=False):
+def get_releases(item_id: str, API_KEY: str, latest=False) -> list:
"""Retrieve release information for an item from the TRUD API."""
url = f"https://{FQDN}/trud/api/v1/keys/{API_KEY}/items/{item_id}/releases"
if latest:
@@ -37,20 +48,23 @@ def get_releases(item_id, API_KEY, latest=False):
response = requests.get(url)
if response.status_code != 200:
- error_exit(
+ _logger.error(
f"Failed to fetch releases for item {item_id}. Status code: {response.status_code}, error {response.json()['message']}. If no releases found for API key, please ensure you are subscribed to the data release and that it is not pending approval"
)
+ response.raise_for_status()
data = response.json()
if data.get("message") != "OK":
- error_exit(data.get("message", "Unknown error occurred"))
+ msg = f"Unknown error occurred {data.get('message')}"
+ _logger.error(msg)
+ raise Exception(msg)
return data.get("releases", [])
def download_release_file(
- item_id, release_ordinal, release, file_json_prefix, file_type=None
-):
+ item_id: str, release_ordinal: str, release: dict, file_json_prefix: str
+) -> Path:
"""Download specified file type for a given release of an item."""
# check folder is a directory
@@ -59,17 +73,23 @@ def download_release_file(
f"Error: '{DOWNLOADS_PATH}' for TRUD resources is not a directory"
)
- file_type = file_type or file_json_prefix
+ file_type = file_json_prefix
file_url = release.get(f"{file_json_prefix}FileUrl")
+ if file_url == None:
+ raise ValueError(f"File url not in json data {file_json_prefix}FileUrl")
+
file_name = release.get(f"{file_json_prefix}FileName")
+ if file_name == None:
+ raise ValueError(f"File name not in json data {file_json_prefix}FileName")
+
file_destination = DOWNLOADS_PATH / file_name
if not file_url or not file_name:
- error_exit(
+ raise ValueError(
f"Missing {file_type} file information for release {release_ordinal} of item {item_id}."
)
- logger.info(
+ _logger.info(
f"Downloading item {item_id} {file_type} file: {file_name} from {file_url} to {file_destination}"
)
response = requests.get(file_url, stream=True)
@@ -77,27 +97,28 @@ def download_release_file(
if response.status_code == 200:
with open(file_destination, "wb") as f:
f.write(response.content)
- return file_destination
else:
- error_exit(
+ _logger.error(
f"Failed to download {file_type} file for item {item_id}. Status code: {response.status_code}"
)
+ response.raise_for_status()
+
+ return file_destination
def validate_download_hash(file_destination: str, item_hash: str):
with open(file_destination, "rb") as f:
hash = hashlib.sha256(f.read()).hexdigest()
- logger.debug(hash)
+ _logger.debug(hash)
if hash.upper() == item_hash.upper():
- logger.debug(f"Verified hash of {file_destination} {hash}")
+ _logger.debug(f"Verified hash of {file_destination} {hash}")
else:
- error_exit(
- f"Could not validate origin of {file_destination}. The SHA-256 hash should be: {item_hash}, but got {hash} instead"
- )
+ msg = f"Could not validate origin of {file_destination}. The SHA-256 hash should be: {item_hash}, but got {hash} instead"
+ _logger.error(msg)
+ raise ValueError(msg)
def unzip_download(file_destination: str):
-
# check folder is a directory
if not DOWNLOADS_PATH.is_dir():
raise NotADirectoryError(
@@ -123,7 +144,7 @@ def extract_icd10():
)
output_path = PROCESSED_PATH / "icd10.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
def extract_opsc4():
@@ -138,7 +159,7 @@ def extract_opsc4():
output_path = PROCESSED_PATH / "opcs4.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
def extract_nhs_data_migrations():
@@ -160,7 +181,7 @@ def extract_nhs_data_migrations():
output_path = PROCESSED_PATH / "snomed.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r2 -> r3
file_path = (
@@ -176,7 +197,7 @@ def extract_nhs_data_migrations():
output_path = PROCESSED_PATH / "read2_to_read3.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r3->r2
file_path = (
@@ -194,7 +215,7 @@ def extract_nhs_data_migrations():
output_path = PROCESSED_PATH / "read3_to_read2.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r2 -> snomed
file_path = (
@@ -210,7 +231,7 @@ def extract_nhs_data_migrations():
output_path = PROCESSED_PATH / "read2_to_snomed.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r3->snomed
file_path = (
@@ -228,7 +249,7 @@ def extract_nhs_data_migrations():
output_path = PROCESSED_PATH / "read3_to_snomed.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
def extract_nhs_read_browser():
@@ -240,7 +261,7 @@ def extract_nhs_read_browser():
df = df.rename(columns={0: "read2"})
output_path = PROCESSED_PATH / "read2.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r2 -> atc
input_path = DOWNLOADS_PATH / "Standard" / "V2" / "ATC.DBF"
@@ -249,7 +270,7 @@ def extract_nhs_read_browser():
df = df.rename(columns={"READCODE": "read2", "ATC": "atc"})
output_path = PROCESSED_PATH / "read2_to_atc.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r2 -> icd10
input_path = DOWNLOADS_PATH / "Standard" / "V2" / "ICD10.DBF"
@@ -260,7 +281,7 @@ def extract_nhs_read_browser():
df = df[~df["read2"].str.match("^.*-.*$")] # remove codes with '-'
output_path = PROCESSED_PATH / "read2_to_icd10.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r2 -> opcs4
input_path = DOWNLOADS_PATH / "Standard" / "V2" / "OPCS4V3.DBF"
@@ -271,7 +292,7 @@ def extract_nhs_read_browser():
df = df[~df["read2"].str.match("^.*-.*$")] # remove codes with '-'
output_path = PROCESSED_PATH / "read2_to_opcs4.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r3 only
input_path = DOWNLOADS_PATH / "Standard" / "V3" / "ANCESTOR.DBF"
@@ -281,7 +302,7 @@ def extract_nhs_read_browser():
df = df.rename(columns={0: "read3"})
output_path = PROCESSED_PATH / "read3.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r3 -> icd10
input_path = DOWNLOADS_PATH / "Standard" / "V3" / "ICD10.DBF"
@@ -292,7 +313,7 @@ def extract_nhs_read_browser():
df = df[~df["read3"].str.match("^.*-.*$")] # remove codes with '-'
output_path = PROCESSED_PATH / "read3_to_icd10.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
# r3 -> icd9
# dbf = simpledbf.Dbf5('build/maps/downloads/Standard/V3/ICD9V3.DBF')
@@ -306,7 +327,7 @@ def extract_nhs_read_browser():
df = df[~df["read3"].str.match("^.*-.*$")] # remove codes with '-'
output_path = PROCESSED_PATH / "read3_to_opcs4.parquet"
df.to_parquet(output_path, index=False)
- logger.info(f"Extracted: {output_path}")
+ _logger.info(f"Extracted: {output_path}")
def create_map_directories():
@@ -327,7 +348,7 @@ def create_map_directories():
shutil.rmtree(VOCAB_PATH)
create_map_dirs = True
elif user_input == "n":
- logger.info("Exiting TRUD installation")
+ _logger.info("Exiting TRUD installation")
sys.exit(0)
else:
create_map_dirs = True
@@ -340,7 +361,7 @@ def create_map_directories():
def install():
- logger.info(f"Installing TRUD")
+ _logger.info(f"Installing TRUD")
# get TRUD api key from environment variable
api_key = os.getenv("ACMC_TRUD_API_KEY")
@@ -384,16 +405,23 @@ def install():
data = [{k: v for k, v in d.items() if k != "extract"} for d in items]
# save TRUD versions to file to main record of what was downloaded
with open(VERSION_PATH, "w") as file:
- yaml.dump(data, file, default_flow_style=False, sort_keys=False)
+ yaml.dump(
+ data,
+ file,
+ Dumper=util.QuotedDumper,
+ default_flow_style=False,
+ sort_keys=False,
+ default_style='"',
+ )
# Validate and process each item ID
for item in items:
item_id = item["id"]
- logger.info(f"--- {item['name']} ---")
+ _logger.info(f"--- {item['name']} ---")
releases = get_releases(item_id, API_KEY=api_key, latest=items_latest)
if not releases:
- error_exit(f"No releases found for item {item_id}.")
+ raise ValueError(f"No releases found for item {item_id}.")
# Process each release in reverse order
for release_ordinal, release in enumerate(releases[::-1], 1):
@@ -421,6 +449,6 @@ def install():
if "extract" in item:
item["extract"]()
- logger.info(f"Downloaded {release_ordinal} release(s) for item {item_id}.")
+ _logger.info(f"Downloaded {release_ordinal} release(s) for item {item_id}.")
- logger.info(f"TRUD installation completed")
+ _logger.info(f"TRUD installation completed")
diff --git a/acmc/util.py b/acmc/util.py
new file mode 100644
index 0000000000000000000000000000000000000000..958a4a8845ad4cac85ed4dc087ba9e74f8ca4877
--- /dev/null
+++ b/acmc/util.py
@@ -0,0 +1,15 @@
+import yaml
+
+"""
+util.py module
+
+This module provides general utility functions for the acmc modules
+
+"""
+
+
+class QuotedDumper(yaml.Dumper):
+ """Custom Dumper to retain quotes on strings in yaml library"""
+
+ def increase_indent(self, flow=False, indentless=False):
+ return super(QuotedDumper, self).increase_indent(flow, indentless)
diff --git a/docs/contributing.md b/contributing.md
similarity index 76%
rename from docs/contributing.md
rename to contributing.md
index 1c6ba5ecfdc8608ba2d51f0f346364c6afb4253c..07fadecfff120a7aaa8ffdae83206c7b9fe2d9e3 100644
--- a/docs/contributing.md
+++ b/contributing.md
@@ -1,4 +1,6 @@
-# Contributing - Development Guide
+# Contributing
+
+This document summaries the development guide for contributing to the ACMC project.
## Prerequisites
To install the ACMC package for development, ensure you have Python 3.9 or later installed.
@@ -67,6 +69,19 @@ To run tests using `pytest`, use:
hatch run pytest
```
+To run a specific test and display print outputs `-s`, use
+
+```sh
+hatch run pytest -s -v tests/test_acmc.py::<test_function_name>
+```
+
+## Running the CLI
+The package provides a command-line interface (CLI) entry point.
+
+```sh
+acmc --help
+```
+
### All code checks
The project run all type and formatting checking
@@ -86,14 +101,14 @@ hatch run black --check .
To format the coode and modify the files, use
```sh
-hatch run black .
+hatch run format
```
### Type Checking
The project uses `mypy` for type checking:
```sh
-hatch run mypy .
+hatch run type
```
## Building the Package
@@ -103,16 +118,62 @@ To build the package, use:
hatch build
```
-## Running the CLI
-The package provides a command-line interface (CLI) entry point.
+## Releasing the Package
+
+Update the version number in pyproject.toml
+
+Build the package using:
```sh
-acmc --help
+hatch build
```
-## Contributing
+Creates file `acmc-<version>-py3-none-any.whl` in the `./dist` directory
+
+Check package using:
+
+```sh
+twine check dist/acmc-<version>-py3-none-any.whl
+```
+Upload to pypi using:
+
+```sh
+twine upload dist/acmc-<version>-py3-none-any.whl --username __token__ --password <your pypi token>
+```
+
+## Updating docs
+
+Docs are created using mkdocs and published at readthedocs.org. mkdocs is configured with `mkdocs.yaml` in the root repo directory. Mkdocs includes the navigation structure of the document linking to general docs and API docs.
+
+Docs are published at readthedocs https://acmc.readthedocs.io/en/latest/. readthedocs is configured with `readthedocs.yml` in the root repo directory.
+
+### General docs
+
+General docs are written in markdown and stored within the `docs` directory, e.g. the user guide is `./docs/index.md`
+
+### API docs
+
+API documentation are automatically generated from docstrings using pdoc, through the following command
+
+```bash
+hatch run docs
+```
+
+### Publishing to readthedocs
+
+1. Get an account a [readthedocs](https://app.readthedocs.org/dashboard)
+
+2. Ask to become a maintainer of the acmc project
+
+3. Select the acmc project from the dash board
+
+4. Select builds and press the rebuild button
+
+5. Check the documentation at https://acmc.readthedocs.io/en/latest/
+
+## GitLab Workflows
-### GitLab Basic Workflow Overview
+### Basic
1. Create an new issue in the [Issue Tracker](https://git.soton.ac.uk/meldb/concepts-processing/-/issues)
@@ -146,7 +207,7 @@ You can also create a new branch in your local repo if it does not exist and it
If you make changes, ensure all tests pass and code formatting is correct before submitting a merge request.
-## GitLab Workflow for Managing Releases
+## Branches
### 1. Main Branch (`main`)
- Represents the stable production-ready code.
@@ -192,7 +253,7 @@ If you make changes, ensure all tests pass and code formatting is correct before
- Deploy to production after approval.
- Requires authorization from a repository maintainer or admin before merging.
-### Commit Message Guidelines
+## Commit Message Guidelines
- Use clear and descriptive commit messages.
- Format: `<type>(<scope>): <description>`
- Example: `docs: git workflow documentation (#issue)`
diff --git a/docs/api.md b/docs/api.md
deleted file mode 100644
index fe113c6a1500759d413611699ecd0b4002ec9902..0000000000000000000000000000000000000000
--- a/docs/api.md
+++ /dev/null
@@ -1,3 +0,0 @@
-# API Reference
-
-::: acmc.module_name
\ No newline at end of file
diff --git a/docs/api/acmc.html b/docs/api/acmc.html
new file mode 100644
index 0000000000000000000000000000000000000000..7fb0357e7df7298fcf5b2846ddb12b75f7af2d2c
--- /dev/null
+++ b/docs/api/acmc.html
@@ -0,0 +1,249 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div>
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+ <h2>Submodules</h2>
+ <ul>
+ <li><a href="acmc/logging_config.html">logging_config</a></li>
+ <li><a href="acmc/main.html">main</a></li>
+ <li><a href="acmc/omop.html">omop</a></li>
+ <li><a href="acmc/parse.html">parse</a></li>
+ <li><a href="acmc/phen.html">phen</a></li>
+ <li><a href="acmc/trud.html">trud</a></li>
+ <li><a href="acmc/util.html">util</a></li>
+ </ul>
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+acmc </h1>
+
+
+ <input id="mod-acmc-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-acmc-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos">1</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">importlib.metadata</span><span class="w"> </span><span class="kn">import</span> <span class="n">version</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos">2</span></a>
+</span><span id="L-3"><a href="#L-3"><span class="linenos">3</span></a><span class="n">__version__</span> <span class="o">=</span> <span class="n">version</span><span class="p">(</span><span class="s2">"acmc"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/logging_config.html b/docs/api/acmc/logging_config.html
new file mode 100644
index 0000000000000000000000000000000000000000..c2ef24cc4e36f5d56fa3ff265a445b72f715843e
--- /dev/null
+++ b/docs/api/acmc/logging_config.html
@@ -0,0 +1,416 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.logging_config API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#DEFAULT_LOG_FILE">DEFAULT_LOG_FILE</a>
+ </li>
+ <li>
+ <a class="function" href="#setup_logger">setup_logger</a>
+ </li>
+ <li>
+ <a class="function" href="#set_log_level">set_log_level</a>
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.logging_config </h1>
+
+ <div class="docstring"><p>logging_config.py</p>
+
+<p>This module defines functions to setup logging for acmc across all module.</p>
+</div>
+
+ <input id="mod-logging_config-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-logging_config-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="sd">"""</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="sd">logging_config.py</span>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">This module defines functions to setup logging for acmc across all module. </span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="sd">"""</span>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">logging</span>
+</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a>
+</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="n">DEFAULT_LOG_FILE</span> <span class="o">=</span> <span class="s2">"acmc.log"</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="sd">"""The default acmc application log filename."""</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a>
+</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a>
+</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a><span class="k">def</span><span class="w"> </span><span class="nf">setup_logger</span><span class="p">(</span><span class="n">log_level</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">):</span>
+</span><span id="L-16"><a href="#L-16"><span class="linenos">16</span></a><span class="w"> </span><span class="sd">"""Sets up acmc logger as a singleton outputing to file and sysout syserr."""</span>
+</span><span id="L-17"><a href="#L-17"><span class="linenos">17</span></a>
+</span><span id="L-18"><a href="#L-18"><span class="linenos">18</span></a> <span class="c1"># Create a logger</span>
+</span><span id="L-19"><a href="#L-19"><span class="linenos">19</span></a> <span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">"acmc_logger"</span><span class="p">)</span>
+</span><span id="L-20"><a href="#L-20"><span class="linenos">20</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
+</span><span id="L-21"><a href="#L-21"><span class="linenos">21</span></a>
+</span><span id="L-22"><a href="#L-22"><span class="linenos">22</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">logger</span><span class="o">.</span><span class="n">hasHandlers</span><span class="p">():</span>
+</span><span id="L-23"><a href="#L-23"><span class="linenos">23</span></a> <span class="c1"># Create a file handler that logs to a file</span>
+</span><span id="L-24"><a href="#L-24"><span class="linenos">24</span></a> <span class="n">file_handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="n">DEFAULT_LOG_FILE</span><span class="p">)</span>
+</span><span id="L-25"><a href="#L-25"><span class="linenos">25</span></a> <span class="n">file_handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
+</span><span id="L-26"><a href="#L-26"><span class="linenos">26</span></a>
+</span><span id="L-27"><a href="#L-27"><span class="linenos">27</span></a> <span class="c1"># Create a stream handler that prints to the console</span>
+</span><span id="L-28"><a href="#L-28"><span class="linenos">28</span></a> <span class="n">stream_handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
+</span><span id="L-29"><a href="#L-29"><span class="linenos">29</span></a> <span class="n">stream_handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
+</span><span id="L-30"><a href="#L-30"><span class="linenos">30</span></a>
+</span><span id="L-31"><a href="#L-31"><span class="linenos">31</span></a> <span class="c1"># Create a formatter for how the log messages should look</span>
+</span><span id="L-32"><a href="#L-32"><span class="linenos">32</span></a> <span class="c1"># Add the formatter to both handlers</span>
+</span><span id="L-33"><a href="#L-33"><span class="linenos">33</span></a> <span class="n">file_formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span>
+</span><span id="L-34"><a href="#L-34"><span class="linenos">34</span></a> <span class="s2">"</span><span class="si">%(asctime)s</span><span class="s2"> - - </span><span class="si">%(levelname)s</span><span class="s2"> - </span><span class="si">%(message)s</span><span class="s2">"</span>
+</span><span id="L-35"><a href="#L-35"><span class="linenos">35</span></a> <span class="p">)</span>
+</span><span id="L-36"><a href="#L-36"><span class="linenos">36</span></a> <span class="n">file_handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">file_formatter</span><span class="p">)</span>
+</span><span id="L-37"><a href="#L-37"><span class="linenos">37</span></a> <span class="n">stream_formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s2">"[</span><span class="si">%(levelname)s</span><span class="s2">] - </span><span class="si">%(message)s</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-38"><a href="#L-38"><span class="linenos">38</span></a> <span class="n">stream_handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">stream_formatter</span><span class="p">)</span>
+</span><span id="L-39"><a href="#L-39"><span class="linenos">39</span></a>
+</span><span id="L-40"><a href="#L-40"><span class="linenos">40</span></a> <span class="c1"># Add the handlers to the logger</span>
+</span><span id="L-41"><a href="#L-41"><span class="linenos">41</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">file_handler</span><span class="p">)</span>
+</span><span id="L-42"><a href="#L-42"><span class="linenos">42</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">stream_handler</span><span class="p">)</span>
+</span><span id="L-43"><a href="#L-43"><span class="linenos">43</span></a>
+</span><span id="L-44"><a href="#L-44"><span class="linenos">44</span></a> <span class="k">return</span> <span class="n">logger</span>
+</span><span id="L-45"><a href="#L-45"><span class="linenos">45</span></a>
+</span><span id="L-46"><a href="#L-46"><span class="linenos">46</span></a>
+</span><span id="L-47"><a href="#L-47"><span class="linenos">47</span></a><span class="k">def</span><span class="w"> </span><span class="nf">set_log_level</span><span class="p">(</span><span class="n">log_level</span><span class="p">:</span> <span class="nb">int</span><span class="p">):</span>
+</span><span id="L-48"><a href="#L-48"><span class="linenos">48</span></a><span class="w"> </span><span class="sd">"""Sets the log level for the acmc logger.</span>
+</span><span id="L-49"><a href="#L-49"><span class="linenos">49</span></a>
+</span><span id="L-50"><a href="#L-50"><span class="linenos">50</span></a><span class="sd"> Args:</span>
+</span><span id="L-51"><a href="#L-51"><span class="linenos">51</span></a><span class="sd"> log_level (int): log level from the python logging libraru</span>
+</span><span id="L-52"><a href="#L-52"><span class="linenos">52</span></a>
+</span><span id="L-53"><a href="#L-53"><span class="linenos">53</span></a><span class="sd"> """</span>
+</span><span id="L-54"><a href="#L-54"><span class="linenos">54</span></a>
+</span><span id="L-55"><a href="#L-55"><span class="linenos">55</span></a> <span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">"acmc_logger"</span><span class="p">)</span>
+</span><span id="L-56"><a href="#L-56"><span class="linenos">56</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">log_level</span><span class="p">)</span>
+</span><span id="L-57"><a href="#L-57"><span class="linenos">57</span></a>
+</span><span id="L-58"><a href="#L-58"><span class="linenos">58</span></a> <span class="c1"># Also update handlers to match the new level</span>
+</span><span id="L-59"><a href="#L-59"><span class="linenos">59</span></a> <span class="k">for</span> <span class="n">handler</span> <span class="ow">in</span> <span class="n">logger</span><span class="o">.</span><span class="n">handlers</span><span class="p">:</span>
+</span><span id="L-60"><a href="#L-60"><span class="linenos">60</span></a> <span class="n">handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">log_level</span><span class="p">)</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="DEFAULT_LOG_FILE">
+ <div class="attr variable">
+ <span class="name">DEFAULT_LOG_FILE</span> =
+<span class="default_value">'acmc.log'</span>
+
+
+ </div>
+ <a class="headerlink" href="#DEFAULT_LOG_FILE"></a>
+
+ <div class="docstring"><p>The default acmc application log filename.</p>
+</div>
+
+
+ </section>
+ <section id="setup_logger">
+ <input id="setup_logger-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">setup_logger</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">log_level</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="mi">20</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="setup_logger-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#setup_logger"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="setup_logger-16"><a href="#setup_logger-16"><span class="linenos">16</span></a><span class="k">def</span><span class="w"> </span><span class="nf">setup_logger</span><span class="p">(</span><span class="n">log_level</span><span class="p">:</span> <span class="nb">int</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">):</span>
+</span><span id="setup_logger-17"><a href="#setup_logger-17"><span class="linenos">17</span></a><span class="w"> </span><span class="sd">"""Sets up acmc logger as a singleton outputing to file and sysout syserr."""</span>
+</span><span id="setup_logger-18"><a href="#setup_logger-18"><span class="linenos">18</span></a>
+</span><span id="setup_logger-19"><a href="#setup_logger-19"><span class="linenos">19</span></a> <span class="c1"># Create a logger</span>
+</span><span id="setup_logger-20"><a href="#setup_logger-20"><span class="linenos">20</span></a> <span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">"acmc_logger"</span><span class="p">)</span>
+</span><span id="setup_logger-21"><a href="#setup_logger-21"><span class="linenos">21</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
+</span><span id="setup_logger-22"><a href="#setup_logger-22"><span class="linenos">22</span></a>
+</span><span id="setup_logger-23"><a href="#setup_logger-23"><span class="linenos">23</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">logger</span><span class="o">.</span><span class="n">hasHandlers</span><span class="p">():</span>
+</span><span id="setup_logger-24"><a href="#setup_logger-24"><span class="linenos">24</span></a> <span class="c1"># Create a file handler that logs to a file</span>
+</span><span id="setup_logger-25"><a href="#setup_logger-25"><span class="linenos">25</span></a> <span class="n">file_handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">FileHandler</span><span class="p">(</span><span class="n">DEFAULT_LOG_FILE</span><span class="p">)</span>
+</span><span id="setup_logger-26"><a href="#setup_logger-26"><span class="linenos">26</span></a> <span class="n">file_handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
+</span><span id="setup_logger-27"><a href="#setup_logger-27"><span class="linenos">27</span></a>
+</span><span id="setup_logger-28"><a href="#setup_logger-28"><span class="linenos">28</span></a> <span class="c1"># Create a stream handler that prints to the console</span>
+</span><span id="setup_logger-29"><a href="#setup_logger-29"><span class="linenos">29</span></a> <span class="n">stream_handler</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">StreamHandler</span><span class="p">()</span>
+</span><span id="setup_logger-30"><a href="#setup_logger-30"><span class="linenos">30</span></a> <span class="n">stream_handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">INFO</span><span class="p">)</span>
+</span><span id="setup_logger-31"><a href="#setup_logger-31"><span class="linenos">31</span></a>
+</span><span id="setup_logger-32"><a href="#setup_logger-32"><span class="linenos">32</span></a> <span class="c1"># Create a formatter for how the log messages should look</span>
+</span><span id="setup_logger-33"><a href="#setup_logger-33"><span class="linenos">33</span></a> <span class="c1"># Add the formatter to both handlers</span>
+</span><span id="setup_logger-34"><a href="#setup_logger-34"><span class="linenos">34</span></a> <span class="n">file_formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span>
+</span><span id="setup_logger-35"><a href="#setup_logger-35"><span class="linenos">35</span></a> <span class="s2">"</span><span class="si">%(asctime)s</span><span class="s2"> - - </span><span class="si">%(levelname)s</span><span class="s2"> - </span><span class="si">%(message)s</span><span class="s2">"</span>
+</span><span id="setup_logger-36"><a href="#setup_logger-36"><span class="linenos">36</span></a> <span class="p">)</span>
+</span><span id="setup_logger-37"><a href="#setup_logger-37"><span class="linenos">37</span></a> <span class="n">file_handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">file_formatter</span><span class="p">)</span>
+</span><span id="setup_logger-38"><a href="#setup_logger-38"><span class="linenos">38</span></a> <span class="n">stream_formatter</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">Formatter</span><span class="p">(</span><span class="s2">"[</span><span class="si">%(levelname)s</span><span class="s2">] - </span><span class="si">%(message)s</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="setup_logger-39"><a href="#setup_logger-39"><span class="linenos">39</span></a> <span class="n">stream_handler</span><span class="o">.</span><span class="n">setFormatter</span><span class="p">(</span><span class="n">stream_formatter</span><span class="p">)</span>
+</span><span id="setup_logger-40"><a href="#setup_logger-40"><span class="linenos">40</span></a>
+</span><span id="setup_logger-41"><a href="#setup_logger-41"><span class="linenos">41</span></a> <span class="c1"># Add the handlers to the logger</span>
+</span><span id="setup_logger-42"><a href="#setup_logger-42"><span class="linenos">42</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">file_handler</span><span class="p">)</span>
+</span><span id="setup_logger-43"><a href="#setup_logger-43"><span class="linenos">43</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">addHandler</span><span class="p">(</span><span class="n">stream_handler</span><span class="p">)</span>
+</span><span id="setup_logger-44"><a href="#setup_logger-44"><span class="linenos">44</span></a>
+</span><span id="setup_logger-45"><a href="#setup_logger-45"><span class="linenos">45</span></a> <span class="k">return</span> <span class="n">logger</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Sets up acmc logger as a singleton outputing to file and sysout syserr.</p>
+</div>
+
+
+ </section>
+ <section id="set_log_level">
+ <input id="set_log_level-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">set_log_level</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">log_level</span><span class="p">:</span> <span class="nb">int</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="set_log_level-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#set_log_level"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="set_log_level-48"><a href="#set_log_level-48"><span class="linenos">48</span></a><span class="k">def</span><span class="w"> </span><span class="nf">set_log_level</span><span class="p">(</span><span class="n">log_level</span><span class="p">:</span> <span class="nb">int</span><span class="p">):</span>
+</span><span id="set_log_level-49"><a href="#set_log_level-49"><span class="linenos">49</span></a><span class="w"> </span><span class="sd">"""Sets the log level for the acmc logger.</span>
+</span><span id="set_log_level-50"><a href="#set_log_level-50"><span class="linenos">50</span></a>
+</span><span id="set_log_level-51"><a href="#set_log_level-51"><span class="linenos">51</span></a><span class="sd"> Args:</span>
+</span><span id="set_log_level-52"><a href="#set_log_level-52"><span class="linenos">52</span></a><span class="sd"> log_level (int): log level from the python logging libraru</span>
+</span><span id="set_log_level-53"><a href="#set_log_level-53"><span class="linenos">53</span></a>
+</span><span id="set_log_level-54"><a href="#set_log_level-54"><span class="linenos">54</span></a><span class="sd"> """</span>
+</span><span id="set_log_level-55"><a href="#set_log_level-55"><span class="linenos">55</span></a>
+</span><span id="set_log_level-56"><a href="#set_log_level-56"><span class="linenos">56</span></a> <span class="n">logger</span> <span class="o">=</span> <span class="n">logging</span><span class="o">.</span><span class="n">getLogger</span><span class="p">(</span><span class="s2">"acmc_logger"</span><span class="p">)</span>
+</span><span id="set_log_level-57"><a href="#set_log_level-57"><span class="linenos">57</span></a> <span class="n">logger</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">log_level</span><span class="p">)</span>
+</span><span id="set_log_level-58"><a href="#set_log_level-58"><span class="linenos">58</span></a>
+</span><span id="set_log_level-59"><a href="#set_log_level-59"><span class="linenos">59</span></a> <span class="c1"># Also update handlers to match the new level</span>
+</span><span id="set_log_level-60"><a href="#set_log_level-60"><span class="linenos">60</span></a> <span class="k">for</span> <span class="n">handler</span> <span class="ow">in</span> <span class="n">logger</span><span class="o">.</span><span class="n">handlers</span><span class="p">:</span>
+</span><span id="set_log_level-61"><a href="#set_log_level-61"><span class="linenos">61</span></a> <span class="n">handler</span><span class="o">.</span><span class="n">setLevel</span><span class="p">(</span><span class="n">log_level</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Sets the log level for the acmc logger.</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>log_level (int):</strong> log level from the python logging libraru</li>
+</ul>
+</div>
+
+
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/main.html b/docs/api/acmc/main.html
new file mode 100644
index 0000000000000000000000000000000000000000..665fb689a9dcd5a7c1a5bc01671de363bad1360c
--- /dev/null
+++ b/docs/api/acmc/main.html
@@ -0,0 +1,875 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.main API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#DEFAULT_WORKSPACE_PATH">DEFAULT_WORKSPACE_PATH</a>
+ </li>
+ <li>
+ <a class="function" href="#main">main</a>
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.main </h1>
+
+ <div class="docstring"><p>main.py module</p>
+
+<p>This module provides the functionality for the acmc command line interface</p>
+</div>
+
+ <input id="mod-main-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-main-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="sd">"""</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="sd">main.py module</span>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">This module provides the functionality for the acmc command line interface</span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="sd">"""</span>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">argparse</span>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">logging</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">pathlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">Path</span>
+</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a>
+</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">acmc</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">acmc</span><span class="w"> </span><span class="kn">import</span> <span class="n">trud</span><span class="p">,</span> <span class="n">omop</span><span class="p">,</span> <span class="n">phen</span><span class="p">,</span> <span class="n">parse</span><span class="p">,</span> <span class="n">logging_config</span> <span class="k">as</span> <span class="n">lc</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a>
+</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a>
+</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="n">DEFAULT_WORKSPACE_PATH</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s2">"./workspace"</span><span class="p">)</span>
+</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="sd">"""Default phenotype workspace path"""</span>
+</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a>
+</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a>
+</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_trud_install</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="w"> </span><span class="sd">"""Handle the `trud install` command."""</span>
+</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a> <span class="n">trud</span><span class="o">.</span><span class="n">install</span><span class="p">()</span>
+</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a>
+</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a>
+</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_omop_install</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="w"> </span><span class="sd">"""Handle the `omop install` command."""</span>
+</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="n">omop</span><span class="o">.</span><span class="n">install</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">omop_zip_file</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">version</span><span class="p">)</span>
+</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
+</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a>
+</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_omop_clear</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="w"> </span><span class="sd">"""Handle the `omop clear` command."""</span>
+</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a> <span class="n">omop</span><span class="o">.</span><span class="n">clear</span><span class="p">(</span><span class="n">omop</span><span class="o">.</span><span class="n">DB_PATH</span><span class="p">)</span>
+</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a>
+</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a>
+</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_omop_delete</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="w"> </span><span class="sd">"""Handle the `omop delete` command."""</span>
+</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="n">omop</span><span class="o">.</span><span class="n">delete</span><span class="p">(</span><span class="n">omop</span><span class="o">.</span><span class="n">DB_PATH</span><span class="p">)</span>
+</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a>
+</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
+</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_init</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="w"> </span><span class="sd">"""Handle the `phen init` command."""</span>
+</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">init</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">remote_url</span><span class="p">)</span>
+</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a>
+</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a>
+</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_fork</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="w"> </span><span class="sd">"""Handle the `phen fork` command."""</span>
+</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">fork</span><span class="p">(</span>
+</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span>
+</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">args</span><span class="o">.</span><span class="n">upstream_url</span><span class="p">,</span>
+</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">args</span><span class="o">.</span><span class="n">upstream_version</span><span class="p">,</span>
+</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="n">new_origin_url</span><span class="o">=</span><span class="n">args</span><span class="o">.</span><span class="n">remote_url</span><span class="p">,</span>
+</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="p">)</span>
+</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a>
+</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a>
+</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_validate</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="w"> </span><span class="sd">"""Handle the `phen validate` command."""</span>
+</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">validate</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a>
+</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a>
+</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_map</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a><span class="w"> </span><span class="sd">"""Handle the `phen map` command."""</span>
+</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">target_coding</span><span class="p">)</span>
+</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
+</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
+</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_export</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a><span class="w"> </span><span class="sd">"""Handle the `phen copy` command."""</span>
+</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">export</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">version</span><span class="p">)</span>
+</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a>
+</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
+</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_publish</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="w"> </span><span class="sd">"""Handle the `phen publish` command."""</span>
+</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">publish</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">msg</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">remote_url</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">increment</span><span class="p">)</span>
+</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a>
+</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
+</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_copy</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="w"> </span><span class="sd">"""Handle the `phen copy` command."""</span>
+</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">target_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">version</span><span class="p">)</span>
+</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
+</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a>
+</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_phen_diff</span><span class="p">(</span><span class="n">args</span><span class="p">:</span> <span class="n">argparse</span><span class="o">.</span><span class="n">Namespace</span><span class="p">):</span>
+</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a><span class="w"> </span><span class="sd">"""Handle the `phen diff` command."""</span>
+</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">phen</span><span class="o">.</span><span class="n">diff</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">version</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">old_phen_dir</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">old_version</span><span class="p">)</span>
+</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
+</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a>
+</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="k">def</span><span class="w"> </span><span class="nf">main</span><span class="p">():</span>
+</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">parser</span> <span class="o">=</span> <span class="n">argparse</span><span class="o">.</span><span class="n">ArgumentParser</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s2">"ACMC command-line tool"</span><span class="p">)</span>
+</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a> <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"--debug"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s2">"store_true"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Enable debug mode"</span><span class="p">)</span>
+</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="s2">"--version"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s2">"version"</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="sa">f</span><span class="s2">"acmc </span><span class="si">{</span><span class="n">acmc</span><span class="o">.</span><span class="n">__version__</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="p">)</span>
+</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a>
+</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="c1"># Top-level commands</span>
+</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">subparsers</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"command"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Available commands"</span>
+</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="p">)</span>
+</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a>
+</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="c1">### TRUD Command ###</span>
+</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="n">trud_parser</span> <span class="o">=</span> <span class="n">subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"trud"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"TRUD commands"</span><span class="p">)</span>
+</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">trud_subparsers</span> <span class="o">=</span> <span class="n">trud_parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"subcommand"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"TRUD subcommands"</span>
+</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="p">)</span>
+</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a>
+</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="c1"># trud install</span>
+</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">trud_install_parser</span> <span class="o">=</span> <span class="n">trud_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="s2">"install"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Install TRUD components"</span>
+</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="p">)</span>
+</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">trud_install_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_trud_install</span><span class="p">)</span>
+</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
+</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="c1">### OMOP Command ###</span>
+</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">omop_parser</span> <span class="o">=</span> <span class="n">subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"omop"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"OMOP commands"</span><span class="p">)</span>
+</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">omop_subparsers</span> <span class="o">=</span> <span class="n">omop_parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"subcommand"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"OMOP subcommands"</span>
+</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="p">)</span>
+</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a>
+</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="c1"># omop install</span>
+</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">omop_install_parser</span> <span class="o">=</span> <span class="n">omop_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="s2">"install"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Install OMOP codes within database"</span>
+</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="p">)</span>
+</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">omop_install_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a> <span class="s2">"-f"</span><span class="p">,</span> <span class="s2">"--omop-zip-file"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Path to downloaded OMOP zip file"</span>
+</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="p">)</span>
+</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">omop_install_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="s2">"-v"</span><span class="p">,</span> <span class="s2">"--version"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"OMOP vocabularies release version"</span>
+</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="p">)</span>
+</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">omop_install_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_omop_install</span><span class="p">)</span>
+</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a>
+</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="c1"># omop clear</span>
+</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="n">omop_clear_parser</span> <span class="o">=</span> <span class="n">omop_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="s2">"clear"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Clear OMOP data from database"</span>
+</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="p">)</span>
+</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="n">omop_clear_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_omop_clear</span><span class="p">)</span>
+</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
+</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="c1"># omop delete</span>
+</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">omop_delete_parser</span> <span class="o">=</span> <span class="n">omop_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="s2">"delete"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Delete OMOP database"</span>
+</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="p">)</span>
+</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">omop_delete_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_omop_delete</span><span class="p">)</span>
+</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
+</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="c1">### PHEN Command ###</span>
+</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">phen_parser</span> <span class="o">=</span> <span class="n">subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"phen"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Phen commands"</span><span class="p">)</span>
+</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">phen_subparsers</span> <span class="o">=</span> <span class="n">phen_parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"subcommand"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Phen subcommands"</span>
+</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="p">)</span>
+</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
+</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="c1"># phen init</span>
+</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">phen_init_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="s2">"init"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Initiatise phenotype directory"</span>
+</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="p">)</span>
+</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="n">phen_init_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="p">)</span>
+</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="n">phen_init_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="s2">"-r"</span><span class="p">,</span>
+</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="s2">"--remote_url"</span><span class="p">,</span>
+</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) URL to repository where the phenotype will be published."</span><span class="p">,</span>
+</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="p">)</span>
+</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">phen_init_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_init</span><span class="p">)</span>
+</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a>
+</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="c1"># phen fork</span>
+</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">phen_fork_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="s2">"fork"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Fork an existing phenotype"</span>
+</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="p">)</span>
+</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="p">)</span>
+</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="s2">"-r"</span><span class="p">,</span>
+</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="s2">"--remote_url"</span><span class="p">,</span>
+</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) URL to repository where the forked phenotype will be published."</span><span class="p">,</span>
+</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="p">)</span>
+</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="s2">"-u"</span><span class="p">,</span>
+</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="s2">"--upstream-url"</span><span class="p">,</span>
+</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Required) URL to the phenotype repository to fork."</span><span class="p">,</span>
+</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="p">)</span>
+</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="s2">"--upstream-version"</span><span class="p">,</span>
+</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Required) Phenotype version to fork."</span><span class="p">,</span>
+</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="p">)</span>
+</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_fork</span><span class="p">)</span>
+</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a>
+</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="c1"># phen validate</span>
+</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">phen_validate_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="s2">"validate"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Validate phenotype configuration"</span>
+</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="p">)</span>
+</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">phen_validate_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="p">)</span>
+</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">phen_validate_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_validate</span><span class="p">)</span>
+</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a>
+</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="c1"># phen map</span>
+</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="n">phen_map_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"map"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Process phen mapping"</span><span class="p">)</span>
+</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="n">phen_map_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="p">)</span>
+</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="n">phen_map_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="s2">"-t"</span><span class="p">,</span>
+</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="s2">"--target-coding"</span><span class="p">,</span>
+</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="n">choices</span><span class="o">=</span><span class="n">parse</span><span class="o">.</span><span class="n">SUPPORTED_CODE_TYPES</span><span class="p">,</span>
+</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">help</span><span class="o">=</span><span class="sa">f</span><span class="s2">"Specify the target coding </span><span class="si">{</span><span class="n">parse</span><span class="o">.</span><span class="n">SUPPORTED_CODE_TYPES</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span>
+</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="p">)</span>
+</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="n">phen_map_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_map</span><span class="p">)</span>
+</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a>
+</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="c1"># phen export</span>
+</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="n">phen_export_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="s2">"export"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Export phen to OMOP database"</span>
+</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="p">)</span>
+</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="n">phen_export_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="p">)</span>
+</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="n">phen_export_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="s2">"--version"</span><span class="p">,</span>
+</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="n">default</span><span class="o">=</span><span class="s2">"latest"</span><span class="p">,</span>
+</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Phenotype version to export, defaults to the latest version"</span><span class="p">,</span>
+</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="p">)</span>
+</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="n">phen_export_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_export</span><span class="p">)</span>
+</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a>
+</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="c1"># phen publish</span>
+</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="n">phen_publish_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="s2">"publish"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Publish phenotype configuration"</span>
+</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="p">)</span>
+</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="p">)</span>
+</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="s2">"-i"</span><span class="p">,</span>
+</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="s2">"--increment"</span><span class="p">,</span>
+</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="n">default</span><span class="o">=</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_VERSION_INC</span><span class="p">,</span>
+</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="n">choices</span><span class="o">=</span><span class="n">phen</span><span class="o">.</span><span class="n">SEMANTIC_VERSION_TYPES</span><span class="p">,</span>
+</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="n">help</span><span class="o">=</span><span class="sa">f</span><span class="s2">"Version increment: </span><span class="si">{</span><span class="n">phen</span><span class="o">.</span><span class="n">SEMANTIC_VERSION_TYPES</span><span class="si">}</span><span class="s2">, default is </span><span class="si">{</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_VERSION_INC</span><span class="si">}</span><span class="s2"> increment"</span><span class="p">,</span>
+</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="p">)</span>
+</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="s2">"-m"</span><span class="p">,</span> <span class="s2">"--msg"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Message to include with the published version"</span>
+</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="p">)</span>
+</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a> <span class="s2">"-r"</span><span class="p">,</span> <span class="s2">"--remote_url"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"URL to remote git repository"</span>
+</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="p">)</span>
+</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_publish</span><span class="p">)</span>
+</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a>
+</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="c1"># phen copy</span>
+</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="n">phen_copy_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="s2">"copy"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Publish phenotype configuration"</span>
+</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="p">)</span>
+</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="p">)</span>
+</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="s2">"-td"</span><span class="p">,</span>
+</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="s2">"--target-dir"</span><span class="p">,</span>
+</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">DEFAULT_WORKSPACE_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Target directory for the copy"</span><span class="p">,</span>
+</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="p">)</span>
+</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a> <span class="s2">"--version"</span><span class="p">,</span>
+</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="n">default</span><span class="o">=</span><span class="s2">"latest"</span><span class="p">,</span>
+</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Phenotype version to copy, defaults to the latest version"</span><span class="p">,</span>
+</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="p">)</span>
+</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_copy</span><span class="p">)</span>
+</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a>
+</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="c1"># phen diff</span>
+</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="n">phen_diff_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="s2">"diff"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Publish phenotype configuration"</span>
+</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="p">)</span>
+</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="p">)</span>
+</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="s2">"--version"</span><span class="p">,</span>
+</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="n">default</span><span class="o">=</span><span class="s2">"latest"</span><span class="p">,</span>
+</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Phenotype version to compare with an old version, defaults to the HEAD of the workspace directory"</span><span class="p">,</span>
+</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="p">)</span>
+</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="s2">"-od"</span><span class="p">,</span>
+</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="s2">"--old-phen-dir"</span><span class="p">,</span>
+</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Directory for the old phenotype version, defaults to workspace directory"</span><span class="p">,</span>
+</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="p">)</span>
+</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="s2">"-ov"</span><span class="p">,</span>
+</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="s2">"--old-version"</span><span class="p">,</span>
+</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Old phenotype version to compare with the changed version"</span><span class="p">,</span>
+</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="p">)</span>
+</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_diff</span><span class="p">)</span>
+</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a>
+</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a> <span class="c1"># Parse arguments</span>
+</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="n">args</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">parse_args</span><span class="p">()</span>
+</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a>
+</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a> <span class="c1"># setup logging</span>
+</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="k">if</span> <span class="n">args</span><span class="o">.</span><span class="n">debug</span><span class="p">:</span>
+</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="n">lc</span><span class="o">.</span><span class="n">set_log_level</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
+</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a>
+</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a> <span class="c1"># Call the function associated with the command</span>
+</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="n">args</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="n">args</span><span class="p">)</span>
+</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a>
+</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a>
+</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
+</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="n">main</span><span class="p">()</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="DEFAULT_WORKSPACE_PATH">
+ <div class="attr variable">
+ <span class="name">DEFAULT_WORKSPACE_PATH</span> =
+<span class="default_value">PosixPath('workspace')</span>
+
+
+ </div>
+ <a class="headerlink" href="#DEFAULT_WORKSPACE_PATH"></a>
+
+ <div class="docstring"><p>Default phenotype workspace path</p>
+</div>
+
+
+ </section>
+ <section id="main">
+ <input id="main-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">main</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="main-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#main"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="main-85"><a href="#main-85"><span class="linenos"> 85</span></a><span class="k">def</span><span class="w"> </span><span class="nf">main</span><span class="p">():</span>
+</span><span id="main-86"><a href="#main-86"><span class="linenos"> 86</span></a> <span class="n">parser</span> <span class="o">=</span> <span class="n">argparse</span><span class="o">.</span><span class="n">ArgumentParser</span><span class="p">(</span><span class="n">description</span><span class="o">=</span><span class="s2">"ACMC command-line tool"</span><span class="p">)</span>
+</span><span id="main-87"><a href="#main-87"><span class="linenos"> 87</span></a> <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"--debug"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s2">"store_true"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Enable debug mode"</span><span class="p">)</span>
+</span><span id="main-88"><a href="#main-88"><span class="linenos"> 88</span></a> <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-89"><a href="#main-89"><span class="linenos"> 89</span></a> <span class="s2">"--version"</span><span class="p">,</span> <span class="n">action</span><span class="o">=</span><span class="s2">"version"</span><span class="p">,</span> <span class="n">version</span><span class="o">=</span><span class="sa">f</span><span class="s2">"acmc </span><span class="si">{</span><span class="n">acmc</span><span class="o">.</span><span class="n">__version__</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="main-90"><a href="#main-90"><span class="linenos"> 90</span></a> <span class="p">)</span>
+</span><span id="main-91"><a href="#main-91"><span class="linenos"> 91</span></a>
+</span><span id="main-92"><a href="#main-92"><span class="linenos"> 92</span></a> <span class="c1"># Top-level commands</span>
+</span><span id="main-93"><a href="#main-93"><span class="linenos"> 93</span></a> <span class="n">subparsers</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="main-94"><a href="#main-94"><span class="linenos"> 94</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"command"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Available commands"</span>
+</span><span id="main-95"><a href="#main-95"><span class="linenos"> 95</span></a> <span class="p">)</span>
+</span><span id="main-96"><a href="#main-96"><span class="linenos"> 96</span></a>
+</span><span id="main-97"><a href="#main-97"><span class="linenos"> 97</span></a> <span class="c1">### TRUD Command ###</span>
+</span><span id="main-98"><a href="#main-98"><span class="linenos"> 98</span></a> <span class="n">trud_parser</span> <span class="o">=</span> <span class="n">subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"trud"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"TRUD commands"</span><span class="p">)</span>
+</span><span id="main-99"><a href="#main-99"><span class="linenos"> 99</span></a> <span class="n">trud_subparsers</span> <span class="o">=</span> <span class="n">trud_parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="main-100"><a href="#main-100"><span class="linenos">100</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"subcommand"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"TRUD subcommands"</span>
+</span><span id="main-101"><a href="#main-101"><span class="linenos">101</span></a> <span class="p">)</span>
+</span><span id="main-102"><a href="#main-102"><span class="linenos">102</span></a>
+</span><span id="main-103"><a href="#main-103"><span class="linenos">103</span></a> <span class="c1"># trud install</span>
+</span><span id="main-104"><a href="#main-104"><span class="linenos">104</span></a> <span class="n">trud_install_parser</span> <span class="o">=</span> <span class="n">trud_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-105"><a href="#main-105"><span class="linenos">105</span></a> <span class="s2">"install"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Install TRUD components"</span>
+</span><span id="main-106"><a href="#main-106"><span class="linenos">106</span></a> <span class="p">)</span>
+</span><span id="main-107"><a href="#main-107"><span class="linenos">107</span></a> <span class="n">trud_install_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_trud_install</span><span class="p">)</span>
+</span><span id="main-108"><a href="#main-108"><span class="linenos">108</span></a>
+</span><span id="main-109"><a href="#main-109"><span class="linenos">109</span></a> <span class="c1">### OMOP Command ###</span>
+</span><span id="main-110"><a href="#main-110"><span class="linenos">110</span></a> <span class="n">omop_parser</span> <span class="o">=</span> <span class="n">subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"omop"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"OMOP commands"</span><span class="p">)</span>
+</span><span id="main-111"><a href="#main-111"><span class="linenos">111</span></a> <span class="n">omop_subparsers</span> <span class="o">=</span> <span class="n">omop_parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="main-112"><a href="#main-112"><span class="linenos">112</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"subcommand"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"OMOP subcommands"</span>
+</span><span id="main-113"><a href="#main-113"><span class="linenos">113</span></a> <span class="p">)</span>
+</span><span id="main-114"><a href="#main-114"><span class="linenos">114</span></a>
+</span><span id="main-115"><a href="#main-115"><span class="linenos">115</span></a> <span class="c1"># omop install</span>
+</span><span id="main-116"><a href="#main-116"><span class="linenos">116</span></a> <span class="n">omop_install_parser</span> <span class="o">=</span> <span class="n">omop_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-117"><a href="#main-117"><span class="linenos">117</span></a> <span class="s2">"install"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Install OMOP codes within database"</span>
+</span><span id="main-118"><a href="#main-118"><span class="linenos">118</span></a> <span class="p">)</span>
+</span><span id="main-119"><a href="#main-119"><span class="linenos">119</span></a> <span class="n">omop_install_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-120"><a href="#main-120"><span class="linenos">120</span></a> <span class="s2">"-f"</span><span class="p">,</span> <span class="s2">"--omop-zip-file"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Path to downloaded OMOP zip file"</span>
+</span><span id="main-121"><a href="#main-121"><span class="linenos">121</span></a> <span class="p">)</span>
+</span><span id="main-122"><a href="#main-122"><span class="linenos">122</span></a> <span class="n">omop_install_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-123"><a href="#main-123"><span class="linenos">123</span></a> <span class="s2">"-v"</span><span class="p">,</span> <span class="s2">"--version"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"OMOP vocabularies release version"</span>
+</span><span id="main-124"><a href="#main-124"><span class="linenos">124</span></a> <span class="p">)</span>
+</span><span id="main-125"><a href="#main-125"><span class="linenos">125</span></a> <span class="n">omop_install_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_omop_install</span><span class="p">)</span>
+</span><span id="main-126"><a href="#main-126"><span class="linenos">126</span></a>
+</span><span id="main-127"><a href="#main-127"><span class="linenos">127</span></a> <span class="c1"># omop clear</span>
+</span><span id="main-128"><a href="#main-128"><span class="linenos">128</span></a> <span class="n">omop_clear_parser</span> <span class="o">=</span> <span class="n">omop_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-129"><a href="#main-129"><span class="linenos">129</span></a> <span class="s2">"clear"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Clear OMOP data from database"</span>
+</span><span id="main-130"><a href="#main-130"><span class="linenos">130</span></a> <span class="p">)</span>
+</span><span id="main-131"><a href="#main-131"><span class="linenos">131</span></a> <span class="n">omop_clear_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_omop_clear</span><span class="p">)</span>
+</span><span id="main-132"><a href="#main-132"><span class="linenos">132</span></a>
+</span><span id="main-133"><a href="#main-133"><span class="linenos">133</span></a> <span class="c1"># omop delete</span>
+</span><span id="main-134"><a href="#main-134"><span class="linenos">134</span></a> <span class="n">omop_delete_parser</span> <span class="o">=</span> <span class="n">omop_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-135"><a href="#main-135"><span class="linenos">135</span></a> <span class="s2">"delete"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Delete OMOP database"</span>
+</span><span id="main-136"><a href="#main-136"><span class="linenos">136</span></a> <span class="p">)</span>
+</span><span id="main-137"><a href="#main-137"><span class="linenos">137</span></a> <span class="n">omop_delete_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_omop_delete</span><span class="p">)</span>
+</span><span id="main-138"><a href="#main-138"><span class="linenos">138</span></a>
+</span><span id="main-139"><a href="#main-139"><span class="linenos">139</span></a> <span class="c1">### PHEN Command ###</span>
+</span><span id="main-140"><a href="#main-140"><span class="linenos">140</span></a> <span class="n">phen_parser</span> <span class="o">=</span> <span class="n">subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"phen"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Phen commands"</span><span class="p">)</span>
+</span><span id="main-141"><a href="#main-141"><span class="linenos">141</span></a> <span class="n">phen_subparsers</span> <span class="o">=</span> <span class="n">phen_parser</span><span class="o">.</span><span class="n">add_subparsers</span><span class="p">(</span>
+</span><span id="main-142"><a href="#main-142"><span class="linenos">142</span></a> <span class="n">dest</span><span class="o">=</span><span class="s2">"subcommand"</span><span class="p">,</span> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Phen subcommands"</span>
+</span><span id="main-143"><a href="#main-143"><span class="linenos">143</span></a> <span class="p">)</span>
+</span><span id="main-144"><a href="#main-144"><span class="linenos">144</span></a>
+</span><span id="main-145"><a href="#main-145"><span class="linenos">145</span></a> <span class="c1"># phen init</span>
+</span><span id="main-146"><a href="#main-146"><span class="linenos">146</span></a> <span class="n">phen_init_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-147"><a href="#main-147"><span class="linenos">147</span></a> <span class="s2">"init"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Initiatise phenotype directory"</span>
+</span><span id="main-148"><a href="#main-148"><span class="linenos">148</span></a> <span class="p">)</span>
+</span><span id="main-149"><a href="#main-149"><span class="linenos">149</span></a> <span class="n">phen_init_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-150"><a href="#main-150"><span class="linenos">150</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-151"><a href="#main-151"><span class="linenos">151</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-152"><a href="#main-152"><span class="linenos">152</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-153"><a href="#main-153"><span class="linenos">153</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-154"><a href="#main-154"><span class="linenos">154</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-155"><a href="#main-155"><span class="linenos">155</span></a> <span class="p">)</span>
+</span><span id="main-156"><a href="#main-156"><span class="linenos">156</span></a> <span class="n">phen_init_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-157"><a href="#main-157"><span class="linenos">157</span></a> <span class="s2">"-r"</span><span class="p">,</span>
+</span><span id="main-158"><a href="#main-158"><span class="linenos">158</span></a> <span class="s2">"--remote_url"</span><span class="p">,</span>
+</span><span id="main-159"><a href="#main-159"><span class="linenos">159</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) URL to repository where the phenotype will be published."</span><span class="p">,</span>
+</span><span id="main-160"><a href="#main-160"><span class="linenos">160</span></a> <span class="p">)</span>
+</span><span id="main-161"><a href="#main-161"><span class="linenos">161</span></a> <span class="n">phen_init_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_init</span><span class="p">)</span>
+</span><span id="main-162"><a href="#main-162"><span class="linenos">162</span></a>
+</span><span id="main-163"><a href="#main-163"><span class="linenos">163</span></a> <span class="c1"># phen fork</span>
+</span><span id="main-164"><a href="#main-164"><span class="linenos">164</span></a> <span class="n">phen_fork_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-165"><a href="#main-165"><span class="linenos">165</span></a> <span class="s2">"fork"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Fork an existing phenotype"</span>
+</span><span id="main-166"><a href="#main-166"><span class="linenos">166</span></a> <span class="p">)</span>
+</span><span id="main-167"><a href="#main-167"><span class="linenos">167</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-168"><a href="#main-168"><span class="linenos">168</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-169"><a href="#main-169"><span class="linenos">169</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-170"><a href="#main-170"><span class="linenos">170</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-171"><a href="#main-171"><span class="linenos">171</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-172"><a href="#main-172"><span class="linenos">172</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-173"><a href="#main-173"><span class="linenos">173</span></a> <span class="p">)</span>
+</span><span id="main-174"><a href="#main-174"><span class="linenos">174</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-175"><a href="#main-175"><span class="linenos">175</span></a> <span class="s2">"-r"</span><span class="p">,</span>
+</span><span id="main-176"><a href="#main-176"><span class="linenos">176</span></a> <span class="s2">"--remote_url"</span><span class="p">,</span>
+</span><span id="main-177"><a href="#main-177"><span class="linenos">177</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) URL to repository where the forked phenotype will be published."</span><span class="p">,</span>
+</span><span id="main-178"><a href="#main-178"><span class="linenos">178</span></a> <span class="p">)</span>
+</span><span id="main-179"><a href="#main-179"><span class="linenos">179</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-180"><a href="#main-180"><span class="linenos">180</span></a> <span class="s2">"-u"</span><span class="p">,</span>
+</span><span id="main-181"><a href="#main-181"><span class="linenos">181</span></a> <span class="s2">"--upstream-url"</span><span class="p">,</span>
+</span><span id="main-182"><a href="#main-182"><span class="linenos">182</span></a> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="main-183"><a href="#main-183"><span class="linenos">183</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Required) URL to the phenotype repository to fork."</span><span class="p">,</span>
+</span><span id="main-184"><a href="#main-184"><span class="linenos">184</span></a> <span class="p">)</span>
+</span><span id="main-185"><a href="#main-185"><span class="linenos">185</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-186"><a href="#main-186"><span class="linenos">186</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="main-187"><a href="#main-187"><span class="linenos">187</span></a> <span class="s2">"--upstream-version"</span><span class="p">,</span>
+</span><span id="main-188"><a href="#main-188"><span class="linenos">188</span></a> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="main-189"><a href="#main-189"><span class="linenos">189</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Required) Phenotype version to fork."</span><span class="p">,</span>
+</span><span id="main-190"><a href="#main-190"><span class="linenos">190</span></a> <span class="p">)</span>
+</span><span id="main-191"><a href="#main-191"><span class="linenos">191</span></a> <span class="n">phen_fork_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_fork</span><span class="p">)</span>
+</span><span id="main-192"><a href="#main-192"><span class="linenos">192</span></a>
+</span><span id="main-193"><a href="#main-193"><span class="linenos">193</span></a> <span class="c1"># phen validate</span>
+</span><span id="main-194"><a href="#main-194"><span class="linenos">194</span></a> <span class="n">phen_validate_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-195"><a href="#main-195"><span class="linenos">195</span></a> <span class="s2">"validate"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Validate phenotype configuration"</span>
+</span><span id="main-196"><a href="#main-196"><span class="linenos">196</span></a> <span class="p">)</span>
+</span><span id="main-197"><a href="#main-197"><span class="linenos">197</span></a> <span class="n">phen_validate_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-198"><a href="#main-198"><span class="linenos">198</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-199"><a href="#main-199"><span class="linenos">199</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-200"><a href="#main-200"><span class="linenos">200</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-201"><a href="#main-201"><span class="linenos">201</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-202"><a href="#main-202"><span class="linenos">202</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-203"><a href="#main-203"><span class="linenos">203</span></a> <span class="p">)</span>
+</span><span id="main-204"><a href="#main-204"><span class="linenos">204</span></a> <span class="n">phen_validate_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_validate</span><span class="p">)</span>
+</span><span id="main-205"><a href="#main-205"><span class="linenos">205</span></a>
+</span><span id="main-206"><a href="#main-206"><span class="linenos">206</span></a> <span class="c1"># phen map</span>
+</span><span id="main-207"><a href="#main-207"><span class="linenos">207</span></a> <span class="n">phen_map_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span><span class="s2">"map"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Process phen mapping"</span><span class="p">)</span>
+</span><span id="main-208"><a href="#main-208"><span class="linenos">208</span></a> <span class="n">phen_map_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-209"><a href="#main-209"><span class="linenos">209</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-210"><a href="#main-210"><span class="linenos">210</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-211"><a href="#main-211"><span class="linenos">211</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-212"><a href="#main-212"><span class="linenos">212</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-213"><a href="#main-213"><span class="linenos">213</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-214"><a href="#main-214"><span class="linenos">214</span></a> <span class="p">)</span>
+</span><span id="main-215"><a href="#main-215"><span class="linenos">215</span></a> <span class="n">phen_map_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-216"><a href="#main-216"><span class="linenos">216</span></a> <span class="s2">"-t"</span><span class="p">,</span>
+</span><span id="main-217"><a href="#main-217"><span class="linenos">217</span></a> <span class="s2">"--target-coding"</span><span class="p">,</span>
+</span><span id="main-218"><a href="#main-218"><span class="linenos">218</span></a> <span class="n">choices</span><span class="o">=</span><span class="n">parse</span><span class="o">.</span><span class="n">SUPPORTED_CODE_TYPES</span><span class="p">,</span>
+</span><span id="main-219"><a href="#main-219"><span class="linenos">219</span></a> <span class="n">help</span><span class="o">=</span><span class="sa">f</span><span class="s2">"Specify the target coding </span><span class="si">{</span><span class="n">parse</span><span class="o">.</span><span class="n">SUPPORTED_CODE_TYPES</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span>
+</span><span id="main-220"><a href="#main-220"><span class="linenos">220</span></a> <span class="p">)</span>
+</span><span id="main-221"><a href="#main-221"><span class="linenos">221</span></a> <span class="n">phen_map_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_map</span><span class="p">)</span>
+</span><span id="main-222"><a href="#main-222"><span class="linenos">222</span></a>
+</span><span id="main-223"><a href="#main-223"><span class="linenos">223</span></a> <span class="c1"># phen export</span>
+</span><span id="main-224"><a href="#main-224"><span class="linenos">224</span></a> <span class="n">phen_export_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-225"><a href="#main-225"><span class="linenos">225</span></a> <span class="s2">"export"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Export phen to OMOP database"</span>
+</span><span id="main-226"><a href="#main-226"><span class="linenos">226</span></a> <span class="p">)</span>
+</span><span id="main-227"><a href="#main-227"><span class="linenos">227</span></a> <span class="n">phen_export_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-228"><a href="#main-228"><span class="linenos">228</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-229"><a href="#main-229"><span class="linenos">229</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-230"><a href="#main-230"><span class="linenos">230</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-231"><a href="#main-231"><span class="linenos">231</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-232"><a href="#main-232"><span class="linenos">232</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-233"><a href="#main-233"><span class="linenos">233</span></a> <span class="p">)</span>
+</span><span id="main-234"><a href="#main-234"><span class="linenos">234</span></a> <span class="n">phen_export_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-235"><a href="#main-235"><span class="linenos">235</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="main-236"><a href="#main-236"><span class="linenos">236</span></a> <span class="s2">"--version"</span><span class="p">,</span>
+</span><span id="main-237"><a href="#main-237"><span class="linenos">237</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-238"><a href="#main-238"><span class="linenos">238</span></a> <span class="n">default</span><span class="o">=</span><span class="s2">"latest"</span><span class="p">,</span>
+</span><span id="main-239"><a href="#main-239"><span class="linenos">239</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Phenotype version to export, defaults to the latest version"</span><span class="p">,</span>
+</span><span id="main-240"><a href="#main-240"><span class="linenos">240</span></a> <span class="p">)</span>
+</span><span id="main-241"><a href="#main-241"><span class="linenos">241</span></a> <span class="n">phen_export_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_export</span><span class="p">)</span>
+</span><span id="main-242"><a href="#main-242"><span class="linenos">242</span></a>
+</span><span id="main-243"><a href="#main-243"><span class="linenos">243</span></a> <span class="c1"># phen publish</span>
+</span><span id="main-244"><a href="#main-244"><span class="linenos">244</span></a> <span class="n">phen_publish_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-245"><a href="#main-245"><span class="linenos">245</span></a> <span class="s2">"publish"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Publish phenotype configuration"</span>
+</span><span id="main-246"><a href="#main-246"><span class="linenos">246</span></a> <span class="p">)</span>
+</span><span id="main-247"><a href="#main-247"><span class="linenos">247</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-248"><a href="#main-248"><span class="linenos">248</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-249"><a href="#main-249"><span class="linenos">249</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-250"><a href="#main-250"><span class="linenos">250</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-251"><a href="#main-251"><span class="linenos">251</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-252"><a href="#main-252"><span class="linenos">252</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-253"><a href="#main-253"><span class="linenos">253</span></a> <span class="p">)</span>
+</span><span id="main-254"><a href="#main-254"><span class="linenos">254</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-255"><a href="#main-255"><span class="linenos">255</span></a> <span class="s2">"-i"</span><span class="p">,</span>
+</span><span id="main-256"><a href="#main-256"><span class="linenos">256</span></a> <span class="s2">"--increment"</span><span class="p">,</span>
+</span><span id="main-257"><a href="#main-257"><span class="linenos">257</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-258"><a href="#main-258"><span class="linenos">258</span></a> <span class="n">default</span><span class="o">=</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_VERSION_INC</span><span class="p">,</span>
+</span><span id="main-259"><a href="#main-259"><span class="linenos">259</span></a> <span class="n">choices</span><span class="o">=</span><span class="n">phen</span><span class="o">.</span><span class="n">SEMANTIC_VERSION_TYPES</span><span class="p">,</span>
+</span><span id="main-260"><a href="#main-260"><span class="linenos">260</span></a> <span class="n">help</span><span class="o">=</span><span class="sa">f</span><span class="s2">"Version increment: </span><span class="si">{</span><span class="n">phen</span><span class="o">.</span><span class="n">SEMANTIC_VERSION_TYPES</span><span class="si">}</span><span class="s2">, default is </span><span class="si">{</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_VERSION_INC</span><span class="si">}</span><span class="s2"> increment"</span><span class="p">,</span>
+</span><span id="main-261"><a href="#main-261"><span class="linenos">261</span></a> <span class="p">)</span>
+</span><span id="main-262"><a href="#main-262"><span class="linenos">262</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-263"><a href="#main-263"><span class="linenos">263</span></a> <span class="s2">"-m"</span><span class="p">,</span> <span class="s2">"--msg"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Message to include with the published version"</span>
+</span><span id="main-264"><a href="#main-264"><span class="linenos">264</span></a> <span class="p">)</span>
+</span><span id="main-265"><a href="#main-265"><span class="linenos">265</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-266"><a href="#main-266"><span class="linenos">266</span></a> <span class="s2">"-r"</span><span class="p">,</span> <span class="s2">"--remote_url"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"URL to remote git repository"</span>
+</span><span id="main-267"><a href="#main-267"><span class="linenos">267</span></a> <span class="p">)</span>
+</span><span id="main-268"><a href="#main-268"><span class="linenos">268</span></a> <span class="n">phen_publish_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_publish</span><span class="p">)</span>
+</span><span id="main-269"><a href="#main-269"><span class="linenos">269</span></a>
+</span><span id="main-270"><a href="#main-270"><span class="linenos">270</span></a> <span class="c1"># phen copy</span>
+</span><span id="main-271"><a href="#main-271"><span class="linenos">271</span></a> <span class="n">phen_copy_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-272"><a href="#main-272"><span class="linenos">272</span></a> <span class="s2">"copy"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Publish phenotype configuration"</span>
+</span><span id="main-273"><a href="#main-273"><span class="linenos">273</span></a> <span class="p">)</span>
+</span><span id="main-274"><a href="#main-274"><span class="linenos">274</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-275"><a href="#main-275"><span class="linenos">275</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-276"><a href="#main-276"><span class="linenos">276</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-277"><a href="#main-277"><span class="linenos">277</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-278"><a href="#main-278"><span class="linenos">278</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-279"><a href="#main-279"><span class="linenos">279</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-280"><a href="#main-280"><span class="linenos">280</span></a> <span class="p">)</span>
+</span><span id="main-281"><a href="#main-281"><span class="linenos">281</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-282"><a href="#main-282"><span class="linenos">282</span></a> <span class="s2">"-td"</span><span class="p">,</span>
+</span><span id="main-283"><a href="#main-283"><span class="linenos">283</span></a> <span class="s2">"--target-dir"</span><span class="p">,</span>
+</span><span id="main-284"><a href="#main-284"><span class="linenos">284</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-285"><a href="#main-285"><span class="linenos">285</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">DEFAULT_WORKSPACE_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-286"><a href="#main-286"><span class="linenos">286</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Target directory for the copy"</span><span class="p">,</span>
+</span><span id="main-287"><a href="#main-287"><span class="linenos">287</span></a> <span class="p">)</span>
+</span><span id="main-288"><a href="#main-288"><span class="linenos">288</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-289"><a href="#main-289"><span class="linenos">289</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="main-290"><a href="#main-290"><span class="linenos">290</span></a> <span class="s2">"--version"</span><span class="p">,</span>
+</span><span id="main-291"><a href="#main-291"><span class="linenos">291</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-292"><a href="#main-292"><span class="linenos">292</span></a> <span class="n">default</span><span class="o">=</span><span class="s2">"latest"</span><span class="p">,</span>
+</span><span id="main-293"><a href="#main-293"><span class="linenos">293</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Phenotype version to copy, defaults to the latest version"</span><span class="p">,</span>
+</span><span id="main-294"><a href="#main-294"><span class="linenos">294</span></a> <span class="p">)</span>
+</span><span id="main-295"><a href="#main-295"><span class="linenos">295</span></a> <span class="n">phen_copy_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_copy</span><span class="p">)</span>
+</span><span id="main-296"><a href="#main-296"><span class="linenos">296</span></a>
+</span><span id="main-297"><a href="#main-297"><span class="linenos">297</span></a> <span class="c1"># phen diff</span>
+</span><span id="main-298"><a href="#main-298"><span class="linenos">298</span></a> <span class="n">phen_diff_parser</span> <span class="o">=</span> <span class="n">phen_subparsers</span><span class="o">.</span><span class="n">add_parser</span><span class="p">(</span>
+</span><span id="main-299"><a href="#main-299"><span class="linenos">299</span></a> <span class="s2">"diff"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"Publish phenotype configuration"</span>
+</span><span id="main-300"><a href="#main-300"><span class="linenos">300</span></a> <span class="p">)</span>
+</span><span id="main-301"><a href="#main-301"><span class="linenos">301</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-302"><a href="#main-302"><span class="linenos">302</span></a> <span class="s2">"-d"</span><span class="p">,</span>
+</span><span id="main-303"><a href="#main-303"><span class="linenos">303</span></a> <span class="s2">"--phen-dir"</span><span class="p">,</span>
+</span><span id="main-304"><a href="#main-304"><span class="linenos">304</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-305"><a href="#main-305"><span class="linenos">305</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-306"><a href="#main-306"><span class="linenos">306</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"(Optional) Local phenotype workspace directory (default is ./workspace/phen)."</span><span class="p">,</span>
+</span><span id="main-307"><a href="#main-307"><span class="linenos">307</span></a> <span class="p">)</span>
+</span><span id="main-308"><a href="#main-308"><span class="linenos">308</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-309"><a href="#main-309"><span class="linenos">309</span></a> <span class="s2">"-v"</span><span class="p">,</span>
+</span><span id="main-310"><a href="#main-310"><span class="linenos">310</span></a> <span class="s2">"--version"</span><span class="p">,</span>
+</span><span id="main-311"><a href="#main-311"><span class="linenos">311</span></a> <span class="n">default</span><span class="o">=</span><span class="s2">"latest"</span><span class="p">,</span>
+</span><span id="main-312"><a href="#main-312"><span class="linenos">312</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Phenotype version to compare with an old version, defaults to the HEAD of the workspace directory"</span><span class="p">,</span>
+</span><span id="main-313"><a href="#main-313"><span class="linenos">313</span></a> <span class="p">)</span>
+</span><span id="main-314"><a href="#main-314"><span class="linenos">314</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-315"><a href="#main-315"><span class="linenos">315</span></a> <span class="s2">"-od"</span><span class="p">,</span>
+</span><span id="main-316"><a href="#main-316"><span class="linenos">316</span></a> <span class="s2">"--old-phen-dir"</span><span class="p">,</span>
+</span><span id="main-317"><a href="#main-317"><span class="linenos">317</span></a> <span class="nb">type</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span>
+</span><span id="main-318"><a href="#main-318"><span class="linenos">318</span></a> <span class="n">default</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">phen</span><span class="o">.</span><span class="n">DEFAULT_PHEN_PATH</span><span class="o">.</span><span class="n">resolve</span><span class="p">()),</span>
+</span><span id="main-319"><a href="#main-319"><span class="linenos">319</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Directory for the old phenotype version, defaults to workspace directory"</span><span class="p">,</span>
+</span><span id="main-320"><a href="#main-320"><span class="linenos">320</span></a> <span class="p">)</span>
+</span><span id="main-321"><a href="#main-321"><span class="linenos">321</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span>
+</span><span id="main-322"><a href="#main-322"><span class="linenos">322</span></a> <span class="s2">"-ov"</span><span class="p">,</span>
+</span><span id="main-323"><a href="#main-323"><span class="linenos">323</span></a> <span class="s2">"--old-version"</span><span class="p">,</span>
+</span><span id="main-324"><a href="#main-324"><span class="linenos">324</span></a> <span class="n">required</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="main-325"><a href="#main-325"><span class="linenos">325</span></a> <span class="n">help</span><span class="o">=</span><span class="s2">"Old phenotype version to compare with the changed version"</span><span class="p">,</span>
+</span><span id="main-326"><a href="#main-326"><span class="linenos">326</span></a> <span class="p">)</span>
+</span><span id="main-327"><a href="#main-327"><span class="linenos">327</span></a> <span class="n">phen_diff_parser</span><span class="o">.</span><span class="n">set_defaults</span><span class="p">(</span><span class="n">func</span><span class="o">=</span><span class="n">_phen_diff</span><span class="p">)</span>
+</span><span id="main-328"><a href="#main-328"><span class="linenos">328</span></a>
+</span><span id="main-329"><a href="#main-329"><span class="linenos">329</span></a> <span class="c1"># Parse arguments</span>
+</span><span id="main-330"><a href="#main-330"><span class="linenos">330</span></a> <span class="n">args</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">parse_args</span><span class="p">()</span>
+</span><span id="main-331"><a href="#main-331"><span class="linenos">331</span></a>
+</span><span id="main-332"><a href="#main-332"><span class="linenos">332</span></a> <span class="c1"># setup logging</span>
+</span><span id="main-333"><a href="#main-333"><span class="linenos">333</span></a> <span class="k">if</span> <span class="n">args</span><span class="o">.</span><span class="n">debug</span><span class="p">:</span>
+</span><span id="main-334"><a href="#main-334"><span class="linenos">334</span></a> <span class="n">lc</span><span class="o">.</span><span class="n">set_log_level</span><span class="p">(</span><span class="n">logging</span><span class="o">.</span><span class="n">DEBUG</span><span class="p">)</span>
+</span><span id="main-335"><a href="#main-335"><span class="linenos">335</span></a>
+</span><span id="main-336"><a href="#main-336"><span class="linenos">336</span></a> <span class="c1"># Call the function associated with the command</span>
+</span><span id="main-337"><a href="#main-337"><span class="linenos">337</span></a> <span class="n">args</span><span class="o">.</span><span class="n">func</span><span class="p">(</span><span class="n">args</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/omop.html b/docs/api/acmc/omop.html
new file mode 100644
index 0000000000000000000000000000000000000000..adb10a9357eb6c14fb00bd3f064879a7a7481a92
--- /dev/null
+++ b/docs/api/acmc/omop.html
@@ -0,0 +1,1420 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.omop API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#VOCAB_PATH">VOCAB_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#OMOP_CDM_Version">OMOP_CDM_Version</a>
+ </li>
+ <li>
+ <a class="variable" href="#OMOP_DB_FILENAME">OMOP_DB_FILENAME</a>
+ </li>
+ <li>
+ <a class="variable" href="#DB_PATH">DB_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#VERSION_FILE">VERSION_FILE</a>
+ </li>
+ <li>
+ <a class="variable" href="#VERSION_PATH">VERSION_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#EXPORT_FILE">EXPORT_FILE</a>
+ </li>
+ <li>
+ <a class="variable" href="#vocabularies">vocabularies</a>
+ </li>
+ <li>
+ <a class="variable" href="#omop_vocab_types">omop_vocab_types</a>
+ </li>
+ <li>
+ <a class="function" href="#install">install</a>
+ </li>
+ <li>
+ <a class="function" href="#write_version_file">write_version_file</a>
+ </li>
+ <li>
+ <a class="function" href="#clear">clear</a>
+ </li>
+ <li>
+ <a class="function" href="#delete">delete</a>
+ </li>
+ <li>
+ <a class="function" href="#table_exists">table_exists</a>
+ </li>
+ <li>
+ <a class="function" href="#vocab_exists">vocab_exists</a>
+ </li>
+ <li>
+ <a class="function" href="#concept_set_exist">concept_set_exist</a>
+ </li>
+ <li>
+ <a class="function" href="#export">export</a>
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.omop </h1>
+
+ <div class="docstring"><p>omop.py module</p>
+
+<p>This module provides functionality to manage OMOP vocabularies.</p>
+</div>
+
+ <input id="mod-omop-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-omop-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="sd">"""</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="sd">omop.py module</span>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">This module provides functionality to manage OMOP vocabularies.</span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="sd">"""</span>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">os</span>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">argparse</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">sqlite3</span>
+</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
+</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">logging</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">zipfile</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">shutil</span>
+</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">json</span>
+</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">yaml</span>
+</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">pathlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">Path</span>
+</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">acmc</span><span class="w"> </span><span class="kn">import</span> <span class="n">util</span><span class="p">,</span> <span class="n">logging_config</span>
+</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a>
+</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="c1"># setup logging</span>
+</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="n">_logger</span> <span class="o">=</span> <span class="n">logging_config</span><span class="o">.</span><span class="n">setup_logger</span><span class="p">()</span>
+</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a>
+</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="n">VOCAB_PATH</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s2">"./vocab/omop"</span><span class="p">)</span>
+</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="sd">"""Default OMOP vocabulary path"""</span>
+</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a>
+</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="n">OMOP_CDM_Version</span> <span class="o">=</span> <span class="s2">"54"</span>
+</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="sd">"""Supported OMOP CDM version number"""</span>
+</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a>
+</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="n">OMOP_DB_FILENAME</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"omop_</span><span class="si">{</span><span class="n">OMOP_CDM_Version</span><span class="si">}</span><span class="s2">.sqlite"</span>
+</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="sd">"""Default OMOP sqllite database filename"""</span>
+</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
+</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="n">DB_PATH</span> <span class="o">=</span> <span class="n">VOCAB_PATH</span> <span class="o">/</span> <span class="n">OMOP_DB_FILENAME</span>
+</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="sd">"""Default OMOP sqllite database path"""</span>
+</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a>
+</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="n">VERSION_FILE</span> <span class="o">=</span> <span class="s2">"omop_version.yml"</span>
+</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a><span class="sd">"""Default OMOP version file"""</span>
+</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a>
+</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="n">VERSION_PATH</span> <span class="o">=</span> <span class="n">VOCAB_PATH</span> <span class="o">/</span> <span class="n">VERSION_FILE</span>
+</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a><span class="sd">"""Default OMOP version path"""</span>
+</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a>
+</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="n">EXPORT_FILE</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"omop_</span><span class="si">{</span><span class="n">OMOP_CDM_Version</span><span class="si">}</span><span class="s2">_export.sqlite"</span>
+</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="sd">"""Default OMOP export database filename"""</span>
+</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a>
+</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="n">vocabularies</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a> <span class="s2">"source"</span><span class="p">:</span> <span class="s2">"OHDSI Athena"</span><span class="p">,</span>
+</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="s2">"url"</span><span class="p">:</span> <span class="s2">"https://athena.ohdsi.org/vocabulary/list"</span><span class="p">,</span>
+</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="s2">"cdm_version"</span><span class="p">:</span> <span class="n">OMOP_CDM_Version</span><span class="p">,</span>
+</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="s2">"version"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="s2">"vocabularies"</span><span class="p">:</span> <span class="p">[</span>
+</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"SNOMED"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"ICD9CM"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">17</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"Readv2"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">21</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"ATC"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">55</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"OPCS4"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">57</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"HES Specialty"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">70</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"ICD10CM"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">75</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"dm+d"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">144</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"UK Biobank"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">154</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS Ethnic Category"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="p">{</span><span class="s2">"id"</span><span class="p">:</span> <span class="mi">155</span><span class="p">,</span> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS Place of Service"</span><span class="p">},</span> <span class="c1"># No license required</span>
+</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="p">],</span>
+</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a> <span class="s2">"tables"</span><span class="p">:</span> <span class="p">[],</span>
+</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a><span class="p">}</span>
+</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a><span class="sd">"""Required OMOP vocabularies definition"""</span>
+</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
+</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a><span class="n">omop_vocab_types</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="s2">"read2"</span><span class="p">:</span> <span class="s2">"Read"</span><span class="p">,</span>
+</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="s2">"read3"</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span>
+</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="s2">"icd10"</span><span class="p">:</span> <span class="s2">"ICD10CM"</span><span class="p">,</span>
+</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="s2">"snomed"</span><span class="p">:</span> <span class="s2">"SNOMED"</span><span class="p">,</span>
+</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="s2">"opcs4"</span><span class="p">:</span> <span class="s2">"OPCS4"</span><span class="p">,</span>
+</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="s2">"atc"</span><span class="p">:</span> <span class="s2">"ATC"</span><span class="p">,</span>
+</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="s2">"med"</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span>
+</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="s2">"cprd"</span><span class="p">:</span> <span class="kc">None</span><span class="p">,</span>
+</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a><span class="p">}</span>
+</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="sd">"""Type mappings from acmc medical coding types to OMOP vocabulary types"""</span>
+</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a>
+</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
+</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a><span class="k">def</span><span class="w"> </span><span class="nf">install</span><span class="p">(</span><span class="n">omop_zip_file</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a><span class="w"> </span><span class="sd">""" "Installs the OMOP release csv files in a file-based sql database</span>
+</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
+</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="sd"> Args:</span>
+</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="sd"> omop_zip_file (str): vocabularies zip file distributed by OHDSI Athena</span>
+</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a><span class="sd"> version (str): version of the vocabularies distributed by OHDSI Athena</span>
+</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
+</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="sd"> Raises:</span>
+</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a><span class="sd"> ValueError: if the zip file does not exist</span>
+</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="sd"> ValueError: if the file is not a zip file</span>
+</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="sd"> Exception: if error reading omop csv files</span>
+</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a><span class="sd"> """</span>
+</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Installing OMOP from zip file: </span><span class="si">{</span><span class="n">omop_zip_file</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="n">omop_zip_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">omop_zip_file</span><span class="p">)</span>
+</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a>
+</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="c1"># Check if the file exists and is a ZIP file</span>
+</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">omop_zip_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">omop_zip_path</span><span class="si">}</span><span class="s2"> does not exist."</span>
+</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a>
+</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">zipfile</span><span class="o">.</span><span class="n">is_zipfile</span><span class="p">(</span><span class="n">omop_zip_path</span><span class="p">):</span>
+</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Error: </span><span class="si">{</span><span class="n">omop_zip_path</span><span class="si">}</span><span class="s2"> is not a valid ZIP file."</span>
+</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a>
+</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="c1"># check codes directory exists and if not create it</span>
+</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP directory '</span><span class="si">{</span><span class="n">VOCAB_PATH</span><span class="si">}</span><span class="s2">' created."</span><span class="p">)</span>
+</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="c1"># removing existing OMOP files</span>
+</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="n">csv_files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"*.csv"</span><span class="p">))</span>
+</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">csv_files</span><span class="p">:</span>
+</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">file</span><span class="o">.</span><span class="n">unlink</span><span class="p">()</span>
+</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Deleted OMOP csv file: </span><span class="si">{</span><span class="n">file</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a>
+</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="c1"># Extract ZIP contents</span>
+</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="k">with</span> <span class="n">zipfile</span><span class="o">.</span><span class="n">ZipFile</span><span class="p">(</span><span class="n">omop_zip_path</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">zip_ref</span><span class="p">:</span>
+</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">zip_ref</span><span class="o">.</span><span class="n">extractall</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="p">)</span>
+</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted OMOP zip file </span><span class="si">{</span><span class="n">omop_zip_path</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">VOCAB_PATH</span><span class="si">}</span><span class="s2">/"</span><span class="p">)</span>
+</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
+</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a> <span class="c1"># connect to database, if it does not exist it will be created</span>
+</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a> <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">DB_PATH</span><span class="p">)</span>
+</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
+</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="c1"># Iterate through files in the folder</span>
+</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="n">csv_files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"*.csv"</span><span class="p">))</span>
+</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="n">total_tables_count</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">csv_files</span><span class="p">)</span>
+</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="n">table_count</span> <span class="o">=</span> <span class="mi">1</span>
+</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="k">for</span> <span class="n">filename</span> <span class="ow">in</span> <span class="n">csv_files</span><span class="p">:</span>
+</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
+</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="sa">f</span><span class="s2">"Processing </span><span class="si">{</span><span class="n">table_count</span><span class="si">}</span><span class="s2"> of </span><span class="si">{</span><span class="n">total_tables_count</span><span class="si">}</span><span class="s2"> tables: </span><span class="si">{</span><span class="n">filename</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="p">)</span>
+</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="c1"># read the CSV file with the specified delimiter</span>
+</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">delimiter</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">low_memory</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a>
+</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="c1"># export Table to sqlite db</span>
+</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_sql</span><span class="p">(</span><span class="n">filename</span><span class="o">.</span><span class="n">stem</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">if_exists</span><span class="o">=</span><span class="s2">"replace"</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a>
+</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="c1"># add to the metadata</span>
+</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="nb">list</span><span class="p">(</span><span class="n">vocabularies</span><span class="p">[</span><span class="s2">"tables"</span><span class="p">])</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">filename</span><span class="o">.</span><span class="n">stem</span><span class="p">)</span>
+</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">table_count</span> <span class="o">=</span> <span class="n">table_count</span> <span class="o">+</span> <span class="mi">1</span>
+</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error reading file </span><span class="si">{</span><span class="n">filename</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a>
+</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a>
+</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="c1"># write version file</span>
+</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">write_version_file</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
+</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
+</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP installation completed"</span><span class="p">)</span>
+</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a>
+</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a>
+</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a><span class="k">def</span><span class="w"> </span><span class="nf">write_version_file</span><span class="p">(</span><span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a><span class="w"> </span><span class="sd">"""Writes the OMOP vocaburaries and version to a file</span>
+</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a>
+</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a><span class="sd"> Args:</span>
+</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a><span class="sd"> version (str): version of the vocabularies distributed by OHDSI Athena</span>
+</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a><span class="sd"> """</span>
+</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">vocabularies</span><span class="p">[</span><span class="s2">"version"</span><span class="p">]</span> <span class="o">=</span> <span class="n">version</span>
+</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">VERSION_PATH</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">vocabularies</span><span class="p">,</span>
+</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="p">)</span>
+</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a>
+</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a>
+</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a><span class="k">def</span><span class="w"> </span><span class="nf">clear</span><span class="p">(</span><span class="n">db_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a><span class="w"> </span><span class="sd">"""Clears the OMOP sql database</span>
+</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a>
+</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a><span class="sd"> Args:</span>
+</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a><span class="sd"> db_path (Path): the path to the omop sqllite database</span>
+</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a>
+</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a><span class="sd"> Raises:</span>
+</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a><span class="sd"> FileNotFoundError: if the omop sqllite database does not exist</span>
+</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a><span class="sd"> """</span>
+</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Clearing OMOP data from database"</span><span class="p">)</span>
+</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">db_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error: OMOP DB file '</span><span class="si">{</span><span class="n">db_path</span><span class="si">}</span><span class="s2">' does not exist."</span><span class="p">)</span>
+</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">db_path</span><span class="p">)</span>
+</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
+</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"SELECT name FROM sqlite_master WHERE type='table';"</span><span class="p">)</span>
+</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a>
+</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="c1"># Fetch and print table names</span>
+</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchall</span><span class="p">()</span>
+</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Tables in database:"</span><span class="p">,</span> <span class="p">[</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">])</span>
+</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a>
+</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="c1"># cur.execute("DROP TABLE CONCEPT_SET;")</span>
+</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="c1"># cur.execute("DROP TABLE CONCEPT_SET_ITEM;")</span>
+</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a>
+</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP database cleared"</span><span class="p">)</span>
+</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a>
+</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a>
+</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a><span class="k">def</span><span class="w"> </span><span class="nf">delete</span><span class="p">(</span><span class="n">db_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a><span class="w"> </span><span class="sd">"""Deletes the OMOP sql database</span>
+</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a>
+</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a><span class="sd"> Args:</span>
+</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a><span class="sd"> db_path (Path): the path to the omop sqllite database</span>
+</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a>
+</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a><span class="sd"> Raises:</span>
+</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a><span class="sd"> FileNotFoundError: if the omop sqllite database does not exist</span>
+</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a><span class="sd"> """</span>
+</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a>
+</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Deleting OMOP database"</span><span class="p">)</span>
+</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">db_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error: OMOP DB file '</span><span class="si">{</span><span class="n">db_path</span><span class="si">}</span><span class="s2">' does not exist."</span><span class="p">)</span>
+</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a>
+</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">db_path</span><span class="o">.</span><span class="n">unlink</span><span class="p">()</span>
+</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP database deleted"</span><span class="p">)</span>
+</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a>
+</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a>
+</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a><span class="k">def</span><span class="w"> </span><span class="nf">table_exists</span><span class="p">(</span><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span><span class="p">,</span> <span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a><span class="w"> </span><span class="sd">"""Query to check if the table exists</span>
+</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a>
+</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a><span class="sd"> Args:</span>
+</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a><span class="sd"> cursor (sqlite3.Cursor): a sqllite database cursor</span>
+</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a><span class="sd"> table_name (str): the table name to check</span>
+</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a>
+</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a><span class="sd"> Returns:</span>
+</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a><span class="sd"> bool: true if table exists</span>
+</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a><span class="sd"> """</span>
+</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a>
+</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a><span class="sd"> SELECT name</span>
+</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a><span class="sd"> FROM sqlite_master</span>
+</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a><span class="sd"> WHERE type='table' AND name=?</span>
+</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a><span class="sd"> """</span><span class="p">,</span>
+</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="p">(</span><span class="n">table_name</span><span class="p">,),</span>
+</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="p">)</span>
+</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a>
+</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="c1"># Fetch the result</span>
+</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>
+</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a>
+</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="k">return</span> <span class="n">result</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
+</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a>
+</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a>
+</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a><span class="k">def</span><span class="w"> </span><span class="nf">vocab_exists</span><span class="p">(</span><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span><span class="p">,</span> <span class="n">vocab_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a><span class="w"> </span><span class="sd">"""Query to check if the vocabulary exists</span>
+</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a>
+</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a><span class="sd"> Args:</span>
+</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a><span class="sd"> cursor (sqlite3.Cursor): a sqllite database cursor</span>
+</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a><span class="sd"> vocab_id (str): the vocabulary id to check</span>
+</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a>
+</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a><span class="sd"> Returns:</span>
+</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a><span class="sd"> bool: true if vocabulary id exists</span>
+</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a><span class="sd"> """</span>
+</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a>
+</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a><span class="sd"> SELECT vocabulary_id </span>
+</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a><span class="sd"> FROM VOCABULARY</span>
+</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a><span class="sd"> WHERE vocabulary_id=?</span>
+</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a><span class="sd"> """</span><span class="p">,</span>
+</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="p">(</span><span class="n">vocab_id</span><span class="p">,),</span>
+</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="p">)</span>
+</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a>
+</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="c1"># Fetch the result</span>
+</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>
+</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a>
+</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="k">return</span> <span class="n">result</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
+</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a>
+</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a>
+</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a><span class="k">def</span><span class="w"> </span><span class="nf">concept_set_exist</span><span class="p">(</span><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span><span class="p">,</span> <span class="n">concept_set_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a><span class="w"> </span><span class="sd">"""Query to check if the concept set exists</span>
+</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a>
+</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a><span class="sd"> Args:</span>
+</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a><span class="sd"> cursor (sqlite3.Cursor): a sqllite database cursor</span>
+</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a><span class="sd"> concept_set_name (str): the concept set name to check</span>
+</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a>
+</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a><span class="sd"> Returns:</span>
+</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a><span class="sd"> bool: true if concept set exists</span>
+</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a><span class="sd"> """</span>
+</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a>
+</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="n">query</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"SELECT EXISTS (SELECT 1 FROM CONCEPT_SET WHERE concept_set_name = ?)"</span>
+</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="p">(</span><span class="n">concept_set_name</span><span class="p">,))</span>
+</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a>
+</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="c1"># 1 if exists, 0 otherwise</span>
+</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="k">return</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="mi">1</span>
+</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a>
+</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a>
+</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a><span class="k">def</span><span class="w"> </span><span class="nf">export</span><span class="p">(</span>
+</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="n">map_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">export_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">omop_metadata</span><span class="p">:</span> <span class="nb">dict</span>
+</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Path</span><span class="p">:</span>
+</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a><span class="w"> </span><span class="sd">"""Export concept sets to omop database in csv format</span>
+</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a>
+</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a><span class="sd"> Args:</span>
+</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a><span class="sd"> map_path (Path): path to the acmc map directory containing concept sets in csv format</span>
+</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a><span class="sd"> export_path (Path): path to the directory where the omop database csv files are to be written</span>
+</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a><span class="sd"> version (str): phenotype version for omop vocabulary version</span>
+</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a><span class="sd"> omop_metadata (dict): phenotype omop metadata for omop vocabulary metadata</span>
+</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a>
+</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a><span class="sd"> Returns:</span>
+</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a><span class="sd"> Path: path to the exported sqllite database</span>
+</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a><span class="sd"> """</span>
+</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a>
+</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"exporting with metadata </span><span class="si">{</span><span class="n">omop_metadata</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a>
+</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="c1"># copy the baseline omop database</span>
+</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="n">export_db_path</span> <span class="o">=</span> <span class="n">export_path</span> <span class="o">/</span> <span class="n">EXPORT_FILE</span>
+</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">DB_PATH</span><span class="p">,</span> <span class="n">export_db_path</span><span class="p">)</span>
+</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a>
+</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="c1"># connect to db</span>
+</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">export_db_path</span><span class="p">)</span>
+</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
+</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a>
+</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="c1"># Create VOCABULARY</span>
+</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">df_test</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="p">[</span>
+</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="p">{</span>
+</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="s2">"vocabulary_id"</span><span class="p">:</span> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_id"</span><span class="p">],</span>
+</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="s2">"vocabulary_name"</span><span class="p">:</span> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_name"</span><span class="p">],</span>
+</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="s2">"vocabulary_reference"</span><span class="p">:</span> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_reference"</span><span class="p">],</span>
+</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="s2">"vocabulary_version"</span><span class="p">:</span> <span class="n">version</span><span class="p">,</span>
+</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="c1"># "vocabulary_concept_id": 0,</span>
+</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="p">}</span>
+</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="p">]</span>
+</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="p">)</span>
+</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="n">df_test</span><span class="o">.</span><span class="n">to_sql</span><span class="p">(</span><span class="s2">"VOCABULARY"</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">if_exists</span><span class="o">=</span><span class="s2">"append"</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a>
+</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="c1"># Create CONCEPT_SET</span>
+</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a><span class="sd"> CREATE TABLE CONCEPT_SET (</span>
+</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a><span class="sd"> concept_set_id INTEGER PRIMARY KEY AUTOINCREMENT, -- Unique identifier for each concept set</span>
+</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a><span class="sd"> atlas_id INTEGER, -- Unique identifier generated by ATLAS</span>
+</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a><span class="sd"> concept_set_name TEXT, -- Optional name for the concept set</span>
+</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a><span class="sd"> concept_set_description TEXT, -- Optional description for the concept set</span>
+</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a><span class="sd"> vocabulary_id TEXT NOT NULL, -- Foreign key to VOCABULARY table</span>
+</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a><span class="sd"> FOREIGN KEY (vocabulary_id) REFERENCES VOCABULARY(vocabulary_id)</span>
+</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a><span class="sd"> );"""</span>
+</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a> <span class="p">)</span>
+</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a>
+</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="c1"># Create CONCEPT_SET_ITEM</span>
+</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a><span class="sd"> CREATE TABLE CONCEPT_SET_ITEM (</span>
+</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a><span class="sd"> concept_set_item_id INTEGER PRIMARY KEY AUTOINCREMENT, -- Unique identifier for each mapping</span>
+</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a><span class="sd"> concept_set_id INTEGER NOT NULL, -- Foreign key to CONCEPT_SET table</span>
+</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a><span class="sd"> concept_id INTEGER NOT NULL, -- Foreign key to CONCEPT table</span>
+</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a><span class="sd"> FOREIGN KEY (concept_set_id) REFERENCES CONCEPT_SET(concept_set_id),</span>
+</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a><span class="sd"> FOREIGN KEY (concept_id) REFERENCES CONCEPT(concept_id)</span>
+</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a><span class="sd"> );"""</span>
+</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="p">)</span>
+</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a>
+</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="c1"># read map files</span>
+</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="n">map_files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">map_path</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"*.csv"</span><span class="p">))</span>
+</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="n">total</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">map_files</span><span class="p">)</span>
+</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exporting </span><span class="si">{</span><span class="n">total</span><span class="si">}</span><span class="s2"> map files"</span><span class="p">)</span>
+</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="k">for</span> <span class="n">index</span><span class="p">,</span> <span class="n">map_file</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">map_files</span><span class="p">):</span>
+</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Processing </span><span class="si">{</span><span class="n">index</span><span class="o">+</span><span class="mi">1</span><span class="si">}</span><span class="s2"> of </span><span class="si">{</span><span class="n">total</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">map_file</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">map_file</span><span class="p">)</span>
+</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a>
+</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a> <span class="k">for</span> <span class="n">concept_set_name</span><span class="p">,</span> <span class="n">grp</span> <span class="ow">in</span> <span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">):</span>
+</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a> <span class="c1"># create Concept_Set</span>
+</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">concept_set_exist</span><span class="p">(</span><span class="n">cur</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">concept_set_name</span><span class="p">)):</span>
+</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a> <span class="sa">f</span><span class="s2">"INSERT INTO CONCEPT_SET (concept_set_name, vocabulary_id) VALUES ('</span><span class="si">{</span><span class="n">concept_set_name</span><span class="si">}</span><span class="s2">', '</span><span class="si">{</span><span class="n">omop_metadata</span><span class="p">[</span><span class="s1">'vocabulary_id'</span><span class="p">]</span><span class="si">}</span><span class="s2">');"</span>
+</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a> <span class="p">)</span>
+</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Concept_set </span><span class="si">{</span><span class="n">concept_set_name</span><span class="si">}</span><span class="s2"> already exists"</span><span class="p">)</span>
+</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="c1"># TODO: ask to remove old concept_set?</span>
+</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a>
+</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="c1"># get Concept_set_Id</span>
+</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="n">query</span> <span class="o">=</span> <span class="s2">"SELECT concept_set_id FROM CONCEPT_SET WHERE concept_set_name = ? AND vocabulary_id = ?;"</span>
+</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="n">target_code_type</span> <span class="o">=</span> <span class="n">map_file</span><span class="o">.</span><span class="n">stem</span>
+</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="n">query</span><span class="p">,</span>
+</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="p">(</span>
+</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="n">concept_set_name</span><span class="p">,</span>
+</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_id"</span><span class="p">],</span>
+</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="p">),</span>
+</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="p">)</span>
+</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="c1"># FAILS HERE WITH NONE REUR</span>
+</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"omop code type </span><span class="si">{</span><span class="n">omop_vocab_types</span><span class="p">[</span><span class="n">target_code_type</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="n">concept_set_id</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span>
+</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"concept set id </span><span class="si">{</span><span class="n">concept_set_id</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a>
+</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="c1"># get corresponing Concept_id (OMOP) for each Concept_code (e.g. SNOMED)</span>
+</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="n">concept_codes</span> <span class="o">=</span> <span class="s2">"'"</span> <span class="o">+</span> <span class="s2">"', '"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">grp</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">]</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)))</span> <span class="o">+</span> <span class="s2">"'"</span>
+</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="n">query</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"SELECT concept_id FROM CONCEPT WHERE vocabulary_id = ? AND concept_code IN (</span><span class="si">{</span><span class="n">concept_codes</span><span class="si">}</span><span class="s2">);"</span>
+</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="p">(</span><span class="n">omop_vocab_types</span><span class="p">[</span><span class="n">target_code_type</span><span class="p">],))</span>
+</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a> <span class="n">df_out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">cur</span><span class="o">.</span><span class="n">fetchall</span><span class="p">(),</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s2">"concept_id"</span><span class="p">])</span>
+</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a>
+</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">len</span><span class="p">(</span><span class="n">grp</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">df_out</span><span class="p">):</span>
+</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
+</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a> <span class="sa">f</span><span class="s2">"ERROR: Some </span><span class="si">{</span><span class="n">omop_vocab_types</span><span class="p">[</span><span class="n">target_code_type</span><span class="p">]</span><span class="si">}</span><span class="s2"> Codes do not exist in OMOP Database"</span>
+</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a> <span class="p">)</span>
+</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a>
+</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="c1"># Create Concept_set_item</span>
+</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="n">df_out</span><span class="p">[</span><span class="s2">"concept_set_id"</span><span class="p">]</span> <span class="o">=</span> <span class="n">concept_set_id</span>
+</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="n">df_out</span><span class="o">.</span><span class="n">to_sql</span><span class="p">(</span><span class="s2">"CONCEPT_SET_ITEM"</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">if_exists</span><span class="o">=</span><span class="s2">"append"</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a>
+</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="c1"># Output all tables to CSV</span>
+</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="c1"># Get the list of all tables</span>
+</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"SELECT name FROM sqlite_master WHERE type='table';"</span><span class="p">)</span>
+</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchall</span><span class="p">()</span> <span class="c1"># List of tables</span>
+</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a>
+</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a> <span class="c1"># Export each table to a separate CSV file</span>
+</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
+</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
+</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a> <span class="c1"># ignore SQLite's internal system table</span>
+</span><span id="L-407"><a href="#L-407"><span class="linenos">407</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="o">!=</span> <span class="s2">"sqlite_sequence"</span><span class="p">:</span>
+</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_sql_query</span><span class="p">(</span><span class="sa">f</span><span class="s2">"SELECT * FROM </span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="n">conn</span><span class="p">)</span>
+</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="n">output_file</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2">.csv"</span>
+</span><span id="L-410"><a href="#L-410"><span class="linenos">410</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">export_path</span> <span class="o">/</span> <span class="n">output_file</span>
+</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="c1"># Save as CSV</span>
+</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exported </span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2">.csv"</span><span class="p">)</span>
+</span><span id="L-413"><a href="#L-413"><span class="linenos">413</span></a>
+</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a>
+</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Created export db successfully"</span><span class="p">)</span>
+</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a>
+</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="k">return</span> <span class="n">export_db_path</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="VOCAB_PATH">
+ <div class="attr variable">
+ <span class="name">VOCAB_PATH</span> =
+<span class="default_value">PosixPath('vocab/omop')</span>
+
+
+ </div>
+ <a class="headerlink" href="#VOCAB_PATH"></a>
+
+ <div class="docstring"><p>Default OMOP vocabulary path</p>
+</div>
+
+
+ </section>
+ <section id="OMOP_CDM_Version">
+ <div class="attr variable">
+ <span class="name">OMOP_CDM_Version</span> =
+<span class="default_value">'54'</span>
+
+
+ </div>
+ <a class="headerlink" href="#OMOP_CDM_Version"></a>
+
+ <div class="docstring"><p>Supported OMOP CDM version number</p>
+</div>
+
+
+ </section>
+ <section id="OMOP_DB_FILENAME">
+ <div class="attr variable">
+ <span class="name">OMOP_DB_FILENAME</span> =
+<span class="default_value">'omop_54.sqlite'</span>
+
+
+ </div>
+ <a class="headerlink" href="#OMOP_DB_FILENAME"></a>
+
+ <div class="docstring"><p>Default OMOP sqllite database filename</p>
+</div>
+
+
+ </section>
+ <section id="DB_PATH">
+ <div class="attr variable">
+ <span class="name">DB_PATH</span> =
+<span class="default_value">PosixPath('vocab/omop/omop_54.sqlite')</span>
+
+
+ </div>
+ <a class="headerlink" href="#DB_PATH"></a>
+
+ <div class="docstring"><p>Default OMOP sqllite database path</p>
+</div>
+
+
+ </section>
+ <section id="VERSION_FILE">
+ <div class="attr variable">
+ <span class="name">VERSION_FILE</span> =
+<span class="default_value">'omop_version.yml'</span>
+
+
+ </div>
+ <a class="headerlink" href="#VERSION_FILE"></a>
+
+ <div class="docstring"><p>Default OMOP version file</p>
+</div>
+
+
+ </section>
+ <section id="VERSION_PATH">
+ <div class="attr variable">
+ <span class="name">VERSION_PATH</span> =
+<span class="default_value">PosixPath('vocab/omop/omop_version.yml')</span>
+
+
+ </div>
+ <a class="headerlink" href="#VERSION_PATH"></a>
+
+ <div class="docstring"><p>Default OMOP version path</p>
+</div>
+
+
+ </section>
+ <section id="EXPORT_FILE">
+ <div class="attr variable">
+ <span class="name">EXPORT_FILE</span> =
+<span class="default_value">'omop_54_export.sqlite'</span>
+
+
+ </div>
+ <a class="headerlink" href="#EXPORT_FILE"></a>
+
+ <div class="docstring"><p>Default OMOP export database filename</p>
+</div>
+
+
+ </section>
+ <section id="vocabularies">
+ <div class="attr variable">
+ <span class="name">vocabularies</span> =
+<input id="vocabularies-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+ <label class="view-value-button pdoc-button" for="vocabularies-view-value"></label><span class="default_value">{'source': 'OHDSI Athena', 'url': 'https://athena.ohdsi.org/vocabulary/list', 'cdm_version': '54', 'version': '', 'vocabularies': [{'id': 1, 'name': 'SNOMED'}, {'id': 2, 'name': 'ICD9CM'}, {'id': 17, 'name': 'Readv2'}, {'id': 21, 'name': 'ATC'}, {'id': 55, 'name': 'OPCS4'}, {'id': 57, 'name': 'HES Specialty'}, {'id': 70, 'name': 'ICD10CM'}, {'id': 75, 'name': 'dm+d'}, {'id': 144, 'name': 'UK Biobank'}, {'id': 154, 'name': 'NHS Ethnic Category'}, {'id': 155, 'name': 'NHS Place of Service'}], 'tables': []}</span>
+
+
+ </div>
+ <a class="headerlink" href="#vocabularies"></a>
+
+ <div class="docstring"><p>Required OMOP vocabularies definition</p>
+</div>
+
+
+ </section>
+ <section id="omop_vocab_types">
+ <div class="attr variable">
+ <span class="name">omop_vocab_types</span> =
+<input id="omop_vocab_types-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+ <label class="view-value-button pdoc-button" for="omop_vocab_types-view-value"></label><span class="default_value">{'read2': 'Read', 'read3': None, 'icd10': 'ICD10CM', 'snomed': 'SNOMED', 'opcs4': 'OPCS4', 'atc': 'ATC', 'med': None, 'cprd': None}</span>
+
+
+ </div>
+ <a class="headerlink" href="#omop_vocab_types"></a>
+
+ <div class="docstring"><p>Type mappings from acmc medical coding types to OMOP vocabulary types</p>
+</div>
+
+
+ </section>
+ <section id="install">
+ <input id="install-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">install</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">omop_zip_file</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">version</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="install-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#install"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="install-79"><a href="#install-79"><span class="linenos"> 79</span></a><span class="k">def</span><span class="w"> </span><span class="nf">install</span><span class="p">(</span><span class="n">omop_zip_file</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="install-80"><a href="#install-80"><span class="linenos"> 80</span></a><span class="w"> </span><span class="sd">""" "Installs the OMOP release csv files in a file-based sql database</span>
+</span><span id="install-81"><a href="#install-81"><span class="linenos"> 81</span></a>
+</span><span id="install-82"><a href="#install-82"><span class="linenos"> 82</span></a><span class="sd"> Args:</span>
+</span><span id="install-83"><a href="#install-83"><span class="linenos"> 83</span></a><span class="sd"> omop_zip_file (str): vocabularies zip file distributed by OHDSI Athena</span>
+</span><span id="install-84"><a href="#install-84"><span class="linenos"> 84</span></a><span class="sd"> version (str): version of the vocabularies distributed by OHDSI Athena</span>
+</span><span id="install-85"><a href="#install-85"><span class="linenos"> 85</span></a>
+</span><span id="install-86"><a href="#install-86"><span class="linenos"> 86</span></a><span class="sd"> Raises:</span>
+</span><span id="install-87"><a href="#install-87"><span class="linenos"> 87</span></a><span class="sd"> ValueError: if the zip file does not exist</span>
+</span><span id="install-88"><a href="#install-88"><span class="linenos"> 88</span></a><span class="sd"> ValueError: if the file is not a zip file</span>
+</span><span id="install-89"><a href="#install-89"><span class="linenos"> 89</span></a><span class="sd"> Exception: if error reading omop csv files</span>
+</span><span id="install-90"><a href="#install-90"><span class="linenos"> 90</span></a><span class="sd"> """</span>
+</span><span id="install-91"><a href="#install-91"><span class="linenos"> 91</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Installing OMOP from zip file: </span><span class="si">{</span><span class="n">omop_zip_file</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="install-92"><a href="#install-92"><span class="linenos"> 92</span></a> <span class="n">omop_zip_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">omop_zip_file</span><span class="p">)</span>
+</span><span id="install-93"><a href="#install-93"><span class="linenos"> 93</span></a>
+</span><span id="install-94"><a href="#install-94"><span class="linenos"> 94</span></a> <span class="c1"># Check if the file exists and is a ZIP file</span>
+</span><span id="install-95"><a href="#install-95"><span class="linenos"> 95</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">omop_zip_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="install-96"><a href="#install-96"><span class="linenos"> 96</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">omop_zip_path</span><span class="si">}</span><span class="s2"> does not exist."</span>
+</span><span id="install-97"><a href="#install-97"><span class="linenos"> 97</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="install-98"><a href="#install-98"><span class="linenos"> 98</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="install-99"><a href="#install-99"><span class="linenos"> 99</span></a>
+</span><span id="install-100"><a href="#install-100"><span class="linenos">100</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">zipfile</span><span class="o">.</span><span class="n">is_zipfile</span><span class="p">(</span><span class="n">omop_zip_path</span><span class="p">):</span>
+</span><span id="install-101"><a href="#install-101"><span class="linenos">101</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Error: </span><span class="si">{</span><span class="n">omop_zip_path</span><span class="si">}</span><span class="s2"> is not a valid ZIP file."</span>
+</span><span id="install-102"><a href="#install-102"><span class="linenos">102</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="install-103"><a href="#install-103"><span class="linenos">103</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="install-104"><a href="#install-104"><span class="linenos">104</span></a>
+</span><span id="install-105"><a href="#install-105"><span class="linenos">105</span></a> <span class="c1"># check codes directory exists and if not create it</span>
+</span><span id="install-106"><a href="#install-106"><span class="linenos">106</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="install-107"><a href="#install-107"><span class="linenos">107</span></a> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="install-108"><a href="#install-108"><span class="linenos">108</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP directory '</span><span class="si">{</span><span class="n">VOCAB_PATH</span><span class="si">}</span><span class="s2">' created."</span><span class="p">)</span>
+</span><span id="install-109"><a href="#install-109"><span class="linenos">109</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="install-110"><a href="#install-110"><span class="linenos">110</span></a> <span class="c1"># removing existing OMOP files</span>
+</span><span id="install-111"><a href="#install-111"><span class="linenos">111</span></a> <span class="n">csv_files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"*.csv"</span><span class="p">))</span>
+</span><span id="install-112"><a href="#install-112"><span class="linenos">112</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">csv_files</span><span class="p">:</span>
+</span><span id="install-113"><a href="#install-113"><span class="linenos">113</span></a> <span class="n">file</span><span class="o">.</span><span class="n">unlink</span><span class="p">()</span>
+</span><span id="install-114"><a href="#install-114"><span class="linenos">114</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Deleted OMOP csv file: </span><span class="si">{</span><span class="n">file</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="install-115"><a href="#install-115"><span class="linenos">115</span></a>
+</span><span id="install-116"><a href="#install-116"><span class="linenos">116</span></a> <span class="c1"># Extract ZIP contents</span>
+</span><span id="install-117"><a href="#install-117"><span class="linenos">117</span></a> <span class="k">with</span> <span class="n">zipfile</span><span class="o">.</span><span class="n">ZipFile</span><span class="p">(</span><span class="n">omop_zip_path</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">zip_ref</span><span class="p">:</span>
+</span><span id="install-118"><a href="#install-118"><span class="linenos">118</span></a> <span class="n">zip_ref</span><span class="o">.</span><span class="n">extractall</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="p">)</span>
+</span><span id="install-119"><a href="#install-119"><span class="linenos">119</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted OMOP zip file </span><span class="si">{</span><span class="n">omop_zip_path</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">VOCAB_PATH</span><span class="si">}</span><span class="s2">/"</span><span class="p">)</span>
+</span><span id="install-120"><a href="#install-120"><span class="linenos">120</span></a>
+</span><span id="install-121"><a href="#install-121"><span class="linenos">121</span></a> <span class="c1"># connect to database, if it does not exist it will be created</span>
+</span><span id="install-122"><a href="#install-122"><span class="linenos">122</span></a> <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">DB_PATH</span><span class="p">)</span>
+</span><span id="install-123"><a href="#install-123"><span class="linenos">123</span></a>
+</span><span id="install-124"><a href="#install-124"><span class="linenos">124</span></a> <span class="c1"># Iterate through files in the folder</span>
+</span><span id="install-125"><a href="#install-125"><span class="linenos">125</span></a> <span class="n">csv_files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"*.csv"</span><span class="p">))</span>
+</span><span id="install-126"><a href="#install-126"><span class="linenos">126</span></a> <span class="n">total_tables_count</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">csv_files</span><span class="p">)</span>
+</span><span id="install-127"><a href="#install-127"><span class="linenos">127</span></a> <span class="n">table_count</span> <span class="o">=</span> <span class="mi">1</span>
+</span><span id="install-128"><a href="#install-128"><span class="linenos">128</span></a> <span class="k">for</span> <span class="n">filename</span> <span class="ow">in</span> <span class="n">csv_files</span><span class="p">:</span>
+</span><span id="install-129"><a href="#install-129"><span class="linenos">129</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="install-130"><a href="#install-130"><span class="linenos">130</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
+</span><span id="install-131"><a href="#install-131"><span class="linenos">131</span></a> <span class="sa">f</span><span class="s2">"Processing </span><span class="si">{</span><span class="n">table_count</span><span class="si">}</span><span class="s2"> of </span><span class="si">{</span><span class="n">total_tables_count</span><span class="si">}</span><span class="s2"> tables: </span><span class="si">{</span><span class="n">filename</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="install-132"><a href="#install-132"><span class="linenos">132</span></a> <span class="p">)</span>
+</span><span id="install-133"><a href="#install-133"><span class="linenos">133</span></a> <span class="c1"># read the CSV file with the specified delimiter</span>
+</span><span id="install-134"><a href="#install-134"><span class="linenos">134</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">delimiter</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">low_memory</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="install-135"><a href="#install-135"><span class="linenos">135</span></a>
+</span><span id="install-136"><a href="#install-136"><span class="linenos">136</span></a> <span class="c1"># export Table to sqlite db</span>
+</span><span id="install-137"><a href="#install-137"><span class="linenos">137</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_sql</span><span class="p">(</span><span class="n">filename</span><span class="o">.</span><span class="n">stem</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">if_exists</span><span class="o">=</span><span class="s2">"replace"</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="install-138"><a href="#install-138"><span class="linenos">138</span></a>
+</span><span id="install-139"><a href="#install-139"><span class="linenos">139</span></a> <span class="c1"># add to the metadata</span>
+</span><span id="install-140"><a href="#install-140"><span class="linenos">140</span></a> <span class="nb">list</span><span class="p">(</span><span class="n">vocabularies</span><span class="p">[</span><span class="s2">"tables"</span><span class="p">])</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">filename</span><span class="o">.</span><span class="n">stem</span><span class="p">)</span>
+</span><span id="install-141"><a href="#install-141"><span class="linenos">141</span></a> <span class="n">table_count</span> <span class="o">=</span> <span class="n">table_count</span> <span class="o">+</span> <span class="mi">1</span>
+</span><span id="install-142"><a href="#install-142"><span class="linenos">142</span></a> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="install-143"><a href="#install-143"><span class="linenos">143</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error reading file </span><span class="si">{</span><span class="n">filename</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="install-144"><a href="#install-144"><span class="linenos">144</span></a>
+</span><span id="install-145"><a href="#install-145"><span class="linenos">145</span></a> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="install-146"><a href="#install-146"><span class="linenos">146</span></a>
+</span><span id="install-147"><a href="#install-147"><span class="linenos">147</span></a> <span class="c1"># write version file</span>
+</span><span id="install-148"><a href="#install-148"><span class="linenos">148</span></a> <span class="n">write_version_file</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
+</span><span id="install-149"><a href="#install-149"><span class="linenos">149</span></a>
+</span><span id="install-150"><a href="#install-150"><span class="linenos">150</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP installation completed"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>"Installs the OMOP release csv files in a file-based sql database</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>omop_zip_file (str):</strong> vocabularies zip file distributed by OHDSI Athena</li>
+<li><strong>version (str):</strong> version of the vocabularies distributed by OHDSI Athena</li>
+</ul>
+
+<h6 id="raises">Raises:</h6>
+
+<ul>
+<li><strong>ValueError:</strong> if the zip file does not exist</li>
+<li><strong>ValueError:</strong> if the file is not a zip file</li>
+<li><strong>Exception:</strong> if error reading omop csv files</li>
+</ul>
+</div>
+
+
+ </section>
+ <section id="write_version_file">
+ <input id="write_version_file-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">write_version_file</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">version</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="write_version_file-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#write_version_file"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="write_version_file-153"><a href="#write_version_file-153"><span class="linenos">153</span></a><span class="k">def</span><span class="w"> </span><span class="nf">write_version_file</span><span class="p">(</span><span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="write_version_file-154"><a href="#write_version_file-154"><span class="linenos">154</span></a><span class="w"> </span><span class="sd">"""Writes the OMOP vocaburaries and version to a file</span>
+</span><span id="write_version_file-155"><a href="#write_version_file-155"><span class="linenos">155</span></a>
+</span><span id="write_version_file-156"><a href="#write_version_file-156"><span class="linenos">156</span></a><span class="sd"> Args:</span>
+</span><span id="write_version_file-157"><a href="#write_version_file-157"><span class="linenos">157</span></a><span class="sd"> version (str): version of the vocabularies distributed by OHDSI Athena</span>
+</span><span id="write_version_file-158"><a href="#write_version_file-158"><span class="linenos">158</span></a><span class="sd"> """</span>
+</span><span id="write_version_file-159"><a href="#write_version_file-159"><span class="linenos">159</span></a> <span class="n">vocabularies</span><span class="p">[</span><span class="s2">"version"</span><span class="p">]</span> <span class="o">=</span> <span class="n">version</span>
+</span><span id="write_version_file-160"><a href="#write_version_file-160"><span class="linenos">160</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">VERSION_PATH</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="write_version_file-161"><a href="#write_version_file-161"><span class="linenos">161</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="write_version_file-162"><a href="#write_version_file-162"><span class="linenos">162</span></a> <span class="n">vocabularies</span><span class="p">,</span>
+</span><span id="write_version_file-163"><a href="#write_version_file-163"><span class="linenos">163</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="write_version_file-164"><a href="#write_version_file-164"><span class="linenos">164</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="write_version_file-165"><a href="#write_version_file-165"><span class="linenos">165</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="write_version_file-166"><a href="#write_version_file-166"><span class="linenos">166</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="write_version_file-167"><a href="#write_version_file-167"><span class="linenos">167</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="write_version_file-168"><a href="#write_version_file-168"><span class="linenos">168</span></a> <span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Writes the OMOP vocaburaries and version to a file</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>version (str):</strong> version of the vocabularies distributed by OHDSI Athena</li>
+</ul>
+</div>
+
+
+ </section>
+ <section id="clear">
+ <input id="clear-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">clear</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">db_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="clear-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#clear"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="clear-171"><a href="#clear-171"><span class="linenos">171</span></a><span class="k">def</span><span class="w"> </span><span class="nf">clear</span><span class="p">(</span><span class="n">db_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="clear-172"><a href="#clear-172"><span class="linenos">172</span></a><span class="w"> </span><span class="sd">"""Clears the OMOP sql database</span>
+</span><span id="clear-173"><a href="#clear-173"><span class="linenos">173</span></a>
+</span><span id="clear-174"><a href="#clear-174"><span class="linenos">174</span></a><span class="sd"> Args:</span>
+</span><span id="clear-175"><a href="#clear-175"><span class="linenos">175</span></a><span class="sd"> db_path (Path): the path to the omop sqllite database</span>
+</span><span id="clear-176"><a href="#clear-176"><span class="linenos">176</span></a>
+</span><span id="clear-177"><a href="#clear-177"><span class="linenos">177</span></a><span class="sd"> Raises:</span>
+</span><span id="clear-178"><a href="#clear-178"><span class="linenos">178</span></a><span class="sd"> FileNotFoundError: if the omop sqllite database does not exist</span>
+</span><span id="clear-179"><a href="#clear-179"><span class="linenos">179</span></a><span class="sd"> """</span>
+</span><span id="clear-180"><a href="#clear-180"><span class="linenos">180</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Clearing OMOP data from database"</span><span class="p">)</span>
+</span><span id="clear-181"><a href="#clear-181"><span class="linenos">181</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">db_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="clear-182"><a href="#clear-182"><span class="linenos">182</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error: OMOP DB file '</span><span class="si">{</span><span class="n">db_path</span><span class="si">}</span><span class="s2">' does not exist."</span><span class="p">)</span>
+</span><span id="clear-183"><a href="#clear-183"><span class="linenos">183</span></a> <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">db_path</span><span class="p">)</span>
+</span><span id="clear-184"><a href="#clear-184"><span class="linenos">184</span></a> <span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
+</span><span id="clear-185"><a href="#clear-185"><span class="linenos">185</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"SELECT name FROM sqlite_master WHERE type='table';"</span><span class="p">)</span>
+</span><span id="clear-186"><a href="#clear-186"><span class="linenos">186</span></a>
+</span><span id="clear-187"><a href="#clear-187"><span class="linenos">187</span></a> <span class="c1"># Fetch and print table names</span>
+</span><span id="clear-188"><a href="#clear-188"><span class="linenos">188</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchall</span><span class="p">()</span>
+</span><span id="clear-189"><a href="#clear-189"><span class="linenos">189</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Tables in database:"</span><span class="p">,</span> <span class="p">[</span><span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">])</span>
+</span><span id="clear-190"><a href="#clear-190"><span class="linenos">190</span></a>
+</span><span id="clear-191"><a href="#clear-191"><span class="linenos">191</span></a> <span class="c1"># cur.execute("DROP TABLE CONCEPT_SET;")</span>
+</span><span id="clear-192"><a href="#clear-192"><span class="linenos">192</span></a> <span class="c1"># cur.execute("DROP TABLE CONCEPT_SET_ITEM;")</span>
+</span><span id="clear-193"><a href="#clear-193"><span class="linenos">193</span></a>
+</span><span id="clear-194"><a href="#clear-194"><span class="linenos">194</span></a> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="clear-195"><a href="#clear-195"><span class="linenos">195</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP database cleared"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Clears the OMOP sql database</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>db_path (Path):</strong> the path to the omop sqllite database</li>
+</ul>
+
+<h6 id="raises">Raises:</h6>
+
+<ul>
+<li><strong>FileNotFoundError:</strong> if the omop sqllite database does not exist</li>
+</ul>
+</div>
+
+
+ </section>
+ <section id="delete">
+ <input id="delete-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">delete</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">db_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="delete-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#delete"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="delete-198"><a href="#delete-198"><span class="linenos">198</span></a><span class="k">def</span><span class="w"> </span><span class="nf">delete</span><span class="p">(</span><span class="n">db_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="delete-199"><a href="#delete-199"><span class="linenos">199</span></a><span class="w"> </span><span class="sd">"""Deletes the OMOP sql database</span>
+</span><span id="delete-200"><a href="#delete-200"><span class="linenos">200</span></a>
+</span><span id="delete-201"><a href="#delete-201"><span class="linenos">201</span></a><span class="sd"> Args:</span>
+</span><span id="delete-202"><a href="#delete-202"><span class="linenos">202</span></a><span class="sd"> db_path (Path): the path to the omop sqllite database</span>
+</span><span id="delete-203"><a href="#delete-203"><span class="linenos">203</span></a>
+</span><span id="delete-204"><a href="#delete-204"><span class="linenos">204</span></a><span class="sd"> Raises:</span>
+</span><span id="delete-205"><a href="#delete-205"><span class="linenos">205</span></a><span class="sd"> FileNotFoundError: if the omop sqllite database does not exist</span>
+</span><span id="delete-206"><a href="#delete-206"><span class="linenos">206</span></a><span class="sd"> """</span>
+</span><span id="delete-207"><a href="#delete-207"><span class="linenos">207</span></a>
+</span><span id="delete-208"><a href="#delete-208"><span class="linenos">208</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Deleting OMOP database"</span><span class="p">)</span>
+</span><span id="delete-209"><a href="#delete-209"><span class="linenos">209</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">db_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="delete-210"><a href="#delete-210"><span class="linenos">210</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error: OMOP DB file '</span><span class="si">{</span><span class="n">db_path</span><span class="si">}</span><span class="s2">' does not exist."</span><span class="p">)</span>
+</span><span id="delete-211"><a href="#delete-211"><span class="linenos">211</span></a>
+</span><span id="delete-212"><a href="#delete-212"><span class="linenos">212</span></a> <span class="n">db_path</span><span class="o">.</span><span class="n">unlink</span><span class="p">()</span>
+</span><span id="delete-213"><a href="#delete-213"><span class="linenos">213</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP database deleted"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Deletes the OMOP sql database</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>db_path (Path):</strong> the path to the omop sqllite database</li>
+</ul>
+
+<h6 id="raises">Raises:</h6>
+
+<ul>
+<li><strong>FileNotFoundError:</strong> if the omop sqllite database does not exist</li>
+</ul>
+</div>
+
+
+ </section>
+ <section id="table_exists">
+ <input id="table_exists-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">table_exists</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span>, </span><span class="param"><span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
+
+ <label class="view-source-button" for="table_exists-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#table_exists"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="table_exists-216"><a href="#table_exists-216"><span class="linenos">216</span></a><span class="k">def</span><span class="w"> </span><span class="nf">table_exists</span><span class="p">(</span><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span><span class="p">,</span> <span class="n">table_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="table_exists-217"><a href="#table_exists-217"><span class="linenos">217</span></a><span class="w"> </span><span class="sd">"""Query to check if the table exists</span>
+</span><span id="table_exists-218"><a href="#table_exists-218"><span class="linenos">218</span></a>
+</span><span id="table_exists-219"><a href="#table_exists-219"><span class="linenos">219</span></a><span class="sd"> Args:</span>
+</span><span id="table_exists-220"><a href="#table_exists-220"><span class="linenos">220</span></a><span class="sd"> cursor (sqlite3.Cursor): a sqllite database cursor</span>
+</span><span id="table_exists-221"><a href="#table_exists-221"><span class="linenos">221</span></a><span class="sd"> table_name (str): the table name to check</span>
+</span><span id="table_exists-222"><a href="#table_exists-222"><span class="linenos">222</span></a>
+</span><span id="table_exists-223"><a href="#table_exists-223"><span class="linenos">223</span></a><span class="sd"> Returns:</span>
+</span><span id="table_exists-224"><a href="#table_exists-224"><span class="linenos">224</span></a><span class="sd"> bool: true if table exists</span>
+</span><span id="table_exists-225"><a href="#table_exists-225"><span class="linenos">225</span></a><span class="sd"> """</span>
+</span><span id="table_exists-226"><a href="#table_exists-226"><span class="linenos">226</span></a>
+</span><span id="table_exists-227"><a href="#table_exists-227"><span class="linenos">227</span></a> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="table_exists-228"><a href="#table_exists-228"><span class="linenos">228</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="table_exists-229"><a href="#table_exists-229"><span class="linenos">229</span></a><span class="sd"> SELECT name</span>
+</span><span id="table_exists-230"><a href="#table_exists-230"><span class="linenos">230</span></a><span class="sd"> FROM sqlite_master</span>
+</span><span id="table_exists-231"><a href="#table_exists-231"><span class="linenos">231</span></a><span class="sd"> WHERE type='table' AND name=?</span>
+</span><span id="table_exists-232"><a href="#table_exists-232"><span class="linenos">232</span></a><span class="sd"> """</span><span class="p">,</span>
+</span><span id="table_exists-233"><a href="#table_exists-233"><span class="linenos">233</span></a> <span class="p">(</span><span class="n">table_name</span><span class="p">,),</span>
+</span><span id="table_exists-234"><a href="#table_exists-234"><span class="linenos">234</span></a> <span class="p">)</span>
+</span><span id="table_exists-235"><a href="#table_exists-235"><span class="linenos">235</span></a>
+</span><span id="table_exists-236"><a href="#table_exists-236"><span class="linenos">236</span></a> <span class="c1"># Fetch the result</span>
+</span><span id="table_exists-237"><a href="#table_exists-237"><span class="linenos">237</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>
+</span><span id="table_exists-238"><a href="#table_exists-238"><span class="linenos">238</span></a>
+</span><span id="table_exists-239"><a href="#table_exists-239"><span class="linenos">239</span></a> <span class="k">return</span> <span class="n">result</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Query to check if the table exists</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>cursor (sqlite3.Cursor):</strong> a sqllite database cursor</li>
+<li><strong>table_name (str):</strong> the table name to check</li>
+</ul>
+
+<h6 id="returns">Returns:</h6>
+
+<blockquote>
+ <p>bool: true if table exists</p>
+</blockquote>
+</div>
+
+
+ </section>
+ <section id="vocab_exists">
+ <input id="vocab_exists-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">vocab_exists</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span>, </span><span class="param"><span class="n">vocab_id</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
+
+ <label class="view-source-button" for="vocab_exists-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#vocab_exists"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="vocab_exists-242"><a href="#vocab_exists-242"><span class="linenos">242</span></a><span class="k">def</span><span class="w"> </span><span class="nf">vocab_exists</span><span class="p">(</span><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span><span class="p">,</span> <span class="n">vocab_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="vocab_exists-243"><a href="#vocab_exists-243"><span class="linenos">243</span></a><span class="w"> </span><span class="sd">"""Query to check if the vocabulary exists</span>
+</span><span id="vocab_exists-244"><a href="#vocab_exists-244"><span class="linenos">244</span></a>
+</span><span id="vocab_exists-245"><a href="#vocab_exists-245"><span class="linenos">245</span></a><span class="sd"> Args:</span>
+</span><span id="vocab_exists-246"><a href="#vocab_exists-246"><span class="linenos">246</span></a><span class="sd"> cursor (sqlite3.Cursor): a sqllite database cursor</span>
+</span><span id="vocab_exists-247"><a href="#vocab_exists-247"><span class="linenos">247</span></a><span class="sd"> vocab_id (str): the vocabulary id to check</span>
+</span><span id="vocab_exists-248"><a href="#vocab_exists-248"><span class="linenos">248</span></a>
+</span><span id="vocab_exists-249"><a href="#vocab_exists-249"><span class="linenos">249</span></a><span class="sd"> Returns:</span>
+</span><span id="vocab_exists-250"><a href="#vocab_exists-250"><span class="linenos">250</span></a><span class="sd"> bool: true if vocabulary id exists</span>
+</span><span id="vocab_exists-251"><a href="#vocab_exists-251"><span class="linenos">251</span></a><span class="sd"> """</span>
+</span><span id="vocab_exists-252"><a href="#vocab_exists-252"><span class="linenos">252</span></a>
+</span><span id="vocab_exists-253"><a href="#vocab_exists-253"><span class="linenos">253</span></a> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="vocab_exists-254"><a href="#vocab_exists-254"><span class="linenos">254</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="vocab_exists-255"><a href="#vocab_exists-255"><span class="linenos">255</span></a><span class="sd"> SELECT vocabulary_id </span>
+</span><span id="vocab_exists-256"><a href="#vocab_exists-256"><span class="linenos">256</span></a><span class="sd"> FROM VOCABULARY</span>
+</span><span id="vocab_exists-257"><a href="#vocab_exists-257"><span class="linenos">257</span></a><span class="sd"> WHERE vocabulary_id=?</span>
+</span><span id="vocab_exists-258"><a href="#vocab_exists-258"><span class="linenos">258</span></a><span class="sd"> """</span><span class="p">,</span>
+</span><span id="vocab_exists-259"><a href="#vocab_exists-259"><span class="linenos">259</span></a> <span class="p">(</span><span class="n">vocab_id</span><span class="p">,),</span>
+</span><span id="vocab_exists-260"><a href="#vocab_exists-260"><span class="linenos">260</span></a> <span class="p">)</span>
+</span><span id="vocab_exists-261"><a href="#vocab_exists-261"><span class="linenos">261</span></a>
+</span><span id="vocab_exists-262"><a href="#vocab_exists-262"><span class="linenos">262</span></a> <span class="c1"># Fetch the result</span>
+</span><span id="vocab_exists-263"><a href="#vocab_exists-263"><span class="linenos">263</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>
+</span><span id="vocab_exists-264"><a href="#vocab_exists-264"><span class="linenos">264</span></a>
+</span><span id="vocab_exists-265"><a href="#vocab_exists-265"><span class="linenos">265</span></a> <span class="k">return</span> <span class="n">result</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Query to check if the vocabulary exists</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>cursor (sqlite3.Cursor):</strong> a sqllite database cursor</li>
+<li><strong>vocab_id (str):</strong> the vocabulary id to check</li>
+</ul>
+
+<h6 id="returns">Returns:</h6>
+
+<blockquote>
+ <p>bool: true if vocabulary id exists</p>
+</blockquote>
+</div>
+
+
+ </section>
+ <section id="concept_set_exist">
+ <input id="concept_set_exist-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">concept_set_exist</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span>, </span><span class="param"><span class="n">concept_set_name</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">) -> <span class="nb">bool</span>:</span></span>
+
+ <label class="view-source-button" for="concept_set_exist-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#concept_set_exist"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="concept_set_exist-268"><a href="#concept_set_exist-268"><span class="linenos">268</span></a><span class="k">def</span><span class="w"> </span><span class="nf">concept_set_exist</span><span class="p">(</span><span class="n">cursor</span><span class="p">:</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">Cursor</span><span class="p">,</span> <span class="n">concept_set_name</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="concept_set_exist-269"><a href="#concept_set_exist-269"><span class="linenos">269</span></a><span class="w"> </span><span class="sd">"""Query to check if the concept set exists</span>
+</span><span id="concept_set_exist-270"><a href="#concept_set_exist-270"><span class="linenos">270</span></a>
+</span><span id="concept_set_exist-271"><a href="#concept_set_exist-271"><span class="linenos">271</span></a><span class="sd"> Args:</span>
+</span><span id="concept_set_exist-272"><a href="#concept_set_exist-272"><span class="linenos">272</span></a><span class="sd"> cursor (sqlite3.Cursor): a sqllite database cursor</span>
+</span><span id="concept_set_exist-273"><a href="#concept_set_exist-273"><span class="linenos">273</span></a><span class="sd"> concept_set_name (str): the concept set name to check</span>
+</span><span id="concept_set_exist-274"><a href="#concept_set_exist-274"><span class="linenos">274</span></a>
+</span><span id="concept_set_exist-275"><a href="#concept_set_exist-275"><span class="linenos">275</span></a><span class="sd"> Returns:</span>
+</span><span id="concept_set_exist-276"><a href="#concept_set_exist-276"><span class="linenos">276</span></a><span class="sd"> bool: true if concept set exists</span>
+</span><span id="concept_set_exist-277"><a href="#concept_set_exist-277"><span class="linenos">277</span></a><span class="sd"> """</span>
+</span><span id="concept_set_exist-278"><a href="#concept_set_exist-278"><span class="linenos">278</span></a>
+</span><span id="concept_set_exist-279"><a href="#concept_set_exist-279"><span class="linenos">279</span></a> <span class="n">query</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"SELECT EXISTS (SELECT 1 FROM CONCEPT_SET WHERE concept_set_name = ?)"</span>
+</span><span id="concept_set_exist-280"><a href="#concept_set_exist-280"><span class="linenos">280</span></a> <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="p">(</span><span class="n">concept_set_name</span><span class="p">,))</span>
+</span><span id="concept_set_exist-281"><a href="#concept_set_exist-281"><span class="linenos">281</span></a>
+</span><span id="concept_set_exist-282"><a href="#concept_set_exist-282"><span class="linenos">282</span></a> <span class="c1"># 1 if exists, 0 otherwise</span>
+</span><span id="concept_set_exist-283"><a href="#concept_set_exist-283"><span class="linenos">283</span></a> <span class="k">return</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="mi">1</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Query to check if the concept set exists</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>cursor (sqlite3.Cursor):</strong> a sqllite database cursor</li>
+<li><strong>concept_set_name (str):</strong> the concept set name to check</li>
+</ul>
+
+<h6 id="returns">Returns:</h6>
+
+<blockquote>
+ <p>bool: true if concept set exists</p>
+</blockquote>
+</div>
+
+
+ </section>
+ <section id="export">
+ <input id="export-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">export</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">map_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>,</span><span class="param"> <span class="n">export_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>,</span><span class="param"> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">omop_metadata</span><span class="p">:</span> <span class="nb">dict</span></span><span class="return-annotation">) -> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>:</span></span>
+
+ <label class="view-source-button" for="export-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#export"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="export-286"><a href="#export-286"><span class="linenos">286</span></a><span class="k">def</span><span class="w"> </span><span class="nf">export</span><span class="p">(</span>
+</span><span id="export-287"><a href="#export-287"><span class="linenos">287</span></a> <span class="n">map_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">export_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">omop_metadata</span><span class="p">:</span> <span class="nb">dict</span>
+</span><span id="export-288"><a href="#export-288"><span class="linenos">288</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Path</span><span class="p">:</span>
+</span><span id="export-289"><a href="#export-289"><span class="linenos">289</span></a><span class="w"> </span><span class="sd">"""Export concept sets to omop database in csv format</span>
+</span><span id="export-290"><a href="#export-290"><span class="linenos">290</span></a>
+</span><span id="export-291"><a href="#export-291"><span class="linenos">291</span></a><span class="sd"> Args:</span>
+</span><span id="export-292"><a href="#export-292"><span class="linenos">292</span></a><span class="sd"> map_path (Path): path to the acmc map directory containing concept sets in csv format</span>
+</span><span id="export-293"><a href="#export-293"><span class="linenos">293</span></a><span class="sd"> export_path (Path): path to the directory where the omop database csv files are to be written</span>
+</span><span id="export-294"><a href="#export-294"><span class="linenos">294</span></a><span class="sd"> version (str): phenotype version for omop vocabulary version</span>
+</span><span id="export-295"><a href="#export-295"><span class="linenos">295</span></a><span class="sd"> omop_metadata (dict): phenotype omop metadata for omop vocabulary metadata</span>
+</span><span id="export-296"><a href="#export-296"><span class="linenos">296</span></a>
+</span><span id="export-297"><a href="#export-297"><span class="linenos">297</span></a><span class="sd"> Returns:</span>
+</span><span id="export-298"><a href="#export-298"><span class="linenos">298</span></a><span class="sd"> Path: path to the exported sqllite database</span>
+</span><span id="export-299"><a href="#export-299"><span class="linenos">299</span></a><span class="sd"> """</span>
+</span><span id="export-300"><a href="#export-300"><span class="linenos">300</span></a>
+</span><span id="export-301"><a href="#export-301"><span class="linenos">301</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"exporting with metadata </span><span class="si">{</span><span class="n">omop_metadata</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="export-302"><a href="#export-302"><span class="linenos">302</span></a>
+</span><span id="export-303"><a href="#export-303"><span class="linenos">303</span></a> <span class="c1"># copy the baseline omop database</span>
+</span><span id="export-304"><a href="#export-304"><span class="linenos">304</span></a> <span class="n">export_db_path</span> <span class="o">=</span> <span class="n">export_path</span> <span class="o">/</span> <span class="n">EXPORT_FILE</span>
+</span><span id="export-305"><a href="#export-305"><span class="linenos">305</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">copy</span><span class="p">(</span><span class="n">DB_PATH</span><span class="p">,</span> <span class="n">export_db_path</span><span class="p">)</span>
+</span><span id="export-306"><a href="#export-306"><span class="linenos">306</span></a>
+</span><span id="export-307"><a href="#export-307"><span class="linenos">307</span></a> <span class="c1"># connect to db</span>
+</span><span id="export-308"><a href="#export-308"><span class="linenos">308</span></a> <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">export_db_path</span><span class="p">)</span>
+</span><span id="export-309"><a href="#export-309"><span class="linenos">309</span></a> <span class="n">cur</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
+</span><span id="export-310"><a href="#export-310"><span class="linenos">310</span></a>
+</span><span id="export-311"><a href="#export-311"><span class="linenos">311</span></a> <span class="c1"># Create VOCABULARY</span>
+</span><span id="export-312"><a href="#export-312"><span class="linenos">312</span></a> <span class="n">df_test</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="export-313"><a href="#export-313"><span class="linenos">313</span></a> <span class="p">[</span>
+</span><span id="export-314"><a href="#export-314"><span class="linenos">314</span></a> <span class="p">{</span>
+</span><span id="export-315"><a href="#export-315"><span class="linenos">315</span></a> <span class="s2">"vocabulary_id"</span><span class="p">:</span> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_id"</span><span class="p">],</span>
+</span><span id="export-316"><a href="#export-316"><span class="linenos">316</span></a> <span class="s2">"vocabulary_name"</span><span class="p">:</span> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_name"</span><span class="p">],</span>
+</span><span id="export-317"><a href="#export-317"><span class="linenos">317</span></a> <span class="s2">"vocabulary_reference"</span><span class="p">:</span> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_reference"</span><span class="p">],</span>
+</span><span id="export-318"><a href="#export-318"><span class="linenos">318</span></a> <span class="s2">"vocabulary_version"</span><span class="p">:</span> <span class="n">version</span><span class="p">,</span>
+</span><span id="export-319"><a href="#export-319"><span class="linenos">319</span></a> <span class="c1"># "vocabulary_concept_id": 0,</span>
+</span><span id="export-320"><a href="#export-320"><span class="linenos">320</span></a> <span class="p">}</span>
+</span><span id="export-321"><a href="#export-321"><span class="linenos">321</span></a> <span class="p">]</span>
+</span><span id="export-322"><a href="#export-322"><span class="linenos">322</span></a> <span class="p">)</span>
+</span><span id="export-323"><a href="#export-323"><span class="linenos">323</span></a> <span class="n">df_test</span><span class="o">.</span><span class="n">to_sql</span><span class="p">(</span><span class="s2">"VOCABULARY"</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">if_exists</span><span class="o">=</span><span class="s2">"append"</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="export-324"><a href="#export-324"><span class="linenos">324</span></a>
+</span><span id="export-325"><a href="#export-325"><span class="linenos">325</span></a> <span class="c1"># Create CONCEPT_SET</span>
+</span><span id="export-326"><a href="#export-326"><span class="linenos">326</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="export-327"><a href="#export-327"><span class="linenos">327</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="export-328"><a href="#export-328"><span class="linenos">328</span></a><span class="sd"> CREATE TABLE CONCEPT_SET (</span>
+</span><span id="export-329"><a href="#export-329"><span class="linenos">329</span></a><span class="sd"> concept_set_id INTEGER PRIMARY KEY AUTOINCREMENT, -- Unique identifier for each concept set</span>
+</span><span id="export-330"><a href="#export-330"><span class="linenos">330</span></a><span class="sd"> atlas_id INTEGER, -- Unique identifier generated by ATLAS</span>
+</span><span id="export-331"><a href="#export-331"><span class="linenos">331</span></a><span class="sd"> concept_set_name TEXT, -- Optional name for the concept set</span>
+</span><span id="export-332"><a href="#export-332"><span class="linenos">332</span></a><span class="sd"> concept_set_description TEXT, -- Optional description for the concept set</span>
+</span><span id="export-333"><a href="#export-333"><span class="linenos">333</span></a><span class="sd"> vocabulary_id TEXT NOT NULL, -- Foreign key to VOCABULARY table</span>
+</span><span id="export-334"><a href="#export-334"><span class="linenos">334</span></a><span class="sd"> FOREIGN KEY (vocabulary_id) REFERENCES VOCABULARY(vocabulary_id)</span>
+</span><span id="export-335"><a href="#export-335"><span class="linenos">335</span></a><span class="sd"> );"""</span>
+</span><span id="export-336"><a href="#export-336"><span class="linenos">336</span></a> <span class="p">)</span>
+</span><span id="export-337"><a href="#export-337"><span class="linenos">337</span></a>
+</span><span id="export-338"><a href="#export-338"><span class="linenos">338</span></a> <span class="c1"># Create CONCEPT_SET_ITEM</span>
+</span><span id="export-339"><a href="#export-339"><span class="linenos">339</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="export-340"><a href="#export-340"><span class="linenos">340</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="export-341"><a href="#export-341"><span class="linenos">341</span></a><span class="sd"> CREATE TABLE CONCEPT_SET_ITEM (</span>
+</span><span id="export-342"><a href="#export-342"><span class="linenos">342</span></a><span class="sd"> concept_set_item_id INTEGER PRIMARY KEY AUTOINCREMENT, -- Unique identifier for each mapping</span>
+</span><span id="export-343"><a href="#export-343"><span class="linenos">343</span></a><span class="sd"> concept_set_id INTEGER NOT NULL, -- Foreign key to CONCEPT_SET table</span>
+</span><span id="export-344"><a href="#export-344"><span class="linenos">344</span></a><span class="sd"> concept_id INTEGER NOT NULL, -- Foreign key to CONCEPT table</span>
+</span><span id="export-345"><a href="#export-345"><span class="linenos">345</span></a><span class="sd"> FOREIGN KEY (concept_set_id) REFERENCES CONCEPT_SET(concept_set_id),</span>
+</span><span id="export-346"><a href="#export-346"><span class="linenos">346</span></a><span class="sd"> FOREIGN KEY (concept_id) REFERENCES CONCEPT(concept_id)</span>
+</span><span id="export-347"><a href="#export-347"><span class="linenos">347</span></a><span class="sd"> );"""</span>
+</span><span id="export-348"><a href="#export-348"><span class="linenos">348</span></a> <span class="p">)</span>
+</span><span id="export-349"><a href="#export-349"><span class="linenos">349</span></a>
+</span><span id="export-350"><a href="#export-350"><span class="linenos">350</span></a> <span class="c1"># read map files</span>
+</span><span id="export-351"><a href="#export-351"><span class="linenos">351</span></a> <span class="n">map_files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">map_path</span><span class="o">.</span><span class="n">glob</span><span class="p">(</span><span class="s2">"*.csv"</span><span class="p">))</span>
+</span><span id="export-352"><a href="#export-352"><span class="linenos">352</span></a> <span class="n">total</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">map_files</span><span class="p">)</span>
+</span><span id="export-353"><a href="#export-353"><span class="linenos">353</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exporting </span><span class="si">{</span><span class="n">total</span><span class="si">}</span><span class="s2"> map files"</span><span class="p">)</span>
+</span><span id="export-354"><a href="#export-354"><span class="linenos">354</span></a> <span class="k">for</span> <span class="n">index</span><span class="p">,</span> <span class="n">map_file</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">map_files</span><span class="p">):</span>
+</span><span id="export-355"><a href="#export-355"><span class="linenos">355</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Processing </span><span class="si">{</span><span class="n">index</span><span class="o">+</span><span class="mi">1</span><span class="si">}</span><span class="s2"> of </span><span class="si">{</span><span class="n">total</span><span class="si">}</span><span class="s2">: </span><span class="si">{</span><span class="n">map_file</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="export-356"><a href="#export-356"><span class="linenos">356</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">map_file</span><span class="p">)</span>
+</span><span id="export-357"><a href="#export-357"><span class="linenos">357</span></a>
+</span><span id="export-358"><a href="#export-358"><span class="linenos">358</span></a> <span class="k">for</span> <span class="n">concept_set_name</span><span class="p">,</span> <span class="n">grp</span> <span class="ow">in</span> <span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">):</span>
+</span><span id="export-359"><a href="#export-359"><span class="linenos">359</span></a> <span class="c1"># create Concept_Set</span>
+</span><span id="export-360"><a href="#export-360"><span class="linenos">360</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">concept_set_exist</span><span class="p">(</span><span class="n">cur</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">concept_set_name</span><span class="p">)):</span>
+</span><span id="export-361"><a href="#export-361"><span class="linenos">361</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="export-362"><a href="#export-362"><span class="linenos">362</span></a> <span class="sa">f</span><span class="s2">"INSERT INTO CONCEPT_SET (concept_set_name, vocabulary_id) VALUES ('</span><span class="si">{</span><span class="n">concept_set_name</span><span class="si">}</span><span class="s2">', '</span><span class="si">{</span><span class="n">omop_metadata</span><span class="p">[</span><span class="s1">'vocabulary_id'</span><span class="p">]</span><span class="si">}</span><span class="s2">');"</span>
+</span><span id="export-363"><a href="#export-363"><span class="linenos">363</span></a> <span class="p">)</span>
+</span><span id="export-364"><a href="#export-364"><span class="linenos">364</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="export-365"><a href="#export-365"><span class="linenos">365</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Concept_set </span><span class="si">{</span><span class="n">concept_set_name</span><span class="si">}</span><span class="s2"> already exists"</span><span class="p">)</span>
+</span><span id="export-366"><a href="#export-366"><span class="linenos">366</span></a> <span class="c1"># TODO: ask to remove old concept_set?</span>
+</span><span id="export-367"><a href="#export-367"><span class="linenos">367</span></a>
+</span><span id="export-368"><a href="#export-368"><span class="linenos">368</span></a> <span class="c1"># get Concept_set_Id</span>
+</span><span id="export-369"><a href="#export-369"><span class="linenos">369</span></a> <span class="n">query</span> <span class="o">=</span> <span class="s2">"SELECT concept_set_id FROM CONCEPT_SET WHERE concept_set_name = ? AND vocabulary_id = ?;"</span>
+</span><span id="export-370"><a href="#export-370"><span class="linenos">370</span></a> <span class="n">target_code_type</span> <span class="o">=</span> <span class="n">map_file</span><span class="o">.</span><span class="n">stem</span>
+</span><span id="export-371"><a href="#export-371"><span class="linenos">371</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span>
+</span><span id="export-372"><a href="#export-372"><span class="linenos">372</span></a> <span class="n">query</span><span class="p">,</span>
+</span><span id="export-373"><a href="#export-373"><span class="linenos">373</span></a> <span class="p">(</span>
+</span><span id="export-374"><a href="#export-374"><span class="linenos">374</span></a> <span class="n">concept_set_name</span><span class="p">,</span>
+</span><span id="export-375"><a href="#export-375"><span class="linenos">375</span></a> <span class="n">omop_metadata</span><span class="p">[</span><span class="s2">"vocabulary_id"</span><span class="p">],</span>
+</span><span id="export-376"><a href="#export-376"><span class="linenos">376</span></a> <span class="p">),</span>
+</span><span id="export-377"><a href="#export-377"><span class="linenos">377</span></a> <span class="p">)</span>
+</span><span id="export-378"><a href="#export-378"><span class="linenos">378</span></a> <span class="c1"># FAILS HERE WITH NONE REUR</span>
+</span><span id="export-379"><a href="#export-379"><span class="linenos">379</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="export-380"><a href="#export-380"><span class="linenos">380</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"omop code type </span><span class="si">{</span><span class="n">omop_vocab_types</span><span class="p">[</span><span class="n">target_code_type</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="export-381"><a href="#export-381"><span class="linenos">381</span></a> <span class="n">concept_set_id</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span>
+</span><span id="export-382"><a href="#export-382"><span class="linenos">382</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"concept set id </span><span class="si">{</span><span class="n">concept_set_id</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="export-383"><a href="#export-383"><span class="linenos">383</span></a>
+</span><span id="export-384"><a href="#export-384"><span class="linenos">384</span></a> <span class="c1"># get corresponing Concept_id (OMOP) for each Concept_code (e.g. SNOMED)</span>
+</span><span id="export-385"><a href="#export-385"><span class="linenos">385</span></a> <span class="n">concept_codes</span> <span class="o">=</span> <span class="s2">"'"</span> <span class="o">+</span> <span class="s2">"', '"</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">grp</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">]</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)))</span> <span class="o">+</span> <span class="s2">"'"</span>
+</span><span id="export-386"><a href="#export-386"><span class="linenos">386</span></a> <span class="n">query</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"SELECT concept_id FROM CONCEPT WHERE vocabulary_id = ? AND concept_code IN (</span><span class="si">{</span><span class="n">concept_codes</span><span class="si">}</span><span class="s2">);"</span>
+</span><span id="export-387"><a href="#export-387"><span class="linenos">387</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="p">(</span><span class="n">omop_vocab_types</span><span class="p">[</span><span class="n">target_code_type</span><span class="p">],))</span>
+</span><span id="export-388"><a href="#export-388"><span class="linenos">388</span></a> <span class="n">df_out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">cur</span><span class="o">.</span><span class="n">fetchall</span><span class="p">(),</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s2">"concept_id"</span><span class="p">])</span>
+</span><span id="export-389"><a href="#export-389"><span class="linenos">389</span></a>
+</span><span id="export-390"><a href="#export-390"><span class="linenos">390</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="nb">len</span><span class="p">(</span><span class="n">grp</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">df_out</span><span class="p">):</span>
+</span><span id="export-391"><a href="#export-391"><span class="linenos">391</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
+</span><span id="export-392"><a href="#export-392"><span class="linenos">392</span></a> <span class="sa">f</span><span class="s2">"ERROR: Some </span><span class="si">{</span><span class="n">omop_vocab_types</span><span class="p">[</span><span class="n">target_code_type</span><span class="p">]</span><span class="si">}</span><span class="s2"> Codes do not exist in OMOP Database"</span>
+</span><span id="export-393"><a href="#export-393"><span class="linenos">393</span></a> <span class="p">)</span>
+</span><span id="export-394"><a href="#export-394"><span class="linenos">394</span></a>
+</span><span id="export-395"><a href="#export-395"><span class="linenos">395</span></a> <span class="c1"># Create Concept_set_item</span>
+</span><span id="export-396"><a href="#export-396"><span class="linenos">396</span></a> <span class="n">df_out</span><span class="p">[</span><span class="s2">"concept_set_id"</span><span class="p">]</span> <span class="o">=</span> <span class="n">concept_set_id</span>
+</span><span id="export-397"><a href="#export-397"><span class="linenos">397</span></a> <span class="n">df_out</span><span class="o">.</span><span class="n">to_sql</span><span class="p">(</span><span class="s2">"CONCEPT_SET_ITEM"</span><span class="p">,</span> <span class="n">conn</span><span class="p">,</span> <span class="n">if_exists</span><span class="o">=</span><span class="s2">"append"</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="export-398"><a href="#export-398"><span class="linenos">398</span></a>
+</span><span id="export-399"><a href="#export-399"><span class="linenos">399</span></a> <span class="c1"># Output all tables to CSV</span>
+</span><span id="export-400"><a href="#export-400"><span class="linenos">400</span></a> <span class="c1"># Get the list of all tables</span>
+</span><span id="export-401"><a href="#export-401"><span class="linenos">401</span></a> <span class="n">cur</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"SELECT name FROM sqlite_master WHERE type='table';"</span><span class="p">)</span>
+</span><span id="export-402"><a href="#export-402"><span class="linenos">402</span></a> <span class="n">tables</span> <span class="o">=</span> <span class="n">cur</span><span class="o">.</span><span class="n">fetchall</span><span class="p">()</span> <span class="c1"># List of tables</span>
+</span><span id="export-403"><a href="#export-403"><span class="linenos">403</span></a>
+</span><span id="export-404"><a href="#export-404"><span class="linenos">404</span></a> <span class="c1"># Export each table to a separate CSV file</span>
+</span><span id="export-405"><a href="#export-405"><span class="linenos">405</span></a> <span class="k">for</span> <span class="n">table</span> <span class="ow">in</span> <span class="n">tables</span><span class="p">:</span>
+</span><span id="export-406"><a href="#export-406"><span class="linenos">406</span></a> <span class="n">table_name</span> <span class="o">=</span> <span class="n">table</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
+</span><span id="export-407"><a href="#export-407"><span class="linenos">407</span></a> <span class="c1"># ignore SQLite's internal system table</span>
+</span><span id="export-408"><a href="#export-408"><span class="linenos">408</span></a> <span class="k">if</span> <span class="n">table_name</span> <span class="o">!=</span> <span class="s2">"sqlite_sequence"</span><span class="p">:</span>
+</span><span id="export-409"><a href="#export-409"><span class="linenos">409</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_sql_query</span><span class="p">(</span><span class="sa">f</span><span class="s2">"SELECT * FROM </span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="n">conn</span><span class="p">)</span>
+</span><span id="export-410"><a href="#export-410"><span class="linenos">410</span></a> <span class="n">output_file</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2">.csv"</span>
+</span><span id="export-411"><a href="#export-411"><span class="linenos">411</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">export_path</span> <span class="o">/</span> <span class="n">output_file</span>
+</span><span id="export-412"><a href="#export-412"><span class="linenos">412</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="c1"># Save as CSV</span>
+</span><span id="export-413"><a href="#export-413"><span class="linenos">413</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exported </span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">table_name</span><span class="si">}</span><span class="s2">.csv"</span><span class="p">)</span>
+</span><span id="export-414"><a href="#export-414"><span class="linenos">414</span></a>
+</span><span id="export-415"><a href="#export-415"><span class="linenos">415</span></a> <span class="n">conn</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="export-416"><a href="#export-416"><span class="linenos">416</span></a>
+</span><span id="export-417"><a href="#export-417"><span class="linenos">417</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Created export db successfully"</span><span class="p">)</span>
+</span><span id="export-418"><a href="#export-418"><span class="linenos">418</span></a>
+</span><span id="export-419"><a href="#export-419"><span class="linenos">419</span></a> <span class="k">return</span> <span class="n">export_db_path</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Export concept sets to omop database in csv format</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>map_path (Path):</strong> path to the acmc map directory containing concept sets in csv format</li>
+<li><strong>export_path (Path):</strong> path to the directory where the omop database csv files are to be written</li>
+<li><strong>version (str):</strong> phenotype version for omop vocabulary version</li>
+<li><strong>omop_metadata (dict):</strong> phenotype omop metadata for omop vocabulary metadata</li>
+</ul>
+
+<h6 id="returns">Returns:</h6>
+
+<blockquote>
+ <p>Path: path to the exported sqllite database</p>
+</blockquote>
+</div>
+
+
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/parse.html b/docs/api/acmc/parse.html
new file mode 100644
index 0000000000000000000000000000000000000000..9aae143bd59130f29c9569be435280607a9a5593
--- /dev/null
+++ b/docs/api/acmc/parse.html
@@ -0,0 +1,2203 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.parse API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#SUPPORTED_CODE_TYPES">SUPPORTED_CODE_TYPES</a>
+ </li>
+ <li>
+ <a class="class" href="#CodesError">CodesError</a>
+ <ul class="memberlist">
+ <li>
+ <a class="function" href="#CodesError.__init__">CodesError</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#InvalidCodesException">InvalidCodesException</a>
+ <ul class="memberlist">
+ <li>
+ <a class="function" href="#InvalidCodesException.__init__">InvalidCodesException</a>
+ </li>
+ <li>
+ <a class="variable" href="#InvalidCodesException.error">error</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Proto">Proto</a>
+ <ul class="memberlist">
+ <li>
+ <a class="function" href="#Proto.__init__">Proto</a>
+ </li>
+ <li>
+ <a class="variable" href="#Proto.checks">checks</a>
+ </li>
+ <li>
+ <a class="variable" href="#Proto.name">name</a>
+ </li>
+ <li>
+ <a class="function" href="#Proto.raise_exception">raise_exception</a>
+ </li>
+ <li>
+ <a class="function" href="#Proto.in_database">in_database</a>
+ </li>
+ <li>
+ <a class="function" href="#Proto.process">process</a>
+ </li>
+ <li>
+ <a class="function" href="#Proto.verify">verify</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Read2">Read2</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Read2.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Read3">Read3</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Read3.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Icd10">Icd10</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Icd10.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Snomed">Snomed</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Snomed.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Opcs4">Opcs4</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Opcs4.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Atc">Atc</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Atc.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Med">Med</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Med.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#Cprd">Cprd</a>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#Cprd.checks">checks</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="class" href="#CodeTypeParser">CodeTypeParser</a>
+ <ul class="memberlist">
+ <li>
+ <a class="function" href="#CodeTypeParser.__init__">CodeTypeParser</a>
+ </li>
+ <li>
+ <a class="variable" href="#CodeTypeParser.code_types">code_types</a>
+ </li>
+ </ul>
+
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.parse </h1>
+
+ <div class="docstring"><p>parse.py module</p>
+
+<p>This module provides functionality to set up medical code translation classes</p>
+</div>
+
+ <input id="mod-parse-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-parse-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="sd">"""</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="sd">parse.py module</span>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">This module provides functionality to set up medical code translation classes</span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="sd">"""</span>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
+</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">os</span>
+</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Callable</span><span class="p">,</span> <span class="n">Optional</span><span class="p">,</span> <span class="n">Tuple</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">pathlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">Path</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">acmc</span><span class="w"> </span><span class="kn">import</span> <span class="n">trud</span><span class="p">,</span> <span class="n">logging_config</span> <span class="k">as</span> <span class="n">lc</span>
+</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a>
+</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="c1"># setup logging</span>
+</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="n">_logger</span> <span class="o">=</span> <span class="n">lc</span><span class="o">.</span><span class="n">setup_logger</span><span class="p">()</span>
+</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a>
+</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="n">SUPPORTED_CODE_TYPES</span> <span class="o">=</span> <span class="p">{</span><span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"icd10"</span><span class="p">,</span> <span class="s2">"snomed"</span><span class="p">,</span> <span class="s2">"opcs4"</span><span class="p">,</span> <span class="s2">"atc"</span><span class="p">}</span>
+</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="sd">"""List of support medical coding types"""</span>
+</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a>
+</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a>
+</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="k">class</span><span class="w"> </span><span class="nc">CodesError</span><span class="p">:</span>
+</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="w"> </span><span class="sd">"""A class used in InvalidCodesException to report an error if a code parser check fails"""</span>
+</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a>
+</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">codes</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">codes_file</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">code_type</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a> <span class="c1"># initialise class variables with provided parameters</span>
+</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="nb">locals</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
+</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">!=</span> <span class="s2">"self"</span><span class="p">:</span>
+</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a> <span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
+</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
+</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a>
+</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="k">class</span><span class="w"> </span><span class="nc">InvalidCodesException</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span>
+</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="w"> </span><span class="sd">"""Custom exception class raised when invalid codes are found that cannot be resolved by processing"""</span>
+</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a>
+</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">error</span><span class="p">):</span>
+</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">error</span><span class="o">.</span><span class="n">message</span><span class="p">)</span>
+</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">error</span> <span class="o">=</span> <span class="n">error</span>
+</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
+</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a>
+</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Proto</span><span class="p">:</span>
+</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="sd"> Define checks as list of 3 tuple: (Message, Condition, Process)</span>
+</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd"> - Message = The name of the condition (what is printed and logged)</span>
+</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="sd"> - Condition = True if Passed, and False if Failed</span>
+</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="sd"> - Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</span>
+</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd"> """</span>
+</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a>
+</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a> <span class="n">checks</span><span class="p">:</span> <span class="nb">list</span><span class="p">[</span>
+</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="nb">tuple</span><span class="p">[</span>
+</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="nb">str</span><span class="p">,</span> <span class="c1"># The description, e.g., "Not Empty"</span>
+</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">Callable</span><span class="p">[</span>
+</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">],</span>
+</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">,</span>
+</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="p">],</span> <span class="c1"># The first lambda function: takes a list and returns a pd.Series of booleans</span>
+</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a> <span class="n">Callable</span><span class="p">[</span>
+</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">Path</span><span class="p">],</span>
+</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span>
+</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="p">],</span> <span class="c1"># The second lambda function: takes a list and a string, and returns nothing</span>
+</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="p">]</span>
+</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="p">]</span>
+</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a>
+</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Path</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
+</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a> <span class="k">if</span> <span class="n">trud_codes_path</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="n">trud_codes_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Path</span> <span class="o">=</span> <span class="n">trud_codes_path</span>
+</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">trud_codes_path</span><span class="p">)</span>
+</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a> <span class="sa">f</span><span class="s2">"Error: Read2 code file '</span><span class="si">{</span><span class="n">trud_codes_path</span><span class="si">}</span><span class="s2">' does not exist. Please ensure you have installed TRUD correctly"</span>
+</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="p">)</span>
+</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
+</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="n">name</span>
+</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a>
+</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">raise_exception</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ex</span><span class="p">:</span> <span class="ne">Exception</span><span class="p">):</span>
+</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="w"> </span><span class="sd">"""Raises an exception inside a lambda function. Python does not allow using raise statement inside lambda because lambda can only contain expressions, not statements. Using raise_exception not raise_ as it's more explict"""</span>
+</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="k">raise</span> <span class="n">ex</span>
+</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
+</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">in_database</span><span class="p">(</span>
+</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">db</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">col</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">:</span>
+</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="k">return</span> <span class="n">codes</span><span class="o">.</span><span class="n">isin</span><span class="p">(</span><span class="n">db</span><span class="p">[</span><span class="n">col</span><span class="p">])</span>
+</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a>
+</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">process</span><span class="p">(</span>
+</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">Path</span>
+</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="nb">list</span><span class="p">]:</span>
+</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a><span class="w"> </span><span class="sd">"""identify issues that do not pass and fix them with define/d process"""</span>
+</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="n">errors</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="c1"># Iter through each item in check.</span>
+</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="k">for</span> <span class="n">msg</span><span class="p">,</span> <span class="n">cond</span><span class="p">,</span> <span class="n">fix</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span><span class="p">:</span>
+</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="c1"># Check if any codes fail the check to False</span>
+</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">():</span>
+</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="c1"># Log the number of codes that failed</span>
+</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="sa">f</span><span class="s2">"Check: </span><span class="si">{</span><span class="n">msg</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="p">(</span><span class="o">~</span><span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">))</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span><span class="si">}</span><span class="s2"> failed, trying to fix"</span>
+</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="p">)</span>
+</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="c1"># try fix errors by running lamba "process" function</span>
+</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">fix</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">)</span>
+</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Check: Fixed"</span><span class="p">)</span>
+</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">except</span> <span class="n">InvalidCodesException</span> <span class="k">as</span> <span class="n">ex</span><span class="p">:</span>
+</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ex</span><span class="o">.</span><span class="n">error</span><span class="p">)</span>
+</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Check: passed"</span><span class="p">)</span>
+</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a>
+</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a> <span class="k">return</span> <span class="n">codes</span><span class="p">,</span> <span class="n">errors</span>
+</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a>
+</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">verify</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a><span class="w"> </span><span class="sd">"""verify codes in codes file"""</span>
+</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a> <span class="n">conds</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([])</span>
+</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a>
+</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="c1"># Iter through each item in check.</span>
+</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="k">for</span> <span class="n">msg</span><span class="p">,</span> <span class="n">cond</span><span class="p">,</span> <span class="n">process</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span><span class="p">:</span>
+</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="c1"># run conditional check</span>
+</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span>
+</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="n">conds</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">conds</span><span class="p">,</span> <span class="n">out</span><span class="o">.</span><span class="n">all</span><span class="p">())</span>
+</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a>
+</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="k">return</span> <span class="n">conds</span>
+</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a>
+</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
+</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Read2</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a><span class="w"> </span><span class="sd">"""This Read2 class extends Proto, adding custom validation checks for a dataset of "Read2" codes. It ensures that the dataset is loaded, validates the codes based on several rules, and applies corrections or logs errors when necessary."""</span>
+</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a>
+</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"read2"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2.parquet"</span><span class="p">)</span>
+</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a>
+</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="c1"># validate checks</span>
+</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="p">(</span>
+</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="c1"># check codes are not empty, if empty throw an exception</span>
+</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="p">)</span>
+</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="p">)</span>
+</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="p">),</span>
+</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="p">),</span>
+</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="p">(</span>
+</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="c1"># check codes <5 characters, if too short pads it with . (dots) to reach 5 characters</span>
+</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">pad</span><span class="p">(</span>
+</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a> <span class="n">width</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">side</span><span class="o">=</span><span class="s2">"right"</span><span class="p">,</span> <span class="n">fillchar</span><span class="o">=</span><span class="s2">"."</span>
+</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a> <span class="p">),</span>
+</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="p">),</span>
+</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="p">(</span>
+</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="c1"># check codes > 5 characters, If too long, truncates them to 5 characters</span>
+</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="s2">"Too Long"</span><span class="p">,</span>
+</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="p">[:</span><span class="mi">5</span><span class="p">],</span>
+</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="p">),</span>
+</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="p">(</span>
+</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a> <span class="c1"># checks codes contain numbers, or dots (.), if not logs invalid code error</span>
+</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="s2">"Alphanumeric Dot"</span><span class="p">,</span>
+</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a> <span class="sa">f</span><span class="s2">"Illegal code format, not alphanumeric dot"</span><span class="p">,</span>
+</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="p">)</span>
+</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="p">)</span>
+</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="p">),</span>
+</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="p">),</span>
+</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="p">(</span>
+</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="c1"># checks code exists in self.db (the Read2 dataset). If missing log invalid codes.</span>
+</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a> <span class="sa">f</span><span class="s2">"Codes do not exist in database"</span><span class="p">,</span>
+</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">mask</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="p">)</span>
+</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="p">)</span>
+</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="p">),</span>
+</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="p">),</span>
+</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="p">]</span>
+</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a>
+</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a>
+</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Read3</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"Read3"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3.parquet"</span><span class="p">)</span>
+</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a>
+</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="p">(</span>
+</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="p">)</span>
+</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="p">)</span>
+</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="p">),</span>
+</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="p">),</span>
+</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="p">(</span>
+</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">pad</span><span class="p">(</span>
+</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="n">width</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">side</span><span class="o">=</span><span class="s2">"right"</span><span class="p">,</span> <span class="n">fillchar</span><span class="o">=</span><span class="s2">"."</span>
+</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="p">),</span>
+</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a> <span class="p">),</span>
+</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="p">(</span>
+</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="s2">"Too Long"</span><span class="p">,</span>
+</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="p">[:</span><span class="mi">5</span><span class="p">],</span>
+</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="p">),</span>
+</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="p">(</span>
+</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="s2">"Alphanumeric Dot"</span><span class="p">,</span>
+</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a> <span class="sa">f</span><span class="s2">"QA Alphanumeric Dot"</span><span class="p">,</span>
+</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="n">check_regex</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="p">)</span>
+</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="p">)</span>
+</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="p">),</span>
+</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="p">),</span>
+</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="p">(</span>
+</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a> <span class="n">check_regex</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="p">)</span>
+</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="p">)</span>
+</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a> <span class="p">),</span>
+</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a> <span class="p">),</span>
+</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a> <span class="p">]</span>
+</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a>
+</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a>
+</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Icd10</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"icd10"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"icd10.parquet"</span><span class="p">)</span>
+</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a>
+</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="p">(</span>
+</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="sa">f</span><span class="s2">"Code list is empty </span><span class="si">{</span><span class="n">codes_file</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span>
+</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a> <span class="p">)</span>
+</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="p">)</span>
+</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="p">),</span>
+</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="p">),</span>
+</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="p">(</span>
+</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">3</span><span class="p">),</span>
+</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="sa">f</span><span class="s2">"QA Too Short"</span><span class="p">,</span>
+</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">3</span><span class="p">),</span>
+</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a> <span class="p">)</span>
+</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="p">)</span>
+</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="p">),</span>
+</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="p">),</span>
+</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="p">(</span>
+</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="s2">"Has Dot"</span><span class="p">,</span>
+</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">".*\..*"</span><span class="p">)),</span> <span class="c1"># check if contains dot</span>
+</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
+</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span>
+</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="p">),</span> <span class="c1"># delete any dots in string</span>
+</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="c1"># lambda codes : codes.str.split('\.').apply(lambda ls: ls[0]) #only get part before dot</span>
+</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="p">),</span>
+</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="p">(</span>
+</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="s2">"Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="sa">f</span><span class="s2">"QA Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="p">)</span>
+</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="p">)</span>
+</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="p">),</span>
+</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="p">),</span>
+</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="p">(</span>
+</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span>
+</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
+</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a> <span class="o">&</span> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">+</span> <span class="s2">"_alt"</span><span class="p">)</span>
+</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="p">),</span>
+</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span>
+</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
+</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="o">&</span> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">+</span> <span class="s2">"_alt"</span><span class="p">)</span>
+</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a> <span class="p">),</span>
+</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a> <span class="p">)</span>
+</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a> <span class="p">)</span>
+</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a> <span class="p">),</span>
+</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="p">),</span>
+</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="c1"># (</span>
+</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="c1"># "ICD10 Regex",</span>
+</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="c1"># lambda codes : codes.str.match("[a-zA-Z][0-9][0-9]\.?[a-zA-Z0-9]*$"), #Alpha, Num, Num , Dot?, 4xAlphNum*</span>
+</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="c1"># lambda codes : lc.log_invalid_code(codes,</span>
+</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a> <span class="c1"># codes.str.match("[a-zA-Z][0-9][0-9]\.?[a-zA-Z0-9]*$"), #Log non-matching rows</span>
+</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="c1"># code_type="icd10",</span>
+</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="c1">#</span>
+</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="c1"># )</span>
+</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="p">]</span>
+</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a>
+</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a>
+</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Snomed</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"snomed"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"snomed.parquet"</span><span class="p">)</span>
+</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a>
+</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="c1"># (</span>
+</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a> <span class="c1"># "Not Empty",</span>
+</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a> <span class="c1"># lambda codes : pd.Series([len(codes) > 0]),</span>
+</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a> <span class="c1"># lambda codes : raise_exception(Exception("Code List is Empty"))</span>
+</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a> <span class="c1"># ),</span>
+</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a> <span class="p">(</span>
+</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">6</span><span class="p">),</span>
+</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="sa">f</span><span class="s2">"QA Too Short"</span><span class="p">,</span>
+</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">6</span><span class="p">),</span>
+</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="p">)</span>
+</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="p">)</span>
+</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="p">),</span>
+</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a> <span class="p">),</span>
+</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="p">(</span>
+</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a> <span class="s2">"Too Long"</span><span class="p">,</span>
+</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">18</span><span class="p">),</span>
+</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="sa">f</span><span class="s2">"QA Too Long"</span><span class="p">,</span>
+</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">18</span><span class="p">),</span>
+</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="p">)</span>
+</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="p">)</span>
+</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a> <span class="p">),</span>
+</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a> <span class="p">),</span>
+</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a> <span class="p">(</span>
+</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a> <span class="s2">"Numeric"</span><span class="p">,</span>
+</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"[0-9]+$"</span><span class="p">),</span>
+</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="sa">f</span><span class="s2">"QA Numeric"</span><span class="p">,</span>
+</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"[0-9]+$"</span><span class="p">),</span>
+</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="p">)</span>
+</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="p">)</span>
+</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a> <span class="p">),</span>
+</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a> <span class="p">),</span>
+</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a> <span class="c1"># (</span>
+</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a> <span class="c1"># "Is Integer",</span>
+</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a> <span class="c1"># lambda codes : codes.dtype == int,</span>
+</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a> <span class="c1"># lambda codes : codes.astype(int) #Convert to integer</span>
+</span><span id="L-407"><a href="#L-407"><span class="linenos">407</span></a> <span class="c1"># ),</span>
+</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a> <span class="p">(</span>
+</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="L-410"><a href="#L-410"><span class="linenos">410</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-413"><a href="#L-413"><span class="linenos">413</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a> <span class="n">mask</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a> <span class="p">)</span>
+</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a> <span class="p">)</span>
+</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a> <span class="p">),</span>
+</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a> <span class="p">),</span>
+</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a> <span class="p">]</span>
+</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a>
+</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a>
+</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Opcs4</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"opcs4"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"opcs4.parquet"</span><span class="p">)</span>
+</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a>
+</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a> <span class="p">(</span>
+</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a> <span class="p">)</span>
+</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a> <span class="p">)</span>
+</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a> <span class="p">),</span>
+</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a> <span class="p">),</span>
+</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a> <span class="p">(</span>
+</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-455"><a href="#L-455"><span class="linenos">455</span></a> <span class="n">mask</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="L-456"><a href="#L-456"><span class="linenos">456</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-457"><a href="#L-457"><span class="linenos">457</span></a> <span class="p">)</span>
+</span><span id="L-458"><a href="#L-458"><span class="linenos">458</span></a> <span class="p">)</span>
+</span><span id="L-459"><a href="#L-459"><span class="linenos">459</span></a> <span class="p">),</span>
+</span><span id="L-460"><a href="#L-460"><span class="linenos">460</span></a> <span class="p">),</span>
+</span><span id="L-461"><a href="#L-461"><span class="linenos">461</span></a> <span class="p">]</span>
+</span><span id="L-462"><a href="#L-462"><span class="linenos">462</span></a>
+</span><span id="L-463"><a href="#L-463"><span class="linenos">463</span></a>
+</span><span id="L-464"><a href="#L-464"><span class="linenos">464</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Atc</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-465"><a href="#L-465"><span class="linenos">465</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-466"><a href="#L-466"><span class="linenos">466</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"atc"</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="L-467"><a href="#L-467"><span class="linenos">467</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-468"><a href="#L-468"><span class="linenos">468</span></a> <span class="p">(</span>
+</span><span id="L-469"><a href="#L-469"><span class="linenos">469</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-470"><a href="#L-470"><span class="linenos">470</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-471"><a href="#L-471"><span class="linenos">471</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-472"><a href="#L-472"><span class="linenos">472</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-473"><a href="#L-473"><span class="linenos">473</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-474"><a href="#L-474"><span class="linenos">474</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="L-475"><a href="#L-475"><span class="linenos">475</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-476"><a href="#L-476"><span class="linenos">476</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-477"><a href="#L-477"><span class="linenos">477</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-478"><a href="#L-478"><span class="linenos">478</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-479"><a href="#L-479"><span class="linenos">479</span></a> <span class="p">)</span>
+</span><span id="L-480"><a href="#L-480"><span class="linenos">480</span></a> <span class="p">)</span>
+</span><span id="L-481"><a href="#L-481"><span class="linenos">481</span></a> <span class="p">),</span>
+</span><span id="L-482"><a href="#L-482"><span class="linenos">482</span></a> <span class="p">),</span>
+</span><span id="L-483"><a href="#L-483"><span class="linenos">483</span></a> <span class="p">(</span>
+</span><span id="L-484"><a href="#L-484"><span class="linenos">484</span></a> <span class="s2">"Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="L-485"><a href="#L-485"><span class="linenos">485</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="L-486"><a href="#L-486"><span class="linenos">486</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-487"><a href="#L-487"><span class="linenos">487</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-488"><a href="#L-488"><span class="linenos">488</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-489"><a href="#L-489"><span class="linenos">489</span></a> <span class="sa">f</span><span class="s2">"QA Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="L-490"><a href="#L-490"><span class="linenos">490</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-491"><a href="#L-491"><span class="linenos">491</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-492"><a href="#L-492"><span class="linenos">492</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="L-493"><a href="#L-493"><span class="linenos">493</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-494"><a href="#L-494"><span class="linenos">494</span></a> <span class="p">)</span>
+</span><span id="L-495"><a href="#L-495"><span class="linenos">495</span></a> <span class="p">)</span>
+</span><span id="L-496"><a href="#L-496"><span class="linenos">496</span></a> <span class="p">),</span>
+</span><span id="L-497"><a href="#L-497"><span class="linenos">497</span></a> <span class="p">),</span>
+</span><span id="L-498"><a href="#L-498"><span class="linenos">498</span></a> <span class="p">]</span>
+</span><span id="L-499"><a href="#L-499"><span class="linenos">499</span></a>
+</span><span id="L-500"><a href="#L-500"><span class="linenos">500</span></a>
+</span><span id="L-501"><a href="#L-501"><span class="linenos">501</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Med</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-502"><a href="#L-502"><span class="linenos">502</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-503"><a href="#L-503"><span class="linenos">503</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"med"</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="L-504"><a href="#L-504"><span class="linenos">504</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-505"><a href="#L-505"><span class="linenos">505</span></a> <span class="p">(</span>
+</span><span id="L-506"><a href="#L-506"><span class="linenos">506</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-507"><a href="#L-507"><span class="linenos">507</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-508"><a href="#L-508"><span class="linenos">508</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-509"><a href="#L-509"><span class="linenos">509</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-510"><a href="#L-510"><span class="linenos">510</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-511"><a href="#L-511"><span class="linenos">511</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="L-512"><a href="#L-512"><span class="linenos">512</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-513"><a href="#L-513"><span class="linenos">513</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-514"><a href="#L-514"><span class="linenos">514</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-515"><a href="#L-515"><span class="linenos">515</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-516"><a href="#L-516"><span class="linenos">516</span></a> <span class="p">)</span>
+</span><span id="L-517"><a href="#L-517"><span class="linenos">517</span></a> <span class="p">)</span>
+</span><span id="L-518"><a href="#L-518"><span class="linenos">518</span></a> <span class="p">),</span>
+</span><span id="L-519"><a href="#L-519"><span class="linenos">519</span></a> <span class="p">)</span>
+</span><span id="L-520"><a href="#L-520"><span class="linenos">520</span></a> <span class="p">]</span>
+</span><span id="L-521"><a href="#L-521"><span class="linenos">521</span></a>
+</span><span id="L-522"><a href="#L-522"><span class="linenos">522</span></a>
+</span><span id="L-523"><a href="#L-523"><span class="linenos">523</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Cprd</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="L-524"><a href="#L-524"><span class="linenos">524</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="L-525"><a href="#L-525"><span class="linenos">525</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"cprd"</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="L-526"><a href="#L-526"><span class="linenos">526</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-527"><a href="#L-527"><span class="linenos">527</span></a> <span class="p">(</span>
+</span><span id="L-528"><a href="#L-528"><span class="linenos">528</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="L-529"><a href="#L-529"><span class="linenos">529</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="L-530"><a href="#L-530"><span class="linenos">530</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="L-531"><a href="#L-531"><span class="linenos">531</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="L-532"><a href="#L-532"><span class="linenos">532</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="L-533"><a href="#L-533"><span class="linenos">533</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="L-534"><a href="#L-534"><span class="linenos">534</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="L-535"><a href="#L-535"><span class="linenos">535</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-536"><a href="#L-536"><span class="linenos">536</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="L-537"><a href="#L-537"><span class="linenos">537</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="L-538"><a href="#L-538"><span class="linenos">538</span></a> <span class="p">)</span>
+</span><span id="L-539"><a href="#L-539"><span class="linenos">539</span></a> <span class="p">)</span>
+</span><span id="L-540"><a href="#L-540"><span class="linenos">540</span></a> <span class="p">),</span>
+</span><span id="L-541"><a href="#L-541"><span class="linenos">541</span></a> <span class="p">)</span>
+</span><span id="L-542"><a href="#L-542"><span class="linenos">542</span></a> <span class="p">]</span>
+</span><span id="L-543"><a href="#L-543"><span class="linenos">543</span></a>
+</span><span id="L-544"><a href="#L-544"><span class="linenos">544</span></a>
+</span><span id="L-545"><a href="#L-545"><span class="linenos">545</span></a><span class="k">class</span><span class="w"> </span><span class="nc">CodeTypeParser</span><span class="p">:</span>
+</span><span id="L-546"><a href="#L-546"><span class="linenos">546</span></a><span class="w"> </span><span class="sd">"""A class used in InvalidCodesException to report an error if a code parser check fails"""</span>
+</span><span id="L-547"><a href="#L-547"><span class="linenos">547</span></a>
+</span><span id="L-548"><a href="#L-548"><span class="linenos">548</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">trud_processed_dir</span><span class="p">:</span> <span class="n">Path</span> <span class="o">=</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span><span class="p">):</span>
+</span><span id="L-549"><a href="#L-549"><span class="linenos">549</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">trud_processed_dir</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">trud_processed_dir</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="L-550"><a href="#L-550"><span class="linenos">550</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="L-551"><a href="#L-551"><span class="linenos">551</span></a> <span class="sa">f</span><span class="s2">"Cannot initialise parsers as the TRUD processed directory </span><span class="si">{</span><span class="n">trud_processed_dir</span><span class="si">}</span><span class="s2"> does not exist, please check that TRUD has been installed: acmc trud install"</span>
+</span><span id="L-552"><a href="#L-552"><span class="linenos">552</span></a> <span class="p">)</span>
+</span><span id="L-553"><a href="#L-553"><span class="linenos">553</span></a>
+</span><span id="L-554"><a href="#L-554"><span class="linenos">554</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">code_types</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-555"><a href="#L-555"><span class="linenos">555</span></a> <span class="s2">"read2"</span><span class="p">:</span> <span class="n">Read2</span><span class="p">(),</span>
+</span><span id="L-556"><a href="#L-556"><span class="linenos">556</span></a> <span class="s2">"read3"</span><span class="p">:</span> <span class="n">Read3</span><span class="p">(),</span>
+</span><span id="L-557"><a href="#L-557"><span class="linenos">557</span></a> <span class="s2">"icd10"</span><span class="p">:</span> <span class="n">Icd10</span><span class="p">(),</span>
+</span><span id="L-558"><a href="#L-558"><span class="linenos">558</span></a> <span class="s2">"snomed"</span><span class="p">:</span> <span class="n">Snomed</span><span class="p">(),</span>
+</span><span id="L-559"><a href="#L-559"><span class="linenos">559</span></a> <span class="s2">"opcs4"</span><span class="p">:</span> <span class="n">Opcs4</span><span class="p">(),</span>
+</span><span id="L-560"><a href="#L-560"><span class="linenos">560</span></a> <span class="s2">"atc"</span><span class="p">:</span> <span class="n">Atc</span><span class="p">(),</span>
+</span><span id="L-561"><a href="#L-561"><span class="linenos">561</span></a> <span class="s2">"med"</span><span class="p">:</span> <span class="n">Med</span><span class="p">(),</span>
+</span><span id="L-562"><a href="#L-562"><span class="linenos">562</span></a> <span class="s2">"cprd"</span><span class="p">:</span> <span class="n">Cprd</span><span class="p">(),</span>
+</span><span id="L-563"><a href="#L-563"><span class="linenos">563</span></a> <span class="p">}</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="SUPPORTED_CODE_TYPES">
+ <div class="attr variable">
+ <span class="name">SUPPORTED_CODE_TYPES</span> =
+<span class="default_value">{'snomed', 'atc', 'read2', 'read3', 'icd10', 'opcs4'}</span>
+
+
+ </div>
+ <a class="headerlink" href="#SUPPORTED_CODE_TYPES"></a>
+
+ <div class="docstring"><p>List of support medical coding types</p>
+</div>
+
+
+ </section>
+ <section id="CodesError">
+ <input id="CodesError-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">CodesError</span>:
+
+ <label class="view-source-button" for="CodesError-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#CodesError"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="CodesError-23"><a href="#CodesError-23"><span class="linenos">23</span></a><span class="k">class</span><span class="w"> </span><span class="nc">CodesError</span><span class="p">:</span>
+</span><span id="CodesError-24"><a href="#CodesError-24"><span class="linenos">24</span></a><span class="w"> </span><span class="sd">"""A class used in InvalidCodesException to report an error if a code parser check fails"""</span>
+</span><span id="CodesError-25"><a href="#CodesError-25"><span class="linenos">25</span></a>
+</span><span id="CodesError-26"><a href="#CodesError-26"><span class="linenos">26</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">codes</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">codes_file</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">code_type</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+</span><span id="CodesError-27"><a href="#CodesError-27"><span class="linenos">27</span></a> <span class="c1"># initialise class variables with provided parameters</span>
+</span><span id="CodesError-28"><a href="#CodesError-28"><span class="linenos">28</span></a> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="nb">locals</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
+</span><span id="CodesError-29"><a href="#CodesError-29"><span class="linenos">29</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">!=</span> <span class="s2">"self"</span><span class="p">:</span>
+</span><span id="CodesError-30"><a href="#CodesError-30"><span class="linenos">30</span></a> <span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>A class used in InvalidCodesException to report an error if a code parser check fails</p>
+</div>
+
+
+ <div id="CodesError.__init__" class="classattr">
+ <input id="CodesError.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="name">CodesError</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">message</span>, </span><span class="param"><span class="n">codes</span><span class="o">=</span><span class="kc">None</span>, </span><span class="param"><span class="n">codes_file</span><span class="o">=</span><span class="kc">None</span>, </span><span class="param"><span class="n">mask</span><span class="o">=</span><span class="kc">None</span>, </span><span class="param"><span class="n">code_type</span><span class="o">=</span><span class="kc">None</span></span>)</span>
+
+ <label class="view-source-button" for="CodesError.__init__-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#CodesError.__init__"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="CodesError.__init__-26"><a href="#CodesError.__init__-26"><span class="linenos">26</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">codes</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">codes_file</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">code_type</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+</span><span id="CodesError.__init__-27"><a href="#CodesError.__init__-27"><span class="linenos">27</span></a> <span class="c1"># initialise class variables with provided parameters</span>
+</span><span id="CodesError.__init__-28"><a href="#CodesError.__init__-28"><span class="linenos">28</span></a> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span> <span class="ow">in</span> <span class="nb">locals</span><span class="p">()</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
+</span><span id="CodesError.__init__-29"><a href="#CodesError.__init__-29"><span class="linenos">29</span></a> <span class="k">if</span> <span class="n">key</span> <span class="o">!=</span> <span class="s2">"self"</span><span class="p">:</span>
+</span><span id="CodesError.__init__-30"><a href="#CodesError.__init__-30"><span class="linenos">30</span></a> <span class="nb">setattr</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ </section>
+ <section id="InvalidCodesException">
+ <input id="InvalidCodesException-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">InvalidCodesException</span><wbr>(<span class="base">builtins.Exception</span>):
+
+ <label class="view-source-button" for="InvalidCodesException-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#InvalidCodesException"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="InvalidCodesException-33"><a href="#InvalidCodesException-33"><span class="linenos">33</span></a><span class="k">class</span><span class="w"> </span><span class="nc">InvalidCodesException</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span>
+</span><span id="InvalidCodesException-34"><a href="#InvalidCodesException-34"><span class="linenos">34</span></a><span class="w"> </span><span class="sd">"""Custom exception class raised when invalid codes are found that cannot be resolved by processing"""</span>
+</span><span id="InvalidCodesException-35"><a href="#InvalidCodesException-35"><span class="linenos">35</span></a>
+</span><span id="InvalidCodesException-36"><a href="#InvalidCodesException-36"><span class="linenos">36</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">error</span><span class="p">):</span>
+</span><span id="InvalidCodesException-37"><a href="#InvalidCodesException-37"><span class="linenos">37</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">error</span><span class="o">.</span><span class="n">message</span><span class="p">)</span>
+</span><span id="InvalidCodesException-38"><a href="#InvalidCodesException-38"><span class="linenos">38</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">error</span> <span class="o">=</span> <span class="n">error</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Custom exception class raised when invalid codes are found that cannot be resolved by processing</p>
+</div>
+
+
+ <div id="InvalidCodesException.__init__" class="classattr">
+ <input id="InvalidCodesException.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="name">InvalidCodesException</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">error</span></span>)</span>
+
+ <label class="view-source-button" for="InvalidCodesException.__init__-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#InvalidCodesException.__init__"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="InvalidCodesException.__init__-36"><a href="#InvalidCodesException.__init__-36"><span class="linenos">36</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">error</span><span class="p">):</span>
+</span><span id="InvalidCodesException.__init__-37"><a href="#InvalidCodesException.__init__-37"><span class="linenos">37</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">error</span><span class="o">.</span><span class="n">message</span><span class="p">)</span>
+</span><span id="InvalidCodesException.__init__-38"><a href="#InvalidCodesException.__init__-38"><span class="linenos">38</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">error</span> <span class="o">=</span> <span class="n">error</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ <div id="InvalidCodesException.error" class="classattr">
+ <div class="attr variable">
+ <span class="name">error</span>
+
+
+ </div>
+ <a class="headerlink" href="#InvalidCodesException.error"></a>
+
+
+
+ </div>
+ </section>
+ <section id="Proto">
+ <input id="Proto-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Proto</span>:
+
+ <label class="view-source-button" for="Proto-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Proto"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Proto-41"><a href="#Proto-41"><span class="linenos"> 41</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Proto</span><span class="p">:</span>
+</span><span id="Proto-42"><a href="#Proto-42"><span class="linenos"> 42</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="Proto-43"><a href="#Proto-43"><span class="linenos"> 43</span></a><span class="sd"> Define checks as list of 3 tuple: (Message, Condition, Process)</span>
+</span><span id="Proto-44"><a href="#Proto-44"><span class="linenos"> 44</span></a><span class="sd"> - Message = The name of the condition (what is printed and logged)</span>
+</span><span id="Proto-45"><a href="#Proto-45"><span class="linenos"> 45</span></a><span class="sd"> - Condition = True if Passed, and False if Failed</span>
+</span><span id="Proto-46"><a href="#Proto-46"><span class="linenos"> 46</span></a><span class="sd"> - Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</span>
+</span><span id="Proto-47"><a href="#Proto-47"><span class="linenos"> 47</span></a><span class="sd"> """</span>
+</span><span id="Proto-48"><a href="#Proto-48"><span class="linenos"> 48</span></a>
+</span><span id="Proto-49"><a href="#Proto-49"><span class="linenos"> 49</span></a> <span class="n">checks</span><span class="p">:</span> <span class="nb">list</span><span class="p">[</span>
+</span><span id="Proto-50"><a href="#Proto-50"><span class="linenos"> 50</span></a> <span class="nb">tuple</span><span class="p">[</span>
+</span><span id="Proto-51"><a href="#Proto-51"><span class="linenos"> 51</span></a> <span class="nb">str</span><span class="p">,</span> <span class="c1"># The description, e.g., "Not Empty"</span>
+</span><span id="Proto-52"><a href="#Proto-52"><span class="linenos"> 52</span></a> <span class="n">Callable</span><span class="p">[</span>
+</span><span id="Proto-53"><a href="#Proto-53"><span class="linenos"> 53</span></a> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">],</span>
+</span><span id="Proto-54"><a href="#Proto-54"><span class="linenos"> 54</span></a> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">,</span>
+</span><span id="Proto-55"><a href="#Proto-55"><span class="linenos"> 55</span></a> <span class="p">],</span> <span class="c1"># The first lambda function: takes a list and returns a pd.Series of booleans</span>
+</span><span id="Proto-56"><a href="#Proto-56"><span class="linenos"> 56</span></a> <span class="n">Callable</span><span class="p">[</span>
+</span><span id="Proto-57"><a href="#Proto-57"><span class="linenos"> 57</span></a> <span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">Path</span><span class="p">],</span>
+</span><span id="Proto-58"><a href="#Proto-58"><span class="linenos"> 58</span></a> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span>
+</span><span id="Proto-59"><a href="#Proto-59"><span class="linenos"> 59</span></a> <span class="p">],</span> <span class="c1"># The second lambda function: takes a list and a string, and returns nothing</span>
+</span><span id="Proto-60"><a href="#Proto-60"><span class="linenos"> 60</span></a> <span class="p">]</span>
+</span><span id="Proto-61"><a href="#Proto-61"><span class="linenos"> 61</span></a> <span class="p">]</span>
+</span><span id="Proto-62"><a href="#Proto-62"><span class="linenos"> 62</span></a>
+</span><span id="Proto-63"><a href="#Proto-63"><span class="linenos"> 63</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Path</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
+</span><span id="Proto-64"><a href="#Proto-64"><span class="linenos"> 64</span></a> <span class="k">if</span> <span class="n">trud_codes_path</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="Proto-65"><a href="#Proto-65"><span class="linenos"> 65</span></a> <span class="k">if</span> <span class="n">trud_codes_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="Proto-66"><a href="#Proto-66"><span class="linenos"> 66</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Path</span> <span class="o">=</span> <span class="n">trud_codes_path</span>
+</span><span id="Proto-67"><a href="#Proto-67"><span class="linenos"> 67</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">trud_codes_path</span><span class="p">)</span>
+</span><span id="Proto-68"><a href="#Proto-68"><span class="linenos"> 68</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="Proto-69"><a href="#Proto-69"><span class="linenos"> 69</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="Proto-70"><a href="#Proto-70"><span class="linenos"> 70</span></a> <span class="sa">f</span><span class="s2">"Error: Read2 code file '</span><span class="si">{</span><span class="n">trud_codes_path</span><span class="si">}</span><span class="s2">' does not exist. Please ensure you have installed TRUD correctly"</span>
+</span><span id="Proto-71"><a href="#Proto-71"><span class="linenos"> 71</span></a> <span class="p">)</span>
+</span><span id="Proto-72"><a href="#Proto-72"><span class="linenos"> 72</span></a>
+</span><span id="Proto-73"><a href="#Proto-73"><span class="linenos"> 73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="n">name</span>
+</span><span id="Proto-74"><a href="#Proto-74"><span class="linenos"> 74</span></a>
+</span><span id="Proto-75"><a href="#Proto-75"><span class="linenos"> 75</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">raise_exception</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ex</span><span class="p">:</span> <span class="ne">Exception</span><span class="p">):</span>
+</span><span id="Proto-76"><a href="#Proto-76"><span class="linenos"> 76</span></a><span class="w"> </span><span class="sd">"""Raises an exception inside a lambda function. Python does not allow using raise statement inside lambda because lambda can only contain expressions, not statements. Using raise_exception not raise_ as it's more explict"""</span>
+</span><span id="Proto-77"><a href="#Proto-77"><span class="linenos"> 77</span></a> <span class="k">raise</span> <span class="n">ex</span>
+</span><span id="Proto-78"><a href="#Proto-78"><span class="linenos"> 78</span></a>
+</span><span id="Proto-79"><a href="#Proto-79"><span class="linenos"> 79</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">in_database</span><span class="p">(</span>
+</span><span id="Proto-80"><a href="#Proto-80"><span class="linenos"> 80</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">db</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">col</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="Proto-81"><a href="#Proto-81"><span class="linenos"> 81</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">:</span>
+</span><span id="Proto-82"><a href="#Proto-82"><span class="linenos"> 82</span></a> <span class="k">return</span> <span class="n">codes</span><span class="o">.</span><span class="n">isin</span><span class="p">(</span><span class="n">db</span><span class="p">[</span><span class="n">col</span><span class="p">])</span>
+</span><span id="Proto-83"><a href="#Proto-83"><span class="linenos"> 83</span></a>
+</span><span id="Proto-84"><a href="#Proto-84"><span class="linenos"> 84</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">process</span><span class="p">(</span>
+</span><span id="Proto-85"><a href="#Proto-85"><span class="linenos"> 85</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">Path</span>
+</span><span id="Proto-86"><a href="#Proto-86"><span class="linenos"> 86</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="nb">list</span><span class="p">]:</span>
+</span><span id="Proto-87"><a href="#Proto-87"><span class="linenos"> 87</span></a><span class="w"> </span><span class="sd">"""identify issues that do not pass and fix them with define/d process"""</span>
+</span><span id="Proto-88"><a href="#Proto-88"><span class="linenos"> 88</span></a> <span class="n">errors</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="Proto-89"><a href="#Proto-89"><span class="linenos"> 89</span></a> <span class="c1"># Iter through each item in check.</span>
+</span><span id="Proto-90"><a href="#Proto-90"><span class="linenos"> 90</span></a> <span class="k">for</span> <span class="n">msg</span><span class="p">,</span> <span class="n">cond</span><span class="p">,</span> <span class="n">fix</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span><span class="p">:</span>
+</span><span id="Proto-91"><a href="#Proto-91"><span class="linenos"> 91</span></a> <span class="c1"># Check if any codes fail the check to False</span>
+</span><span id="Proto-92"><a href="#Proto-92"><span class="linenos"> 92</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">():</span>
+</span><span id="Proto-93"><a href="#Proto-93"><span class="linenos"> 93</span></a> <span class="c1"># Log the number of codes that failed</span>
+</span><span id="Proto-94"><a href="#Proto-94"><span class="linenos"> 94</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="Proto-95"><a href="#Proto-95"><span class="linenos"> 95</span></a> <span class="sa">f</span><span class="s2">"Check: </span><span class="si">{</span><span class="n">msg</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="p">(</span><span class="o">~</span><span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">))</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span><span class="si">}</span><span class="s2"> failed, trying to fix"</span>
+</span><span id="Proto-96"><a href="#Proto-96"><span class="linenos"> 96</span></a> <span class="p">)</span>
+</span><span id="Proto-97"><a href="#Proto-97"><span class="linenos"> 97</span></a> <span class="c1"># try fix errors by running lamba "process" function</span>
+</span><span id="Proto-98"><a href="#Proto-98"><span class="linenos"> 98</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="Proto-99"><a href="#Proto-99"><span class="linenos"> 99</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">fix</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">)</span>
+</span><span id="Proto-100"><a href="#Proto-100"><span class="linenos">100</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Check: Fixed"</span><span class="p">)</span>
+</span><span id="Proto-101"><a href="#Proto-101"><span class="linenos">101</span></a> <span class="k">except</span> <span class="n">InvalidCodesException</span> <span class="k">as</span> <span class="n">ex</span><span class="p">:</span>
+</span><span id="Proto-102"><a href="#Proto-102"><span class="linenos">102</span></a> <span class="n">errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ex</span><span class="o">.</span><span class="n">error</span><span class="p">)</span>
+</span><span id="Proto-103"><a href="#Proto-103"><span class="linenos">103</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="Proto-104"><a href="#Proto-104"><span class="linenos">104</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Check: passed"</span><span class="p">)</span>
+</span><span id="Proto-105"><a href="#Proto-105"><span class="linenos">105</span></a>
+</span><span id="Proto-106"><a href="#Proto-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">codes</span><span class="p">,</span> <span class="n">errors</span>
+</span><span id="Proto-107"><a href="#Proto-107"><span class="linenos">107</span></a>
+</span><span id="Proto-108"><a href="#Proto-108"><span class="linenos">108</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">verify</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="Proto-109"><a href="#Proto-109"><span class="linenos">109</span></a><span class="w"> </span><span class="sd">"""verify codes in codes file"""</span>
+</span><span id="Proto-110"><a href="#Proto-110"><span class="linenos">110</span></a> <span class="n">conds</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([])</span>
+</span><span id="Proto-111"><a href="#Proto-111"><span class="linenos">111</span></a>
+</span><span id="Proto-112"><a href="#Proto-112"><span class="linenos">112</span></a> <span class="c1"># Iter through each item in check.</span>
+</span><span id="Proto-113"><a href="#Proto-113"><span class="linenos">113</span></a> <span class="k">for</span> <span class="n">msg</span><span class="p">,</span> <span class="n">cond</span><span class="p">,</span> <span class="n">process</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span><span class="p">:</span>
+</span><span id="Proto-114"><a href="#Proto-114"><span class="linenos">114</span></a> <span class="c1"># run conditional check</span>
+</span><span id="Proto-115"><a href="#Proto-115"><span class="linenos">115</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span>
+</span><span id="Proto-116"><a href="#Proto-116"><span class="linenos">116</span></a> <span class="n">conds</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">conds</span><span class="p">,</span> <span class="n">out</span><span class="o">.</span><span class="n">all</span><span class="p">())</span>
+</span><span id="Proto-117"><a href="#Proto-117"><span class="linenos">117</span></a>
+</span><span id="Proto-118"><a href="#Proto-118"><span class="linenos">118</span></a> <span class="k">return</span> <span class="n">conds</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Proto.__init__" class="classattr">
+ <input id="Proto.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="name">Proto</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">name</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span></span>)</span>
+
+ <label class="view-source-button" for="Proto.__init__-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Proto.__init__"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Proto.__init__-63"><a href="#Proto.__init__-63"><span class="linenos">63</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="n">Path</span><span class="p">]</span> <span class="o">=</span> <span class="kc">None</span><span class="p">):</span>
+</span><span id="Proto.__init__-64"><a href="#Proto.__init__-64"><span class="linenos">64</span></a> <span class="k">if</span> <span class="n">trud_codes_path</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="Proto.__init__-65"><a href="#Proto.__init__-65"><span class="linenos">65</span></a> <span class="k">if</span> <span class="n">trud_codes_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="Proto.__init__-66"><a href="#Proto.__init__-66"><span class="linenos">66</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">trud_codes_path</span><span class="p">:</span> <span class="n">Path</span> <span class="o">=</span> <span class="n">trud_codes_path</span>
+</span><span id="Proto.__init__-67"><a href="#Proto.__init__-67"><span class="linenos">67</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">trud_codes_path</span><span class="p">)</span>
+</span><span id="Proto.__init__-68"><a href="#Proto.__init__-68"><span class="linenos">68</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="Proto.__init__-69"><a href="#Proto.__init__-69"><span class="linenos">69</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="Proto.__init__-70"><a href="#Proto.__init__-70"><span class="linenos">70</span></a> <span class="sa">f</span><span class="s2">"Error: Read2 code file '</span><span class="si">{</span><span class="n">trud_codes_path</span><span class="si">}</span><span class="s2">' does not exist. Please ensure you have installed TRUD correctly"</span>
+</span><span id="Proto.__init__-71"><a href="#Proto.__init__-71"><span class="linenos">71</span></a> <span class="p">)</span>
+</span><span id="Proto.__init__-72"><a href="#Proto.__init__-72"><span class="linenos">72</span></a>
+</span><span id="Proto.__init__-73"><a href="#Proto.__init__-73"><span class="linenos">73</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="n">name</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ <div id="Proto.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span><span class="annotation">: list[tuple[str, typing.Callable[[pandas.core.frame.DataFrame], pandas.core.series.Series], typing.Callable[[pandas.core.frame.DataFrame, pathlib.Path], pandas.core.frame.DataFrame]]]</span>
+
+
+ </div>
+ <a class="headerlink" href="#Proto.checks"></a>
+
+
+
+ </div>
+ <div id="Proto.name" class="classattr">
+ <div class="attr variable">
+ <span class="name">name</span><span class="annotation">: str</span>
+
+
+ </div>
+ <a class="headerlink" href="#Proto.name"></a>
+
+
+
+ </div>
+ <div id="Proto.raise_exception" class="classattr">
+ <input id="Proto.raise_exception-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">raise_exception</span><span class="signature pdoc-code condensed">(<span class="param"><span class="bp">self</span>, </span><span class="param"><span class="n">ex</span><span class="p">:</span> <span class="ne">Exception</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="Proto.raise_exception-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Proto.raise_exception"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Proto.raise_exception-75"><a href="#Proto.raise_exception-75"><span class="linenos">75</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">raise_exception</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ex</span><span class="p">:</span> <span class="ne">Exception</span><span class="p">):</span>
+</span><span id="Proto.raise_exception-76"><a href="#Proto.raise_exception-76"><span class="linenos">76</span></a><span class="w"> </span><span class="sd">"""Raises an exception inside a lambda function. Python does not allow using raise statement inside lambda because lambda can only contain expressions, not statements. Using raise_exception not raise_ as it's more explict"""</span>
+</span><span id="Proto.raise_exception-77"><a href="#Proto.raise_exception-77"><span class="linenos">77</span></a> <span class="k">raise</span> <span class="n">ex</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Raises an exception inside a lambda function. Python does not allow using raise statement inside lambda because lambda can only contain expressions, not statements. Using raise_exception not raise_ as it's more explict</p>
+</div>
+
+
+ </div>
+ <div id="Proto.in_database" class="classattr">
+ <input id="Proto.in_database-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">in_database</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">codes</span><span class="p">:</span> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>,</span><span class="param"> <span class="n">db</span><span class="p">:</span> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>,</span><span class="param"> <span class="n">col</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">) -> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>:</span></span>
+
+ <label class="view-source-button" for="Proto.in_database-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Proto.in_database"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Proto.in_database-79"><a href="#Proto.in_database-79"><span class="linenos">79</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">in_database</span><span class="p">(</span>
+</span><span id="Proto.in_database-80"><a href="#Proto.in_database-80"><span class="linenos">80</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">db</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">col</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="Proto.in_database-81"><a href="#Proto.in_database-81"><span class="linenos">81</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">:</span>
+</span><span id="Proto.in_database-82"><a href="#Proto.in_database-82"><span class="linenos">82</span></a> <span class="k">return</span> <span class="n">codes</span><span class="o">.</span><span class="n">isin</span><span class="p">(</span><span class="n">db</span><span class="p">[</span><span class="n">col</span><span class="p">])</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ <div id="Proto.process" class="classattr">
+ <input id="Proto.process-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">process</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="bp">self</span>,</span><span class="param"> <span class="n">codes</span><span class="p">:</span> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>,</span><span class="param"> <span class="n">codes_file</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">) -> <span class="n">Tuple</span><span class="p">[</span><span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="nb">list</span><span class="p">]</span>:</span></span>
+
+ <label class="view-source-button" for="Proto.process-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Proto.process"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Proto.process-84"><a href="#Proto.process-84"><span class="linenos"> 84</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">process</span><span class="p">(</span>
+</span><span id="Proto.process-85"><a href="#Proto.process-85"><span class="linenos"> 85</span></a> <span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">Path</span>
+</span><span id="Proto.process-86"><a href="#Proto.process-86"><span class="linenos"> 86</span></a> <span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="nb">list</span><span class="p">]:</span>
+</span><span id="Proto.process-87"><a href="#Proto.process-87"><span class="linenos"> 87</span></a><span class="w"> </span><span class="sd">"""identify issues that do not pass and fix them with define/d process"""</span>
+</span><span id="Proto.process-88"><a href="#Proto.process-88"><span class="linenos"> 88</span></a> <span class="n">errors</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="Proto.process-89"><a href="#Proto.process-89"><span class="linenos"> 89</span></a> <span class="c1"># Iter through each item in check.</span>
+</span><span id="Proto.process-90"><a href="#Proto.process-90"><span class="linenos"> 90</span></a> <span class="k">for</span> <span class="n">msg</span><span class="p">,</span> <span class="n">cond</span><span class="p">,</span> <span class="n">fix</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span><span class="p">:</span>
+</span><span id="Proto.process-91"><a href="#Proto.process-91"><span class="linenos"> 91</span></a> <span class="c1"># Check if any codes fail the check to False</span>
+</span><span id="Proto.process-92"><a href="#Proto.process-92"><span class="linenos"> 92</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">():</span>
+</span><span id="Proto.process-93"><a href="#Proto.process-93"><span class="linenos"> 93</span></a> <span class="c1"># Log the number of codes that failed</span>
+</span><span id="Proto.process-94"><a href="#Proto.process-94"><span class="linenos"> 94</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="Proto.process-95"><a href="#Proto.process-95"><span class="linenos"> 95</span></a> <span class="sa">f</span><span class="s2">"Check: </span><span class="si">{</span><span class="n">msg</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="p">(</span><span class="o">~</span><span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">))</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span><span class="si">}</span><span class="s2"> failed, trying to fix"</span>
+</span><span id="Proto.process-96"><a href="#Proto.process-96"><span class="linenos"> 96</span></a> <span class="p">)</span>
+</span><span id="Proto.process-97"><a href="#Proto.process-97"><span class="linenos"> 97</span></a> <span class="c1"># try fix errors by running lamba "process" function</span>
+</span><span id="Proto.process-98"><a href="#Proto.process-98"><span class="linenos"> 98</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="Proto.process-99"><a href="#Proto.process-99"><span class="linenos"> 99</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">fix</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">)</span>
+</span><span id="Proto.process-100"><a href="#Proto.process-100"><span class="linenos">100</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Check: Fixed"</span><span class="p">)</span>
+</span><span id="Proto.process-101"><a href="#Proto.process-101"><span class="linenos">101</span></a> <span class="k">except</span> <span class="n">InvalidCodesException</span> <span class="k">as</span> <span class="n">ex</span><span class="p">:</span>
+</span><span id="Proto.process-102"><a href="#Proto.process-102"><span class="linenos">102</span></a> <span class="n">errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ex</span><span class="o">.</span><span class="n">error</span><span class="p">)</span>
+</span><span id="Proto.process-103"><a href="#Proto.process-103"><span class="linenos">103</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="Proto.process-104"><a href="#Proto.process-104"><span class="linenos">104</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Check: passed"</span><span class="p">)</span>
+</span><span id="Proto.process-105"><a href="#Proto.process-105"><span class="linenos">105</span></a>
+</span><span id="Proto.process-106"><a href="#Proto.process-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">codes</span><span class="p">,</span> <span class="n">errors</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>identify issues that do not pass and fix them with define/d process</p>
+</div>
+
+
+ </div>
+ <div id="Proto.verify" class="classattr">
+ <input id="Proto.verify-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">verify</span><span class="signature pdoc-code condensed">(<span class="param"><span class="bp">self</span>, </span><span class="param"><span class="n">codes</span><span class="p">:</span> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>, </span><span class="param"><span class="n">codes_file</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="Proto.verify-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Proto.verify"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Proto.verify-108"><a href="#Proto.verify-108"><span class="linenos">108</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">verify</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="Proto.verify-109"><a href="#Proto.verify-109"><span class="linenos">109</span></a><span class="w"> </span><span class="sd">"""verify codes in codes file"""</span>
+</span><span id="Proto.verify-110"><a href="#Proto.verify-110"><span class="linenos">110</span></a> <span class="n">conds</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([])</span>
+</span><span id="Proto.verify-111"><a href="#Proto.verify-111"><span class="linenos">111</span></a>
+</span><span id="Proto.verify-112"><a href="#Proto.verify-112"><span class="linenos">112</span></a> <span class="c1"># Iter through each item in check.</span>
+</span><span id="Proto.verify-113"><a href="#Proto.verify-113"><span class="linenos">113</span></a> <span class="k">for</span> <span class="n">msg</span><span class="p">,</span> <span class="n">cond</span><span class="p">,</span> <span class="n">process</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span><span class="p">:</span>
+</span><span id="Proto.verify-114"><a href="#Proto.verify-114"><span class="linenos">114</span></a> <span class="c1"># run conditional check</span>
+</span><span id="Proto.verify-115"><a href="#Proto.verify-115"><span class="linenos">115</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">cond</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span>
+</span><span id="Proto.verify-116"><a href="#Proto.verify-116"><span class="linenos">116</span></a> <span class="n">conds</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">conds</span><span class="p">,</span> <span class="n">out</span><span class="o">.</span><span class="n">all</span><span class="p">())</span>
+</span><span id="Proto.verify-117"><a href="#Proto.verify-117"><span class="linenos">117</span></a>
+</span><span id="Proto.verify-118"><a href="#Proto.verify-118"><span class="linenos">118</span></a> <span class="k">return</span> <span class="n">conds</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>verify codes in codes file</p>
+</div>
+
+
+ </div>
+ </section>
+ <section id="Read2">
+ <input id="Read2-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Read2</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Read2-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Read2"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Read2-121"><a href="#Read2-121"><span class="linenos">121</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Read2</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Read2-122"><a href="#Read2-122"><span class="linenos">122</span></a><span class="w"> </span><span class="sd">"""This Read2 class extends Proto, adding custom validation checks for a dataset of "Read2" codes. It ensures that the dataset is loaded, validates the codes based on several rules, and applies corrections or logs errors when necessary."""</span>
+</span><span id="Read2-123"><a href="#Read2-123"><span class="linenos">123</span></a>
+</span><span id="Read2-124"><a href="#Read2-124"><span class="linenos">124</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Read2-125"><a href="#Read2-125"><span class="linenos">125</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"read2"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2.parquet"</span><span class="p">)</span>
+</span><span id="Read2-126"><a href="#Read2-126"><span class="linenos">126</span></a>
+</span><span id="Read2-127"><a href="#Read2-127"><span class="linenos">127</span></a> <span class="c1"># validate checks</span>
+</span><span id="Read2-128"><a href="#Read2-128"><span class="linenos">128</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Read2-129"><a href="#Read2-129"><span class="linenos">129</span></a> <span class="p">(</span>
+</span><span id="Read2-130"><a href="#Read2-130"><span class="linenos">130</span></a> <span class="c1"># check codes are not empty, if empty throw an exception</span>
+</span><span id="Read2-131"><a href="#Read2-131"><span class="linenos">131</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Read2-132"><a href="#Read2-132"><span class="linenos">132</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Read2-133"><a href="#Read2-133"><span class="linenos">133</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Read2-134"><a href="#Read2-134"><span class="linenos">134</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Read2-135"><a href="#Read2-135"><span class="linenos">135</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Read2-136"><a href="#Read2-136"><span class="linenos">136</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="Read2-137"><a href="#Read2-137"><span class="linenos">137</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Read2-138"><a href="#Read2-138"><span class="linenos">138</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Read2-139"><a href="#Read2-139"><span class="linenos">139</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Read2-140"><a href="#Read2-140"><span class="linenos">140</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Read2-141"><a href="#Read2-141"><span class="linenos">141</span></a> <span class="p">)</span>
+</span><span id="Read2-142"><a href="#Read2-142"><span class="linenos">142</span></a> <span class="p">)</span>
+</span><span id="Read2-143"><a href="#Read2-143"><span class="linenos">143</span></a> <span class="p">),</span>
+</span><span id="Read2-144"><a href="#Read2-144"><span class="linenos">144</span></a> <span class="p">),</span>
+</span><span id="Read2-145"><a href="#Read2-145"><span class="linenos">145</span></a> <span class="p">(</span>
+</span><span id="Read2-146"><a href="#Read2-146"><span class="linenos">146</span></a> <span class="c1"># check codes <5 characters, if too short pads it with . (dots) to reach 5 characters</span>
+</span><span id="Read2-147"><a href="#Read2-147"><span class="linenos">147</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="Read2-148"><a href="#Read2-148"><span class="linenos">148</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="Read2-149"><a href="#Read2-149"><span class="linenos">149</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">pad</span><span class="p">(</span>
+</span><span id="Read2-150"><a href="#Read2-150"><span class="linenos">150</span></a> <span class="n">width</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">side</span><span class="o">=</span><span class="s2">"right"</span><span class="p">,</span> <span class="n">fillchar</span><span class="o">=</span><span class="s2">"."</span>
+</span><span id="Read2-151"><a href="#Read2-151"><span class="linenos">151</span></a> <span class="p">),</span>
+</span><span id="Read2-152"><a href="#Read2-152"><span class="linenos">152</span></a> <span class="p">),</span>
+</span><span id="Read2-153"><a href="#Read2-153"><span class="linenos">153</span></a> <span class="p">(</span>
+</span><span id="Read2-154"><a href="#Read2-154"><span class="linenos">154</span></a> <span class="c1"># check codes > 5 characters, If too long, truncates them to 5 characters</span>
+</span><span id="Read2-155"><a href="#Read2-155"><span class="linenos">155</span></a> <span class="s2">"Too Long"</span><span class="p">,</span>
+</span><span id="Read2-156"><a href="#Read2-156"><span class="linenos">156</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="Read2-157"><a href="#Read2-157"><span class="linenos">157</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="p">[:</span><span class="mi">5</span><span class="p">],</span>
+</span><span id="Read2-158"><a href="#Read2-158"><span class="linenos">158</span></a> <span class="p">),</span>
+</span><span id="Read2-159"><a href="#Read2-159"><span class="linenos">159</span></a> <span class="p">(</span>
+</span><span id="Read2-160"><a href="#Read2-160"><span class="linenos">160</span></a> <span class="c1"># checks codes contain numbers, or dots (.), if not logs invalid code error</span>
+</span><span id="Read2-161"><a href="#Read2-161"><span class="linenos">161</span></a> <span class="s2">"Alphanumeric Dot"</span><span class="p">,</span>
+</span><span id="Read2-162"><a href="#Read2-162"><span class="linenos">162</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="Read2-163"><a href="#Read2-163"><span class="linenos">163</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Read2-164"><a href="#Read2-164"><span class="linenos">164</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Read2-165"><a href="#Read2-165"><span class="linenos">165</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Read2-166"><a href="#Read2-166"><span class="linenos">166</span></a> <span class="sa">f</span><span class="s2">"Illegal code format, not alphanumeric dot"</span><span class="p">,</span>
+</span><span id="Read2-167"><a href="#Read2-167"><span class="linenos">167</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Read2-168"><a href="#Read2-168"><span class="linenos">168</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Read2-169"><a href="#Read2-169"><span class="linenos">169</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="Read2-170"><a href="#Read2-170"><span class="linenos">170</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Read2-171"><a href="#Read2-171"><span class="linenos">171</span></a> <span class="p">)</span>
+</span><span id="Read2-172"><a href="#Read2-172"><span class="linenos">172</span></a> <span class="p">)</span>
+</span><span id="Read2-173"><a href="#Read2-173"><span class="linenos">173</span></a> <span class="p">),</span>
+</span><span id="Read2-174"><a href="#Read2-174"><span class="linenos">174</span></a> <span class="p">),</span>
+</span><span id="Read2-175"><a href="#Read2-175"><span class="linenos">175</span></a> <span class="p">(</span>
+</span><span id="Read2-176"><a href="#Read2-176"><span class="linenos">176</span></a> <span class="c1"># checks code exists in self.db (the Read2 dataset). If missing log invalid codes.</span>
+</span><span id="Read2-177"><a href="#Read2-177"><span class="linenos">177</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="Read2-178"><a href="#Read2-178"><span class="linenos">178</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Read2-179"><a href="#Read2-179"><span class="linenos">179</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Read2-180"><a href="#Read2-180"><span class="linenos">180</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Read2-181"><a href="#Read2-181"><span class="linenos">181</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Read2-182"><a href="#Read2-182"><span class="linenos">182</span></a> <span class="sa">f</span><span class="s2">"Codes do not exist in database"</span><span class="p">,</span>
+</span><span id="Read2-183"><a href="#Read2-183"><span class="linenos">183</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Read2-184"><a href="#Read2-184"><span class="linenos">184</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Read2-185"><a href="#Read2-185"><span class="linenos">185</span></a> <span class="n">mask</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Read2-186"><a href="#Read2-186"><span class="linenos">186</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Read2-187"><a href="#Read2-187"><span class="linenos">187</span></a> <span class="p">)</span>
+</span><span id="Read2-188"><a href="#Read2-188"><span class="linenos">188</span></a> <span class="p">)</span>
+</span><span id="Read2-189"><a href="#Read2-189"><span class="linenos">189</span></a> <span class="p">),</span>
+</span><span id="Read2-190"><a href="#Read2-190"><span class="linenos">190</span></a> <span class="p">),</span>
+</span><span id="Read2-191"><a href="#Read2-191"><span class="linenos">191</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>This Read2 class extends Proto, adding custom validation checks for a dataset of "Read2" codes. It ensures that the dataset is loaded, validates the codes based on several rules, and applies corrections or logs errors when necessary.</p>
+</div>
+
+
+ <div id="Read2.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Read2.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Read2.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Read2.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Read2.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Read2.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Read2.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Read3">
+ <input id="Read3-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Read3</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Read3-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Read3"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Read3-194"><a href="#Read3-194"><span class="linenos">194</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Read3</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Read3-195"><a href="#Read3-195"><span class="linenos">195</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Read3-196"><a href="#Read3-196"><span class="linenos">196</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"Read3"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3.parquet"</span><span class="p">)</span>
+</span><span id="Read3-197"><a href="#Read3-197"><span class="linenos">197</span></a>
+</span><span id="Read3-198"><a href="#Read3-198"><span class="linenos">198</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Read3-199"><a href="#Read3-199"><span class="linenos">199</span></a> <span class="p">(</span>
+</span><span id="Read3-200"><a href="#Read3-200"><span class="linenos">200</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Read3-201"><a href="#Read3-201"><span class="linenos">201</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Read3-202"><a href="#Read3-202"><span class="linenos">202</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Read3-203"><a href="#Read3-203"><span class="linenos">203</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Read3-204"><a href="#Read3-204"><span class="linenos">204</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Read3-205"><a href="#Read3-205"><span class="linenos">205</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="Read3-206"><a href="#Read3-206"><span class="linenos">206</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Read3-207"><a href="#Read3-207"><span class="linenos">207</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Read3-208"><a href="#Read3-208"><span class="linenos">208</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Read3-209"><a href="#Read3-209"><span class="linenos">209</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Read3-210"><a href="#Read3-210"><span class="linenos">210</span></a> <span class="p">)</span>
+</span><span id="Read3-211"><a href="#Read3-211"><span class="linenos">211</span></a> <span class="p">)</span>
+</span><span id="Read3-212"><a href="#Read3-212"><span class="linenos">212</span></a> <span class="p">),</span>
+</span><span id="Read3-213"><a href="#Read3-213"><span class="linenos">213</span></a> <span class="p">),</span>
+</span><span id="Read3-214"><a href="#Read3-214"><span class="linenos">214</span></a> <span class="p">(</span>
+</span><span id="Read3-215"><a href="#Read3-215"><span class="linenos">215</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="Read3-216"><a href="#Read3-216"><span class="linenos">216</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="Read3-217"><a href="#Read3-217"><span class="linenos">217</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">pad</span><span class="p">(</span>
+</span><span id="Read3-218"><a href="#Read3-218"><span class="linenos">218</span></a> <span class="n">width</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">side</span><span class="o">=</span><span class="s2">"right"</span><span class="p">,</span> <span class="n">fillchar</span><span class="o">=</span><span class="s2">"."</span>
+</span><span id="Read3-219"><a href="#Read3-219"><span class="linenos">219</span></a> <span class="p">),</span>
+</span><span id="Read3-220"><a href="#Read3-220"><span class="linenos">220</span></a> <span class="p">),</span>
+</span><span id="Read3-221"><a href="#Read3-221"><span class="linenos">221</span></a> <span class="p">(</span>
+</span><span id="Read3-222"><a href="#Read3-222"><span class="linenos">222</span></a> <span class="s2">"Too Long"</span><span class="p">,</span>
+</span><span id="Read3-223"><a href="#Read3-223"><span class="linenos">223</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">5</span><span class="p">),</span>
+</span><span id="Read3-224"><a href="#Read3-224"><span class="linenos">224</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="p">[:</span><span class="mi">5</span><span class="p">],</span>
+</span><span id="Read3-225"><a href="#Read3-225"><span class="linenos">225</span></a> <span class="p">),</span>
+</span><span id="Read3-226"><a href="#Read3-226"><span class="linenos">226</span></a> <span class="p">(</span>
+</span><span id="Read3-227"><a href="#Read3-227"><span class="linenos">227</span></a> <span class="s2">"Alphanumeric Dot"</span><span class="p">,</span>
+</span><span id="Read3-228"><a href="#Read3-228"><span class="linenos">228</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="Read3-229"><a href="#Read3-229"><span class="linenos">229</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Read3-230"><a href="#Read3-230"><span class="linenos">230</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Read3-231"><a href="#Read3-231"><span class="linenos">231</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Read3-232"><a href="#Read3-232"><span class="linenos">232</span></a> <span class="sa">f</span><span class="s2">"QA Alphanumeric Dot"</span><span class="p">,</span>
+</span><span id="Read3-233"><a href="#Read3-233"><span class="linenos">233</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Read3-234"><a href="#Read3-234"><span class="linenos">234</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Read3-235"><a href="#Read3-235"><span class="linenos">235</span></a> <span class="n">check_regex</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[a-zA-Z0-9.]+$"</span><span class="p">),</span>
+</span><span id="Read3-236"><a href="#Read3-236"><span class="linenos">236</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Read3-237"><a href="#Read3-237"><span class="linenos">237</span></a> <span class="p">)</span>
+</span><span id="Read3-238"><a href="#Read3-238"><span class="linenos">238</span></a> <span class="p">)</span>
+</span><span id="Read3-239"><a href="#Read3-239"><span class="linenos">239</span></a> <span class="p">),</span>
+</span><span id="Read3-240"><a href="#Read3-240"><span class="linenos">240</span></a> <span class="p">),</span>
+</span><span id="Read3-241"><a href="#Read3-241"><span class="linenos">241</span></a> <span class="p">(</span>
+</span><span id="Read3-242"><a href="#Read3-242"><span class="linenos">242</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="Read3-243"><a href="#Read3-243"><span class="linenos">243</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Read3-244"><a href="#Read3-244"><span class="linenos">244</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Read3-245"><a href="#Read3-245"><span class="linenos">245</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Read3-246"><a href="#Read3-246"><span class="linenos">246</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Read3-247"><a href="#Read3-247"><span class="linenos">247</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="Read3-248"><a href="#Read3-248"><span class="linenos">248</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Read3-249"><a href="#Read3-249"><span class="linenos">249</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Read3-250"><a href="#Read3-250"><span class="linenos">250</span></a> <span class="n">check_regex</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Read3-251"><a href="#Read3-251"><span class="linenos">251</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Read3-252"><a href="#Read3-252"><span class="linenos">252</span></a> <span class="p">)</span>
+</span><span id="Read3-253"><a href="#Read3-253"><span class="linenos">253</span></a> <span class="p">)</span>
+</span><span id="Read3-254"><a href="#Read3-254"><span class="linenos">254</span></a> <span class="p">),</span>
+</span><span id="Read3-255"><a href="#Read3-255"><span class="linenos">255</span></a> <span class="p">),</span>
+</span><span id="Read3-256"><a href="#Read3-256"><span class="linenos">256</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Read3.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Read3.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Read3.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Read3.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Read3.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Read3.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Read3.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Icd10">
+ <input id="Icd10-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Icd10</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Icd10-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Icd10"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Icd10-259"><a href="#Icd10-259"><span class="linenos">259</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Icd10</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Icd10-260"><a href="#Icd10-260"><span class="linenos">260</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Icd10-261"><a href="#Icd10-261"><span class="linenos">261</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"icd10"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"icd10.parquet"</span><span class="p">)</span>
+</span><span id="Icd10-262"><a href="#Icd10-262"><span class="linenos">262</span></a>
+</span><span id="Icd10-263"><a href="#Icd10-263"><span class="linenos">263</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Icd10-264"><a href="#Icd10-264"><span class="linenos">264</span></a> <span class="p">(</span>
+</span><span id="Icd10-265"><a href="#Icd10-265"><span class="linenos">265</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Icd10-266"><a href="#Icd10-266"><span class="linenos">266</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Icd10-267"><a href="#Icd10-267"><span class="linenos">267</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Icd10-268"><a href="#Icd10-268"><span class="linenos">268</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Icd10-269"><a href="#Icd10-269"><span class="linenos">269</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Icd10-270"><a href="#Icd10-270"><span class="linenos">270</span></a> <span class="sa">f</span><span class="s2">"Code list is empty </span><span class="si">{</span><span class="n">codes_file</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span>
+</span><span id="Icd10-271"><a href="#Icd10-271"><span class="linenos">271</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Icd10-272"><a href="#Icd10-272"><span class="linenos">272</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Icd10-273"><a href="#Icd10-273"><span class="linenos">273</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Icd10-274"><a href="#Icd10-274"><span class="linenos">274</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Icd10-275"><a href="#Icd10-275"><span class="linenos">275</span></a> <span class="p">)</span>
+</span><span id="Icd10-276"><a href="#Icd10-276"><span class="linenos">276</span></a> <span class="p">)</span>
+</span><span id="Icd10-277"><a href="#Icd10-277"><span class="linenos">277</span></a> <span class="p">),</span>
+</span><span id="Icd10-278"><a href="#Icd10-278"><span class="linenos">278</span></a> <span class="p">),</span>
+</span><span id="Icd10-279"><a href="#Icd10-279"><span class="linenos">279</span></a> <span class="p">(</span>
+</span><span id="Icd10-280"><a href="#Icd10-280"><span class="linenos">280</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="Icd10-281"><a href="#Icd10-281"><span class="linenos">281</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">3</span><span class="p">),</span>
+</span><span id="Icd10-282"><a href="#Icd10-282"><span class="linenos">282</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Icd10-283"><a href="#Icd10-283"><span class="linenos">283</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Icd10-284"><a href="#Icd10-284"><span class="linenos">284</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Icd10-285"><a href="#Icd10-285"><span class="linenos">285</span></a> <span class="sa">f</span><span class="s2">"QA Too Short"</span><span class="p">,</span>
+</span><span id="Icd10-286"><a href="#Icd10-286"><span class="linenos">286</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Icd10-287"><a href="#Icd10-287"><span class="linenos">287</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Icd10-288"><a href="#Icd10-288"><span class="linenos">288</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">3</span><span class="p">),</span>
+</span><span id="Icd10-289"><a href="#Icd10-289"><span class="linenos">289</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Icd10-290"><a href="#Icd10-290"><span class="linenos">290</span></a> <span class="p">)</span>
+</span><span id="Icd10-291"><a href="#Icd10-291"><span class="linenos">291</span></a> <span class="p">)</span>
+</span><span id="Icd10-292"><a href="#Icd10-292"><span class="linenos">292</span></a> <span class="p">),</span>
+</span><span id="Icd10-293"><a href="#Icd10-293"><span class="linenos">293</span></a> <span class="p">),</span>
+</span><span id="Icd10-294"><a href="#Icd10-294"><span class="linenos">294</span></a> <span class="p">(</span>
+</span><span id="Icd10-295"><a href="#Icd10-295"><span class="linenos">295</span></a> <span class="s2">"Has Dot"</span><span class="p">,</span>
+</span><span id="Icd10-296"><a href="#Icd10-296"><span class="linenos">296</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">".*\..*"</span><span class="p">)),</span> <span class="c1"># check if contains dot</span>
+</span><span id="Icd10-297"><a href="#Icd10-297"><span class="linenos">297</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span>
+</span><span id="Icd10-298"><a href="#Icd10-298"><span class="linenos">298</span></a> <span class="s2">"."</span><span class="p">,</span> <span class="s2">""</span>
+</span><span id="Icd10-299"><a href="#Icd10-299"><span class="linenos">299</span></a> <span class="p">),</span> <span class="c1"># delete any dots in string</span>
+</span><span id="Icd10-300"><a href="#Icd10-300"><span class="linenos">300</span></a> <span class="c1"># lambda codes : codes.str.split('\.').apply(lambda ls: ls[0]) #only get part before dot</span>
+</span><span id="Icd10-301"><a href="#Icd10-301"><span class="linenos">301</span></a> <span class="p">),</span>
+</span><span id="Icd10-302"><a href="#Icd10-302"><span class="linenos">302</span></a> <span class="p">(</span>
+</span><span id="Icd10-303"><a href="#Icd10-303"><span class="linenos">303</span></a> <span class="s2">"Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="Icd10-304"><a href="#Icd10-304"><span class="linenos">304</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="Icd10-305"><a href="#Icd10-305"><span class="linenos">305</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Icd10-306"><a href="#Icd10-306"><span class="linenos">306</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Icd10-307"><a href="#Icd10-307"><span class="linenos">307</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Icd10-308"><a href="#Icd10-308"><span class="linenos">308</span></a> <span class="sa">f</span><span class="s2">"QA Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="Icd10-309"><a href="#Icd10-309"><span class="linenos">309</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Icd10-310"><a href="#Icd10-310"><span class="linenos">310</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Icd10-311"><a href="#Icd10-311"><span class="linenos">311</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="Icd10-312"><a href="#Icd10-312"><span class="linenos">312</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Icd10-313"><a href="#Icd10-313"><span class="linenos">313</span></a> <span class="p">)</span>
+</span><span id="Icd10-314"><a href="#Icd10-314"><span class="linenos">314</span></a> <span class="p">)</span>
+</span><span id="Icd10-315"><a href="#Icd10-315"><span class="linenos">315</span></a> <span class="p">),</span>
+</span><span id="Icd10-316"><a href="#Icd10-316"><span class="linenos">316</span></a> <span class="p">),</span>
+</span><span id="Icd10-317"><a href="#Icd10-317"><span class="linenos">317</span></a> <span class="p">(</span>
+</span><span id="Icd10-318"><a href="#Icd10-318"><span class="linenos">318</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="Icd10-319"><a href="#Icd10-319"><span class="linenos">319</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span>
+</span><span id="Icd10-320"><a href="#Icd10-320"><span class="linenos">320</span></a> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
+</span><span id="Icd10-321"><a href="#Icd10-321"><span class="linenos">321</span></a> <span class="o">&</span> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">+</span> <span class="s2">"_alt"</span><span class="p">)</span>
+</span><span id="Icd10-322"><a href="#Icd10-322"><span class="linenos">322</span></a> <span class="p">),</span>
+</span><span id="Icd10-323"><a href="#Icd10-323"><span class="linenos">323</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Icd10-324"><a href="#Icd10-324"><span class="linenos">324</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Icd10-325"><a href="#Icd10-325"><span class="linenos">325</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Icd10-326"><a href="#Icd10-326"><span class="linenos">326</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="Icd10-327"><a href="#Icd10-327"><span class="linenos">327</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Icd10-328"><a href="#Icd10-328"><span class="linenos">328</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Icd10-329"><a href="#Icd10-329"><span class="linenos">329</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span>
+</span><span id="Icd10-330"><a href="#Icd10-330"><span class="linenos">330</span></a> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">)</span>
+</span><span id="Icd10-331"><a href="#Icd10-331"><span class="linenos">331</span></a> <span class="o">&</span> <span class="o">~</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span> <span class="o">+</span> <span class="s2">"_alt"</span><span class="p">)</span>
+</span><span id="Icd10-332"><a href="#Icd10-332"><span class="linenos">332</span></a> <span class="p">),</span>
+</span><span id="Icd10-333"><a href="#Icd10-333"><span class="linenos">333</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Icd10-334"><a href="#Icd10-334"><span class="linenos">334</span></a> <span class="p">)</span>
+</span><span id="Icd10-335"><a href="#Icd10-335"><span class="linenos">335</span></a> <span class="p">)</span>
+</span><span id="Icd10-336"><a href="#Icd10-336"><span class="linenos">336</span></a> <span class="p">),</span>
+</span><span id="Icd10-337"><a href="#Icd10-337"><span class="linenos">337</span></a> <span class="p">),</span>
+</span><span id="Icd10-338"><a href="#Icd10-338"><span class="linenos">338</span></a> <span class="c1"># (</span>
+</span><span id="Icd10-339"><a href="#Icd10-339"><span class="linenos">339</span></a> <span class="c1"># "ICD10 Regex",</span>
+</span><span id="Icd10-340"><a href="#Icd10-340"><span class="linenos">340</span></a> <span class="c1"># lambda codes : codes.str.match("[a-zA-Z][0-9][0-9]\.?[a-zA-Z0-9]*$"), #Alpha, Num, Num , Dot?, 4xAlphNum*</span>
+</span><span id="Icd10-341"><a href="#Icd10-341"><span class="linenos">341</span></a> <span class="c1"># lambda codes : lc.log_invalid_code(codes,</span>
+</span><span id="Icd10-342"><a href="#Icd10-342"><span class="linenos">342</span></a> <span class="c1"># codes.str.match("[a-zA-Z][0-9][0-9]\.?[a-zA-Z0-9]*$"), #Log non-matching rows</span>
+</span><span id="Icd10-343"><a href="#Icd10-343"><span class="linenos">343</span></a> <span class="c1"># code_type="icd10",</span>
+</span><span id="Icd10-344"><a href="#Icd10-344"><span class="linenos">344</span></a> <span class="c1">#</span>
+</span><span id="Icd10-345"><a href="#Icd10-345"><span class="linenos">345</span></a> <span class="c1"># )</span>
+</span><span id="Icd10-346"><a href="#Icd10-346"><span class="linenos">346</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Icd10.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Icd10.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Icd10.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Icd10.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Icd10.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Icd10.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Icd10.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Snomed">
+ <input id="Snomed-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Snomed</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Snomed-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Snomed"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Snomed-349"><a href="#Snomed-349"><span class="linenos">349</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Snomed</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Snomed-350"><a href="#Snomed-350"><span class="linenos">350</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Snomed-351"><a href="#Snomed-351"><span class="linenos">351</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"snomed"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"snomed.parquet"</span><span class="p">)</span>
+</span><span id="Snomed-352"><a href="#Snomed-352"><span class="linenos">352</span></a>
+</span><span id="Snomed-353"><a href="#Snomed-353"><span class="linenos">353</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Snomed-354"><a href="#Snomed-354"><span class="linenos">354</span></a> <span class="c1"># (</span>
+</span><span id="Snomed-355"><a href="#Snomed-355"><span class="linenos">355</span></a> <span class="c1"># "Not Empty",</span>
+</span><span id="Snomed-356"><a href="#Snomed-356"><span class="linenos">356</span></a> <span class="c1"># lambda codes : pd.Series([len(codes) > 0]),</span>
+</span><span id="Snomed-357"><a href="#Snomed-357"><span class="linenos">357</span></a> <span class="c1"># lambda codes : raise_exception(Exception("Code List is Empty"))</span>
+</span><span id="Snomed-358"><a href="#Snomed-358"><span class="linenos">358</span></a> <span class="c1"># ),</span>
+</span><span id="Snomed-359"><a href="#Snomed-359"><span class="linenos">359</span></a> <span class="p">(</span>
+</span><span id="Snomed-360"><a href="#Snomed-360"><span class="linenos">360</span></a> <span class="s2">"Too Short"</span><span class="p">,</span>
+</span><span id="Snomed-361"><a href="#Snomed-361"><span class="linenos">361</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">6</span><span class="p">),</span>
+</span><span id="Snomed-362"><a href="#Snomed-362"><span class="linenos">362</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Snomed-363"><a href="#Snomed-363"><span class="linenos">363</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Snomed-364"><a href="#Snomed-364"><span class="linenos">364</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Snomed-365"><a href="#Snomed-365"><span class="linenos">365</span></a> <span class="sa">f</span><span class="s2">"QA Too Short"</span><span class="p">,</span>
+</span><span id="Snomed-366"><a href="#Snomed-366"><span class="linenos">366</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Snomed-367"><a href="#Snomed-367"><span class="linenos">367</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Snomed-368"><a href="#Snomed-368"><span class="linenos">368</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o"><</span> <span class="mi">6</span><span class="p">),</span>
+</span><span id="Snomed-369"><a href="#Snomed-369"><span class="linenos">369</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Snomed-370"><a href="#Snomed-370"><span class="linenos">370</span></a> <span class="p">)</span>
+</span><span id="Snomed-371"><a href="#Snomed-371"><span class="linenos">371</span></a> <span class="p">)</span>
+</span><span id="Snomed-372"><a href="#Snomed-372"><span class="linenos">372</span></a> <span class="p">),</span>
+</span><span id="Snomed-373"><a href="#Snomed-373"><span class="linenos">373</span></a> <span class="p">),</span>
+</span><span id="Snomed-374"><a href="#Snomed-374"><span class="linenos">374</span></a> <span class="p">(</span>
+</span><span id="Snomed-375"><a href="#Snomed-375"><span class="linenos">375</span></a> <span class="s2">"Too Long"</span><span class="p">,</span>
+</span><span id="Snomed-376"><a href="#Snomed-376"><span class="linenos">376</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="o">~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">18</span><span class="p">),</span>
+</span><span id="Snomed-377"><a href="#Snomed-377"><span class="linenos">377</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Snomed-378"><a href="#Snomed-378"><span class="linenos">378</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Snomed-379"><a href="#Snomed-379"><span class="linenos">379</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Snomed-380"><a href="#Snomed-380"><span class="linenos">380</span></a> <span class="sa">f</span><span class="s2">"QA Too Long"</span><span class="p">,</span>
+</span><span id="Snomed-381"><a href="#Snomed-381"><span class="linenos">381</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Snomed-382"><a href="#Snomed-382"><span class="linenos">382</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Snomed-383"><a href="#Snomed-383"><span class="linenos">383</span></a> <span class="n">mask</span><span class="o">=~</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">len</span><span class="p">()</span> <span class="o">></span> <span class="mi">18</span><span class="p">),</span>
+</span><span id="Snomed-384"><a href="#Snomed-384"><span class="linenos">384</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Snomed-385"><a href="#Snomed-385"><span class="linenos">385</span></a> <span class="p">)</span>
+</span><span id="Snomed-386"><a href="#Snomed-386"><span class="linenos">386</span></a> <span class="p">)</span>
+</span><span id="Snomed-387"><a href="#Snomed-387"><span class="linenos">387</span></a> <span class="p">),</span>
+</span><span id="Snomed-388"><a href="#Snomed-388"><span class="linenos">388</span></a> <span class="p">),</span>
+</span><span id="Snomed-389"><a href="#Snomed-389"><span class="linenos">389</span></a> <span class="p">(</span>
+</span><span id="Snomed-390"><a href="#Snomed-390"><span class="linenos">390</span></a> <span class="s2">"Numeric"</span><span class="p">,</span>
+</span><span id="Snomed-391"><a href="#Snomed-391"><span class="linenos">391</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"[0-9]+$"</span><span class="p">),</span>
+</span><span id="Snomed-392"><a href="#Snomed-392"><span class="linenos">392</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Snomed-393"><a href="#Snomed-393"><span class="linenos">393</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Snomed-394"><a href="#Snomed-394"><span class="linenos">394</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Snomed-395"><a href="#Snomed-395"><span class="linenos">395</span></a> <span class="sa">f</span><span class="s2">"QA Numeric"</span><span class="p">,</span>
+</span><span id="Snomed-396"><a href="#Snomed-396"><span class="linenos">396</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Snomed-397"><a href="#Snomed-397"><span class="linenos">397</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Snomed-398"><a href="#Snomed-398"><span class="linenos">398</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"[0-9]+$"</span><span class="p">),</span>
+</span><span id="Snomed-399"><a href="#Snomed-399"><span class="linenos">399</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Snomed-400"><a href="#Snomed-400"><span class="linenos">400</span></a> <span class="p">)</span>
+</span><span id="Snomed-401"><a href="#Snomed-401"><span class="linenos">401</span></a> <span class="p">)</span>
+</span><span id="Snomed-402"><a href="#Snomed-402"><span class="linenos">402</span></a> <span class="p">),</span>
+</span><span id="Snomed-403"><a href="#Snomed-403"><span class="linenos">403</span></a> <span class="p">),</span>
+</span><span id="Snomed-404"><a href="#Snomed-404"><span class="linenos">404</span></a> <span class="c1"># (</span>
+</span><span id="Snomed-405"><a href="#Snomed-405"><span class="linenos">405</span></a> <span class="c1"># "Is Integer",</span>
+</span><span id="Snomed-406"><a href="#Snomed-406"><span class="linenos">406</span></a> <span class="c1"># lambda codes : codes.dtype == int,</span>
+</span><span id="Snomed-407"><a href="#Snomed-407"><span class="linenos">407</span></a> <span class="c1"># lambda codes : codes.astype(int) #Convert to integer</span>
+</span><span id="Snomed-408"><a href="#Snomed-408"><span class="linenos">408</span></a> <span class="c1"># ),</span>
+</span><span id="Snomed-409"><a href="#Snomed-409"><span class="linenos">409</span></a> <span class="p">(</span>
+</span><span id="Snomed-410"><a href="#Snomed-410"><span class="linenos">410</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="Snomed-411"><a href="#Snomed-411"><span class="linenos">411</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Snomed-412"><a href="#Snomed-412"><span class="linenos">412</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Snomed-413"><a href="#Snomed-413"><span class="linenos">413</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Snomed-414"><a href="#Snomed-414"><span class="linenos">414</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Snomed-415"><a href="#Snomed-415"><span class="linenos">415</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="Snomed-416"><a href="#Snomed-416"><span class="linenos">416</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Snomed-417"><a href="#Snomed-417"><span class="linenos">417</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Snomed-418"><a href="#Snomed-418"><span class="linenos">418</span></a> <span class="n">mask</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Snomed-419"><a href="#Snomed-419"><span class="linenos">419</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Snomed-420"><a href="#Snomed-420"><span class="linenos">420</span></a> <span class="p">)</span>
+</span><span id="Snomed-421"><a href="#Snomed-421"><span class="linenos">421</span></a> <span class="p">)</span>
+</span><span id="Snomed-422"><a href="#Snomed-422"><span class="linenos">422</span></a> <span class="p">),</span>
+</span><span id="Snomed-423"><a href="#Snomed-423"><span class="linenos">423</span></a> <span class="p">),</span>
+</span><span id="Snomed-424"><a href="#Snomed-424"><span class="linenos">424</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Snomed.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Snomed.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Snomed.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Snomed.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Snomed.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Snomed.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Snomed.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Opcs4">
+ <input id="Opcs4-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Opcs4</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Opcs4-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Opcs4"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Opcs4-427"><a href="#Opcs4-427"><span class="linenos">427</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Opcs4</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Opcs4-428"><a href="#Opcs4-428"><span class="linenos">428</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Opcs4-429"><a href="#Opcs4-429"><span class="linenos">429</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"opcs4"</span><span class="p">,</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"opcs4.parquet"</span><span class="p">)</span>
+</span><span id="Opcs4-430"><a href="#Opcs4-430"><span class="linenos">430</span></a>
+</span><span id="Opcs4-431"><a href="#Opcs4-431"><span class="linenos">431</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Opcs4-432"><a href="#Opcs4-432"><span class="linenos">432</span></a> <span class="p">(</span>
+</span><span id="Opcs4-433"><a href="#Opcs4-433"><span class="linenos">433</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Opcs4-434"><a href="#Opcs4-434"><span class="linenos">434</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Opcs4-435"><a href="#Opcs4-435"><span class="linenos">435</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Opcs4-436"><a href="#Opcs4-436"><span class="linenos">436</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Opcs4-437"><a href="#Opcs4-437"><span class="linenos">437</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Opcs4-438"><a href="#Opcs4-438"><span class="linenos">438</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="Opcs4-439"><a href="#Opcs4-439"><span class="linenos">439</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Opcs4-440"><a href="#Opcs4-440"><span class="linenos">440</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Opcs4-441"><a href="#Opcs4-441"><span class="linenos">441</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Opcs4-442"><a href="#Opcs4-442"><span class="linenos">442</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Opcs4-443"><a href="#Opcs4-443"><span class="linenos">443</span></a> <span class="p">)</span>
+</span><span id="Opcs4-444"><a href="#Opcs4-444"><span class="linenos">444</span></a> <span class="p">)</span>
+</span><span id="Opcs4-445"><a href="#Opcs4-445"><span class="linenos">445</span></a> <span class="p">),</span>
+</span><span id="Opcs4-446"><a href="#Opcs4-446"><span class="linenos">446</span></a> <span class="p">),</span>
+</span><span id="Opcs4-447"><a href="#Opcs4-447"><span class="linenos">447</span></a> <span class="p">(</span>
+</span><span id="Opcs4-448"><a href="#Opcs4-448"><span class="linenos">448</span></a> <span class="s2">"In Database"</span><span class="p">,</span>
+</span><span id="Opcs4-449"><a href="#Opcs4-449"><span class="linenos">449</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Opcs4-450"><a href="#Opcs4-450"><span class="linenos">450</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Opcs4-451"><a href="#Opcs4-451"><span class="linenos">451</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Opcs4-452"><a href="#Opcs4-452"><span class="linenos">452</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Opcs4-453"><a href="#Opcs4-453"><span class="linenos">453</span></a> <span class="sa">f</span><span class="s2">"QA In Database"</span><span class="p">,</span>
+</span><span id="Opcs4-454"><a href="#Opcs4-454"><span class="linenos">454</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Opcs4-455"><a href="#Opcs4-455"><span class="linenos">455</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Opcs4-456"><a href="#Opcs4-456"><span class="linenos">456</span></a> <span class="n">mask</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">in_database</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">db</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">),</span>
+</span><span id="Opcs4-457"><a href="#Opcs4-457"><span class="linenos">457</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Opcs4-458"><a href="#Opcs4-458"><span class="linenos">458</span></a> <span class="p">)</span>
+</span><span id="Opcs4-459"><a href="#Opcs4-459"><span class="linenos">459</span></a> <span class="p">)</span>
+</span><span id="Opcs4-460"><a href="#Opcs4-460"><span class="linenos">460</span></a> <span class="p">),</span>
+</span><span id="Opcs4-461"><a href="#Opcs4-461"><span class="linenos">461</span></a> <span class="p">),</span>
+</span><span id="Opcs4-462"><a href="#Opcs4-462"><span class="linenos">462</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Opcs4.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Opcs4.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Opcs4.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Opcs4.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Opcs4.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Opcs4.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Opcs4.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Atc">
+ <input id="Atc-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Atc</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Atc-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Atc"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Atc-465"><a href="#Atc-465"><span class="linenos">465</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Atc</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Atc-466"><a href="#Atc-466"><span class="linenos">466</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Atc-467"><a href="#Atc-467"><span class="linenos">467</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"atc"</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="Atc-468"><a href="#Atc-468"><span class="linenos">468</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Atc-469"><a href="#Atc-469"><span class="linenos">469</span></a> <span class="p">(</span>
+</span><span id="Atc-470"><a href="#Atc-470"><span class="linenos">470</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Atc-471"><a href="#Atc-471"><span class="linenos">471</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Atc-472"><a href="#Atc-472"><span class="linenos">472</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Atc-473"><a href="#Atc-473"><span class="linenos">473</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Atc-474"><a href="#Atc-474"><span class="linenos">474</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Atc-475"><a href="#Atc-475"><span class="linenos">475</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="Atc-476"><a href="#Atc-476"><span class="linenos">476</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Atc-477"><a href="#Atc-477"><span class="linenos">477</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Atc-478"><a href="#Atc-478"><span class="linenos">478</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Atc-479"><a href="#Atc-479"><span class="linenos">479</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Atc-480"><a href="#Atc-480"><span class="linenos">480</span></a> <span class="p">)</span>
+</span><span id="Atc-481"><a href="#Atc-481"><span class="linenos">481</span></a> <span class="p">)</span>
+</span><span id="Atc-482"><a href="#Atc-482"><span class="linenos">482</span></a> <span class="p">),</span>
+</span><span id="Atc-483"><a href="#Atc-483"><span class="linenos">483</span></a> <span class="p">),</span>
+</span><span id="Atc-484"><a href="#Atc-484"><span class="linenos">484</span></a> <span class="p">(</span>
+</span><span id="Atc-485"><a href="#Atc-485"><span class="linenos">485</span></a> <span class="s2">"Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="Atc-486"><a href="#Atc-486"><span class="linenos">486</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="Atc-487"><a href="#Atc-487"><span class="linenos">487</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Atc-488"><a href="#Atc-488"><span class="linenos">488</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Atc-489"><a href="#Atc-489"><span class="linenos">489</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Atc-490"><a href="#Atc-490"><span class="linenos">490</span></a> <span class="sa">f</span><span class="s2">"QA Alphanumeric Capital"</span><span class="p">,</span>
+</span><span id="Atc-491"><a href="#Atc-491"><span class="linenos">491</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Atc-492"><a href="#Atc-492"><span class="linenos">492</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Atc-493"><a href="#Atc-493"><span class="linenos">493</span></a> <span class="n">mask</span><span class="o">=</span><span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"^[A-Z0-9]+$"</span><span class="p">),</span>
+</span><span id="Atc-494"><a href="#Atc-494"><span class="linenos">494</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Atc-495"><a href="#Atc-495"><span class="linenos">495</span></a> <span class="p">)</span>
+</span><span id="Atc-496"><a href="#Atc-496"><span class="linenos">496</span></a> <span class="p">)</span>
+</span><span id="Atc-497"><a href="#Atc-497"><span class="linenos">497</span></a> <span class="p">),</span>
+</span><span id="Atc-498"><a href="#Atc-498"><span class="linenos">498</span></a> <span class="p">),</span>
+</span><span id="Atc-499"><a href="#Atc-499"><span class="linenos">499</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Atc.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Atc.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Atc.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Atc.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Atc.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Atc.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Atc.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Med">
+ <input id="Med-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Med</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Med-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Med"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Med-502"><a href="#Med-502"><span class="linenos">502</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Med</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Med-503"><a href="#Med-503"><span class="linenos">503</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Med-504"><a href="#Med-504"><span class="linenos">504</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"med"</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="Med-505"><a href="#Med-505"><span class="linenos">505</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Med-506"><a href="#Med-506"><span class="linenos">506</span></a> <span class="p">(</span>
+</span><span id="Med-507"><a href="#Med-507"><span class="linenos">507</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Med-508"><a href="#Med-508"><span class="linenos">508</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Med-509"><a href="#Med-509"><span class="linenos">509</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Med-510"><a href="#Med-510"><span class="linenos">510</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Med-511"><a href="#Med-511"><span class="linenos">511</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Med-512"><a href="#Med-512"><span class="linenos">512</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="Med-513"><a href="#Med-513"><span class="linenos">513</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Med-514"><a href="#Med-514"><span class="linenos">514</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Med-515"><a href="#Med-515"><span class="linenos">515</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Med-516"><a href="#Med-516"><span class="linenos">516</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Med-517"><a href="#Med-517"><span class="linenos">517</span></a> <span class="p">)</span>
+</span><span id="Med-518"><a href="#Med-518"><span class="linenos">518</span></a> <span class="p">)</span>
+</span><span id="Med-519"><a href="#Med-519"><span class="linenos">519</span></a> <span class="p">),</span>
+</span><span id="Med-520"><a href="#Med-520"><span class="linenos">520</span></a> <span class="p">)</span>
+</span><span id="Med-521"><a href="#Med-521"><span class="linenos">521</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Med.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Med.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Med.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Med.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Med.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Med.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Med.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="Cprd">
+ <input id="Cprd-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">Cprd</span><wbr>(<span class="base"><a href="#Proto">Proto</a></span>):
+
+ <label class="view-source-button" for="Cprd-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#Cprd"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="Cprd-524"><a href="#Cprd-524"><span class="linenos">524</span></a><span class="k">class</span><span class="w"> </span><span class="nc">Cprd</span><span class="p">(</span><span class="n">Proto</span><span class="p">):</span>
+</span><span id="Cprd-525"><a href="#Cprd-525"><span class="linenos">525</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+</span><span id="Cprd-526"><a href="#Cprd-526"><span class="linenos">526</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">"cprd"</span><span class="p">,</span> <span class="n">trud_codes_path</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="Cprd-527"><a href="#Cprd-527"><span class="linenos">527</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">checks</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="Cprd-528"><a href="#Cprd-528"><span class="linenos">528</span></a> <span class="p">(</span>
+</span><span id="Cprd-529"><a href="#Cprd-529"><span class="linenos">529</span></a> <span class="s2">"Not Empty"</span><span class="p">,</span>
+</span><span id="Cprd-530"><a href="#Cprd-530"><span class="linenos">530</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">Series</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">]),</span>
+</span><span id="Cprd-531"><a href="#Cprd-531"><span class="linenos">531</span></a> <span class="k">lambda</span> <span class="n">codes</span><span class="p">,</span> <span class="n">codes_file</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">raise_exception</span><span class="p">(</span>
+</span><span id="Cprd-532"><a href="#Cprd-532"><span class="linenos">532</span></a> <span class="n">InvalidCodesException</span><span class="p">(</span>
+</span><span id="Cprd-533"><a href="#Cprd-533"><span class="linenos">533</span></a> <span class="n">CodesError</span><span class="p">(</span>
+</span><span id="Cprd-534"><a href="#Cprd-534"><span class="linenos">534</span></a> <span class="sa">f</span><span class="s2">"Code list is empty"</span><span class="p">,</span>
+</span><span id="Cprd-535"><a href="#Cprd-535"><span class="linenos">535</span></a> <span class="n">codes</span><span class="o">=</span><span class="n">codes</span><span class="p">,</span>
+</span><span id="Cprd-536"><a href="#Cprd-536"><span class="linenos">536</span></a> <span class="n">codes_file</span><span class="o">=</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="Cprd-537"><a href="#Cprd-537"><span class="linenos">537</span></a> <span class="n">mask</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
+</span><span id="Cprd-538"><a href="#Cprd-538"><span class="linenos">538</span></a> <span class="n">code_type</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">name</span><span class="p">,</span>
+</span><span id="Cprd-539"><a href="#Cprd-539"><span class="linenos">539</span></a> <span class="p">)</span>
+</span><span id="Cprd-540"><a href="#Cprd-540"><span class="linenos">540</span></a> <span class="p">)</span>
+</span><span id="Cprd-541"><a href="#Cprd-541"><span class="linenos">541</span></a> <span class="p">),</span>
+</span><span id="Cprd-542"><a href="#Cprd-542"><span class="linenos">542</span></a> <span class="p">)</span>
+</span><span id="Cprd-543"><a href="#Cprd-543"><span class="linenos">543</span></a> <span class="p">]</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>
+
+<ul>
+<li>Message = The name of the condition (what is printed and logged)</li>
+<li>Condition = True if Passed, and False if Failed</li>
+<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>
+</ul>
+</div>
+
+
+ <div id="Cprd.checks" class="classattr">
+ <div class="attr variable">
+ <span class="name">checks</span>
+
+
+ </div>
+ <a class="headerlink" href="#Cprd.checks"></a>
+
+
+
+ </div>
+ <div class="inherited">
+ <h5>Inherited Members</h5>
+ <dl>
+ <div><dt><a href="#Proto">Proto</a></dt>
+ <dd id="Cprd.name" class="variable"><a href="#Proto.name">name</a></dd>
+ <dd id="Cprd.raise_exception" class="function"><a href="#Proto.raise_exception">raise_exception</a></dd>
+ <dd id="Cprd.in_database" class="function"><a href="#Proto.in_database">in_database</a></dd>
+ <dd id="Cprd.process" class="function"><a href="#Proto.process">process</a></dd>
+ <dd id="Cprd.verify" class="function"><a href="#Proto.verify">verify</a></dd>
+
+ </div>
+ </dl>
+ </div>
+ </section>
+ <section id="CodeTypeParser">
+ <input id="CodeTypeParser-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">CodeTypeParser</span>:
+
+ <label class="view-source-button" for="CodeTypeParser-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#CodeTypeParser"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="CodeTypeParser-546"><a href="#CodeTypeParser-546"><span class="linenos">546</span></a><span class="k">class</span><span class="w"> </span><span class="nc">CodeTypeParser</span><span class="p">:</span>
+</span><span id="CodeTypeParser-547"><a href="#CodeTypeParser-547"><span class="linenos">547</span></a><span class="w"> </span><span class="sd">"""A class used in InvalidCodesException to report an error if a code parser check fails"""</span>
+</span><span id="CodeTypeParser-548"><a href="#CodeTypeParser-548"><span class="linenos">548</span></a>
+</span><span id="CodeTypeParser-549"><a href="#CodeTypeParser-549"><span class="linenos">549</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">trud_processed_dir</span><span class="p">:</span> <span class="n">Path</span> <span class="o">=</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span><span class="p">):</span>
+</span><span id="CodeTypeParser-550"><a href="#CodeTypeParser-550"><span class="linenos">550</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">trud_processed_dir</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">trud_processed_dir</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="CodeTypeParser-551"><a href="#CodeTypeParser-551"><span class="linenos">551</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="CodeTypeParser-552"><a href="#CodeTypeParser-552"><span class="linenos">552</span></a> <span class="sa">f</span><span class="s2">"Cannot initialise parsers as the TRUD processed directory </span><span class="si">{</span><span class="n">trud_processed_dir</span><span class="si">}</span><span class="s2"> does not exist, please check that TRUD has been installed: acmc trud install"</span>
+</span><span id="CodeTypeParser-553"><a href="#CodeTypeParser-553"><span class="linenos">553</span></a> <span class="p">)</span>
+</span><span id="CodeTypeParser-554"><a href="#CodeTypeParser-554"><span class="linenos">554</span></a>
+</span><span id="CodeTypeParser-555"><a href="#CodeTypeParser-555"><span class="linenos">555</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">code_types</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="CodeTypeParser-556"><a href="#CodeTypeParser-556"><span class="linenos">556</span></a> <span class="s2">"read2"</span><span class="p">:</span> <span class="n">Read2</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-557"><a href="#CodeTypeParser-557"><span class="linenos">557</span></a> <span class="s2">"read3"</span><span class="p">:</span> <span class="n">Read3</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-558"><a href="#CodeTypeParser-558"><span class="linenos">558</span></a> <span class="s2">"icd10"</span><span class="p">:</span> <span class="n">Icd10</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-559"><a href="#CodeTypeParser-559"><span class="linenos">559</span></a> <span class="s2">"snomed"</span><span class="p">:</span> <span class="n">Snomed</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-560"><a href="#CodeTypeParser-560"><span class="linenos">560</span></a> <span class="s2">"opcs4"</span><span class="p">:</span> <span class="n">Opcs4</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-561"><a href="#CodeTypeParser-561"><span class="linenos">561</span></a> <span class="s2">"atc"</span><span class="p">:</span> <span class="n">Atc</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-562"><a href="#CodeTypeParser-562"><span class="linenos">562</span></a> <span class="s2">"med"</span><span class="p">:</span> <span class="n">Med</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-563"><a href="#CodeTypeParser-563"><span class="linenos">563</span></a> <span class="s2">"cprd"</span><span class="p">:</span> <span class="n">Cprd</span><span class="p">(),</span>
+</span><span id="CodeTypeParser-564"><a href="#CodeTypeParser-564"><span class="linenos">564</span></a> <span class="p">}</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>A class used in InvalidCodesException to report an error if a code parser check fails</p>
+</div>
+
+
+ <div id="CodeTypeParser.__init__" class="classattr">
+ <input id="CodeTypeParser.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="name">CodeTypeParser</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">trud_processed_dir</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span> <span class="o">=</span> <span class="n">PosixPath</span><span class="p">(</span><span class="s1">'vocab/trud/processed'</span><span class="p">)</span></span>)</span>
+
+ <label class="view-source-button" for="CodeTypeParser.__init__-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#CodeTypeParser.__init__"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="CodeTypeParser.__init__-549"><a href="#CodeTypeParser.__init__-549"><span class="linenos">549</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">trud_processed_dir</span><span class="p">:</span> <span class="n">Path</span> <span class="o">=</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span><span class="p">):</span>
+</span><span id="CodeTypeParser.__init__-550"><a href="#CodeTypeParser.__init__-550"><span class="linenos">550</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">trud_processed_dir</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">trud_processed_dir</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="CodeTypeParser.__init__-551"><a href="#CodeTypeParser.__init__-551"><span class="linenos">551</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="CodeTypeParser.__init__-552"><a href="#CodeTypeParser.__init__-552"><span class="linenos">552</span></a> <span class="sa">f</span><span class="s2">"Cannot initialise parsers as the TRUD processed directory </span><span class="si">{</span><span class="n">trud_processed_dir</span><span class="si">}</span><span class="s2"> does not exist, please check that TRUD has been installed: acmc trud install"</span>
+</span><span id="CodeTypeParser.__init__-553"><a href="#CodeTypeParser.__init__-553"><span class="linenos">553</span></a> <span class="p">)</span>
+</span><span id="CodeTypeParser.__init__-554"><a href="#CodeTypeParser.__init__-554"><span class="linenos">554</span></a>
+</span><span id="CodeTypeParser.__init__-555"><a href="#CodeTypeParser.__init__-555"><span class="linenos">555</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">code_types</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="CodeTypeParser.__init__-556"><a href="#CodeTypeParser.__init__-556"><span class="linenos">556</span></a> <span class="s2">"read2"</span><span class="p">:</span> <span class="n">Read2</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-557"><a href="#CodeTypeParser.__init__-557"><span class="linenos">557</span></a> <span class="s2">"read3"</span><span class="p">:</span> <span class="n">Read3</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-558"><a href="#CodeTypeParser.__init__-558"><span class="linenos">558</span></a> <span class="s2">"icd10"</span><span class="p">:</span> <span class="n">Icd10</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-559"><a href="#CodeTypeParser.__init__-559"><span class="linenos">559</span></a> <span class="s2">"snomed"</span><span class="p">:</span> <span class="n">Snomed</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-560"><a href="#CodeTypeParser.__init__-560"><span class="linenos">560</span></a> <span class="s2">"opcs4"</span><span class="p">:</span> <span class="n">Opcs4</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-561"><a href="#CodeTypeParser.__init__-561"><span class="linenos">561</span></a> <span class="s2">"atc"</span><span class="p">:</span> <span class="n">Atc</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-562"><a href="#CodeTypeParser.__init__-562"><span class="linenos">562</span></a> <span class="s2">"med"</span><span class="p">:</span> <span class="n">Med</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-563"><a href="#CodeTypeParser.__init__-563"><span class="linenos">563</span></a> <span class="s2">"cprd"</span><span class="p">:</span> <span class="n">Cprd</span><span class="p">(),</span>
+</span><span id="CodeTypeParser.__init__-564"><a href="#CodeTypeParser.__init__-564"><span class="linenos">564</span></a> <span class="p">}</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ <div id="CodeTypeParser.code_types" class="classattr">
+ <div class="attr variable">
+ <span class="name">code_types</span>
+
+
+ </div>
+ <a class="headerlink" href="#CodeTypeParser.code_types"></a>
+
+
+
+ </div>
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/phen.html b/docs/api/acmc/phen.html
new file mode 100644
index 0000000000000000000000000000000000000000..e09b51cd960f02a7fcafc0e726d4f9e998aadd71
--- /dev/null
+++ b/docs/api/acmc/phen.html
@@ -0,0 +1,3272 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.phen API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#PHEN_DIR">PHEN_DIR</a>
+ </li>
+ <li>
+ <a class="variable" href="#DEFAULT_PHEN_PATH">DEFAULT_PHEN_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#CONCEPTS_DIR">CONCEPTS_DIR</a>
+ </li>
+ <li>
+ <a class="variable" href="#MAP_DIR">MAP_DIR</a>
+ </li>
+ <li>
+ <a class="variable" href="#CONCEPT_SET_DIR">CONCEPT_SET_DIR</a>
+ </li>
+ <li>
+ <a class="variable" href="#CSV_PATH">CSV_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#OMOP_PATH">OMOP_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#DEFAULT_PHEN_DIR_LIST">DEFAULT_PHEN_DIR_LIST</a>
+ </li>
+ <li>
+ <a class="variable" href="#CONFIG_FILE">CONFIG_FILE</a>
+ </li>
+ <li>
+ <a class="variable" href="#VOCAB_VERSION_FILE">VOCAB_VERSION_FILE</a>
+ </li>
+ <li>
+ <a class="variable" href="#SEMANTIC_VERSION_TYPES">SEMANTIC_VERSION_TYPES</a>
+ </li>
+ <li>
+ <a class="variable" href="#DEFAULT_VERSION_INC">DEFAULT_VERSION_INC</a>
+ </li>
+ <li>
+ <a class="variable" href="#DEFAULT_GIT_BRANCH">DEFAULT_GIT_BRANCH</a>
+ </li>
+ <li>
+ <a class="variable" href="#SPLIT_COL_ACTION">SPLIT_COL_ACTION</a>
+ </li>
+ <li>
+ <a class="variable" href="#CODES_COL_ACTION">CODES_COL_ACTION</a>
+ </li>
+ <li>
+ <a class="variable" href="#DIVIDE_COL_ACTION">DIVIDE_COL_ACTION</a>
+ </li>
+ <li>
+ <a class="variable" href="#COL_ACTIONS">COL_ACTIONS</a>
+ </li>
+ <li>
+ <a class="variable" href="#CODE_FILE_TYPES">CODE_FILE_TYPES</a>
+ </li>
+ <li>
+ <a class="variable" href="#CONFIG_SCHEMA">CONFIG_SCHEMA</a>
+ </li>
+ <li>
+ <a class="class" href="#PhenValidationException">PhenValidationException</a>
+ <ul class="memberlist">
+ <li>
+ <a class="function" href="#PhenValidationException.__init__">PhenValidationException</a>
+ </li>
+ <li>
+ <a class="variable" href="#PhenValidationException.validation_errors">validation_errors</a>
+ </li>
+ </ul>
+
+ </li>
+ <li>
+ <a class="function" href="#init">init</a>
+ </li>
+ <li>
+ <a class="function" href="#fork">fork</a>
+ </li>
+ <li>
+ <a class="function" href="#validate">validate</a>
+ </li>
+ <li>
+ <a class="function" href="#translate_codes">translate_codes</a>
+ </li>
+ <li>
+ <a class="function" href="#write_vocab_version">write_vocab_version</a>
+ </li>
+ <li>
+ <a class="function" href="#map">map</a>
+ </li>
+ <li>
+ <a class="function" href="#publish">publish</a>
+ </li>
+ <li>
+ <a class="function" href="#export">export</a>
+ </li>
+ <li>
+ <a class="function" href="#copy">copy</a>
+ </li>
+ <li>
+ <a class="function" href="#extract_concepts">extract_concepts</a>
+ </li>
+ <li>
+ <a class="function" href="#diff_config">diff_config</a>
+ </li>
+ <li>
+ <a class="function" href="#diff_map_files">diff_map_files</a>
+ </li>
+ <li>
+ <a class="function" href="#diff_phen">diff_phen</a>
+ </li>
+ <li>
+ <a class="function" href="#diff">diff</a>
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.phen </h1>
+
+ <div class="docstring"><p>phenotype.py module</p>
+
+<p>This module provides functionality for managing phenotypes.</p>
+</div>
+
+ <input id="mod-phen-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-phen-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="sd">"""</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="sd">phenotype.py module</span>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">This module provides functionality for managing phenotypes.</span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a><span class="sd">"""</span>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">argparse</span>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
+</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">json</span>
+</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">os</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">sqlite3</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">sys</span>
+</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">shutil</span>
+</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">time</span>
+</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">git</span>
+</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">re</span>
+</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">logging</span>
+</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">requests</span>
+</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">yaml</span>
+</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">semver</span>
+</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">git</span><span class="w"> </span><span class="kn">import</span> <span class="n">Repo</span>
+</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">cerberus</span><span class="w"> </span><span class="kn">import</span> <span class="n">Validator</span> <span class="c1"># type: ignore</span>
+</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">deepdiff</span><span class="w"> </span><span class="kn">import</span> <span class="n">DeepDiff</span>
+</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">pathlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">Path</span>
+</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">urllib.parse</span><span class="w"> </span><span class="kn">import</span> <span class="n">urlparse</span><span class="p">,</span> <span class="n">urlunparse</span>
+</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Tuple</span><span class="p">,</span> <span class="n">Set</span><span class="p">,</span> <span class="n">Any</span>
+</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">acmc</span>
+</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">acmc</span><span class="w"> </span><span class="kn">import</span> <span class="n">trud</span><span class="p">,</span> <span class="n">omop</span><span class="p">,</span> <span class="n">parse</span><span class="p">,</span> <span class="n">util</span><span class="p">,</span> <span class="n">logging_config</span> <span class="k">as</span> <span class="n">lc</span>
+</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a>
+</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="c1"># set up logging</span>
+</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a><span class="n">_logger</span> <span class="o">=</span> <span class="n">lc</span><span class="o">.</span><span class="n">setup_logger</span><span class="p">()</span>
+</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a>
+</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="n">pd</span><span class="o">.</span><span class="n">set_option</span><span class="p">(</span><span class="s2">"mode.chained_assignment"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
+</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a>
+</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="n">PHEN_DIR</span> <span class="o">=</span> <span class="s2">"phen"</span>
+</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd">"""Default phenotype directory name"""</span>
+</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
+</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="n">DEFAULT_PHEN_PATH</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s2">"./workspace"</span><span class="p">)</span> <span class="o">/</span> <span class="n">PHEN_DIR</span>
+</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd">"""Default phenotype directory path"""</span>
+</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a>
+</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a><span class="n">CONCEPTS_DIR</span> <span class="o">=</span> <span class="s2">"concepts"</span>
+</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="sd">"""Default concepts directory name"""</span>
+</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a>
+</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a><span class="n">MAP_DIR</span> <span class="o">=</span> <span class="s2">"map"</span>
+</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a><span class="sd">"""Default map directory name"""</span>
+</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a>
+</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a><span class="n">CONCEPT_SET_DIR</span> <span class="o">=</span> <span class="s2">"concept-sets"</span>
+</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a><span class="sd">"""Default concept set directory name"""</span>
+</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a>
+</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a><span class="n">CSV_PATH</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">CONCEPT_SET_DIR</span><span class="p">)</span> <span class="o">/</span> <span class="s2">"csv"</span>
+</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a><span class="sd">"""Default CSV concept set directory path"""</span>
+</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a>
+</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a><span class="n">OMOP_PATH</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">CONCEPT_SET_DIR</span><span class="p">)</span> <span class="o">/</span> <span class="s2">"omop"</span>
+</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a><span class="sd">"""Default OMOP concept set directory path"""</span>
+</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a>
+</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a><span class="n">DEFAULT_PHEN_DIR_LIST</span> <span class="o">=</span> <span class="p">[</span><span class="n">CONCEPTS_DIR</span><span class="p">,</span> <span class="n">MAP_DIR</span><span class="p">,</span> <span class="n">CONCEPT_SET_DIR</span><span class="p">]</span>
+</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a><span class="sd">"""List of default phenotype directories"""</span>
+</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a>
+</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a><span class="n">CONFIG_FILE</span> <span class="o">=</span> <span class="s2">"config.yml"</span>
+</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a><span class="sd">"""Default configuration filename"""</span>
+</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a>
+</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a><span class="n">VOCAB_VERSION_FILE</span> <span class="o">=</span> <span class="s2">"vocab_version.yml"</span>
+</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a><span class="sd">"""Default vocabulary version filename"""</span>
+</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a>
+</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a><span class="n">SEMANTIC_VERSION_TYPES</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"major"</span><span class="p">,</span> <span class="s2">"minor"</span><span class="p">,</span> <span class="s2">"patch"</span><span class="p">]</span>
+</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="sd">"""List of semantic version increment types"""</span>
+</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a>
+</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a><span class="n">DEFAULT_VERSION_INC</span> <span class="o">=</span> <span class="s2">"patch"</span>
+</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a><span class="sd">"""Default semantic version increment type"""</span>
+</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a>
+</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a><span class="n">DEFAULT_GIT_BRANCH</span> <span class="o">=</span> <span class="s2">"main"</span>
+</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a><span class="sd">"""Default phenotype repo branch name"""</span>
+</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a>
+</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a><span class="n">SPLIT_COL_ACTION</span> <span class="o">=</span> <span class="s2">"split_col"</span>
+</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a><span class="sd">"""Split column preprocessing action type"""</span>
+</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a>
+</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a><span class="n">CODES_COL_ACTION</span> <span class="o">=</span> <span class="s2">"codes_col"</span>
+</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a><span class="sd">"""Codes column preprocessing action type"""</span>
+</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
+</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a><span class="n">DIVIDE_COL_ACTION</span> <span class="o">=</span> <span class="s2">"divide_col"</span>
+</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a><span class="sd">"""Divide column preprocessing action type"""</span>
+</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a>
+</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a><span class="n">COL_ACTIONS</span> <span class="o">=</span> <span class="p">[</span><span class="n">SPLIT_COL_ACTION</span><span class="p">,</span> <span class="n">CODES_COL_ACTION</span><span class="p">,</span> <span class="n">DIVIDE_COL_ACTION</span><span class="p">]</span>
+</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a><span class="sd">"""List of column preprocessing action types"""</span>
+</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
+</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a><span class="n">CODE_FILE_TYPES</span> <span class="o">=</span> <span class="p">[</span><span class="s2">".xlsx"</span><span class="p">,</span> <span class="s2">".xls"</span><span class="p">,</span> <span class="s2">".csv"</span><span class="p">]</span>
+</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a><span class="sd">"""List of supported source concept coding list file types"""</span>
+</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a>
+</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a><span class="c1"># config.yaml schema</span>
+</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a><span class="n">CONFIG_SCHEMA</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="s2">"phenotype"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span>
+</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
+</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a> <span class="s2">"version"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span>
+</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
+</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="s2">"regex"</span><span class="p">:</span> <span class="sa">r</span><span class="s2">"^\d+\.\d+\.\d+$"</span><span class="p">,</span> <span class="c1"># Enforces 'vN.N.N' format</span>
+</span><span id="L-100"><a href="#L-100"><span class="linenos"> 100</span></a> <span class="p">},</span>
+</span><span id="L-101"><a href="#L-101"><span class="linenos"> 101</span></a> <span class="s2">"omop"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-102"><a href="#L-102"><span class="linenos"> 102</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span>
+</span><span id="L-103"><a href="#L-103"><span class="linenos"> 103</span></a> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
+</span><span id="L-104"><a href="#L-104"><span class="linenos"> 104</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-105"><a href="#L-105"><span class="linenos"> 105</span></a> <span class="s2">"vocabulary_id"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">},</span>
+</span><span id="L-106"><a href="#L-106"><span class="linenos"> 106</span></a> <span class="s2">"vocabulary_name"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">},</span>
+</span><span id="L-107"><a href="#L-107"><span class="linenos"> 107</span></a> <span class="s2">"vocabulary_reference"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-108"><a href="#L-108"><span class="linenos"> 108</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span>
+</span><span id="L-109"><a href="#L-109"><span class="linenos"> 109</span></a> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
+</span><span id="L-110"><a href="#L-110"><span class="linenos"> 110</span></a> <span class="s2">"regex"</span><span class="p">:</span> <span class="sa">r</span><span class="s2">"^https?://.*"</span><span class="p">,</span> <span class="c1"># Ensures it's a URL</span>
+</span><span id="L-111"><a href="#L-111"><span class="linenos"> 111</span></a> <span class="p">},</span>
+</span><span id="L-112"><a href="#L-112"><span class="linenos"> 112</span></a> <span class="p">},</span>
+</span><span id="L-113"><a href="#L-113"><span class="linenos"> 113</span></a> <span class="p">},</span>
+</span><span id="L-114"><a href="#L-114"><span class="linenos"> 114</span></a> <span class="s2">"map"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-115"><a href="#L-115"><span class="linenos"> 115</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"list"</span><span class="p">,</span>
+</span><span id="L-116"><a href="#L-116"><span class="linenos"> 116</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-117"><a href="#L-117"><span class="linenos"> 117</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span>
+</span><span id="L-118"><a href="#L-118"><span class="linenos"> 118</span></a> <span class="s2">"allowed"</span><span class="p">:</span> <span class="nb">list</span><span class="p">(</span>
+</span><span id="L-119"><a href="#L-119"><span class="linenos"> 119</span></a> <span class="n">parse</span><span class="o">.</span><span class="n">SUPPORTED_CODE_TYPES</span>
+</span><span id="L-120"><a href="#L-120"><span class="linenos"> 120</span></a> <span class="p">),</span> <span class="c1"># Ensure only predefined values are allowed</span>
+</span><span id="L-121"><a href="#L-121"><span class="linenos"> 121</span></a> <span class="p">},</span>
+</span><span id="L-122"><a href="#L-122"><span class="linenos"> 122</span></a> <span class="p">},</span>
+</span><span id="L-123"><a href="#L-123"><span class="linenos"> 123</span></a> <span class="s2">"concept_sets"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-124"><a href="#L-124"><span class="linenos"> 124</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"list"</span><span class="p">,</span>
+</span><span id="L-125"><a href="#L-125"><span class="linenos"> 125</span></a> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">,</span>
+</span><span id="L-126"><a href="#L-126"><span class="linenos"> 126</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-127"><a href="#L-127"><span class="linenos"> 127</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span>
+</span><span id="L-128"><a href="#L-128"><span class="linenos"> 128</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-129"><a href="#L-129"><span class="linenos"> 129</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">},</span>
+</span><span id="L-130"><a href="#L-130"><span class="linenos"> 130</span></a> <span class="s2">"file"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-131"><a href="#L-131"><span class="linenos"> 131</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span>
+</span><span id="L-132"><a href="#L-132"><span class="linenos"> 132</span></a> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
+</span><span id="L-133"><a href="#L-133"><span class="linenos"> 133</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-134"><a href="#L-134"><span class="linenos"> 134</span></a> <span class="s2">"path"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">,</span> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">},</span>
+</span><span id="L-135"><a href="#L-135"><span class="linenos"> 135</span></a> <span class="s2">"columns"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">},</span>
+</span><span id="L-136"><a href="#L-136"><span class="linenos"> 136</span></a> <span class="s2">"category"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-137"><a href="#L-137"><span class="linenos"> 137</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span>
+</span><span id="L-138"><a href="#L-138"><span class="linenos"> 138</span></a> <span class="p">},</span> <span class="c1"># Optional but must be string if present</span>
+</span><span id="L-139"><a href="#L-139"><span class="linenos"> 139</span></a> <span class="s2">"actions"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-140"><a href="#L-140"><span class="linenos"> 140</span></a> <span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span>
+</span><span id="L-141"><a href="#L-141"><span class="linenos"> 141</span></a> <span class="s2">"schema"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"divide_col"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"string"</span><span class="p">}},</span>
+</span><span id="L-142"><a href="#L-142"><span class="linenos"> 142</span></a> <span class="p">},</span>
+</span><span id="L-143"><a href="#L-143"><span class="linenos"> 143</span></a> <span class="p">},</span>
+</span><span id="L-144"><a href="#L-144"><span class="linenos"> 144</span></a> <span class="p">},</span>
+</span><span id="L-145"><a href="#L-145"><span class="linenos"> 145</span></a> <span class="s2">"metadata"</span><span class="p">:</span> <span class="p">{</span><span class="s2">"type"</span><span class="p">:</span> <span class="s2">"dict"</span><span class="p">,</span> <span class="s2">"required"</span><span class="p">:</span> <span class="kc">True</span><span class="p">},</span>
+</span><span id="L-146"><a href="#L-146"><span class="linenos"> 146</span></a> <span class="p">},</span>
+</span><span id="L-147"><a href="#L-147"><span class="linenos"> 147</span></a> <span class="p">},</span>
+</span><span id="L-148"><a href="#L-148"><span class="linenos"> 148</span></a> <span class="p">},</span>
+</span><span id="L-149"><a href="#L-149"><span class="linenos"> 149</span></a> <span class="p">},</span>
+</span><span id="L-150"><a href="#L-150"><span class="linenos"> 150</span></a> <span class="p">}</span>
+</span><span id="L-151"><a href="#L-151"><span class="linenos"> 151</span></a><span class="p">}</span>
+</span><span id="L-152"><a href="#L-152"><span class="linenos"> 152</span></a><span class="sd">"""Phenotype config.yml schema definition"""</span>
+</span><span id="L-153"><a href="#L-153"><span class="linenos"> 153</span></a>
+</span><span id="L-154"><a href="#L-154"><span class="linenos"> 154</span></a>
+</span><span id="L-155"><a href="#L-155"><span class="linenos"> 155</span></a><span class="k">class</span><span class="w"> </span><span class="nc">PhenValidationException</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span>
+</span><span id="L-156"><a href="#L-156"><span class="linenos"> 156</span></a><span class="w"> </span><span class="sd">"""Custom exception class raised when validation errors in phenotype configuration file"""</span>
+</span><span id="L-157"><a href="#L-157"><span class="linenos"> 157</span></a>
+</span><span id="L-158"><a href="#L-158"><span class="linenos"> 158</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">validation_errors</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+</span><span id="L-159"><a href="#L-159"><span class="linenos"> 159</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">message</span><span class="p">)</span>
+</span><span id="L-160"><a href="#L-160"><span class="linenos"> 160</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">validation_errors</span> <span class="o">=</span> <span class="n">validation_errors</span>
+</span><span id="L-161"><a href="#L-161"><span class="linenos"> 161</span></a>
+</span><span id="L-162"><a href="#L-162"><span class="linenos"> 162</span></a>
+</span><span id="L-163"><a href="#L-163"><span class="linenos"> 163</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_construct_git_url</span><span class="p">(</span><span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-164"><a href="#L-164"><span class="linenos"> 164</span></a><span class="w"> </span><span class="sd">"""Constructs a git url for github or gitlab including a PAT token environment variable"""</span>
+</span><span id="L-165"><a href="#L-165"><span class="linenos"> 165</span></a> <span class="c1"># check the url</span>
+</span><span id="L-166"><a href="#L-166"><span class="linenos"> 166</span></a> <span class="n">parsed_url</span> <span class="o">=</span> <span class="n">urlparse</span><span class="p">(</span><span class="n">remote_url</span><span class="p">)</span>
+</span><span id="L-167"><a href="#L-167"><span class="linenos"> 167</span></a>
+</span><span id="L-168"><a href="#L-168"><span class="linenos"> 168</span></a> <span class="c1"># if github in the URL otherwise assume it's gitlab, if we want to use others such as codeberg we'd</span>
+</span><span id="L-169"><a href="#L-169"><span class="linenos"> 169</span></a> <span class="c1"># need to update this function if the URL scheme is different.</span>
+</span><span id="L-170"><a href="#L-170"><span class="linenos"> 170</span></a> <span class="k">if</span> <span class="s2">"github.com"</span> <span class="ow">in</span> <span class="n">parsed_url</span><span class="o">.</span><span class="n">netloc</span><span class="p">:</span>
+</span><span id="L-171"><a href="#L-171"><span class="linenos"> 171</span></a> <span class="c1"># get GitHub PAT from environment variable</span>
+</span><span id="L-172"><a href="#L-172"><span class="linenos"> 172</span></a> <span class="n">auth</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"ACMC_GITHUB_PAT"</span><span class="p">)</span>
+</span><span id="L-173"><a href="#L-173"><span class="linenos"> 173</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">auth</span><span class="p">:</span>
+</span><span id="L-174"><a href="#L-174"><span class="linenos"> 174</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-175"><a href="#L-175"><span class="linenos"> 175</span></a> <span class="s2">"GitHub PAT not found. Set the ACMC_GITHUB_PAT environment variable."</span>
+</span><span id="L-176"><a href="#L-176"><span class="linenos"> 176</span></a> <span class="p">)</span>
+</span><span id="L-177"><a href="#L-177"><span class="linenos"> 177</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-178"><a href="#L-178"><span class="linenos"> 178</span></a> <span class="c1"># get GitLab PAT from environment variable</span>
+</span><span id="L-179"><a href="#L-179"><span class="linenos"> 179</span></a> <span class="n">auth</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"ACMC_GITLAB_PAT"</span><span class="p">)</span>
+</span><span id="L-180"><a href="#L-180"><span class="linenos"> 180</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">auth</span><span class="p">:</span>
+</span><span id="L-181"><a href="#L-181"><span class="linenos"> 181</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-182"><a href="#L-182"><span class="linenos"> 182</span></a> <span class="s2">"GitLab PAT not found. Set the ACMC_GITLAB_PAT environment variable."</span>
+</span><span id="L-183"><a href="#L-183"><span class="linenos"> 183</span></a> <span class="p">)</span>
+</span><span id="L-184"><a href="#L-184"><span class="linenos"> 184</span></a> <span class="n">auth</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"oauth2:</span><span class="si">{</span><span class="n">auth</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-185"><a href="#L-185"><span class="linenos"> 185</span></a>
+</span><span id="L-186"><a href="#L-186"><span class="linenos"> 186</span></a> <span class="c1"># Construct the new URL with credentials</span>
+</span><span id="L-187"><a href="#L-187"><span class="linenos"> 187</span></a> <span class="n">new_netloc</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">auth</span><span class="si">}</span><span class="s2">@</span><span class="si">{</span><span class="n">parsed_url</span><span class="o">.</span><span class="n">netloc</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-188"><a href="#L-188"><span class="linenos"> 188</span></a> <span class="k">return</span> <span class="n">urlunparse</span><span class="p">(</span>
+</span><span id="L-189"><a href="#L-189"><span class="linenos"> 189</span></a> <span class="p">(</span>
+</span><span id="L-190"><a href="#L-190"><span class="linenos"> 190</span></a> <span class="n">parsed_url</span><span class="o">.</span><span class="n">scheme</span><span class="p">,</span>
+</span><span id="L-191"><a href="#L-191"><span class="linenos"> 191</span></a> <span class="n">new_netloc</span><span class="p">,</span>
+</span><span id="L-192"><a href="#L-192"><span class="linenos"> 192</span></a> <span class="n">parsed_url</span><span class="o">.</span><span class="n">path</span><span class="p">,</span>
+</span><span id="L-193"><a href="#L-193"><span class="linenos"> 193</span></a> <span class="n">parsed_url</span><span class="o">.</span><span class="n">params</span><span class="p">,</span>
+</span><span id="L-194"><a href="#L-194"><span class="linenos"> 194</span></a> <span class="n">parsed_url</span><span class="o">.</span><span class="n">query</span><span class="p">,</span>
+</span><span id="L-195"><a href="#L-195"><span class="linenos"> 195</span></a> <span class="n">parsed_url</span><span class="o">.</span><span class="n">fragment</span><span class="p">,</span>
+</span><span id="L-196"><a href="#L-196"><span class="linenos"> 196</span></a> <span class="p">)</span>
+</span><span id="L-197"><a href="#L-197"><span class="linenos"> 197</span></a> <span class="p">)</span>
+</span><span id="L-198"><a href="#L-198"><span class="linenos"> 198</span></a>
+</span><span id="L-199"><a href="#L-199"><span class="linenos"> 199</span></a>
+</span><span id="L-200"><a href="#L-200"><span class="linenos"> 200</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_create_empty_git_dir</span><span class="p">(</span><span class="n">path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="L-201"><a href="#L-201"><span class="linenos"> 201</span></a><span class="w"> </span><span class="sd">"""Creates a directory with a .gitkeep file so that it's tracked in git"""</span>
+</span><span id="L-202"><a href="#L-202"><span class="linenos"> 202</span></a> <span class="n">path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-203"><a href="#L-203"><span class="linenos"> 203</span></a> <span class="n">keep_path</span> <span class="o">=</span> <span class="n">path</span> <span class="o">/</span> <span class="s2">".gitkeep"</span>
+</span><span id="L-204"><a href="#L-204"><span class="linenos"> 204</span></a> <span class="n">keep_path</span><span class="o">.</span><span class="n">touch</span><span class="p">(</span><span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-205"><a href="#L-205"><span class="linenos"> 205</span></a>
+</span><span id="L-206"><a href="#L-206"><span class="linenos"> 206</span></a>
+</span><span id="L-207"><a href="#L-207"><span class="linenos"> 207</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_check_delete_dir</span><span class="p">(</span><span class="n">path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">msg</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="nb">bool</span><span class="p">:</span>
+</span><span id="L-208"><a href="#L-208"><span class="linenos"> 208</span></a><span class="w"> </span><span class="sd">"""Checks on the command line if a user wants to delete a directory</span>
+</span><span id="L-209"><a href="#L-209"><span class="linenos"> 209</span></a>
+</span><span id="L-210"><a href="#L-210"><span class="linenos"> 210</span></a><span class="sd"> Args:</span>
+</span><span id="L-211"><a href="#L-211"><span class="linenos"> 211</span></a><span class="sd"> path (Path): path of the directory to be deleted</span>
+</span><span id="L-212"><a href="#L-212"><span class="linenos"> 212</span></a><span class="sd"> msg (str): message to be displayed to the user</span>
+</span><span id="L-213"><a href="#L-213"><span class="linenos"> 213</span></a>
+</span><span id="L-214"><a href="#L-214"><span class="linenos"> 214</span></a><span class="sd"> Returns:</span>
+</span><span id="L-215"><a href="#L-215"><span class="linenos"> 215</span></a><span class="sd"> Boolean: True if deleted</span>
+</span><span id="L-216"><a href="#L-216"><span class="linenos"> 216</span></a><span class="sd"> """</span>
+</span><span id="L-217"><a href="#L-217"><span class="linenos"> 217</span></a> <span class="n">deleted</span> <span class="o">=</span> <span class="kc">False</span>
+</span><span id="L-218"><a href="#L-218"><span class="linenos"> 218</span></a>
+</span><span id="L-219"><a href="#L-219"><span class="linenos"> 219</span></a> <span class="n">user_input</span> <span class="o">=</span> <span class="nb">input</span><span class="p">(</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">msg</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
+</span><span id="L-220"><a href="#L-220"><span class="linenos"> 220</span></a> <span class="k">if</span> <span class="n">user_input</span> <span class="ow">in</span> <span class="p">[</span><span class="s2">"yes"</span><span class="p">,</span> <span class="s2">"y"</span><span class="p">]:</span>
+</span><span id="L-221"><a href="#L-221"><span class="linenos"> 221</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>
+</span><span id="L-222"><a href="#L-222"><span class="linenos"> 222</span></a> <span class="n">deleted</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-223"><a href="#L-223"><span class="linenos"> 223</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-224"><a href="#L-224"><span class="linenos"> 224</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Directory was not deleted."</span><span class="p">)</span>
+</span><span id="L-225"><a href="#L-225"><span class="linenos"> 225</span></a>
+</span><span id="L-226"><a href="#L-226"><span class="linenos"> 226</span></a> <span class="k">return</span> <span class="n">deleted</span>
+</span><span id="L-227"><a href="#L-227"><span class="linenos"> 227</span></a>
+</span><span id="L-228"><a href="#L-228"><span class="linenos"> 228</span></a>
+</span><span id="L-229"><a href="#L-229"><span class="linenos"> 229</span></a><span class="k">def</span><span class="w"> </span><span class="nf">init</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-230"><a href="#L-230"><span class="linenos"> 230</span></a><span class="w"> </span><span class="sd">"""Initial phenotype directory as git repo with standard structure"""</span>
+</span><span id="L-231"><a href="#L-231"><span class="linenos"> 231</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Initialising Phenotype in directory: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-232"><a href="#L-232"><span class="linenos"> 232</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-233"><a href="#L-233"><span class="linenos"> 233</span></a>
+</span><span id="L-234"><a href="#L-234"><span class="linenos"> 234</span></a> <span class="c1"># check if directory already exists and ask user if they want to recreate it</span>
+</span><span id="L-235"><a href="#L-235"><span class="linenos"> 235</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="L-236"><a href="#L-236"><span class="linenos"> 236</span></a> <span class="n">phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">and</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span>
+</span><span id="L-237"><a href="#L-237"><span class="linenos"> 237</span></a> <span class="p">):</span> <span class="c1"># Check if it exists and is a directory</span>
+</span><span id="L-238"><a href="#L-238"><span class="linenos"> 238</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="n">_check_delete_dir</span><span class="p">(</span>
+</span><span id="L-239"><a href="#L-239"><span class="linenos"> 239</span></a> <span class="n">phen_path</span><span class="p">,</span>
+</span><span id="L-240"><a href="#L-240"><span class="linenos"> 240</span></a> <span class="sa">f</span><span class="s2">"The phen directory already exists. Do you want to reinitialise? (yes/no): "</span><span class="p">,</span>
+</span><span id="L-241"><a href="#L-241"><span class="linenos"> 241</span></a> <span class="p">)</span>
+</span><span id="L-242"><a href="#L-242"><span class="linenos"> 242</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-243"><a href="#L-243"><span class="linenos"> 243</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-244"><a href="#L-244"><span class="linenos"> 244</span></a>
+</span><span id="L-245"><a href="#L-245"><span class="linenos"> 245</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">configure</span><span class="p">:</span>
+</span><span id="L-246"><a href="#L-246"><span class="linenos"> 246</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exiting, phenotype not initiatised"</span><span class="p">)</span>
+</span><span id="L-247"><a href="#L-247"><span class="linenos"> 247</span></a> <span class="k">return</span>
+</span><span id="L-248"><a href="#L-248"><span class="linenos"> 248</span></a>
+</span><span id="L-249"><a href="#L-249"><span class="linenos"> 249</span></a> <span class="c1"># Initialise repo from local or remote</span>
+</span><span id="L-250"><a href="#L-250"><span class="linenos"> 250</span></a> <span class="n">repo</span><span class="p">:</span> <span class="n">Repo</span>
+</span><span id="L-251"><a href="#L-251"><span class="linenos"> 251</span></a>
+</span><span id="L-252"><a href="#L-252"><span class="linenos"> 252</span></a> <span class="c1"># if remote then clone the repo otherwise init a local repo</span>
+</span><span id="L-253"><a href="#L-253"><span class="linenos"> 253</span></a> <span class="k">if</span> <span class="n">remote_url</span> <span class="o">!=</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="L-254"><a href="#L-254"><span class="linenos"> 254</span></a> <span class="c1"># add PAT token to the URL</span>
+</span><span id="L-255"><a href="#L-255"><span class="linenos"> 255</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">remote_url</span><span class="p">)</span>
+</span><span id="L-256"><a href="#L-256"><span class="linenos"> 256</span></a>
+</span><span id="L-257"><a href="#L-257"><span class="linenos"> 257</span></a> <span class="c1"># clone the repo</span>
+</span><span id="L-258"><a href="#L-258"><span class="linenos"> 258</span></a> <span class="n">git_cmd</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">cmd</span><span class="o">.</span><span class="n">Git</span><span class="p">()</span>
+</span><span id="L-259"><a href="#L-259"><span class="linenos"> 259</span></a> <span class="n">git_cmd</span><span class="o">.</span><span class="n">clone</span><span class="p">(</span><span class="n">git_url</span><span class="p">,</span> <span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-260"><a href="#L-260"><span class="linenos"> 260</span></a>
+</span><span id="L-261"><a href="#L-261"><span class="linenos"> 261</span></a> <span class="c1"># open repo</span>
+</span><span id="L-262"><a href="#L-262"><span class="linenos"> 262</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">Repo</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-263"><a href="#L-263"><span class="linenos"> 263</span></a> <span class="c1"># check if there are any commits (new repo has no commits)</span>
+</span><span id="L-264"><a href="#L-264"><span class="linenos"> 264</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="L-265"><a href="#L-265"><span class="linenos"> 265</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">repo</span><span class="o">.</span><span class="n">branches</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">repo</span><span class="o">.</span><span class="n">head</span><span class="o">.</span><span class="n">is_detached</span>
+</span><span id="L-266"><a href="#L-266"><span class="linenos"> 266</span></a> <span class="p">):</span> <span class="c1"># Handle detached HEAD (e.g., after init)</span>
+</span><span id="L-267"><a href="#L-267"><span class="linenos"> 267</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"The phen repository has no commits yet."</span><span class="p">)</span>
+</span><span id="L-268"><a href="#L-268"><span class="linenos"> 268</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="mi">0</span>
+</span><span id="L-269"><a href="#L-269"><span class="linenos"> 269</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-270"><a href="#L-270"><span class="linenos"> 270</span></a> <span class="c1"># Get the total number of commits in the default branch</span>
+</span><span id="L-271"><a href="#L-271"><span class="linenos"> 271</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">iter_commits</span><span class="p">())</span>
+</span><span id="L-272"><a href="#L-272"><span class="linenos"> 272</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Repo has previous commits: </span><span class="si">{</span><span class="n">commit_count</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-273"><a href="#L-273"><span class="linenos"> 273</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-274"><a href="#L-274"><span class="linenos"> 274</span></a> <span class="c1"># local repo, create the directories and init</span>
+</span><span id="L-275"><a href="#L-275"><span class="linenos"> 275</span></a> <span class="n">phen_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-276"><a href="#L-276"><span class="linenos"> 276</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phen directory '</span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2">' has been created."</span><span class="p">)</span>
+</span><span id="L-277"><a href="#L-277"><span class="linenos"> 277</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">init</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-278"><a href="#L-278"><span class="linenos"> 278</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="mi">0</span>
+</span><span id="L-279"><a href="#L-279"><span class="linenos"> 279</span></a>
+</span><span id="L-280"><a href="#L-280"><span class="linenos"> 280</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()</span>
+</span><span id="L-281"><a href="#L-281"><span class="linenos"> 281</span></a> <span class="c1"># initialise empty repos</span>
+</span><span id="L-282"><a href="#L-282"><span class="linenos"> 282</span></a> <span class="k">if</span> <span class="n">commit_count</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-283"><a href="#L-283"><span class="linenos"> 283</span></a> <span class="c1"># create initial commit</span>
+</span><span id="L-284"><a href="#L-284"><span class="linenos"> 284</span></a> <span class="n">initial_file_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="s2">"README.md"</span>
+</span><span id="L-285"><a href="#L-285"><span class="linenos"> 285</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">initial_file_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-286"><a href="#L-286"><span class="linenos"> 286</span></a> <span class="n">file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span>
+</span><span id="L-287"><a href="#L-287"><span class="linenos"> 287</span></a> <span class="s2">"# Initial commit</span><span class="se">\n</span><span class="s2">This is the first commit in the phen repository.</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-288"><a href="#L-288"><span class="linenos"> 288</span></a> <span class="p">)</span>
+</span><span id="L-289"><a href="#L-289"><span class="linenos"> 289</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">add</span><span class="p">([</span><span class="n">initial_file_path</span><span class="p">])</span>
+</span><span id="L-290"><a href="#L-290"><span class="linenos"> 290</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">commit</span><span class="p">(</span><span class="s2">"Initial commit"</span><span class="p">)</span>
+</span><span id="L-291"><a href="#L-291"><span class="linenos"> 291</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="mi">1</span>
+</span><span id="L-292"><a href="#L-292"><span class="linenos"> 292</span></a>
+</span><span id="L-293"><a href="#L-293"><span class="linenos"> 293</span></a> <span class="c1"># Checkout the phens default branch, creating it if it does not exist</span>
+</span><span id="L-294"><a href="#L-294"><span class="linenos"> 294</span></a> <span class="k">if</span> <span class="n">DEFAULT_GIT_BRANCH</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">branches</span><span class="p">:</span>
+</span><span id="L-295"><a href="#L-295"><span class="linenos"> 295</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">heads</span><span class="p">[</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">]</span>
+</span><span id="L-296"><a href="#L-296"><span class="linenos"> 296</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="L-297"><a href="#L-297"><span class="linenos"> 297</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-298"><a href="#L-298"><span class="linenos"> 298</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">create_head</span><span class="p">(</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">)</span>
+</span><span id="L-299"><a href="#L-299"><span class="linenos"> 299</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="L-300"><a href="#L-300"><span class="linenos"> 300</span></a>
+</span><span id="L-301"><a href="#L-301"><span class="linenos"> 301</span></a> <span class="c1"># if the phen path does not contain the config file then initialise the phen type</span>
+</span><span id="L-302"><a href="#L-302"><span class="linenos"> 302</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-303"><a href="#L-303"><span class="linenos"> 303</span></a> <span class="k">if</span> <span class="n">config_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-304"><a href="#L-304"><span class="linenos"> 304</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype configuration files already exist"</span><span class="p">)</span>
+</span><span id="L-305"><a href="#L-305"><span class="linenos"> 305</span></a> <span class="k">return</span>
+</span><span id="L-306"><a href="#L-306"><span class="linenos"> 306</span></a>
+</span><span id="L-307"><a href="#L-307"><span class="linenos"> 307</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Creating phen directory structure and config files"</span><span class="p">)</span>
+</span><span id="L-308"><a href="#L-308"><span class="linenos"> 308</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">DEFAULT_PHEN_DIR_LIST</span><span class="p">:</span>
+</span><span id="L-309"><a href="#L-309"><span class="linenos"> 309</span></a> <span class="n">_create_empty_git_dir</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">d</span><span class="p">)</span>
+</span><span id="L-310"><a href="#L-310"><span class="linenos"> 310</span></a>
+</span><span id="L-311"><a href="#L-311"><span class="linenos"> 311</span></a> <span class="c1"># create empty phen config file</span>
+</span><span id="L-312"><a href="#L-312"><span class="linenos"> 312</span></a> <span class="n">config</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-313"><a href="#L-313"><span class="linenos"> 313</span></a> <span class="s2">"phenotype"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-314"><a href="#L-314"><span class="linenos"> 314</span></a> <span class="s2">"version"</span><span class="p">:</span> <span class="s2">"0.0.0"</span><span class="p">,</span>
+</span><span id="L-315"><a href="#L-315"><span class="linenos"> 315</span></a> <span class="s2">"omop"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-316"><a href="#L-316"><span class="linenos"> 316</span></a> <span class="s2">"vocabulary_id"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="L-317"><a href="#L-317"><span class="linenos"> 317</span></a> <span class="s2">"vocabulary_name"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="L-318"><a href="#L-318"><span class="linenos"> 318</span></a> <span class="s2">"vocabulary_reference"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="L-319"><a href="#L-319"><span class="linenos"> 319</span></a> <span class="p">},</span>
+</span><span id="L-320"><a href="#L-320"><span class="linenos"> 320</span></a> <span class="s2">"translate"</span><span class="p">:</span> <span class="p">[],</span>
+</span><span id="L-321"><a href="#L-321"><span class="linenos"> 321</span></a> <span class="s2">"concept_sets"</span><span class="p">:</span> <span class="p">[],</span>
+</span><span id="L-322"><a href="#L-322"><span class="linenos"> 322</span></a> <span class="p">}</span>
+</span><span id="L-323"><a href="#L-323"><span class="linenos"> 323</span></a> <span class="p">}</span>
+</span><span id="L-324"><a href="#L-324"><span class="linenos"> 324</span></a>
+</span><span id="L-325"><a href="#L-325"><span class="linenos"> 325</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-326"><a href="#L-326"><span class="linenos"> 326</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="L-327"><a href="#L-327"><span class="linenos"> 327</span></a> <span class="n">config</span><span class="p">,</span>
+</span><span id="L-328"><a href="#L-328"><span class="linenos"> 328</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="L-329"><a href="#L-329"><span class="linenos"> 329</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="L-330"><a href="#L-330"><span class="linenos"> 330</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-331"><a href="#L-331"><span class="linenos"> 331</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-332"><a href="#L-332"><span class="linenos"> 332</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="L-333"><a href="#L-333"><span class="linenos"> 333</span></a> <span class="p">)</span>
+</span><span id="L-334"><a href="#L-334"><span class="linenos"> 334</span></a>
+</span><span id="L-335"><a href="#L-335"><span class="linenos"> 335</span></a> <span class="c1"># add git ignore</span>
+</span><span id="L-336"><a href="#L-336"><span class="linenos"> 336</span></a> <span class="n">ignore_content</span> <span class="o">=</span> <span class="s2">"""# Ignore SQLite database files</span>
+</span><span id="L-337"><a href="#L-337"><span class="linenos"> 337</span></a><span class="s2">*.db</span>
+</span><span id="L-338"><a href="#L-338"><span class="linenos"> 338</span></a><span class="s2">*.sqlite3</span>
+</span><span id="L-339"><a href="#L-339"><span class="linenos"> 339</span></a><span class="s2"> </span>
+</span><span id="L-340"><a href="#L-340"><span class="linenos"> 340</span></a><span class="s2"># Ignore SQLite journal and metadata files</span>
+</span><span id="L-341"><a href="#L-341"><span class="linenos"> 341</span></a><span class="s2">*.db-journal</span>
+</span><span id="L-342"><a href="#L-342"><span class="linenos"> 342</span></a><span class="s2">*.sqlite3-journal</span>
+</span><span id="L-343"><a href="#L-343"><span class="linenos"> 343</span></a>
+</span><span id="L-344"><a href="#L-344"><span class="linenos"> 344</span></a><span class="s2"># python</span>
+</span><span id="L-345"><a href="#L-345"><span class="linenos"> 345</span></a><span class="s2">.ipynb_checkpoints</span>
+</span><span id="L-346"><a href="#L-346"><span class="linenos"> 346</span></a><span class="s2"> """</span>
+</span><span id="L-347"><a href="#L-347"><span class="linenos"> 347</span></a> <span class="n">ignore_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="s2">".gitignore"</span>
+</span><span id="L-348"><a href="#L-348"><span class="linenos"> 348</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">ignore_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-349"><a href="#L-349"><span class="linenos"> 349</span></a> <span class="n">file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">ignore_content</span><span class="p">)</span>
+</span><span id="L-350"><a href="#L-350"><span class="linenos"> 350</span></a>
+</span><span id="L-351"><a href="#L-351"><span class="linenos"> 351</span></a> <span class="c1"># add to git repo and commit</span>
+</span><span id="L-352"><a href="#L-352"><span class="linenos"> 352</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">DEFAULT_PHEN_DIR_LIST</span><span class="p">:</span>
+</span><span id="L-353"><a href="#L-353"><span class="linenos"> 353</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">d</span><span class="p">)</span>
+</span><span id="L-354"><a href="#L-354"><span class="linenos"> 354</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">all</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-355"><a href="#L-355"><span class="linenos"> 355</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">commit</span><span class="p">(</span><span class="s2">"initialised the phen git repo."</span><span class="p">)</span>
+</span><span id="L-356"><a href="#L-356"><span class="linenos"> 356</span></a>
+</span><span id="L-357"><a href="#L-357"><span class="linenos"> 357</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype initialised successfully"</span><span class="p">)</span>
+</span><span id="L-358"><a href="#L-358"><span class="linenos"> 358</span></a>
+</span><span id="L-359"><a href="#L-359"><span class="linenos"> 359</span></a>
+</span><span id="L-360"><a href="#L-360"><span class="linenos"> 360</span></a><span class="k">def</span><span class="w"> </span><span class="nf">fork</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">upstream_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">upstream_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">new_origin_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-361"><a href="#L-361"><span class="linenos"> 361</span></a><span class="w"> </span><span class="sd">"""Forks an upstream phenotype in a remote repo at a specific version to a local director, and optionally sets to a new remote origin"</span>
+</span><span id="L-362"><a href="#L-362"><span class="linenos"> 362</span></a>
+</span><span id="L-363"><a href="#L-363"><span class="linenos"> 363</span></a><span class="sd"> Args:</span>
+</span><span id="L-364"><a href="#L-364"><span class="linenos"> 364</span></a><span class="sd"> phen_dir (str): local directory path where the upstream repo is to be cloned</span>
+</span><span id="L-365"><a href="#L-365"><span class="linenos"> 365</span></a><span class="sd"> upstream_url (str): url to the upstream repo</span>
+</span><span id="L-366"><a href="#L-366"><span class="linenos"> 366</span></a><span class="sd"> upstream_version (str): version in the upstream repo to clone</span>
+</span><span id="L-367"><a href="#L-367"><span class="linenos"> 367</span></a><span class="sd"> new_origin_url (str, optional): url of the remote repo to set as the new origin. Defaults to None.</span>
+</span><span id="L-368"><a href="#L-368"><span class="linenos"> 368</span></a>
+</span><span id="L-369"><a href="#L-369"><span class="linenos"> 369</span></a><span class="sd"> Raises:</span>
+</span><span id="L-370"><a href="#L-370"><span class="linenos"> 370</span></a><span class="sd"> ValueError: if the specified version is not in the upstream repo</span>
+</span><span id="L-371"><a href="#L-371"><span class="linenos"> 371</span></a><span class="sd"> ValueError: if the upstream repo is not a valid phenotype repo</span>
+</span><span id="L-372"><a href="#L-372"><span class="linenos"> 372</span></a><span class="sd"> ValueError: if there's any other problems with Git</span>
+</span><span id="L-373"><a href="#L-373"><span class="linenos"> 373</span></a><span class="sd"> """</span>
+</span><span id="L-374"><a href="#L-374"><span class="linenos"> 374</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
+</span><span id="L-375"><a href="#L-375"><span class="linenos"> 375</span></a> <span class="sa">f</span><span class="s2">"Forking upstream repo </span><span class="si">{</span><span class="n">upstream_url</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">upstream_version</span><span class="si">}</span><span class="s2"> into directory: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-376"><a href="#L-376"><span class="linenos"> 376</span></a> <span class="p">)</span>
+</span><span id="L-377"><a href="#L-377"><span class="linenos"> 377</span></a>
+</span><span id="L-378"><a href="#L-378"><span class="linenos"> 378</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-379"><a href="#L-379"><span class="linenos"> 379</span></a> <span class="c1"># check if directory already exists and ask user if they want to recreate it</span>
+</span><span id="L-380"><a href="#L-380"><span class="linenos"> 380</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="L-381"><a href="#L-381"><span class="linenos"> 381</span></a> <span class="n">phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">and</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span>
+</span><span id="L-382"><a href="#L-382"><span class="linenos"> 382</span></a> <span class="p">):</span> <span class="c1"># Check if it exists and is a directory</span>
+</span><span id="L-383"><a href="#L-383"><span class="linenos"> 383</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="n">_check_delete_dir</span><span class="p">(</span>
+</span><span id="L-384"><a href="#L-384"><span class="linenos"> 384</span></a> <span class="n">phen_path</span><span class="p">,</span>
+</span><span id="L-385"><a href="#L-385"><span class="linenos"> 385</span></a> <span class="sa">f</span><span class="s2">"The phen directory already exists. Do you want to reinitialise? (yes/no): "</span><span class="p">,</span>
+</span><span id="L-386"><a href="#L-386"><span class="linenos"> 386</span></a> <span class="p">)</span>
+</span><span id="L-387"><a href="#L-387"><span class="linenos"> 387</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-388"><a href="#L-388"><span class="linenos"> 388</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-389"><a href="#L-389"><span class="linenos"> 389</span></a>
+</span><span id="L-390"><a href="#L-390"><span class="linenos"> 390</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">configure</span><span class="p">:</span>
+</span><span id="L-391"><a href="#L-391"><span class="linenos"> 391</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exiting, phenotype not initiatised"</span><span class="p">)</span>
+</span><span id="L-392"><a href="#L-392"><span class="linenos"> 392</span></a> <span class="k">return</span>
+</span><span id="L-393"><a href="#L-393"><span class="linenos"> 393</span></a>
+</span><span id="L-394"><a href="#L-394"><span class="linenos"> 394</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-395"><a href="#L-395"><span class="linenos"> 395</span></a> <span class="c1"># Clone repo</span>
+</span><span id="L-396"><a href="#L-396"><span class="linenos"> 396</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">upstream_url</span><span class="p">)</span>
+</span><span id="L-397"><a href="#L-397"><span class="linenos"> 397</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">git_url</span><span class="p">,</span> <span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-398"><a href="#L-398"><span class="linenos"> 398</span></a>
+</span><span id="L-399"><a href="#L-399"><span class="linenos"> 399</span></a> <span class="c1"># Fetch all branches and tags</span>
+</span><span id="L-400"><a href="#L-400"><span class="linenos"> 400</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="o">.</span><span class="n">origin</span><span class="o">.</span><span class="n">fetch</span><span class="p">()</span>
+</span><span id="L-401"><a href="#L-401"><span class="linenos"> 401</span></a>
+</span><span id="L-402"><a href="#L-402"><span class="linenos"> 402</span></a> <span class="c1"># Check if the version exists</span>
+</span><span id="L-403"><a href="#L-403"><span class="linenos"> 403</span></a> <span class="n">available_refs</span> <span class="o">=</span> <span class="p">[</span><span class="n">ref</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"/"</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">ref</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">references</span><span class="p">]</span>
+</span><span id="L-404"><a href="#L-404"><span class="linenos"> 404</span></a> <span class="k">if</span> <span class="n">upstream_version</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">available_refs</span><span class="p">:</span>
+</span><span id="L-405"><a href="#L-405"><span class="linenos"> 405</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-406"><a href="#L-406"><span class="linenos"> 406</span></a> <span class="sa">f</span><span class="s2">"Version '</span><span class="si">{</span><span class="n">upstream_version</span><span class="si">}</span><span class="s2">' not found in the repository: </span><span class="si">{</span><span class="n">upstream_url</span><span class="si">}</span><span class="s2">."</span>
+</span><span id="L-407"><a href="#L-407"><span class="linenos"> 407</span></a> <span class="p">)</span>
+</span><span id="L-408"><a href="#L-408"><span class="linenos"> 408</span></a>
+</span><span id="L-409"><a href="#L-409"><span class="linenos"> 409</span></a> <span class="c1"># Checkout the specified version</span>
+</span><span id="L-410"><a href="#L-410"><span class="linenos"> 410</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">upstream_version</span><span class="p">)</span>
+</span><span id="L-411"><a href="#L-411"><span class="linenos"> 411</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">heads</span><span class="p">[</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">]</span>
+</span><span id="L-412"><a href="#L-412"><span class="linenos"> 412</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="L-413"><a href="#L-413"><span class="linenos"> 413</span></a>
+</span><span id="L-414"><a href="#L-414"><span class="linenos"> 414</span></a> <span class="c1"># Check if 'config.yml' exists in the root directory</span>
+</span><span id="L-415"><a href="#L-415"><span class="linenos"> 415</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="s2">"config.yml"</span>
+</span><span id="L-416"><a href="#L-416"><span class="linenos"> 416</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">isfile</span><span class="p">(</span><span class="n">config_path</span><span class="p">):</span>
+</span><span id="L-417"><a href="#L-417"><span class="linenos"> 417</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-418"><a href="#L-418"><span class="linenos"> 418</span></a> <span class="sa">f</span><span class="s2">"The forked repository is not a valid ACMC repo because 'config.yml' is missing in the root directory."</span>
+</span><span id="L-419"><a href="#L-419"><span class="linenos"> 419</span></a> <span class="p">)</span>
+</span><span id="L-420"><a href="#L-420"><span class="linenos"> 420</span></a>
+</span><span id="L-421"><a href="#L-421"><span class="linenos"> 421</span></a> <span class="c1"># Validate the phenotype is compatible with the acmc tool</span>
+</span><span id="L-422"><a href="#L-422"><span class="linenos"> 422</span></a> <span class="n">validate</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()))</span>
+</span><span id="L-423"><a href="#L-423"><span class="linenos"> 423</span></a>
+</span><span id="L-424"><a href="#L-424"><span class="linenos"> 424</span></a> <span class="c1"># Delete each tag locally</span>
+</span><span id="L-425"><a href="#L-425"><span class="linenos"> 425</span></a> <span class="n">tags</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">tags</span>
+</span><span id="L-426"><a href="#L-426"><span class="linenos"> 426</span></a> <span class="k">for</span> <span class="n">tag</span> <span class="ow">in</span> <span class="n">tags</span><span class="p">:</span>
+</span><span id="L-427"><a href="#L-427"><span class="linenos"> 427</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">delete_tag</span><span class="p">(</span><span class="n">tag</span><span class="p">)</span>
+</span><span id="L-428"><a href="#L-428"><span class="linenos"> 428</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Deleted tags from forked repo: </span><span class="si">{</span><span class="n">tag</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-429"><a href="#L-429"><span class="linenos"> 429</span></a>
+</span><span id="L-430"><a href="#L-430"><span class="linenos"> 430</span></a> <span class="c1"># Add upstream remote</span>
+</span><span id="L-431"><a href="#L-431"><span class="linenos"> 431</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_remote</span><span class="p">(</span><span class="s2">"upstream"</span><span class="p">,</span> <span class="n">upstream_url</span><span class="p">)</span>
+</span><span id="L-432"><a href="#L-432"><span class="linenos"> 432</span></a> <span class="n">remote</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">[</span><span class="s2">"origin"</span><span class="p">]</span>
+</span><span id="L-433"><a href="#L-433"><span class="linenos"> 433</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">delete_remote</span><span class="p">(</span><span class="n">remote</span><span class="p">)</span> <span class="c1"># Remove existing origin</span>
+</span><span id="L-434"><a href="#L-434"><span class="linenos"> 434</span></a>
+</span><span id="L-435"><a href="#L-435"><span class="linenos"> 435</span></a> <span class="c1"># Optionally set a new origin remote</span>
+</span><span id="L-436"><a href="#L-436"><span class="linenos"> 436</span></a> <span class="k">if</span> <span class="n">new_origin_url</span><span class="p">:</span>
+</span><span id="L-437"><a href="#L-437"><span class="linenos"> 437</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">new_origin_url</span><span class="p">)</span>
+</span><span id="L-438"><a href="#L-438"><span class="linenos"> 438</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_remote</span><span class="p">(</span><span class="s2">"origin"</span><span class="p">,</span> <span class="n">git_url</span><span class="p">)</span>
+</span><span id="L-439"><a href="#L-439"><span class="linenos"> 439</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="s2">"--set-upstream"</span><span class="p">,</span> <span class="s2">"origin"</span><span class="p">,</span> <span class="s2">"main"</span><span class="p">)</span>
+</span><span id="L-440"><a href="#L-440"><span class="linenos"> 440</span></a>
+</span><span id="L-441"><a href="#L-441"><span class="linenos"> 441</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Repository forked successfully at </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-442"><a href="#L-442"><span class="linenos"> 442</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Upstream set to </span><span class="si">{</span><span class="n">upstream_url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-443"><a href="#L-443"><span class="linenos"> 443</span></a> <span class="k">if</span> <span class="n">new_origin_url</span><span class="p">:</span>
+</span><span id="L-444"><a href="#L-444"><span class="linenos"> 444</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Origin set to </span><span class="si">{</span><span class="n">new_origin_url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-445"><a href="#L-445"><span class="linenos"> 445</span></a>
+</span><span id="L-446"><a href="#L-446"><span class="linenos"> 446</span></a> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="L-447"><a href="#L-447"><span class="linenos"> 447</span></a> <span class="k">if</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-448"><a href="#L-448"><span class="linenos"> 448</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-449"><a href="#L-449"><span class="linenos"> 449</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error occurred during repository fork: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-450"><a href="#L-450"><span class="linenos"> 450</span></a>
+</span><span id="L-451"><a href="#L-451"><span class="linenos"> 451</span></a>
+</span><span id="L-452"><a href="#L-452"><span class="linenos"> 452</span></a><span class="k">def</span><span class="w"> </span><span class="nf">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-453"><a href="#L-453"><span class="linenos"> 453</span></a><span class="w"> </span><span class="sd">"""Validates the phenotype directory is a git repo with standard structure"""</span>
+</span><span id="L-454"><a href="#L-454"><span class="linenos"> 454</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Validating phenotype: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-455"><a href="#L-455"><span class="linenos"> 455</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-456"><a href="#L-456"><span class="linenos"> 456</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="L-457"><a href="#L-457"><span class="linenos"> 457</span></a> <span class="k">raise</span> <span class="ne">NotADirectoryError</span><span class="p">(</span>
+</span><span id="L-458"><a href="#L-458"><span class="linenos"> 458</span></a> <span class="sa">f</span><span class="s2">"Error: '</span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">' is not a directory"</span>
+</span><span id="L-459"><a href="#L-459"><span class="linenos"> 459</span></a> <span class="p">)</span>
+</span><span id="L-460"><a href="#L-460"><span class="linenos"> 460</span></a>
+</span><span id="L-461"><a href="#L-461"><span class="linenos"> 461</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-462"><a href="#L-462"><span class="linenos"> 462</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">config_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="L-463"><a href="#L-463"><span class="linenos"> 463</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="L-464"><a href="#L-464"><span class="linenos"> 464</span></a> <span class="sa">f</span><span class="s2">"Error: phen configuration file '</span><span class="si">{</span><span class="n">config_path</span><span class="si">}</span><span class="s2">' does not exist."</span>
+</span><span id="L-465"><a href="#L-465"><span class="linenos"> 465</span></a> <span class="p">)</span>
+</span><span id="L-466"><a href="#L-466"><span class="linenos"> 466</span></a>
+</span><span id="L-467"><a href="#L-467"><span class="linenos"> 467</span></a> <span class="n">concepts_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONCEPTS_DIR</span>
+</span><span id="L-468"><a href="#L-468"><span class="linenos"> 468</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">concepts_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="L-469"><a href="#L-469"><span class="linenos"> 469</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="L-470"><a href="#L-470"><span class="linenos"> 470</span></a> <span class="sa">f</span><span class="s2">"Error: source concepts directory </span><span class="si">{</span><span class="n">concepts_path</span><span class="si">}</span><span class="s2"> does not exist."</span>
+</span><span id="L-471"><a href="#L-471"><span class="linenos"> 471</span></a> <span class="p">)</span>
+</span><span id="L-472"><a href="#L-472"><span class="linenos"> 472</span></a>
+</span><span id="L-473"><a href="#L-473"><span class="linenos"> 473</span></a> <span class="c1"># Calidate the directory is a git repo</span>
+</span><span id="L-474"><a href="#L-474"><span class="linenos"> 474</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-475"><a href="#L-475"><span class="linenos"> 475</span></a> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-476"><a href="#L-476"><span class="linenos"> 476</span></a> <span class="k">except</span> <span class="p">(</span><span class="n">git</span><span class="o">.</span><span class="n">exc</span><span class="o">.</span><span class="n">InvalidGitRepositoryError</span><span class="p">,</span> <span class="n">git</span><span class="o">.</span><span class="n">exc</span><span class="o">.</span><span class="n">NoSuchPathError</span><span class="p">):</span>
+</span><span id="L-477"><a href="#L-477"><span class="linenos"> 477</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phen directory </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> is not a git repo"</span><span class="p">)</span>
+</span><span id="L-478"><a href="#L-478"><span class="linenos"> 478</span></a>
+</span><span id="L-479"><a href="#L-479"><span class="linenos"> 479</span></a> <span class="c1"># Load configuration File</span>
+</span><span id="L-480"><a href="#L-480"><span class="linenos"> 480</span></a> <span class="k">if</span> <span class="n">config_path</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">".yml"</span><span class="p">:</span>
+</span><span id="L-481"><a href="#L-481"><span class="linenos"> 481</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-482"><a href="#L-482"><span class="linenos"> 482</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-483"><a href="#L-483"><span class="linenos"> 483</span></a> <span class="n">phenotype</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-484"><a href="#L-484"><span class="linenos"> 484</span></a>
+</span><span id="L-485"><a href="#L-485"><span class="linenos"> 485</span></a> <span class="n">validator</span> <span class="o">=</span> <span class="n">Validator</span><span class="p">(</span><span class="n">CONFIG_SCHEMA</span><span class="p">)</span>
+</span><span id="L-486"><a href="#L-486"><span class="linenos"> 486</span></a> <span class="k">if</span> <span class="n">validator</span><span class="o">.</span><span class="n">validate</span><span class="p">(</span><span class="n">phenotype</span><span class="p">):</span>
+</span><span id="L-487"><a href="#L-487"><span class="linenos"> 487</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"YAML structure is valid."</span><span class="p">)</span>
+</span><span id="L-488"><a href="#L-488"><span class="linenos"> 488</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-489"><a href="#L-489"><span class="linenos"> 489</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">"YAML structure validation failed: </span><span class="si">{</span><span class="n">validator</span><span class="o">.</span><span class="n">errors</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-490"><a href="#L-490"><span class="linenos"> 490</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="sa">f</span><span class="s2">"YAML structure validation failed: </span><span class="si">{</span><span class="n">validator</span><span class="o">.</span><span class="n">errors</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-491"><a href="#L-491"><span class="linenos"> 491</span></a> <span class="k">except</span> <span class="n">yaml</span><span class="o">.</span><span class="n">YAMLError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="L-492"><a href="#L-492"><span class="linenos"> 492</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">"YAML syntax error: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-493"><a href="#L-493"><span class="linenos"> 493</span></a> <span class="k">raise</span> <span class="n">e</span>
+</span><span id="L-494"><a href="#L-494"><span class="linenos"> 494</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-495"><a href="#L-495"><span class="linenos"> 495</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span>
+</span><span id="L-496"><a href="#L-496"><span class="linenos"> 496</span></a> <span class="sa">f</span><span class="s2">"Unsupported configuration filetype: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">config_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-497"><a href="#L-497"><span class="linenos"> 497</span></a> <span class="p">)</span>
+</span><span id="L-498"><a href="#L-498"><span class="linenos"> 498</span></a>
+</span><span id="L-499"><a href="#L-499"><span class="linenos"> 499</span></a> <span class="c1"># initiatise</span>
+</span><span id="L-500"><a href="#L-500"><span class="linenos"> 500</span></a> <span class="n">validation_errors</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-501"><a href="#L-501"><span class="linenos"> 501</span></a> <span class="n">phenotype</span> <span class="o">=</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">]</span>
+</span><span id="L-502"><a href="#L-502"><span class="linenos"> 502</span></a> <span class="n">code_types</span> <span class="o">=</span> <span class="n">parse</span><span class="o">.</span><span class="n">CodeTypeParser</span><span class="p">()</span><span class="o">.</span><span class="n">code_types</span>
+</span><span id="L-503"><a href="#L-503"><span class="linenos"> 503</span></a>
+</span><span id="L-504"><a href="#L-504"><span class="linenos"> 504</span></a> <span class="c1"># check the version number is of the format vn.n.n</span>
+</span><span id="L-505"><a href="#L-505"><span class="linenos"> 505</span></a> <span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"(\d+\.\d+\.\d+)"</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"version"</span><span class="p">])</span>
+</span><span id="L-506"><a href="#L-506"><span class="linenos"> 506</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">match</span><span class="p">:</span>
+</span><span id="L-507"><a href="#L-507"><span class="linenos"> 507</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="L-508"><a href="#L-508"><span class="linenos"> 508</span></a> <span class="sa">f</span><span class="s2">"Invalid version format in configuration file: </span><span class="si">{</span><span class="n">phenotype</span><span class="p">[</span><span class="s1">'version'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-509"><a href="#L-509"><span class="linenos"> 509</span></a> <span class="p">)</span>
+</span><span id="L-510"><a href="#L-510"><span class="linenos"> 510</span></a>
+</span><span id="L-511"><a href="#L-511"><span class="linenos"> 511</span></a> <span class="c1"># create a list of all the concept set names defined in the concept set configuration</span>
+</span><span id="L-512"><a href="#L-512"><span class="linenos"> 512</span></a> <span class="n">concept_set_names</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-513"><a href="#L-513"><span class="linenos"> 513</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"concept_sets"</span><span class="p">]:</span>
+</span><span id="L-514"><a href="#L-514"><span class="linenos"> 514</span></a> <span class="k">if</span> <span class="n">item</span><span class="p">[</span><span class="s2">"name"</span><span class="p">]</span> <span class="ow">in</span> <span class="n">concept_set_names</span><span class="p">:</span>
+</span><span id="L-515"><a href="#L-515"><span class="linenos"> 515</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="L-516"><a href="#L-516"><span class="linenos"> 516</span></a> <span class="sa">f</span><span class="s2">"Duplicate concept set defined in concept sets </span><span class="si">{</span><span class="n">item</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span><span class="w"> </span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-517"><a href="#L-517"><span class="linenos"> 517</span></a> <span class="p">)</span>
+</span><span id="L-518"><a href="#L-518"><span class="linenos"> 518</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-519"><a href="#L-519"><span class="linenos"> 519</span></a> <span class="n">concept_set_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">item</span><span class="p">[</span><span class="s2">"name"</span><span class="p">])</span>
+</span><span id="L-520"><a href="#L-520"><span class="linenos"> 520</span></a>
+</span><span id="L-521"><a href="#L-521"><span class="linenos"> 521</span></a> <span class="c1"># check codes definition</span>
+</span><span id="L-522"><a href="#L-522"><span class="linenos"> 522</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"concept_sets"</span><span class="p">]:</span>
+</span><span id="L-523"><a href="#L-523"><span class="linenos"> 523</span></a> <span class="c1"># check concepte code file exists</span>
+</span><span id="L-524"><a href="#L-524"><span class="linenos"> 524</span></a> <span class="n">concept_code_file_path</span> <span class="o">=</span> <span class="n">concepts_path</span> <span class="o">/</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"path"</span><span class="p">]</span>
+</span><span id="L-525"><a href="#L-525"><span class="linenos"> 525</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-526"><a href="#L-526"><span class="linenos"> 526</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="L-527"><a href="#L-527"><span class="linenos"> 527</span></a> <span class="sa">f</span><span class="s2">"Coding file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> does not exist"</span>
+</span><span id="L-528"><a href="#L-528"><span class="linenos"> 528</span></a> <span class="p">)</span>
+</span><span id="L-529"><a href="#L-529"><span class="linenos"> 529</span></a>
+</span><span id="L-530"><a href="#L-530"><span class="linenos"> 530</span></a> <span class="c1"># check concepte code file is not empty</span>
+</span><span id="L-531"><a href="#L-531"><span class="linenos"> 531</span></a> <span class="k">if</span> <span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">stat</span><span class="p">()</span><span class="o">.</span><span class="n">st_size</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-532"><a href="#L-532"><span class="linenos"> 532</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="L-533"><a href="#L-533"><span class="linenos"> 533</span></a> <span class="sa">f</span><span class="s2">"Coding file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> is an empty file"</span>
+</span><span id="L-534"><a href="#L-534"><span class="linenos"> 534</span></a> <span class="p">)</span>
+</span><span id="L-535"><a href="#L-535"><span class="linenos"> 535</span></a>
+</span><span id="L-536"><a href="#L-536"><span class="linenos"> 536</span></a> <span class="c1"># check code file type is supported</span>
+</span><span id="L-537"><a href="#L-537"><span class="linenos"> 537</span></a> <span class="k">if</span> <span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">suffix</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">CODE_FILE_TYPES</span><span class="p">:</span>
+</span><span id="L-538"><a href="#L-538"><span class="linenos"> 538</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-539"><a href="#L-539"><span class="linenos"> 539</span></a> <span class="sa">f</span><span class="s2">"Unsupported filetype </span><span class="si">{</span><span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">suffix</span><span class="si">}</span><span class="s2">, only support csv, xlsx, xls code file types"</span>
+</span><span id="L-540"><a href="#L-540"><span class="linenos"> 540</span></a> <span class="p">)</span>
+</span><span id="L-541"><a href="#L-541"><span class="linenos"> 541</span></a>
+</span><span id="L-542"><a href="#L-542"><span class="linenos"> 542</span></a> <span class="c1"># check columns specified are a supported medical coding type</span>
+</span><span id="L-543"><a href="#L-543"><span class="linenos"> 543</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"columns"</span><span class="p">]:</span>
+</span><span id="L-544"><a href="#L-544"><span class="linenos"> 544</span></a> <span class="k">if</span> <span class="n">column</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">code_types</span><span class="p">:</span>
+</span><span id="L-545"><a href="#L-545"><span class="linenos"> 545</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="L-546"><a href="#L-546"><span class="linenos"> 546</span></a> <span class="sa">f</span><span class="s2">"Column type </span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2"> for file </span><span class="si">{</span><span class="n">concept_code_file_path</span><span class="si">}</span><span class="s2"> is not supported"</span>
+</span><span id="L-547"><a href="#L-547"><span class="linenos"> 547</span></a> <span class="p">)</span>
+</span><span id="L-548"><a href="#L-548"><span class="linenos"> 548</span></a>
+</span><span id="L-549"><a href="#L-549"><span class="linenos"> 549</span></a> <span class="c1"># check the actions are supported</span>
+</span><span id="L-550"><a href="#L-550"><span class="linenos"> 550</span></a> <span class="k">if</span> <span class="s2">"actions"</span> <span class="ow">in</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">]:</span>
+</span><span id="L-551"><a href="#L-551"><span class="linenos"> 551</span></a> <span class="k">for</span> <span class="n">action</span> <span class="ow">in</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">]:</span>
+</span><span id="L-552"><a href="#L-552"><span class="linenos"> 552</span></a> <span class="k">if</span> <span class="n">action</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">COL_ACTIONS</span><span class="p">:</span>
+</span><span id="L-553"><a href="#L-553"><span class="linenos"> 553</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Action </span><span class="si">{</span><span class="n">action</span><span class="si">}</span><span class="s2"> is not supported"</span><span class="p">)</span>
+</span><span id="L-554"><a href="#L-554"><span class="linenos"> 554</span></a>
+</span><span id="L-555"><a href="#L-555"><span class="linenos"> 555</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">validation_errors</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-556"><a href="#L-556"><span class="linenos"> 556</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">validation_errors</span><span class="p">)</span>
+</span><span id="L-557"><a href="#L-557"><span class="linenos"> 557</span></a> <span class="k">raise</span> <span class="n">PhenValidationException</span><span class="p">(</span>
+</span><span id="L-558"><a href="#L-558"><span class="linenos"> 558</span></a> <span class="sa">f</span><span class="s2">"Configuration file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">config_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> failed validation"</span><span class="p">,</span>
+</span><span id="L-559"><a href="#L-559"><span class="linenos"> 559</span></a> <span class="n">validation_errors</span><span class="p">,</span>
+</span><span id="L-560"><a href="#L-560"><span class="linenos"> 560</span></a> <span class="p">)</span>
+</span><span id="L-561"><a href="#L-561"><span class="linenos"> 561</span></a>
+</span><span id="L-562"><a href="#L-562"><span class="linenos"> 562</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype validated successfully"</span><span class="p">)</span>
+</span><span id="L-563"><a href="#L-563"><span class="linenos"> 563</span></a>
+</span><span id="L-564"><a href="#L-564"><span class="linenos"> 564</span></a>
+</span><span id="L-565"><a href="#L-565"><span class="linenos"> 565</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_read_table_file</span><span class="p">(</span><span class="n">path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">excel_sheet</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">""</span><span class="p">):</span>
+</span><span id="L-566"><a href="#L-566"><span class="linenos"> 566</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-567"><a href="#L-567"><span class="linenos"> 567</span></a><span class="sd"> Load Code List File</span>
+</span><span id="L-568"><a href="#L-568"><span class="linenos"> 568</span></a><span class="sd"> """</span>
+</span><span id="L-569"><a href="#L-569"><span class="linenos"> 569</span></a>
+</span><span id="L-570"><a href="#L-570"><span class="linenos"> 570</span></a> <span class="n">path</span> <span class="o">=</span> <span class="n">path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()</span>
+</span><span id="L-571"><a href="#L-571"><span class="linenos"> 571</span></a> <span class="k">if</span> <span class="n">path</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">".csv"</span><span class="p">:</span>
+</span><span id="L-572"><a href="#L-572"><span class="linenos"> 572</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-573"><a href="#L-573"><span class="linenos"> 573</span></a> <span class="k">elif</span> <span class="n">path</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">".xlsx"</span> <span class="ow">or</span> <span class="n">path</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">".xls"</span><span class="p">:</span>
+</span><span id="L-574"><a href="#L-574"><span class="linenos"> 574</span></a> <span class="k">if</span> <span class="n">excel_sheet</span> <span class="o">!=</span> <span class="s2">""</span><span class="p">:</span>
+</span><span id="L-575"><a href="#L-575"><span class="linenos"> 575</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_excel</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">sheet_name</span><span class="o">=</span><span class="n">excel_sheet</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-576"><a href="#L-576"><span class="linenos"> 576</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-577"><a href="#L-577"><span class="linenos"> 577</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_excel</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-578"><a href="#L-578"><span class="linenos"> 578</span></a> <span class="k">elif</span> <span class="n">path</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">".dta"</span><span class="p">:</span>
+</span><span id="L-579"><a href="#L-579"><span class="linenos"> 579</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_stata</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>
+</span><span id="L-580"><a href="#L-580"><span class="linenos"> 580</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-581"><a href="#L-581"><span class="linenos"> 581</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-582"><a href="#L-582"><span class="linenos"> 582</span></a> <span class="sa">f</span><span class="s2">"Unsupported filetype </span><span class="si">{</span><span class="n">path</span><span class="o">.</span><span class="n">suffix</span><span class="si">}</span><span class="s2">, only support</span><span class="si">{</span><span class="n">CODE_FILE_TYPES</span><span class="si">}</span><span class="s2"> code file types"</span>
+</span><span id="L-583"><a href="#L-583"><span class="linenos"> 583</span></a> <span class="p">)</span>
+</span><span id="L-584"><a href="#L-584"><span class="linenos"> 584</span></a>
+</span><span id="L-585"><a href="#L-585"><span class="linenos"> 585</span></a> <span class="k">return</span> <span class="n">df</span>
+</span><span id="L-586"><a href="#L-586"><span class="linenos"> 586</span></a>
+</span><span id="L-587"><a href="#L-587"><span class="linenos"> 587</span></a>
+</span><span id="L-588"><a href="#L-588"><span class="linenos"> 588</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_process_actions</span><span class="p">(</span><span class="n">df</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">concept_set</span><span class="p">:</span> <span class="nb">dict</span><span class="p">)</span> <span class="o">-></span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">:</span>
+</span><span id="L-589"><a href="#L-589"><span class="linenos"> 589</span></a> <span class="c1"># Perform Structural Changes to file before preprocessing</span>
+</span><span id="L-590"><a href="#L-590"><span class="linenos"> 590</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Processing file structural actions"</span><span class="p">)</span>
+</span><span id="L-591"><a href="#L-591"><span class="linenos"> 591</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="L-592"><a href="#L-592"><span class="linenos"> 592</span></a> <span class="s2">"actions"</span> <span class="ow">in</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">]</span>
+</span><span id="L-593"><a href="#L-593"><span class="linenos"> 593</span></a> <span class="ow">and</span> <span class="s2">"split_col"</span> <span class="ow">in</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">]</span>
+</span><span id="L-594"><a href="#L-594"><span class="linenos"> 594</span></a> <span class="ow">and</span> <span class="s2">"codes_col"</span> <span class="ow">in</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">]</span>
+</span><span id="L-595"><a href="#L-595"><span class="linenos"> 595</span></a> <span class="p">):</span>
+</span><span id="L-596"><a href="#L-596"><span class="linenos"> 596</span></a> <span class="n">split_col</span> <span class="o">=</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">][</span><span class="s2">"split_col"</span><span class="p">]</span>
+</span><span id="L-597"><a href="#L-597"><span class="linenos"> 597</span></a> <span class="n">codes_col</span> <span class="o">=</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">][</span><span class="s2">"codes_col"</span><span class="p">]</span>
+</span><span id="L-598"><a href="#L-598"><span class="linenos"> 598</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-599"><a href="#L-599"><span class="linenos"> 599</span></a> <span class="s2">"Action: Splitting"</span><span class="p">,</span>
+</span><span id="L-600"><a href="#L-600"><span class="linenos"> 600</span></a> <span class="n">split_col</span><span class="p">,</span>
+</span><span id="L-601"><a href="#L-601"><span class="linenos"> 601</span></a> <span class="s2">"column into:"</span><span class="p">,</span>
+</span><span id="L-602"><a href="#L-602"><span class="linenos"> 602</span></a> <span class="n">df</span><span class="p">[</span><span class="n">split_col</span><span class="p">]</span><span class="o">.</span><span class="n">unique</span><span class="p">(),</span>
+</span><span id="L-603"><a href="#L-603"><span class="linenos"> 603</span></a> <span class="p">)</span>
+</span><span id="L-604"><a href="#L-604"><span class="linenos"> 604</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="n">codes_col</span><span class="p">]</span>
+</span><span id="L-605"><a href="#L-605"><span class="linenos"> 605</span></a> <span class="n">oh</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">get_dummies</span><span class="p">(</span><span class="n">df</span><span class="p">[</span><span class="n">split_col</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">bool</span><span class="p">)</span> <span class="c1"># one hot encode</span>
+</span><span id="L-606"><a href="#L-606"><span class="linenos"> 606</span></a> <span class="n">oh</span> <span class="o">=</span> <span class="n">oh</span><span class="o">.</span><span class="n">where</span><span class="p">((</span><span class="n">oh</span> <span class="o">!=</span> <span class="kc">True</span><span class="p">),</span> <span class="n">codes</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span> <span class="c1"># fill in 1s with codes</span>
+</span><span id="L-607"><a href="#L-607"><span class="linenos"> 607</span></a> <span class="n">oh</span><span class="p">[</span><span class="n">oh</span> <span class="o">==</span> <span class="kc">False</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">nan</span> <span class="c1"># replace 0s with None</span>
+</span><span id="L-608"><a href="#L-608"><span class="linenos"> 608</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">df</span><span class="p">,</span> <span class="n">oh</span><span class="p">],</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># merge in new columns</span>
+</span><span id="L-609"><a href="#L-609"><span class="linenos"> 609</span></a>
+</span><span id="L-610"><a href="#L-610"><span class="linenos"> 610</span></a> <span class="k">return</span> <span class="n">df</span>
+</span><span id="L-611"><a href="#L-611"><span class="linenos"> 611</span></a>
+</span><span id="L-612"><a href="#L-612"><span class="linenos"> 612</span></a>
+</span><span id="L-613"><a href="#L-613"><span class="linenos"> 613</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_preprocess_source_concepts</span><span class="p">(</span>
+</span><span id="L-614"><a href="#L-614"><span class="linenos"> 614</span></a> <span class="n">df</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">concept_set</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">code_file_path</span><span class="p">:</span> <span class="n">Path</span>
+</span><span id="L-615"><a href="#L-615"><span class="linenos"> 615</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="nb">list</span><span class="p">]:</span>
+</span><span id="L-616"><a href="#L-616"><span class="linenos"> 616</span></a><span class="w"> </span><span class="sd">"""Perform QA Checks on columns individually and append to df"""</span>
+</span><span id="L-617"><a href="#L-617"><span class="linenos"> 617</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">([])</span> <span class="c1"># create output df to append to</span>
+</span><span id="L-618"><a href="#L-618"><span class="linenos"> 618</span></a> <span class="n">code_errors</span> <span class="o">=</span> <span class="p">[]</span> <span class="c1"># list of errors from processing</span>
+</span><span id="L-619"><a href="#L-619"><span class="linenos"> 619</span></a>
+</span><span id="L-620"><a href="#L-620"><span class="linenos"> 620</span></a> <span class="c1"># remove unnamed columns due to extra commas, missing headers, or incorrect parsing</span>
+</span><span id="L-621"><a href="#L-621"><span class="linenos"> 621</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">col</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">df</span><span class="o">.</span><span class="n">columns</span> <span class="k">if</span> <span class="s2">"Unnamed"</span> <span class="ow">in</span> <span class="n">col</span><span class="p">])</span>
+</span><span id="L-622"><a href="#L-622"><span class="linenos"> 622</span></a>
+</span><span id="L-623"><a href="#L-623"><span class="linenos"> 623</span></a> <span class="c1"># Preprocess codes</span>
+</span><span id="L-624"><a href="#L-624"><span class="linenos"> 624</span></a> <span class="n">code_types</span> <span class="o">=</span> <span class="n">parse</span><span class="o">.</span><span class="n">CodeTypeParser</span><span class="p">()</span><span class="o">.</span><span class="n">code_types</span>
+</span><span id="L-625"><a href="#L-625"><span class="linenos"> 625</span></a> <span class="k">for</span> <span class="n">code_type</span> <span class="ow">in</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"columns"</span><span class="p">]:</span>
+</span><span id="L-626"><a href="#L-626"><span class="linenos"> 626</span></a> <span class="n">parser</span> <span class="o">=</span> <span class="n">code_types</span><span class="p">[</span><span class="n">code_type</span><span class="p">]</span>
+</span><span id="L-627"><a href="#L-627"><span class="linenos"> 627</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Processing </span><span class="si">{</span><span class="n">code_type</span><span class="si">}</span><span class="s2"> codes for </span><span class="si">{</span><span class="n">code_file_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-628"><a href="#L-628"><span class="linenos"> 628</span></a>
+</span><span id="L-629"><a href="#L-629"><span class="linenos"> 629</span></a> <span class="c1"># get codes by column name</span>
+</span><span id="L-630"><a href="#L-630"><span class="linenos"> 630</span></a> <span class="n">source_col_name</span> <span class="o">=</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"columns"</span><span class="p">][</span><span class="n">code_type</span><span class="p">]</span>
+</span><span id="L-631"><a href="#L-631"><span class="linenos"> 631</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="n">source_col_name</span><span class="p">]</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span>
+</span><span id="L-632"><a href="#L-632"><span class="linenos"> 632</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">codes</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span> <span class="c1"># convert to string</span>
+</span><span id="L-633"><a href="#L-633"><span class="linenos"> 633</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">codes</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span> <span class="c1"># remove excess spaces</span>
+</span><span id="L-634"><a href="#L-634"><span class="linenos"> 634</span></a>
+</span><span id="L-635"><a href="#L-635"><span class="linenos"> 635</span></a> <span class="c1"># process codes, validating them using parser and returning the errors</span>
+</span><span id="L-636"><a href="#L-636"><span class="linenos"> 636</span></a> <span class="n">codes</span><span class="p">,</span> <span class="n">errors</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">process</span><span class="p">(</span><span class="n">codes</span><span class="p">,</span> <span class="n">code_file_path</span><span class="p">)</span>
+</span><span id="L-637"><a href="#L-637"><span class="linenos"> 637</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-638"><a href="#L-638"><span class="linenos"> 638</span></a> <span class="n">code_errors</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span>
+</span><span id="L-639"><a href="#L-639"><span class="linenos"> 639</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Codes validation failed with </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span><span class="si">}</span><span class="s2"> errors"</span><span class="p">)</span>
+</span><span id="L-640"><a href="#L-640"><span class="linenos"> 640</span></a>
+</span><span id="L-641"><a href="#L-641"><span class="linenos"> 641</span></a> <span class="c1"># add processed codes to df</span>
+</span><span id="L-642"><a href="#L-642"><span class="linenos"> 642</span></a> <span class="n">new_col_name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">source_col_name</span><span class="si">}</span><span class="s2">_SOURCE"</span>
+</span><span id="L-643"><a href="#L-643"><span class="linenos"> 643</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="n">source_col_name</span><span class="p">:</span> <span class="n">new_col_name</span><span class="p">})</span>
+</span><span id="L-644"><a href="#L-644"><span class="linenos"> 644</span></a> <span class="n">process_codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">({</span><span class="n">code_type</span><span class="p">:</span> <span class="n">codes</span><span class="p">})</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
+</span><span id="L-645"><a href="#L-645"><span class="linenos"> 645</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span>
+</span><span id="L-646"><a href="#L-646"><span class="linenos"> 646</span></a> <span class="p">[</span><span class="n">out</span><span class="p">,</span> <span class="n">process_codes</span><span class="p">],</span>
+</span><span id="L-647"><a href="#L-647"><span class="linenos"> 647</span></a> <span class="n">ignore_index</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
+</span><span id="L-648"><a href="#L-648"><span class="linenos"> 648</span></a> <span class="p">)</span>
+</span><span id="L-649"><a href="#L-649"><span class="linenos"> 649</span></a>
+</span><span id="L-650"><a href="#L-650"><span class="linenos"> 650</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="n">out</span><span class="o">.</span><span class="n">head</span><span class="p">())</span>
+</span><span id="L-651"><a href="#L-651"><span class="linenos"> 651</span></a>
+</span><span id="L-652"><a href="#L-652"><span class="linenos"> 652</span></a> <span class="k">return</span> <span class="n">out</span><span class="p">,</span> <span class="n">code_errors</span>
+</span><span id="L-653"><a href="#L-653"><span class="linenos"> 653</span></a>
+</span><span id="L-654"><a href="#L-654"><span class="linenos"> 654</span></a>
+</span><span id="L-655"><a href="#L-655"><span class="linenos"> 655</span></a><span class="c1"># Translate Df with multiple codes into single code type Series</span>
+</span><span id="L-656"><a href="#L-656"><span class="linenos"> 656</span></a><span class="k">def</span><span class="w"> </span><span class="nf">translate_codes</span><span class="p">(</span>
+</span><span id="L-657"><a href="#L-657"><span class="linenos"> 657</span></a> <span class="n">source_df</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">concept_name</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="L-658"><a href="#L-658"><span class="linenos"> 658</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">:</span>
+</span><span id="L-659"><a href="#L-659"><span class="linenos"> 659</span></a><span class="w"> </span><span class="sd">"""Translates each source code type the source coding list into a target type and returns all conversions as a concept set"""</span>
+</span><span id="L-660"><a href="#L-660"><span class="linenos"> 660</span></a>
+</span><span id="L-661"><a href="#L-661"><span class="linenos"> 661</span></a> <span class="c1"># codes = pd.DataFrame([], dtype=str)</span>
+</span><span id="L-662"><a href="#L-662"><span class="linenos"> 662</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="L-663"><a href="#L-663"><span class="linenos"> 663</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s2">"SOURCE_CONCEPT"</span><span class="p">,</span> <span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="s2">"string"</span>
+</span><span id="L-664"><a href="#L-664"><span class="linenos"> 664</span></a> <span class="p">)</span>
+</span><span id="L-665"><a href="#L-665"><span class="linenos"> 665</span></a> <span class="c1"># Convert codes to target type</span>
+</span><span id="L-666"><a href="#L-666"><span class="linenos"> 666</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Converting to target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-667"><a href="#L-667"><span class="linenos"> 667</span></a>
+</span><span id="L-668"><a href="#L-668"><span class="linenos"> 668</span></a> <span class="k">for</span> <span class="n">source_code_type</span> <span class="ow">in</span> <span class="n">source_df</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
+</span><span id="L-669"><a href="#L-669"><span class="linenos"> 669</span></a> <span class="c1"># if target code type is the same as thet source code type, no translation, just appending source as target</span>
+</span><span id="L-670"><a href="#L-670"><span class="linenos"> 670</span></a> <span class="k">if</span> <span class="n">source_code_type</span> <span class="o">==</span> <span class="n">target_code_type</span><span class="p">:</span>
+</span><span id="L-671"><a href="#L-671"><span class="linenos"> 671</span></a> <span class="n">copy_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="L-672"><a href="#L-672"><span class="linenos"> 672</span></a> <span class="p">{</span>
+</span><span id="L-673"><a href="#L-673"><span class="linenos"> 673</span></a> <span class="s2">"SOURCE_CONCEPT"</span><span class="p">:</span> <span class="n">source_df</span><span class="p">[</span><span class="n">source_code_type</span><span class="p">],</span>
+</span><span id="L-674"><a href="#L-674"><span class="linenos"> 674</span></a> <span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">:</span> <span class="n">source_code_type</span><span class="p">,</span>
+</span><span id="L-675"><a href="#L-675"><span class="linenos"> 675</span></a> <span class="s2">"CONCEPT"</span><span class="p">:</span> <span class="n">source_df</span><span class="p">[</span><span class="n">source_code_type</span><span class="p">],</span>
+</span><span id="L-676"><a href="#L-676"><span class="linenos"> 676</span></a> <span class="p">}</span>
+</span><span id="L-677"><a href="#L-677"><span class="linenos"> 677</span></a> <span class="p">)</span>
+</span><span id="L-678"><a href="#L-678"><span class="linenos"> 678</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">codes</span><span class="p">,</span> <span class="n">copy_df</span><span class="p">])</span>
+</span><span id="L-679"><a href="#L-679"><span class="linenos"> 679</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-680"><a href="#L-680"><span class="linenos"> 680</span></a> <span class="sa">f</span><span class="s2">"Target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2"> is the same as source code type </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">source_df</span><span class="p">)</span><span class="si">}</span><span class="s2">, copying codes rather than translating"</span>
+</span><span id="L-681"><a href="#L-681"><span class="linenos"> 681</span></a> <span class="p">)</span>
+</span><span id="L-682"><a href="#L-682"><span class="linenos"> 682</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-683"><a href="#L-683"><span class="linenos"> 683</span></a> <span class="c1"># get the translation filename using source to target code types</span>
+</span><span id="L-684"><a href="#L-684"><span class="linenos"> 684</span></a> <span class="n">filename</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">source_code_type</span><span class="si">}</span><span class="s2">_to_</span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">.parquet"</span>
+</span><span id="L-685"><a href="#L-685"><span class="linenos"> 685</span></a> <span class="n">map_path</span> <span class="o">=</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="n">filename</span>
+</span><span id="L-686"><a href="#L-686"><span class="linenos"> 686</span></a>
+</span><span id="L-687"><a href="#L-687"><span class="linenos"> 687</span></a> <span class="c1"># do the mapping if it exists</span>
+</span><span id="L-688"><a href="#L-688"><span class="linenos"> 688</span></a> <span class="k">if</span> <span class="n">map_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-689"><a href="#L-689"><span class="linenos"> 689</span></a> <span class="c1"># get mapping</span>
+</span><span id="L-690"><a href="#L-690"><span class="linenos"> 690</span></a> <span class="n">df_map</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="n">map_path</span><span class="p">)</span>
+</span><span id="L-691"><a href="#L-691"><span class="linenos"> 691</span></a>
+</span><span id="L-692"><a href="#L-692"><span class="linenos"> 692</span></a> <span class="c1"># do mapping</span>
+</span><span id="L-693"><a href="#L-693"><span class="linenos"> 693</span></a> <span class="n">translated_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">merge</span><span class="p">(</span>
+</span><span id="L-694"><a href="#L-694"><span class="linenos"> 694</span></a> <span class="n">source_df</span><span class="p">[</span><span class="n">source_code_type</span><span class="p">],</span> <span class="n">df_map</span><span class="p">,</span> <span class="n">how</span><span class="o">=</span><span class="s2">"left"</span>
+</span><span id="L-695"><a href="#L-695"><span class="linenos"> 695</span></a> <span class="p">)</span>
+</span><span id="L-696"><a href="#L-696"><span class="linenos"> 696</span></a>
+</span><span id="L-697"><a href="#L-697"><span class="linenos"> 697</span></a> <span class="c1"># normalise the output</span>
+</span><span id="L-698"><a href="#L-698"><span class="linenos"> 698</span></a> <span class="n">translated_df</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Index</span><span class="p">([</span><span class="s2">"SOURCE_CONCEPT"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="L-699"><a href="#L-699"><span class="linenos"> 699</span></a> <span class="n">translated_df</span><span class="p">[</span><span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">]</span> <span class="o">=</span> <span class="n">source_code_type</span>
+</span><span id="L-700"><a href="#L-700"><span class="linenos"> 700</span></a>
+</span><span id="L-701"><a href="#L-701"><span class="linenos"> 701</span></a> <span class="c1"># add to list of codes</span>
+</span><span id="L-702"><a href="#L-702"><span class="linenos"> 702</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">codes</span><span class="p">,</span> <span class="n">translated_df</span><span class="p">])</span>
+</span><span id="L-703"><a href="#L-703"><span class="linenos"> 703</span></a>
+</span><span id="L-704"><a href="#L-704"><span class="linenos"> 704</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-705"><a href="#L-705"><span class="linenos"> 705</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span>
+</span><span id="L-706"><a href="#L-706"><span class="linenos"> 706</span></a> <span class="sa">f</span><span class="s2">"No mapping from </span><span class="si">{</span><span class="n">source_code_type</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">, file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">map_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> does not exist"</span>
+</span><span id="L-707"><a href="#L-707"><span class="linenos"> 707</span></a> <span class="p">)</span>
+</span><span id="L-708"><a href="#L-708"><span class="linenos"> 708</span></a>
+</span><span id="L-709"><a href="#L-709"><span class="linenos"> 709</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">codes</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span> <span class="c1"># delete NaNs</span>
+</span><span id="L-710"><a href="#L-710"><span class="linenos"> 710</span></a>
+</span><span id="L-711"><a href="#L-711"><span class="linenos"> 711</span></a> <span class="c1"># added concept set type to output if any translations</span>
+</span><span id="L-712"><a href="#L-712"><span class="linenos"> 712</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-713"><a href="#L-713"><span class="linenos"> 713</span></a> <span class="n">codes</span><span class="p">[</span><span class="s2">"CONCEPT_SET"</span><span class="p">]</span> <span class="o">=</span> <span class="n">concept_name</span>
+</span><span id="L-714"><a href="#L-714"><span class="linenos"> 714</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-715"><a href="#L-715"><span class="linenos"> 715</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No codes converted with target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-716"><a href="#L-716"><span class="linenos"> 716</span></a>
+</span><span id="L-717"><a href="#L-717"><span class="linenos"> 717</span></a> <span class="k">return</span> <span class="n">codes</span>
+</span><span id="L-718"><a href="#L-718"><span class="linenos"> 718</span></a>
+</span><span id="L-719"><a href="#L-719"><span class="linenos"> 719</span></a>
+</span><span id="L-720"><a href="#L-720"><span class="linenos"> 720</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_write_code_errors</span><span class="p">(</span><span class="n">code_errors</span><span class="p">:</span> <span class="nb">list</span><span class="p">,</span> <span class="n">code_errors_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="L-721"><a href="#L-721"><span class="linenos"> 721</span></a> <span class="n">err_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="L-722"><a href="#L-722"><span class="linenos"> 722</span></a> <span class="p">[</span>
+</span><span id="L-723"><a href="#L-723"><span class="linenos"> 723</span></a> <span class="p">{</span>
+</span><span id="L-724"><a href="#L-724"><span class="linenos"> 724</span></a> <span class="s2">"CONCEPT"</span><span class="p">:</span> <span class="s2">", "</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">err</span><span class="o">.</span><span class="n">codes</span><span class="p">[</span><span class="o">~</span><span class="n">err</span><span class="o">.</span><span class="n">mask</span><span class="p">]</span><span class="o">.</span><span class="n">tolist</span><span class="p">()),</span>
+</span><span id="L-725"><a href="#L-725"><span class="linenos"> 725</span></a> <span class="s2">"VOCABULARY"</span><span class="p">:</span> <span class="n">err</span><span class="o">.</span><span class="n">code_type</span><span class="p">,</span>
+</span><span id="L-726"><a href="#L-726"><span class="linenos"> 726</span></a> <span class="s2">"SOURCE"</span><span class="p">:</span> <span class="n">err</span><span class="o">.</span><span class="n">codes_file</span><span class="p">,</span>
+</span><span id="L-727"><a href="#L-727"><span class="linenos"> 727</span></a> <span class="s2">"CAUSE"</span><span class="p">:</span> <span class="n">err</span><span class="o">.</span><span class="n">message</span><span class="p">,</span>
+</span><span id="L-728"><a href="#L-728"><span class="linenos"> 728</span></a> <span class="p">}</span>
+</span><span id="L-729"><a href="#L-729"><span class="linenos"> 729</span></a> <span class="k">for</span> <span class="n">err</span> <span class="ow">in</span> <span class="n">code_errors</span>
+</span><span id="L-730"><a href="#L-730"><span class="linenos"> 730</span></a> <span class="p">]</span>
+</span><span id="L-731"><a href="#L-731"><span class="linenos"> 731</span></a> <span class="p">)</span>
+</span><span id="L-732"><a href="#L-732"><span class="linenos"> 732</span></a>
+</span><span id="L-733"><a href="#L-733"><span class="linenos"> 733</span></a> <span class="n">err_df</span> <span class="o">=</span> <span class="n">err_df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">()</span> <span class="c1"># Remove Duplicates from Error file</span>
+</span><span id="L-734"><a href="#L-734"><span class="linenos"> 734</span></a> <span class="n">err_df</span> <span class="o">=</span> <span class="n">err_df</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="p">[</span><span class="s2">"SOURCE"</span><span class="p">,</span> <span class="s2">"VOCABULARY"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="L-735"><a href="#L-735"><span class="linenos"> 735</span></a> <span class="n">err_df</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="n">code_errors_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">mode</span><span class="o">=</span><span class="s2">"w"</span><span class="p">)</span>
+</span><span id="L-736"><a href="#L-736"><span class="linenos"> 736</span></a>
+</span><span id="L-737"><a href="#L-737"><span class="linenos"> 737</span></a>
+</span><span id="L-738"><a href="#L-738"><span class="linenos"> 738</span></a><span class="k">def</span><span class="w"> </span><span class="nf">write_vocab_version</span><span class="p">(</span><span class="n">phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="L-739"><a href="#L-739"><span class="linenos"> 739</span></a> <span class="c1"># write the vocab version files</span>
+</span><span id="L-740"><a href="#L-740"><span class="linenos"> 740</span></a>
+</span><span id="L-741"><a href="#L-741"><span class="linenos"> 741</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">trud</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-742"><a href="#L-742"><span class="linenos"> 742</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="L-743"><a href="#L-743"><span class="linenos"> 743</span></a> <span class="sa">f</span><span class="s2">"TRUD version path </span><span class="si">{</span><span class="n">trud</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="si">}</span><span class="s2"> does not exist, please check TRUD is installed"</span>
+</span><span id="L-744"><a href="#L-744"><span class="linenos"> 744</span></a> <span class="p">)</span>
+</span><span id="L-745"><a href="#L-745"><span class="linenos"> 745</span></a>
+</span><span id="L-746"><a href="#L-746"><span class="linenos"> 746</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">omop</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-747"><a href="#L-747"><span class="linenos"> 747</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="L-748"><a href="#L-748"><span class="linenos"> 748</span></a> <span class="sa">f</span><span class="s2">"OMOP version path </span><span class="si">{</span><span class="n">omop</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="si">}</span><span class="s2"> does not exist, please check OMOP is installed"</span>
+</span><span id="L-749"><a href="#L-749"><span class="linenos"> 749</span></a> <span class="p">)</span>
+</span><span id="L-750"><a href="#L-750"><span class="linenos"> 750</span></a>
+</span><span id="L-751"><a href="#L-751"><span class="linenos"> 751</span></a> <span class="k">with</span> <span class="n">trud</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-752"><a href="#L-752"><span class="linenos"> 752</span></a> <span class="n">trud_version</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-753"><a href="#L-753"><span class="linenos"> 753</span></a>
+</span><span id="L-754"><a href="#L-754"><span class="linenos"> 754</span></a> <span class="k">with</span> <span class="n">omop</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-755"><a href="#L-755"><span class="linenos"> 755</span></a> <span class="n">omop_version</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-756"><a href="#L-756"><span class="linenos"> 756</span></a>
+</span><span id="L-757"><a href="#L-757"><span class="linenos"> 757</span></a> <span class="c1"># Create the combined YAML structure</span>
+</span><span id="L-758"><a href="#L-758"><span class="linenos"> 758</span></a> <span class="n">version_data</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-759"><a href="#L-759"><span class="linenos"> 759</span></a> <span class="s2">"versions"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="L-760"><a href="#L-760"><span class="linenos"> 760</span></a> <span class="s2">"acmc"</span><span class="p">:</span> <span class="n">acmc</span><span class="o">.</span><span class="n">__version__</span><span class="p">,</span>
+</span><span id="L-761"><a href="#L-761"><span class="linenos"> 761</span></a> <span class="s2">"trud"</span><span class="p">:</span> <span class="n">trud_version</span><span class="p">,</span>
+</span><span id="L-762"><a href="#L-762"><span class="linenos"> 762</span></a> <span class="s2">"omop"</span><span class="p">:</span> <span class="n">omop_version</span><span class="p">,</span>
+</span><span id="L-763"><a href="#L-763"><span class="linenos"> 763</span></a> <span class="p">}</span>
+</span><span id="L-764"><a href="#L-764"><span class="linenos"> 764</span></a> <span class="p">}</span>
+</span><span id="L-765"><a href="#L-765"><span class="linenos"> 765</span></a>
+</span><span id="L-766"><a href="#L-766"><span class="linenos"> 766</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">VOCAB_VERSION_FILE</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-767"><a href="#L-767"><span class="linenos"> 767</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="L-768"><a href="#L-768"><span class="linenos"> 768</span></a> <span class="n">version_data</span><span class="p">,</span>
+</span><span id="L-769"><a href="#L-769"><span class="linenos"> 769</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="L-770"><a href="#L-770"><span class="linenos"> 770</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="L-771"><a href="#L-771"><span class="linenos"> 771</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-772"><a href="#L-772"><span class="linenos"> 772</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-773"><a href="#L-773"><span class="linenos"> 773</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="L-774"><a href="#L-774"><span class="linenos"> 774</span></a> <span class="p">)</span>
+</span><span id="L-775"><a href="#L-775"><span class="linenos"> 775</span></a>
+</span><span id="L-776"><a href="#L-776"><span class="linenos"> 776</span></a>
+</span><span id="L-777"><a href="#L-777"><span class="linenos"> 777</span></a><span class="k">def</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-778"><a href="#L-778"><span class="linenos"> 778</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Processing phenotype: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-779"><a href="#L-779"><span class="linenos"> 779</span></a>
+</span><span id="L-780"><a href="#L-780"><span class="linenos"> 780</span></a> <span class="c1"># Validate configuration</span>
+</span><span id="L-781"><a href="#L-781"><span class="linenos"> 781</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-782"><a href="#L-782"><span class="linenos"> 782</span></a>
+</span><span id="L-783"><a href="#L-783"><span class="linenos"> 783</span></a> <span class="c1"># initialise paths</span>
+</span><span id="L-784"><a href="#L-784"><span class="linenos"> 784</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-785"><a href="#L-785"><span class="linenos"> 785</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-786"><a href="#L-786"><span class="linenos"> 786</span></a>
+</span><span id="L-787"><a href="#L-787"><span class="linenos"> 787</span></a> <span class="c1"># load configuration</span>
+</span><span id="L-788"><a href="#L-788"><span class="linenos"> 788</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-789"><a href="#L-789"><span class="linenos"> 789</span></a> <span class="n">config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-790"><a href="#L-790"><span class="linenos"> 790</span></a> <span class="n">phenotype</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">]</span>
+</span><span id="L-791"><a href="#L-791"><span class="linenos"> 791</span></a>
+</span><span id="L-792"><a href="#L-792"><span class="linenos"> 792</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">phenotype</span><span class="p">[</span><span class="s2">"map"</span><span class="p">])</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-793"><a href="#L-793"><span class="linenos"> 793</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No map codes defined in the phenotype configuration"</span><span class="p">)</span>
+</span><span id="L-794"><a href="#L-794"><span class="linenos"> 794</span></a>
+</span><span id="L-795"><a href="#L-795"><span class="linenos"> 795</span></a> <span class="k">if</span> <span class="n">target_code_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="n">target_code_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"map"</span><span class="p">]:</span>
+</span><span id="L-796"><a href="#L-796"><span class="linenos"> 796</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-797"><a href="#L-797"><span class="linenos"> 797</span></a> <span class="sa">f</span><span class="s2">"Target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2"> not in phenotype configuration map </span><span class="si">{</span><span class="n">phenotype</span><span class="p">[</span><span class="s1">'map'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-798"><a href="#L-798"><span class="linenos"> 798</span></a> <span class="p">)</span>
+</span><span id="L-799"><a href="#L-799"><span class="linenos"> 799</span></a>
+</span><span id="L-800"><a href="#L-800"><span class="linenos"> 800</span></a> <span class="k">if</span> <span class="n">target_code_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="L-801"><a href="#L-801"><span class="linenos"> 801</span></a> <span class="n">_map_target_code_type</span><span class="p">(</span><span class="n">phen_path</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">)</span>
+</span><span id="L-802"><a href="#L-802"><span class="linenos"> 802</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-803"><a href="#L-803"><span class="linenos"> 803</span></a> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"map"</span><span class="p">]:</span>
+</span><span id="L-804"><a href="#L-804"><span class="linenos"> 804</span></a> <span class="n">_map_target_code_type</span><span class="p">(</span><span class="n">phen_path</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">,</span> <span class="n">t</span><span class="p">)</span>
+</span><span id="L-805"><a href="#L-805"><span class="linenos"> 805</span></a>
+</span><span id="L-806"><a href="#L-806"><span class="linenos"> 806</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype processed successfully"</span><span class="p">)</span>
+</span><span id="L-807"><a href="#L-807"><span class="linenos"> 807</span></a>
+</span><span id="L-808"><a href="#L-808"><span class="linenos"> 808</span></a>
+</span><span id="L-809"><a href="#L-809"><span class="linenos"> 809</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_map_target_code_type</span><span class="p">(</span><span class="n">phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-810"><a href="#L-810"><span class="linenos"> 810</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Target coding format: </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-811"><a href="#L-811"><span class="linenos"> 811</span></a> <span class="n">concepts_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONCEPTS_DIR</span>
+</span><span id="L-812"><a href="#L-812"><span class="linenos"> 812</span></a> <span class="c1"># Create output dataframe</span>
+</span><span id="L-813"><a href="#L-813"><span class="linenos"> 813</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">([])</span>
+</span><span id="L-814"><a href="#L-814"><span class="linenos"> 814</span></a> <span class="n">code_errors</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-815"><a href="#L-815"><span class="linenos"> 815</span></a>
+</span><span id="L-816"><a href="#L-816"><span class="linenos"> 816</span></a> <span class="c1"># Process each folder in codes section</span>
+</span><span id="L-817"><a href="#L-817"><span class="linenos"> 817</span></a> <span class="k">for</span> <span class="n">concept_set</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"concept_sets"</span><span class="p">]:</span>
+</span><span id="L-818"><a href="#L-818"><span class="linenos"> 818</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"--- </span><span class="si">{</span><span class="n">concept_set</span><span class="p">[</span><span class="s1">'file'</span><span class="p">]</span><span class="si">}</span><span class="s2"> ---"</span><span class="p">)</span>
+</span><span id="L-819"><a href="#L-819"><span class="linenos"> 819</span></a>
+</span><span id="L-820"><a href="#L-820"><span class="linenos"> 820</span></a> <span class="c1"># Load code file</span>
+</span><span id="L-821"><a href="#L-821"><span class="linenos"> 821</span></a> <span class="n">codes_file_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">concepts_path</span> <span class="o">/</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"path"</span><span class="p">])</span>
+</span><span id="L-822"><a href="#L-822"><span class="linenos"> 822</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">_read_table_file</span><span class="p">(</span><span class="n">codes_file_path</span><span class="p">)</span>
+</span><span id="L-823"><a href="#L-823"><span class="linenos"> 823</span></a>
+</span><span id="L-824"><a href="#L-824"><span class="linenos"> 824</span></a> <span class="c1"># process structural actions</span>
+</span><span id="L-825"><a href="#L-825"><span class="linenos"> 825</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">_process_actions</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">concept_set</span><span class="p">)</span>
+</span><span id="L-826"><a href="#L-826"><span class="linenos"> 826</span></a>
+</span><span id="L-827"><a href="#L-827"><span class="linenos"> 827</span></a> <span class="c1"># preprocessing and validate of source concepts</span>
+</span><span id="L-828"><a href="#L-828"><span class="linenos"> 828</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Processing and validating source concept codes"</span><span class="p">)</span>
+</span><span id="L-829"><a href="#L-829"><span class="linenos"> 829</span></a> <span class="n">df</span><span class="p">,</span> <span class="n">errors</span> <span class="o">=</span> <span class="n">_preprocess_source_concepts</span><span class="p">(</span>
+</span><span id="L-830"><a href="#L-830"><span class="linenos"> 830</span></a> <span class="n">df</span><span class="p">,</span>
+</span><span id="L-831"><a href="#L-831"><span class="linenos"> 831</span></a> <span class="n">concept_set</span><span class="p">,</span>
+</span><span id="L-832"><a href="#L-832"><span class="linenos"> 832</span></a> <span class="n">codes_file_path</span><span class="p">,</span>
+</span><span id="L-833"><a href="#L-833"><span class="linenos"> 833</span></a> <span class="p">)</span>
+</span><span id="L-834"><a href="#L-834"><span class="linenos"> 834</span></a>
+</span><span id="L-835"><a href="#L-835"><span class="linenos"> 835</span></a> <span class="c1"># create df with just the source code columns</span>
+</span><span id="L-836"><a href="#L-836"><span class="linenos"> 836</span></a> <span class="n">source_column_names</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"columns"</span><span class="p">]</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
+</span><span id="L-837"><a href="#L-837"><span class="linenos"> 837</span></a> <span class="n">source_df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="n">source_column_names</span><span class="p">]</span>
+</span><span id="L-838"><a href="#L-838"><span class="linenos"> 838</span></a>
+</span><span id="L-839"><a href="#L-839"><span class="linenos"> 839</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="n">source_df</span><span class="o">.</span><span class="n">columns</span><span class="p">)</span>
+</span><span id="L-840"><a href="#L-840"><span class="linenos"> 840</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="n">source_df</span><span class="o">.</span><span class="n">head</span><span class="p">())</span>
+</span><span id="L-841"><a href="#L-841"><span class="linenos"> 841</span></a>
+</span><span id="L-842"><a href="#L-842"><span class="linenos"> 842</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-843"><a href="#L-843"><span class="linenos"> 843</span></a> <span class="sa">f</span><span class="s2">"Length of errors from _preprocess_source_concepts </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-844"><a href="#L-844"><span class="linenos"> 844</span></a> <span class="p">)</span>
+</span><span id="L-845"><a href="#L-845"><span class="linenos"> 845</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-846"><a href="#L-846"><span class="linenos"> 846</span></a> <span class="n">code_errors</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">errors</span><span class="p">)</span>
+</span><span id="L-847"><a href="#L-847"><span class="linenos"> 847</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">" Length of code_errors </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">code_errors</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-848"><a href="#L-848"><span class="linenos"> 848</span></a>
+</span><span id="L-849"><a href="#L-849"><span class="linenos"> 849</span></a> <span class="c1"># Map source concepts codes to target codes</span>
+</span><span id="L-850"><a href="#L-850"><span class="linenos"> 850</span></a> <span class="c1"># if processing a source coding list with categorical data</span>
+</span><span id="L-851"><a href="#L-851"><span class="linenos"> 851</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="L-852"><a href="#L-852"><span class="linenos"> 852</span></a> <span class="s2">"actions"</span> <span class="ow">in</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">]</span>
+</span><span id="L-853"><a href="#L-853"><span class="linenos"> 853</span></a> <span class="ow">and</span> <span class="s2">"divide_col"</span> <span class="ow">in</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">]</span>
+</span><span id="L-854"><a href="#L-854"><span class="linenos"> 854</span></a> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">df</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span>
+</span><span id="L-855"><a href="#L-855"><span class="linenos"> 855</span></a> <span class="p">):</span>
+</span><span id="L-856"><a href="#L-856"><span class="linenos"> 856</span></a> <span class="n">divide_col</span> <span class="o">=</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">][</span><span class="s2">"divide_col"</span><span class="p">]</span>
+</span><span id="L-857"><a href="#L-857"><span class="linenos"> 857</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Action: Dividing Table by </span><span class="si">{</span><span class="n">divide_col</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-858"><a href="#L-858"><span class="linenos"> 858</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"column into: </span><span class="si">{</span><span class="n">df</span><span class="p">[</span><span class="n">divide_col</span><span class="p">]</span><span class="o">.</span><span class="n">unique</span><span class="p">()</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-859"><a href="#L-859"><span class="linenos"> 859</span></a> <span class="n">df_grp</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="n">divide_col</span><span class="p">)</span>
+</span><span id="L-860"><a href="#L-860"><span class="linenos"> 860</span></a> <span class="k">for</span> <span class="n">cat</span><span class="p">,</span> <span class="n">grp</span> <span class="ow">in</span> <span class="n">df_grp</span><span class="p">:</span>
+</span><span id="L-861"><a href="#L-861"><span class="linenos"> 861</span></a> <span class="k">if</span> <span class="n">cat</span> <span class="o">==</span> <span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"category"</span><span class="p">]:</span>
+</span><span id="L-862"><a href="#L-862"><span class="linenos"> 862</span></a> <span class="n">grp</span> <span class="o">=</span> <span class="n">grp</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="n">divide_col</span><span class="p">])</span> <span class="c1"># delete categorical column</span>
+</span><span id="L-863"><a href="#L-863"><span class="linenos"> 863</span></a> <span class="n">source_df</span> <span class="o">=</span> <span class="n">grp</span><span class="p">[</span><span class="n">source_column_names</span><span class="p">]</span>
+</span><span id="L-864"><a href="#L-864"><span class="linenos"> 864</span></a> <span class="n">trans_out</span> <span class="o">=</span> <span class="n">translate_codes</span><span class="p">(</span>
+</span><span id="L-865"><a href="#L-865"><span class="linenos"> 865</span></a> <span class="n">source_df</span><span class="p">,</span>
+</span><span id="L-866"><a href="#L-866"><span class="linenos"> 866</span></a> <span class="n">target_code_type</span><span class="o">=</span><span class="n">target_code_type</span><span class="p">,</span>
+</span><span id="L-867"><a href="#L-867"><span class="linenos"> 867</span></a> <span class="n">concept_name</span><span class="o">=</span><span class="n">concept_set</span><span class="p">[</span><span class="s2">"name"</span><span class="p">],</span>
+</span><span id="L-868"><a href="#L-868"><span class="linenos"> 868</span></a> <span class="p">)</span>
+</span><span id="L-869"><a href="#L-869"><span class="linenos"> 869</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">out</span><span class="p">,</span> <span class="n">trans_out</span><span class="p">])</span>
+</span><span id="L-870"><a href="#L-870"><span class="linenos"> 870</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-871"><a href="#L-871"><span class="linenos"> 871</span></a> <span class="n">source_df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="n">source_column_names</span><span class="p">]</span>
+</span><span id="L-872"><a href="#L-872"><span class="linenos"> 872</span></a> <span class="n">trans_out</span> <span class="o">=</span> <span class="n">translate_codes</span><span class="p">(</span>
+</span><span id="L-873"><a href="#L-873"><span class="linenos"> 873</span></a> <span class="n">source_df</span><span class="p">,</span>
+</span><span id="L-874"><a href="#L-874"><span class="linenos"> 874</span></a> <span class="n">target_code_type</span><span class="o">=</span><span class="n">target_code_type</span><span class="p">,</span>
+</span><span id="L-875"><a href="#L-875"><span class="linenos"> 875</span></a> <span class="n">concept_name</span><span class="o">=</span><span class="n">concept_set</span><span class="p">[</span><span class="s2">"name"</span><span class="p">],</span>
+</span><span id="L-876"><a href="#L-876"><span class="linenos"> 876</span></a> <span class="p">)</span>
+</span><span id="L-877"><a href="#L-877"><span class="linenos"> 877</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">out</span><span class="p">,</span> <span class="n">trans_out</span><span class="p">])</span>
+</span><span id="L-878"><a href="#L-878"><span class="linenos"> 878</span></a>
+</span><span id="L-879"><a href="#L-879"><span class="linenos"> 879</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">code_errors</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-880"><a href="#L-880"><span class="linenos"> 880</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">"The map processing has </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">code_errors</span><span class="p">)</span><span class="si">}</span><span class="s2"> errors"</span><span class="p">)</span>
+</span><span id="L-881"><a href="#L-881"><span class="linenos"> 881</span></a> <span class="n">error_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span> <span class="o">/</span> <span class="s2">"errors"</span>
+</span><span id="L-882"><a href="#L-882"><span class="linenos"> 882</span></a> <span class="n">error_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-883"><a href="#L-883"><span class="linenos"> 883</span></a> <span class="n">error_filename</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">-code-errors.csv"</span>
+</span><span id="L-884"><a href="#L-884"><span class="linenos"> 884</span></a> <span class="n">_write_code_errors</span><span class="p">(</span><span class="n">code_errors</span><span class="p">,</span> <span class="n">error_path</span> <span class="o">/</span> <span class="n">error_filename</span><span class="p">)</span>
+</span><span id="L-885"><a href="#L-885"><span class="linenos"> 885</span></a>
+</span><span id="L-886"><a href="#L-886"><span class="linenos"> 886</span></a> <span class="c1"># Check there is output from processing</span>
+</span><span id="L-887"><a href="#L-887"><span class="linenos"> 887</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">out</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-888"><a href="#L-888"><span class="linenos"> 888</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No output after map processing"</span><span class="p">)</span>
+</span><span id="L-889"><a href="#L-889"><span class="linenos"> 889</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span>
+</span><span id="L-890"><a href="#L-890"><span class="linenos"> 890</span></a> <span class="sa">f</span><span class="s2">"No output after map processing, check config </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-891"><a href="#L-891"><span class="linenos"> 891</span></a> <span class="p">)</span>
+</span><span id="L-892"><a href="#L-892"><span class="linenos"> 892</span></a>
+</span><span id="L-893"><a href="#L-893"><span class="linenos"> 893</span></a> <span class="c1"># final processing</span>
+</span><span id="L-894"><a href="#L-894"><span class="linenos"> 894</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">out</span><span class="o">.</span><span class="n">reset_index</span><span class="p">(</span><span class="n">drop</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-895"><a href="#L-895"><span class="linenos"> 895</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">out</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">(</span><span class="n">subset</span><span class="o">=</span><span class="p">[</span><span class="s2">"CONCEPT_SET"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="L-896"><a href="#L-896"><span class="linenos"> 896</span></a> <span class="n">out</span> <span class="o">=</span> <span class="n">out</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="p">[</span><span class="s2">"CONCEPT_SET"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="L-897"><a href="#L-897"><span class="linenos"> 897</span></a>
+</span><span id="L-898"><a href="#L-898"><span class="linenos"> 898</span></a> <span class="n">out_count</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">out</span><span class="o">.</span><span class="n">index</span><span class="p">)</span>
+</span><span id="L-899"><a href="#L-899"><span class="linenos"> 899</span></a> <span class="c1"># added metadata</span>
+</span><span id="L-900"><a href="#L-900"><span class="linenos"> 900</span></a> <span class="c1"># Loop over each source_concept_type and perform the left join on all columns apart from source code columns</span>
+</span><span id="L-901"><a href="#L-901"><span class="linenos"> 901</span></a> <span class="n">result_list</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-902"><a href="#L-902"><span class="linenos"> 902</span></a> <span class="n">source_column_names</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">concept_set</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"columns"</span><span class="p">]</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
+</span><span id="L-903"><a href="#L-903"><span class="linenos"> 903</span></a> <span class="k">for</span> <span class="n">source_concept_type</span> <span class="ow">in</span> <span class="n">source_column_names</span><span class="p">:</span>
+</span><span id="L-904"><a href="#L-904"><span class="linenos"> 904</span></a> <span class="c1"># Filter output based on the current source_concept_type</span>
+</span><span id="L-905"><a href="#L-905"><span class="linenos"> 905</span></a> <span class="n">out_filtered_df</span> <span class="o">=</span> <span class="n">out</span><span class="p">[</span><span class="n">out</span><span class="p">[</span><span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">]</span> <span class="o">==</span> <span class="n">source_concept_type</span><span class="p">]</span>
+</span><span id="L-906"><a href="#L-906"><span class="linenos"> 906</span></a> <span class="n">filtered_count</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">out_filtered_df</span><span class="o">.</span><span class="n">index</span><span class="p">)</span>
+</span><span id="L-907"><a href="#L-907"><span class="linenos"> 907</span></a>
+</span><span id="L-908"><a href="#L-908"><span class="linenos"> 908</span></a> <span class="c1"># Remove the source type columns except the current type will leave the metadata and the join</span>
+</span><span id="L-909"><a href="#L-909"><span class="linenos"> 909</span></a> <span class="n">remove_types</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-910"><a href="#L-910"><span class="linenos"> 910</span></a> <span class="nb">type</span> <span class="k">for</span> <span class="nb">type</span> <span class="ow">in</span> <span class="n">source_column_names</span> <span class="k">if</span> <span class="nb">type</span> <span class="o">!=</span> <span class="n">source_concept_type</span>
+</span><span id="L-911"><a href="#L-911"><span class="linenos"> 911</span></a> <span class="p">]</span>
+</span><span id="L-912"><a href="#L-912"><span class="linenos"> 912</span></a> <span class="n">metadata_df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="n">remove_types</span><span class="p">)</span>
+</span><span id="L-913"><a href="#L-913"><span class="linenos"> 913</span></a> <span class="n">metadata_df</span> <span class="o">=</span> <span class="n">metadata_df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span>
+</span><span id="L-914"><a href="#L-914"><span class="linenos"> 914</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="n">source_concept_type</span><span class="p">:</span> <span class="s2">"SOURCE_CONCEPT"</span><span class="p">}</span>
+</span><span id="L-915"><a href="#L-915"><span class="linenos"> 915</span></a> <span class="p">)</span>
+</span><span id="L-916"><a href="#L-916"><span class="linenos"> 916</span></a> <span class="n">metadata_df_count</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">metadata_df</span><span class="o">.</span><span class="n">index</span><span class="p">)</span>
+</span><span id="L-917"><a href="#L-917"><span class="linenos"> 917</span></a>
+</span><span id="L-918"><a href="#L-918"><span class="linenos"> 918</span></a> <span class="c1"># Perform the left join with df2 on SOURCE_CONCEPT to add the metadata</span>
+</span><span id="L-919"><a href="#L-919"><span class="linenos"> 919</span></a> <span class="n">result</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">merge</span><span class="p">(</span><span class="n">out_filtered_df</span><span class="p">,</span> <span class="n">metadata_df</span><span class="p">,</span> <span class="n">how</span><span class="o">=</span><span class="s2">"left"</span><span class="p">,</span> <span class="n">on</span><span class="o">=</span><span class="s2">"SOURCE_CONCEPT"</span><span class="p">)</span>
+</span><span id="L-920"><a href="#L-920"><span class="linenos"> 920</span></a> <span class="n">result_count</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">index</span><span class="p">)</span>
+</span><span id="L-921"><a href="#L-921"><span class="linenos"> 921</span></a>
+</span><span id="L-922"><a href="#L-922"><span class="linenos"> 922</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-923"><a href="#L-923"><span class="linenos"> 923</span></a> <span class="sa">f</span><span class="s2">"Adding metadata for </span><span class="si">{</span><span class="n">source_concept_type</span><span class="si">}</span><span class="s2">: out_count </span><span class="si">{</span><span class="n">out_count</span><span class="si">}</span><span class="s2">, filtered_count </span><span class="si">{</span><span class="n">filtered_count</span><span class="si">}</span><span class="s2">, metadata_df_count </span><span class="si">{</span><span class="n">metadata_df_count</span><span class="si">}</span><span class="s2">, result_count </span><span class="si">{</span><span class="n">result_count</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-924"><a href="#L-924"><span class="linenos"> 924</span></a> <span class="p">)</span>
+</span><span id="L-925"><a href="#L-925"><span class="linenos"> 925</span></a>
+</span><span id="L-926"><a href="#L-926"><span class="linenos"> 926</span></a> <span class="c1"># Append the result to the result_list</span>
+</span><span id="L-927"><a href="#L-927"><span class="linenos"> 927</span></a> <span class="n">result_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
+</span><span id="L-928"><a href="#L-928"><span class="linenos"> 928</span></a>
+</span><span id="L-929"><a href="#L-929"><span class="linenos"> 929</span></a> <span class="c1"># Concatenate all the results into a single DataFrame</span>
+</span><span id="L-930"><a href="#L-930"><span class="linenos"> 930</span></a> <span class="n">final_out</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">result_list</span><span class="p">,</span> <span class="n">ignore_index</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-931"><a href="#L-931"><span class="linenos"> 931</span></a> <span class="n">final_out</span> <span class="o">=</span> <span class="n">final_out</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">(</span><span class="n">subset</span><span class="o">=</span><span class="p">[</span><span class="s2">"CONCEPT_SET"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="L-932"><a href="#L-932"><span class="linenos"> 932</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-933"><a href="#L-933"><span class="linenos"> 933</span></a> <span class="sa">f</span><span class="s2">"Check metadata processing counts: before </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">out</span><span class="o">.</span><span class="n">index</span><span class="p">)</span><span class="si">}</span><span class="s2"> : after </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">final_out</span><span class="o">.</span><span class="n">index</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-934"><a href="#L-934"><span class="linenos"> 934</span></a> <span class="p">)</span>
+</span><span id="L-935"><a href="#L-935"><span class="linenos"> 935</span></a>
+</span><span id="L-936"><a href="#L-936"><span class="linenos"> 936</span></a> <span class="c1"># Save output to map directory</span>
+</span><span id="L-937"><a href="#L-937"><span class="linenos"> 937</span></a> <span class="n">output_filename</span> <span class="o">=</span> <span class="n">target_code_type</span> <span class="o">+</span> <span class="s2">".csv"</span>
+</span><span id="L-938"><a href="#L-938"><span class="linenos"> 938</span></a> <span class="n">map_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span> <span class="o">/</span> <span class="n">output_filename</span>
+</span><span id="L-939"><a href="#L-939"><span class="linenos"> 939</span></a> <span class="n">final_out</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="n">map_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-940"><a href="#L-940"><span class="linenos"> 940</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Saved mapped concepts to </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">map_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-941"><a href="#L-941"><span class="linenos"> 941</span></a>
+</span><span id="L-942"><a href="#L-942"><span class="linenos"> 942</span></a> <span class="c1"># save concept sets as separate files</span>
+</span><span id="L-943"><a href="#L-943"><span class="linenos"> 943</span></a> <span class="n">concept_set_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CSV_PATH</span> <span class="o">/</span> <span class="n">target_code_type</span>
+</span><span id="L-944"><a href="#L-944"><span class="linenos"> 944</span></a>
+</span><span id="L-945"><a href="#L-945"><span class="linenos"> 945</span></a> <span class="c1"># empty the concept-set directory except for hiddle files, e.g. .git</span>
+</span><span id="L-946"><a href="#L-946"><span class="linenos"> 946</span></a> <span class="k">if</span> <span class="n">concept_set_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-947"><a href="#L-947"><span class="linenos"> 947</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">concept_set_path</span><span class="o">.</span><span class="n">iterdir</span><span class="p">():</span>
+</span><span id="L-948"><a href="#L-948"><span class="linenos"> 948</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">item</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"."</span><span class="p">):</span>
+</span><span id="L-949"><a href="#L-949"><span class="linenos"> 949</span></a> <span class="n">item</span><span class="o">.</span><span class="n">unlink</span><span class="p">()</span>
+</span><span id="L-950"><a href="#L-950"><span class="linenos"> 950</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-951"><a href="#L-951"><span class="linenos"> 951</span></a> <span class="n">concept_set_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-952"><a href="#L-952"><span class="linenos"> 952</span></a>
+</span><span id="L-953"><a href="#L-953"><span class="linenos"> 953</span></a> <span class="c1"># write each concept as a separate file</span>
+</span><span id="L-954"><a href="#L-954"><span class="linenos"> 954</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">concept</span> <span class="ow">in</span> <span class="n">final_out</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">):</span>
+</span><span id="L-955"><a href="#L-955"><span class="linenos"> 955</span></a> <span class="n">concept</span> <span class="o">=</span> <span class="n">concept</span><span class="o">.</span><span class="n">sort_values</span><span class="p">(</span><span class="n">by</span><span class="o">=</span><span class="s2">"CONCEPT"</span><span class="p">)</span> <span class="c1"># sort rows</span>
+</span><span id="L-956"><a href="#L-956"><span class="linenos"> 956</span></a> <span class="n">concept</span> <span class="o">=</span> <span class="n">concept</span><span class="o">.</span><span class="n">dropna</span><span class="p">(</span><span class="n">how</span><span class="o">=</span><span class="s2">"all"</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># remove empty cols</span>
+</span><span id="L-957"><a href="#L-957"><span class="linenos"> 957</span></a> <span class="n">concept</span> <span class="o">=</span> <span class="n">concept</span><span class="o">.</span><span class="n">reindex</span><span class="p">(</span>
+</span><span id="L-958"><a href="#L-958"><span class="linenos"> 958</span></a> <span class="nb">sorted</span><span class="p">(</span><span class="n">concept</span><span class="o">.</span><span class="n">columns</span><span class="p">),</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span>
+</span><span id="L-959"><a href="#L-959"><span class="linenos"> 959</span></a> <span class="p">)</span> <span class="c1"># sort cols alphabetically</span>
+</span><span id="L-960"><a href="#L-960"><span class="linenos"> 960</span></a> <span class="n">filename</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">.csv"</span>
+</span><span id="L-961"><a href="#L-961"><span class="linenos"> 961</span></a> <span class="n">concept_path</span> <span class="o">=</span> <span class="n">concept_set_path</span> <span class="o">/</span> <span class="n">filename</span>
+</span><span id="L-962"><a href="#L-962"><span class="linenos"> 962</span></a> <span class="n">concept</span><span class="o">.</span><span class="n">to_csv</span><span class="p">(</span><span class="n">concept_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-963"><a href="#L-963"><span class="linenos"> 963</span></a>
+</span><span id="L-964"><a href="#L-964"><span class="linenos"> 964</span></a> <span class="n">write_vocab_version</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-965"><a href="#L-965"><span class="linenos"> 965</span></a>
+</span><span id="L-966"><a href="#L-966"><span class="linenos"> 966</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype processed target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-967"><a href="#L-967"><span class="linenos"> 967</span></a>
+</span><span id="L-968"><a href="#L-968"><span class="linenos"> 968</span></a>
+</span><span id="L-969"><a href="#L-969"><span class="linenos"> 969</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_generate_version_tag</span><span class="p">(</span>
+</span><span id="L-970"><a href="#L-970"><span class="linenos"> 970</span></a> <span class="n">repo</span><span class="p">:</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="p">,</span> <span class="n">increment</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="n">DEFAULT_VERSION_INC</span><span class="p">,</span> <span class="n">use_v_prefix</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span>
+</span><span id="L-971"><a href="#L-971"><span class="linenos"> 971</span></a><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
+</span><span id="L-972"><a href="#L-972"><span class="linenos"> 972</span></a> <span class="c1"># Get all valid semantic version tags</span>
+</span><span id="L-973"><a href="#L-973"><span class="linenos"> 973</span></a> <span class="n">versions</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-974"><a href="#L-974"><span class="linenos"> 974</span></a> <span class="k">for</span> <span class="n">tag</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">tags</span><span class="p">:</span>
+</span><span id="L-975"><a href="#L-975"><span class="linenos"> 975</span></a> <span class="n">tag_name</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-976"><a href="#L-976"><span class="linenos"> 976</span></a> <span class="n">tag</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">lstrip</span><span class="p">(</span><span class="s2">"v"</span><span class="p">)</span> <span class="k">if</span> <span class="n">use_v_prefix</span> <span class="k">else</span> <span class="n">tag</span><span class="o">.</span><span class="n">name</span>
+</span><span id="L-977"><a href="#L-977"><span class="linenos"> 977</span></a> <span class="p">)</span> <span class="c1"># Remove 'v' if needed</span>
+</span><span id="L-978"><a href="#L-978"><span class="linenos"> 978</span></a> <span class="k">if</span> <span class="n">semver</span><span class="o">.</span><span class="n">Version</span><span class="o">.</span><span class="n">is_valid</span><span class="p">(</span><span class="n">tag_name</span><span class="p">):</span>
+</span><span id="L-979"><a href="#L-979"><span class="linenos"> 979</span></a> <span class="n">versions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">semver</span><span class="o">.</span><span class="n">Version</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">tag_name</span><span class="p">))</span>
+</span><span id="L-980"><a href="#L-980"><span class="linenos"> 980</span></a>
+</span><span id="L-981"><a href="#L-981"><span class="linenos"> 981</span></a> <span class="c1"># Determine the next version</span>
+</span><span id="L-982"><a href="#L-982"><span class="linenos"> 982</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">versions</span><span class="p">:</span>
+</span><span id="L-983"><a href="#L-983"><span class="linenos"> 983</span></a> <span class="n">new_version</span> <span class="o">=</span> <span class="n">semver</span><span class="o">.</span><span class="n">Version</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
+</span><span id="L-984"><a href="#L-984"><span class="linenos"> 984</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-985"><a href="#L-985"><span class="linenos"> 985</span></a> <span class="n">latest_version</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">versions</span><span class="p">)</span>
+</span><span id="L-986"><a href="#L-986"><span class="linenos"> 986</span></a> <span class="k">if</span> <span class="n">increment</span> <span class="o">==</span> <span class="s2">"major"</span><span class="p">:</span>
+</span><span id="L-987"><a href="#L-987"><span class="linenos"> 987</span></a> <span class="n">new_version</span> <span class="o">=</span> <span class="n">latest_version</span><span class="o">.</span><span class="n">bump_major</span><span class="p">()</span>
+</span><span id="L-988"><a href="#L-988"><span class="linenos"> 988</span></a> <span class="k">elif</span> <span class="n">increment</span> <span class="o">==</span> <span class="s2">"minor"</span><span class="p">:</span>
+</span><span id="L-989"><a href="#L-989"><span class="linenos"> 989</span></a> <span class="n">new_version</span> <span class="o">=</span> <span class="n">latest_version</span><span class="o">.</span><span class="n">bump_minor</span><span class="p">()</span>
+</span><span id="L-990"><a href="#L-990"><span class="linenos"> 990</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-991"><a href="#L-991"><span class="linenos"> 991</span></a> <span class="n">new_version</span> <span class="o">=</span> <span class="n">latest_version</span><span class="o">.</span><span class="n">bump_patch</span><span class="p">()</span>
+</span><span id="L-992"><a href="#L-992"><span class="linenos"> 992</span></a>
+</span><span id="L-993"><a href="#L-993"><span class="linenos"> 993</span></a> <span class="c1"># Create the new tag</span>
+</span><span id="L-994"><a href="#L-994"><span class="linenos"> 994</span></a> <span class="n">new_version_str</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"v</span><span class="si">{</span><span class="n">new_version</span><span class="si">}</span><span class="s2">"</span> <span class="k">if</span> <span class="n">use_v_prefix</span> <span class="k">else</span> <span class="nb">str</span><span class="p">(</span><span class="n">new_version</span><span class="p">)</span>
+</span><span id="L-995"><a href="#L-995"><span class="linenos"> 995</span></a>
+</span><span id="L-996"><a href="#L-996"><span class="linenos"> 996</span></a> <span class="k">return</span> <span class="n">new_version_str</span>
+</span><span id="L-997"><a href="#L-997"><span class="linenos"> 997</span></a>
+</span><span id="L-998"><a href="#L-998"><span class="linenos"> 998</span></a>
+</span><span id="L-999"><a href="#L-999"><span class="linenos"> 999</span></a><span class="k">def</span><span class="w"> </span><span class="nf">publish</span><span class="p">(</span>
+</span><span id="L-1000"><a href="#L-1000"><span class="linenos">1000</span></a> <span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">msg</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">increment</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="n">DEFAULT_VERSION_INC</span>
+</span><span id="L-1001"><a href="#L-1001"><span class="linenos">1001</span></a><span class="p">):</span>
+</span><span id="L-1002"><a href="#L-1002"><span class="linenos">1002</span></a><span class="w"> </span><span class="sd">"""Publishes updates to the phenotype by commiting all changes to the repo directory"""</span>
+</span><span id="L-1003"><a href="#L-1003"><span class="linenos">1003</span></a>
+</span><span id="L-1004"><a href="#L-1004"><span class="linenos">1004</span></a> <span class="c1"># Validate config</span>
+</span><span id="L-1005"><a href="#L-1005"><span class="linenos">1005</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1006"><a href="#L-1006"><span class="linenos">1006</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1007"><a href="#L-1007"><span class="linenos">1007</span></a>
+</span><span id="L-1008"><a href="#L-1008"><span class="linenos">1008</span></a> <span class="c1"># load git repo and set the branch</span>
+</span><span id="L-1009"><a href="#L-1009"><span class="linenos">1009</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="L-1010"><a href="#L-1010"><span class="linenos">1010</span></a> <span class="k">if</span> <span class="n">DEFAULT_GIT_BRANCH</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">branches</span><span class="p">:</span>
+</span><span id="L-1011"><a href="#L-1011"><span class="linenos">1011</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">heads</span><span class="p">[</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">]</span>
+</span><span id="L-1012"><a href="#L-1012"><span class="linenos">1012</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="L-1013"><a href="#L-1013"><span class="linenos">1013</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1014"><a href="#L-1014"><span class="linenos">1014</span></a> <span class="k">raise</span> <span class="ne">AttributeError</span><span class="p">(</span>
+</span><span id="L-1015"><a href="#L-1015"><span class="linenos">1015</span></a> <span class="sa">f</span><span class="s2">"Phen repo does not contain the default branch </span><span class="si">{</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-1016"><a href="#L-1016"><span class="linenos">1016</span></a> <span class="p">)</span>
+</span><span id="L-1017"><a href="#L-1017"><span class="linenos">1017</span></a>
+</span><span id="L-1018"><a href="#L-1018"><span class="linenos">1018</span></a> <span class="c1"># check if any changes to publish</span>
+</span><span id="L-1019"><a href="#L-1019"><span class="linenos">1019</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">repo</span><span class="o">.</span><span class="n">is_dirty</span><span class="p">()</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">repo</span><span class="o">.</span><span class="n">untracked_files</span><span class="p">:</span>
+</span><span id="L-1020"><a href="#L-1020"><span class="linenos">1020</span></a> <span class="k">if</span> <span class="n">remote_url</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="s2">"origin"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">:</span>
+</span><span id="L-1021"><a href="#L-1021"><span class="linenos">1021</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"First publish to remote url </span><span class="si">{</span><span class="n">remote_url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1022"><a href="#L-1022"><span class="linenos">1022</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1023"><a href="#L-1023"><span class="linenos">1023</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Nothing to publish, no changes to the repo"</span><span class="p">)</span>
+</span><span id="L-1024"><a href="#L-1024"><span class="linenos">1024</span></a> <span class="k">return</span>
+</span><span id="L-1025"><a href="#L-1025"><span class="linenos">1025</span></a>
+</span><span id="L-1026"><a href="#L-1026"><span class="linenos">1026</span></a> <span class="c1"># get next version</span>
+</span><span id="L-1027"><a href="#L-1027"><span class="linenos">1027</span></a> <span class="n">new_version_str</span> <span class="o">=</span> <span class="n">_generate_version_tag</span><span class="p">(</span><span class="n">repo</span><span class="p">,</span> <span class="n">increment</span><span class="p">)</span>
+</span><span id="L-1028"><a href="#L-1028"><span class="linenos">1028</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"New version: </span><span class="si">{</span><span class="n">new_version_str</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1029"><a href="#L-1029"><span class="linenos">1029</span></a>
+</span><span id="L-1030"><a href="#L-1030"><span class="linenos">1030</span></a> <span class="c1"># Write version in configuration file</span>
+</span><span id="L-1031"><a href="#L-1031"><span class="linenos">1031</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-1032"><a href="#L-1032"><span class="linenos">1032</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-1033"><a href="#L-1033"><span class="linenos">1033</span></a> <span class="n">config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-1034"><a href="#L-1034"><span class="linenos">1034</span></a>
+</span><span id="L-1035"><a href="#L-1035"><span class="linenos">1035</span></a> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"version"</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_version_str</span>
+</span><span id="L-1036"><a href="#L-1036"><span class="linenos">1036</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">config_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-1037"><a href="#L-1037"><span class="linenos">1037</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="L-1038"><a href="#L-1038"><span class="linenos">1038</span></a> <span class="n">config</span><span class="p">,</span>
+</span><span id="L-1039"><a href="#L-1039"><span class="linenos">1039</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="L-1040"><a href="#L-1040"><span class="linenos">1040</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="L-1041"><a href="#L-1041"><span class="linenos">1041</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-1042"><a href="#L-1042"><span class="linenos">1042</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-1043"><a href="#L-1043"><span class="linenos">1043</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="L-1044"><a href="#L-1044"><span class="linenos">1044</span></a> <span class="p">)</span>
+</span><span id="L-1045"><a href="#L-1045"><span class="linenos">1045</span></a>
+</span><span id="L-1046"><a href="#L-1046"><span class="linenos">1046</span></a> <span class="c1"># Add and commit changes to repo including version updates</span>
+</span><span id="L-1047"><a href="#L-1047"><span class="linenos">1047</span></a> <span class="n">commit_message</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Committing updates to phenotype </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-1048"><a href="#L-1048"><span class="linenos">1048</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="s2">"--all"</span><span class="p">)</span>
+</span><span id="L-1049"><a href="#L-1049"><span class="linenos">1049</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">commit</span><span class="p">(</span><span class="n">commit_message</span><span class="p">)</span>
+</span><span id="L-1050"><a href="#L-1050"><span class="linenos">1050</span></a>
+</span><span id="L-1051"><a href="#L-1051"><span class="linenos">1051</span></a> <span class="c1"># Add tag to the repo</span>
+</span><span id="L-1052"><a href="#L-1052"><span class="linenos">1052</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_tag</span><span class="p">(</span><span class="n">new_version_str</span><span class="p">)</span>
+</span><span id="L-1053"><a href="#L-1053"><span class="linenos">1053</span></a>
+</span><span id="L-1054"><a href="#L-1054"><span class="linenos">1054</span></a> <span class="c1"># push to origin if a remote repo</span>
+</span><span id="L-1055"><a href="#L-1055"><span class="linenos">1055</span></a> <span class="k">if</span> <span class="n">remote_url</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="s2">"origin"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">:</span>
+</span><span id="L-1056"><a href="#L-1056"><span class="linenos">1056</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">remote_url</span><span class="p">)</span>
+</span><span id="L-1057"><a href="#L-1057"><span class="linenos">1057</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_remote</span><span class="p">(</span><span class="s2">"origin"</span><span class="p">,</span> <span class="n">git_url</span><span class="p">)</span>
+</span><span id="L-1058"><a href="#L-1058"><span class="linenos">1058</span></a>
+</span><span id="L-1059"><a href="#L-1059"><span class="linenos">1059</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-1060"><a href="#L-1060"><span class="linenos">1060</span></a> <span class="k">if</span> <span class="s2">"origin"</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">:</span>
+</span><span id="L-1061"><a href="#L-1061"><span class="linenos">1061</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Remote 'origin' is set </span><span class="si">{</span><span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="o">.</span><span class="n">origin</span><span class="o">.</span><span class="n">url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1062"><a href="#L-1062"><span class="linenos">1062</span></a> <span class="n">origin</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="o">.</span><span class="n">origin</span>
+</span><span id="L-1063"><a href="#L-1063"><span class="linenos">1063</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Pushing main branch to remote repo"</span><span class="p">)</span>
+</span><span id="L-1064"><a href="#L-1064"><span class="linenos">1064</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="s2">"--set-upstream"</span><span class="p">,</span> <span class="s2">"origin"</span><span class="p">,</span> <span class="s2">"main"</span><span class="p">)</span>
+</span><span id="L-1065"><a href="#L-1065"><span class="linenos">1065</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Pushing version tags to remote git repo"</span><span class="p">)</span>
+</span><span id="L-1066"><a href="#L-1066"><span class="linenos">1066</span></a> <span class="n">origin</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="n">tags</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1067"><a href="#L-1067"><span class="linenos">1067</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Changes pushed to 'origin'"</span><span class="p">)</span>
+</span><span id="L-1068"><a href="#L-1068"><span class="linenos">1068</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1069"><a href="#L-1069"><span class="linenos">1069</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Remote 'origin' is not set"</span><span class="p">)</span>
+</span><span id="L-1070"><a href="#L-1070"><span class="linenos">1070</span></a> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="L-1071"><a href="#L-1071"><span class="linenos">1071</span></a> <span class="n">tag_ref</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">tags</span><span class="p">[</span><span class="n">new_version_str</span><span class="p">]</span>
+</span><span id="L-1072"><a href="#L-1072"><span class="linenos">1072</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">delete_tag</span><span class="p">(</span><span class="n">tag_ref</span><span class="p">)</span>
+</span><span id="L-1073"><a href="#L-1073"><span class="linenos">1073</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">reset</span><span class="p">(</span><span class="s2">"--soft"</span><span class="p">,</span> <span class="s2">"HEAD~1"</span><span class="p">)</span>
+</span><span id="L-1074"><a href="#L-1074"><span class="linenos">1074</span></a> <span class="k">raise</span> <span class="n">e</span>
+</span><span id="L-1075"><a href="#L-1075"><span class="linenos">1075</span></a>
+</span><span id="L-1076"><a href="#L-1076"><span class="linenos">1076</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype published successfully"</span><span class="p">)</span>
+</span><span id="L-1077"><a href="#L-1077"><span class="linenos">1077</span></a>
+</span><span id="L-1078"><a href="#L-1078"><span class="linenos">1078</span></a>
+</span><span id="L-1079"><a href="#L-1079"><span class="linenos">1079</span></a><span class="k">def</span><span class="w"> </span><span class="nf">export</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-1080"><a href="#L-1080"><span class="linenos">1080</span></a><span class="w"> </span><span class="sd">"""Exports a phen repo at a specific tagged version into a target directory"""</span>
+</span><span id="L-1081"><a href="#L-1081"><span class="linenos">1081</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exporting phenotype </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1082"><a href="#L-1082"><span class="linenos">1082</span></a>
+</span><span id="L-1083"><a href="#L-1083"><span class="linenos">1083</span></a> <span class="c1"># validate configuration</span>
+</span><span id="L-1084"><a href="#L-1084"><span class="linenos">1084</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1085"><a href="#L-1085"><span class="linenos">1085</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1086"><a href="#L-1086"><span class="linenos">1086</span></a>
+</span><span id="L-1087"><a href="#L-1087"><span class="linenos">1087</span></a> <span class="c1"># load configuration</span>
+</span><span id="L-1088"><a href="#L-1088"><span class="linenos">1088</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-1089"><a href="#L-1089"><span class="linenos">1089</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-1090"><a href="#L-1090"><span class="linenos">1090</span></a> <span class="n">config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-1091"><a href="#L-1091"><span class="linenos">1091</span></a>
+</span><span id="L-1092"><a href="#L-1092"><span class="linenos">1092</span></a> <span class="n">map_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span>
+</span><span id="L-1093"><a href="#L-1093"><span class="linenos">1093</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">map_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-1094"><a href="#L-1094"><span class="linenos">1094</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Map path does not exist '</span><span class="si">{</span><span class="n">map_path</span><span class="si">}</span><span class="s2">'"</span><span class="p">)</span>
+</span><span id="L-1095"><a href="#L-1095"><span class="linenos">1095</span></a>
+</span><span id="L-1096"><a href="#L-1096"><span class="linenos">1096</span></a> <span class="n">export_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">OMOP_PATH</span>
+</span><span id="L-1097"><a href="#L-1097"><span class="linenos">1097</span></a> <span class="c1"># check export directory exists and if not create it</span>
+</span><span id="L-1098"><a href="#L-1098"><span class="linenos">1098</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">export_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-1099"><a href="#L-1099"><span class="linenos">1099</span></a> <span class="n">export_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1100"><a href="#L-1100"><span class="linenos">1100</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP export directory '</span><span class="si">{</span><span class="n">export_path</span><span class="si">}</span><span class="s2">' created."</span><span class="p">)</span>
+</span><span id="L-1101"><a href="#L-1101"><span class="linenos">1101</span></a>
+</span><span id="L-1102"><a href="#L-1102"><span class="linenos">1102</span></a> <span class="c1"># omop export db</span>
+</span><span id="L-1103"><a href="#L-1103"><span class="linenos">1103</span></a> <span class="n">export_db_path</span> <span class="o">=</span> <span class="n">omop</span><span class="o">.</span><span class="n">export</span><span class="p">(</span>
+</span><span id="L-1104"><a href="#L-1104"><span class="linenos">1104</span></a> <span class="n">map_path</span><span class="p">,</span>
+</span><span id="L-1105"><a href="#L-1105"><span class="linenos">1105</span></a> <span class="n">export_path</span><span class="p">,</span>
+</span><span id="L-1106"><a href="#L-1106"><span class="linenos">1106</span></a> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"version"</span><span class="p">],</span>
+</span><span id="L-1107"><a href="#L-1107"><span class="linenos">1107</span></a> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"omop"</span><span class="p">],</span>
+</span><span id="L-1108"><a href="#L-1108"><span class="linenos">1108</span></a> <span class="p">)</span>
+</span><span id="L-1109"><a href="#L-1109"><span class="linenos">1109</span></a>
+</span><span id="L-1110"><a href="#L-1110"><span class="linenos">1110</span></a> <span class="c1"># write to tables</span>
+</span><span id="L-1111"><a href="#L-1111"><span class="linenos">1111</span></a> <span class="c1"># export as csv</span>
+</span><span id="L-1112"><a href="#L-1112"><span class="linenos">1112</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype exported successfully"</span><span class="p">)</span>
+</span><span id="L-1113"><a href="#L-1113"><span class="linenos">1113</span></a>
+</span><span id="L-1114"><a href="#L-1114"><span class="linenos">1114</span></a>
+</span><span id="L-1115"><a href="#L-1115"><span class="linenos">1115</span></a><span class="k">def</span><span class="w"> </span><span class="nf">copy</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">target_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-1116"><a href="#L-1116"><span class="linenos">1116</span></a><span class="w"> </span><span class="sd">"""Copys a phen repo at a specific tagged version into a target directory"""</span>
+</span><span id="L-1117"><a href="#L-1117"><span class="linenos">1117</span></a>
+</span><span id="L-1118"><a href="#L-1118"><span class="linenos">1118</span></a> <span class="c1"># Validate</span>
+</span><span id="L-1119"><a href="#L-1119"><span class="linenos">1119</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1120"><a href="#L-1120"><span class="linenos">1120</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1121"><a href="#L-1121"><span class="linenos">1121</span></a>
+</span><span id="L-1122"><a href="#L-1122"><span class="linenos">1122</span></a> <span class="c1"># Check target directory exists</span>
+</span><span id="L-1123"><a href="#L-1123"><span class="linenos">1123</span></a> <span class="n">target_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">target_dir</span><span class="p">)</span>
+</span><span id="L-1124"><a href="#L-1124"><span class="linenos">1124</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">target_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-1125"><a href="#L-1125"><span class="linenos">1125</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"The target directory </span><span class="si">{</span><span class="n">target_path</span><span class="si">}</span><span class="s2"> does not exist"</span><span class="p">)</span>
+</span><span id="L-1126"><a href="#L-1126"><span class="linenos">1126</span></a>
+</span><span id="L-1127"><a href="#L-1127"><span class="linenos">1127</span></a> <span class="c1"># Set copy directory</span>
+</span><span id="L-1128"><a href="#L-1128"><span class="linenos">1128</span></a> <span class="n">copy_path</span> <span class="o">=</span> <span class="n">target_path</span> <span class="o">/</span> <span class="n">version</span>
+</span><span id="L-1129"><a href="#L-1129"><span class="linenos">1129</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Copying repo </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">copy_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1130"><a href="#L-1130"><span class="linenos">1130</span></a>
+</span><span id="L-1131"><a href="#L-1131"><span class="linenos">1131</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="L-1132"><a href="#L-1132"><span class="linenos">1132</span></a> <span class="n">copy_path</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">and</span> <span class="n">copy_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span>
+</span><span id="L-1133"><a href="#L-1133"><span class="linenos">1133</span></a> <span class="p">):</span> <span class="c1"># Check if it exists and is a directory</span>
+</span><span id="L-1134"><a href="#L-1134"><span class="linenos">1134</span></a> <span class="n">copy</span> <span class="o">=</span> <span class="n">_check_delete_dir</span><span class="p">(</span>
+</span><span id="L-1135"><a href="#L-1135"><span class="linenos">1135</span></a> <span class="n">copy_path</span><span class="p">,</span>
+</span><span id="L-1136"><a href="#L-1136"><span class="linenos">1136</span></a> <span class="sa">f</span><span class="s2">"The directory </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">copy_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> already exists. Do you want to overwrite? (yes/no): "</span><span class="p">,</span>
+</span><span id="L-1137"><a href="#L-1137"><span class="linenos">1137</span></a> <span class="p">)</span>
+</span><span id="L-1138"><a href="#L-1138"><span class="linenos">1138</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1139"><a href="#L-1139"><span class="linenos">1139</span></a> <span class="n">copy</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-1140"><a href="#L-1140"><span class="linenos">1140</span></a>
+</span><span id="L-1141"><a href="#L-1141"><span class="linenos">1141</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">copy</span><span class="p">:</span>
+</span><span id="L-1142"><a href="#L-1142"><span class="linenos">1142</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Not copying the version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1143"><a href="#L-1143"><span class="linenos">1143</span></a> <span class="k">return</span>
+</span><span id="L-1144"><a href="#L-1144"><span class="linenos">1144</span></a>
+</span><span id="L-1145"><a href="#L-1145"><span class="linenos">1145</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Cloning repo from </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">copy_path</span><span class="si">}</span><span class="s2">..."</span><span class="p">)</span>
+</span><span id="L-1146"><a href="#L-1146"><span class="linenos">1146</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">phen_path</span><span class="p">,</span> <span class="n">copy_path</span><span class="p">)</span>
+</span><span id="L-1147"><a href="#L-1147"><span class="linenos">1147</span></a>
+</span><span id="L-1148"><a href="#L-1148"><span class="linenos">1148</span></a> <span class="c1"># Check out the latest commit or specified version</span>
+</span><span id="L-1149"><a href="#L-1149"><span class="linenos">1149</span></a> <span class="k">if</span> <span class="n">version</span><span class="p">:</span>
+</span><span id="L-1150"><a href="#L-1150"><span class="linenos">1150</span></a> <span class="c1"># Checkout a specific version (e.g., branch, tag, or commit hash)</span>
+</span><span id="L-1151"><a href="#L-1151"><span class="linenos">1151</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Checking out version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">..."</span><span class="p">)</span>
+</span><span id="L-1152"><a href="#L-1152"><span class="linenos">1152</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
+</span><span id="L-1153"><a href="#L-1153"><span class="linenos">1153</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1154"><a href="#L-1154"><span class="linenos">1154</span></a> <span class="c1"># Checkout the latest commit (HEAD)</span>
+</span><span id="L-1155"><a href="#L-1155"><span class="linenos">1155</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Checking out the latest commit..."</span><span class="p">)</span>
+</span><span id="L-1156"><a href="#L-1156"><span class="linenos">1156</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="s2">"HEAD"</span><span class="p">)</span>
+</span><span id="L-1157"><a href="#L-1157"><span class="linenos">1157</span></a>
+</span><span id="L-1158"><a href="#L-1158"><span class="linenos">1158</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Copied </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">repo</span><span class="o">.</span><span class="n">head</span><span class="o">.</span><span class="n">commit</span><span class="o">.</span><span class="n">hexsha</span><span class="p">[:</span><span class="mi">7</span><span class="p">]</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">copy_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1159"><a href="#L-1159"><span class="linenos">1159</span></a>
+</span><span id="L-1160"><a href="#L-1160"><span class="linenos">1160</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype copied successfully"</span><span class="p">)</span>
+</span><span id="L-1161"><a href="#L-1161"><span class="linenos">1161</span></a>
+</span><span id="L-1162"><a href="#L-1162"><span class="linenos">1162</span></a>
+</span><span id="L-1163"><a href="#L-1163"><span class="linenos">1163</span></a><span class="c1"># Convert concept_sets list into dictionaries</span>
+</span><span id="L-1164"><a href="#L-1164"><span class="linenos">1164</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_concepts</span><span class="p">(</span><span class="n">config_data</span><span class="p">:</span> <span class="nb">dict</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
+</span><span id="L-1165"><a href="#L-1165"><span class="linenos">1165</span></a><span class="w"> </span><span class="sd">"""Extracts concepts as {name: file_path} dictionary and a name set."""</span>
+</span><span id="L-1166"><a href="#L-1166"><span class="linenos">1166</span></a> <span class="n">concepts_dict</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="L-1167"><a href="#L-1167"><span class="linenos">1167</span></a> <span class="n">item</span><span class="p">[</span><span class="s2">"name"</span><span class="p">]:</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"path"</span><span class="p">]</span>
+</span><span id="L-1168"><a href="#L-1168"><span class="linenos">1168</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">config_data</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"concept_sets"</span><span class="p">]</span>
+</span><span id="L-1169"><a href="#L-1169"><span class="linenos">1169</span></a> <span class="p">}</span>
+</span><span id="L-1170"><a href="#L-1170"><span class="linenos">1170</span></a> <span class="n">name_set</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">concepts_dict</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
+</span><span id="L-1171"><a href="#L-1171"><span class="linenos">1171</span></a> <span class="k">return</span> <span class="n">concepts_dict</span><span class="p">,</span> <span class="n">name_set</span>
+</span><span id="L-1172"><a href="#L-1172"><span class="linenos">1172</span></a>
+</span><span id="L-1173"><a href="#L-1173"><span class="linenos">1173</span></a>
+</span><span id="L-1174"><a href="#L-1174"><span class="linenos">1174</span></a><span class="k">def</span><span class="w"> </span><span class="nf">_extract_clean_deepdiff_keys</span><span class="p">(</span><span class="n">diff</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">key_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-></span> <span class="n">Set</span><span class="p">[</span><span class="n">Any</span><span class="p">]:</span>
+</span><span id="L-1175"><a href="#L-1175"><span class="linenos">1175</span></a><span class="w"> </span><span class="sd">"""</span>
+</span><span id="L-1176"><a href="#L-1176"><span class="linenos">1176</span></a><span class="sd"> Extracts clean keys from a DeepDiff dictionary.</span>
+</span><span id="L-1177"><a href="#L-1177"><span class="linenos">1177</span></a>
+</span><span id="L-1178"><a href="#L-1178"><span class="linenos">1178</span></a><span class="sd"> :param diff: DeepDiff result dictionary</span>
+</span><span id="L-1179"><a href="#L-1179"><span class="linenos">1179</span></a><span class="sd"> :param key_type: The type of change to extract (e.g., "dictionary_item_added", "dictionary_item_removed")</span>
+</span><span id="L-1180"><a href="#L-1180"><span class="linenos">1180</span></a><span class="sd"> :return: A set of clean key names</span>
+</span><span id="L-1181"><a href="#L-1181"><span class="linenos">1181</span></a><span class="sd"> """</span>
+</span><span id="L-1182"><a href="#L-1182"><span class="linenos">1182</span></a> <span class="k">return</span> <span class="p">{</span><span class="n">key</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"root['"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"']"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="n">diff</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">key_type</span><span class="p">,</span> <span class="p">[])}</span>
+</span><span id="L-1183"><a href="#L-1183"><span class="linenos">1183</span></a>
+</span><span id="L-1184"><a href="#L-1184"><span class="linenos">1184</span></a>
+</span><span id="L-1185"><a href="#L-1185"><span class="linenos">1185</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff_config</span><span class="p">(</span><span class="n">old_config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">new_config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
+</span><span id="L-1186"><a href="#L-1186"><span class="linenos">1186</span></a> <span class="n">report</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="se">\n</span><span class="s2"># Changes to phenotype configuration</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1187"><a href="#L-1187"><span class="linenos">1187</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"This compares changes in the phenotype configuration including added, removed and renamed concept sets and changes to concept set source concept code file paths</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1188"><a href="#L-1188"><span class="linenos">1188</span></a>
+</span><span id="L-1189"><a href="#L-1189"><span class="linenos">1189</span></a> <span class="n">old_concepts</span><span class="p">,</span> <span class="n">old_names</span> <span class="o">=</span> <span class="n">extract_concepts</span><span class="p">(</span><span class="n">old_config</span><span class="p">)</span>
+</span><span id="L-1190"><a href="#L-1190"><span class="linenos">1190</span></a> <span class="n">new_concepts</span><span class="p">,</span> <span class="n">new_names</span> <span class="o">=</span> <span class="n">extract_concepts</span><span class="p">(</span><span class="n">new_config</span><span class="p">)</span>
+</span><span id="L-1191"><a href="#L-1191"><span class="linenos">1191</span></a>
+</span><span id="L-1192"><a href="#L-1192"><span class="linenos">1192</span></a> <span class="c1"># Check added and removed names</span>
+</span><span id="L-1193"><a href="#L-1193"><span class="linenos">1193</span></a> <span class="n">added_names</span> <span class="o">=</span> <span class="n">new_names</span> <span class="o">-</span> <span class="n">old_names</span> <span class="c1"># Names that appear in new but not in old</span>
+</span><span id="L-1194"><a href="#L-1194"><span class="linenos">1194</span></a> <span class="n">removed_names</span> <span class="o">=</span> <span class="n">old_names</span> <span class="o">-</span> <span class="n">new_names</span> <span class="c1"># Names that were in old but not in new</span>
+</span><span id="L-1195"><a href="#L-1195"><span class="linenos">1195</span></a>
+</span><span id="L-1196"><a href="#L-1196"><span class="linenos">1196</span></a> <span class="c1"># find file path changes for unchanged names</span>
+</span><span id="L-1197"><a href="#L-1197"><span class="linenos">1197</span></a> <span class="n">unchanged_names</span> <span class="o">=</span> <span class="n">old_names</span> <span class="o">&</span> <span class="n">new_names</span> <span class="c1"># Names that exist in both</span>
+</span><span id="L-1198"><a href="#L-1198"><span class="linenos">1198</span></a> <span class="n">file_diff</span> <span class="o">=</span> <span class="n">DeepDiff</span><span class="p">(</span>
+</span><span id="L-1199"><a href="#L-1199"><span class="linenos">1199</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">old_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">unchanged_names</span><span class="p">},</span>
+</span><span id="L-1200"><a href="#L-1200"><span class="linenos">1200</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">new_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">unchanged_names</span><span class="p">},</span>
+</span><span id="L-1201"><a href="#L-1201"><span class="linenos">1201</span></a> <span class="p">)</span>
+</span><span id="L-1202"><a href="#L-1202"><span class="linenos">1202</span></a>
+</span><span id="L-1203"><a href="#L-1203"><span class="linenos">1203</span></a> <span class="c1"># Find renamed concepts (same file, different name)</span>
+</span><span id="L-1204"><a href="#L-1204"><span class="linenos">1204</span></a> <span class="n">renamed_concepts</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="L-1205"><a href="#L-1205"><span class="linenos">1205</span></a> <span class="k">for</span> <span class="n">removed</span> <span class="ow">in</span> <span class="n">removed_names</span><span class="p">:</span>
+</span><span id="L-1206"><a href="#L-1206"><span class="linenos">1206</span></a> <span class="n">old_path</span> <span class="o">=</span> <span class="n">old_concepts</span><span class="p">[</span><span class="n">removed</span><span class="p">]</span>
+</span><span id="L-1207"><a href="#L-1207"><span class="linenos">1207</span></a> <span class="k">for</span> <span class="n">added</span> <span class="ow">in</span> <span class="n">added_names</span><span class="p">:</span>
+</span><span id="L-1208"><a href="#L-1208"><span class="linenos">1208</span></a> <span class="n">new_path</span> <span class="o">=</span> <span class="n">new_concepts</span><span class="p">[</span><span class="n">added</span><span class="p">]</span>
+</span><span id="L-1209"><a href="#L-1209"><span class="linenos">1209</span></a> <span class="k">if</span> <span class="n">old_path</span> <span class="o">==</span> <span class="n">new_path</span><span class="p">:</span>
+</span><span id="L-1210"><a href="#L-1210"><span class="linenos">1210</span></a> <span class="n">renamed_concepts</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">removed</span><span class="p">,</span> <span class="n">added</span><span class="p">))</span>
+</span><span id="L-1211"><a href="#L-1211"><span class="linenos">1211</span></a>
+</span><span id="L-1212"><a href="#L-1212"><span class="linenos">1212</span></a> <span class="c1"># Remove renamed concepts from added and removed sets</span>
+</span><span id="L-1213"><a href="#L-1213"><span class="linenos">1213</span></a> <span class="k">for</span> <span class="n">old_name</span><span class="p">,</span> <span class="n">new_name</span> <span class="ow">in</span> <span class="n">renamed_concepts</span><span class="p">:</span>
+</span><span id="L-1214"><a href="#L-1214"><span class="linenos">1214</span></a> <span class="n">added_names</span><span class="o">.</span><span class="n">discard</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
+</span><span id="L-1215"><a href="#L-1215"><span class="linenos">1215</span></a> <span class="n">removed_names</span><span class="o">.</span><span class="n">discard</span><span class="p">(</span><span class="n">old_name</span><span class="p">)</span>
+</span><span id="L-1216"><a href="#L-1216"><span class="linenos">1216</span></a>
+</span><span id="L-1217"><a href="#L-1217"><span class="linenos">1217</span></a> <span class="c1"># generate config report</span>
+</span><span id="L-1218"><a href="#L-1218"><span class="linenos">1218</span></a> <span class="k">if</span> <span class="n">added_names</span><span class="p">:</span>
+</span><span id="L-1219"><a href="#L-1219"><span class="linenos">1219</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Added Concepts</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1220"><a href="#L-1220"><span class="linenos">1220</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">added_names</span><span class="p">:</span>
+</span><span id="L-1221"><a href="#L-1221"><span class="linenos">1221</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">` (File: `</span><span class="si">{</span><span class="n">new_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="si">}</span><span class="s2">`)</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1222"><a href="#L-1222"><span class="linenos">1222</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1223"><a href="#L-1223"><span class="linenos">1223</span></a>
+</span><span id="L-1224"><a href="#L-1224"><span class="linenos">1224</span></a> <span class="k">if</span> <span class="n">removed_names</span><span class="p">:</span>
+</span><span id="L-1225"><a href="#L-1225"><span class="linenos">1225</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Removed Concepts</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1226"><a href="#L-1226"><span class="linenos">1226</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">removed_names</span><span class="p">:</span>
+</span><span id="L-1227"><a href="#L-1227"><span class="linenos">1227</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">` (File: `</span><span class="si">{</span><span class="n">old_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="si">}</span><span class="s2">`)</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1228"><a href="#L-1228"><span class="linenos">1228</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1229"><a href="#L-1229"><span class="linenos">1229</span></a>
+</span><span id="L-1230"><a href="#L-1230"><span class="linenos">1230</span></a> <span class="k">if</span> <span class="n">renamed_concepts</span><span class="p">:</span>
+</span><span id="L-1231"><a href="#L-1231"><span class="linenos">1231</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Renamed Concepts</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1232"><a href="#L-1232"><span class="linenos">1232</span></a> <span class="k">for</span> <span class="n">old_name</span><span class="p">,</span> <span class="n">new_name</span> <span class="ow">in</span> <span class="n">renamed_concepts</span><span class="p">:</span>
+</span><span id="L-1233"><a href="#L-1233"><span class="linenos">1233</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="p">(</span>
+</span><span id="L-1234"><a href="#L-1234"><span class="linenos">1234</span></a> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">old_name</span><span class="si">}</span><span class="s2">` ➝ `</span><span class="si">{</span><span class="n">new_name</span><span class="si">}</span><span class="s2">` (File: `</span><span class="si">{</span><span class="n">old_concepts</span><span class="p">[</span><span class="n">old_name</span><span class="p">]</span><span class="si">}</span><span class="s2">`)</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1235"><a href="#L-1235"><span class="linenos">1235</span></a> <span class="p">)</span>
+</span><span id="L-1236"><a href="#L-1236"><span class="linenos">1236</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1237"><a href="#L-1237"><span class="linenos">1237</span></a>
+</span><span id="L-1238"><a href="#L-1238"><span class="linenos">1238</span></a> <span class="k">if</span> <span class="s2">"values_changed"</span> <span class="ow">in</span> <span class="n">file_diff</span><span class="p">:</span>
+</span><span id="L-1239"><a href="#L-1239"><span class="linenos">1239</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Updated File Paths</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1240"><a href="#L-1240"><span class="linenos">1240</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">change</span> <span class="ow">in</span> <span class="n">file_diff</span><span class="p">[</span><span class="s2">"values_changed"</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
+</span><span id="L-1241"><a href="#L-1241"><span class="linenos">1241</span></a> <span class="n">old_file</span> <span class="o">=</span> <span class="n">change</span><span class="p">[</span><span class="s2">"old_value"</span><span class="p">]</span>
+</span><span id="L-1242"><a href="#L-1242"><span class="linenos">1242</span></a> <span class="n">new_file</span> <span class="o">=</span> <span class="n">change</span><span class="p">[</span><span class="s2">"new_value"</span><span class="p">]</span>
+</span><span id="L-1243"><a href="#L-1243"><span class="linenos">1243</span></a> <span class="n">clean_name</span> <span class="o">=</span> <span class="n">name</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"root['"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"']"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
+</span><span id="L-1244"><a href="#L-1244"><span class="linenos">1244</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="p">(</span>
+</span><span id="L-1245"><a href="#L-1245"><span class="linenos">1245</span></a> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">clean_name</span><span class="si">}</span><span class="s2">` changed file from `</span><span class="si">{</span><span class="n">old_file</span><span class="si">}</span><span class="s2">` ➝ `</span><span class="si">{</span><span class="n">new_file</span><span class="si">}</span><span class="s2">`</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1246"><a href="#L-1246"><span class="linenos">1246</span></a> <span class="p">)</span>
+</span><span id="L-1247"><a href="#L-1247"><span class="linenos">1247</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1248"><a href="#L-1248"><span class="linenos">1248</span></a>
+</span><span id="L-1249"><a href="#L-1249"><span class="linenos">1249</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span>
+</span><span id="L-1250"><a href="#L-1250"><span class="linenos">1250</span></a> <span class="n">added_names</span>
+</span><span id="L-1251"><a href="#L-1251"><span class="linenos">1251</span></a> <span class="ow">or</span> <span class="n">removed_names</span>
+</span><span id="L-1252"><a href="#L-1252"><span class="linenos">1252</span></a> <span class="ow">or</span> <span class="n">renamed_concepts</span>
+</span><span id="L-1253"><a href="#L-1253"><span class="linenos">1253</span></a> <span class="ow">or</span> <span class="n">file_diff</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"values_changed"</span><span class="p">)</span>
+</span><span id="L-1254"><a href="#L-1254"><span class="linenos">1254</span></a> <span class="p">):</span>
+</span><span id="L-1255"><a href="#L-1255"><span class="linenos">1255</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"No changes in concept sets.</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1256"><a href="#L-1256"><span class="linenos">1256</span></a>
+</span><span id="L-1257"><a href="#L-1257"><span class="linenos">1257</span></a> <span class="k">return</span> <span class="n">report</span>
+</span><span id="L-1258"><a href="#L-1258"><span class="linenos">1258</span></a>
+</span><span id="L-1259"><a href="#L-1259"><span class="linenos">1259</span></a>
+</span><span id="L-1260"><a href="#L-1260"><span class="linenos">1260</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff_map_files</span><span class="p">(</span><span class="n">old_map_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">new_map_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
+</span><span id="L-1261"><a href="#L-1261"><span class="linenos">1261</span></a> <span class="n">old_output_files</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-1262"><a href="#L-1262"><span class="linenos">1262</span></a> <span class="n">file</span><span class="o">.</span><span class="n">name</span>
+</span><span id="L-1263"><a href="#L-1263"><span class="linenos">1263</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">old_map_path</span><span class="o">.</span><span class="n">iterdir</span><span class="p">()</span>
+</span><span id="L-1264"><a href="#L-1264"><span class="linenos">1264</span></a> <span class="k">if</span> <span class="n">file</span><span class="o">.</span><span class="n">is_file</span><span class="p">()</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">file</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"."</span><span class="p">)</span>
+</span><span id="L-1265"><a href="#L-1265"><span class="linenos">1265</span></a> <span class="p">]</span>
+</span><span id="L-1266"><a href="#L-1266"><span class="linenos">1266</span></a> <span class="n">new_output_files</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-1267"><a href="#L-1267"><span class="linenos">1267</span></a> <span class="n">file</span><span class="o">.</span><span class="n">name</span>
+</span><span id="L-1268"><a href="#L-1268"><span class="linenos">1268</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">new_map_path</span><span class="o">.</span><span class="n">iterdir</span><span class="p">()</span>
+</span><span id="L-1269"><a href="#L-1269"><span class="linenos">1269</span></a> <span class="k">if</span> <span class="n">file</span><span class="o">.</span><span class="n">is_file</span><span class="p">()</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">file</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"."</span><span class="p">)</span>
+</span><span id="L-1270"><a href="#L-1270"><span class="linenos">1270</span></a> <span class="p">]</span>
+</span><span id="L-1271"><a href="#L-1271"><span class="linenos">1271</span></a>
+</span><span id="L-1272"><a href="#L-1272"><span class="linenos">1272</span></a> <span class="c1"># Convert the lists to sets for easy comparison</span>
+</span><span id="L-1273"><a href="#L-1273"><span class="linenos">1273</span></a> <span class="n">old_output_set</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">old_output_files</span><span class="p">)</span>
+</span><span id="L-1274"><a href="#L-1274"><span class="linenos">1274</span></a> <span class="n">new_output_set</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">new_output_files</span><span class="p">)</span>
+</span><span id="L-1275"><a href="#L-1275"><span class="linenos">1275</span></a>
+</span><span id="L-1276"><a href="#L-1276"><span class="linenos">1276</span></a> <span class="c1"># Outputs that are in old_output_set but not in new_output_set (removed files)</span>
+</span><span id="L-1277"><a href="#L-1277"><span class="linenos">1277</span></a> <span class="n">removed_outputs</span> <span class="o">=</span> <span class="n">old_output_set</span> <span class="o">-</span> <span class="n">new_output_set</span>
+</span><span id="L-1278"><a href="#L-1278"><span class="linenos">1278</span></a> <span class="c1"># Outputs that are in new_output_set but not in old_output_set (added files)</span>
+</span><span id="L-1279"><a href="#L-1279"><span class="linenos">1279</span></a> <span class="n">added_outputs</span> <span class="o">=</span> <span class="n">new_output_set</span> <span class="o">-</span> <span class="n">old_output_set</span>
+</span><span id="L-1280"><a href="#L-1280"><span class="linenos">1280</span></a> <span class="c1"># Outputs that are the intersection of old_output_set and new_output_set</span>
+</span><span id="L-1281"><a href="#L-1281"><span class="linenos">1281</span></a> <span class="n">common_outputs</span> <span class="o">=</span> <span class="n">old_output_set</span> <span class="o">&</span> <span class="n">new_output_set</span>
+</span><span id="L-1282"><a href="#L-1282"><span class="linenos">1282</span></a>
+</span><span id="L-1283"><a href="#L-1283"><span class="linenos">1283</span></a> <span class="n">report</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="se">\n</span><span class="s2"># Changes to available translations</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1284"><a href="#L-1284"><span class="linenos">1284</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"This compares the coding translations files available.</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1285"><a href="#L-1285"><span class="linenos">1285</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Removed outputs: </span><span class="si">{</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">removed_outputs</span><span class="p">))</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1286"><a href="#L-1286"><span class="linenos">1286</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Added outputs: </span><span class="si">{</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">added_outputs</span><span class="p">))</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1287"><a href="#L-1287"><span class="linenos">1287</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Common outputs: </span><span class="si">{</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">common_outputs</span><span class="p">))</span><span class="si">}</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1288"><a href="#L-1288"><span class="linenos">1288</span></a>
+</span><span id="L-1289"><a href="#L-1289"><span class="linenos">1289</span></a> <span class="c1"># Step N: Compare common outputs between versions</span>
+</span><span id="L-1290"><a href="#L-1290"><span class="linenos">1290</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"# Changes to concepts in translation files</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1291"><a href="#L-1291"><span class="linenos">1291</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"This compares the added and removed concepts in each of the coding translation files. Note that this might be different to the config.yaml if the translations have not been run for the current config.</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1292"><a href="#L-1292"><span class="linenos">1292</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">common_outputs</span><span class="p">:</span>
+</span><span id="L-1293"><a href="#L-1293"><span class="linenos">1293</span></a> <span class="n">old_output</span> <span class="o">=</span> <span class="n">old_map_path</span> <span class="o">/</span> <span class="n">file</span>
+</span><span id="L-1294"><a href="#L-1294"><span class="linenos">1294</span></a> <span class="n">new_output</span> <span class="o">=</span> <span class="n">new_map_path</span> <span class="o">/</span> <span class="n">file</span>
+</span><span id="L-1295"><a href="#L-1295"><span class="linenos">1295</span></a>
+</span><span id="L-1296"><a href="#L-1296"><span class="linenos">1296</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Old ouptput: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_output</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1297"><a href="#L-1297"><span class="linenos">1297</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"New ouptput: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">new_output</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1298"><a href="#L-1298"><span class="linenos">1298</span></a>
+</span><span id="L-1299"><a href="#L-1299"><span class="linenos">1299</span></a> <span class="n">df1</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">old_output</span><span class="p">)</span>
+</span><span id="L-1300"><a href="#L-1300"><span class="linenos">1300</span></a> <span class="n">df1</span> <span class="o">=</span> <span class="n">df1</span><span class="p">[[</span><span class="s2">"CONCEPT"</span><span class="p">,</span> <span class="s2">"CONCEPT_SET"</span><span class="p">]]</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">)</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
+</span><span id="L-1301"><a href="#L-1301"><span class="linenos">1301</span></a> <span class="n">df2</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">new_output</span><span class="p">)</span>
+</span><span id="L-1302"><a href="#L-1302"><span class="linenos">1302</span></a> <span class="n">df2</span> <span class="o">=</span> <span class="n">df2</span><span class="p">[[</span><span class="s2">"CONCEPT"</span><span class="p">,</span> <span class="s2">"CONCEPT_SET"</span><span class="p">]]</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">)</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
+</span><span id="L-1303"><a href="#L-1303"><span class="linenos">1303</span></a>
+</span><span id="L-1304"><a href="#L-1304"><span class="linenos">1304</span></a> <span class="c1"># Check for added and removed concepts</span>
+</span><span id="L-1305"><a href="#L-1305"><span class="linenos">1305</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- File </span><span class="si">{</span><span class="n">file</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1306"><a href="#L-1306"><span class="linenos">1306</span></a> <span class="n">sorted_list</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">df1</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">-</span> <span class="nb">set</span><span class="p">(</span><span class="n">df2</span><span class="o">.</span><span class="n">index</span><span class="p">)))</span>
+</span><span id="L-1307"><a href="#L-1307"><span class="linenos">1307</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Removed concepts </span><span class="si">{</span><span class="n">sorted_list</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1308"><a href="#L-1308"><span class="linenos">1308</span></a> <span class="n">sorted_list</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">df2</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">-</span> <span class="nb">set</span><span class="p">(</span><span class="n">df1</span><span class="o">.</span><span class="n">index</span><span class="p">)))</span>
+</span><span id="L-1309"><a href="#L-1309"><span class="linenos">1309</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Added concepts </span><span class="si">{</span><span class="n">sorted_list</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1310"><a href="#L-1310"><span class="linenos">1310</span></a>
+</span><span id="L-1311"><a href="#L-1311"><span class="linenos">1311</span></a> <span class="c1"># Check for changed concepts</span>
+</span><span id="L-1312"><a href="#L-1312"><span class="linenos">1312</span></a> <span class="n">diff</span> <span class="o">=</span> <span class="n">df2</span> <span class="o">-</span> <span class="n">df1</span> <span class="c1"># diff in counts</span>
+</span><span id="L-1313"><a href="#L-1313"><span class="linenos">1313</span></a> <span class="n">diff</span> <span class="o">=</span> <span class="n">diff</span><span class="p">[</span>
+</span><span id="L-1314"><a href="#L-1314"><span class="linenos">1314</span></a> <span class="p">(</span><span class="o">~</span><span class="p">(</span><span class="n">diff</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">]</span> <span class="o">==</span> <span class="mf">0.0</span><span class="p">))</span> <span class="o">&</span> <span class="n">diff</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">]</span><span class="o">.</span><span class="n">notna</span><span class="p">()</span>
+</span><span id="L-1315"><a href="#L-1315"><span class="linenos">1315</span></a> <span class="p">]</span> <span class="c1"># get non-zero counts</span>
+</span><span id="L-1316"><a href="#L-1316"><span class="linenos">1316</span></a> <span class="n">s</span> <span class="o">=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1317"><a href="#L-1317"><span class="linenos">1317</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">diff</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="L-1318"><a href="#L-1318"><span class="linenos">1318</span></a> <span class="k">for</span> <span class="n">concept</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">diff</span><span class="o">.</span><span class="n">iterrows</span><span class="p">():</span>
+</span><span id="L-1319"><a href="#L-1319"><span class="linenos">1319</span></a> <span class="n">s</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\t</span><span class="s2"> - </span><span class="si">{}</span><span class="s2"> </span><span class="si">{}</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">concept</span><span class="p">,</span> <span class="n">row</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="L-1320"><a href="#L-1320"><span class="linenos">1320</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Changed concepts </span><span class="si">{</span><span class="n">s</span><span class="si">}</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1321"><a href="#L-1321"><span class="linenos">1321</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1322"><a href="#L-1322"><span class="linenos">1322</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Changed concepts []</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="L-1323"><a href="#L-1323"><span class="linenos">1323</span></a>
+</span><span id="L-1324"><a href="#L-1324"><span class="linenos">1324</span></a> <span class="k">return</span> <span class="n">report</span>
+</span><span id="L-1325"><a href="#L-1325"><span class="linenos">1325</span></a>
+</span><span id="L-1326"><a href="#L-1326"><span class="linenos">1326</span></a>
+</span><span id="L-1327"><a href="#L-1327"><span class="linenos">1327</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff_phen</span><span class="p">(</span>
+</span><span id="L-1328"><a href="#L-1328"><span class="linenos">1328</span></a> <span class="n">new_phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span>
+</span><span id="L-1329"><a href="#L-1329"><span class="linenos">1329</span></a> <span class="n">new_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
+</span><span id="L-1330"><a href="#L-1330"><span class="linenos">1330</span></a> <span class="n">old_phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span>
+</span><span id="L-1331"><a href="#L-1331"><span class="linenos">1331</span></a> <span class="n">old_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
+</span><span id="L-1332"><a href="#L-1332"><span class="linenos">1332</span></a> <span class="n">report_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span>
+</span><span id="L-1333"><a href="#L-1333"><span class="linenos">1333</span></a><span class="p">):</span>
+</span><span id="L-1334"><a href="#L-1334"><span class="linenos">1334</span></a><span class="w"> </span><span class="sd">"""Compare the differences between two versions of a phenotype"""</span>
+</span><span id="L-1335"><a href="#L-1335"><span class="linenos">1335</span></a>
+</span><span id="L-1336"><a href="#L-1336"><span class="linenos">1336</span></a> <span class="c1"># validate phenotypes</span>
+</span><span id="L-1337"><a href="#L-1337"><span class="linenos">1337</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Validating for diff old path: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1338"><a href="#L-1338"><span class="linenos">1338</span></a> <span class="n">validate</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()))</span>
+</span><span id="L-1339"><a href="#L-1339"><span class="linenos">1339</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Validating for diff new path: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">new_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1340"><a href="#L-1340"><span class="linenos">1340</span></a> <span class="n">validate</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">new_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()))</span>
+</span><span id="L-1341"><a href="#L-1341"><span class="linenos">1341</span></a>
+</span><span id="L-1342"><a href="#L-1342"><span class="linenos">1342</span></a> <span class="c1"># get old and new config</span>
+</span><span id="L-1343"><a href="#L-1343"><span class="linenos">1343</span></a> <span class="n">old_config_path</span> <span class="o">=</span> <span class="n">old_phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-1344"><a href="#L-1344"><span class="linenos">1344</span></a> <span class="k">with</span> <span class="n">old_config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-1345"><a href="#L-1345"><span class="linenos">1345</span></a> <span class="n">old_config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-1346"><a href="#L-1346"><span class="linenos">1346</span></a> <span class="n">new_config_path</span> <span class="o">=</span> <span class="n">new_phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="L-1347"><a href="#L-1347"><span class="linenos">1347</span></a> <span class="k">with</span> <span class="n">new_config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-1348"><a href="#L-1348"><span class="linenos">1348</span></a> <span class="n">new_config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="L-1349"><a href="#L-1349"><span class="linenos">1349</span></a>
+</span><span id="L-1350"><a href="#L-1350"><span class="linenos">1350</span></a> <span class="c1"># write report heading</span>
+</span><span id="L-1351"><a href="#L-1351"><span class="linenos">1351</span></a> <span class="n">report</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"# Phenotype Comparison Report</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1352"><a href="#L-1352"><span class="linenos">1352</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"## Original phenotype</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1353"><a href="#L-1353"><span class="linenos">1353</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">old_config</span><span class="p">[</span><span class="s1">'phenotype'</span><span class="p">][</span><span class="s1">'omop'</span><span class="p">][</span><span class="s1">'vocabulary_id'</span><span class="p">]</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1354"><a href="#L-1354"><span class="linenos">1354</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1355"><a href="#L-1355"><span class="linenos">1355</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1356"><a href="#L-1356"><span class="linenos">1356</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"## Changed phenotype:</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1357"><a href="#L-1357"><span class="linenos">1357</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">new_config</span><span class="p">[</span><span class="s1">'phenotype'</span><span class="p">][</span><span class="s1">'omop'</span><span class="p">][</span><span class="s1">'vocabulary_id'</span><span class="p">]</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1358"><a href="#L-1358"><span class="linenos">1358</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">new_version</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1359"><a href="#L-1359"><span class="linenos">1359</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">new_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="L-1360"><a href="#L-1360"><span class="linenos">1360</span></a>
+</span><span id="L-1361"><a href="#L-1361"><span class="linenos">1361</span></a> <span class="c1"># Step 1: check differences configuration files</span>
+</span><span id="L-1362"><a href="#L-1362"><span class="linenos">1362</span></a> <span class="c1"># Convert list of dicts into a dict: {name: file}</span>
+</span><span id="L-1363"><a href="#L-1363"><span class="linenos">1363</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="n">diff_config</span><span class="p">(</span><span class="n">old_config</span><span class="p">,</span> <span class="n">new_config</span><span class="p">)</span>
+</span><span id="L-1364"><a href="#L-1364"><span class="linenos">1364</span></a>
+</span><span id="L-1365"><a href="#L-1365"><span class="linenos">1365</span></a> <span class="c1"># Step 2: check differences between map files</span>
+</span><span id="L-1366"><a href="#L-1366"><span class="linenos">1366</span></a> <span class="c1"># List files from output directories</span>
+</span><span id="L-1367"><a href="#L-1367"><span class="linenos">1367</span></a> <span class="n">old_map_path</span> <span class="o">=</span> <span class="n">old_phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span>
+</span><span id="L-1368"><a href="#L-1368"><span class="linenos">1368</span></a> <span class="n">new_map_path</span> <span class="o">=</span> <span class="n">new_phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span>
+</span><span id="L-1369"><a href="#L-1369"><span class="linenos">1369</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="n">diff_map_files</span><span class="p">(</span><span class="n">old_map_path</span><span class="p">,</span> <span class="n">new_map_path</span><span class="p">)</span>
+</span><span id="L-1370"><a href="#L-1370"><span class="linenos">1370</span></a>
+</span><span id="L-1371"><a href="#L-1371"><span class="linenos">1371</span></a> <span class="c1"># initialise report file</span>
+</span><span id="L-1372"><a href="#L-1372"><span class="linenos">1372</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Writing to report file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">report_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1373"><a href="#L-1373"><span class="linenos">1373</span></a> <span class="n">report_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">report_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span>
+</span><span id="L-1374"><a href="#L-1374"><span class="linenos">1374</span></a> <span class="n">report_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">report</span><span class="p">)</span>
+</span><span id="L-1375"><a href="#L-1375"><span class="linenos">1375</span></a> <span class="n">report_file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="L-1376"><a href="#L-1376"><span class="linenos">1376</span></a>
+</span><span id="L-1377"><a href="#L-1377"><span class="linenos">1377</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotypes diff'd successfully"</span><span class="p">)</span>
+</span><span id="L-1378"><a href="#L-1378"><span class="linenos">1378</span></a>
+</span><span id="L-1379"><a href="#L-1379"><span class="linenos">1379</span></a>
+</span><span id="L-1380"><a href="#L-1380"><span class="linenos">1380</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">old_phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">old_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-1381"><a href="#L-1381"><span class="linenos">1381</span></a> <span class="c1"># make tmp directory .acmc</span>
+</span><span id="L-1382"><a href="#L-1382"><span class="linenos">1382</span></a> <span class="n">timestamp</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">"%Y%m</span><span class="si">%d</span><span class="s2">_%H%M%S"</span><span class="p">)</span>
+</span><span id="L-1383"><a href="#L-1383"><span class="linenos">1383</span></a> <span class="n">temp_dir</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="sa">f</span><span class="s2">".acmc/diff_</span><span class="si">{</span><span class="n">timestamp</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1384"><a href="#L-1384"><span class="linenos">1384</span></a>
+</span><span id="L-1385"><a href="#L-1385"><span class="linenos">1385</span></a> <span class="n">changed_phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="L-1386"><a href="#L-1386"><span class="linenos">1386</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">changed_phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-1387"><a href="#L-1387"><span class="linenos">1387</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-1388"><a href="#L-1388"><span class="linenos">1388</span></a> <span class="sa">f</span><span class="s2">"Changed phenotype directory does not exist: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">changed_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-1389"><a href="#L-1389"><span class="linenos">1389</span></a> <span class="p">)</span>
+</span><span id="L-1390"><a href="#L-1390"><span class="linenos">1390</span></a>
+</span><span id="L-1391"><a href="#L-1391"><span class="linenos">1391</span></a> <span class="n">old_phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">old_phen_dir</span><span class="p">)</span>
+</span><span id="L-1392"><a href="#L-1392"><span class="linenos">1392</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">old_phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-1393"><a href="#L-1393"><span class="linenos">1393</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-1394"><a href="#L-1394"><span class="linenos">1394</span></a> <span class="sa">f</span><span class="s2">"Old phenotype directory does not exist: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-1395"><a href="#L-1395"><span class="linenos">1395</span></a> <span class="p">)</span>
+</span><span id="L-1396"><a href="#L-1396"><span class="linenos">1396</span></a>
+</span><span id="L-1397"><a href="#L-1397"><span class="linenos">1397</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="L-1398"><a href="#L-1398"><span class="linenos">1398</span></a> <span class="c1"># Create the directory</span>
+</span><span id="L-1399"><a href="#L-1399"><span class="linenos">1399</span></a> <span class="n">temp_dir</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1400"><a href="#L-1400"><span class="linenos">1400</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Temporary directory created: </span><span class="si">{</span><span class="n">temp_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-1401"><a href="#L-1401"><span class="linenos">1401</span></a>
+</span><span id="L-1402"><a href="#L-1402"><span class="linenos">1402</span></a> <span class="c1"># Create temporary directories</span>
+</span><span id="L-1403"><a href="#L-1403"><span class="linenos">1403</span></a> <span class="n">changed_path</span> <span class="o">=</span> <span class="n">temp_dir</span> <span class="o">/</span> <span class="s2">"changed"</span>
+</span><span id="L-1404"><a href="#L-1404"><span class="linenos">1404</span></a> <span class="n">changed_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1405"><a href="#L-1405"><span class="linenos">1405</span></a> <span class="n">old_path</span> <span class="o">=</span> <span class="n">temp_dir</span> <span class="o">/</span> <span class="s2">"old"</span>
+</span><span id="L-1406"><a href="#L-1406"><span class="linenos">1406</span></a> <span class="n">old_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1407"><a href="#L-1407"><span class="linenos">1407</span></a>
+</span><span id="L-1408"><a href="#L-1408"><span class="linenos">1408</span></a> <span class="c1"># checkout changed</span>
+</span><span id="L-1409"><a href="#L-1409"><span class="linenos">1409</span></a> <span class="k">if</span> <span class="n">version</span> <span class="o">==</span> <span class="s2">"latest"</span><span class="p">:</span>
+</span><span id="L-1410"><a href="#L-1410"><span class="linenos">1410</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-1411"><a href="#L-1411"><span class="linenos">1411</span></a> <span class="sa">f</span><span class="s2">"Copying changed repo from </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">changed_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="L-1412"><a href="#L-1412"><span class="linenos">1412</span></a> <span class="p">)</span>
+</span><span id="L-1413"><a href="#L-1413"><span class="linenos">1413</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">copytree</span><span class="p">(</span><span class="n">changed_phen_path</span><span class="p">,</span> <span class="n">changed_path</span><span class="p">,</span> <span class="n">dirs_exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1414"><a href="#L-1414"><span class="linenos">1414</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1415"><a href="#L-1415"><span class="linenos">1415</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-1416"><a href="#L-1416"><span class="linenos">1416</span></a> <span class="sa">f</span><span class="s2">"Cloning changed repo from </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">changed_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="L-1417"><a href="#L-1417"><span class="linenos">1417</span></a> <span class="p">)</span>
+</span><span id="L-1418"><a href="#L-1418"><span class="linenos">1418</span></a> <span class="n">changed_repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">changed_phen_path</span><span class="p">,</span> <span class="n">changed_path</span><span class="p">)</span>
+</span><span id="L-1419"><a href="#L-1419"><span class="linenos">1419</span></a> <span class="n">changed_repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
+</span><span id="L-1420"><a href="#L-1420"><span class="linenos">1420</span></a>
+</span><span id="L-1421"><a href="#L-1421"><span class="linenos">1421</span></a> <span class="c1"># checkout old</span>
+</span><span id="L-1422"><a href="#L-1422"><span class="linenos">1422</span></a> <span class="k">if</span> <span class="n">old_version</span> <span class="o">==</span> <span class="s2">"latest"</span><span class="p">:</span>
+</span><span id="L-1423"><a href="#L-1423"><span class="linenos">1423</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-1424"><a href="#L-1424"><span class="linenos">1424</span></a> <span class="sa">f</span><span class="s2">"Copying old repo from </span><span class="si">{</span><span class="n">old_phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">old_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="L-1425"><a href="#L-1425"><span class="linenos">1425</span></a> <span class="p">)</span>
+</span><span id="L-1426"><a href="#L-1426"><span class="linenos">1426</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">copytree</span><span class="p">(</span><span class="n">old_phen_path</span><span class="p">,</span> <span class="n">old_path</span><span class="p">,</span> <span class="n">dirs_exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-1427"><a href="#L-1427"><span class="linenos">1427</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-1428"><a href="#L-1428"><span class="linenos">1428</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="L-1429"><a href="#L-1429"><span class="linenos">1429</span></a> <span class="sa">f</span><span class="s2">"Cloning old repo from </span><span class="si">{</span><span class="n">old_phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">old_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="L-1430"><a href="#L-1430"><span class="linenos">1430</span></a> <span class="p">)</span>
+</span><span id="L-1431"><a href="#L-1431"><span class="linenos">1431</span></a> <span class="n">old_repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">old_phen_dir</span><span class="p">,</span> <span class="n">old_path</span><span class="p">)</span>
+</span><span id="L-1432"><a href="#L-1432"><span class="linenos">1432</span></a> <span class="n">old_repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">old_version</span><span class="p">)</span>
+</span><span id="L-1433"><a href="#L-1433"><span class="linenos">1433</span></a>
+</span><span id="L-1434"><a href="#L-1434"><span class="linenos">1434</span></a> <span class="n">report_filename</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">_</span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="s2">_diff.md"</span>
+</span><span id="L-1435"><a href="#L-1435"><span class="linenos">1435</span></a> <span class="n">report_path</span> <span class="o">=</span> <span class="n">changed_phen_path</span> <span class="o">/</span> <span class="n">report_filename</span>
+</span><span id="L-1436"><a href="#L-1436"><span class="linenos">1436</span></a> <span class="c1"># diff old with new</span>
+</span><span id="L-1437"><a href="#L-1437"><span class="linenos">1437</span></a> <span class="n">diff_phen</span><span class="p">(</span><span class="n">changed_path</span><span class="p">,</span> <span class="n">version</span><span class="p">,</span> <span class="n">old_path</span><span class="p">,</span> <span class="n">old_version</span><span class="p">,</span> <span class="n">report_path</span><span class="p">)</span>
+</span><span id="L-1438"><a href="#L-1438"><span class="linenos">1438</span></a>
+</span><span id="L-1439"><a href="#L-1439"><span class="linenos">1439</span></a> <span class="k">finally</span><span class="p">:</span>
+</span><span id="L-1440"><a href="#L-1440"><span class="linenos">1440</span></a> <span class="c1"># clean up tmp directory</span>
+</span><span id="L-1441"><a href="#L-1441"><span class="linenos">1441</span></a> <span class="k">if</span> <span class="n">temp_dir</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-1442"><a href="#L-1442"><span class="linenos">1442</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">temp_dir</span><span class="p">)</span>
+</span><span id="L-1443"><a href="#L-1443"><span class="linenos">1443</span></a> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Temporary directory removed: </span><span class="si">{</span><span class="n">temp_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="PHEN_DIR">
+ <div class="attr variable">
+ <span class="name">PHEN_DIR</span> =
+<span class="default_value">'phen'</span>
+
+
+ </div>
+ <a class="headerlink" href="#PHEN_DIR"></a>
+
+ <div class="docstring"><p>Default phenotype directory name</p>
+</div>
+
+
+ </section>
+ <section id="DEFAULT_PHEN_PATH">
+ <div class="attr variable">
+ <span class="name">DEFAULT_PHEN_PATH</span> =
+<span class="default_value">PosixPath('workspace/phen')</span>
+
+
+ </div>
+ <a class="headerlink" href="#DEFAULT_PHEN_PATH"></a>
+
+ <div class="docstring"><p>Default phenotype directory path</p>
+</div>
+
+
+ </section>
+ <section id="CONCEPTS_DIR">
+ <div class="attr variable">
+ <span class="name">CONCEPTS_DIR</span> =
+<span class="default_value">'concepts'</span>
+
+
+ </div>
+ <a class="headerlink" href="#CONCEPTS_DIR"></a>
+
+ <div class="docstring"><p>Default concepts directory name</p>
+</div>
+
+
+ </section>
+ <section id="MAP_DIR">
+ <div class="attr variable">
+ <span class="name">MAP_DIR</span> =
+<span class="default_value">'map'</span>
+
+
+ </div>
+ <a class="headerlink" href="#MAP_DIR"></a>
+
+ <div class="docstring"><p>Default map directory name</p>
+</div>
+
+
+ </section>
+ <section id="CONCEPT_SET_DIR">
+ <div class="attr variable">
+ <span class="name">CONCEPT_SET_DIR</span> =
+<span class="default_value">'concept-sets'</span>
+
+
+ </div>
+ <a class="headerlink" href="#CONCEPT_SET_DIR"></a>
+
+ <div class="docstring"><p>Default concept set directory name</p>
+</div>
+
+
+ </section>
+ <section id="CSV_PATH">
+ <div class="attr variable">
+ <span class="name">CSV_PATH</span> =
+<span class="default_value">PosixPath('concept-sets/csv')</span>
+
+
+ </div>
+ <a class="headerlink" href="#CSV_PATH"></a>
+
+ <div class="docstring"><p>Default CSV concept set directory path</p>
+</div>
+
+
+ </section>
+ <section id="OMOP_PATH">
+ <div class="attr variable">
+ <span class="name">OMOP_PATH</span> =
+<span class="default_value">PosixPath('concept-sets/omop')</span>
+
+
+ </div>
+ <a class="headerlink" href="#OMOP_PATH"></a>
+
+ <div class="docstring"><p>Default OMOP concept set directory path</p>
+</div>
+
+
+ </section>
+ <section id="DEFAULT_PHEN_DIR_LIST">
+ <div class="attr variable">
+ <span class="name">DEFAULT_PHEN_DIR_LIST</span> =
+<span class="default_value">['concepts', 'map', 'concept-sets']</span>
+
+
+ </div>
+ <a class="headerlink" href="#DEFAULT_PHEN_DIR_LIST"></a>
+
+ <div class="docstring"><p>List of default phenotype directories</p>
+</div>
+
+
+ </section>
+ <section id="CONFIG_FILE">
+ <div class="attr variable">
+ <span class="name">CONFIG_FILE</span> =
+<span class="default_value">'config.yml'</span>
+
+
+ </div>
+ <a class="headerlink" href="#CONFIG_FILE"></a>
+
+ <div class="docstring"><p>Default configuration filename</p>
+</div>
+
+
+ </section>
+ <section id="VOCAB_VERSION_FILE">
+ <div class="attr variable">
+ <span class="name">VOCAB_VERSION_FILE</span> =
+<span class="default_value">'vocab_version.yml'</span>
+
+
+ </div>
+ <a class="headerlink" href="#VOCAB_VERSION_FILE"></a>
+
+ <div class="docstring"><p>Default vocabulary version filename</p>
+</div>
+
+
+ </section>
+ <section id="SEMANTIC_VERSION_TYPES">
+ <div class="attr variable">
+ <span class="name">SEMANTIC_VERSION_TYPES</span> =
+<span class="default_value">['major', 'minor', 'patch']</span>
+
+
+ </div>
+ <a class="headerlink" href="#SEMANTIC_VERSION_TYPES"></a>
+
+ <div class="docstring"><p>List of semantic version increment types</p>
+</div>
+
+
+ </section>
+ <section id="DEFAULT_VERSION_INC">
+ <div class="attr variable">
+ <span class="name">DEFAULT_VERSION_INC</span> =
+<span class="default_value">'patch'</span>
+
+
+ </div>
+ <a class="headerlink" href="#DEFAULT_VERSION_INC"></a>
+
+ <div class="docstring"><p>Default semantic version increment type</p>
+</div>
+
+
+ </section>
+ <section id="DEFAULT_GIT_BRANCH">
+ <div class="attr variable">
+ <span class="name">DEFAULT_GIT_BRANCH</span> =
+<span class="default_value">'main'</span>
+
+
+ </div>
+ <a class="headerlink" href="#DEFAULT_GIT_BRANCH"></a>
+
+ <div class="docstring"><p>Default phenotype repo branch name</p>
+</div>
+
+
+ </section>
+ <section id="SPLIT_COL_ACTION">
+ <div class="attr variable">
+ <span class="name">SPLIT_COL_ACTION</span> =
+<span class="default_value">'split_col'</span>
+
+
+ </div>
+ <a class="headerlink" href="#SPLIT_COL_ACTION"></a>
+
+ <div class="docstring"><p>Split column preprocessing action type</p>
+</div>
+
+
+ </section>
+ <section id="CODES_COL_ACTION">
+ <div class="attr variable">
+ <span class="name">CODES_COL_ACTION</span> =
+<span class="default_value">'codes_col'</span>
+
+
+ </div>
+ <a class="headerlink" href="#CODES_COL_ACTION"></a>
+
+ <div class="docstring"><p>Codes column preprocessing action type</p>
+</div>
+
+
+ </section>
+ <section id="DIVIDE_COL_ACTION">
+ <div class="attr variable">
+ <span class="name">DIVIDE_COL_ACTION</span> =
+<span class="default_value">'divide_col'</span>
+
+
+ </div>
+ <a class="headerlink" href="#DIVIDE_COL_ACTION"></a>
+
+ <div class="docstring"><p>Divide column preprocessing action type</p>
+</div>
+
+
+ </section>
+ <section id="COL_ACTIONS">
+ <div class="attr variable">
+ <span class="name">COL_ACTIONS</span> =
+<span class="default_value">['split_col', 'codes_col', 'divide_col']</span>
+
+
+ </div>
+ <a class="headerlink" href="#COL_ACTIONS"></a>
+
+ <div class="docstring"><p>List of column preprocessing action types</p>
+</div>
+
+
+ </section>
+ <section id="CODE_FILE_TYPES">
+ <div class="attr variable">
+ <span class="name">CODE_FILE_TYPES</span> =
+<span class="default_value">['.xlsx', '.xls', '.csv']</span>
+
+
+ </div>
+ <a class="headerlink" href="#CODE_FILE_TYPES"></a>
+
+ <div class="docstring"><p>List of supported source concept coding list file types</p>
+</div>
+
+
+ </section>
+ <section id="CONFIG_SCHEMA">
+ <div class="attr variable">
+ <span class="name">CONFIG_SCHEMA</span> =
+<input id="CONFIG_SCHEMA-view-value" class="view-value-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+ <label class="view-value-button pdoc-button" for="CONFIG_SCHEMA-view-value"></label><span class="default_value">{'phenotype': {'type': 'dict', 'required': True, 'schema': {'version': {'type': 'string', 'required': True, 'regex': '^\\d+\\.\\d+\\.\\d+$'}, 'omop': {'type': 'dict', 'required': True, 'schema': {'vocabulary_id': {'type': 'string', 'required': True}, 'vocabulary_name': {'type': 'string', 'required': True}, 'vocabulary_reference': {'type': 'string', 'required': True, 'regex': '^https?://.*'}}}, 'map': {'type': 'list', 'schema': {'type': 'string', 'allowed': ['snomed', 'atc', 'read2', 'read3', 'icd10', 'opcs4']}}, 'concept_sets': {'type': 'list', 'required': True, 'schema': {'type': 'dict', 'schema': {'name': {'type': 'string', 'required': True}, 'file': {'type': 'dict', 'required': False, 'schema': {'path': {'type': 'string', 'required': True}, 'columns': {'type': 'dict', 'required': True}, 'category': {'type': 'string'}, 'actions': {'type': 'dict', 'schema': {'divide_col': {'type': 'string'}}}}}, 'metadata': {'type': 'dict', 'required': True}}}}}}}</span>
+
+
+ </div>
+ <a class="headerlink" href="#CONFIG_SCHEMA"></a>
+
+ <div class="docstring"><p>Phenotype config.yml schema definition</p>
+</div>
+
+
+ </section>
+ <section id="PhenValidationException">
+ <input id="PhenValidationException-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">PhenValidationException</span><wbr>(<span class="base">builtins.Exception</span>):
+
+ <label class="view-source-button" for="PhenValidationException-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#PhenValidationException"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="PhenValidationException-156"><a href="#PhenValidationException-156"><span class="linenos">156</span></a><span class="k">class</span><span class="w"> </span><span class="nc">PhenValidationException</span><span class="p">(</span><span class="ne">Exception</span><span class="p">):</span>
+</span><span id="PhenValidationException-157"><a href="#PhenValidationException-157"><span class="linenos">157</span></a><span class="w"> </span><span class="sd">"""Custom exception class raised when validation errors in phenotype configuration file"""</span>
+</span><span id="PhenValidationException-158"><a href="#PhenValidationException-158"><span class="linenos">158</span></a>
+</span><span id="PhenValidationException-159"><a href="#PhenValidationException-159"><span class="linenos">159</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">validation_errors</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+</span><span id="PhenValidationException-160"><a href="#PhenValidationException-160"><span class="linenos">160</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">message</span><span class="p">)</span>
+</span><span id="PhenValidationException-161"><a href="#PhenValidationException-161"><span class="linenos">161</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">validation_errors</span> <span class="o">=</span> <span class="n">validation_errors</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Custom exception class raised when validation errors in phenotype configuration file</p>
+</div>
+
+
+ <div id="PhenValidationException.__init__" class="classattr">
+ <input id="PhenValidationException.__init__-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="name">PhenValidationException</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">message</span>, </span><span class="param"><span class="n">validation_errors</span><span class="o">=</span><span class="kc">None</span></span>)</span>
+
+ <label class="view-source-button" for="PhenValidationException.__init__-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#PhenValidationException.__init__"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="PhenValidationException.__init__-159"><a href="#PhenValidationException.__init__-159"><span class="linenos">159</span></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">message</span><span class="p">,</span> <span class="n">validation_errors</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
+</span><span id="PhenValidationException.__init__-160"><a href="#PhenValidationException.__init__-160"><span class="linenos">160</span></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">message</span><span class="p">)</span>
+</span><span id="PhenValidationException.__init__-161"><a href="#PhenValidationException.__init__-161"><span class="linenos">161</span></a> <span class="bp">self</span><span class="o">.</span><span class="n">validation_errors</span> <span class="o">=</span> <span class="n">validation_errors</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ <div id="PhenValidationException.validation_errors" class="classattr">
+ <div class="attr variable">
+ <span class="name">validation_errors</span>
+
+
+ </div>
+ <a class="headerlink" href="#PhenValidationException.validation_errors"></a>
+
+
+
+ </div>
+ </section>
+ <section id="init">
+ <input id="init-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">init</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="init-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#init"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="init-230"><a href="#init-230"><span class="linenos">230</span></a><span class="k">def</span><span class="w"> </span><span class="nf">init</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="init-231"><a href="#init-231"><span class="linenos">231</span></a><span class="w"> </span><span class="sd">"""Initial phenotype directory as git repo with standard structure"""</span>
+</span><span id="init-232"><a href="#init-232"><span class="linenos">232</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Initialising Phenotype in directory: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="init-233"><a href="#init-233"><span class="linenos">233</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="init-234"><a href="#init-234"><span class="linenos">234</span></a>
+</span><span id="init-235"><a href="#init-235"><span class="linenos">235</span></a> <span class="c1"># check if directory already exists and ask user if they want to recreate it</span>
+</span><span id="init-236"><a href="#init-236"><span class="linenos">236</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="init-237"><a href="#init-237"><span class="linenos">237</span></a> <span class="n">phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">and</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span>
+</span><span id="init-238"><a href="#init-238"><span class="linenos">238</span></a> <span class="p">):</span> <span class="c1"># Check if it exists and is a directory</span>
+</span><span id="init-239"><a href="#init-239"><span class="linenos">239</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="n">_check_delete_dir</span><span class="p">(</span>
+</span><span id="init-240"><a href="#init-240"><span class="linenos">240</span></a> <span class="n">phen_path</span><span class="p">,</span>
+</span><span id="init-241"><a href="#init-241"><span class="linenos">241</span></a> <span class="sa">f</span><span class="s2">"The phen directory already exists. Do you want to reinitialise? (yes/no): "</span><span class="p">,</span>
+</span><span id="init-242"><a href="#init-242"><span class="linenos">242</span></a> <span class="p">)</span>
+</span><span id="init-243"><a href="#init-243"><span class="linenos">243</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="init-244"><a href="#init-244"><span class="linenos">244</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="init-245"><a href="#init-245"><span class="linenos">245</span></a>
+</span><span id="init-246"><a href="#init-246"><span class="linenos">246</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">configure</span><span class="p">:</span>
+</span><span id="init-247"><a href="#init-247"><span class="linenos">247</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exiting, phenotype not initiatised"</span><span class="p">)</span>
+</span><span id="init-248"><a href="#init-248"><span class="linenos">248</span></a> <span class="k">return</span>
+</span><span id="init-249"><a href="#init-249"><span class="linenos">249</span></a>
+</span><span id="init-250"><a href="#init-250"><span class="linenos">250</span></a> <span class="c1"># Initialise repo from local or remote</span>
+</span><span id="init-251"><a href="#init-251"><span class="linenos">251</span></a> <span class="n">repo</span><span class="p">:</span> <span class="n">Repo</span>
+</span><span id="init-252"><a href="#init-252"><span class="linenos">252</span></a>
+</span><span id="init-253"><a href="#init-253"><span class="linenos">253</span></a> <span class="c1"># if remote then clone the repo otherwise init a local repo</span>
+</span><span id="init-254"><a href="#init-254"><span class="linenos">254</span></a> <span class="k">if</span> <span class="n">remote_url</span> <span class="o">!=</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="init-255"><a href="#init-255"><span class="linenos">255</span></a> <span class="c1"># add PAT token to the URL</span>
+</span><span id="init-256"><a href="#init-256"><span class="linenos">256</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">remote_url</span><span class="p">)</span>
+</span><span id="init-257"><a href="#init-257"><span class="linenos">257</span></a>
+</span><span id="init-258"><a href="#init-258"><span class="linenos">258</span></a> <span class="c1"># clone the repo</span>
+</span><span id="init-259"><a href="#init-259"><span class="linenos">259</span></a> <span class="n">git_cmd</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">cmd</span><span class="o">.</span><span class="n">Git</span><span class="p">()</span>
+</span><span id="init-260"><a href="#init-260"><span class="linenos">260</span></a> <span class="n">git_cmd</span><span class="o">.</span><span class="n">clone</span><span class="p">(</span><span class="n">git_url</span><span class="p">,</span> <span class="n">phen_path</span><span class="p">)</span>
+</span><span id="init-261"><a href="#init-261"><span class="linenos">261</span></a>
+</span><span id="init-262"><a href="#init-262"><span class="linenos">262</span></a> <span class="c1"># open repo</span>
+</span><span id="init-263"><a href="#init-263"><span class="linenos">263</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">Repo</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="init-264"><a href="#init-264"><span class="linenos">264</span></a> <span class="c1"># check if there are any commits (new repo has no commits)</span>
+</span><span id="init-265"><a href="#init-265"><span class="linenos">265</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="init-266"><a href="#init-266"><span class="linenos">266</span></a> <span class="nb">len</span><span class="p">(</span><span class="n">repo</span><span class="o">.</span><span class="n">branches</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">repo</span><span class="o">.</span><span class="n">head</span><span class="o">.</span><span class="n">is_detached</span>
+</span><span id="init-267"><a href="#init-267"><span class="linenos">267</span></a> <span class="p">):</span> <span class="c1"># Handle detached HEAD (e.g., after init)</span>
+</span><span id="init-268"><a href="#init-268"><span class="linenos">268</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"The phen repository has no commits yet."</span><span class="p">)</span>
+</span><span id="init-269"><a href="#init-269"><span class="linenos">269</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="mi">0</span>
+</span><span id="init-270"><a href="#init-270"><span class="linenos">270</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="init-271"><a href="#init-271"><span class="linenos">271</span></a> <span class="c1"># Get the total number of commits in the default branch</span>
+</span><span id="init-272"><a href="#init-272"><span class="linenos">272</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="nb">sum</span><span class="p">(</span><span class="mi">1</span> <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">iter_commits</span><span class="p">())</span>
+</span><span id="init-273"><a href="#init-273"><span class="linenos">273</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Repo has previous commits: </span><span class="si">{</span><span class="n">commit_count</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="init-274"><a href="#init-274"><span class="linenos">274</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="init-275"><a href="#init-275"><span class="linenos">275</span></a> <span class="c1"># local repo, create the directories and init</span>
+</span><span id="init-276"><a href="#init-276"><span class="linenos">276</span></a> <span class="n">phen_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="init-277"><a href="#init-277"><span class="linenos">277</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phen directory '</span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2">' has been created."</span><span class="p">)</span>
+</span><span id="init-278"><a href="#init-278"><span class="linenos">278</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">init</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="init-279"><a href="#init-279"><span class="linenos">279</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="mi">0</span>
+</span><span id="init-280"><a href="#init-280"><span class="linenos">280</span></a>
+</span><span id="init-281"><a href="#init-281"><span class="linenos">281</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()</span>
+</span><span id="init-282"><a href="#init-282"><span class="linenos">282</span></a> <span class="c1"># initialise empty repos</span>
+</span><span id="init-283"><a href="#init-283"><span class="linenos">283</span></a> <span class="k">if</span> <span class="n">commit_count</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="init-284"><a href="#init-284"><span class="linenos">284</span></a> <span class="c1"># create initial commit</span>
+</span><span id="init-285"><a href="#init-285"><span class="linenos">285</span></a> <span class="n">initial_file_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="s2">"README.md"</span>
+</span><span id="init-286"><a href="#init-286"><span class="linenos">286</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">initial_file_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="init-287"><a href="#init-287"><span class="linenos">287</span></a> <span class="n">file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span>
+</span><span id="init-288"><a href="#init-288"><span class="linenos">288</span></a> <span class="s2">"# Initial commit</span><span class="se">\n</span><span class="s2">This is the first commit in the phen repository.</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="init-289"><a href="#init-289"><span class="linenos">289</span></a> <span class="p">)</span>
+</span><span id="init-290"><a href="#init-290"><span class="linenos">290</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">add</span><span class="p">([</span><span class="n">initial_file_path</span><span class="p">])</span>
+</span><span id="init-291"><a href="#init-291"><span class="linenos">291</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">commit</span><span class="p">(</span><span class="s2">"Initial commit"</span><span class="p">)</span>
+</span><span id="init-292"><a href="#init-292"><span class="linenos">292</span></a> <span class="n">commit_count</span> <span class="o">=</span> <span class="mi">1</span>
+</span><span id="init-293"><a href="#init-293"><span class="linenos">293</span></a>
+</span><span id="init-294"><a href="#init-294"><span class="linenos">294</span></a> <span class="c1"># Checkout the phens default branch, creating it if it does not exist</span>
+</span><span id="init-295"><a href="#init-295"><span class="linenos">295</span></a> <span class="k">if</span> <span class="n">DEFAULT_GIT_BRANCH</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">branches</span><span class="p">:</span>
+</span><span id="init-296"><a href="#init-296"><span class="linenos">296</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">heads</span><span class="p">[</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">]</span>
+</span><span id="init-297"><a href="#init-297"><span class="linenos">297</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="init-298"><a href="#init-298"><span class="linenos">298</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="init-299"><a href="#init-299"><span class="linenos">299</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">create_head</span><span class="p">(</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">)</span>
+</span><span id="init-300"><a href="#init-300"><span class="linenos">300</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="init-301"><a href="#init-301"><span class="linenos">301</span></a>
+</span><span id="init-302"><a href="#init-302"><span class="linenos">302</span></a> <span class="c1"># if the phen path does not contain the config file then initialise the phen type</span>
+</span><span id="init-303"><a href="#init-303"><span class="linenos">303</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="init-304"><a href="#init-304"><span class="linenos">304</span></a> <span class="k">if</span> <span class="n">config_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="init-305"><a href="#init-305"><span class="linenos">305</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype configuration files already exist"</span><span class="p">)</span>
+</span><span id="init-306"><a href="#init-306"><span class="linenos">306</span></a> <span class="k">return</span>
+</span><span id="init-307"><a href="#init-307"><span class="linenos">307</span></a>
+</span><span id="init-308"><a href="#init-308"><span class="linenos">308</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Creating phen directory structure and config files"</span><span class="p">)</span>
+</span><span id="init-309"><a href="#init-309"><span class="linenos">309</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">DEFAULT_PHEN_DIR_LIST</span><span class="p">:</span>
+</span><span id="init-310"><a href="#init-310"><span class="linenos">310</span></a> <span class="n">_create_empty_git_dir</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">d</span><span class="p">)</span>
+</span><span id="init-311"><a href="#init-311"><span class="linenos">311</span></a>
+</span><span id="init-312"><a href="#init-312"><span class="linenos">312</span></a> <span class="c1"># create empty phen config file</span>
+</span><span id="init-313"><a href="#init-313"><span class="linenos">313</span></a> <span class="n">config</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="init-314"><a href="#init-314"><span class="linenos">314</span></a> <span class="s2">"phenotype"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="init-315"><a href="#init-315"><span class="linenos">315</span></a> <span class="s2">"version"</span><span class="p">:</span> <span class="s2">"0.0.0"</span><span class="p">,</span>
+</span><span id="init-316"><a href="#init-316"><span class="linenos">316</span></a> <span class="s2">"omop"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="init-317"><a href="#init-317"><span class="linenos">317</span></a> <span class="s2">"vocabulary_id"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="init-318"><a href="#init-318"><span class="linenos">318</span></a> <span class="s2">"vocabulary_name"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="init-319"><a href="#init-319"><span class="linenos">319</span></a> <span class="s2">"vocabulary_reference"</span><span class="p">:</span> <span class="s2">""</span><span class="p">,</span>
+</span><span id="init-320"><a href="#init-320"><span class="linenos">320</span></a> <span class="p">},</span>
+</span><span id="init-321"><a href="#init-321"><span class="linenos">321</span></a> <span class="s2">"translate"</span><span class="p">:</span> <span class="p">[],</span>
+</span><span id="init-322"><a href="#init-322"><span class="linenos">322</span></a> <span class="s2">"concept_sets"</span><span class="p">:</span> <span class="p">[],</span>
+</span><span id="init-323"><a href="#init-323"><span class="linenos">323</span></a> <span class="p">}</span>
+</span><span id="init-324"><a href="#init-324"><span class="linenos">324</span></a> <span class="p">}</span>
+</span><span id="init-325"><a href="#init-325"><span class="linenos">325</span></a>
+</span><span id="init-326"><a href="#init-326"><span class="linenos">326</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="init-327"><a href="#init-327"><span class="linenos">327</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="init-328"><a href="#init-328"><span class="linenos">328</span></a> <span class="n">config</span><span class="p">,</span>
+</span><span id="init-329"><a href="#init-329"><span class="linenos">329</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="init-330"><a href="#init-330"><span class="linenos">330</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="init-331"><a href="#init-331"><span class="linenos">331</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="init-332"><a href="#init-332"><span class="linenos">332</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="init-333"><a href="#init-333"><span class="linenos">333</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="init-334"><a href="#init-334"><span class="linenos">334</span></a> <span class="p">)</span>
+</span><span id="init-335"><a href="#init-335"><span class="linenos">335</span></a>
+</span><span id="init-336"><a href="#init-336"><span class="linenos">336</span></a> <span class="c1"># add git ignore</span>
+</span><span id="init-337"><a href="#init-337"><span class="linenos">337</span></a> <span class="n">ignore_content</span> <span class="o">=</span> <span class="s2">"""# Ignore SQLite database files</span>
+</span><span id="init-338"><a href="#init-338"><span class="linenos">338</span></a><span class="s2">*.db</span>
+</span><span id="init-339"><a href="#init-339"><span class="linenos">339</span></a><span class="s2">*.sqlite3</span>
+</span><span id="init-340"><a href="#init-340"><span class="linenos">340</span></a><span class="s2"> </span>
+</span><span id="init-341"><a href="#init-341"><span class="linenos">341</span></a><span class="s2"># Ignore SQLite journal and metadata files</span>
+</span><span id="init-342"><a href="#init-342"><span class="linenos">342</span></a><span class="s2">*.db-journal</span>
+</span><span id="init-343"><a href="#init-343"><span class="linenos">343</span></a><span class="s2">*.sqlite3-journal</span>
+</span><span id="init-344"><a href="#init-344"><span class="linenos">344</span></a>
+</span><span id="init-345"><a href="#init-345"><span class="linenos">345</span></a><span class="s2"># python</span>
+</span><span id="init-346"><a href="#init-346"><span class="linenos">346</span></a><span class="s2">.ipynb_checkpoints</span>
+</span><span id="init-347"><a href="#init-347"><span class="linenos">347</span></a><span class="s2"> """</span>
+</span><span id="init-348"><a href="#init-348"><span class="linenos">348</span></a> <span class="n">ignore_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="s2">".gitignore"</span>
+</span><span id="init-349"><a href="#init-349"><span class="linenos">349</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">ignore_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="init-350"><a href="#init-350"><span class="linenos">350</span></a> <span class="n">file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">ignore_content</span><span class="p">)</span>
+</span><span id="init-351"><a href="#init-351"><span class="linenos">351</span></a>
+</span><span id="init-352"><a href="#init-352"><span class="linenos">352</span></a> <span class="c1"># add to git repo and commit</span>
+</span><span id="init-353"><a href="#init-353"><span class="linenos">353</span></a> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">DEFAULT_PHEN_DIR_LIST</span><span class="p">:</span>
+</span><span id="init-354"><a href="#init-354"><span class="linenos">354</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">d</span><span class="p">)</span>
+</span><span id="init-355"><a href="#init-355"><span class="linenos">355</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="nb">all</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="init-356"><a href="#init-356"><span class="linenos">356</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">commit</span><span class="p">(</span><span class="s2">"initialised the phen git repo."</span><span class="p">)</span>
+</span><span id="init-357"><a href="#init-357"><span class="linenos">357</span></a>
+</span><span id="init-358"><a href="#init-358"><span class="linenos">358</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype initialised successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Initial phenotype directory as git repo with standard structure</p>
+</div>
+
+
+ </section>
+ <section id="fork">
+ <input id="fork-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">fork</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">upstream_url</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">upstream_version</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">new_origin_url</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="fork-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#fork"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="fork-361"><a href="#fork-361"><span class="linenos">361</span></a><span class="k">def</span><span class="w"> </span><span class="nf">fork</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">upstream_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">upstream_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">new_origin_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="fork-362"><a href="#fork-362"><span class="linenos">362</span></a><span class="w"> </span><span class="sd">"""Forks an upstream phenotype in a remote repo at a specific version to a local director, and optionally sets to a new remote origin"</span>
+</span><span id="fork-363"><a href="#fork-363"><span class="linenos">363</span></a>
+</span><span id="fork-364"><a href="#fork-364"><span class="linenos">364</span></a><span class="sd"> Args:</span>
+</span><span id="fork-365"><a href="#fork-365"><span class="linenos">365</span></a><span class="sd"> phen_dir (str): local directory path where the upstream repo is to be cloned</span>
+</span><span id="fork-366"><a href="#fork-366"><span class="linenos">366</span></a><span class="sd"> upstream_url (str): url to the upstream repo</span>
+</span><span id="fork-367"><a href="#fork-367"><span class="linenos">367</span></a><span class="sd"> upstream_version (str): version in the upstream repo to clone</span>
+</span><span id="fork-368"><a href="#fork-368"><span class="linenos">368</span></a><span class="sd"> new_origin_url (str, optional): url of the remote repo to set as the new origin. Defaults to None.</span>
+</span><span id="fork-369"><a href="#fork-369"><span class="linenos">369</span></a>
+</span><span id="fork-370"><a href="#fork-370"><span class="linenos">370</span></a><span class="sd"> Raises:</span>
+</span><span id="fork-371"><a href="#fork-371"><span class="linenos">371</span></a><span class="sd"> ValueError: if the specified version is not in the upstream repo</span>
+</span><span id="fork-372"><a href="#fork-372"><span class="linenos">372</span></a><span class="sd"> ValueError: if the upstream repo is not a valid phenotype repo</span>
+</span><span id="fork-373"><a href="#fork-373"><span class="linenos">373</span></a><span class="sd"> ValueError: if there's any other problems with Git</span>
+</span><span id="fork-374"><a href="#fork-374"><span class="linenos">374</span></a><span class="sd"> """</span>
+</span><span id="fork-375"><a href="#fork-375"><span class="linenos">375</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
+</span><span id="fork-376"><a href="#fork-376"><span class="linenos">376</span></a> <span class="sa">f</span><span class="s2">"Forking upstream repo </span><span class="si">{</span><span class="n">upstream_url</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">upstream_version</span><span class="si">}</span><span class="s2"> into directory: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="fork-377"><a href="#fork-377"><span class="linenos">377</span></a> <span class="p">)</span>
+</span><span id="fork-378"><a href="#fork-378"><span class="linenos">378</span></a>
+</span><span id="fork-379"><a href="#fork-379"><span class="linenos">379</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="fork-380"><a href="#fork-380"><span class="linenos">380</span></a> <span class="c1"># check if directory already exists and ask user if they want to recreate it</span>
+</span><span id="fork-381"><a href="#fork-381"><span class="linenos">381</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="fork-382"><a href="#fork-382"><span class="linenos">382</span></a> <span class="n">phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">and</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span>
+</span><span id="fork-383"><a href="#fork-383"><span class="linenos">383</span></a> <span class="p">):</span> <span class="c1"># Check if it exists and is a directory</span>
+</span><span id="fork-384"><a href="#fork-384"><span class="linenos">384</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="n">_check_delete_dir</span><span class="p">(</span>
+</span><span id="fork-385"><a href="#fork-385"><span class="linenos">385</span></a> <span class="n">phen_path</span><span class="p">,</span>
+</span><span id="fork-386"><a href="#fork-386"><span class="linenos">386</span></a> <span class="sa">f</span><span class="s2">"The phen directory already exists. Do you want to reinitialise? (yes/no): "</span><span class="p">,</span>
+</span><span id="fork-387"><a href="#fork-387"><span class="linenos">387</span></a> <span class="p">)</span>
+</span><span id="fork-388"><a href="#fork-388"><span class="linenos">388</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="fork-389"><a href="#fork-389"><span class="linenos">389</span></a> <span class="n">configure</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="fork-390"><a href="#fork-390"><span class="linenos">390</span></a>
+</span><span id="fork-391"><a href="#fork-391"><span class="linenos">391</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">configure</span><span class="p">:</span>
+</span><span id="fork-392"><a href="#fork-392"><span class="linenos">392</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exiting, phenotype not initiatised"</span><span class="p">)</span>
+</span><span id="fork-393"><a href="#fork-393"><span class="linenos">393</span></a> <span class="k">return</span>
+</span><span id="fork-394"><a href="#fork-394"><span class="linenos">394</span></a>
+</span><span id="fork-395"><a href="#fork-395"><span class="linenos">395</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="fork-396"><a href="#fork-396"><span class="linenos">396</span></a> <span class="c1"># Clone repo</span>
+</span><span id="fork-397"><a href="#fork-397"><span class="linenos">397</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">upstream_url</span><span class="p">)</span>
+</span><span id="fork-398"><a href="#fork-398"><span class="linenos">398</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">git_url</span><span class="p">,</span> <span class="n">phen_path</span><span class="p">)</span>
+</span><span id="fork-399"><a href="#fork-399"><span class="linenos">399</span></a>
+</span><span id="fork-400"><a href="#fork-400"><span class="linenos">400</span></a> <span class="c1"># Fetch all branches and tags</span>
+</span><span id="fork-401"><a href="#fork-401"><span class="linenos">401</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="o">.</span><span class="n">origin</span><span class="o">.</span><span class="n">fetch</span><span class="p">()</span>
+</span><span id="fork-402"><a href="#fork-402"><span class="linenos">402</span></a>
+</span><span id="fork-403"><a href="#fork-403"><span class="linenos">403</span></a> <span class="c1"># Check if the version exists</span>
+</span><span id="fork-404"><a href="#fork-404"><span class="linenos">404</span></a> <span class="n">available_refs</span> <span class="o">=</span> <span class="p">[</span><span class="n">ref</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"/"</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="k">for</span> <span class="n">ref</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">references</span><span class="p">]</span>
+</span><span id="fork-405"><a href="#fork-405"><span class="linenos">405</span></a> <span class="k">if</span> <span class="n">upstream_version</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">available_refs</span><span class="p">:</span>
+</span><span id="fork-406"><a href="#fork-406"><span class="linenos">406</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="fork-407"><a href="#fork-407"><span class="linenos">407</span></a> <span class="sa">f</span><span class="s2">"Version '</span><span class="si">{</span><span class="n">upstream_version</span><span class="si">}</span><span class="s2">' not found in the repository: </span><span class="si">{</span><span class="n">upstream_url</span><span class="si">}</span><span class="s2">."</span>
+</span><span id="fork-408"><a href="#fork-408"><span class="linenos">408</span></a> <span class="p">)</span>
+</span><span id="fork-409"><a href="#fork-409"><span class="linenos">409</span></a>
+</span><span id="fork-410"><a href="#fork-410"><span class="linenos">410</span></a> <span class="c1"># Checkout the specified version</span>
+</span><span id="fork-411"><a href="#fork-411"><span class="linenos">411</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">upstream_version</span><span class="p">)</span>
+</span><span id="fork-412"><a href="#fork-412"><span class="linenos">412</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">heads</span><span class="p">[</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">]</span>
+</span><span id="fork-413"><a href="#fork-413"><span class="linenos">413</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="fork-414"><a href="#fork-414"><span class="linenos">414</span></a>
+</span><span id="fork-415"><a href="#fork-415"><span class="linenos">415</span></a> <span class="c1"># Check if 'config.yml' exists in the root directory</span>
+</span><span id="fork-416"><a href="#fork-416"><span class="linenos">416</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="s2">"config.yml"</span>
+</span><span id="fork-417"><a href="#fork-417"><span class="linenos">417</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">isfile</span><span class="p">(</span><span class="n">config_path</span><span class="p">):</span>
+</span><span id="fork-418"><a href="#fork-418"><span class="linenos">418</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="fork-419"><a href="#fork-419"><span class="linenos">419</span></a> <span class="sa">f</span><span class="s2">"The forked repository is not a valid ACMC repo because 'config.yml' is missing in the root directory."</span>
+</span><span id="fork-420"><a href="#fork-420"><span class="linenos">420</span></a> <span class="p">)</span>
+</span><span id="fork-421"><a href="#fork-421"><span class="linenos">421</span></a>
+</span><span id="fork-422"><a href="#fork-422"><span class="linenos">422</span></a> <span class="c1"># Validate the phenotype is compatible with the acmc tool</span>
+</span><span id="fork-423"><a href="#fork-423"><span class="linenos">423</span></a> <span class="n">validate</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()))</span>
+</span><span id="fork-424"><a href="#fork-424"><span class="linenos">424</span></a>
+</span><span id="fork-425"><a href="#fork-425"><span class="linenos">425</span></a> <span class="c1"># Delete each tag locally</span>
+</span><span id="fork-426"><a href="#fork-426"><span class="linenos">426</span></a> <span class="n">tags</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">tags</span>
+</span><span id="fork-427"><a href="#fork-427"><span class="linenos">427</span></a> <span class="k">for</span> <span class="n">tag</span> <span class="ow">in</span> <span class="n">tags</span><span class="p">:</span>
+</span><span id="fork-428"><a href="#fork-428"><span class="linenos">428</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">delete_tag</span><span class="p">(</span><span class="n">tag</span><span class="p">)</span>
+</span><span id="fork-429"><a href="#fork-429"><span class="linenos">429</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Deleted tags from forked repo: </span><span class="si">{</span><span class="n">tag</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="fork-430"><a href="#fork-430"><span class="linenos">430</span></a>
+</span><span id="fork-431"><a href="#fork-431"><span class="linenos">431</span></a> <span class="c1"># Add upstream remote</span>
+</span><span id="fork-432"><a href="#fork-432"><span class="linenos">432</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_remote</span><span class="p">(</span><span class="s2">"upstream"</span><span class="p">,</span> <span class="n">upstream_url</span><span class="p">)</span>
+</span><span id="fork-433"><a href="#fork-433"><span class="linenos">433</span></a> <span class="n">remote</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">[</span><span class="s2">"origin"</span><span class="p">]</span>
+</span><span id="fork-434"><a href="#fork-434"><span class="linenos">434</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">delete_remote</span><span class="p">(</span><span class="n">remote</span><span class="p">)</span> <span class="c1"># Remove existing origin</span>
+</span><span id="fork-435"><a href="#fork-435"><span class="linenos">435</span></a>
+</span><span id="fork-436"><a href="#fork-436"><span class="linenos">436</span></a> <span class="c1"># Optionally set a new origin remote</span>
+</span><span id="fork-437"><a href="#fork-437"><span class="linenos">437</span></a> <span class="k">if</span> <span class="n">new_origin_url</span><span class="p">:</span>
+</span><span id="fork-438"><a href="#fork-438"><span class="linenos">438</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">new_origin_url</span><span class="p">)</span>
+</span><span id="fork-439"><a href="#fork-439"><span class="linenos">439</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_remote</span><span class="p">(</span><span class="s2">"origin"</span><span class="p">,</span> <span class="n">git_url</span><span class="p">)</span>
+</span><span id="fork-440"><a href="#fork-440"><span class="linenos">440</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="s2">"--set-upstream"</span><span class="p">,</span> <span class="s2">"origin"</span><span class="p">,</span> <span class="s2">"main"</span><span class="p">)</span>
+</span><span id="fork-441"><a href="#fork-441"><span class="linenos">441</span></a>
+</span><span id="fork-442"><a href="#fork-442"><span class="linenos">442</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Repository forked successfully at </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="fork-443"><a href="#fork-443"><span class="linenos">443</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Upstream set to </span><span class="si">{</span><span class="n">upstream_url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="fork-444"><a href="#fork-444"><span class="linenos">444</span></a> <span class="k">if</span> <span class="n">new_origin_url</span><span class="p">:</span>
+</span><span id="fork-445"><a href="#fork-445"><span class="linenos">445</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Origin set to </span><span class="si">{</span><span class="n">new_origin_url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="fork-446"><a href="#fork-446"><span class="linenos">446</span></a>
+</span><span id="fork-447"><a href="#fork-447"><span class="linenos">447</span></a> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="fork-448"><a href="#fork-448"><span class="linenos">448</span></a> <span class="k">if</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="fork-449"><a href="#fork-449"><span class="linenos">449</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="fork-450"><a href="#fork-450"><span class="linenos">450</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Error occurred during repository fork: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Forks an upstream phenotype in a remote repo at a specific version to a local director, and optionally sets to a new remote origin"</p>
+
+<h6 id="arguments">Arguments:</h6>
+
+<ul>
+<li><strong>phen_dir (str):</strong> local directory path where the upstream repo is to be cloned</li>
+<li><strong>upstream_url (str):</strong> url to the upstream repo</li>
+<li><strong>upstream_version (str):</strong> version in the upstream repo to clone</li>
+<li><strong>new_origin_url (str, optional):</strong> url of the remote repo to set as the new origin. Defaults to None.</li>
+</ul>
+
+<h6 id="raises">Raises:</h6>
+
+<ul>
+<li><strong>ValueError:</strong> if the specified version is not in the upstream repo</li>
+<li><strong>ValueError:</strong> if the upstream repo is not a valid phenotype repo</li>
+<li><strong>ValueError:</strong> if there's any other problems with Git</li>
+</ul>
+</div>
+
+
+ </section>
+ <section id="validate">
+ <input id="validate-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">validate</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="validate-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#validate"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="validate-453"><a href="#validate-453"><span class="linenos">453</span></a><span class="k">def</span><span class="w"> </span><span class="nf">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="validate-454"><a href="#validate-454"><span class="linenos">454</span></a><span class="w"> </span><span class="sd">"""Validates the phenotype directory is a git repo with standard structure"""</span>
+</span><span id="validate-455"><a href="#validate-455"><span class="linenos">455</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Validating phenotype: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="validate-456"><a href="#validate-456"><span class="linenos">456</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="validate-457"><a href="#validate-457"><span class="linenos">457</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">phen_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="validate-458"><a href="#validate-458"><span class="linenos">458</span></a> <span class="k">raise</span> <span class="ne">NotADirectoryError</span><span class="p">(</span>
+</span><span id="validate-459"><a href="#validate-459"><span class="linenos">459</span></a> <span class="sa">f</span><span class="s2">"Error: '</span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">' is not a directory"</span>
+</span><span id="validate-460"><a href="#validate-460"><span class="linenos">460</span></a> <span class="p">)</span>
+</span><span id="validate-461"><a href="#validate-461"><span class="linenos">461</span></a>
+</span><span id="validate-462"><a href="#validate-462"><span class="linenos">462</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="validate-463"><a href="#validate-463"><span class="linenos">463</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">config_path</span><span class="o">.</span><span class="n">is_file</span><span class="p">():</span>
+</span><span id="validate-464"><a href="#validate-464"><span class="linenos">464</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="validate-465"><a href="#validate-465"><span class="linenos">465</span></a> <span class="sa">f</span><span class="s2">"Error: phen configuration file '</span><span class="si">{</span><span class="n">config_path</span><span class="si">}</span><span class="s2">' does not exist."</span>
+</span><span id="validate-466"><a href="#validate-466"><span class="linenos">466</span></a> <span class="p">)</span>
+</span><span id="validate-467"><a href="#validate-467"><span class="linenos">467</span></a>
+</span><span id="validate-468"><a href="#validate-468"><span class="linenos">468</span></a> <span class="n">concepts_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONCEPTS_DIR</span>
+</span><span id="validate-469"><a href="#validate-469"><span class="linenos">469</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">concepts_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="validate-470"><a href="#validate-470"><span class="linenos">470</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="validate-471"><a href="#validate-471"><span class="linenos">471</span></a> <span class="sa">f</span><span class="s2">"Error: source concepts directory </span><span class="si">{</span><span class="n">concepts_path</span><span class="si">}</span><span class="s2"> does not exist."</span>
+</span><span id="validate-472"><a href="#validate-472"><span class="linenos">472</span></a> <span class="p">)</span>
+</span><span id="validate-473"><a href="#validate-473"><span class="linenos">473</span></a>
+</span><span id="validate-474"><a href="#validate-474"><span class="linenos">474</span></a> <span class="c1"># Calidate the directory is a git repo</span>
+</span><span id="validate-475"><a href="#validate-475"><span class="linenos">475</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="validate-476"><a href="#validate-476"><span class="linenos">476</span></a> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="validate-477"><a href="#validate-477"><span class="linenos">477</span></a> <span class="k">except</span> <span class="p">(</span><span class="n">git</span><span class="o">.</span><span class="n">exc</span><span class="o">.</span><span class="n">InvalidGitRepositoryError</span><span class="p">,</span> <span class="n">git</span><span class="o">.</span><span class="n">exc</span><span class="o">.</span><span class="n">NoSuchPathError</span><span class="p">):</span>
+</span><span id="validate-478"><a href="#validate-478"><span class="linenos">478</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phen directory </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> is not a git repo"</span><span class="p">)</span>
+</span><span id="validate-479"><a href="#validate-479"><span class="linenos">479</span></a>
+</span><span id="validate-480"><a href="#validate-480"><span class="linenos">480</span></a> <span class="c1"># Load configuration File</span>
+</span><span id="validate-481"><a href="#validate-481"><span class="linenos">481</span></a> <span class="k">if</span> <span class="n">config_path</span><span class="o">.</span><span class="n">suffix</span> <span class="o">==</span> <span class="s2">".yml"</span><span class="p">:</span>
+</span><span id="validate-482"><a href="#validate-482"><span class="linenos">482</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="validate-483"><a href="#validate-483"><span class="linenos">483</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="validate-484"><a href="#validate-484"><span class="linenos">484</span></a> <span class="n">phenotype</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="validate-485"><a href="#validate-485"><span class="linenos">485</span></a>
+</span><span id="validate-486"><a href="#validate-486"><span class="linenos">486</span></a> <span class="n">validator</span> <span class="o">=</span> <span class="n">Validator</span><span class="p">(</span><span class="n">CONFIG_SCHEMA</span><span class="p">)</span>
+</span><span id="validate-487"><a href="#validate-487"><span class="linenos">487</span></a> <span class="k">if</span> <span class="n">validator</span><span class="o">.</span><span class="n">validate</span><span class="p">(</span><span class="n">phenotype</span><span class="p">):</span>
+</span><span id="validate-488"><a href="#validate-488"><span class="linenos">488</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"YAML structure is valid."</span><span class="p">)</span>
+</span><span id="validate-489"><a href="#validate-489"><span class="linenos">489</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="validate-490"><a href="#validate-490"><span class="linenos">490</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">"YAML structure validation failed: </span><span class="si">{</span><span class="n">validator</span><span class="o">.</span><span class="n">errors</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="validate-491"><a href="#validate-491"><span class="linenos">491</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="sa">f</span><span class="s2">"YAML structure validation failed: </span><span class="si">{</span><span class="n">validator</span><span class="o">.</span><span class="n">errors</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="validate-492"><a href="#validate-492"><span class="linenos">492</span></a> <span class="k">except</span> <span class="n">yaml</span><span class="o">.</span><span class="n">YAMLError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="validate-493"><a href="#validate-493"><span class="linenos">493</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="sa">f</span><span class="s2">"YAML syntax error: </span><span class="si">{</span><span class="n">e</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="validate-494"><a href="#validate-494"><span class="linenos">494</span></a> <span class="k">raise</span> <span class="n">e</span>
+</span><span id="validate-495"><a href="#validate-495"><span class="linenos">495</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="validate-496"><a href="#validate-496"><span class="linenos">496</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span>
+</span><span id="validate-497"><a href="#validate-497"><span class="linenos">497</span></a> <span class="sa">f</span><span class="s2">"Unsupported configuration filetype: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">config_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="validate-498"><a href="#validate-498"><span class="linenos">498</span></a> <span class="p">)</span>
+</span><span id="validate-499"><a href="#validate-499"><span class="linenos">499</span></a>
+</span><span id="validate-500"><a href="#validate-500"><span class="linenos">500</span></a> <span class="c1"># initiatise</span>
+</span><span id="validate-501"><a href="#validate-501"><span class="linenos">501</span></a> <span class="n">validation_errors</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="validate-502"><a href="#validate-502"><span class="linenos">502</span></a> <span class="n">phenotype</span> <span class="o">=</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">]</span>
+</span><span id="validate-503"><a href="#validate-503"><span class="linenos">503</span></a> <span class="n">code_types</span> <span class="o">=</span> <span class="n">parse</span><span class="o">.</span><span class="n">CodeTypeParser</span><span class="p">()</span><span class="o">.</span><span class="n">code_types</span>
+</span><span id="validate-504"><a href="#validate-504"><span class="linenos">504</span></a>
+</span><span id="validate-505"><a href="#validate-505"><span class="linenos">505</span></a> <span class="c1"># check the version number is of the format vn.n.n</span>
+</span><span id="validate-506"><a href="#validate-506"><span class="linenos">506</span></a> <span class="n">match</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="sa">r</span><span class="s2">"(\d+\.\d+\.\d+)"</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"version"</span><span class="p">])</span>
+</span><span id="validate-507"><a href="#validate-507"><span class="linenos">507</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">match</span><span class="p">:</span>
+</span><span id="validate-508"><a href="#validate-508"><span class="linenos">508</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="validate-509"><a href="#validate-509"><span class="linenos">509</span></a> <span class="sa">f</span><span class="s2">"Invalid version format in configuration file: </span><span class="si">{</span><span class="n">phenotype</span><span class="p">[</span><span class="s1">'version'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="validate-510"><a href="#validate-510"><span class="linenos">510</span></a> <span class="p">)</span>
+</span><span id="validate-511"><a href="#validate-511"><span class="linenos">511</span></a>
+</span><span id="validate-512"><a href="#validate-512"><span class="linenos">512</span></a> <span class="c1"># create a list of all the concept set names defined in the concept set configuration</span>
+</span><span id="validate-513"><a href="#validate-513"><span class="linenos">513</span></a> <span class="n">concept_set_names</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="validate-514"><a href="#validate-514"><span class="linenos">514</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"concept_sets"</span><span class="p">]:</span>
+</span><span id="validate-515"><a href="#validate-515"><span class="linenos">515</span></a> <span class="k">if</span> <span class="n">item</span><span class="p">[</span><span class="s2">"name"</span><span class="p">]</span> <span class="ow">in</span> <span class="n">concept_set_names</span><span class="p">:</span>
+</span><span id="validate-516"><a href="#validate-516"><span class="linenos">516</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="validate-517"><a href="#validate-517"><span class="linenos">517</span></a> <span class="sa">f</span><span class="s2">"Duplicate concept set defined in concept sets </span><span class="si">{</span><span class="n">item</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span><span class="w"> </span><span class="si">}</span><span class="s2">"</span>
+</span><span id="validate-518"><a href="#validate-518"><span class="linenos">518</span></a> <span class="p">)</span>
+</span><span id="validate-519"><a href="#validate-519"><span class="linenos">519</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="validate-520"><a href="#validate-520"><span class="linenos">520</span></a> <span class="n">concept_set_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">item</span><span class="p">[</span><span class="s2">"name"</span><span class="p">])</span>
+</span><span id="validate-521"><a href="#validate-521"><span class="linenos">521</span></a>
+</span><span id="validate-522"><a href="#validate-522"><span class="linenos">522</span></a> <span class="c1"># check codes definition</span>
+</span><span id="validate-523"><a href="#validate-523"><span class="linenos">523</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"concept_sets"</span><span class="p">]:</span>
+</span><span id="validate-524"><a href="#validate-524"><span class="linenos">524</span></a> <span class="c1"># check concepte code file exists</span>
+</span><span id="validate-525"><a href="#validate-525"><span class="linenos">525</span></a> <span class="n">concept_code_file_path</span> <span class="o">=</span> <span class="n">concepts_path</span> <span class="o">/</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"path"</span><span class="p">]</span>
+</span><span id="validate-526"><a href="#validate-526"><span class="linenos">526</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="validate-527"><a href="#validate-527"><span class="linenos">527</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="validate-528"><a href="#validate-528"><span class="linenos">528</span></a> <span class="sa">f</span><span class="s2">"Coding file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> does not exist"</span>
+</span><span id="validate-529"><a href="#validate-529"><span class="linenos">529</span></a> <span class="p">)</span>
+</span><span id="validate-530"><a href="#validate-530"><span class="linenos">530</span></a>
+</span><span id="validate-531"><a href="#validate-531"><span class="linenos">531</span></a> <span class="c1"># check concepte code file is not empty</span>
+</span><span id="validate-532"><a href="#validate-532"><span class="linenos">532</span></a> <span class="k">if</span> <span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">stat</span><span class="p">()</span><span class="o">.</span><span class="n">st_size</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="validate-533"><a href="#validate-533"><span class="linenos">533</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="validate-534"><a href="#validate-534"><span class="linenos">534</span></a> <span class="sa">f</span><span class="s2">"Coding file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> is an empty file"</span>
+</span><span id="validate-535"><a href="#validate-535"><span class="linenos">535</span></a> <span class="p">)</span>
+</span><span id="validate-536"><a href="#validate-536"><span class="linenos">536</span></a>
+</span><span id="validate-537"><a href="#validate-537"><span class="linenos">537</span></a> <span class="c1"># check code file type is supported</span>
+</span><span id="validate-538"><a href="#validate-538"><span class="linenos">538</span></a> <span class="k">if</span> <span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">suffix</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">CODE_FILE_TYPES</span><span class="p">:</span>
+</span><span id="validate-539"><a href="#validate-539"><span class="linenos">539</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="validate-540"><a href="#validate-540"><span class="linenos">540</span></a> <span class="sa">f</span><span class="s2">"Unsupported filetype </span><span class="si">{</span><span class="n">concept_code_file_path</span><span class="o">.</span><span class="n">suffix</span><span class="si">}</span><span class="s2">, only support csv, xlsx, xls code file types"</span>
+</span><span id="validate-541"><a href="#validate-541"><span class="linenos">541</span></a> <span class="p">)</span>
+</span><span id="validate-542"><a href="#validate-542"><span class="linenos">542</span></a>
+</span><span id="validate-543"><a href="#validate-543"><span class="linenos">543</span></a> <span class="c1"># check columns specified are a supported medical coding type</span>
+</span><span id="validate-544"><a href="#validate-544"><span class="linenos">544</span></a> <span class="k">for</span> <span class="n">column</span> <span class="ow">in</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"columns"</span><span class="p">]:</span>
+</span><span id="validate-545"><a href="#validate-545"><span class="linenos">545</span></a> <span class="k">if</span> <span class="n">column</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">code_types</span><span class="p">:</span>
+</span><span id="validate-546"><a href="#validate-546"><span class="linenos">546</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span>
+</span><span id="validate-547"><a href="#validate-547"><span class="linenos">547</span></a> <span class="sa">f</span><span class="s2">"Column type </span><span class="si">{</span><span class="n">column</span><span class="si">}</span><span class="s2"> for file </span><span class="si">{</span><span class="n">concept_code_file_path</span><span class="si">}</span><span class="s2"> is not supported"</span>
+</span><span id="validate-548"><a href="#validate-548"><span class="linenos">548</span></a> <span class="p">)</span>
+</span><span id="validate-549"><a href="#validate-549"><span class="linenos">549</span></a>
+</span><span id="validate-550"><a href="#validate-550"><span class="linenos">550</span></a> <span class="c1"># check the actions are supported</span>
+</span><span id="validate-551"><a href="#validate-551"><span class="linenos">551</span></a> <span class="k">if</span> <span class="s2">"actions"</span> <span class="ow">in</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">]:</span>
+</span><span id="validate-552"><a href="#validate-552"><span class="linenos">552</span></a> <span class="k">for</span> <span class="n">action</span> <span class="ow">in</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"actions"</span><span class="p">]:</span>
+</span><span id="validate-553"><a href="#validate-553"><span class="linenos">553</span></a> <span class="k">if</span> <span class="n">action</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">COL_ACTIONS</span><span class="p">:</span>
+</span><span id="validate-554"><a href="#validate-554"><span class="linenos">554</span></a> <span class="n">validation_errors</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Action </span><span class="si">{</span><span class="n">action</span><span class="si">}</span><span class="s2"> is not supported"</span><span class="p">)</span>
+</span><span id="validate-555"><a href="#validate-555"><span class="linenos">555</span></a>
+</span><span id="validate-556"><a href="#validate-556"><span class="linenos">556</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">validation_errors</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="validate-557"><a href="#validate-557"><span class="linenos">557</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">validation_errors</span><span class="p">)</span>
+</span><span id="validate-558"><a href="#validate-558"><span class="linenos">558</span></a> <span class="k">raise</span> <span class="n">PhenValidationException</span><span class="p">(</span>
+</span><span id="validate-559"><a href="#validate-559"><span class="linenos">559</span></a> <span class="sa">f</span><span class="s2">"Configuration file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">config_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> failed validation"</span><span class="p">,</span>
+</span><span id="validate-560"><a href="#validate-560"><span class="linenos">560</span></a> <span class="n">validation_errors</span><span class="p">,</span>
+</span><span id="validate-561"><a href="#validate-561"><span class="linenos">561</span></a> <span class="p">)</span>
+</span><span id="validate-562"><a href="#validate-562"><span class="linenos">562</span></a>
+</span><span id="validate-563"><a href="#validate-563"><span class="linenos">563</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype validated successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Validates the phenotype directory is a git repo with standard structure</p>
+</div>
+
+
+ </section>
+ <section id="translate_codes">
+ <input id="translate_codes-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">translate_codes</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">source_df</span><span class="p">:</span> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>,</span><span class="param"> <span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">concept_name</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">) -> <span class="n">pandas</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">frame</span><span class="o">.</span><span class="n">DataFrame</span>:</span></span>
+
+ <label class="view-source-button" for="translate_codes-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#translate_codes"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="translate_codes-657"><a href="#translate_codes-657"><span class="linenos">657</span></a><span class="k">def</span><span class="w"> </span><span class="nf">translate_codes</span><span class="p">(</span>
+</span><span id="translate_codes-658"><a href="#translate_codes-658"><span class="linenos">658</span></a> <span class="n">source_df</span><span class="p">:</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">concept_name</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="translate_codes-659"><a href="#translate_codes-659"><span class="linenos">659</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">:</span>
+</span><span id="translate_codes-660"><a href="#translate_codes-660"><span class="linenos">660</span></a><span class="w"> </span><span class="sd">"""Translates each source code type the source coding list into a target type and returns all conversions as a concept set"""</span>
+</span><span id="translate_codes-661"><a href="#translate_codes-661"><span class="linenos">661</span></a>
+</span><span id="translate_codes-662"><a href="#translate_codes-662"><span class="linenos">662</span></a> <span class="c1"># codes = pd.DataFrame([], dtype=str)</span>
+</span><span id="translate_codes-663"><a href="#translate_codes-663"><span class="linenos">663</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="translate_codes-664"><a href="#translate_codes-664"><span class="linenos">664</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s2">"SOURCE_CONCEPT"</span><span class="p">,</span> <span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="s2">"string"</span>
+</span><span id="translate_codes-665"><a href="#translate_codes-665"><span class="linenos">665</span></a> <span class="p">)</span>
+</span><span id="translate_codes-666"><a href="#translate_codes-666"><span class="linenos">666</span></a> <span class="c1"># Convert codes to target type</span>
+</span><span id="translate_codes-667"><a href="#translate_codes-667"><span class="linenos">667</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Converting to target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="translate_codes-668"><a href="#translate_codes-668"><span class="linenos">668</span></a>
+</span><span id="translate_codes-669"><a href="#translate_codes-669"><span class="linenos">669</span></a> <span class="k">for</span> <span class="n">source_code_type</span> <span class="ow">in</span> <span class="n">source_df</span><span class="o">.</span><span class="n">columns</span><span class="p">:</span>
+</span><span id="translate_codes-670"><a href="#translate_codes-670"><span class="linenos">670</span></a> <span class="c1"># if target code type is the same as thet source code type, no translation, just appending source as target</span>
+</span><span id="translate_codes-671"><a href="#translate_codes-671"><span class="linenos">671</span></a> <span class="k">if</span> <span class="n">source_code_type</span> <span class="o">==</span> <span class="n">target_code_type</span><span class="p">:</span>
+</span><span id="translate_codes-672"><a href="#translate_codes-672"><span class="linenos">672</span></a> <span class="n">copy_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span>
+</span><span id="translate_codes-673"><a href="#translate_codes-673"><span class="linenos">673</span></a> <span class="p">{</span>
+</span><span id="translate_codes-674"><a href="#translate_codes-674"><span class="linenos">674</span></a> <span class="s2">"SOURCE_CONCEPT"</span><span class="p">:</span> <span class="n">source_df</span><span class="p">[</span><span class="n">source_code_type</span><span class="p">],</span>
+</span><span id="translate_codes-675"><a href="#translate_codes-675"><span class="linenos">675</span></a> <span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">:</span> <span class="n">source_code_type</span><span class="p">,</span>
+</span><span id="translate_codes-676"><a href="#translate_codes-676"><span class="linenos">676</span></a> <span class="s2">"CONCEPT"</span><span class="p">:</span> <span class="n">source_df</span><span class="p">[</span><span class="n">source_code_type</span><span class="p">],</span>
+</span><span id="translate_codes-677"><a href="#translate_codes-677"><span class="linenos">677</span></a> <span class="p">}</span>
+</span><span id="translate_codes-678"><a href="#translate_codes-678"><span class="linenos">678</span></a> <span class="p">)</span>
+</span><span id="translate_codes-679"><a href="#translate_codes-679"><span class="linenos">679</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">codes</span><span class="p">,</span> <span class="n">copy_df</span><span class="p">])</span>
+</span><span id="translate_codes-680"><a href="#translate_codes-680"><span class="linenos">680</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="translate_codes-681"><a href="#translate_codes-681"><span class="linenos">681</span></a> <span class="sa">f</span><span class="s2">"Target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2"> is the same as source code type </span><span class="si">{</span><span class="nb">len</span><span class="p">(</span><span class="n">source_df</span><span class="p">)</span><span class="si">}</span><span class="s2">, copying codes rather than translating"</span>
+</span><span id="translate_codes-682"><a href="#translate_codes-682"><span class="linenos">682</span></a> <span class="p">)</span>
+</span><span id="translate_codes-683"><a href="#translate_codes-683"><span class="linenos">683</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="translate_codes-684"><a href="#translate_codes-684"><span class="linenos">684</span></a> <span class="c1"># get the translation filename using source to target code types</span>
+</span><span id="translate_codes-685"><a href="#translate_codes-685"><span class="linenos">685</span></a> <span class="n">filename</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">source_code_type</span><span class="si">}</span><span class="s2">_to_</span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">.parquet"</span>
+</span><span id="translate_codes-686"><a href="#translate_codes-686"><span class="linenos">686</span></a> <span class="n">map_path</span> <span class="o">=</span> <span class="n">trud</span><span class="o">.</span><span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="n">filename</span>
+</span><span id="translate_codes-687"><a href="#translate_codes-687"><span class="linenos">687</span></a>
+</span><span id="translate_codes-688"><a href="#translate_codes-688"><span class="linenos">688</span></a> <span class="c1"># do the mapping if it exists</span>
+</span><span id="translate_codes-689"><a href="#translate_codes-689"><span class="linenos">689</span></a> <span class="k">if</span> <span class="n">map_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="translate_codes-690"><a href="#translate_codes-690"><span class="linenos">690</span></a> <span class="c1"># get mapping</span>
+</span><span id="translate_codes-691"><a href="#translate_codes-691"><span class="linenos">691</span></a> <span class="n">df_map</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="n">map_path</span><span class="p">)</span>
+</span><span id="translate_codes-692"><a href="#translate_codes-692"><span class="linenos">692</span></a>
+</span><span id="translate_codes-693"><a href="#translate_codes-693"><span class="linenos">693</span></a> <span class="c1"># do mapping</span>
+</span><span id="translate_codes-694"><a href="#translate_codes-694"><span class="linenos">694</span></a> <span class="n">translated_df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">merge</span><span class="p">(</span>
+</span><span id="translate_codes-695"><a href="#translate_codes-695"><span class="linenos">695</span></a> <span class="n">source_df</span><span class="p">[</span><span class="n">source_code_type</span><span class="p">],</span> <span class="n">df_map</span><span class="p">,</span> <span class="n">how</span><span class="o">=</span><span class="s2">"left"</span>
+</span><span id="translate_codes-696"><a href="#translate_codes-696"><span class="linenos">696</span></a> <span class="p">)</span>
+</span><span id="translate_codes-697"><a href="#translate_codes-697"><span class="linenos">697</span></a>
+</span><span id="translate_codes-698"><a href="#translate_codes-698"><span class="linenos">698</span></a> <span class="c1"># normalise the output</span>
+</span><span id="translate_codes-699"><a href="#translate_codes-699"><span class="linenos">699</span></a> <span class="n">translated_df</span><span class="o">.</span><span class="n">columns</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">Index</span><span class="p">([</span><span class="s2">"SOURCE_CONCEPT"</span><span class="p">,</span> <span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="translate_codes-700"><a href="#translate_codes-700"><span class="linenos">700</span></a> <span class="n">translated_df</span><span class="p">[</span><span class="s2">"SOURCE_CONCEPT_TYPE"</span><span class="p">]</span> <span class="o">=</span> <span class="n">source_code_type</span>
+</span><span id="translate_codes-701"><a href="#translate_codes-701"><span class="linenos">701</span></a>
+</span><span id="translate_codes-702"><a href="#translate_codes-702"><span class="linenos">702</span></a> <span class="c1"># add to list of codes</span>
+</span><span id="translate_codes-703"><a href="#translate_codes-703"><span class="linenos">703</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">codes</span><span class="p">,</span> <span class="n">translated_df</span><span class="p">])</span>
+</span><span id="translate_codes-704"><a href="#translate_codes-704"><span class="linenos">704</span></a>
+</span><span id="translate_codes-705"><a href="#translate_codes-705"><span class="linenos">705</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="translate_codes-706"><a href="#translate_codes-706"><span class="linenos">706</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span>
+</span><span id="translate_codes-707"><a href="#translate_codes-707"><span class="linenos">707</span></a> <span class="sa">f</span><span class="s2">"No mapping from </span><span class="si">{</span><span class="n">source_code_type</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">, file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">map_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> does not exist"</span>
+</span><span id="translate_codes-708"><a href="#translate_codes-708"><span class="linenos">708</span></a> <span class="p">)</span>
+</span><span id="translate_codes-709"><a href="#translate_codes-709"><span class="linenos">709</span></a>
+</span><span id="translate_codes-710"><a href="#translate_codes-710"><span class="linenos">710</span></a> <span class="n">codes</span> <span class="o">=</span> <span class="n">codes</span><span class="o">.</span><span class="n">dropna</span><span class="p">()</span> <span class="c1"># delete NaNs</span>
+</span><span id="translate_codes-711"><a href="#translate_codes-711"><span class="linenos">711</span></a>
+</span><span id="translate_codes-712"><a href="#translate_codes-712"><span class="linenos">712</span></a> <span class="c1"># added concept set type to output if any translations</span>
+</span><span id="translate_codes-713"><a href="#translate_codes-713"><span class="linenos">713</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">codes</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="translate_codes-714"><a href="#translate_codes-714"><span class="linenos">714</span></a> <span class="n">codes</span><span class="p">[</span><span class="s2">"CONCEPT_SET"</span><span class="p">]</span> <span class="o">=</span> <span class="n">concept_name</span>
+</span><span id="translate_codes-715"><a href="#translate_codes-715"><span class="linenos">715</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="translate_codes-716"><a href="#translate_codes-716"><span class="linenos">716</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No codes converted with target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="translate_codes-717"><a href="#translate_codes-717"><span class="linenos">717</span></a>
+</span><span id="translate_codes-718"><a href="#translate_codes-718"><span class="linenos">718</span></a> <span class="k">return</span> <span class="n">codes</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Translates each source code type the source coding list into a target type and returns all conversions as a concept set</p>
+</div>
+
+
+ </section>
+ <section id="write_vocab_version">
+ <input id="write_vocab_version-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">write_vocab_version</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="write_vocab_version-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#write_vocab_version"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="write_vocab_version-739"><a href="#write_vocab_version-739"><span class="linenos">739</span></a><span class="k">def</span><span class="w"> </span><span class="nf">write_vocab_version</span><span class="p">(</span><span class="n">phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">):</span>
+</span><span id="write_vocab_version-740"><a href="#write_vocab_version-740"><span class="linenos">740</span></a> <span class="c1"># write the vocab version files</span>
+</span><span id="write_vocab_version-741"><a href="#write_vocab_version-741"><span class="linenos">741</span></a>
+</span><span id="write_vocab_version-742"><a href="#write_vocab_version-742"><span class="linenos">742</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">trud</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="write_vocab_version-743"><a href="#write_vocab_version-743"><span class="linenos">743</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="write_vocab_version-744"><a href="#write_vocab_version-744"><span class="linenos">744</span></a> <span class="sa">f</span><span class="s2">"TRUD version path </span><span class="si">{</span><span class="n">trud</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="si">}</span><span class="s2"> does not exist, please check TRUD is installed"</span>
+</span><span id="write_vocab_version-745"><a href="#write_vocab_version-745"><span class="linenos">745</span></a> <span class="p">)</span>
+</span><span id="write_vocab_version-746"><a href="#write_vocab_version-746"><span class="linenos">746</span></a>
+</span><span id="write_vocab_version-747"><a href="#write_vocab_version-747"><span class="linenos">747</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">omop</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="write_vocab_version-748"><a href="#write_vocab_version-748"><span class="linenos">748</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span>
+</span><span id="write_vocab_version-749"><a href="#write_vocab_version-749"><span class="linenos">749</span></a> <span class="sa">f</span><span class="s2">"OMOP version path </span><span class="si">{</span><span class="n">omop</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="si">}</span><span class="s2"> does not exist, please check OMOP is installed"</span>
+</span><span id="write_vocab_version-750"><a href="#write_vocab_version-750"><span class="linenos">750</span></a> <span class="p">)</span>
+</span><span id="write_vocab_version-751"><a href="#write_vocab_version-751"><span class="linenos">751</span></a>
+</span><span id="write_vocab_version-752"><a href="#write_vocab_version-752"><span class="linenos">752</span></a> <span class="k">with</span> <span class="n">trud</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="write_vocab_version-753"><a href="#write_vocab_version-753"><span class="linenos">753</span></a> <span class="n">trud_version</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="write_vocab_version-754"><a href="#write_vocab_version-754"><span class="linenos">754</span></a>
+</span><span id="write_vocab_version-755"><a href="#write_vocab_version-755"><span class="linenos">755</span></a> <span class="k">with</span> <span class="n">omop</span><span class="o">.</span><span class="n">VERSION_PATH</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="write_vocab_version-756"><a href="#write_vocab_version-756"><span class="linenos">756</span></a> <span class="n">omop_version</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="write_vocab_version-757"><a href="#write_vocab_version-757"><span class="linenos">757</span></a>
+</span><span id="write_vocab_version-758"><a href="#write_vocab_version-758"><span class="linenos">758</span></a> <span class="c1"># Create the combined YAML structure</span>
+</span><span id="write_vocab_version-759"><a href="#write_vocab_version-759"><span class="linenos">759</span></a> <span class="n">version_data</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="write_vocab_version-760"><a href="#write_vocab_version-760"><span class="linenos">760</span></a> <span class="s2">"versions"</span><span class="p">:</span> <span class="p">{</span>
+</span><span id="write_vocab_version-761"><a href="#write_vocab_version-761"><span class="linenos">761</span></a> <span class="s2">"acmc"</span><span class="p">:</span> <span class="n">acmc</span><span class="o">.</span><span class="n">__version__</span><span class="p">,</span>
+</span><span id="write_vocab_version-762"><a href="#write_vocab_version-762"><span class="linenos">762</span></a> <span class="s2">"trud"</span><span class="p">:</span> <span class="n">trud_version</span><span class="p">,</span>
+</span><span id="write_vocab_version-763"><a href="#write_vocab_version-763"><span class="linenos">763</span></a> <span class="s2">"omop"</span><span class="p">:</span> <span class="n">omop_version</span><span class="p">,</span>
+</span><span id="write_vocab_version-764"><a href="#write_vocab_version-764"><span class="linenos">764</span></a> <span class="p">}</span>
+</span><span id="write_vocab_version-765"><a href="#write_vocab_version-765"><span class="linenos">765</span></a> <span class="p">}</span>
+</span><span id="write_vocab_version-766"><a href="#write_vocab_version-766"><span class="linenos">766</span></a>
+</span><span id="write_vocab_version-767"><a href="#write_vocab_version-767"><span class="linenos">767</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">phen_path</span> <span class="o">/</span> <span class="n">VOCAB_VERSION_FILE</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="write_vocab_version-768"><a href="#write_vocab_version-768"><span class="linenos">768</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="write_vocab_version-769"><a href="#write_vocab_version-769"><span class="linenos">769</span></a> <span class="n">version_data</span><span class="p">,</span>
+</span><span id="write_vocab_version-770"><a href="#write_vocab_version-770"><span class="linenos">770</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="write_vocab_version-771"><a href="#write_vocab_version-771"><span class="linenos">771</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="write_vocab_version-772"><a href="#write_vocab_version-772"><span class="linenos">772</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="write_vocab_version-773"><a href="#write_vocab_version-773"><span class="linenos">773</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="write_vocab_version-774"><a href="#write_vocab_version-774"><span class="linenos">774</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="write_vocab_version-775"><a href="#write_vocab_version-775"><span class="linenos">775</span></a> <span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="map">
+ <input id="map-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">map</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="map-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#map"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="map-778"><a href="#map-778"><span class="linenos">778</span></a><span class="k">def</span><span class="w"> </span><span class="nf">map</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="map-779"><a href="#map-779"><span class="linenos">779</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Processing phenotype: </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="map-780"><a href="#map-780"><span class="linenos">780</span></a>
+</span><span id="map-781"><a href="#map-781"><span class="linenos">781</span></a> <span class="c1"># Validate configuration</span>
+</span><span id="map-782"><a href="#map-782"><span class="linenos">782</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="map-783"><a href="#map-783"><span class="linenos">783</span></a>
+</span><span id="map-784"><a href="#map-784"><span class="linenos">784</span></a> <span class="c1"># initialise paths</span>
+</span><span id="map-785"><a href="#map-785"><span class="linenos">785</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="map-786"><a href="#map-786"><span class="linenos">786</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="map-787"><a href="#map-787"><span class="linenos">787</span></a>
+</span><span id="map-788"><a href="#map-788"><span class="linenos">788</span></a> <span class="c1"># load configuration</span>
+</span><span id="map-789"><a href="#map-789"><span class="linenos">789</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="map-790"><a href="#map-790"><span class="linenos">790</span></a> <span class="n">config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="map-791"><a href="#map-791"><span class="linenos">791</span></a> <span class="n">phenotype</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">]</span>
+</span><span id="map-792"><a href="#map-792"><span class="linenos">792</span></a>
+</span><span id="map-793"><a href="#map-793"><span class="linenos">793</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">phenotype</span><span class="p">[</span><span class="s2">"map"</span><span class="p">])</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="map-794"><a href="#map-794"><span class="linenos">794</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No map codes defined in the phenotype configuration"</span><span class="p">)</span>
+</span><span id="map-795"><a href="#map-795"><span class="linenos">795</span></a>
+</span><span id="map-796"><a href="#map-796"><span class="linenos">796</span></a> <span class="k">if</span> <span class="n">target_code_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="n">target_code_type</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"map"</span><span class="p">]:</span>
+</span><span id="map-797"><a href="#map-797"><span class="linenos">797</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="map-798"><a href="#map-798"><span class="linenos">798</span></a> <span class="sa">f</span><span class="s2">"Target code type </span><span class="si">{</span><span class="n">target_code_type</span><span class="si">}</span><span class="s2"> not in phenotype configuration map </span><span class="si">{</span><span class="n">phenotype</span><span class="p">[</span><span class="s1">'map'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="map-799"><a href="#map-799"><span class="linenos">799</span></a> <span class="p">)</span>
+</span><span id="map-800"><a href="#map-800"><span class="linenos">800</span></a>
+</span><span id="map-801"><a href="#map-801"><span class="linenos">801</span></a> <span class="k">if</span> <span class="n">target_code_type</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="map-802"><a href="#map-802"><span class="linenos">802</span></a> <span class="n">_map_target_code_type</span><span class="p">(</span><span class="n">phen_path</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">,</span> <span class="n">target_code_type</span><span class="p">)</span>
+</span><span id="map-803"><a href="#map-803"><span class="linenos">803</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="map-804"><a href="#map-804"><span class="linenos">804</span></a> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">phenotype</span><span class="p">[</span><span class="s2">"map"</span><span class="p">]:</span>
+</span><span id="map-805"><a href="#map-805"><span class="linenos">805</span></a> <span class="n">_map_target_code_type</span><span class="p">(</span><span class="n">phen_path</span><span class="p">,</span> <span class="n">phenotype</span><span class="p">,</span> <span class="n">t</span><span class="p">)</span>
+</span><span id="map-806"><a href="#map-806"><span class="linenos">806</span></a>
+</span><span id="map-807"><a href="#map-807"><span class="linenos">807</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype processed successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="publish">
+ <input id="publish-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">publish</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">msg</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">increment</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s1">'patch'</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="publish-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#publish"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="publish-1000"><a href="#publish-1000"><span class="linenos">1000</span></a><span class="k">def</span><span class="w"> </span><span class="nf">publish</span><span class="p">(</span>
+</span><span id="publish-1001"><a href="#publish-1001"><span class="linenos">1001</span></a> <span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">msg</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">remote_url</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">increment</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="n">DEFAULT_VERSION_INC</span>
+</span><span id="publish-1002"><a href="#publish-1002"><span class="linenos">1002</span></a><span class="p">):</span>
+</span><span id="publish-1003"><a href="#publish-1003"><span class="linenos">1003</span></a><span class="w"> </span><span class="sd">"""Publishes updates to the phenotype by commiting all changes to the repo directory"""</span>
+</span><span id="publish-1004"><a href="#publish-1004"><span class="linenos">1004</span></a>
+</span><span id="publish-1005"><a href="#publish-1005"><span class="linenos">1005</span></a> <span class="c1"># Validate config</span>
+</span><span id="publish-1006"><a href="#publish-1006"><span class="linenos">1006</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="publish-1007"><a href="#publish-1007"><span class="linenos">1007</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="publish-1008"><a href="#publish-1008"><span class="linenos">1008</span></a>
+</span><span id="publish-1009"><a href="#publish-1009"><span class="linenos">1009</span></a> <span class="c1"># load git repo and set the branch</span>
+</span><span id="publish-1010"><a href="#publish-1010"><span class="linenos">1010</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="p">(</span><span class="n">phen_path</span><span class="p">)</span>
+</span><span id="publish-1011"><a href="#publish-1011"><span class="linenos">1011</span></a> <span class="k">if</span> <span class="n">DEFAULT_GIT_BRANCH</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">branches</span><span class="p">:</span>
+</span><span id="publish-1012"><a href="#publish-1012"><span class="linenos">1012</span></a> <span class="n">main_branch</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">heads</span><span class="p">[</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="p">]</span>
+</span><span id="publish-1013"><a href="#publish-1013"><span class="linenos">1013</span></a> <span class="n">main_branch</span><span class="o">.</span><span class="n">checkout</span><span class="p">()</span>
+</span><span id="publish-1014"><a href="#publish-1014"><span class="linenos">1014</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="publish-1015"><a href="#publish-1015"><span class="linenos">1015</span></a> <span class="k">raise</span> <span class="ne">AttributeError</span><span class="p">(</span>
+</span><span id="publish-1016"><a href="#publish-1016"><span class="linenos">1016</span></a> <span class="sa">f</span><span class="s2">"Phen repo does not contain the default branch </span><span class="si">{</span><span class="n">DEFAULT_GIT_BRANCH</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="publish-1017"><a href="#publish-1017"><span class="linenos">1017</span></a> <span class="p">)</span>
+</span><span id="publish-1018"><a href="#publish-1018"><span class="linenos">1018</span></a>
+</span><span id="publish-1019"><a href="#publish-1019"><span class="linenos">1019</span></a> <span class="c1"># check if any changes to publish</span>
+</span><span id="publish-1020"><a href="#publish-1020"><span class="linenos">1020</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">repo</span><span class="o">.</span><span class="n">is_dirty</span><span class="p">()</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">repo</span><span class="o">.</span><span class="n">untracked_files</span><span class="p">:</span>
+</span><span id="publish-1021"><a href="#publish-1021"><span class="linenos">1021</span></a> <span class="k">if</span> <span class="n">remote_url</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="s2">"origin"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">:</span>
+</span><span id="publish-1022"><a href="#publish-1022"><span class="linenos">1022</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"First publish to remote url </span><span class="si">{</span><span class="n">remote_url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="publish-1023"><a href="#publish-1023"><span class="linenos">1023</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="publish-1024"><a href="#publish-1024"><span class="linenos">1024</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Nothing to publish, no changes to the repo"</span><span class="p">)</span>
+</span><span id="publish-1025"><a href="#publish-1025"><span class="linenos">1025</span></a> <span class="k">return</span>
+</span><span id="publish-1026"><a href="#publish-1026"><span class="linenos">1026</span></a>
+</span><span id="publish-1027"><a href="#publish-1027"><span class="linenos">1027</span></a> <span class="c1"># get next version</span>
+</span><span id="publish-1028"><a href="#publish-1028"><span class="linenos">1028</span></a> <span class="n">new_version_str</span> <span class="o">=</span> <span class="n">_generate_version_tag</span><span class="p">(</span><span class="n">repo</span><span class="p">,</span> <span class="n">increment</span><span class="p">)</span>
+</span><span id="publish-1029"><a href="#publish-1029"><span class="linenos">1029</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"New version: </span><span class="si">{</span><span class="n">new_version_str</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="publish-1030"><a href="#publish-1030"><span class="linenos">1030</span></a>
+</span><span id="publish-1031"><a href="#publish-1031"><span class="linenos">1031</span></a> <span class="c1"># Write version in configuration file</span>
+</span><span id="publish-1032"><a href="#publish-1032"><span class="linenos">1032</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="publish-1033"><a href="#publish-1033"><span class="linenos">1033</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="publish-1034"><a href="#publish-1034"><span class="linenos">1034</span></a> <span class="n">config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="publish-1035"><a href="#publish-1035"><span class="linenos">1035</span></a>
+</span><span id="publish-1036"><a href="#publish-1036"><span class="linenos">1036</span></a> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"version"</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_version_str</span>
+</span><span id="publish-1037"><a href="#publish-1037"><span class="linenos">1037</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">config_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="publish-1038"><a href="#publish-1038"><span class="linenos">1038</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="publish-1039"><a href="#publish-1039"><span class="linenos">1039</span></a> <span class="n">config</span><span class="p">,</span>
+</span><span id="publish-1040"><a href="#publish-1040"><span class="linenos">1040</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="publish-1041"><a href="#publish-1041"><span class="linenos">1041</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="publish-1042"><a href="#publish-1042"><span class="linenos">1042</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="publish-1043"><a href="#publish-1043"><span class="linenos">1043</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="publish-1044"><a href="#publish-1044"><span class="linenos">1044</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="publish-1045"><a href="#publish-1045"><span class="linenos">1045</span></a> <span class="p">)</span>
+</span><span id="publish-1046"><a href="#publish-1046"><span class="linenos">1046</span></a>
+</span><span id="publish-1047"><a href="#publish-1047"><span class="linenos">1047</span></a> <span class="c1"># Add and commit changes to repo including version updates</span>
+</span><span id="publish-1048"><a href="#publish-1048"><span class="linenos">1048</span></a> <span class="n">commit_message</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Committing updates to phenotype </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="publish-1049"><a href="#publish-1049"><span class="linenos">1049</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="s2">"--all"</span><span class="p">)</span>
+</span><span id="publish-1050"><a href="#publish-1050"><span class="linenos">1050</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">index</span><span class="o">.</span><span class="n">commit</span><span class="p">(</span><span class="n">commit_message</span><span class="p">)</span>
+</span><span id="publish-1051"><a href="#publish-1051"><span class="linenos">1051</span></a>
+</span><span id="publish-1052"><a href="#publish-1052"><span class="linenos">1052</span></a> <span class="c1"># Add tag to the repo</span>
+</span><span id="publish-1053"><a href="#publish-1053"><span class="linenos">1053</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_tag</span><span class="p">(</span><span class="n">new_version_str</span><span class="p">)</span>
+</span><span id="publish-1054"><a href="#publish-1054"><span class="linenos">1054</span></a>
+</span><span id="publish-1055"><a href="#publish-1055"><span class="linenos">1055</span></a> <span class="c1"># push to origin if a remote repo</span>
+</span><span id="publish-1056"><a href="#publish-1056"><span class="linenos">1056</span></a> <span class="k">if</span> <span class="n">remote_url</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="s2">"origin"</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">:</span>
+</span><span id="publish-1057"><a href="#publish-1057"><span class="linenos">1057</span></a> <span class="n">git_url</span> <span class="o">=</span> <span class="n">_construct_git_url</span><span class="p">(</span><span class="n">remote_url</span><span class="p">)</span>
+</span><span id="publish-1058"><a href="#publish-1058"><span class="linenos">1058</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">create_remote</span><span class="p">(</span><span class="s2">"origin"</span><span class="p">,</span> <span class="n">git_url</span><span class="p">)</span>
+</span><span id="publish-1059"><a href="#publish-1059"><span class="linenos">1059</span></a>
+</span><span id="publish-1060"><a href="#publish-1060"><span class="linenos">1060</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="publish-1061"><a href="#publish-1061"><span class="linenos">1061</span></a> <span class="k">if</span> <span class="s2">"origin"</span> <span class="ow">in</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="p">:</span>
+</span><span id="publish-1062"><a href="#publish-1062"><span class="linenos">1062</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Remote 'origin' is set </span><span class="si">{</span><span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="o">.</span><span class="n">origin</span><span class="o">.</span><span class="n">url</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="publish-1063"><a href="#publish-1063"><span class="linenos">1063</span></a> <span class="n">origin</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">remotes</span><span class="o">.</span><span class="n">origin</span>
+</span><span id="publish-1064"><a href="#publish-1064"><span class="linenos">1064</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Pushing main branch to remote repo"</span><span class="p">)</span>
+</span><span id="publish-1065"><a href="#publish-1065"><span class="linenos">1065</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="s2">"--set-upstream"</span><span class="p">,</span> <span class="s2">"origin"</span><span class="p">,</span> <span class="s2">"main"</span><span class="p">)</span>
+</span><span id="publish-1066"><a href="#publish-1066"><span class="linenos">1066</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Pushing version tags to remote git repo"</span><span class="p">)</span>
+</span><span id="publish-1067"><a href="#publish-1067"><span class="linenos">1067</span></a> <span class="n">origin</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="n">tags</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="publish-1068"><a href="#publish-1068"><span class="linenos">1068</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Changes pushed to 'origin'"</span><span class="p">)</span>
+</span><span id="publish-1069"><a href="#publish-1069"><span class="linenos">1069</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="publish-1070"><a href="#publish-1070"><span class="linenos">1070</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="s2">"Remote 'origin' is not set"</span><span class="p">)</span>
+</span><span id="publish-1071"><a href="#publish-1071"><span class="linenos">1071</span></a> <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
+</span><span id="publish-1072"><a href="#publish-1072"><span class="linenos">1072</span></a> <span class="n">tag_ref</span> <span class="o">=</span> <span class="n">repo</span><span class="o">.</span><span class="n">tags</span><span class="p">[</span><span class="n">new_version_str</span><span class="p">]</span>
+</span><span id="publish-1073"><a href="#publish-1073"><span class="linenos">1073</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">delete_tag</span><span class="p">(</span><span class="n">tag_ref</span><span class="p">)</span>
+</span><span id="publish-1074"><a href="#publish-1074"><span class="linenos">1074</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">reset</span><span class="p">(</span><span class="s2">"--soft"</span><span class="p">,</span> <span class="s2">"HEAD~1"</span><span class="p">)</span>
+</span><span id="publish-1075"><a href="#publish-1075"><span class="linenos">1075</span></a> <span class="k">raise</span> <span class="n">e</span>
+</span><span id="publish-1076"><a href="#publish-1076"><span class="linenos">1076</span></a>
+</span><span id="publish-1077"><a href="#publish-1077"><span class="linenos">1077</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype published successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Publishes updates to the phenotype by commiting all changes to the repo directory</p>
+</div>
+
+
+ </section>
+ <section id="export">
+ <input id="export-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">export</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">version</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="export-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#export"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="export-1080"><a href="#export-1080"><span class="linenos">1080</span></a><span class="k">def</span><span class="w"> </span><span class="nf">export</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="export-1081"><a href="#export-1081"><span class="linenos">1081</span></a><span class="w"> </span><span class="sd">"""Exports a phen repo at a specific tagged version into a target directory"""</span>
+</span><span id="export-1082"><a href="#export-1082"><span class="linenos">1082</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Exporting phenotype </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="export-1083"><a href="#export-1083"><span class="linenos">1083</span></a>
+</span><span id="export-1084"><a href="#export-1084"><span class="linenos">1084</span></a> <span class="c1"># validate configuration</span>
+</span><span id="export-1085"><a href="#export-1085"><span class="linenos">1085</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="export-1086"><a href="#export-1086"><span class="linenos">1086</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="export-1087"><a href="#export-1087"><span class="linenos">1087</span></a>
+</span><span id="export-1088"><a href="#export-1088"><span class="linenos">1088</span></a> <span class="c1"># load configuration</span>
+</span><span id="export-1089"><a href="#export-1089"><span class="linenos">1089</span></a> <span class="n">config_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="export-1090"><a href="#export-1090"><span class="linenos">1090</span></a> <span class="k">with</span> <span class="n">config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="export-1091"><a href="#export-1091"><span class="linenos">1091</span></a> <span class="n">config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="export-1092"><a href="#export-1092"><span class="linenos">1092</span></a>
+</span><span id="export-1093"><a href="#export-1093"><span class="linenos">1093</span></a> <span class="n">map_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span>
+</span><span id="export-1094"><a href="#export-1094"><span class="linenos">1094</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">map_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="export-1095"><a href="#export-1095"><span class="linenos">1095</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">warning</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Map path does not exist '</span><span class="si">{</span><span class="n">map_path</span><span class="si">}</span><span class="s2">'"</span><span class="p">)</span>
+</span><span id="export-1096"><a href="#export-1096"><span class="linenos">1096</span></a>
+</span><span id="export-1097"><a href="#export-1097"><span class="linenos">1097</span></a> <span class="n">export_path</span> <span class="o">=</span> <span class="n">phen_path</span> <span class="o">/</span> <span class="n">OMOP_PATH</span>
+</span><span id="export-1098"><a href="#export-1098"><span class="linenos">1098</span></a> <span class="c1"># check export directory exists and if not create it</span>
+</span><span id="export-1099"><a href="#export-1099"><span class="linenos">1099</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">export_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="export-1100"><a href="#export-1100"><span class="linenos">1100</span></a> <span class="n">export_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="export-1101"><a href="#export-1101"><span class="linenos">1101</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"OMOP export directory '</span><span class="si">{</span><span class="n">export_path</span><span class="si">}</span><span class="s2">' created."</span><span class="p">)</span>
+</span><span id="export-1102"><a href="#export-1102"><span class="linenos">1102</span></a>
+</span><span id="export-1103"><a href="#export-1103"><span class="linenos">1103</span></a> <span class="c1"># omop export db</span>
+</span><span id="export-1104"><a href="#export-1104"><span class="linenos">1104</span></a> <span class="n">export_db_path</span> <span class="o">=</span> <span class="n">omop</span><span class="o">.</span><span class="n">export</span><span class="p">(</span>
+</span><span id="export-1105"><a href="#export-1105"><span class="linenos">1105</span></a> <span class="n">map_path</span><span class="p">,</span>
+</span><span id="export-1106"><a href="#export-1106"><span class="linenos">1106</span></a> <span class="n">export_path</span><span class="p">,</span>
+</span><span id="export-1107"><a href="#export-1107"><span class="linenos">1107</span></a> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"version"</span><span class="p">],</span>
+</span><span id="export-1108"><a href="#export-1108"><span class="linenos">1108</span></a> <span class="n">config</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"omop"</span><span class="p">],</span>
+</span><span id="export-1109"><a href="#export-1109"><span class="linenos">1109</span></a> <span class="p">)</span>
+</span><span id="export-1110"><a href="#export-1110"><span class="linenos">1110</span></a>
+</span><span id="export-1111"><a href="#export-1111"><span class="linenos">1111</span></a> <span class="c1"># write to tables</span>
+</span><span id="export-1112"><a href="#export-1112"><span class="linenos">1112</span></a> <span class="c1"># export as csv</span>
+</span><span id="export-1113"><a href="#export-1113"><span class="linenos">1113</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype exported successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Exports a phen repo at a specific tagged version into a target directory</p>
+</div>
+
+
+ </section>
+ <section id="copy">
+ <input id="copy-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">copy</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">target_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">version</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="copy-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#copy"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="copy-1116"><a href="#copy-1116"><span class="linenos">1116</span></a><span class="k">def</span><span class="w"> </span><span class="nf">copy</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">target_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="copy-1117"><a href="#copy-1117"><span class="linenos">1117</span></a><span class="w"> </span><span class="sd">"""Copys a phen repo at a specific tagged version into a target directory"""</span>
+</span><span id="copy-1118"><a href="#copy-1118"><span class="linenos">1118</span></a>
+</span><span id="copy-1119"><a href="#copy-1119"><span class="linenos">1119</span></a> <span class="c1"># Validate</span>
+</span><span id="copy-1120"><a href="#copy-1120"><span class="linenos">1120</span></a> <span class="n">validate</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="copy-1121"><a href="#copy-1121"><span class="linenos">1121</span></a> <span class="n">phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="copy-1122"><a href="#copy-1122"><span class="linenos">1122</span></a>
+</span><span id="copy-1123"><a href="#copy-1123"><span class="linenos">1123</span></a> <span class="c1"># Check target directory exists</span>
+</span><span id="copy-1124"><a href="#copy-1124"><span class="linenos">1124</span></a> <span class="n">target_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">target_dir</span><span class="p">)</span>
+</span><span id="copy-1125"><a href="#copy-1125"><span class="linenos">1125</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">target_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="copy-1126"><a href="#copy-1126"><span class="linenos">1126</span></a> <span class="k">raise</span> <span class="ne">FileNotFoundError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"The target directory </span><span class="si">{</span><span class="n">target_path</span><span class="si">}</span><span class="s2"> does not exist"</span><span class="p">)</span>
+</span><span id="copy-1127"><a href="#copy-1127"><span class="linenos">1127</span></a>
+</span><span id="copy-1128"><a href="#copy-1128"><span class="linenos">1128</span></a> <span class="c1"># Set copy directory</span>
+</span><span id="copy-1129"><a href="#copy-1129"><span class="linenos">1129</span></a> <span class="n">copy_path</span> <span class="o">=</span> <span class="n">target_path</span> <span class="o">/</span> <span class="n">version</span>
+</span><span id="copy-1130"><a href="#copy-1130"><span class="linenos">1130</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Copying repo </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">copy_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="copy-1131"><a href="#copy-1131"><span class="linenos">1131</span></a>
+</span><span id="copy-1132"><a href="#copy-1132"><span class="linenos">1132</span></a> <span class="k">if</span> <span class="p">(</span>
+</span><span id="copy-1133"><a href="#copy-1133"><span class="linenos">1133</span></a> <span class="n">copy_path</span><span class="o">.</span><span class="n">exists</span><span class="p">()</span> <span class="ow">and</span> <span class="n">copy_path</span><span class="o">.</span><span class="n">is_dir</span><span class="p">()</span>
+</span><span id="copy-1134"><a href="#copy-1134"><span class="linenos">1134</span></a> <span class="p">):</span> <span class="c1"># Check if it exists and is a directory</span>
+</span><span id="copy-1135"><a href="#copy-1135"><span class="linenos">1135</span></a> <span class="n">copy</span> <span class="o">=</span> <span class="n">_check_delete_dir</span><span class="p">(</span>
+</span><span id="copy-1136"><a href="#copy-1136"><span class="linenos">1136</span></a> <span class="n">copy_path</span><span class="p">,</span>
+</span><span id="copy-1137"><a href="#copy-1137"><span class="linenos">1137</span></a> <span class="sa">f</span><span class="s2">"The directory </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">copy_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2"> already exists. Do you want to overwrite? (yes/no): "</span><span class="p">,</span>
+</span><span id="copy-1138"><a href="#copy-1138"><span class="linenos">1138</span></a> <span class="p">)</span>
+</span><span id="copy-1139"><a href="#copy-1139"><span class="linenos">1139</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="copy-1140"><a href="#copy-1140"><span class="linenos">1140</span></a> <span class="n">copy</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="copy-1141"><a href="#copy-1141"><span class="linenos">1141</span></a>
+</span><span id="copy-1142"><a href="#copy-1142"><span class="linenos">1142</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">copy</span><span class="p">:</span>
+</span><span id="copy-1143"><a href="#copy-1143"><span class="linenos">1143</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Not copying the version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="copy-1144"><a href="#copy-1144"><span class="linenos">1144</span></a> <span class="k">return</span>
+</span><span id="copy-1145"><a href="#copy-1145"><span class="linenos">1145</span></a>
+</span><span id="copy-1146"><a href="#copy-1146"><span class="linenos">1146</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Cloning repo from </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">copy_path</span><span class="si">}</span><span class="s2">..."</span><span class="p">)</span>
+</span><span id="copy-1147"><a href="#copy-1147"><span class="linenos">1147</span></a> <span class="n">repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">phen_path</span><span class="p">,</span> <span class="n">copy_path</span><span class="p">)</span>
+</span><span id="copy-1148"><a href="#copy-1148"><span class="linenos">1148</span></a>
+</span><span id="copy-1149"><a href="#copy-1149"><span class="linenos">1149</span></a> <span class="c1"># Check out the latest commit or specified version</span>
+</span><span id="copy-1150"><a href="#copy-1150"><span class="linenos">1150</span></a> <span class="k">if</span> <span class="n">version</span><span class="p">:</span>
+</span><span id="copy-1151"><a href="#copy-1151"><span class="linenos">1151</span></a> <span class="c1"># Checkout a specific version (e.g., branch, tag, or commit hash)</span>
+</span><span id="copy-1152"><a href="#copy-1152"><span class="linenos">1152</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Checking out version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">..."</span><span class="p">)</span>
+</span><span id="copy-1153"><a href="#copy-1153"><span class="linenos">1153</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
+</span><span id="copy-1154"><a href="#copy-1154"><span class="linenos">1154</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="copy-1155"><a href="#copy-1155"><span class="linenos">1155</span></a> <span class="c1"># Checkout the latest commit (HEAD)</span>
+</span><span id="copy-1156"><a href="#copy-1156"><span class="linenos">1156</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Checking out the latest commit..."</span><span class="p">)</span>
+</span><span id="copy-1157"><a href="#copy-1157"><span class="linenos">1157</span></a> <span class="n">repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="s2">"HEAD"</span><span class="p">)</span>
+</span><span id="copy-1158"><a href="#copy-1158"><span class="linenos">1158</span></a>
+</span><span id="copy-1159"><a href="#copy-1159"><span class="linenos">1159</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Copied </span><span class="si">{</span><span class="n">phen_path</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">repo</span><span class="o">.</span><span class="n">head</span><span class="o">.</span><span class="n">commit</span><span class="o">.</span><span class="n">hexsha</span><span class="p">[:</span><span class="mi">7</span><span class="p">]</span><span class="si">}</span><span class="s2"> in </span><span class="si">{</span><span class="n">copy_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="copy-1160"><a href="#copy-1160"><span class="linenos">1160</span></a>
+</span><span id="copy-1161"><a href="#copy-1161"><span class="linenos">1161</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotype copied successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Copys a phen repo at a specific tagged version into a target directory</p>
+</div>
+
+
+ </section>
+ <section id="extract_concepts">
+ <input id="extract_concepts-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">extract_concepts</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">config_data</span><span class="p">:</span> <span class="nb">dict</span></span><span class="return-annotation">) -> <span class="n">Tuple</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>:</span></span>
+
+ <label class="view-source-button" for="extract_concepts-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#extract_concepts"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="extract_concepts-1165"><a href="#extract_concepts-1165"><span class="linenos">1165</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_concepts</span><span class="p">(</span><span class="n">config_data</span><span class="p">:</span> <span class="nb">dict</span><span class="p">)</span> <span class="o">-></span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">dict</span><span class="p">,</span> <span class="n">Set</span><span class="p">[</span><span class="nb">str</span><span class="p">]]:</span>
+</span><span id="extract_concepts-1166"><a href="#extract_concepts-1166"><span class="linenos">1166</span></a><span class="w"> </span><span class="sd">"""Extracts concepts as {name: file_path} dictionary and a name set."""</span>
+</span><span id="extract_concepts-1167"><a href="#extract_concepts-1167"><span class="linenos">1167</span></a> <span class="n">concepts_dict</span> <span class="o">=</span> <span class="p">{</span>
+</span><span id="extract_concepts-1168"><a href="#extract_concepts-1168"><span class="linenos">1168</span></a> <span class="n">item</span><span class="p">[</span><span class="s2">"name"</span><span class="p">]:</span> <span class="n">item</span><span class="p">[</span><span class="s2">"file"</span><span class="p">][</span><span class="s2">"path"</span><span class="p">]</span>
+</span><span id="extract_concepts-1169"><a href="#extract_concepts-1169"><span class="linenos">1169</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">config_data</span><span class="p">[</span><span class="s2">"phenotype"</span><span class="p">][</span><span class="s2">"concept_sets"</span><span class="p">]</span>
+</span><span id="extract_concepts-1170"><a href="#extract_concepts-1170"><span class="linenos">1170</span></a> <span class="p">}</span>
+</span><span id="extract_concepts-1171"><a href="#extract_concepts-1171"><span class="linenos">1171</span></a> <span class="n">name_set</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">concepts_dict</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
+</span><span id="extract_concepts-1172"><a href="#extract_concepts-1172"><span class="linenos">1172</span></a> <span class="k">return</span> <span class="n">concepts_dict</span><span class="p">,</span> <span class="n">name_set</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Extracts concepts as {name: file_path} dictionary and a name set.</p>
+</div>
+
+
+ </section>
+ <section id="diff_config">
+ <input id="diff_config-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">diff_config</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">old_config</span><span class="p">:</span> <span class="nb">dict</span>, </span><span class="param"><span class="n">new_config</span><span class="p">:</span> <span class="nb">dict</span></span><span class="return-annotation">) -> <span class="nb">str</span>:</span></span>
+
+ <label class="view-source-button" for="diff_config-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#diff_config"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="diff_config-1186"><a href="#diff_config-1186"><span class="linenos">1186</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff_config</span><span class="p">(</span><span class="n">old_config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">new_config</span><span class="p">:</span> <span class="nb">dict</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
+</span><span id="diff_config-1187"><a href="#diff_config-1187"><span class="linenos">1187</span></a> <span class="n">report</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="se">\n</span><span class="s2"># Changes to phenotype configuration</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1188"><a href="#diff_config-1188"><span class="linenos">1188</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"This compares changes in the phenotype configuration including added, removed and renamed concept sets and changes to concept set source concept code file paths</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_config-1189"><a href="#diff_config-1189"><span class="linenos">1189</span></a>
+</span><span id="diff_config-1190"><a href="#diff_config-1190"><span class="linenos">1190</span></a> <span class="n">old_concepts</span><span class="p">,</span> <span class="n">old_names</span> <span class="o">=</span> <span class="n">extract_concepts</span><span class="p">(</span><span class="n">old_config</span><span class="p">)</span>
+</span><span id="diff_config-1191"><a href="#diff_config-1191"><span class="linenos">1191</span></a> <span class="n">new_concepts</span><span class="p">,</span> <span class="n">new_names</span> <span class="o">=</span> <span class="n">extract_concepts</span><span class="p">(</span><span class="n">new_config</span><span class="p">)</span>
+</span><span id="diff_config-1192"><a href="#diff_config-1192"><span class="linenos">1192</span></a>
+</span><span id="diff_config-1193"><a href="#diff_config-1193"><span class="linenos">1193</span></a> <span class="c1"># Check added and removed names</span>
+</span><span id="diff_config-1194"><a href="#diff_config-1194"><span class="linenos">1194</span></a> <span class="n">added_names</span> <span class="o">=</span> <span class="n">new_names</span> <span class="o">-</span> <span class="n">old_names</span> <span class="c1"># Names that appear in new but not in old</span>
+</span><span id="diff_config-1195"><a href="#diff_config-1195"><span class="linenos">1195</span></a> <span class="n">removed_names</span> <span class="o">=</span> <span class="n">old_names</span> <span class="o">-</span> <span class="n">new_names</span> <span class="c1"># Names that were in old but not in new</span>
+</span><span id="diff_config-1196"><a href="#diff_config-1196"><span class="linenos">1196</span></a>
+</span><span id="diff_config-1197"><a href="#diff_config-1197"><span class="linenos">1197</span></a> <span class="c1"># find file path changes for unchanged names</span>
+</span><span id="diff_config-1198"><a href="#diff_config-1198"><span class="linenos">1198</span></a> <span class="n">unchanged_names</span> <span class="o">=</span> <span class="n">old_names</span> <span class="o">&</span> <span class="n">new_names</span> <span class="c1"># Names that exist in both</span>
+</span><span id="diff_config-1199"><a href="#diff_config-1199"><span class="linenos">1199</span></a> <span class="n">file_diff</span> <span class="o">=</span> <span class="n">DeepDiff</span><span class="p">(</span>
+</span><span id="diff_config-1200"><a href="#diff_config-1200"><span class="linenos">1200</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">old_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">unchanged_names</span><span class="p">},</span>
+</span><span id="diff_config-1201"><a href="#diff_config-1201"><span class="linenos">1201</span></a> <span class="p">{</span><span class="n">name</span><span class="p">:</span> <span class="n">new_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">unchanged_names</span><span class="p">},</span>
+</span><span id="diff_config-1202"><a href="#diff_config-1202"><span class="linenos">1202</span></a> <span class="p">)</span>
+</span><span id="diff_config-1203"><a href="#diff_config-1203"><span class="linenos">1203</span></a>
+</span><span id="diff_config-1204"><a href="#diff_config-1204"><span class="linenos">1204</span></a> <span class="c1"># Find renamed concepts (same file, different name)</span>
+</span><span id="diff_config-1205"><a href="#diff_config-1205"><span class="linenos">1205</span></a> <span class="n">renamed_concepts</span> <span class="o">=</span> <span class="p">[]</span>
+</span><span id="diff_config-1206"><a href="#diff_config-1206"><span class="linenos">1206</span></a> <span class="k">for</span> <span class="n">removed</span> <span class="ow">in</span> <span class="n">removed_names</span><span class="p">:</span>
+</span><span id="diff_config-1207"><a href="#diff_config-1207"><span class="linenos">1207</span></a> <span class="n">old_path</span> <span class="o">=</span> <span class="n">old_concepts</span><span class="p">[</span><span class="n">removed</span><span class="p">]</span>
+</span><span id="diff_config-1208"><a href="#diff_config-1208"><span class="linenos">1208</span></a> <span class="k">for</span> <span class="n">added</span> <span class="ow">in</span> <span class="n">added_names</span><span class="p">:</span>
+</span><span id="diff_config-1209"><a href="#diff_config-1209"><span class="linenos">1209</span></a> <span class="n">new_path</span> <span class="o">=</span> <span class="n">new_concepts</span><span class="p">[</span><span class="n">added</span><span class="p">]</span>
+</span><span id="diff_config-1210"><a href="#diff_config-1210"><span class="linenos">1210</span></a> <span class="k">if</span> <span class="n">old_path</span> <span class="o">==</span> <span class="n">new_path</span><span class="p">:</span>
+</span><span id="diff_config-1211"><a href="#diff_config-1211"><span class="linenos">1211</span></a> <span class="n">renamed_concepts</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">removed</span><span class="p">,</span> <span class="n">added</span><span class="p">))</span>
+</span><span id="diff_config-1212"><a href="#diff_config-1212"><span class="linenos">1212</span></a>
+</span><span id="diff_config-1213"><a href="#diff_config-1213"><span class="linenos">1213</span></a> <span class="c1"># Remove renamed concepts from added and removed sets</span>
+</span><span id="diff_config-1214"><a href="#diff_config-1214"><span class="linenos">1214</span></a> <span class="k">for</span> <span class="n">old_name</span><span class="p">,</span> <span class="n">new_name</span> <span class="ow">in</span> <span class="n">renamed_concepts</span><span class="p">:</span>
+</span><span id="diff_config-1215"><a href="#diff_config-1215"><span class="linenos">1215</span></a> <span class="n">added_names</span><span class="o">.</span><span class="n">discard</span><span class="p">(</span><span class="n">new_name</span><span class="p">)</span>
+</span><span id="diff_config-1216"><a href="#diff_config-1216"><span class="linenos">1216</span></a> <span class="n">removed_names</span><span class="o">.</span><span class="n">discard</span><span class="p">(</span><span class="n">old_name</span><span class="p">)</span>
+</span><span id="diff_config-1217"><a href="#diff_config-1217"><span class="linenos">1217</span></a>
+</span><span id="diff_config-1218"><a href="#diff_config-1218"><span class="linenos">1218</span></a> <span class="c1"># generate config report</span>
+</span><span id="diff_config-1219"><a href="#diff_config-1219"><span class="linenos">1219</span></a> <span class="k">if</span> <span class="n">added_names</span><span class="p">:</span>
+</span><span id="diff_config-1220"><a href="#diff_config-1220"><span class="linenos">1220</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Added Concepts</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1221"><a href="#diff_config-1221"><span class="linenos">1221</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">added_names</span><span class="p">:</span>
+</span><span id="diff_config-1222"><a href="#diff_config-1222"><span class="linenos">1222</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">` (File: `</span><span class="si">{</span><span class="n">new_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="si">}</span><span class="s2">`)</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1223"><a href="#diff_config-1223"><span class="linenos">1223</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1224"><a href="#diff_config-1224"><span class="linenos">1224</span></a>
+</span><span id="diff_config-1225"><a href="#diff_config-1225"><span class="linenos">1225</span></a> <span class="k">if</span> <span class="n">removed_names</span><span class="p">:</span>
+</span><span id="diff_config-1226"><a href="#diff_config-1226"><span class="linenos">1226</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Removed Concepts</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1227"><a href="#diff_config-1227"><span class="linenos">1227</span></a> <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">removed_names</span><span class="p">:</span>
+</span><span id="diff_config-1228"><a href="#diff_config-1228"><span class="linenos">1228</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2">` (File: `</span><span class="si">{</span><span class="n">old_concepts</span><span class="p">[</span><span class="n">name</span><span class="p">]</span><span class="si">}</span><span class="s2">`)</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1229"><a href="#diff_config-1229"><span class="linenos">1229</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1230"><a href="#diff_config-1230"><span class="linenos">1230</span></a>
+</span><span id="diff_config-1231"><a href="#diff_config-1231"><span class="linenos">1231</span></a> <span class="k">if</span> <span class="n">renamed_concepts</span><span class="p">:</span>
+</span><span id="diff_config-1232"><a href="#diff_config-1232"><span class="linenos">1232</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Renamed Concepts</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1233"><a href="#diff_config-1233"><span class="linenos">1233</span></a> <span class="k">for</span> <span class="n">old_name</span><span class="p">,</span> <span class="n">new_name</span> <span class="ow">in</span> <span class="n">renamed_concepts</span><span class="p">:</span>
+</span><span id="diff_config-1234"><a href="#diff_config-1234"><span class="linenos">1234</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="p">(</span>
+</span><span id="diff_config-1235"><a href="#diff_config-1235"><span class="linenos">1235</span></a> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">old_name</span><span class="si">}</span><span class="s2">` ➝ `</span><span class="si">{</span><span class="n">new_name</span><span class="si">}</span><span class="s2">` (File: `</span><span class="si">{</span><span class="n">old_concepts</span><span class="p">[</span><span class="n">old_name</span><span class="p">]</span><span class="si">}</span><span class="s2">`)</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1236"><a href="#diff_config-1236"><span class="linenos">1236</span></a> <span class="p">)</span>
+</span><span id="diff_config-1237"><a href="#diff_config-1237"><span class="linenos">1237</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1238"><a href="#diff_config-1238"><span class="linenos">1238</span></a>
+</span><span id="diff_config-1239"><a href="#diff_config-1239"><span class="linenos">1239</span></a> <span class="k">if</span> <span class="s2">"values_changed"</span> <span class="ow">in</span> <span class="n">file_diff</span><span class="p">:</span>
+</span><span id="diff_config-1240"><a href="#diff_config-1240"><span class="linenos">1240</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"## Updated File Paths</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1241"><a href="#diff_config-1241"><span class="linenos">1241</span></a> <span class="k">for</span> <span class="n">name</span><span class="p">,</span> <span class="n">change</span> <span class="ow">in</span> <span class="n">file_diff</span><span class="p">[</span><span class="s2">"values_changed"</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
+</span><span id="diff_config-1242"><a href="#diff_config-1242"><span class="linenos">1242</span></a> <span class="n">old_file</span> <span class="o">=</span> <span class="n">change</span><span class="p">[</span><span class="s2">"old_value"</span><span class="p">]</span>
+</span><span id="diff_config-1243"><a href="#diff_config-1243"><span class="linenos">1243</span></a> <span class="n">new_file</span> <span class="o">=</span> <span class="n">change</span><span class="p">[</span><span class="s2">"new_value"</span><span class="p">]</span>
+</span><span id="diff_config-1244"><a href="#diff_config-1244"><span class="linenos">1244</span></a> <span class="n">clean_name</span> <span class="o">=</span> <span class="n">name</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"root['"</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">"']"</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
+</span><span id="diff_config-1245"><a href="#diff_config-1245"><span class="linenos">1245</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="p">(</span>
+</span><span id="diff_config-1246"><a href="#diff_config-1246"><span class="linenos">1246</span></a> <span class="sa">f</span><span class="s2">"- `</span><span class="si">{</span><span class="n">clean_name</span><span class="si">}</span><span class="s2">` changed file from `</span><span class="si">{</span><span class="n">old_file</span><span class="si">}</span><span class="s2">` ➝ `</span><span class="si">{</span><span class="n">new_file</span><span class="si">}</span><span class="s2">`</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1247"><a href="#diff_config-1247"><span class="linenos">1247</span></a> <span class="p">)</span>
+</span><span id="diff_config-1248"><a href="#diff_config-1248"><span class="linenos">1248</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1249"><a href="#diff_config-1249"><span class="linenos">1249</span></a>
+</span><span id="diff_config-1250"><a href="#diff_config-1250"><span class="linenos">1250</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span>
+</span><span id="diff_config-1251"><a href="#diff_config-1251"><span class="linenos">1251</span></a> <span class="n">added_names</span>
+</span><span id="diff_config-1252"><a href="#diff_config-1252"><span class="linenos">1252</span></a> <span class="ow">or</span> <span class="n">removed_names</span>
+</span><span id="diff_config-1253"><a href="#diff_config-1253"><span class="linenos">1253</span></a> <span class="ow">or</span> <span class="n">renamed_concepts</span>
+</span><span id="diff_config-1254"><a href="#diff_config-1254"><span class="linenos">1254</span></a> <span class="ow">or</span> <span class="n">file_diff</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"values_changed"</span><span class="p">)</span>
+</span><span id="diff_config-1255"><a href="#diff_config-1255"><span class="linenos">1255</span></a> <span class="p">):</span>
+</span><span id="diff_config-1256"><a href="#diff_config-1256"><span class="linenos">1256</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="s2">"No changes in concept sets.</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_config-1257"><a href="#diff_config-1257"><span class="linenos">1257</span></a>
+</span><span id="diff_config-1258"><a href="#diff_config-1258"><span class="linenos">1258</span></a> <span class="k">return</span> <span class="n">report</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="diff_map_files">
+ <input id="diff_map_files-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">diff_map_files</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">old_map_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>, </span><span class="param"><span class="n">new_map_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">) -> <span class="nb">str</span>:</span></span>
+
+ <label class="view-source-button" for="diff_map_files-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#diff_map_files"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="diff_map_files-1261"><a href="#diff_map_files-1261"><span class="linenos">1261</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff_map_files</span><span class="p">(</span><span class="n">old_map_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span> <span class="n">new_map_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">)</span> <span class="o">-></span> <span class="nb">str</span><span class="p">:</span>
+</span><span id="diff_map_files-1262"><a href="#diff_map_files-1262"><span class="linenos">1262</span></a> <span class="n">old_output_files</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="diff_map_files-1263"><a href="#diff_map_files-1263"><span class="linenos">1263</span></a> <span class="n">file</span><span class="o">.</span><span class="n">name</span>
+</span><span id="diff_map_files-1264"><a href="#diff_map_files-1264"><span class="linenos">1264</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">old_map_path</span><span class="o">.</span><span class="n">iterdir</span><span class="p">()</span>
+</span><span id="diff_map_files-1265"><a href="#diff_map_files-1265"><span class="linenos">1265</span></a> <span class="k">if</span> <span class="n">file</span><span class="o">.</span><span class="n">is_file</span><span class="p">()</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">file</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"."</span><span class="p">)</span>
+</span><span id="diff_map_files-1266"><a href="#diff_map_files-1266"><span class="linenos">1266</span></a> <span class="p">]</span>
+</span><span id="diff_map_files-1267"><a href="#diff_map_files-1267"><span class="linenos">1267</span></a> <span class="n">new_output_files</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="diff_map_files-1268"><a href="#diff_map_files-1268"><span class="linenos">1268</span></a> <span class="n">file</span><span class="o">.</span><span class="n">name</span>
+</span><span id="diff_map_files-1269"><a href="#diff_map_files-1269"><span class="linenos">1269</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">new_map_path</span><span class="o">.</span><span class="n">iterdir</span><span class="p">()</span>
+</span><span id="diff_map_files-1270"><a href="#diff_map_files-1270"><span class="linenos">1270</span></a> <span class="k">if</span> <span class="n">file</span><span class="o">.</span><span class="n">is_file</span><span class="p">()</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">file</span><span class="o">.</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">"."</span><span class="p">)</span>
+</span><span id="diff_map_files-1271"><a href="#diff_map_files-1271"><span class="linenos">1271</span></a> <span class="p">]</span>
+</span><span id="diff_map_files-1272"><a href="#diff_map_files-1272"><span class="linenos">1272</span></a>
+</span><span id="diff_map_files-1273"><a href="#diff_map_files-1273"><span class="linenos">1273</span></a> <span class="c1"># Convert the lists to sets for easy comparison</span>
+</span><span id="diff_map_files-1274"><a href="#diff_map_files-1274"><span class="linenos">1274</span></a> <span class="n">old_output_set</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">old_output_files</span><span class="p">)</span>
+</span><span id="diff_map_files-1275"><a href="#diff_map_files-1275"><span class="linenos">1275</span></a> <span class="n">new_output_set</span> <span class="o">=</span> <span class="nb">set</span><span class="p">(</span><span class="n">new_output_files</span><span class="p">)</span>
+</span><span id="diff_map_files-1276"><a href="#diff_map_files-1276"><span class="linenos">1276</span></a>
+</span><span id="diff_map_files-1277"><a href="#diff_map_files-1277"><span class="linenos">1277</span></a> <span class="c1"># Outputs that are in old_output_set but not in new_output_set (removed files)</span>
+</span><span id="diff_map_files-1278"><a href="#diff_map_files-1278"><span class="linenos">1278</span></a> <span class="n">removed_outputs</span> <span class="o">=</span> <span class="n">old_output_set</span> <span class="o">-</span> <span class="n">new_output_set</span>
+</span><span id="diff_map_files-1279"><a href="#diff_map_files-1279"><span class="linenos">1279</span></a> <span class="c1"># Outputs that are in new_output_set but not in old_output_set (added files)</span>
+</span><span id="diff_map_files-1280"><a href="#diff_map_files-1280"><span class="linenos">1280</span></a> <span class="n">added_outputs</span> <span class="o">=</span> <span class="n">new_output_set</span> <span class="o">-</span> <span class="n">old_output_set</span>
+</span><span id="diff_map_files-1281"><a href="#diff_map_files-1281"><span class="linenos">1281</span></a> <span class="c1"># Outputs that are the intersection of old_output_set and new_output_set</span>
+</span><span id="diff_map_files-1282"><a href="#diff_map_files-1282"><span class="linenos">1282</span></a> <span class="n">common_outputs</span> <span class="o">=</span> <span class="n">old_output_set</span> <span class="o">&</span> <span class="n">new_output_set</span>
+</span><span id="diff_map_files-1283"><a href="#diff_map_files-1283"><span class="linenos">1283</span></a>
+</span><span id="diff_map_files-1284"><a href="#diff_map_files-1284"><span class="linenos">1284</span></a> <span class="n">report</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="se">\n</span><span class="s2"># Changes to available translations</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1285"><a href="#diff_map_files-1285"><span class="linenos">1285</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"This compares the coding translations files available.</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1286"><a href="#diff_map_files-1286"><span class="linenos">1286</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Removed outputs: </span><span class="si">{</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">removed_outputs</span><span class="p">))</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1287"><a href="#diff_map_files-1287"><span class="linenos">1287</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Added outputs: </span><span class="si">{</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">added_outputs</span><span class="p">))</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1288"><a href="#diff_map_files-1288"><span class="linenos">1288</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Common outputs: </span><span class="si">{</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">common_outputs</span><span class="p">))</span><span class="si">}</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1289"><a href="#diff_map_files-1289"><span class="linenos">1289</span></a>
+</span><span id="diff_map_files-1290"><a href="#diff_map_files-1290"><span class="linenos">1290</span></a> <span class="c1"># Step N: Compare common outputs between versions</span>
+</span><span id="diff_map_files-1291"><a href="#diff_map_files-1291"><span class="linenos">1291</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"# Changes to concepts in translation files</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1292"><a href="#diff_map_files-1292"><span class="linenos">1292</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"This compares the added and removed concepts in each of the coding translation files. Note that this might be different to the config.yaml if the translations have not been run for the current config.</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1293"><a href="#diff_map_files-1293"><span class="linenos">1293</span></a> <span class="k">for</span> <span class="n">file</span> <span class="ow">in</span> <span class="n">common_outputs</span><span class="p">:</span>
+</span><span id="diff_map_files-1294"><a href="#diff_map_files-1294"><span class="linenos">1294</span></a> <span class="n">old_output</span> <span class="o">=</span> <span class="n">old_map_path</span> <span class="o">/</span> <span class="n">file</span>
+</span><span id="diff_map_files-1295"><a href="#diff_map_files-1295"><span class="linenos">1295</span></a> <span class="n">new_output</span> <span class="o">=</span> <span class="n">new_map_path</span> <span class="o">/</span> <span class="n">file</span>
+</span><span id="diff_map_files-1296"><a href="#diff_map_files-1296"><span class="linenos">1296</span></a>
+</span><span id="diff_map_files-1297"><a href="#diff_map_files-1297"><span class="linenos">1297</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Old ouptput: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_output</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff_map_files-1298"><a href="#diff_map_files-1298"><span class="linenos">1298</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"New ouptput: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">new_output</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff_map_files-1299"><a href="#diff_map_files-1299"><span class="linenos">1299</span></a>
+</span><span id="diff_map_files-1300"><a href="#diff_map_files-1300"><span class="linenos">1300</span></a> <span class="n">df1</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">old_output</span><span class="p">)</span>
+</span><span id="diff_map_files-1301"><a href="#diff_map_files-1301"><span class="linenos">1301</span></a> <span class="n">df1</span> <span class="o">=</span> <span class="n">df1</span><span class="p">[[</span><span class="s2">"CONCEPT"</span><span class="p">,</span> <span class="s2">"CONCEPT_SET"</span><span class="p">]]</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">)</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
+</span><span id="diff_map_files-1302"><a href="#diff_map_files-1302"><span class="linenos">1302</span></a> <span class="n">df2</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">new_output</span><span class="p">)</span>
+</span><span id="diff_map_files-1303"><a href="#diff_map_files-1303"><span class="linenos">1303</span></a> <span class="n">df2</span> <span class="o">=</span> <span class="n">df2</span><span class="p">[[</span><span class="s2">"CONCEPT"</span><span class="p">,</span> <span class="s2">"CONCEPT_SET"</span><span class="p">]]</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s2">"CONCEPT_SET"</span><span class="p">)</span><span class="o">.</span><span class="n">count</span><span class="p">()</span>
+</span><span id="diff_map_files-1304"><a href="#diff_map_files-1304"><span class="linenos">1304</span></a>
+</span><span id="diff_map_files-1305"><a href="#diff_map_files-1305"><span class="linenos">1305</span></a> <span class="c1"># Check for added and removed concepts</span>
+</span><span id="diff_map_files-1306"><a href="#diff_map_files-1306"><span class="linenos">1306</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- File </span><span class="si">{</span><span class="n">file</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1307"><a href="#diff_map_files-1307"><span class="linenos">1307</span></a> <span class="n">sorted_list</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">df1</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">-</span> <span class="nb">set</span><span class="p">(</span><span class="n">df2</span><span class="o">.</span><span class="n">index</span><span class="p">)))</span>
+</span><span id="diff_map_files-1308"><a href="#diff_map_files-1308"><span class="linenos">1308</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Removed concepts </span><span class="si">{</span><span class="n">sorted_list</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1309"><a href="#diff_map_files-1309"><span class="linenos">1309</span></a> <span class="n">sorted_list</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">df2</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">-</span> <span class="nb">set</span><span class="p">(</span><span class="n">df1</span><span class="o">.</span><span class="n">index</span><span class="p">)))</span>
+</span><span id="diff_map_files-1310"><a href="#diff_map_files-1310"><span class="linenos">1310</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Added concepts </span><span class="si">{</span><span class="n">sorted_list</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1311"><a href="#diff_map_files-1311"><span class="linenos">1311</span></a>
+</span><span id="diff_map_files-1312"><a href="#diff_map_files-1312"><span class="linenos">1312</span></a> <span class="c1"># Check for changed concepts</span>
+</span><span id="diff_map_files-1313"><a href="#diff_map_files-1313"><span class="linenos">1313</span></a> <span class="n">diff</span> <span class="o">=</span> <span class="n">df2</span> <span class="o">-</span> <span class="n">df1</span> <span class="c1"># diff in counts</span>
+</span><span id="diff_map_files-1314"><a href="#diff_map_files-1314"><span class="linenos">1314</span></a> <span class="n">diff</span> <span class="o">=</span> <span class="n">diff</span><span class="p">[</span>
+</span><span id="diff_map_files-1315"><a href="#diff_map_files-1315"><span class="linenos">1315</span></a> <span class="p">(</span><span class="o">~</span><span class="p">(</span><span class="n">diff</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">]</span> <span class="o">==</span> <span class="mf">0.0</span><span class="p">))</span> <span class="o">&</span> <span class="n">diff</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">]</span><span class="o">.</span><span class="n">notna</span><span class="p">()</span>
+</span><span id="diff_map_files-1316"><a href="#diff_map_files-1316"><span class="linenos">1316</span></a> <span class="p">]</span> <span class="c1"># get non-zero counts</span>
+</span><span id="diff_map_files-1317"><a href="#diff_map_files-1317"><span class="linenos">1317</span></a> <span class="n">s</span> <span class="o">=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1318"><a href="#diff_map_files-1318"><span class="linenos">1318</span></a> <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">diff</span><span class="o">.</span><span class="n">index</span><span class="p">)</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span>
+</span><span id="diff_map_files-1319"><a href="#diff_map_files-1319"><span class="linenos">1319</span></a> <span class="k">for</span> <span class="n">concept</span><span class="p">,</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">diff</span><span class="o">.</span><span class="n">iterrows</span><span class="p">():</span>
+</span><span id="diff_map_files-1320"><a href="#diff_map_files-1320"><span class="linenos">1320</span></a> <span class="n">s</span> <span class="o">+=</span> <span class="s2">"</span><span class="se">\t</span><span class="s2"> - </span><span class="si">{}</span><span class="s2"> </span><span class="si">{}</span><span class="se">\n</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">concept</span><span class="p">,</span> <span class="n">row</span><span class="p">[</span><span class="s2">"CONCEPT"</span><span class="p">])</span>
+</span><span id="diff_map_files-1321"><a href="#diff_map_files-1321"><span class="linenos">1321</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Changed concepts </span><span class="si">{</span><span class="n">s</span><span class="si">}</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1322"><a href="#diff_map_files-1322"><span class="linenos">1322</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="diff_map_files-1323"><a href="#diff_map_files-1323"><span class="linenos">1323</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"- Changed concepts []</span><span class="se">\n\n</span><span class="s2">"</span>
+</span><span id="diff_map_files-1324"><a href="#diff_map_files-1324"><span class="linenos">1324</span></a>
+</span><span id="diff_map_files-1325"><a href="#diff_map_files-1325"><span class="linenos">1325</span></a> <span class="k">return</span> <span class="n">report</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="diff_phen">
+ <input id="diff_phen-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">diff_phen</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">new_phen_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>,</span><span class="param"> <span class="n">new_version</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">old_phen_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>,</span><span class="param"> <span class="n">old_version</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">report_path</span><span class="p">:</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="diff_phen-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#diff_phen"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="diff_phen-1328"><a href="#diff_phen-1328"><span class="linenos">1328</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff_phen</span><span class="p">(</span>
+</span><span id="diff_phen-1329"><a href="#diff_phen-1329"><span class="linenos">1329</span></a> <span class="n">new_phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span>
+</span><span id="diff_phen-1330"><a href="#diff_phen-1330"><span class="linenos">1330</span></a> <span class="n">new_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
+</span><span id="diff_phen-1331"><a href="#diff_phen-1331"><span class="linenos">1331</span></a> <span class="n">old_phen_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span>
+</span><span id="diff_phen-1332"><a href="#diff_phen-1332"><span class="linenos">1332</span></a> <span class="n">old_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
+</span><span id="diff_phen-1333"><a href="#diff_phen-1333"><span class="linenos">1333</span></a> <span class="n">report_path</span><span class="p">:</span> <span class="n">Path</span><span class="p">,</span>
+</span><span id="diff_phen-1334"><a href="#diff_phen-1334"><span class="linenos">1334</span></a><span class="p">):</span>
+</span><span id="diff_phen-1335"><a href="#diff_phen-1335"><span class="linenos">1335</span></a><span class="w"> </span><span class="sd">"""Compare the differences between two versions of a phenotype"""</span>
+</span><span id="diff_phen-1336"><a href="#diff_phen-1336"><span class="linenos">1336</span></a>
+</span><span id="diff_phen-1337"><a href="#diff_phen-1337"><span class="linenos">1337</span></a> <span class="c1"># validate phenotypes</span>
+</span><span id="diff_phen-1338"><a href="#diff_phen-1338"><span class="linenos">1338</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Validating for diff old path: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff_phen-1339"><a href="#diff_phen-1339"><span class="linenos">1339</span></a> <span class="n">validate</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()))</span>
+</span><span id="diff_phen-1340"><a href="#diff_phen-1340"><span class="linenos">1340</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Validating for diff new path: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">new_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff_phen-1341"><a href="#diff_phen-1341"><span class="linenos">1341</span></a> <span class="n">validate</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">new_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">()))</span>
+</span><span id="diff_phen-1342"><a href="#diff_phen-1342"><span class="linenos">1342</span></a>
+</span><span id="diff_phen-1343"><a href="#diff_phen-1343"><span class="linenos">1343</span></a> <span class="c1"># get old and new config</span>
+</span><span id="diff_phen-1344"><a href="#diff_phen-1344"><span class="linenos">1344</span></a> <span class="n">old_config_path</span> <span class="o">=</span> <span class="n">old_phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="diff_phen-1345"><a href="#diff_phen-1345"><span class="linenos">1345</span></a> <span class="k">with</span> <span class="n">old_config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="diff_phen-1346"><a href="#diff_phen-1346"><span class="linenos">1346</span></a> <span class="n">old_config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="diff_phen-1347"><a href="#diff_phen-1347"><span class="linenos">1347</span></a> <span class="n">new_config_path</span> <span class="o">=</span> <span class="n">new_phen_path</span> <span class="o">/</span> <span class="n">CONFIG_FILE</span>
+</span><span id="diff_phen-1348"><a href="#diff_phen-1348"><span class="linenos">1348</span></a> <span class="k">with</span> <span class="n">new_config_path</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="diff_phen-1349"><a href="#diff_phen-1349"><span class="linenos">1349</span></a> <span class="n">new_config</span> <span class="o">=</span> <span class="n">yaml</span><span class="o">.</span><span class="n">safe_load</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
+</span><span id="diff_phen-1350"><a href="#diff_phen-1350"><span class="linenos">1350</span></a>
+</span><span id="diff_phen-1351"><a href="#diff_phen-1351"><span class="linenos">1351</span></a> <span class="c1"># write report heading</span>
+</span><span id="diff_phen-1352"><a href="#diff_phen-1352"><span class="linenos">1352</span></a> <span class="n">report</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"# Phenotype Comparison Report</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1353"><a href="#diff_phen-1353"><span class="linenos">1353</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"## Original phenotype</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1354"><a href="#diff_phen-1354"><span class="linenos">1354</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">old_config</span><span class="p">[</span><span class="s1">'phenotype'</span><span class="p">][</span><span class="s1">'omop'</span><span class="p">][</span><span class="s1">'vocabulary_id'</span><span class="p">]</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1355"><a href="#diff_phen-1355"><span class="linenos">1355</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1356"><a href="#diff_phen-1356"><span class="linenos">1356</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1357"><a href="#diff_phen-1357"><span class="linenos">1357</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">"## Changed phenotype:</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1358"><a href="#diff_phen-1358"><span class="linenos">1358</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">new_config</span><span class="p">[</span><span class="s1">'phenotype'</span><span class="p">][</span><span class="s1">'omop'</span><span class="p">][</span><span class="s1">'vocabulary_id'</span><span class="p">]</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1359"><a href="#diff_phen-1359"><span class="linenos">1359</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="n">new_version</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1360"><a href="#diff_phen-1360"><span class="linenos">1360</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="sa">f</span><span class="s2">" - </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">new_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="se">\n</span><span class="s2">"</span>
+</span><span id="diff_phen-1361"><a href="#diff_phen-1361"><span class="linenos">1361</span></a>
+</span><span id="diff_phen-1362"><a href="#diff_phen-1362"><span class="linenos">1362</span></a> <span class="c1"># Step 1: check differences configuration files</span>
+</span><span id="diff_phen-1363"><a href="#diff_phen-1363"><span class="linenos">1363</span></a> <span class="c1"># Convert list of dicts into a dict: {name: file}</span>
+</span><span id="diff_phen-1364"><a href="#diff_phen-1364"><span class="linenos">1364</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="n">diff_config</span><span class="p">(</span><span class="n">old_config</span><span class="p">,</span> <span class="n">new_config</span><span class="p">)</span>
+</span><span id="diff_phen-1365"><a href="#diff_phen-1365"><span class="linenos">1365</span></a>
+</span><span id="diff_phen-1366"><a href="#diff_phen-1366"><span class="linenos">1366</span></a> <span class="c1"># Step 2: check differences between map files</span>
+</span><span id="diff_phen-1367"><a href="#diff_phen-1367"><span class="linenos">1367</span></a> <span class="c1"># List files from output directories</span>
+</span><span id="diff_phen-1368"><a href="#diff_phen-1368"><span class="linenos">1368</span></a> <span class="n">old_map_path</span> <span class="o">=</span> <span class="n">old_phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span>
+</span><span id="diff_phen-1369"><a href="#diff_phen-1369"><span class="linenos">1369</span></a> <span class="n">new_map_path</span> <span class="o">=</span> <span class="n">new_phen_path</span> <span class="o">/</span> <span class="n">MAP_DIR</span>
+</span><span id="diff_phen-1370"><a href="#diff_phen-1370"><span class="linenos">1370</span></a> <span class="n">report</span> <span class="o">+=</span> <span class="n">diff_map_files</span><span class="p">(</span><span class="n">old_map_path</span><span class="p">,</span> <span class="n">new_map_path</span><span class="p">)</span>
+</span><span id="diff_phen-1371"><a href="#diff_phen-1371"><span class="linenos">1371</span></a>
+</span><span id="diff_phen-1372"><a href="#diff_phen-1372"><span class="linenos">1372</span></a> <span class="c1"># initialise report file</span>
+</span><span id="diff_phen-1373"><a href="#diff_phen-1373"><span class="linenos">1373</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Writing to report file </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">report_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff_phen-1374"><a href="#diff_phen-1374"><span class="linenos">1374</span></a> <span class="n">report_file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">report_path</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span>
+</span><span id="diff_phen-1375"><a href="#diff_phen-1375"><span class="linenos">1375</span></a> <span class="n">report_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">report</span><span class="p">)</span>
+</span><span id="diff_phen-1376"><a href="#diff_phen-1376"><span class="linenos">1376</span></a> <span class="n">report_file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+</span><span id="diff_phen-1377"><a href="#diff_phen-1377"><span class="linenos">1377</span></a>
+</span><span id="diff_phen-1378"><a href="#diff_phen-1378"><span class="linenos">1378</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Phenotypes diff'd successfully"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Compare the differences between two versions of a phenotype</p>
+</div>
+
+
+ </section>
+ <section id="diff">
+ <input id="diff-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">diff</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">version</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">old_phen_dir</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">old_version</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="diff-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#diff"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="diff-1381"><a href="#diff-1381"><span class="linenos">1381</span></a><span class="k">def</span><span class="w"> </span><span class="nf">diff</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">version</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">old_phen_dir</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">old_version</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="diff-1382"><a href="#diff-1382"><span class="linenos">1382</span></a> <span class="c1"># make tmp directory .acmc</span>
+</span><span id="diff-1383"><a href="#diff-1383"><span class="linenos">1383</span></a> <span class="n">timestamp</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="s2">"%Y%m</span><span class="si">%d</span><span class="s2">_%H%M%S"</span><span class="p">)</span>
+</span><span id="diff-1384"><a href="#diff-1384"><span class="linenos">1384</span></a> <span class="n">temp_dir</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="sa">f</span><span class="s2">".acmc/diff_</span><span class="si">{</span><span class="n">timestamp</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff-1385"><a href="#diff-1385"><span class="linenos">1385</span></a>
+</span><span id="diff-1386"><a href="#diff-1386"><span class="linenos">1386</span></a> <span class="n">changed_phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">phen_dir</span><span class="p">)</span>
+</span><span id="diff-1387"><a href="#diff-1387"><span class="linenos">1387</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">changed_phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="diff-1388"><a href="#diff-1388"><span class="linenos">1388</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="diff-1389"><a href="#diff-1389"><span class="linenos">1389</span></a> <span class="sa">f</span><span class="s2">"Changed phenotype directory does not exist: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">changed_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="diff-1390"><a href="#diff-1390"><span class="linenos">1390</span></a> <span class="p">)</span>
+</span><span id="diff-1391"><a href="#diff-1391"><span class="linenos">1391</span></a>
+</span><span id="diff-1392"><a href="#diff-1392"><span class="linenos">1392</span></a> <span class="n">old_phen_path</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="n">old_phen_dir</span><span class="p">)</span>
+</span><span id="diff-1393"><a href="#diff-1393"><span class="linenos">1393</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">old_phen_path</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="diff-1394"><a href="#diff-1394"><span class="linenos">1394</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="diff-1395"><a href="#diff-1395"><span class="linenos">1395</span></a> <span class="sa">f</span><span class="s2">"Old phenotype directory does not exist: </span><span class="si">{</span><span class="nb">str</span><span class="p">(</span><span class="n">old_phen_path</span><span class="o">.</span><span class="n">resolve</span><span class="p">())</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="diff-1396"><a href="#diff-1396"><span class="linenos">1396</span></a> <span class="p">)</span>
+</span><span id="diff-1397"><a href="#diff-1397"><span class="linenos">1397</span></a>
+</span><span id="diff-1398"><a href="#diff-1398"><span class="linenos">1398</span></a> <span class="k">try</span><span class="p">:</span>
+</span><span id="diff-1399"><a href="#diff-1399"><span class="linenos">1399</span></a> <span class="c1"># Create the directory</span>
+</span><span id="diff-1400"><a href="#diff-1400"><span class="linenos">1400</span></a> <span class="n">temp_dir</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="diff-1401"><a href="#diff-1401"><span class="linenos">1401</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Temporary directory created: </span><span class="si">{</span><span class="n">temp_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="diff-1402"><a href="#diff-1402"><span class="linenos">1402</span></a>
+</span><span id="diff-1403"><a href="#diff-1403"><span class="linenos">1403</span></a> <span class="c1"># Create temporary directories</span>
+</span><span id="diff-1404"><a href="#diff-1404"><span class="linenos">1404</span></a> <span class="n">changed_path</span> <span class="o">=</span> <span class="n">temp_dir</span> <span class="o">/</span> <span class="s2">"changed"</span>
+</span><span id="diff-1405"><a href="#diff-1405"><span class="linenos">1405</span></a> <span class="n">changed_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="diff-1406"><a href="#diff-1406"><span class="linenos">1406</span></a> <span class="n">old_path</span> <span class="o">=</span> <span class="n">temp_dir</span> <span class="o">/</span> <span class="s2">"old"</span>
+</span><span id="diff-1407"><a href="#diff-1407"><span class="linenos">1407</span></a> <span class="n">old_path</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="diff-1408"><a href="#diff-1408"><span class="linenos">1408</span></a>
+</span><span id="diff-1409"><a href="#diff-1409"><span class="linenos">1409</span></a> <span class="c1"># checkout changed</span>
+</span><span id="diff-1410"><a href="#diff-1410"><span class="linenos">1410</span></a> <span class="k">if</span> <span class="n">version</span> <span class="o">==</span> <span class="s2">"latest"</span><span class="p">:</span>
+</span><span id="diff-1411"><a href="#diff-1411"><span class="linenos">1411</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="diff-1412"><a href="#diff-1412"><span class="linenos">1412</span></a> <span class="sa">f</span><span class="s2">"Copying changed repo from </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">changed_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="diff-1413"><a href="#diff-1413"><span class="linenos">1413</span></a> <span class="p">)</span>
+</span><span id="diff-1414"><a href="#diff-1414"><span class="linenos">1414</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">copytree</span><span class="p">(</span><span class="n">changed_phen_path</span><span class="p">,</span> <span class="n">changed_path</span><span class="p">,</span> <span class="n">dirs_exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="diff-1415"><a href="#diff-1415"><span class="linenos">1415</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="diff-1416"><a href="#diff-1416"><span class="linenos">1416</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="diff-1417"><a href="#diff-1417"><span class="linenos">1417</span></a> <span class="sa">f</span><span class="s2">"Cloning changed repo from </span><span class="si">{</span><span class="n">phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">changed_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="diff-1418"><a href="#diff-1418"><span class="linenos">1418</span></a> <span class="p">)</span>
+</span><span id="diff-1419"><a href="#diff-1419"><span class="linenos">1419</span></a> <span class="n">changed_repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">changed_phen_path</span><span class="p">,</span> <span class="n">changed_path</span><span class="p">)</span>
+</span><span id="diff-1420"><a href="#diff-1420"><span class="linenos">1420</span></a> <span class="n">changed_repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">version</span><span class="p">)</span>
+</span><span id="diff-1421"><a href="#diff-1421"><span class="linenos">1421</span></a>
+</span><span id="diff-1422"><a href="#diff-1422"><span class="linenos">1422</span></a> <span class="c1"># checkout old</span>
+</span><span id="diff-1423"><a href="#diff-1423"><span class="linenos">1423</span></a> <span class="k">if</span> <span class="n">old_version</span> <span class="o">==</span> <span class="s2">"latest"</span><span class="p">:</span>
+</span><span id="diff-1424"><a href="#diff-1424"><span class="linenos">1424</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="diff-1425"><a href="#diff-1425"><span class="linenos">1425</span></a> <span class="sa">f</span><span class="s2">"Copying old repo from </span><span class="si">{</span><span class="n">old_phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">old_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="diff-1426"><a href="#diff-1426"><span class="linenos">1426</span></a> <span class="p">)</span>
+</span><span id="diff-1427"><a href="#diff-1427"><span class="linenos">1427</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">copytree</span><span class="p">(</span><span class="n">old_phen_path</span><span class="p">,</span> <span class="n">old_path</span><span class="p">,</span> <span class="n">dirs_exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="diff-1428"><a href="#diff-1428"><span class="linenos">1428</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="diff-1429"><a href="#diff-1429"><span class="linenos">1429</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span>
+</span><span id="diff-1430"><a href="#diff-1430"><span class="linenos">1430</span></a> <span class="sa">f</span><span class="s2">"Cloning old repo from </span><span class="si">{</span><span class="n">old_phen_dir</span><span class="si">}</span><span class="s2"> into </span><span class="si">{</span><span class="n">old_path</span><span class="si">}</span><span class="s2"> at version </span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="s2">..."</span>
+</span><span id="diff-1431"><a href="#diff-1431"><span class="linenos">1431</span></a> <span class="p">)</span>
+</span><span id="diff-1432"><a href="#diff-1432"><span class="linenos">1432</span></a> <span class="n">old_repo</span> <span class="o">=</span> <span class="n">git</span><span class="o">.</span><span class="n">Repo</span><span class="o">.</span><span class="n">clone_from</span><span class="p">(</span><span class="n">old_phen_dir</span><span class="p">,</span> <span class="n">old_path</span><span class="p">)</span>
+</span><span id="diff-1433"><a href="#diff-1433"><span class="linenos">1433</span></a> <span class="n">old_repo</span><span class="o">.</span><span class="n">git</span><span class="o">.</span><span class="n">checkout</span><span class="p">(</span><span class="n">old_version</span><span class="p">)</span>
+</span><span id="diff-1434"><a href="#diff-1434"><span class="linenos">1434</span></a>
+</span><span id="diff-1435"><a href="#diff-1435"><span class="linenos">1435</span></a> <span class="n">report_filename</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">version</span><span class="si">}</span><span class="s2">_</span><span class="si">{</span><span class="n">old_version</span><span class="si">}</span><span class="s2">_diff.md"</span>
+</span><span id="diff-1436"><a href="#diff-1436"><span class="linenos">1436</span></a> <span class="n">report_path</span> <span class="o">=</span> <span class="n">changed_phen_path</span> <span class="o">/</span> <span class="n">report_filename</span>
+</span><span id="diff-1437"><a href="#diff-1437"><span class="linenos">1437</span></a> <span class="c1"># diff old with new</span>
+</span><span id="diff-1438"><a href="#diff-1438"><span class="linenos">1438</span></a> <span class="n">diff_phen</span><span class="p">(</span><span class="n">changed_path</span><span class="p">,</span> <span class="n">version</span><span class="p">,</span> <span class="n">old_path</span><span class="p">,</span> <span class="n">old_version</span><span class="p">,</span> <span class="n">report_path</span><span class="p">)</span>
+</span><span id="diff-1439"><a href="#diff-1439"><span class="linenos">1439</span></a>
+</span><span id="diff-1440"><a href="#diff-1440"><span class="linenos">1440</span></a> <span class="k">finally</span><span class="p">:</span>
+</span><span id="diff-1441"><a href="#diff-1441"><span class="linenos">1441</span></a> <span class="c1"># clean up tmp directory</span>
+</span><span id="diff-1442"><a href="#diff-1442"><span class="linenos">1442</span></a> <span class="k">if</span> <span class="n">temp_dir</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="diff-1443"><a href="#diff-1443"><span class="linenos">1443</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">temp_dir</span><span class="p">)</span>
+</span><span id="diff-1444"><a href="#diff-1444"><span class="linenos">1444</span></a> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Temporary directory removed: </span><span class="si">{</span><span class="n">temp_dir</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/trud.html b/docs/api/acmc/trud.html
new file mode 100644
index 0000000000000000000000000000000000000000..c423eaacaa4d3e876c612af3db9f12a6f178dbdd
--- /dev/null
+++ b/docs/api/acmc/trud.html
@@ -0,0 +1,1401 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.trud API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="variable" href="#FQDN">FQDN</a>
+ </li>
+ <li>
+ <a class="variable" href="#VOCAB_PATH">VOCAB_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#VERSION_FILE">VERSION_FILE</a>
+ </li>
+ <li>
+ <a class="variable" href="#VERSION_PATH">VERSION_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#DOWNLOADS_PATH">DOWNLOADS_PATH</a>
+ </li>
+ <li>
+ <a class="variable" href="#PROCESSED_PATH">PROCESSED_PATH</a>
+ </li>
+ <li>
+ <a class="function" href="#get_releases">get_releases</a>
+ </li>
+ <li>
+ <a class="function" href="#download_release_file">download_release_file</a>
+ </li>
+ <li>
+ <a class="function" href="#validate_download_hash">validate_download_hash</a>
+ </li>
+ <li>
+ <a class="function" href="#unzip_download">unzip_download</a>
+ </li>
+ <li>
+ <a class="function" href="#extract_icd10">extract_icd10</a>
+ </li>
+ <li>
+ <a class="function" href="#extract_opsc4">extract_opsc4</a>
+ </li>
+ <li>
+ <a class="function" href="#extract_nhs_data_migrations">extract_nhs_data_migrations</a>
+ </li>
+ <li>
+ <a class="function" href="#extract_nhs_read_browser">extract_nhs_read_browser</a>
+ </li>
+ <li>
+ <a class="function" href="#create_map_directories">create_map_directories</a>
+ </li>
+ <li>
+ <a class="function" href="#install">install</a>
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.trud </h1>
+
+ <div class="docstring"><p>trud.py module</p>
+
+<p>This module provides functionality to manage installation of the NHS TRUD vocabularies.</p>
+</div>
+
+ <input id="mod-trud-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-trud-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="sd">"""</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a><span class="sd">trud.py module</span>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">This module provides functionality to manage installation of the NHS TRUD vocabularies.</span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="sd">"""</span>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">os</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">sys</span>
+</span><span id="L-10"><a href="#L-10"><span class="linenos"> 10</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">requests</span>
+</span><span id="L-11"><a href="#L-11"><span class="linenos"> 11</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">argparse</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos"> 12</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">shutil</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos"> 13</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">hashlib</span>
+</span><span id="L-14"><a href="#L-14"><span class="linenos"> 14</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">zipfile</span>
+</span><span id="L-15"><a href="#L-15"><span class="linenos"> 15</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
+</span><span id="L-16"><a href="#L-16"><span class="linenos"> 16</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">simpledbf</span> <span class="c1"># type: ignore</span>
+</span><span id="L-17"><a href="#L-17"><span class="linenos"> 17</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">yaml</span>
+</span><span id="L-18"><a href="#L-18"><span class="linenos"> 18</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">pathlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">Path</span>
+</span><span id="L-19"><a href="#L-19"><span class="linenos"> 19</span></a><span class="kn">from</span><span class="w"> </span><span class="nn">acmc</span><span class="w"> </span><span class="kn">import</span> <span class="n">util</span><span class="p">,</span> <span class="n">logging_config</span> <span class="k">as</span> <span class="n">lc</span>
+</span><span id="L-20"><a href="#L-20"><span class="linenos"> 20</span></a>
+</span><span id="L-21"><a href="#L-21"><span class="linenos"> 21</span></a><span class="c1"># setup logging</span>
+</span><span id="L-22"><a href="#L-22"><span class="linenos"> 22</span></a><span class="n">_logger</span> <span class="o">=</span> <span class="n">lc</span><span class="o">.</span><span class="n">setup_logger</span><span class="p">()</span>
+</span><span id="L-23"><a href="#L-23"><span class="linenos"> 23</span></a>
+</span><span id="L-24"><a href="#L-24"><span class="linenos"> 24</span></a><span class="n">FQDN</span> <span class="o">=</span> <span class="s2">"isd.digital.nhs.uk"</span>
+</span><span id="L-25"><a href="#L-25"><span class="linenos"> 25</span></a><span class="sd">"""Fully Qualified Domain Name of NHS digital TRUD service API"""</span>
+</span><span id="L-26"><a href="#L-26"><span class="linenos"> 26</span></a>
+</span><span id="L-27"><a href="#L-27"><span class="linenos"> 27</span></a><span class="n">VOCAB_PATH</span> <span class="o">=</span> <span class="n">Path</span><span class="p">(</span><span class="s2">"./vocab/trud"</span><span class="p">)</span>
+</span><span id="L-28"><a href="#L-28"><span class="linenos"> 28</span></a><span class="sd">"""Default path to the TRUD vocabulary directory relative to the the acmc execution directory"""</span>
+</span><span id="L-29"><a href="#L-29"><span class="linenos"> 29</span></a>
+</span><span id="L-30"><a href="#L-30"><span class="linenos"> 30</span></a><span class="n">VERSION_FILE</span> <span class="o">=</span> <span class="s2">"trud_version.yml"</span>
+</span><span id="L-31"><a href="#L-31"><span class="linenos"> 31</span></a><span class="sd">"""TRUD version file"""</span>
+</span><span id="L-32"><a href="#L-32"><span class="linenos"> 32</span></a>
+</span><span id="L-33"><a href="#L-33"><span class="linenos"> 33</span></a><span class="n">VERSION_PATH</span> <span class="o">=</span> <span class="n">VOCAB_PATH</span> <span class="o">/</span> <span class="n">VERSION_FILE</span>
+</span><span id="L-34"><a href="#L-34"><span class="linenos"> 34</span></a><span class="sd">"""Default path to the TRUD version file"""</span>
+</span><span id="L-35"><a href="#L-35"><span class="linenos"> 35</span></a>
+</span><span id="L-36"><a href="#L-36"><span class="linenos"> 36</span></a><span class="n">DOWNLOADS_PATH</span> <span class="o">=</span> <span class="n">VOCAB_PATH</span> <span class="o">/</span> <span class="s2">"downloads"</span>
+</span><span id="L-37"><a href="#L-37"><span class="linenos"> 37</span></a><span class="sd">"""Default path to the TRUD vocabulary downloads directory"""</span>
+</span><span id="L-38"><a href="#L-38"><span class="linenos"> 38</span></a>
+</span><span id="L-39"><a href="#L-39"><span class="linenos"> 39</span></a><span class="n">PROCESSED_PATH</span> <span class="o">=</span> <span class="n">VOCAB_PATH</span> <span class="o">/</span> <span class="s2">"processed"</span>
+</span><span id="L-40"><a href="#L-40"><span class="linenos"> 40</span></a><span class="sd">""" Default path to the processed TRUD mappings directory"""</span>
+</span><span id="L-41"><a href="#L-41"><span class="linenos"> 41</span></a>
+</span><span id="L-42"><a href="#L-42"><span class="linenos"> 42</span></a>
+</span><span id="L-43"><a href="#L-43"><span class="linenos"> 43</span></a><span class="k">def</span><span class="w"> </span><span class="nf">get_releases</span><span class="p">(</span><span class="n">item_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">API_KEY</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">latest</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="o">-></span> <span class="nb">list</span><span class="p">:</span>
+</span><span id="L-44"><a href="#L-44"><span class="linenos"> 44</span></a><span class="w"> </span><span class="sd">"""Retrieve release information for an item from the TRUD API."""</span>
+</span><span id="L-45"><a href="#L-45"><span class="linenos"> 45</span></a> <span class="n">url</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"https://</span><span class="si">{</span><span class="n">FQDN</span><span class="si">}</span><span class="s2">/trud/api/v1/keys/</span><span class="si">{</span><span class="n">API_KEY</span><span class="si">}</span><span class="s2">/items/</span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">/releases"</span>
+</span><span id="L-46"><a href="#L-46"><span class="linenos"> 46</span></a> <span class="k">if</span> <span class="n">latest</span><span class="p">:</span>
+</span><span id="L-47"><a href="#L-47"><span class="linenos"> 47</span></a> <span class="n">url</span> <span class="o">+=</span> <span class="s2">"?latest"</span>
+</span><span id="L-48"><a href="#L-48"><span class="linenos"> 48</span></a>
+</span><span id="L-49"><a href="#L-49"><span class="linenos"> 49</span></a> <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
+</span><span id="L-50"><a href="#L-50"><span class="linenos"> 50</span></a> <span class="k">if</span> <span class="n">response</span><span class="o">.</span><span class="n">status_code</span> <span class="o">!=</span> <span class="mi">200</span><span class="p">:</span>
+</span><span id="L-51"><a href="#L-51"><span class="linenos"> 51</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
+</span><span id="L-52"><a href="#L-52"><span class="linenos"> 52</span></a> <span class="sa">f</span><span class="s2">"Failed to fetch releases for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">. Status code: </span><span class="si">{</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="si">}</span><span class="s2">, error </span><span class="si">{</span><span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()[</span><span class="s1">'message'</span><span class="p">]</span><span class="si">}</span><span class="s2">. If no releases found for API key, please ensure you are subscribed to the data release and that it is not pending approval"</span>
+</span><span id="L-53"><a href="#L-53"><span class="linenos"> 53</span></a> <span class="p">)</span>
+</span><span id="L-54"><a href="#L-54"><span class="linenos"> 54</span></a> <span class="n">response</span><span class="o">.</span><span class="n">raise_for_status</span><span class="p">()</span>
+</span><span id="L-55"><a href="#L-55"><span class="linenos"> 55</span></a>
+</span><span id="L-56"><a href="#L-56"><span class="linenos"> 56</span></a> <span class="n">data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
+</span><span id="L-57"><a href="#L-57"><span class="linenos"> 57</span></a> <span class="k">if</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"message"</span><span class="p">)</span> <span class="o">!=</span> <span class="s2">"OK"</span><span class="p">:</span>
+</span><span id="L-58"><a href="#L-58"><span class="linenos"> 58</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Unknown error occurred </span><span class="si">{</span><span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'message'</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-59"><a href="#L-59"><span class="linenos"> 59</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-60"><a href="#L-60"><span class="linenos"> 60</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-61"><a href="#L-61"><span class="linenos"> 61</span></a>
+</span><span id="L-62"><a href="#L-62"><span class="linenos"> 62</span></a> <span class="k">return</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"releases"</span><span class="p">,</span> <span class="p">[])</span>
+</span><span id="L-63"><a href="#L-63"><span class="linenos"> 63</span></a>
+</span><span id="L-64"><a href="#L-64"><span class="linenos"> 64</span></a>
+</span><span id="L-65"><a href="#L-65"><span class="linenos"> 65</span></a><span class="k">def</span><span class="w"> </span><span class="nf">download_release_file</span><span class="p">(</span>
+</span><span id="L-66"><a href="#L-66"><span class="linenos"> 66</span></a> <span class="n">item_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">release_ordinal</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">release</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">file_json_prefix</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="L-67"><a href="#L-67"><span class="linenos"> 67</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Path</span><span class="p">:</span>
+</span><span id="L-68"><a href="#L-68"><span class="linenos"> 68</span></a><span class="w"> </span><span class="sd">"""Download specified file type for a given release of an item."""</span>
+</span><span id="L-69"><a href="#L-69"><span class="linenos"> 69</span></a>
+</span><span id="L-70"><a href="#L-70"><span class="linenos"> 70</span></a> <span class="c1"># check folder is a directory</span>
+</span><span id="L-71"><a href="#L-71"><span class="linenos"> 71</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">DOWNLOADS_PATH</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="L-72"><a href="#L-72"><span class="linenos"> 72</span></a> <span class="k">raise</span> <span class="ne">NotADirectoryError</span><span class="p">(</span>
+</span><span id="L-73"><a href="#L-73"><span class="linenos"> 73</span></a> <span class="sa">f</span><span class="s2">"Error: '</span><span class="si">{</span><span class="n">DOWNLOADS_PATH</span><span class="si">}</span><span class="s2">' for TRUD resources is not a directory"</span>
+</span><span id="L-74"><a href="#L-74"><span class="linenos"> 74</span></a> <span class="p">)</span>
+</span><span id="L-75"><a href="#L-75"><span class="linenos"> 75</span></a>
+</span><span id="L-76"><a href="#L-76"><span class="linenos"> 76</span></a> <span class="n">file_type</span> <span class="o">=</span> <span class="n">file_json_prefix</span>
+</span><span id="L-77"><a href="#L-77"><span class="linenos"> 77</span></a> <span class="n">file_url</span> <span class="o">=</span> <span class="n">release</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileUrl"</span><span class="p">)</span>
+</span><span id="L-78"><a href="#L-78"><span class="linenos"> 78</span></a> <span class="k">if</span> <span class="n">file_url</span> <span class="o">==</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="L-79"><a href="#L-79"><span class="linenos"> 79</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"File url not in json data </span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileUrl"</span><span class="p">)</span>
+</span><span id="L-80"><a href="#L-80"><span class="linenos"> 80</span></a>
+</span><span id="L-81"><a href="#L-81"><span class="linenos"> 81</span></a> <span class="n">file_name</span> <span class="o">=</span> <span class="n">release</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileName"</span><span class="p">)</span>
+</span><span id="L-82"><a href="#L-82"><span class="linenos"> 82</span></a> <span class="k">if</span> <span class="n">file_name</span> <span class="o">==</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="L-83"><a href="#L-83"><span class="linenos"> 83</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"File name not in json data </span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileName"</span><span class="p">)</span>
+</span><span id="L-84"><a href="#L-84"><span class="linenos"> 84</span></a>
+</span><span id="L-85"><a href="#L-85"><span class="linenos"> 85</span></a> <span class="n">file_destination</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="n">file_name</span>
+</span><span id="L-86"><a href="#L-86"><span class="linenos"> 86</span></a>
+</span><span id="L-87"><a href="#L-87"><span class="linenos"> 87</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">file_url</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">file_name</span><span class="p">:</span>
+</span><span id="L-88"><a href="#L-88"><span class="linenos"> 88</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-89"><a href="#L-89"><span class="linenos"> 89</span></a> <span class="sa">f</span><span class="s2">"Missing </span><span class="si">{</span><span class="n">file_type</span><span class="si">}</span><span class="s2"> file information for release </span><span class="si">{</span><span class="n">release_ordinal</span><span class="si">}</span><span class="s2"> of item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">."</span>
+</span><span id="L-90"><a href="#L-90"><span class="linenos"> 90</span></a> <span class="p">)</span>
+</span><span id="L-91"><a href="#L-91"><span class="linenos"> 91</span></a>
+</span><span id="L-92"><a href="#L-92"><span class="linenos"> 92</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
+</span><span id="L-93"><a href="#L-93"><span class="linenos"> 93</span></a> <span class="sa">f</span><span class="s2">"Downloading item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">file_type</span><span class="si">}</span><span class="s2"> file: </span><span class="si">{</span><span class="n">file_name</span><span class="si">}</span><span class="s2"> from </span><span class="si">{</span><span class="n">file_url</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">file_destination</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-94"><a href="#L-94"><span class="linenos"> 94</span></a> <span class="p">)</span>
+</span><span id="L-95"><a href="#L-95"><span class="linenos"> 95</span></a> <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">file_url</span><span class="p">,</span> <span class="n">stream</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-96"><a href="#L-96"><span class="linenos"> 96</span></a>
+</span><span id="L-97"><a href="#L-97"><span class="linenos"> 97</span></a> <span class="k">if</span> <span class="n">response</span><span class="o">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span><span class="p">:</span>
+</span><span id="L-98"><a href="#L-98"><span class="linenos"> 98</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
+</span><span id="L-99"><a href="#L-99"><span class="linenos"> 99</span></a> <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">content</span><span class="p">)</span>
+</span><span id="L-100"><a href="#L-100"><span class="linenos">100</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-101"><a href="#L-101"><span class="linenos">101</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
+</span><span id="L-102"><a href="#L-102"><span class="linenos">102</span></a> <span class="sa">f</span><span class="s2">"Failed to download </span><span class="si">{</span><span class="n">file_type</span><span class="si">}</span><span class="s2"> file for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">. Status code: </span><span class="si">{</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="L-103"><a href="#L-103"><span class="linenos">103</span></a> <span class="p">)</span>
+</span><span id="L-104"><a href="#L-104"><span class="linenos">104</span></a> <span class="n">response</span><span class="o">.</span><span class="n">raise_for_status</span><span class="p">()</span>
+</span><span id="L-105"><a href="#L-105"><span class="linenos">105</span></a>
+</span><span id="L-106"><a href="#L-106"><span class="linenos">106</span></a> <span class="k">return</span> <span class="n">file_destination</span>
+</span><span id="L-107"><a href="#L-107"><span class="linenos">107</span></a>
+</span><span id="L-108"><a href="#L-108"><span class="linenos">108</span></a>
+</span><span id="L-109"><a href="#L-109"><span class="linenos">109</span></a><span class="k">def</span><span class="w"> </span><span class="nf">validate_download_hash</span><span class="p">(</span><span class="n">file_destination</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">item_hash</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-110"><a href="#L-110"><span class="linenos">110</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
+</span><span id="L-111"><a href="#L-111"><span class="linenos">111</span></a> <span class="nb">hash</span> <span class="o">=</span> <span class="n">hashlib</span><span class="o">.</span><span class="n">sha256</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">())</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">()</span>
+</span><span id="L-112"><a href="#L-112"><span class="linenos">112</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="nb">hash</span><span class="p">)</span>
+</span><span id="L-113"><a href="#L-113"><span class="linenos">113</span></a> <span class="k">if</span> <span class="nb">hash</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="o">==</span> <span class="n">item_hash</span><span class="o">.</span><span class="n">upper</span><span class="p">():</span>
+</span><span id="L-114"><a href="#L-114"><span class="linenos">114</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Verified hash of </span><span class="si">{</span><span class="n">file_destination</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="nb">hash</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-115"><a href="#L-115"><span class="linenos">115</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-116"><a href="#L-116"><span class="linenos">116</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Could not validate origin of </span><span class="si">{</span><span class="n">file_destination</span><span class="si">}</span><span class="s2">. The SHA-256 hash should be: </span><span class="si">{</span><span class="n">item_hash</span><span class="si">}</span><span class="s2">, but got </span><span class="si">{</span><span class="nb">hash</span><span class="si">}</span><span class="s2"> instead"</span>
+</span><span id="L-117"><a href="#L-117"><span class="linenos">117</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-118"><a href="#L-118"><span class="linenos">118</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="L-119"><a href="#L-119"><span class="linenos">119</span></a>
+</span><span id="L-120"><a href="#L-120"><span class="linenos">120</span></a>
+</span><span id="L-121"><a href="#L-121"><span class="linenos">121</span></a><span class="k">def</span><span class="w"> </span><span class="nf">unzip_download</span><span class="p">(</span><span class="n">file_destination</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="L-122"><a href="#L-122"><span class="linenos">122</span></a> <span class="c1"># check folder is a directory</span>
+</span><span id="L-123"><a href="#L-123"><span class="linenos">123</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">DOWNLOADS_PATH</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="L-124"><a href="#L-124"><span class="linenos">124</span></a> <span class="k">raise</span> <span class="ne">NotADirectoryError</span><span class="p">(</span>
+</span><span id="L-125"><a href="#L-125"><span class="linenos">125</span></a> <span class="sa">f</span><span class="s2">"Error: '</span><span class="si">{</span><span class="n">DOWNLOADS_PATH</span><span class="si">}</span><span class="s2">' for TRUD resoruces is not a directory"</span>
+</span><span id="L-126"><a href="#L-126"><span class="linenos">126</span></a> <span class="p">)</span>
+</span><span id="L-127"><a href="#L-127"><span class="linenos">127</span></a>
+</span><span id="L-128"><a href="#L-128"><span class="linenos">128</span></a> <span class="k">with</span> <span class="n">zipfile</span><span class="o">.</span><span class="n">ZipFile</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">zip_ref</span><span class="p">:</span>
+</span><span id="L-129"><a href="#L-129"><span class="linenos">129</span></a> <span class="n">zip_ref</span><span class="o">.</span><span class="n">extractall</span><span class="p">(</span><span class="n">DOWNLOADS_PATH</span><span class="p">)</span>
+</span><span id="L-130"><a href="#L-130"><span class="linenos">130</span></a>
+</span><span id="L-131"><a href="#L-131"><span class="linenos">131</span></a>
+</span><span id="L-132"><a href="#L-132"><span class="linenos">132</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_icd10</span><span class="p">():</span>
+</span><span id="L-133"><a href="#L-133"><span class="linenos">133</span></a> <span class="c1"># ICD10_edition5</span>
+</span><span id="L-134"><a href="#L-134"><span class="linenos">134</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-135"><a href="#L-135"><span class="linenos">135</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-136"><a href="#L-136"><span class="linenos">136</span></a> <span class="o">/</span> <span class="s2">"ICD10_Edition5_XML_20160401"</span>
+</span><span id="L-137"><a href="#L-137"><span class="linenos">137</span></a> <span class="o">/</span> <span class="s2">"Content"</span>
+</span><span id="L-138"><a href="#L-138"><span class="linenos">138</span></a> <span class="o">/</span> <span class="s2">"ICD10_Edition5_CodesAndTitlesAndMetadata_GB_20160401.xml"</span>
+</span><span id="L-139"><a href="#L-139"><span class="linenos">139</span></a> <span class="p">)</span>
+</span><span id="L-140"><a href="#L-140"><span class="linenos">140</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_xml</span><span class="p">(</span><span class="n">file_path</span><span class="p">)</span>
+</span><span id="L-141"><a href="#L-141"><span class="linenos">141</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"CODE"</span><span class="p">,</span> <span class="s2">"ALT_CODE"</span><span class="p">,</span> <span class="s2">"DESCRIPTION"</span><span class="p">]]</span>
+</span><span id="L-142"><a href="#L-142"><span class="linenos">142</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span>
+</span><span id="L-143"><a href="#L-143"><span class="linenos">143</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"CODE"</span><span class="p">:</span> <span class="s2">"icd10"</span><span class="p">,</span> <span class="s2">"ALT_CODE"</span><span class="p">:</span> <span class="s2">"icd10_alt"</span><span class="p">,</span> <span class="s2">"DESCRIPTION"</span><span class="p">:</span> <span class="s2">"description"</span><span class="p">}</span>
+</span><span id="L-144"><a href="#L-144"><span class="linenos">144</span></a> <span class="p">)</span>
+</span><span id="L-145"><a href="#L-145"><span class="linenos">145</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"icd10.parquet"</span>
+</span><span id="L-146"><a href="#L-146"><span class="linenos">146</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-147"><a href="#L-147"><span class="linenos">147</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-148"><a href="#L-148"><span class="linenos">148</span></a>
+</span><span id="L-149"><a href="#L-149"><span class="linenos">149</span></a>
+</span><span id="L-150"><a href="#L-150"><span class="linenos">150</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_opsc4</span><span class="p">():</span>
+</span><span id="L-151"><a href="#L-151"><span class="linenos">151</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-152"><a href="#L-152"><span class="linenos">152</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-153"><a href="#L-153"><span class="linenos">153</span></a> <span class="o">/</span> <span class="s2">"OPCS410 Data files txt"</span>
+</span><span id="L-154"><a href="#L-154"><span class="linenos">154</span></a> <span class="o">/</span> <span class="s2">"OPCS410 CodesAndTitles Nov 2022 V1.0.txt"</span>
+</span><span id="L-155"><a href="#L-155"><span class="linenos">155</span></a> <span class="p">)</span>
+</span><span id="L-156"><a href="#L-156"><span class="linenos">156</span></a>
+</span><span id="L-157"><a href="#L-157"><span class="linenos">157</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span> <span class="n">header</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="L-158"><a href="#L-158"><span class="linenos">158</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="mi">0</span><span class="p">:</span> <span class="s2">"opcs4"</span><span class="p">,</span> <span class="mi">1</span><span class="p">:</span> <span class="s2">"description"</span><span class="p">})</span>
+</span><span id="L-159"><a href="#L-159"><span class="linenos">159</span></a>
+</span><span id="L-160"><a href="#L-160"><span class="linenos">160</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"opcs4.parquet"</span>
+</span><span id="L-161"><a href="#L-161"><span class="linenos">161</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-162"><a href="#L-162"><span class="linenos">162</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-163"><a href="#L-163"><span class="linenos">163</span></a>
+</span><span id="L-164"><a href="#L-164"><span class="linenos">164</span></a>
+</span><span id="L-165"><a href="#L-165"><span class="linenos">165</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_nhs_data_migrations</span><span class="p">():</span>
+</span><span id="L-166"><a href="#L-166"><span class="linenos">166</span></a> <span class="c1"># NHS Data Migrations</span>
+</span><span id="L-167"><a href="#L-167"><span class="linenos">167</span></a>
+</span><span id="L-168"><a href="#L-168"><span class="linenos">168</span></a> <span class="c1"># snomed only</span>
+</span><span id="L-169"><a href="#L-169"><span class="linenos">169</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-170"><a href="#L-170"><span class="linenos">170</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-171"><a href="#L-171"><span class="linenos">171</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="L-172"><a href="#L-172"><span class="linenos">172</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="L-173"><a href="#L-173"><span class="linenos">173</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="L-174"><a href="#L-174"><span class="linenos">174</span></a> <span class="o">/</span> <span class="s2">"sctcremap_uk_20200401000001.txt"</span>
+</span><span id="L-175"><a href="#L-175"><span class="linenos">175</span></a> <span class="p">)</span>
+</span><span id="L-176"><a href="#L-176"><span class="linenos">176</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-177"><a href="#L-177"><span class="linenos">177</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"SCT_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="L-178"><a href="#L-178"><span class="linenos">178</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"SCT_CONCEPTID"</span><span class="p">:</span> <span class="s2">"snomed"</span><span class="p">})</span>
+</span><span id="L-179"><a href="#L-179"><span class="linenos">179</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">()</span>
+</span><span id="L-180"><a href="#L-180"><span class="linenos">180</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-181"><a href="#L-181"><span class="linenos">181</span></a>
+</span><span id="L-182"><a href="#L-182"><span class="linenos">182</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"snomed.parquet"</span>
+</span><span id="L-183"><a href="#L-183"><span class="linenos">183</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-184"><a href="#L-184"><span class="linenos">184</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-185"><a href="#L-185"><span class="linenos">185</span></a>
+</span><span id="L-186"><a href="#L-186"><span class="linenos">186</span></a> <span class="c1"># r2 -> r3</span>
+</span><span id="L-187"><a href="#L-187"><span class="linenos">187</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-188"><a href="#L-188"><span class="linenos">188</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-189"><a href="#L-189"><span class="linenos">189</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="L-190"><a href="#L-190"><span class="linenos">190</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="L-191"><a href="#L-191"><span class="linenos">191</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="L-192"><a href="#L-192"><span class="linenos">192</span></a> <span class="o">/</span> <span class="s2">"rctctv3map_uk_20200401000001.txt"</span>
+</span><span id="L-193"><a href="#L-193"><span class="linenos">193</span></a> <span class="p">)</span>
+</span><span id="L-194"><a href="#L-194"><span class="linenos">194</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-195"><a href="#L-195"><span class="linenos">195</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"V2_CONCEPTID"</span><span class="p">,</span> <span class="s2">"CTV3_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="L-196"><a href="#L-196"><span class="linenos">196</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"V2_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"CTV3_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">})</span>
+</span><span id="L-197"><a href="#L-197"><span class="linenos">197</span></a>
+</span><span id="L-198"><a href="#L-198"><span class="linenos">198</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_read3.parquet"</span>
+</span><span id="L-199"><a href="#L-199"><span class="linenos">199</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-200"><a href="#L-200"><span class="linenos">200</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-201"><a href="#L-201"><span class="linenos">201</span></a>
+</span><span id="L-202"><a href="#L-202"><span class="linenos">202</span></a> <span class="c1"># r3->r2</span>
+</span><span id="L-203"><a href="#L-203"><span class="linenos">203</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-204"><a href="#L-204"><span class="linenos">204</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-205"><a href="#L-205"><span class="linenos">205</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="L-206"><a href="#L-206"><span class="linenos">206</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="L-207"><a href="#L-207"><span class="linenos">207</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="L-208"><a href="#L-208"><span class="linenos">208</span></a> <span class="o">/</span> <span class="s2">"ctv3rctmap_uk_20200401000002.txt"</span>
+</span><span id="L-209"><a href="#L-209"><span class="linenos">209</span></a> <span class="p">)</span>
+</span><span id="L-210"><a href="#L-210"><span class="linenos">210</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-211"><a href="#L-211"><span class="linenos">211</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"CTV3_CONCEPTID"</span><span class="p">,</span> <span class="s2">"V2_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="L-212"><a href="#L-212"><span class="linenos">212</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"CTV3_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"V2_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">})</span>
+</span><span id="L-213"><a href="#L-213"><span class="linenos">213</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">()</span>
+</span><span id="L-214"><a href="#L-214"><span class="linenos">214</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read2"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*_.*$"</span><span class="p">)]</span> <span class="c1"># remove r2 codes with '_'</span>
+</span><span id="L-215"><a href="#L-215"><span class="linenos">215</span></a>
+</span><span id="L-216"><a href="#L-216"><span class="linenos">216</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_read2.parquet"</span>
+</span><span id="L-217"><a href="#L-217"><span class="linenos">217</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-218"><a href="#L-218"><span class="linenos">218</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-219"><a href="#L-219"><span class="linenos">219</span></a>
+</span><span id="L-220"><a href="#L-220"><span class="linenos">220</span></a> <span class="c1"># r2 -> snomed</span>
+</span><span id="L-221"><a href="#L-221"><span class="linenos">221</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-222"><a href="#L-222"><span class="linenos">222</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-223"><a href="#L-223"><span class="linenos">223</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="L-224"><a href="#L-224"><span class="linenos">224</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="L-225"><a href="#L-225"><span class="linenos">225</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="L-226"><a href="#L-226"><span class="linenos">226</span></a> <span class="o">/</span> <span class="s2">"rcsctmap2_uk_20200401000001.txt"</span>
+</span><span id="L-227"><a href="#L-227"><span class="linenos">227</span></a> <span class="p">)</span>
+</span><span id="L-228"><a href="#L-228"><span class="linenos">228</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-229"><a href="#L-229"><span class="linenos">229</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"ReadCode"</span><span class="p">,</span> <span class="s2">"ConceptId"</span><span class="p">]]</span>
+</span><span id="L-230"><a href="#L-230"><span class="linenos">230</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"ReadCode"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"ConceptId"</span><span class="p">:</span> <span class="s2">"snomed"</span><span class="p">})</span>
+</span><span id="L-231"><a href="#L-231"><span class="linenos">231</span></a>
+</span><span id="L-232"><a href="#L-232"><span class="linenos">232</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_snomed.parquet"</span>
+</span><span id="L-233"><a href="#L-233"><span class="linenos">233</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-234"><a href="#L-234"><span class="linenos">234</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-235"><a href="#L-235"><span class="linenos">235</span></a>
+</span><span id="L-236"><a href="#L-236"><span class="linenos">236</span></a> <span class="c1"># r3->snomed</span>
+</span><span id="L-237"><a href="#L-237"><span class="linenos">237</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-238"><a href="#L-238"><span class="linenos">238</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="L-239"><a href="#L-239"><span class="linenos">239</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="L-240"><a href="#L-240"><span class="linenos">240</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="L-241"><a href="#L-241"><span class="linenos">241</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="L-242"><a href="#L-242"><span class="linenos">242</span></a> <span class="o">/</span> <span class="s2">"ctv3sctmap2_uk_20200401000001.txt"</span>
+</span><span id="L-243"><a href="#L-243"><span class="linenos">243</span></a> <span class="p">)</span>
+</span><span id="L-244"><a href="#L-244"><span class="linenos">244</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-245"><a href="#L-245"><span class="linenos">245</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"CTV3_TERMID"</span><span class="p">,</span> <span class="s2">"SCT_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="L-246"><a href="#L-246"><span class="linenos">246</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"CTV3_TERMID"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"SCT_CONCEPTID"</span><span class="p">:</span> <span class="s2">"snomed"</span><span class="p">})</span>
+</span><span id="L-247"><a href="#L-247"><span class="linenos">247</span></a> <span class="n">df</span><span class="p">[</span><span class="s2">"snomed"</span><span class="p">]</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s2">"snomed"</span><span class="p">]</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="L-248"><a href="#L-248"><span class="linenos">248</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"snomed"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*_.*$"</span><span class="p">)]</span> <span class="c1"># remove snomed codes with '_'</span>
+</span><span id="L-249"><a href="#L-249"><span class="linenos">249</span></a>
+</span><span id="L-250"><a href="#L-250"><span class="linenos">250</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_snomed.parquet"</span>
+</span><span id="L-251"><a href="#L-251"><span class="linenos">251</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-252"><a href="#L-252"><span class="linenos">252</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-253"><a href="#L-253"><span class="linenos">253</span></a>
+</span><span id="L-254"><a href="#L-254"><span class="linenos">254</span></a>
+</span><span id="L-255"><a href="#L-255"><span class="linenos">255</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_nhs_read_browser</span><span class="p">():</span>
+</span><span id="L-256"><a href="#L-256"><span class="linenos">256</span></a> <span class="c1"># r2 only</span>
+</span><span id="L-257"><a href="#L-257"><span class="linenos">257</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"ANCESTOR.DBF"</span>
+</span><span id="L-258"><a href="#L-258"><span class="linenos">258</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-259"><a href="#L-259"><span class="linenos">259</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">df</span><span class="p">[</span><span class="s2">"READCODE"</span><span class="p">],</span> <span class="n">df</span><span class="p">[</span><span class="s2">"DESCENDANT"</span><span class="p">]])</span>
+</span><span id="L-260"><a href="#L-260"><span class="linenos">260</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">())</span>
+</span><span id="L-261"><a href="#L-261"><span class="linenos">261</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="mi">0</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">})</span>
+</span><span id="L-262"><a href="#L-262"><span class="linenos">262</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2.parquet"</span>
+</span><span id="L-263"><a href="#L-263"><span class="linenos">263</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-264"><a href="#L-264"><span class="linenos">264</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-265"><a href="#L-265"><span class="linenos">265</span></a>
+</span><span id="L-266"><a href="#L-266"><span class="linenos">266</span></a> <span class="c1"># r2 -> atc</span>
+</span><span id="L-267"><a href="#L-267"><span class="linenos">267</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"ATC.DBF"</span>
+</span><span id="L-268"><a href="#L-268"><span class="linenos">268</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-269"><a href="#L-269"><span class="linenos">269</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READCODE"</span><span class="p">,</span> <span class="s2">"ATC"</span><span class="p">]]</span>
+</span><span id="L-270"><a href="#L-270"><span class="linenos">270</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READCODE"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"ATC"</span><span class="p">:</span> <span class="s2">"atc"</span><span class="p">})</span>
+</span><span id="L-271"><a href="#L-271"><span class="linenos">271</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_atc.parquet"</span>
+</span><span id="L-272"><a href="#L-272"><span class="linenos">272</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-273"><a href="#L-273"><span class="linenos">273</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-274"><a href="#L-274"><span class="linenos">274</span></a>
+</span><span id="L-275"><a href="#L-275"><span class="linenos">275</span></a> <span class="c1"># r2 -> icd10</span>
+</span><span id="L-276"><a href="#L-276"><span class="linenos">276</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"ICD10.DBF"</span>
+</span><span id="L-277"><a href="#L-277"><span class="linenos">277</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-278"><a href="#L-278"><span class="linenos">278</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="L-279"><a href="#L-279"><span class="linenos">279</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"icd10"</span><span class="p">})</span>
+</span><span id="L-280"><a href="#L-280"><span class="linenos">280</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"icd10"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-281"><a href="#L-281"><span class="linenos">281</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read2"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-282"><a href="#L-282"><span class="linenos">282</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_icd10.parquet"</span>
+</span><span id="L-283"><a href="#L-283"><span class="linenos">283</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-284"><a href="#L-284"><span class="linenos">284</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-285"><a href="#L-285"><span class="linenos">285</span></a>
+</span><span id="L-286"><a href="#L-286"><span class="linenos">286</span></a> <span class="c1"># r2 -> opcs4</span>
+</span><span id="L-287"><a href="#L-287"><span class="linenos">287</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"OPCS4V3.DBF"</span>
+</span><span id="L-288"><a href="#L-288"><span class="linenos">288</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-289"><a href="#L-289"><span class="linenos">289</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="L-290"><a href="#L-290"><span class="linenos">290</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"opcs4"</span><span class="p">})</span>
+</span><span id="L-291"><a href="#L-291"><span class="linenos">291</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"opcs4"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-292"><a href="#L-292"><span class="linenos">292</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read2"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-293"><a href="#L-293"><span class="linenos">293</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_opcs4.parquet"</span>
+</span><span id="L-294"><a href="#L-294"><span class="linenos">294</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-295"><a href="#L-295"><span class="linenos">295</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-296"><a href="#L-296"><span class="linenos">296</span></a>
+</span><span id="L-297"><a href="#L-297"><span class="linenos">297</span></a> <span class="c1"># r3 only</span>
+</span><span id="L-298"><a href="#L-298"><span class="linenos">298</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V3"</span> <span class="o">/</span> <span class="s2">"ANCESTOR.DBF"</span>
+</span><span id="L-299"><a href="#L-299"><span class="linenos">299</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-300"><a href="#L-300"><span class="linenos">300</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">df</span><span class="p">[</span><span class="s2">"READCODE"</span><span class="p">],</span> <span class="n">df</span><span class="p">[</span><span class="s2">"DESCENDANT"</span><span class="p">]])</span>
+</span><span id="L-301"><a href="#L-301"><span class="linenos">301</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">())</span>
+</span><span id="L-302"><a href="#L-302"><span class="linenos">302</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="mi">0</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">})</span>
+</span><span id="L-303"><a href="#L-303"><span class="linenos">303</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3.parquet"</span>
+</span><span id="L-304"><a href="#L-304"><span class="linenos">304</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-305"><a href="#L-305"><span class="linenos">305</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-306"><a href="#L-306"><span class="linenos">306</span></a>
+</span><span id="L-307"><a href="#L-307"><span class="linenos">307</span></a> <span class="c1"># r3 -> icd10</span>
+</span><span id="L-308"><a href="#L-308"><span class="linenos">308</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V3"</span> <span class="o">/</span> <span class="s2">"ICD10.DBF"</span>
+</span><span id="L-309"><a href="#L-309"><span class="linenos">309</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-310"><a href="#L-310"><span class="linenos">310</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="L-311"><a href="#L-311"><span class="linenos">311</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"icd10"</span><span class="p">})</span>
+</span><span id="L-312"><a href="#L-312"><span class="linenos">312</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"icd10"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-313"><a href="#L-313"><span class="linenos">313</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read3"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-314"><a href="#L-314"><span class="linenos">314</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_icd10.parquet"</span>
+</span><span id="L-315"><a href="#L-315"><span class="linenos">315</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-316"><a href="#L-316"><span class="linenos">316</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-317"><a href="#L-317"><span class="linenos">317</span></a>
+</span><span id="L-318"><a href="#L-318"><span class="linenos">318</span></a> <span class="c1"># r3 -> icd9</span>
+</span><span id="L-319"><a href="#L-319"><span class="linenos">319</span></a> <span class="c1"># dbf = simpledbf.Dbf5('build/maps/downloads/Standard/V3/ICD9V3.DBF')</span>
+</span><span id="L-320"><a href="#L-320"><span class="linenos">320</span></a>
+</span><span id="L-321"><a href="#L-321"><span class="linenos">321</span></a> <span class="c1"># r3 -> opcs4</span>
+</span><span id="L-322"><a href="#L-322"><span class="linenos">322</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V3"</span> <span class="o">/</span> <span class="s2">"OPCS4V3.DBF"</span>
+</span><span id="L-323"><a href="#L-323"><span class="linenos">323</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="L-324"><a href="#L-324"><span class="linenos">324</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="L-325"><a href="#L-325"><span class="linenos">325</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"opcs4"</span><span class="p">})</span>
+</span><span id="L-326"><a href="#L-326"><span class="linenos">326</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"opcs4"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-327"><a href="#L-327"><span class="linenos">327</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read3"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="L-328"><a href="#L-328"><span class="linenos">328</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_opcs4.parquet"</span>
+</span><span id="L-329"><a href="#L-329"><span class="linenos">329</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="L-330"><a href="#L-330"><span class="linenos">330</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="L-331"><a href="#L-331"><span class="linenos">331</span></a>
+</span><span id="L-332"><a href="#L-332"><span class="linenos">332</span></a>
+</span><span id="L-333"><a href="#L-333"><span class="linenos">333</span></a><span class="k">def</span><span class="w"> </span><span class="nf">create_map_directories</span><span class="p">():</span>
+</span><span id="L-334"><a href="#L-334"><span class="linenos">334</span></a><span class="w"> </span><span class="sd">"""Create map directories."""</span>
+</span><span id="L-335"><a href="#L-335"><span class="linenos">335</span></a>
+</span><span id="L-336"><a href="#L-336"><span class="linenos">336</span></a> <span class="c1"># Check if build directory exists</span>
+</span><span id="L-337"><a href="#L-337"><span class="linenos">337</span></a> <span class="n">create_map_dirs</span> <span class="o">=</span> <span class="kc">False</span>
+</span><span id="L-338"><a href="#L-338"><span class="linenos">338</span></a> <span class="k">if</span> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="L-339"><a href="#L-339"><span class="linenos">339</span></a> <span class="n">user_input</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="L-340"><a href="#L-340"><span class="linenos">340</span></a> <span class="nb">input</span><span class="p">(</span>
+</span><span id="L-341"><a href="#L-341"><span class="linenos">341</span></a> <span class="sa">f</span><span class="s2">"The map directory </span><span class="si">{</span><span class="n">VOCAB_PATH</span><span class="si">}</span><span class="s2"> already exists. Do you want to download and process trud data again? (y/n): "</span>
+</span><span id="L-342"><a href="#L-342"><span class="linenos">342</span></a> <span class="p">)</span>
+</span><span id="L-343"><a href="#L-343"><span class="linenos">343</span></a> <span class="o">.</span><span class="n">strip</span><span class="p">()</span>
+</span><span id="L-344"><a href="#L-344"><span class="linenos">344</span></a> <span class="o">.</span><span class="n">lower</span><span class="p">()</span>
+</span><span id="L-345"><a href="#L-345"><span class="linenos">345</span></a> <span class="p">)</span>
+</span><span id="L-346"><a href="#L-346"><span class="linenos">346</span></a> <span class="k">if</span> <span class="n">user_input</span> <span class="o">==</span> <span class="s2">"y"</span><span class="p">:</span>
+</span><span id="L-347"><a href="#L-347"><span class="linenos">347</span></a> <span class="c1"># delete all build files</span>
+</span><span id="L-348"><a href="#L-348"><span class="linenos">348</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="p">)</span>
+</span><span id="L-349"><a href="#L-349"><span class="linenos">349</span></a> <span class="n">create_map_dirs</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-350"><a href="#L-350"><span class="linenos">350</span></a> <span class="k">elif</span> <span class="n">user_input</span> <span class="o">==</span> <span class="s2">"n"</span><span class="p">:</span>
+</span><span id="L-351"><a href="#L-351"><span class="linenos">351</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Exiting TRUD installation"</span><span class="p">)</span>
+</span><span id="L-352"><a href="#L-352"><span class="linenos">352</span></a> <span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+</span><span id="L-353"><a href="#L-353"><span class="linenos">353</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="L-354"><a href="#L-354"><span class="linenos">354</span></a> <span class="n">create_map_dirs</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-355"><a href="#L-355"><span class="linenos">355</span></a>
+</span><span id="L-356"><a href="#L-356"><span class="linenos">356</span></a> <span class="k">if</span> <span class="n">create_map_dirs</span><span class="p">:</span>
+</span><span id="L-357"><a href="#L-357"><span class="linenos">357</span></a> <span class="c1"># create maps directories</span>
+</span><span id="L-358"><a href="#L-358"><span class="linenos">358</span></a> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-359"><a href="#L-359"><span class="linenos">359</span></a> <span class="n">DOWNLOADS_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-360"><a href="#L-360"><span class="linenos">360</span></a> <span class="n">PROCESSED_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="L-361"><a href="#L-361"><span class="linenos">361</span></a>
+</span><span id="L-362"><a href="#L-362"><span class="linenos">362</span></a>
+</span><span id="L-363"><a href="#L-363"><span class="linenos">363</span></a><span class="k">def</span><span class="w"> </span><span class="nf">install</span><span class="p">():</span>
+</span><span id="L-364"><a href="#L-364"><span class="linenos">364</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Installing TRUD"</span><span class="p">)</span>
+</span><span id="L-365"><a href="#L-365"><span class="linenos">365</span></a>
+</span><span id="L-366"><a href="#L-366"><span class="linenos">366</span></a> <span class="c1"># get TRUD api key from environment variable</span>
+</span><span id="L-367"><a href="#L-367"><span class="linenos">367</span></a> <span class="n">api_key</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"ACMC_TRUD_API_KEY"</span><span class="p">)</span>
+</span><span id="L-368"><a href="#L-368"><span class="linenos">368</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">api_key</span><span class="p">:</span>
+</span><span id="L-369"><a href="#L-369"><span class="linenos">369</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="L-370"><a href="#L-370"><span class="linenos">370</span></a> <span class="s2">"TRUD API KEY not found. Set the ACMC_TRUD_API_KEY environment variable."</span>
+</span><span id="L-371"><a href="#L-371"><span class="linenos">371</span></a> <span class="p">)</span>
+</span><span id="L-372"><a href="#L-372"><span class="linenos">372</span></a>
+</span><span id="L-373"><a href="#L-373"><span class="linenos">373</span></a> <span class="n">create_map_directories</span><span class="p">()</span>
+</span><span id="L-374"><a href="#L-374"><span class="linenos">374</span></a>
+</span><span id="L-375"><a href="#L-375"><span class="linenos">375</span></a> <span class="n">items_latest</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="L-376"><a href="#L-376"><span class="linenos">376</span></a> <span class="n">items</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="L-377"><a href="#L-377"><span class="linenos">377</span></a> <span class="p">{</span>
+</span><span id="L-378"><a href="#L-378"><span class="linenos">378</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">259</span><span class="p">,</span>
+</span><span id="L-379"><a href="#L-379"><span class="linenos">379</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS ICD-10 5th Edition XML data files"</span><span class="p">,</span>
+</span><span id="L-380"><a href="#L-380"><span class="linenos">380</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"A4F7BBA6E86349AADD0F4696C5E91152EB99CC06121427FC359160439B9F883F"</span><span class="p">,</span>
+</span><span id="L-381"><a href="#L-381"><span class="linenos">381</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_icd10</span><span class="p">,</span>
+</span><span id="L-382"><a href="#L-382"><span class="linenos">382</span></a> <span class="p">},</span>
+</span><span id="L-383"><a href="#L-383"><span class="linenos">383</span></a> <span class="p">{</span>
+</span><span id="L-384"><a href="#L-384"><span class="linenos">384</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">119</span><span class="p">,</span>
+</span><span id="L-385"><a href="#L-385"><span class="linenos">385</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"OPCS-4 data files"</span><span class="p">,</span>
+</span><span id="L-386"><a href="#L-386"><span class="linenos">386</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"0615A2BF43FFEF94517F1D1E0C05493B627839F323F22C52CBCD8B40BF767CD3"</span><span class="p">,</span>
+</span><span id="L-387"><a href="#L-387"><span class="linenos">387</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_opsc4</span><span class="p">,</span>
+</span><span id="L-388"><a href="#L-388"><span class="linenos">388</span></a> <span class="p">},</span>
+</span><span id="L-389"><a href="#L-389"><span class="linenos">389</span></a> <span class="p">{</span>
+</span><span id="L-390"><a href="#L-390"><span class="linenos">390</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">9</span><span class="p">,</span>
+</span><span id="L-391"><a href="#L-391"><span class="linenos">391</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS Data Migration"</span><span class="p">,</span>
+</span><span id="L-392"><a href="#L-392"><span class="linenos">392</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"D4317B3ADBA6E1247CF17F0B7CD2B8850FD36C0EA2923BF684EA6159F3A54765"</span><span class="p">,</span>
+</span><span id="L-393"><a href="#L-393"><span class="linenos">393</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_nhs_data_migrations</span><span class="p">,</span>
+</span><span id="L-394"><a href="#L-394"><span class="linenos">394</span></a> <span class="p">},</span>
+</span><span id="L-395"><a href="#L-395"><span class="linenos">395</span></a> <span class="p">{</span>
+</span><span id="L-396"><a href="#L-396"><span class="linenos">396</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">8</span><span class="p">,</span>
+</span><span id="L-397"><a href="#L-397"><span class="linenos">397</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS Read Browser"</span><span class="p">,</span>
+</span><span id="L-398"><a href="#L-398"><span class="linenos">398</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"1FFF2CBF11D0E6D7FC6CC6F13DD52D2F459095C3D83A3F754E6C359F16913C5E"</span><span class="p">,</span>
+</span><span id="L-399"><a href="#L-399"><span class="linenos">399</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_nhs_read_browser</span><span class="p">,</span>
+</span><span id="L-400"><a href="#L-400"><span class="linenos">400</span></a> <span class="p">},</span>
+</span><span id="L-401"><a href="#L-401"><span class="linenos">401</span></a> <span class="c1"># TODO: Download BNF from separate site? https://www.nhsbsa.nhs.uk/sites/default/files/2024-10/BNF%20Snomed%20Mapping%20data%2020241016.zip</span>
+</span><span id="L-402"><a href="#L-402"><span class="linenos">402</span></a> <span class="p">]</span>
+</span><span id="L-403"><a href="#L-403"><span class="linenos">403</span></a>
+</span><span id="L-404"><a href="#L-404"><span class="linenos">404</span></a> <span class="c1"># remove function from items to save versions</span>
+</span><span id="L-405"><a href="#L-405"><span class="linenos">405</span></a> <span class="n">data</span> <span class="o">=</span> <span class="p">[{</span><span class="n">k</span><span class="p">:</span> <span class="n">v</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">k</span> <span class="o">!=</span> <span class="s2">"extract"</span><span class="p">}</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">items</span><span class="p">]</span>
+</span><span id="L-406"><a href="#L-406"><span class="linenos">406</span></a> <span class="c1"># save TRUD versions to file to main record of what was downloaded</span>
+</span><span id="L-407"><a href="#L-407"><span class="linenos">407</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">VERSION_PATH</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="L-408"><a href="#L-408"><span class="linenos">408</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="L-409"><a href="#L-409"><span class="linenos">409</span></a> <span class="n">data</span><span class="p">,</span>
+</span><span id="L-410"><a href="#L-410"><span class="linenos">410</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="L-411"><a href="#L-411"><span class="linenos">411</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="L-412"><a href="#L-412"><span class="linenos">412</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-413"><a href="#L-413"><span class="linenos">413</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="L-414"><a href="#L-414"><span class="linenos">414</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="L-415"><a href="#L-415"><span class="linenos">415</span></a> <span class="p">)</span>
+</span><span id="L-416"><a href="#L-416"><span class="linenos">416</span></a>
+</span><span id="L-417"><a href="#L-417"><span class="linenos">417</span></a> <span class="c1"># Validate and process each item ID</span>
+</span><span id="L-418"><a href="#L-418"><span class="linenos">418</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">items</span><span class="p">:</span>
+</span><span id="L-419"><a href="#L-419"><span class="linenos">419</span></a> <span class="n">item_id</span> <span class="o">=</span> <span class="n">item</span><span class="p">[</span><span class="s2">"id"</span><span class="p">]</span>
+</span><span id="L-420"><a href="#L-420"><span class="linenos">420</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"--- </span><span class="si">{</span><span class="n">item</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span><span class="si">}</span><span class="s2"> ---"</span><span class="p">)</span>
+</span><span id="L-421"><a href="#L-421"><span class="linenos">421</span></a>
+</span><span id="L-422"><a href="#L-422"><span class="linenos">422</span></a> <span class="n">releases</span> <span class="o">=</span> <span class="n">get_releases</span><span class="p">(</span><span class="n">item_id</span><span class="p">,</span> <span class="n">API_KEY</span><span class="o">=</span><span class="n">api_key</span><span class="p">,</span> <span class="n">latest</span><span class="o">=</span><span class="n">items_latest</span><span class="p">)</span>
+</span><span id="L-423"><a href="#L-423"><span class="linenos">423</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">releases</span><span class="p">:</span>
+</span><span id="L-424"><a href="#L-424"><span class="linenos">424</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No releases found for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
+</span><span id="L-425"><a href="#L-425"><span class="linenos">425</span></a>
+</span><span id="L-426"><a href="#L-426"><span class="linenos">426</span></a> <span class="c1"># Process each release in reverse order</span>
+</span><span id="L-427"><a href="#L-427"><span class="linenos">427</span></a> <span class="k">for</span> <span class="n">release_ordinal</span><span class="p">,</span> <span class="n">release</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">releases</span><span class="p">[::</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="mi">1</span><span class="p">):</span>
+</span><span id="L-428"><a href="#L-428"><span class="linenos">428</span></a> <span class="c1"># Download archive file</span>
+</span><span id="L-429"><a href="#L-429"><span class="linenos">429</span></a> <span class="n">file_destination</span> <span class="o">=</span> <span class="n">download_release_file</span><span class="p">(</span>
+</span><span id="L-430"><a href="#L-430"><span class="linenos">430</span></a> <span class="n">item_id</span><span class="p">,</span> <span class="n">release_ordinal</span><span class="p">,</span> <span class="n">release</span><span class="p">,</span> <span class="s2">"archive"</span>
+</span><span id="L-431"><a href="#L-431"><span class="linenos">431</span></a> <span class="p">)</span>
+</span><span id="L-432"><a href="#L-432"><span class="linenos">432</span></a>
+</span><span id="L-433"><a href="#L-433"><span class="linenos">433</span></a> <span class="c1"># Optional files</span>
+</span><span id="L-434"><a href="#L-434"><span class="linenos">434</span></a> <span class="c1"># if items.checksum:</span>
+</span><span id="L-435"><a href="#L-435"><span class="linenos">435</span></a> <span class="c1"># download_release_file(item["id"], release_ordinal, release, "checksum")</span>
+</span><span id="L-436"><a href="#L-436"><span class="linenos">436</span></a> <span class="c1"># if items.signature:</span>
+</span><span id="L-437"><a href="#L-437"><span class="linenos">437</span></a> <span class="c1"># download_release_file(item["id"], release_ordinal, release, "signature")</span>
+</span><span id="L-438"><a href="#L-438"><span class="linenos">438</span></a> <span class="c1"># if items.public_key:</span>
+</span><span id="L-439"><a href="#L-439"><span class="linenos">439</span></a> <span class="c1"># download_release_file(item["id"], release_ordinal, release, "publicKey", "public key")</span>
+</span><span id="L-440"><a href="#L-440"><span class="linenos">440</span></a>
+</span><span id="L-441"><a href="#L-441"><span class="linenos">441</span></a> <span class="c1"># Verify Hash if available</span>
+</span><span id="L-442"><a href="#L-442"><span class="linenos">442</span></a> <span class="k">if</span> <span class="s2">"hash"</span> <span class="ow">in</span> <span class="n">item</span><span class="p">:</span>
+</span><span id="L-443"><a href="#L-443"><span class="linenos">443</span></a> <span class="n">validate_download_hash</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="n">item</span><span class="p">[</span><span class="s2">"hash"</span><span class="p">])</span>
+</span><span id="L-444"><a href="#L-444"><span class="linenos">444</span></a>
+</span><span id="L-445"><a href="#L-445"><span class="linenos">445</span></a> <span class="c1"># Unzip downloaded .zip</span>
+</span><span id="L-446"><a href="#L-446"><span class="linenos">446</span></a> <span class="n">unzip_download</span><span class="p">(</span><span class="n">file_destination</span><span class="p">)</span>
+</span><span id="L-447"><a href="#L-447"><span class="linenos">447</span></a>
+</span><span id="L-448"><a href="#L-448"><span class="linenos">448</span></a> <span class="c1"># Extract Tables to parquet</span>
+</span><span id="L-449"><a href="#L-449"><span class="linenos">449</span></a> <span class="k">if</span> <span class="s2">"extract"</span> <span class="ow">in</span> <span class="n">item</span><span class="p">:</span>
+</span><span id="L-450"><a href="#L-450"><span class="linenos">450</span></a> <span class="n">item</span><span class="p">[</span><span class="s2">"extract"</span><span class="p">]()</span>
+</span><span id="L-451"><a href="#L-451"><span class="linenos">451</span></a>
+</span><span id="L-452"><a href="#L-452"><span class="linenos">452</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Downloaded </span><span class="si">{</span><span class="n">release_ordinal</span><span class="si">}</span><span class="s2"> release(s) for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
+</span><span id="L-453"><a href="#L-453"><span class="linenos">453</span></a>
+</span><span id="L-454"><a href="#L-454"><span class="linenos">454</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"TRUD installation completed"</span><span class="p">)</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="FQDN">
+ <div class="attr variable">
+ <span class="name">FQDN</span> =
+<span class="default_value">'isd.digital.nhs.uk'</span>
+
+
+ </div>
+ <a class="headerlink" href="#FQDN"></a>
+
+ <div class="docstring"><p>Fully Qualified Domain Name of NHS digital TRUD service API</p>
+</div>
+
+
+ </section>
+ <section id="VOCAB_PATH">
+ <div class="attr variable">
+ <span class="name">VOCAB_PATH</span> =
+<span class="default_value">PosixPath('vocab/trud')</span>
+
+
+ </div>
+ <a class="headerlink" href="#VOCAB_PATH"></a>
+
+ <div class="docstring"><p>Default path to the TRUD vocabulary directory relative to the the acmc execution directory</p>
+</div>
+
+
+ </section>
+ <section id="VERSION_FILE">
+ <div class="attr variable">
+ <span class="name">VERSION_FILE</span> =
+<span class="default_value">'trud_version.yml'</span>
+
+
+ </div>
+ <a class="headerlink" href="#VERSION_FILE"></a>
+
+ <div class="docstring"><p>TRUD version file</p>
+</div>
+
+
+ </section>
+ <section id="VERSION_PATH">
+ <div class="attr variable">
+ <span class="name">VERSION_PATH</span> =
+<span class="default_value">PosixPath('vocab/trud/trud_version.yml')</span>
+
+
+ </div>
+ <a class="headerlink" href="#VERSION_PATH"></a>
+
+ <div class="docstring"><p>Default path to the TRUD version file</p>
+</div>
+
+
+ </section>
+ <section id="DOWNLOADS_PATH">
+ <div class="attr variable">
+ <span class="name">DOWNLOADS_PATH</span> =
+<span class="default_value">PosixPath('vocab/trud/downloads')</span>
+
+
+ </div>
+ <a class="headerlink" href="#DOWNLOADS_PATH"></a>
+
+ <div class="docstring"><p>Default path to the TRUD vocabulary downloads directory</p>
+</div>
+
+
+ </section>
+ <section id="PROCESSED_PATH">
+ <div class="attr variable">
+ <span class="name">PROCESSED_PATH</span> =
+<span class="default_value">PosixPath('vocab/trud/processed')</span>
+
+
+ </div>
+ <a class="headerlink" href="#PROCESSED_PATH"></a>
+
+ <div class="docstring"><p>Default path to the processed TRUD mappings directory</p>
+</div>
+
+
+ </section>
+ <section id="get_releases">
+ <input id="get_releases-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">get_releases</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">item_id</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">API_KEY</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">latest</span><span class="o">=</span><span class="kc">False</span></span><span class="return-annotation">) -> <span class="nb">list</span>:</span></span>
+
+ <label class="view-source-button" for="get_releases-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#get_releases"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="get_releases-44"><a href="#get_releases-44"><span class="linenos">44</span></a><span class="k">def</span><span class="w"> </span><span class="nf">get_releases</span><span class="p">(</span><span class="n">item_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">API_KEY</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">latest</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="o">-></span> <span class="nb">list</span><span class="p">:</span>
+</span><span id="get_releases-45"><a href="#get_releases-45"><span class="linenos">45</span></a><span class="w"> </span><span class="sd">"""Retrieve release information for an item from the TRUD API."""</span>
+</span><span id="get_releases-46"><a href="#get_releases-46"><span class="linenos">46</span></a> <span class="n">url</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"https://</span><span class="si">{</span><span class="n">FQDN</span><span class="si">}</span><span class="s2">/trud/api/v1/keys/</span><span class="si">{</span><span class="n">API_KEY</span><span class="si">}</span><span class="s2">/items/</span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">/releases"</span>
+</span><span id="get_releases-47"><a href="#get_releases-47"><span class="linenos">47</span></a> <span class="k">if</span> <span class="n">latest</span><span class="p">:</span>
+</span><span id="get_releases-48"><a href="#get_releases-48"><span class="linenos">48</span></a> <span class="n">url</span> <span class="o">+=</span> <span class="s2">"?latest"</span>
+</span><span id="get_releases-49"><a href="#get_releases-49"><span class="linenos">49</span></a>
+</span><span id="get_releases-50"><a href="#get_releases-50"><span class="linenos">50</span></a> <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
+</span><span id="get_releases-51"><a href="#get_releases-51"><span class="linenos">51</span></a> <span class="k">if</span> <span class="n">response</span><span class="o">.</span><span class="n">status_code</span> <span class="o">!=</span> <span class="mi">200</span><span class="p">:</span>
+</span><span id="get_releases-52"><a href="#get_releases-52"><span class="linenos">52</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
+</span><span id="get_releases-53"><a href="#get_releases-53"><span class="linenos">53</span></a> <span class="sa">f</span><span class="s2">"Failed to fetch releases for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">. Status code: </span><span class="si">{</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="si">}</span><span class="s2">, error </span><span class="si">{</span><span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()[</span><span class="s1">'message'</span><span class="p">]</span><span class="si">}</span><span class="s2">. If no releases found for API key, please ensure you are subscribed to the data release and that it is not pending approval"</span>
+</span><span id="get_releases-54"><a href="#get_releases-54"><span class="linenos">54</span></a> <span class="p">)</span>
+</span><span id="get_releases-55"><a href="#get_releases-55"><span class="linenos">55</span></a> <span class="n">response</span><span class="o">.</span><span class="n">raise_for_status</span><span class="p">()</span>
+</span><span id="get_releases-56"><a href="#get_releases-56"><span class="linenos">56</span></a>
+</span><span id="get_releases-57"><a href="#get_releases-57"><span class="linenos">57</span></a> <span class="n">data</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">()</span>
+</span><span id="get_releases-58"><a href="#get_releases-58"><span class="linenos">58</span></a> <span class="k">if</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"message"</span><span class="p">)</span> <span class="o">!=</span> <span class="s2">"OK"</span><span class="p">:</span>
+</span><span id="get_releases-59"><a href="#get_releases-59"><span class="linenos">59</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Unknown error occurred </span><span class="si">{</span><span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">'message'</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="get_releases-60"><a href="#get_releases-60"><span class="linenos">60</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="get_releases-61"><a href="#get_releases-61"><span class="linenos">61</span></a> <span class="k">raise</span> <span class="ne">Exception</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="get_releases-62"><a href="#get_releases-62"><span class="linenos">62</span></a>
+</span><span id="get_releases-63"><a href="#get_releases-63"><span class="linenos">63</span></a> <span class="k">return</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"releases"</span><span class="p">,</span> <span class="p">[])</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Retrieve release information for an item from the TRUD API.</p>
+</div>
+
+
+ </section>
+ <section id="download_release_file">
+ <input id="download_release_file-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">download_release_file</span><span class="signature pdoc-code multiline">(<span class="param"> <span class="n">item_id</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">release_ordinal</span><span class="p">:</span> <span class="nb">str</span>,</span><span class="param"> <span class="n">release</span><span class="p">:</span> <span class="nb">dict</span>,</span><span class="param"> <span class="n">file_json_prefix</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">) -> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span>:</span></span>
+
+ <label class="view-source-button" for="download_release_file-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#download_release_file"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="download_release_file-66"><a href="#download_release_file-66"><span class="linenos"> 66</span></a><span class="k">def</span><span class="w"> </span><span class="nf">download_release_file</span><span class="p">(</span>
+</span><span id="download_release_file-67"><a href="#download_release_file-67"><span class="linenos"> 67</span></a> <span class="n">item_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">release_ordinal</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">release</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span> <span class="n">file_json_prefix</span><span class="p">:</span> <span class="nb">str</span>
+</span><span id="download_release_file-68"><a href="#download_release_file-68"><span class="linenos"> 68</span></a><span class="p">)</span> <span class="o">-></span> <span class="n">Path</span><span class="p">:</span>
+</span><span id="download_release_file-69"><a href="#download_release_file-69"><span class="linenos"> 69</span></a><span class="w"> </span><span class="sd">"""Download specified file type for a given release of an item."""</span>
+</span><span id="download_release_file-70"><a href="#download_release_file-70"><span class="linenos"> 70</span></a>
+</span><span id="download_release_file-71"><a href="#download_release_file-71"><span class="linenos"> 71</span></a> <span class="c1"># check folder is a directory</span>
+</span><span id="download_release_file-72"><a href="#download_release_file-72"><span class="linenos"> 72</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">DOWNLOADS_PATH</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="download_release_file-73"><a href="#download_release_file-73"><span class="linenos"> 73</span></a> <span class="k">raise</span> <span class="ne">NotADirectoryError</span><span class="p">(</span>
+</span><span id="download_release_file-74"><a href="#download_release_file-74"><span class="linenos"> 74</span></a> <span class="sa">f</span><span class="s2">"Error: '</span><span class="si">{</span><span class="n">DOWNLOADS_PATH</span><span class="si">}</span><span class="s2">' for TRUD resources is not a directory"</span>
+</span><span id="download_release_file-75"><a href="#download_release_file-75"><span class="linenos"> 75</span></a> <span class="p">)</span>
+</span><span id="download_release_file-76"><a href="#download_release_file-76"><span class="linenos"> 76</span></a>
+</span><span id="download_release_file-77"><a href="#download_release_file-77"><span class="linenos"> 77</span></a> <span class="n">file_type</span> <span class="o">=</span> <span class="n">file_json_prefix</span>
+</span><span id="download_release_file-78"><a href="#download_release_file-78"><span class="linenos"> 78</span></a> <span class="n">file_url</span> <span class="o">=</span> <span class="n">release</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileUrl"</span><span class="p">)</span>
+</span><span id="download_release_file-79"><a href="#download_release_file-79"><span class="linenos"> 79</span></a> <span class="k">if</span> <span class="n">file_url</span> <span class="o">==</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="download_release_file-80"><a href="#download_release_file-80"><span class="linenos"> 80</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"File url not in json data </span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileUrl"</span><span class="p">)</span>
+</span><span id="download_release_file-81"><a href="#download_release_file-81"><span class="linenos"> 81</span></a>
+</span><span id="download_release_file-82"><a href="#download_release_file-82"><span class="linenos"> 82</span></a> <span class="n">file_name</span> <span class="o">=</span> <span class="n">release</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="sa">f</span><span class="s2">"</span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileName"</span><span class="p">)</span>
+</span><span id="download_release_file-83"><a href="#download_release_file-83"><span class="linenos"> 83</span></a> <span class="k">if</span> <span class="n">file_name</span> <span class="o">==</span> <span class="kc">None</span><span class="p">:</span>
+</span><span id="download_release_file-84"><a href="#download_release_file-84"><span class="linenos"> 84</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"File name not in json data </span><span class="si">{</span><span class="n">file_json_prefix</span><span class="si">}</span><span class="s2">FileName"</span><span class="p">)</span>
+</span><span id="download_release_file-85"><a href="#download_release_file-85"><span class="linenos"> 85</span></a>
+</span><span id="download_release_file-86"><a href="#download_release_file-86"><span class="linenos"> 86</span></a> <span class="n">file_destination</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="n">file_name</span>
+</span><span id="download_release_file-87"><a href="#download_release_file-87"><span class="linenos"> 87</span></a>
+</span><span id="download_release_file-88"><a href="#download_release_file-88"><span class="linenos"> 88</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">file_url</span> <span class="ow">or</span> <span class="ow">not</span> <span class="n">file_name</span><span class="p">:</span>
+</span><span id="download_release_file-89"><a href="#download_release_file-89"><span class="linenos"> 89</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="download_release_file-90"><a href="#download_release_file-90"><span class="linenos"> 90</span></a> <span class="sa">f</span><span class="s2">"Missing </span><span class="si">{</span><span class="n">file_type</span><span class="si">}</span><span class="s2"> file information for release </span><span class="si">{</span><span class="n">release_ordinal</span><span class="si">}</span><span class="s2"> of item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">."</span>
+</span><span id="download_release_file-91"><a href="#download_release_file-91"><span class="linenos"> 91</span></a> <span class="p">)</span>
+</span><span id="download_release_file-92"><a href="#download_release_file-92"><span class="linenos"> 92</span></a>
+</span><span id="download_release_file-93"><a href="#download_release_file-93"><span class="linenos"> 93</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span>
+</span><span id="download_release_file-94"><a href="#download_release_file-94"><span class="linenos"> 94</span></a> <span class="sa">f</span><span class="s2">"Downloading item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">file_type</span><span class="si">}</span><span class="s2"> file: </span><span class="si">{</span><span class="n">file_name</span><span class="si">}</span><span class="s2"> from </span><span class="si">{</span><span class="n">file_url</span><span class="si">}</span><span class="s2"> to </span><span class="si">{</span><span class="n">file_destination</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="download_release_file-95"><a href="#download_release_file-95"><span class="linenos"> 95</span></a> <span class="p">)</span>
+</span><span id="download_release_file-96"><a href="#download_release_file-96"><span class="linenos"> 96</span></a> <span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">file_url</span><span class="p">,</span> <span class="n">stream</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="download_release_file-97"><a href="#download_release_file-97"><span class="linenos"> 97</span></a>
+</span><span id="download_release_file-98"><a href="#download_release_file-98"><span class="linenos"> 98</span></a> <span class="k">if</span> <span class="n">response</span><span class="o">.</span><span class="n">status_code</span> <span class="o">==</span> <span class="mi">200</span><span class="p">:</span>
+</span><span id="download_release_file-99"><a href="#download_release_file-99"><span class="linenos"> 99</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="s2">"wb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
+</span><span id="download_release_file-100"><a href="#download_release_file-100"><span class="linenos">100</span></a> <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">content</span><span class="p">)</span>
+</span><span id="download_release_file-101"><a href="#download_release_file-101"><span class="linenos">101</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="download_release_file-102"><a href="#download_release_file-102"><span class="linenos">102</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span>
+</span><span id="download_release_file-103"><a href="#download_release_file-103"><span class="linenos">103</span></a> <span class="sa">f</span><span class="s2">"Failed to download </span><span class="si">{</span><span class="n">file_type</span><span class="si">}</span><span class="s2"> file for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">. Status code: </span><span class="si">{</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="si">}</span><span class="s2">"</span>
+</span><span id="download_release_file-104"><a href="#download_release_file-104"><span class="linenos">104</span></a> <span class="p">)</span>
+</span><span id="download_release_file-105"><a href="#download_release_file-105"><span class="linenos">105</span></a> <span class="n">response</span><span class="o">.</span><span class="n">raise_for_status</span><span class="p">()</span>
+</span><span id="download_release_file-106"><a href="#download_release_file-106"><span class="linenos">106</span></a>
+</span><span id="download_release_file-107"><a href="#download_release_file-107"><span class="linenos">107</span></a> <span class="k">return</span> <span class="n">file_destination</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Download specified file type for a given release of an item.</p>
+</div>
+
+
+ </section>
+ <section id="validate_download_hash">
+ <input id="validate_download_hash-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">validate_download_hash</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">file_destination</span><span class="p">:</span> <span class="nb">str</span>, </span><span class="param"><span class="n">item_hash</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="validate_download_hash-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#validate_download_hash"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="validate_download_hash-110"><a href="#validate_download_hash-110"><span class="linenos">110</span></a><span class="k">def</span><span class="w"> </span><span class="nf">validate_download_hash</span><span class="p">(</span><span class="n">file_destination</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">item_hash</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="validate_download_hash-111"><a href="#validate_download_hash-111"><span class="linenos">111</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="s2">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
+</span><span id="validate_download_hash-112"><a href="#validate_download_hash-112"><span class="linenos">112</span></a> <span class="nb">hash</span> <span class="o">=</span> <span class="n">hashlib</span><span class="o">.</span><span class="n">sha256</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">())</span><span class="o">.</span><span class="n">hexdigest</span><span class="p">()</span>
+</span><span id="validate_download_hash-113"><a href="#validate_download_hash-113"><span class="linenos">113</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="nb">hash</span><span class="p">)</span>
+</span><span id="validate_download_hash-114"><a href="#validate_download_hash-114"><span class="linenos">114</span></a> <span class="k">if</span> <span class="nb">hash</span><span class="o">.</span><span class="n">upper</span><span class="p">()</span> <span class="o">==</span> <span class="n">item_hash</span><span class="o">.</span><span class="n">upper</span><span class="p">():</span>
+</span><span id="validate_download_hash-115"><a href="#validate_download_hash-115"><span class="linenos">115</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">debug</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Verified hash of </span><span class="si">{</span><span class="n">file_destination</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="nb">hash</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="validate_download_hash-116"><a href="#validate_download_hash-116"><span class="linenos">116</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="validate_download_hash-117"><a href="#validate_download_hash-117"><span class="linenos">117</span></a> <span class="n">msg</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">"Could not validate origin of </span><span class="si">{</span><span class="n">file_destination</span><span class="si">}</span><span class="s2">. The SHA-256 hash should be: </span><span class="si">{</span><span class="n">item_hash</span><span class="si">}</span><span class="s2">, but got </span><span class="si">{</span><span class="nb">hash</span><span class="si">}</span><span class="s2"> instead"</span>
+</span><span id="validate_download_hash-118"><a href="#validate_download_hash-118"><span class="linenos">118</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span><span id="validate_download_hash-119"><a href="#validate_download_hash-119"><span class="linenos">119</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="unzip_download">
+ <input id="unzip_download-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">unzip_download</span><span class="signature pdoc-code condensed">(<span class="param"><span class="n">file_destination</span><span class="p">:</span> <span class="nb">str</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="unzip_download-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#unzip_download"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="unzip_download-122"><a href="#unzip_download-122"><span class="linenos">122</span></a><span class="k">def</span><span class="w"> </span><span class="nf">unzip_download</span><span class="p">(</span><span class="n">file_destination</span><span class="p">:</span> <span class="nb">str</span><span class="p">):</span>
+</span><span id="unzip_download-123"><a href="#unzip_download-123"><span class="linenos">123</span></a> <span class="c1"># check folder is a directory</span>
+</span><span id="unzip_download-124"><a href="#unzip_download-124"><span class="linenos">124</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">DOWNLOADS_PATH</span><span class="o">.</span><span class="n">is_dir</span><span class="p">():</span>
+</span><span id="unzip_download-125"><a href="#unzip_download-125"><span class="linenos">125</span></a> <span class="k">raise</span> <span class="ne">NotADirectoryError</span><span class="p">(</span>
+</span><span id="unzip_download-126"><a href="#unzip_download-126"><span class="linenos">126</span></a> <span class="sa">f</span><span class="s2">"Error: '</span><span class="si">{</span><span class="n">DOWNLOADS_PATH</span><span class="si">}</span><span class="s2">' for TRUD resoruces is not a directory"</span>
+</span><span id="unzip_download-127"><a href="#unzip_download-127"><span class="linenos">127</span></a> <span class="p">)</span>
+</span><span id="unzip_download-128"><a href="#unzip_download-128"><span class="linenos">128</span></a>
+</span><span id="unzip_download-129"><a href="#unzip_download-129"><span class="linenos">129</span></a> <span class="k">with</span> <span class="n">zipfile</span><span class="o">.</span><span class="n">ZipFile</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span> <span class="k">as</span> <span class="n">zip_ref</span><span class="p">:</span>
+</span><span id="unzip_download-130"><a href="#unzip_download-130"><span class="linenos">130</span></a> <span class="n">zip_ref</span><span class="o">.</span><span class="n">extractall</span><span class="p">(</span><span class="n">DOWNLOADS_PATH</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="extract_icd10">
+ <input id="extract_icd10-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">extract_icd10</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="extract_icd10-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#extract_icd10"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="extract_icd10-133"><a href="#extract_icd10-133"><span class="linenos">133</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_icd10</span><span class="p">():</span>
+</span><span id="extract_icd10-134"><a href="#extract_icd10-134"><span class="linenos">134</span></a> <span class="c1"># ICD10_edition5</span>
+</span><span id="extract_icd10-135"><a href="#extract_icd10-135"><span class="linenos">135</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_icd10-136"><a href="#extract_icd10-136"><span class="linenos">136</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_icd10-137"><a href="#extract_icd10-137"><span class="linenos">137</span></a> <span class="o">/</span> <span class="s2">"ICD10_Edition5_XML_20160401"</span>
+</span><span id="extract_icd10-138"><a href="#extract_icd10-138"><span class="linenos">138</span></a> <span class="o">/</span> <span class="s2">"Content"</span>
+</span><span id="extract_icd10-139"><a href="#extract_icd10-139"><span class="linenos">139</span></a> <span class="o">/</span> <span class="s2">"ICD10_Edition5_CodesAndTitlesAndMetadata_GB_20160401.xml"</span>
+</span><span id="extract_icd10-140"><a href="#extract_icd10-140"><span class="linenos">140</span></a> <span class="p">)</span>
+</span><span id="extract_icd10-141"><a href="#extract_icd10-141"><span class="linenos">141</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_xml</span><span class="p">(</span><span class="n">file_path</span><span class="p">)</span>
+</span><span id="extract_icd10-142"><a href="#extract_icd10-142"><span class="linenos">142</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"CODE"</span><span class="p">,</span> <span class="s2">"ALT_CODE"</span><span class="p">,</span> <span class="s2">"DESCRIPTION"</span><span class="p">]]</span>
+</span><span id="extract_icd10-143"><a href="#extract_icd10-143"><span class="linenos">143</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span>
+</span><span id="extract_icd10-144"><a href="#extract_icd10-144"><span class="linenos">144</span></a> <span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"CODE"</span><span class="p">:</span> <span class="s2">"icd10"</span><span class="p">,</span> <span class="s2">"ALT_CODE"</span><span class="p">:</span> <span class="s2">"icd10_alt"</span><span class="p">,</span> <span class="s2">"DESCRIPTION"</span><span class="p">:</span> <span class="s2">"description"</span><span class="p">}</span>
+</span><span id="extract_icd10-145"><a href="#extract_icd10-145"><span class="linenos">145</span></a> <span class="p">)</span>
+</span><span id="extract_icd10-146"><a href="#extract_icd10-146"><span class="linenos">146</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"icd10.parquet"</span>
+</span><span id="extract_icd10-147"><a href="#extract_icd10-147"><span class="linenos">147</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_icd10-148"><a href="#extract_icd10-148"><span class="linenos">148</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="extract_opsc4">
+ <input id="extract_opsc4-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">extract_opsc4</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="extract_opsc4-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#extract_opsc4"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="extract_opsc4-151"><a href="#extract_opsc4-151"><span class="linenos">151</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_opsc4</span><span class="p">():</span>
+</span><span id="extract_opsc4-152"><a href="#extract_opsc4-152"><span class="linenos">152</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_opsc4-153"><a href="#extract_opsc4-153"><span class="linenos">153</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_opsc4-154"><a href="#extract_opsc4-154"><span class="linenos">154</span></a> <span class="o">/</span> <span class="s2">"OPCS410 Data files txt"</span>
+</span><span id="extract_opsc4-155"><a href="#extract_opsc4-155"><span class="linenos">155</span></a> <span class="o">/</span> <span class="s2">"OPCS410 CodesAndTitles Nov 2022 V1.0.txt"</span>
+</span><span id="extract_opsc4-156"><a href="#extract_opsc4-156"><span class="linenos">156</span></a> <span class="p">)</span>
+</span><span id="extract_opsc4-157"><a href="#extract_opsc4-157"><span class="linenos">157</span></a>
+</span><span id="extract_opsc4-158"><a href="#extract_opsc4-158"><span class="linenos">158</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">,</span> <span class="n">header</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
+</span><span id="extract_opsc4-159"><a href="#extract_opsc4-159"><span class="linenos">159</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="mi">0</span><span class="p">:</span> <span class="s2">"opcs4"</span><span class="p">,</span> <span class="mi">1</span><span class="p">:</span> <span class="s2">"description"</span><span class="p">})</span>
+</span><span id="extract_opsc4-160"><a href="#extract_opsc4-160"><span class="linenos">160</span></a>
+</span><span id="extract_opsc4-161"><a href="#extract_opsc4-161"><span class="linenos">161</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"opcs4.parquet"</span>
+</span><span id="extract_opsc4-162"><a href="#extract_opsc4-162"><span class="linenos">162</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_opsc4-163"><a href="#extract_opsc4-163"><span class="linenos">163</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="extract_nhs_data_migrations">
+ <input id="extract_nhs_data_migrations-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">extract_nhs_data_migrations</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="extract_nhs_data_migrations-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#extract_nhs_data_migrations"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="extract_nhs_data_migrations-166"><a href="#extract_nhs_data_migrations-166"><span class="linenos">166</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_nhs_data_migrations</span><span class="p">():</span>
+</span><span id="extract_nhs_data_migrations-167"><a href="#extract_nhs_data_migrations-167"><span class="linenos">167</span></a> <span class="c1"># NHS Data Migrations</span>
+</span><span id="extract_nhs_data_migrations-168"><a href="#extract_nhs_data_migrations-168"><span class="linenos">168</span></a>
+</span><span id="extract_nhs_data_migrations-169"><a href="#extract_nhs_data_migrations-169"><span class="linenos">169</span></a> <span class="c1"># snomed only</span>
+</span><span id="extract_nhs_data_migrations-170"><a href="#extract_nhs_data_migrations-170"><span class="linenos">170</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_nhs_data_migrations-171"><a href="#extract_nhs_data_migrations-171"><span class="linenos">171</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_nhs_data_migrations-172"><a href="#extract_nhs_data_migrations-172"><span class="linenos">172</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="extract_nhs_data_migrations-173"><a href="#extract_nhs_data_migrations-173"><span class="linenos">173</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="extract_nhs_data_migrations-174"><a href="#extract_nhs_data_migrations-174"><span class="linenos">174</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="extract_nhs_data_migrations-175"><a href="#extract_nhs_data_migrations-175"><span class="linenos">175</span></a> <span class="o">/</span> <span class="s2">"sctcremap_uk_20200401000001.txt"</span>
+</span><span id="extract_nhs_data_migrations-176"><a href="#extract_nhs_data_migrations-176"><span class="linenos">176</span></a> <span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-177"><a href="#extract_nhs_data_migrations-177"><span class="linenos">177</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-178"><a href="#extract_nhs_data_migrations-178"><span class="linenos">178</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"SCT_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="extract_nhs_data_migrations-179"><a href="#extract_nhs_data_migrations-179"><span class="linenos">179</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"SCT_CONCEPTID"</span><span class="p">:</span> <span class="s2">"snomed"</span><span class="p">})</span>
+</span><span id="extract_nhs_data_migrations-180"><a href="#extract_nhs_data_migrations-180"><span class="linenos">180</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">()</span>
+</span><span id="extract_nhs_data_migrations-181"><a href="#extract_nhs_data_migrations-181"><span class="linenos">181</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-182"><a href="#extract_nhs_data_migrations-182"><span class="linenos">182</span></a>
+</span><span id="extract_nhs_data_migrations-183"><a href="#extract_nhs_data_migrations-183"><span class="linenos">183</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"snomed.parquet"</span>
+</span><span id="extract_nhs_data_migrations-184"><a href="#extract_nhs_data_migrations-184"><span class="linenos">184</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-185"><a href="#extract_nhs_data_migrations-185"><span class="linenos">185</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-186"><a href="#extract_nhs_data_migrations-186"><span class="linenos">186</span></a>
+</span><span id="extract_nhs_data_migrations-187"><a href="#extract_nhs_data_migrations-187"><span class="linenos">187</span></a> <span class="c1"># r2 -> r3</span>
+</span><span id="extract_nhs_data_migrations-188"><a href="#extract_nhs_data_migrations-188"><span class="linenos">188</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_nhs_data_migrations-189"><a href="#extract_nhs_data_migrations-189"><span class="linenos">189</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_nhs_data_migrations-190"><a href="#extract_nhs_data_migrations-190"><span class="linenos">190</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="extract_nhs_data_migrations-191"><a href="#extract_nhs_data_migrations-191"><span class="linenos">191</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="extract_nhs_data_migrations-192"><a href="#extract_nhs_data_migrations-192"><span class="linenos">192</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="extract_nhs_data_migrations-193"><a href="#extract_nhs_data_migrations-193"><span class="linenos">193</span></a> <span class="o">/</span> <span class="s2">"rctctv3map_uk_20200401000001.txt"</span>
+</span><span id="extract_nhs_data_migrations-194"><a href="#extract_nhs_data_migrations-194"><span class="linenos">194</span></a> <span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-195"><a href="#extract_nhs_data_migrations-195"><span class="linenos">195</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-196"><a href="#extract_nhs_data_migrations-196"><span class="linenos">196</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"V2_CONCEPTID"</span><span class="p">,</span> <span class="s2">"CTV3_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="extract_nhs_data_migrations-197"><a href="#extract_nhs_data_migrations-197"><span class="linenos">197</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"V2_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"CTV3_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">})</span>
+</span><span id="extract_nhs_data_migrations-198"><a href="#extract_nhs_data_migrations-198"><span class="linenos">198</span></a>
+</span><span id="extract_nhs_data_migrations-199"><a href="#extract_nhs_data_migrations-199"><span class="linenos">199</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_read3.parquet"</span>
+</span><span id="extract_nhs_data_migrations-200"><a href="#extract_nhs_data_migrations-200"><span class="linenos">200</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-201"><a href="#extract_nhs_data_migrations-201"><span class="linenos">201</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-202"><a href="#extract_nhs_data_migrations-202"><span class="linenos">202</span></a>
+</span><span id="extract_nhs_data_migrations-203"><a href="#extract_nhs_data_migrations-203"><span class="linenos">203</span></a> <span class="c1"># r3->r2</span>
+</span><span id="extract_nhs_data_migrations-204"><a href="#extract_nhs_data_migrations-204"><span class="linenos">204</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_nhs_data_migrations-205"><a href="#extract_nhs_data_migrations-205"><span class="linenos">205</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_nhs_data_migrations-206"><a href="#extract_nhs_data_migrations-206"><span class="linenos">206</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="extract_nhs_data_migrations-207"><a href="#extract_nhs_data_migrations-207"><span class="linenos">207</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="extract_nhs_data_migrations-208"><a href="#extract_nhs_data_migrations-208"><span class="linenos">208</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="extract_nhs_data_migrations-209"><a href="#extract_nhs_data_migrations-209"><span class="linenos">209</span></a> <span class="o">/</span> <span class="s2">"ctv3rctmap_uk_20200401000002.txt"</span>
+</span><span id="extract_nhs_data_migrations-210"><a href="#extract_nhs_data_migrations-210"><span class="linenos">210</span></a> <span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-211"><a href="#extract_nhs_data_migrations-211"><span class="linenos">211</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-212"><a href="#extract_nhs_data_migrations-212"><span class="linenos">212</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"CTV3_CONCEPTID"</span><span class="p">,</span> <span class="s2">"V2_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="extract_nhs_data_migrations-213"><a href="#extract_nhs_data_migrations-213"><span class="linenos">213</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"CTV3_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"V2_CONCEPTID"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">})</span>
+</span><span id="extract_nhs_data_migrations-214"><a href="#extract_nhs_data_migrations-214"><span class="linenos">214</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">()</span>
+</span><span id="extract_nhs_data_migrations-215"><a href="#extract_nhs_data_migrations-215"><span class="linenos">215</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read2"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*_.*$"</span><span class="p">)]</span> <span class="c1"># remove r2 codes with '_'</span>
+</span><span id="extract_nhs_data_migrations-216"><a href="#extract_nhs_data_migrations-216"><span class="linenos">216</span></a>
+</span><span id="extract_nhs_data_migrations-217"><a href="#extract_nhs_data_migrations-217"><span class="linenos">217</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_read2.parquet"</span>
+</span><span id="extract_nhs_data_migrations-218"><a href="#extract_nhs_data_migrations-218"><span class="linenos">218</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-219"><a href="#extract_nhs_data_migrations-219"><span class="linenos">219</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-220"><a href="#extract_nhs_data_migrations-220"><span class="linenos">220</span></a>
+</span><span id="extract_nhs_data_migrations-221"><a href="#extract_nhs_data_migrations-221"><span class="linenos">221</span></a> <span class="c1"># r2 -> snomed</span>
+</span><span id="extract_nhs_data_migrations-222"><a href="#extract_nhs_data_migrations-222"><span class="linenos">222</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_nhs_data_migrations-223"><a href="#extract_nhs_data_migrations-223"><span class="linenos">223</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_nhs_data_migrations-224"><a href="#extract_nhs_data_migrations-224"><span class="linenos">224</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="extract_nhs_data_migrations-225"><a href="#extract_nhs_data_migrations-225"><span class="linenos">225</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="extract_nhs_data_migrations-226"><a href="#extract_nhs_data_migrations-226"><span class="linenos">226</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="extract_nhs_data_migrations-227"><a href="#extract_nhs_data_migrations-227"><span class="linenos">227</span></a> <span class="o">/</span> <span class="s2">"rcsctmap2_uk_20200401000001.txt"</span>
+</span><span id="extract_nhs_data_migrations-228"><a href="#extract_nhs_data_migrations-228"><span class="linenos">228</span></a> <span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-229"><a href="#extract_nhs_data_migrations-229"><span class="linenos">229</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-230"><a href="#extract_nhs_data_migrations-230"><span class="linenos">230</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"ReadCode"</span><span class="p">,</span> <span class="s2">"ConceptId"</span><span class="p">]]</span>
+</span><span id="extract_nhs_data_migrations-231"><a href="#extract_nhs_data_migrations-231"><span class="linenos">231</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"ReadCode"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"ConceptId"</span><span class="p">:</span> <span class="s2">"snomed"</span><span class="p">})</span>
+</span><span id="extract_nhs_data_migrations-232"><a href="#extract_nhs_data_migrations-232"><span class="linenos">232</span></a>
+</span><span id="extract_nhs_data_migrations-233"><a href="#extract_nhs_data_migrations-233"><span class="linenos">233</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_snomed.parquet"</span>
+</span><span id="extract_nhs_data_migrations-234"><a href="#extract_nhs_data_migrations-234"><span class="linenos">234</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-235"><a href="#extract_nhs_data_migrations-235"><span class="linenos">235</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-236"><a href="#extract_nhs_data_migrations-236"><span class="linenos">236</span></a>
+</span><span id="extract_nhs_data_migrations-237"><a href="#extract_nhs_data_migrations-237"><span class="linenos">237</span></a> <span class="c1"># r3->snomed</span>
+</span><span id="extract_nhs_data_migrations-238"><a href="#extract_nhs_data_migrations-238"><span class="linenos">238</span></a> <span class="n">file_path</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="extract_nhs_data_migrations-239"><a href="#extract_nhs_data_migrations-239"><span class="linenos">239</span></a> <span class="n">DOWNLOADS_PATH</span>
+</span><span id="extract_nhs_data_migrations-240"><a href="#extract_nhs_data_migrations-240"><span class="linenos">240</span></a> <span class="o">/</span> <span class="s2">"Mapping Tables"</span>
+</span><span id="extract_nhs_data_migrations-241"><a href="#extract_nhs_data_migrations-241"><span class="linenos">241</span></a> <span class="o">/</span> <span class="s2">"Updated"</span>
+</span><span id="extract_nhs_data_migrations-242"><a href="#extract_nhs_data_migrations-242"><span class="linenos">242</span></a> <span class="o">/</span> <span class="s2">"Clinically Assured"</span>
+</span><span id="extract_nhs_data_migrations-243"><a href="#extract_nhs_data_migrations-243"><span class="linenos">243</span></a> <span class="o">/</span> <span class="s2">"ctv3sctmap2_uk_20200401000001.txt"</span>
+</span><span id="extract_nhs_data_migrations-244"><a href="#extract_nhs_data_migrations-244"><span class="linenos">244</span></a> <span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-245"><a href="#extract_nhs_data_migrations-245"><span class="linenos">245</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="n">sep</span><span class="o">=</span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-246"><a href="#extract_nhs_data_migrations-246"><span class="linenos">246</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"CTV3_TERMID"</span><span class="p">,</span> <span class="s2">"SCT_CONCEPTID"</span><span class="p">]]</span>
+</span><span id="extract_nhs_data_migrations-247"><a href="#extract_nhs_data_migrations-247"><span class="linenos">247</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"CTV3_TERMID"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"SCT_CONCEPTID"</span><span class="p">:</span> <span class="s2">"snomed"</span><span class="p">})</span>
+</span><span id="extract_nhs_data_migrations-248"><a href="#extract_nhs_data_migrations-248"><span class="linenos">248</span></a> <span class="n">df</span><span class="p">[</span><span class="s2">"snomed"</span><span class="p">]</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="s2">"snomed"</span><span class="p">]</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-249"><a href="#extract_nhs_data_migrations-249"><span class="linenos">249</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"snomed"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*_.*$"</span><span class="p">)]</span> <span class="c1"># remove snomed codes with '_'</span>
+</span><span id="extract_nhs_data_migrations-250"><a href="#extract_nhs_data_migrations-250"><span class="linenos">250</span></a>
+</span><span id="extract_nhs_data_migrations-251"><a href="#extract_nhs_data_migrations-251"><span class="linenos">251</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_snomed.parquet"</span>
+</span><span id="extract_nhs_data_migrations-252"><a href="#extract_nhs_data_migrations-252"><span class="linenos">252</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_data_migrations-253"><a href="#extract_nhs_data_migrations-253"><span class="linenos">253</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="extract_nhs_read_browser">
+ <input id="extract_nhs_read_browser-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">extract_nhs_read_browser</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="extract_nhs_read_browser-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#extract_nhs_read_browser"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="extract_nhs_read_browser-256"><a href="#extract_nhs_read_browser-256"><span class="linenos">256</span></a><span class="k">def</span><span class="w"> </span><span class="nf">extract_nhs_read_browser</span><span class="p">():</span>
+</span><span id="extract_nhs_read_browser-257"><a href="#extract_nhs_read_browser-257"><span class="linenos">257</span></a> <span class="c1"># r2 only</span>
+</span><span id="extract_nhs_read_browser-258"><a href="#extract_nhs_read_browser-258"><span class="linenos">258</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"ANCESTOR.DBF"</span>
+</span><span id="extract_nhs_read_browser-259"><a href="#extract_nhs_read_browser-259"><span class="linenos">259</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-260"><a href="#extract_nhs_read_browser-260"><span class="linenos">260</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">df</span><span class="p">[</span><span class="s2">"READCODE"</span><span class="p">],</span> <span class="n">df</span><span class="p">[</span><span class="s2">"DESCENDANT"</span><span class="p">]])</span>
+</span><span id="extract_nhs_read_browser-261"><a href="#extract_nhs_read_browser-261"><span class="linenos">261</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">())</span>
+</span><span id="extract_nhs_read_browser-262"><a href="#extract_nhs_read_browser-262"><span class="linenos">262</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="mi">0</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-263"><a href="#extract_nhs_read_browser-263"><span class="linenos">263</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2.parquet"</span>
+</span><span id="extract_nhs_read_browser-264"><a href="#extract_nhs_read_browser-264"><span class="linenos">264</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-265"><a href="#extract_nhs_read_browser-265"><span class="linenos">265</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-266"><a href="#extract_nhs_read_browser-266"><span class="linenos">266</span></a>
+</span><span id="extract_nhs_read_browser-267"><a href="#extract_nhs_read_browser-267"><span class="linenos">267</span></a> <span class="c1"># r2 -> atc</span>
+</span><span id="extract_nhs_read_browser-268"><a href="#extract_nhs_read_browser-268"><span class="linenos">268</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"ATC.DBF"</span>
+</span><span id="extract_nhs_read_browser-269"><a href="#extract_nhs_read_browser-269"><span class="linenos">269</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-270"><a href="#extract_nhs_read_browser-270"><span class="linenos">270</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READCODE"</span><span class="p">,</span> <span class="s2">"ATC"</span><span class="p">]]</span>
+</span><span id="extract_nhs_read_browser-271"><a href="#extract_nhs_read_browser-271"><span class="linenos">271</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READCODE"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"ATC"</span><span class="p">:</span> <span class="s2">"atc"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-272"><a href="#extract_nhs_read_browser-272"><span class="linenos">272</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_atc.parquet"</span>
+</span><span id="extract_nhs_read_browser-273"><a href="#extract_nhs_read_browser-273"><span class="linenos">273</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-274"><a href="#extract_nhs_read_browser-274"><span class="linenos">274</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-275"><a href="#extract_nhs_read_browser-275"><span class="linenos">275</span></a>
+</span><span id="extract_nhs_read_browser-276"><a href="#extract_nhs_read_browser-276"><span class="linenos">276</span></a> <span class="c1"># r2 -> icd10</span>
+</span><span id="extract_nhs_read_browser-277"><a href="#extract_nhs_read_browser-277"><span class="linenos">277</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"ICD10.DBF"</span>
+</span><span id="extract_nhs_read_browser-278"><a href="#extract_nhs_read_browser-278"><span class="linenos">278</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-279"><a href="#extract_nhs_read_browser-279"><span class="linenos">279</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="extract_nhs_read_browser-280"><a href="#extract_nhs_read_browser-280"><span class="linenos">280</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"icd10"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-281"><a href="#extract_nhs_read_browser-281"><span class="linenos">281</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"icd10"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-282"><a href="#extract_nhs_read_browser-282"><span class="linenos">282</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read2"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-283"><a href="#extract_nhs_read_browser-283"><span class="linenos">283</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_icd10.parquet"</span>
+</span><span id="extract_nhs_read_browser-284"><a href="#extract_nhs_read_browser-284"><span class="linenos">284</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-285"><a href="#extract_nhs_read_browser-285"><span class="linenos">285</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-286"><a href="#extract_nhs_read_browser-286"><span class="linenos">286</span></a>
+</span><span id="extract_nhs_read_browser-287"><a href="#extract_nhs_read_browser-287"><span class="linenos">287</span></a> <span class="c1"># r2 -> opcs4</span>
+</span><span id="extract_nhs_read_browser-288"><a href="#extract_nhs_read_browser-288"><span class="linenos">288</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V2"</span> <span class="o">/</span> <span class="s2">"OPCS4V3.DBF"</span>
+</span><span id="extract_nhs_read_browser-289"><a href="#extract_nhs_read_browser-289"><span class="linenos">289</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-290"><a href="#extract_nhs_read_browser-290"><span class="linenos">290</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="extract_nhs_read_browser-291"><a href="#extract_nhs_read_browser-291"><span class="linenos">291</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read2"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"opcs4"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-292"><a href="#extract_nhs_read_browser-292"><span class="linenos">292</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"opcs4"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-293"><a href="#extract_nhs_read_browser-293"><span class="linenos">293</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read2"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-294"><a href="#extract_nhs_read_browser-294"><span class="linenos">294</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read2_to_opcs4.parquet"</span>
+</span><span id="extract_nhs_read_browser-295"><a href="#extract_nhs_read_browser-295"><span class="linenos">295</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-296"><a href="#extract_nhs_read_browser-296"><span class="linenos">296</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-297"><a href="#extract_nhs_read_browser-297"><span class="linenos">297</span></a>
+</span><span id="extract_nhs_read_browser-298"><a href="#extract_nhs_read_browser-298"><span class="linenos">298</span></a> <span class="c1"># r3 only</span>
+</span><span id="extract_nhs_read_browser-299"><a href="#extract_nhs_read_browser-299"><span class="linenos">299</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V3"</span> <span class="o">/</span> <span class="s2">"ANCESTOR.DBF"</span>
+</span><span id="extract_nhs_read_browser-300"><a href="#extract_nhs_read_browser-300"><span class="linenos">300</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-301"><a href="#extract_nhs_read_browser-301"><span class="linenos">301</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="n">df</span><span class="p">[</span><span class="s2">"READCODE"</span><span class="p">],</span> <span class="n">df</span><span class="p">[</span><span class="s2">"DESCENDANT"</span><span class="p">]])</span>
+</span><span id="extract_nhs_read_browser-302"><a href="#extract_nhs_read_browser-302"><span class="linenos">302</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">drop_duplicates</span><span class="p">())</span>
+</span><span id="extract_nhs_read_browser-303"><a href="#extract_nhs_read_browser-303"><span class="linenos">303</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="mi">0</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-304"><a href="#extract_nhs_read_browser-304"><span class="linenos">304</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3.parquet"</span>
+</span><span id="extract_nhs_read_browser-305"><a href="#extract_nhs_read_browser-305"><span class="linenos">305</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-306"><a href="#extract_nhs_read_browser-306"><span class="linenos">306</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-307"><a href="#extract_nhs_read_browser-307"><span class="linenos">307</span></a>
+</span><span id="extract_nhs_read_browser-308"><a href="#extract_nhs_read_browser-308"><span class="linenos">308</span></a> <span class="c1"># r3 -> icd10</span>
+</span><span id="extract_nhs_read_browser-309"><a href="#extract_nhs_read_browser-309"><span class="linenos">309</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V3"</span> <span class="o">/</span> <span class="s2">"ICD10.DBF"</span>
+</span><span id="extract_nhs_read_browser-310"><a href="#extract_nhs_read_browser-310"><span class="linenos">310</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-311"><a href="#extract_nhs_read_browser-311"><span class="linenos">311</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="extract_nhs_read_browser-312"><a href="#extract_nhs_read_browser-312"><span class="linenos">312</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"icd10"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-313"><a href="#extract_nhs_read_browser-313"><span class="linenos">313</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"icd10"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-314"><a href="#extract_nhs_read_browser-314"><span class="linenos">314</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read3"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-315"><a href="#extract_nhs_read_browser-315"><span class="linenos">315</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_icd10.parquet"</span>
+</span><span id="extract_nhs_read_browser-316"><a href="#extract_nhs_read_browser-316"><span class="linenos">316</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-317"><a href="#extract_nhs_read_browser-317"><span class="linenos">317</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-318"><a href="#extract_nhs_read_browser-318"><span class="linenos">318</span></a>
+</span><span id="extract_nhs_read_browser-319"><a href="#extract_nhs_read_browser-319"><span class="linenos">319</span></a> <span class="c1"># r3 -> icd9</span>
+</span><span id="extract_nhs_read_browser-320"><a href="#extract_nhs_read_browser-320"><span class="linenos">320</span></a> <span class="c1"># dbf = simpledbf.Dbf5('build/maps/downloads/Standard/V3/ICD9V3.DBF')</span>
+</span><span id="extract_nhs_read_browser-321"><a href="#extract_nhs_read_browser-321"><span class="linenos">321</span></a>
+</span><span id="extract_nhs_read_browser-322"><a href="#extract_nhs_read_browser-322"><span class="linenos">322</span></a> <span class="c1"># r3 -> opcs4</span>
+</span><span id="extract_nhs_read_browser-323"><a href="#extract_nhs_read_browser-323"><span class="linenos">323</span></a> <span class="n">input_path</span> <span class="o">=</span> <span class="n">DOWNLOADS_PATH</span> <span class="o">/</span> <span class="s2">"Standard"</span> <span class="o">/</span> <span class="s2">"V3"</span> <span class="o">/</span> <span class="s2">"OPCS4V3.DBF"</span>
+</span><span id="extract_nhs_read_browser-324"><a href="#extract_nhs_read_browser-324"><span class="linenos">324</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">simpledbf</span><span class="o">.</span><span class="n">Dbf5</span><span class="p">(</span><span class="n">input_path</span><span class="p">)</span><span class="o">.</span><span class="n">to_dataframe</span><span class="p">()</span>
+</span><span id="extract_nhs_read_browser-325"><a href="#extract_nhs_read_browser-325"><span class="linenos">325</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[[</span><span class="s2">"READ_CODE"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">]]</span>
+</span><span id="extract_nhs_read_browser-326"><a href="#extract_nhs_read_browser-326"><span class="linenos">326</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">columns</span><span class="o">=</span><span class="p">{</span><span class="s2">"READ_CODE"</span><span class="p">:</span> <span class="s2">"read3"</span><span class="p">,</span> <span class="s2">"TARG_CODE"</span><span class="p">:</span> <span class="s2">"opcs4"</span><span class="p">})</span>
+</span><span id="extract_nhs_read_browser-327"><a href="#extract_nhs_read_browser-327"><span class="linenos">327</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"opcs4"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-328"><a href="#extract_nhs_read_browser-328"><span class="linenos">328</span></a> <span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="p">[</span><span class="o">~</span><span class="n">df</span><span class="p">[</span><span class="s2">"read3"</span><span class="p">]</span><span class="o">.</span><span class="n">str</span><span class="o">.</span><span class="n">match</span><span class="p">(</span><span class="s2">"^.*-.*$"</span><span class="p">)]</span> <span class="c1"># remove codes with '-'</span>
+</span><span id="extract_nhs_read_browser-329"><a href="#extract_nhs_read_browser-329"><span class="linenos">329</span></a> <span class="n">output_path</span> <span class="o">=</span> <span class="n">PROCESSED_PATH</span> <span class="o">/</span> <span class="s2">"read3_to_opcs4.parquet"</span>
+</span><span id="extract_nhs_read_browser-330"><a href="#extract_nhs_read_browser-330"><span class="linenos">330</span></a> <span class="n">df</span><span class="o">.</span><span class="n">to_parquet</span><span class="p">(</span><span class="n">output_path</span><span class="p">,</span> <span class="n">index</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+</span><span id="extract_nhs_read_browser-331"><a href="#extract_nhs_read_browser-331"><span class="linenos">331</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Extracted: </span><span class="si">{</span><span class="n">output_path</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ <section id="create_map_directories">
+ <input id="create_map_directories-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">create_map_directories</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="create_map_directories-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#create_map_directories"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="create_map_directories-334"><a href="#create_map_directories-334"><span class="linenos">334</span></a><span class="k">def</span><span class="w"> </span><span class="nf">create_map_directories</span><span class="p">():</span>
+</span><span id="create_map_directories-335"><a href="#create_map_directories-335"><span class="linenos">335</span></a><span class="w"> </span><span class="sd">"""Create map directories."""</span>
+</span><span id="create_map_directories-336"><a href="#create_map_directories-336"><span class="linenos">336</span></a>
+</span><span id="create_map_directories-337"><a href="#create_map_directories-337"><span class="linenos">337</span></a> <span class="c1"># Check if build directory exists</span>
+</span><span id="create_map_directories-338"><a href="#create_map_directories-338"><span class="linenos">338</span></a> <span class="n">create_map_dirs</span> <span class="o">=</span> <span class="kc">False</span>
+</span><span id="create_map_directories-339"><a href="#create_map_directories-339"><span class="linenos">339</span></a> <span class="k">if</span> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">exists</span><span class="p">():</span>
+</span><span id="create_map_directories-340"><a href="#create_map_directories-340"><span class="linenos">340</span></a> <span class="n">user_input</span> <span class="o">=</span> <span class="p">(</span>
+</span><span id="create_map_directories-341"><a href="#create_map_directories-341"><span class="linenos">341</span></a> <span class="nb">input</span><span class="p">(</span>
+</span><span id="create_map_directories-342"><a href="#create_map_directories-342"><span class="linenos">342</span></a> <span class="sa">f</span><span class="s2">"The map directory </span><span class="si">{</span><span class="n">VOCAB_PATH</span><span class="si">}</span><span class="s2"> already exists. Do you want to download and process trud data again? (y/n): "</span>
+</span><span id="create_map_directories-343"><a href="#create_map_directories-343"><span class="linenos">343</span></a> <span class="p">)</span>
+</span><span id="create_map_directories-344"><a href="#create_map_directories-344"><span class="linenos">344</span></a> <span class="o">.</span><span class="n">strip</span><span class="p">()</span>
+</span><span id="create_map_directories-345"><a href="#create_map_directories-345"><span class="linenos">345</span></a> <span class="o">.</span><span class="n">lower</span><span class="p">()</span>
+</span><span id="create_map_directories-346"><a href="#create_map_directories-346"><span class="linenos">346</span></a> <span class="p">)</span>
+</span><span id="create_map_directories-347"><a href="#create_map_directories-347"><span class="linenos">347</span></a> <span class="k">if</span> <span class="n">user_input</span> <span class="o">==</span> <span class="s2">"y"</span><span class="p">:</span>
+</span><span id="create_map_directories-348"><a href="#create_map_directories-348"><span class="linenos">348</span></a> <span class="c1"># delete all build files</span>
+</span><span id="create_map_directories-349"><a href="#create_map_directories-349"><span class="linenos">349</span></a> <span class="n">shutil</span><span class="o">.</span><span class="n">rmtree</span><span class="p">(</span><span class="n">VOCAB_PATH</span><span class="p">)</span>
+</span><span id="create_map_directories-350"><a href="#create_map_directories-350"><span class="linenos">350</span></a> <span class="n">create_map_dirs</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="create_map_directories-351"><a href="#create_map_directories-351"><span class="linenos">351</span></a> <span class="k">elif</span> <span class="n">user_input</span> <span class="o">==</span> <span class="s2">"n"</span><span class="p">:</span>
+</span><span id="create_map_directories-352"><a href="#create_map_directories-352"><span class="linenos">352</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Exiting TRUD installation"</span><span class="p">)</span>
+</span><span id="create_map_directories-353"><a href="#create_map_directories-353"><span class="linenos">353</span></a> <span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+</span><span id="create_map_directories-354"><a href="#create_map_directories-354"><span class="linenos">354</span></a> <span class="k">else</span><span class="p">:</span>
+</span><span id="create_map_directories-355"><a href="#create_map_directories-355"><span class="linenos">355</span></a> <span class="n">create_map_dirs</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="create_map_directories-356"><a href="#create_map_directories-356"><span class="linenos">356</span></a>
+</span><span id="create_map_directories-357"><a href="#create_map_directories-357"><span class="linenos">357</span></a> <span class="k">if</span> <span class="n">create_map_dirs</span><span class="p">:</span>
+</span><span id="create_map_directories-358"><a href="#create_map_directories-358"><span class="linenos">358</span></a> <span class="c1"># create maps directories</span>
+</span><span id="create_map_directories-359"><a href="#create_map_directories-359"><span class="linenos">359</span></a> <span class="n">VOCAB_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="create_map_directories-360"><a href="#create_map_directories-360"><span class="linenos">360</span></a> <span class="n">DOWNLOADS_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span><span id="create_map_directories-361"><a href="#create_map_directories-361"><span class="linenos">361</span></a> <span class="n">PROCESSED_PATH</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="n">parents</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Create map directories.</p>
+</div>
+
+
+ </section>
+ <section id="install">
+ <input id="install-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">install</span><span class="signature pdoc-code condensed">(<span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="install-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#install"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="install-364"><a href="#install-364"><span class="linenos">364</span></a><span class="k">def</span><span class="w"> </span><span class="nf">install</span><span class="p">():</span>
+</span><span id="install-365"><a href="#install-365"><span class="linenos">365</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Installing TRUD"</span><span class="p">)</span>
+</span><span id="install-366"><a href="#install-366"><span class="linenos">366</span></a>
+</span><span id="install-367"><a href="#install-367"><span class="linenos">367</span></a> <span class="c1"># get TRUD api key from environment variable</span>
+</span><span id="install-368"><a href="#install-368"><span class="linenos">368</span></a> <span class="n">api_key</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">getenv</span><span class="p">(</span><span class="s2">"ACMC_TRUD_API_KEY"</span><span class="p">)</span>
+</span><span id="install-369"><a href="#install-369"><span class="linenos">369</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">api_key</span><span class="p">:</span>
+</span><span id="install-370"><a href="#install-370"><span class="linenos">370</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
+</span><span id="install-371"><a href="#install-371"><span class="linenos">371</span></a> <span class="s2">"TRUD API KEY not found. Set the ACMC_TRUD_API_KEY environment variable."</span>
+</span><span id="install-372"><a href="#install-372"><span class="linenos">372</span></a> <span class="p">)</span>
+</span><span id="install-373"><a href="#install-373"><span class="linenos">373</span></a>
+</span><span id="install-374"><a href="#install-374"><span class="linenos">374</span></a> <span class="n">create_map_directories</span><span class="p">()</span>
+</span><span id="install-375"><a href="#install-375"><span class="linenos">375</span></a>
+</span><span id="install-376"><a href="#install-376"><span class="linenos">376</span></a> <span class="n">items_latest</span> <span class="o">=</span> <span class="kc">True</span>
+</span><span id="install-377"><a href="#install-377"><span class="linenos">377</span></a> <span class="n">items</span> <span class="o">=</span> <span class="p">[</span>
+</span><span id="install-378"><a href="#install-378"><span class="linenos">378</span></a> <span class="p">{</span>
+</span><span id="install-379"><a href="#install-379"><span class="linenos">379</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">259</span><span class="p">,</span>
+</span><span id="install-380"><a href="#install-380"><span class="linenos">380</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS ICD-10 5th Edition XML data files"</span><span class="p">,</span>
+</span><span id="install-381"><a href="#install-381"><span class="linenos">381</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"A4F7BBA6E86349AADD0F4696C5E91152EB99CC06121427FC359160439B9F883F"</span><span class="p">,</span>
+</span><span id="install-382"><a href="#install-382"><span class="linenos">382</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_icd10</span><span class="p">,</span>
+</span><span id="install-383"><a href="#install-383"><span class="linenos">383</span></a> <span class="p">},</span>
+</span><span id="install-384"><a href="#install-384"><span class="linenos">384</span></a> <span class="p">{</span>
+</span><span id="install-385"><a href="#install-385"><span class="linenos">385</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">119</span><span class="p">,</span>
+</span><span id="install-386"><a href="#install-386"><span class="linenos">386</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"OPCS-4 data files"</span><span class="p">,</span>
+</span><span id="install-387"><a href="#install-387"><span class="linenos">387</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"0615A2BF43FFEF94517F1D1E0C05493B627839F323F22C52CBCD8B40BF767CD3"</span><span class="p">,</span>
+</span><span id="install-388"><a href="#install-388"><span class="linenos">388</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_opsc4</span><span class="p">,</span>
+</span><span id="install-389"><a href="#install-389"><span class="linenos">389</span></a> <span class="p">},</span>
+</span><span id="install-390"><a href="#install-390"><span class="linenos">390</span></a> <span class="p">{</span>
+</span><span id="install-391"><a href="#install-391"><span class="linenos">391</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">9</span><span class="p">,</span>
+</span><span id="install-392"><a href="#install-392"><span class="linenos">392</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS Data Migration"</span><span class="p">,</span>
+</span><span id="install-393"><a href="#install-393"><span class="linenos">393</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"D4317B3ADBA6E1247CF17F0B7CD2B8850FD36C0EA2923BF684EA6159F3A54765"</span><span class="p">,</span>
+</span><span id="install-394"><a href="#install-394"><span class="linenos">394</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_nhs_data_migrations</span><span class="p">,</span>
+</span><span id="install-395"><a href="#install-395"><span class="linenos">395</span></a> <span class="p">},</span>
+</span><span id="install-396"><a href="#install-396"><span class="linenos">396</span></a> <span class="p">{</span>
+</span><span id="install-397"><a href="#install-397"><span class="linenos">397</span></a> <span class="s2">"id"</span><span class="p">:</span> <span class="mi">8</span><span class="p">,</span>
+</span><span id="install-398"><a href="#install-398"><span class="linenos">398</span></a> <span class="s2">"name"</span><span class="p">:</span> <span class="s2">"NHS Read Browser"</span><span class="p">,</span>
+</span><span id="install-399"><a href="#install-399"><span class="linenos">399</span></a> <span class="s2">"hash"</span><span class="p">:</span> <span class="s2">"1FFF2CBF11D0E6D7FC6CC6F13DD52D2F459095C3D83A3F754E6C359F16913C5E"</span><span class="p">,</span>
+</span><span id="install-400"><a href="#install-400"><span class="linenos">400</span></a> <span class="s2">"extract"</span><span class="p">:</span> <span class="n">extract_nhs_read_browser</span><span class="p">,</span>
+</span><span id="install-401"><a href="#install-401"><span class="linenos">401</span></a> <span class="p">},</span>
+</span><span id="install-402"><a href="#install-402"><span class="linenos">402</span></a> <span class="c1"># TODO: Download BNF from separate site? https://www.nhsbsa.nhs.uk/sites/default/files/2024-10/BNF%20Snomed%20Mapping%20data%2020241016.zip</span>
+</span><span id="install-403"><a href="#install-403"><span class="linenos">403</span></a> <span class="p">]</span>
+</span><span id="install-404"><a href="#install-404"><span class="linenos">404</span></a>
+</span><span id="install-405"><a href="#install-405"><span class="linenos">405</span></a> <span class="c1"># remove function from items to save versions</span>
+</span><span id="install-406"><a href="#install-406"><span class="linenos">406</span></a> <span class="n">data</span> <span class="o">=</span> <span class="p">[{</span><span class="n">k</span><span class="p">:</span> <span class="n">v</span> <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">d</span><span class="o">.</span><span class="n">items</span><span class="p">()</span> <span class="k">if</span> <span class="n">k</span> <span class="o">!=</span> <span class="s2">"extract"</span><span class="p">}</span> <span class="k">for</span> <span class="n">d</span> <span class="ow">in</span> <span class="n">items</span><span class="p">]</span>
+</span><span id="install-407"><a href="#install-407"><span class="linenos">407</span></a> <span class="c1"># save TRUD versions to file to main record of what was downloaded</span>
+</span><span id="install-408"><a href="#install-408"><span class="linenos">408</span></a> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">VERSION_PATH</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">file</span><span class="p">:</span>
+</span><span id="install-409"><a href="#install-409"><span class="linenos">409</span></a> <span class="n">yaml</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span>
+</span><span id="install-410"><a href="#install-410"><span class="linenos">410</span></a> <span class="n">data</span><span class="p">,</span>
+</span><span id="install-411"><a href="#install-411"><span class="linenos">411</span></a> <span class="n">file</span><span class="p">,</span>
+</span><span id="install-412"><a href="#install-412"><span class="linenos">412</span></a> <span class="n">Dumper</span><span class="o">=</span><span class="n">util</span><span class="o">.</span><span class="n">QuotedDumper</span><span class="p">,</span>
+</span><span id="install-413"><a href="#install-413"><span class="linenos">413</span></a> <span class="n">default_flow_style</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="install-414"><a href="#install-414"><span class="linenos">414</span></a> <span class="n">sort_keys</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
+</span><span id="install-415"><a href="#install-415"><span class="linenos">415</span></a> <span class="n">default_style</span><span class="o">=</span><span class="s1">'"'</span><span class="p">,</span>
+</span><span id="install-416"><a href="#install-416"><span class="linenos">416</span></a> <span class="p">)</span>
+</span><span id="install-417"><a href="#install-417"><span class="linenos">417</span></a>
+</span><span id="install-418"><a href="#install-418"><span class="linenos">418</span></a> <span class="c1"># Validate and process each item ID</span>
+</span><span id="install-419"><a href="#install-419"><span class="linenos">419</span></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">items</span><span class="p">:</span>
+</span><span id="install-420"><a href="#install-420"><span class="linenos">420</span></a> <span class="n">item_id</span> <span class="o">=</span> <span class="n">item</span><span class="p">[</span><span class="s2">"id"</span><span class="p">]</span>
+</span><span id="install-421"><a href="#install-421"><span class="linenos">421</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"--- </span><span class="si">{</span><span class="n">item</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]</span><span class="si">}</span><span class="s2"> ---"</span><span class="p">)</span>
+</span><span id="install-422"><a href="#install-422"><span class="linenos">422</span></a>
+</span><span id="install-423"><a href="#install-423"><span class="linenos">423</span></a> <span class="n">releases</span> <span class="o">=</span> <span class="n">get_releases</span><span class="p">(</span><span class="n">item_id</span><span class="p">,</span> <span class="n">API_KEY</span><span class="o">=</span><span class="n">api_key</span><span class="p">,</span> <span class="n">latest</span><span class="o">=</span><span class="n">items_latest</span><span class="p">)</span>
+</span><span id="install-424"><a href="#install-424"><span class="linenos">424</span></a> <span class="k">if</span> <span class="ow">not</span> <span class="n">releases</span><span class="p">:</span>
+</span><span id="install-425"><a href="#install-425"><span class="linenos">425</span></a> <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span><span class="sa">f</span><span class="s2">"No releases found for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
+</span><span id="install-426"><a href="#install-426"><span class="linenos">426</span></a>
+</span><span id="install-427"><a href="#install-427"><span class="linenos">427</span></a> <span class="c1"># Process each release in reverse order</span>
+</span><span id="install-428"><a href="#install-428"><span class="linenos">428</span></a> <span class="k">for</span> <span class="n">release_ordinal</span><span class="p">,</span> <span class="n">release</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">releases</span><span class="p">[::</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="mi">1</span><span class="p">):</span>
+</span><span id="install-429"><a href="#install-429"><span class="linenos">429</span></a> <span class="c1"># Download archive file</span>
+</span><span id="install-430"><a href="#install-430"><span class="linenos">430</span></a> <span class="n">file_destination</span> <span class="o">=</span> <span class="n">download_release_file</span><span class="p">(</span>
+</span><span id="install-431"><a href="#install-431"><span class="linenos">431</span></a> <span class="n">item_id</span><span class="p">,</span> <span class="n">release_ordinal</span><span class="p">,</span> <span class="n">release</span><span class="p">,</span> <span class="s2">"archive"</span>
+</span><span id="install-432"><a href="#install-432"><span class="linenos">432</span></a> <span class="p">)</span>
+</span><span id="install-433"><a href="#install-433"><span class="linenos">433</span></a>
+</span><span id="install-434"><a href="#install-434"><span class="linenos">434</span></a> <span class="c1"># Optional files</span>
+</span><span id="install-435"><a href="#install-435"><span class="linenos">435</span></a> <span class="c1"># if items.checksum:</span>
+</span><span id="install-436"><a href="#install-436"><span class="linenos">436</span></a> <span class="c1"># download_release_file(item["id"], release_ordinal, release, "checksum")</span>
+</span><span id="install-437"><a href="#install-437"><span class="linenos">437</span></a> <span class="c1"># if items.signature:</span>
+</span><span id="install-438"><a href="#install-438"><span class="linenos">438</span></a> <span class="c1"># download_release_file(item["id"], release_ordinal, release, "signature")</span>
+</span><span id="install-439"><a href="#install-439"><span class="linenos">439</span></a> <span class="c1"># if items.public_key:</span>
+</span><span id="install-440"><a href="#install-440"><span class="linenos">440</span></a> <span class="c1"># download_release_file(item["id"], release_ordinal, release, "publicKey", "public key")</span>
+</span><span id="install-441"><a href="#install-441"><span class="linenos">441</span></a>
+</span><span id="install-442"><a href="#install-442"><span class="linenos">442</span></a> <span class="c1"># Verify Hash if available</span>
+</span><span id="install-443"><a href="#install-443"><span class="linenos">443</span></a> <span class="k">if</span> <span class="s2">"hash"</span> <span class="ow">in</span> <span class="n">item</span><span class="p">:</span>
+</span><span id="install-444"><a href="#install-444"><span class="linenos">444</span></a> <span class="n">validate_download_hash</span><span class="p">(</span><span class="n">file_destination</span><span class="p">,</span> <span class="n">item</span><span class="p">[</span><span class="s2">"hash"</span><span class="p">])</span>
+</span><span id="install-445"><a href="#install-445"><span class="linenos">445</span></a>
+</span><span id="install-446"><a href="#install-446"><span class="linenos">446</span></a> <span class="c1"># Unzip downloaded .zip</span>
+</span><span id="install-447"><a href="#install-447"><span class="linenos">447</span></a> <span class="n">unzip_download</span><span class="p">(</span><span class="n">file_destination</span><span class="p">)</span>
+</span><span id="install-448"><a href="#install-448"><span class="linenos">448</span></a>
+</span><span id="install-449"><a href="#install-449"><span class="linenos">449</span></a> <span class="c1"># Extract Tables to parquet</span>
+</span><span id="install-450"><a href="#install-450"><span class="linenos">450</span></a> <span class="k">if</span> <span class="s2">"extract"</span> <span class="ow">in</span> <span class="n">item</span><span class="p">:</span>
+</span><span id="install-451"><a href="#install-451"><span class="linenos">451</span></a> <span class="n">item</span><span class="p">[</span><span class="s2">"extract"</span><span class="p">]()</span>
+</span><span id="install-452"><a href="#install-452"><span class="linenos">452</span></a>
+</span><span id="install-453"><a href="#install-453"><span class="linenos">453</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Downloaded </span><span class="si">{</span><span class="n">release_ordinal</span><span class="si">}</span><span class="s2"> release(s) for item </span><span class="si">{</span><span class="n">item_id</span><span class="si">}</span><span class="s2">."</span><span class="p">)</span>
+</span><span id="install-454"><a href="#install-454"><span class="linenos">454</span></a>
+</span><span id="install-455"><a href="#install-455"><span class="linenos">455</span></a> <span class="n">_logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="sa">f</span><span class="s2">"TRUD installation completed"</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/acmc/util.html b/docs/api/acmc/util.html
new file mode 100644
index 0000000000000000000000000000000000000000..f9bb06388e6627867c16fc4f6f566137bc749822
--- /dev/null
+++ b/docs/api/acmc/util.html
@@ -0,0 +1,308 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <meta name="generator" content="pdoc 15.0.1"/>
+ <title>acmc.util API documentation</title>
+
+ <style>/*! * Bootstrap Reboot v5.0.0 (https://getbootstrap.com/) * Copyright 2011-2021 The Bootstrap Authors * Copyright 2011-2021 Twitter, Inc. * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) * Forked from Normalize.css, licensed MIT (https://github.com/necolas/normalize.css/blob/master/LICENSE.md) */*,::after,::before{box-sizing:border-box}@media (prefers-reduced-motion:no-preference){:root{scroll-behavior:smooth}}body{margin:0;font-family:system-ui,-apple-system,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans","Liberation Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:1rem;font-weight:400;line-height:1.5;color:#212529;background-color:#fff;-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:transparent}hr{margin:1rem 0;color:inherit;background-color:currentColor;border:0;opacity:.25}hr:not([size]){height:1px}h1,h2,h3,h4,h5,h6{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}h1{font-size:calc(1.375rem + 1.5vw)}@media (min-width:1200px){h1{font-size:2.5rem}}h2{font-size:calc(1.325rem + .9vw)}@media (min-width:1200px){h2{font-size:2rem}}h3{font-size:calc(1.3rem + .6vw)}@media (min-width:1200px){h3{font-size:1.75rem}}h4{font-size:calc(1.275rem + .3vw)}@media (min-width:1200px){h4{font-size:1.5rem}}h5{font-size:1.25rem}h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[data-bs-original-title],abbr[title]{-webkit-text-decoration:underline dotted;text-decoration:underline dotted;cursor:help;-webkit-text-decoration-skip-ink:none;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}dl,ol,ul{margin-top:0;margin-bottom:1rem}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem}b,strong{font-weight:bolder}small{font-size:.875em}mark{padding:.2em;background-color:#fcf8e3}sub,sup{position:relative;font-size:.75em;line-height:0;vertical-align:baseline}sub{bottom:-.25em}sup{top:-.5em}a{color:#0d6efd;text-decoration:underline}a:hover{color:#0a58ca}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}code,kbd,pre,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em;direction:ltr;unicode-bidi:bidi-override}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:.875em}pre code{font-size:inherit;color:inherit;word-break:normal}code{font-size:.875em;color:#d63384;word-wrap:break-word}a>code{color:inherit}kbd{padding:.2rem .4rem;font-size:.875em;color:#fff;background-color:#212529;border-radius:.2rem}kbd kbd{padding:0;font-size:1em;font-weight:700}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:#6c757d;text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}tbody,td,tfoot,th,thead,tr{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}button,input,optgroup,select,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]::-webkit-calendar-picker-indicator{display:none}[type=button],[type=reset],[type=submit],button{-webkit-appearance:button}[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled),button:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + .3vw);line-height:inherit}@media (min-width:1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-text,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{outline-offset:-2px;-webkit-appearance:textfield}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit}::-webkit-file-upload-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none!important}</style>
+ <style>/*! syntax-highlighting.css */pre{line-height:125%;}span.linenos{color:inherit; background-color:transparent; padding-left:5px; padding-right:20px;}.pdoc-code .hll{background-color:#ffffcc}.pdoc-code{background:#f8f8f8;}.pdoc-code .c{color:#3D7B7B; font-style:italic}.pdoc-code .err{border:1px solid #FF0000}.pdoc-code .k{color:#008000; font-weight:bold}.pdoc-code .o{color:#666666}.pdoc-code .ch{color:#3D7B7B; font-style:italic}.pdoc-code .cm{color:#3D7B7B; font-style:italic}.pdoc-code .cp{color:#9C6500}.pdoc-code .cpf{color:#3D7B7B; font-style:italic}.pdoc-code .c1{color:#3D7B7B; font-style:italic}.pdoc-code .cs{color:#3D7B7B; font-style:italic}.pdoc-code .gd{color:#A00000}.pdoc-code .ge{font-style:italic}.pdoc-code .gr{color:#E40000}.pdoc-code .gh{color:#000080; font-weight:bold}.pdoc-code .gi{color:#008400}.pdoc-code .go{color:#717171}.pdoc-code .gp{color:#000080; font-weight:bold}.pdoc-code .gs{font-weight:bold}.pdoc-code .gu{color:#800080; font-weight:bold}.pdoc-code .gt{color:#0044DD}.pdoc-code .kc{color:#008000; font-weight:bold}.pdoc-code .kd{color:#008000; font-weight:bold}.pdoc-code .kn{color:#008000; font-weight:bold}.pdoc-code .kp{color:#008000}.pdoc-code .kr{color:#008000; font-weight:bold}.pdoc-code .kt{color:#B00040}.pdoc-code .m{color:#666666}.pdoc-code .s{color:#BA2121}.pdoc-code .na{color:#687822}.pdoc-code .nb{color:#008000}.pdoc-code .nc{color:#0000FF; font-weight:bold}.pdoc-code .no{color:#880000}.pdoc-code .nd{color:#AA22FF}.pdoc-code .ni{color:#717171; font-weight:bold}.pdoc-code .ne{color:#CB3F38; font-weight:bold}.pdoc-code .nf{color:#0000FF}.pdoc-code .nl{color:#767600}.pdoc-code .nn{color:#0000FF; font-weight:bold}.pdoc-code .nt{color:#008000; font-weight:bold}.pdoc-code .nv{color:#19177C}.pdoc-code .ow{color:#AA22FF; font-weight:bold}.pdoc-code .w{color:#bbbbbb}.pdoc-code .mb{color:#666666}.pdoc-code .mf{color:#666666}.pdoc-code .mh{color:#666666}.pdoc-code .mi{color:#666666}.pdoc-code .mo{color:#666666}.pdoc-code .sa{color:#BA2121}.pdoc-code .sb{color:#BA2121}.pdoc-code .sc{color:#BA2121}.pdoc-code .dl{color:#BA2121}.pdoc-code .sd{color:#BA2121; font-style:italic}.pdoc-code .s2{color:#BA2121}.pdoc-code .se{color:#AA5D1F; font-weight:bold}.pdoc-code .sh{color:#BA2121}.pdoc-code .si{color:#A45A77; font-weight:bold}.pdoc-code .sx{color:#008000}.pdoc-code .sr{color:#A45A77}.pdoc-code .s1{color:#BA2121}.pdoc-code .ss{color:#19177C}.pdoc-code .bp{color:#008000}.pdoc-code .fm{color:#0000FF}.pdoc-code .vc{color:#19177C}.pdoc-code .vg{color:#19177C}.pdoc-code .vi{color:#19177C}.pdoc-code .vm{color:#19177C}.pdoc-code .il{color:#666666}</style>
+ <style>/*! theme.css */:root{--pdoc-background:#fff;}.pdoc{--text:#212529;--muted:#6c757d;--link:#3660a5;--link-hover:#1659c5;--code:#f8f8f8;--active:#fff598;--accent:#eee;--accent2:#c1c1c1;--nav-hover:rgba(255, 255, 255, 0.5);--name:#0066BB;--def:#008800;--annotation:#007020;}</style>
+ <style>/*! layout.css */html, body{width:100%;height:100%;}html, main{scroll-behavior:smooth;}body{background-color:var(--pdoc-background);}@media (max-width:769px){#navtoggle{cursor:pointer;position:absolute;width:50px;height:40px;top:1rem;right:1rem;border-color:var(--text);color:var(--text);display:flex;opacity:0.8;z-index:999;}#navtoggle:hover{opacity:1;}#togglestate + div{display:none;}#togglestate:checked + div{display:inherit;}main, header{padding:2rem 3vw;}header + main{margin-top:-3rem;}.git-button{display:none !important;}nav input[type="search"]{max-width:77%;}nav input[type="search"]:first-child{margin-top:-6px;}nav input[type="search"]:valid ~ *{display:none !important;}}@media (min-width:770px){:root{--sidebar-width:clamp(12.5rem, 28vw, 22rem);}nav{position:fixed;overflow:auto;height:100vh;width:var(--sidebar-width);}main, header{padding:3rem 2rem 3rem calc(var(--sidebar-width) + 3rem);width:calc(54rem + var(--sidebar-width));max-width:100%;}header + main{margin-top:-4rem;}#navtoggle{display:none;}}#togglestate{position:absolute;height:0;opacity:0;}nav.pdoc{--pad:clamp(0.5rem, 2vw, 1.75rem);--indent:1.5rem;background-color:var(--accent);border-right:1px solid var(--accent2);box-shadow:0 0 20px rgba(50, 50, 50, .2) inset;padding:0 0 0 var(--pad);overflow-wrap:anywhere;scrollbar-width:thin; scrollbar-color:var(--accent2) transparent; z-index:1}nav.pdoc::-webkit-scrollbar{width:.4rem; }nav.pdoc::-webkit-scrollbar-thumb{background-color:var(--accent2); }nav.pdoc > div{padding:var(--pad) 0;}nav.pdoc .module-list-button{display:inline-flex;align-items:center;color:var(--text);border-color:var(--muted);margin-bottom:1rem;}nav.pdoc .module-list-button:hover{border-color:var(--text);}nav.pdoc input[type=search]{display:block;outline-offset:0;width:calc(100% - var(--pad));}nav.pdoc .logo{max-width:calc(100% - var(--pad));max-height:35vh;display:block;margin:0 auto 1rem;transform:translate(calc(-.5 * var(--pad)), 0);}nav.pdoc ul{list-style:none;padding-left:0;}nav.pdoc > div > ul{margin-left:calc(0px - var(--pad));}nav.pdoc li a{padding:.2rem 0 .2rem calc(var(--pad) + var(--indent));}nav.pdoc > div > ul > li > a{padding-left:var(--pad);}nav.pdoc li{transition:all 100ms;}nav.pdoc li:hover{background-color:var(--nav-hover);}nav.pdoc a, nav.pdoc a:hover{color:var(--text);}nav.pdoc a{display:block;}nav.pdoc > h2:first-of-type{margin-top:1.5rem;}nav.pdoc .class:before{content:"class ";color:var(--muted);}nav.pdoc .function:after{content:"()";color:var(--muted);}nav.pdoc footer:before{content:"";display:block;width:calc(100% - var(--pad));border-top:solid var(--accent2) 1px;margin-top:1.5rem;padding-top:.5rem;}nav.pdoc footer{font-size:small;}</style>
+ <style>/*! content.css */.pdoc{color:var(--text);box-sizing:border-box;line-height:1.5;background:none;}.pdoc .pdoc-button{cursor:pointer;display:inline-block;border:solid black 1px;border-radius:2px;font-size:.75rem;padding:calc(0.5em - 1px) 1em;transition:100ms all;}.pdoc .alert{padding:1rem 1rem 1rem calc(1.5rem + 24px);border:1px solid transparent;border-radius:.25rem;background-repeat:no-repeat;background-position:.75rem center;margin-bottom:1rem;}.pdoc .alert > em{display:none;}.pdoc .alert > *:last-child{margin-bottom:0;}.pdoc .alert.note {color:#084298;background-color:#cfe2ff;border-color:#b6d4fe;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23084298%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8%2016A8%208%200%201%200%208%200a8%208%200%200%200%200%2016zm.93-9.412-1%204.705c-.07.34.029.533.304.533.194%200%20.487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703%200-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381%202.29-.287zM8%205.5a1%201%200%201%201%200-2%201%201%200%200%201%200%202z%22/%3E%3C/svg%3E");}.pdoc .alert.warning{color:#664d03;background-color:#fff3cd;border-color:#ffecb5;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23664d03%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M8.982%201.566a1.13%201.13%200%200%200-1.96%200L.165%2013.233c-.457.778.091%201.767.98%201.767h13.713c.889%200%201.438-.99.98-1.767L8.982%201.566zM8%205c.535%200%20.954.462.9.995l-.35%203.507a.552.552%200%200%201-1.1%200L7.1%205.995A.905.905%200%200%201%208%205zm.002%206a1%201%200%201%201%200%202%201%201%200%200%201%200-2z%22/%3E%3C/svg%3E");}.pdoc .alert.danger{color:#842029;background-color:#f8d7da;border-color:#f5c2c7;background-image:url("data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20width%3D%2224%22%20height%3D%2224%22%20fill%3D%22%23842029%22%20viewBox%3D%220%200%2016%2016%22%3E%3Cpath%20d%3D%22M5.52.359A.5.5%200%200%201%206%200h4a.5.5%200%200%201%20.474.658L8.694%206H12.5a.5.5%200%200%201%20.395.807l-7%209a.5.5%200%200%201-.873-.454L6.823%209.5H3.5a.5.5%200%200%201-.48-.641l2.5-8.5z%22/%3E%3C/svg%3E");}.pdoc .visually-hidden{position:absolute !important;width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important;}.pdoc h1, .pdoc h2, .pdoc h3{font-weight:300;margin:.3em 0;padding:.2em 0;}.pdoc > section:not(.module-info) h1{font-size:1.5rem;font-weight:500;}.pdoc > section:not(.module-info) h2{font-size:1.4rem;font-weight:500;}.pdoc > section:not(.module-info) h3{font-size:1.3rem;font-weight:500;}.pdoc > section:not(.module-info) h4{font-size:1.2rem;}.pdoc > section:not(.module-info) h5{font-size:1.1rem;}.pdoc a{text-decoration:none;color:var(--link);}.pdoc a:hover{color:var(--link-hover);}.pdoc blockquote{margin-left:2rem;}.pdoc pre{border-top:1px solid var(--accent2);border-bottom:1px solid var(--accent2);margin-top:0;margin-bottom:1em;padding:.5rem 0 .5rem .5rem;overflow-x:auto;background-color:var(--code);}.pdoc code{color:var(--text);padding:.2em .4em;margin:0;font-size:85%;background-color:var(--accent);border-radius:6px;}.pdoc a > code{color:inherit;}.pdoc pre > code{display:inline-block;font-size:inherit;background:none;border:none;padding:0;}.pdoc > section:not(.module-info){margin-bottom:1.5rem;}.pdoc .modulename{margin-top:0;font-weight:bold;}.pdoc .modulename a{color:var(--link);transition:100ms all;}.pdoc .git-button{float:right;border:solid var(--link) 1px;}.pdoc .git-button:hover{background-color:var(--link);color:var(--pdoc-background);}.view-source-toggle-state,.view-source-toggle-state ~ .pdoc-code{display:none;}.view-source-toggle-state:checked ~ .pdoc-code{display:block;}.view-source-button{display:inline-block;float:right;font-size:.75rem;line-height:1.5rem;color:var(--muted);padding:0 .4rem 0 1.3rem;cursor:pointer;text-indent:-2px;}.view-source-button > span{visibility:hidden;}.module-info .view-source-button{float:none;display:flex;justify-content:flex-end;margin:-1.2rem .4rem -.2rem 0;}.view-source-button::before{position:absolute;content:"View Source";display:list-item;list-style-type:disclosure-closed;}.view-source-toggle-state:checked ~ .attr .view-source-button::before,.view-source-toggle-state:checked ~ .view-source-button::before{list-style-type:disclosure-open;}.pdoc .docstring{margin-bottom:1.5rem;}.pdoc section:not(.module-info) .docstring{margin-left:clamp(0rem, 5vw - 2rem, 1rem);}.pdoc .docstring .pdoc-code{margin-left:1em;margin-right:1em;}.pdoc h1:target,.pdoc h2:target,.pdoc h3:target,.pdoc h4:target,.pdoc h5:target,.pdoc h6:target,.pdoc .pdoc-code > pre > span:target{background-color:var(--active);box-shadow:-1rem 0 0 0 var(--active);}.pdoc .pdoc-code > pre > span:target{display:block;}.pdoc div:target > .attr,.pdoc section:target > .attr,.pdoc dd:target > a{background-color:var(--active);}.pdoc *{scroll-margin:2rem;}.pdoc .pdoc-code .linenos{user-select:none;}.pdoc .attr:hover{filter:contrast(0.95);}.pdoc section, .pdoc .classattr{position:relative;}.pdoc .headerlink{--width:clamp(1rem, 3vw, 2rem);position:absolute;top:0;left:calc(0rem - var(--width));transition:all 100ms ease-in-out;opacity:0;}.pdoc .headerlink::before{content:"#";display:block;text-align:center;width:var(--width);height:2.3rem;line-height:2.3rem;font-size:1.5rem;}.pdoc .attr:hover ~ .headerlink,.pdoc *:target > .headerlink,.pdoc .headerlink:hover{opacity:1;}.pdoc .attr{display:block;margin:.5rem 0 .5rem;padding:.4rem .4rem .4rem 1rem;background-color:var(--accent);overflow-x:auto;}.pdoc .classattr{margin-left:2rem;}.pdoc .decorator-deprecated{color:#842029;}.pdoc .decorator-deprecated ~ span{filter:grayscale(1) opacity(0.8);}.pdoc .name{color:var(--name);font-weight:bold;}.pdoc .def{color:var(--def);font-weight:bold;}.pdoc .signature{background-color:transparent;}.pdoc .param, .pdoc .return-annotation{white-space:pre;}.pdoc .signature.multiline .param{display:block;}.pdoc .signature.condensed .param{display:inline-block;}.pdoc .annotation{color:var(--annotation);}.pdoc .view-value-toggle-state,.pdoc .view-value-toggle-state ~ .default_value{display:none;}.pdoc .view-value-toggle-state:checked ~ .default_value{display:inherit;}.pdoc .view-value-button{font-size:.5rem;vertical-align:middle;border-style:dashed;margin-top:-0.1rem;}.pdoc .view-value-button:hover{background:white;}.pdoc .view-value-button::before{content:"show";text-align:center;width:2.2em;display:inline-block;}.pdoc .view-value-toggle-state:checked ~ .view-value-button::before{content:"hide";}.pdoc .inherited{margin-left:2rem;}.pdoc .inherited dt{font-weight:700;}.pdoc .inherited dt, .pdoc .inherited dd{display:inline;margin-left:0;margin-bottom:.5rem;}.pdoc .inherited dd:not(:last-child):after{content:", ";}.pdoc .inherited .class:before{content:"class ";}.pdoc .inherited .function a:after{content:"()";}.pdoc .search-result .docstring{overflow:auto;max-height:25vh;}.pdoc .search-result.focused > .attr{background-color:var(--active);}.pdoc .attribution{margin-top:2rem;display:block;opacity:0.5;transition:all 200ms;filter:grayscale(100%);}.pdoc .attribution:hover{opacity:1;filter:grayscale(0%);}.pdoc .attribution img{margin-left:5px;height:35px;vertical-align:middle;width:70px;transition:all 200ms;}.pdoc table{display:block;width:max-content;max-width:100%;overflow:auto;margin-bottom:1rem;}.pdoc table th{font-weight:600;}.pdoc table th, .pdoc table td{padding:6px 13px;border:1px solid var(--accent2);}</style>
+ <style>/*! custom.css */</style></head>
+<body>
+ <nav class="pdoc">
+ <label id="navtoggle" for="togglestate" class="pdoc-button"><svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'><path stroke-linecap='round' stroke="currentColor" stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/></svg></label>
+ <input id="togglestate" type="checkbox" aria-hidden="true" tabindex="-1">
+ <div> <a class="pdoc-button module-list-button" href="../acmc.html">
+<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-box-arrow-in-left" viewBox="0 0 16 16">
+ <path fill-rule="evenodd" d="M10 3.5a.5.5 0 0 0-.5-.5h-8a.5.5 0 0 0-.5.5v9a.5.5 0 0 0 .5.5h8a.5.5 0 0 0 .5-.5v-2a.5.5 0 0 1 1 0v2A1.5 1.5 0 0 1 9.5 14h-8A1.5 1.5 0 0 1 0 12.5v-9A1.5 1.5 0 0 1 1.5 2h8A1.5 1.5 0 0 1 11 3.5v2a.5.5 0 0 1-1 0v-2z"/>
+ <path fill-rule="evenodd" d="M4.146 8.354a.5.5 0 0 1 0-.708l3-3a.5.5 0 1 1 .708.708L5.707 7.5H14.5a.5.5 0 0 1 0 1H5.707l2.147 2.146a.5.5 0 0 1-.708.708l-3-3z"/>
+</svg> acmc</a>
+
+
+ <input type="search" placeholder="Search..." role="searchbox" aria-label="search"
+ pattern=".+" required>
+
+
+
+ <h2>API Documentation</h2>
+ <ul class="memberlist">
+ <li>
+ <a class="class" href="#QuotedDumper">QuotedDumper</a>
+ <ul class="memberlist">
+ <li>
+ <a class="function" href="#QuotedDumper.increase_indent">increase_indent</a>
+ </li>
+ </ul>
+
+ </li>
+ </ul>
+
+
+
+ <a class="attribution" title="pdoc: Python API documentation generator" href="https://pdoc.dev" target="_blank">
+ built with <span class="visually-hidden">pdoc</span><img
+ alt="pdoc logo"
+ src="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A//www.w3.org/2000/svg%22%20role%3D%22img%22%20aria-label%3D%22pdoc%20logo%22%20width%3D%22300%22%20height%3D%22150%22%20viewBox%3D%22-1%200%2060%2030%22%3E%3Ctitle%3Epdoc%3C/title%3E%3Cpath%20d%3D%22M29.621%2021.293c-.011-.273-.214-.475-.511-.481a.5.5%200%200%200-.489.503l-.044%201.393c-.097.551-.695%201.215-1.566%201.704-.577.428-1.306.486-2.193.182-1.426-.617-2.467-1.654-3.304-2.487l-.173-.172a3.43%203.43%200%200%200-.365-.306.49.49%200%200%200-.286-.196c-1.718-1.06-4.931-1.47-7.353.191l-.219.15c-1.707%201.187-3.413%202.131-4.328%201.03-.02-.027-.49-.685-.141-1.763.233-.721.546-2.408.772-4.076.042-.09.067-.187.046-.288.166-1.347.277-2.625.241-3.351%201.378-1.008%202.271-2.586%202.271-4.362%200-.976-.272-1.935-.788-2.774-.057-.094-.122-.18-.184-.268.033-.167.052-.339.052-.516%200-1.477-1.202-2.679-2.679-2.679-.791%200-1.496.352-1.987.9a6.3%206.3%200%200%200-1.001.029c-.492-.564-1.207-.929-2.012-.929-1.477%200-2.679%201.202-2.679%202.679A2.65%202.65%200%200%200%20.97%206.554c-.383.747-.595%201.572-.595%202.41%200%202.311%201.507%204.29%203.635%205.107-.037.699-.147%202.27-.423%203.294l-.137.461c-.622%202.042-2.515%208.257%201.727%2010.643%201.614.908%203.06%201.248%204.317%201.248%202.665%200%204.492-1.524%205.322-2.401%201.476-1.559%202.886-1.854%206.491.82%201.877%201.393%203.514%201.753%204.861%201.068%202.223-1.713%202.811-3.867%203.399-6.374.077-.846.056-1.469.054-1.537zm-4.835%204.313c-.054.305-.156.586-.242.629-.034-.007-.131-.022-.307-.157-.145-.111-.314-.478-.456-.908.221.121.432.25.675.355.115.039.219.051.33.081zm-2.251-1.238c-.05.33-.158.648-.252.694-.022.001-.125-.018-.307-.157-.217-.166-.488-.906-.639-1.573.358.344.754.693%201.198%201.036zm-3.887-2.337c-.006-.116-.018-.231-.041-.342.635.145%201.189.368%201.599.625.097.231.166.481.174.642-.03.049-.055.101-.067.158-.046.013-.128.026-.298.004-.278-.037-.901-.57-1.367-1.087zm-1.127-.497c.116.306.176.625.12.71-.019.014-.117.045-.345.016-.206-.027-.604-.332-.986-.695.41-.051.816-.056%201.211-.031zm-4.535%201.535c.209.22.379.47.358.598-.006.041-.088.138-.351.234-.144.055-.539-.063-.979-.259a11.66%2011.66%200%200%200%20.972-.573zm.983-.664c.359-.237.738-.418%201.126-.554.25.237.479.548.457.694-.006.042-.087.138-.351.235-.174.064-.694-.105-1.232-.375zm-3.381%201.794c-.022.145-.061.29-.149.401-.133.166-.358.248-.69.251h-.002c-.133%200-.306-.26-.45-.621.417.091.854.07%201.291-.031zm-2.066-8.077a4.78%204.78%200%200%201-.775-.584c.172-.115.505-.254.88-.378l-.105.962zm-.331%202.302a10.32%2010.32%200%200%201-.828-.502c.202-.143.576-.328.984-.49l-.156.992zm-.45%202.157l-.701-.403c.214-.115.536-.249.891-.376a11.57%2011.57%200%200%201-.19.779zm-.181%201.716c.064.398.194.702.298.893-.194-.051-.435-.162-.736-.398.061-.119.224-.3.438-.495zM8.87%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zm-.735-.389a1.15%201.15%200%200%200-.314.783%201.16%201.16%200%200%200%201.162%201.162c.457%200%20.842-.27%201.032-.653.026.117.042.238.042.362a1.68%201.68%200%200%201-1.679%201.679%201.68%201.68%200%200%201-1.679-1.679c0-.843.626-1.535%201.436-1.654zM5.059%205.406A1.68%201.68%200%200%201%203.38%207.085a1.68%201.68%200%200%201-1.679-1.679c0-.037.009-.072.011-.109.21.3.541.508.935.508a1.16%201.16%200%200%200%201.162-1.162%201.14%201.14%200%200%200-.474-.912c.015%200%20.03-.005.045-.005.926.001%201.679.754%201.679%201.68zM3.198%204.141c0%20.152-.123.276-.276.276s-.275-.124-.275-.276.123-.276.276-.276.275.124.275.276zM1.375%208.964c0-.52.103-1.035.288-1.52.466.394%201.06.64%201.717.64%201.144%200%202.116-.725%202.499-1.738.383%201.012%201.355%201.738%202.499%201.738.867%200%201.631-.421%202.121-1.062.307.605.478%201.267.478%201.942%200%202.486-2.153%204.51-4.801%204.51s-4.801-2.023-4.801-4.51zm24.342%2019.349c-.985.498-2.267.168-3.813-.979-3.073-2.281-5.453-3.199-7.813-.705-1.315%201.391-4.163%203.365-8.423.97-3.174-1.786-2.239-6.266-1.261-9.479l.146-.492c.276-1.02.395-2.457.444-3.268a6.11%206.11%200%200%200%201.18.115%206.01%206.01%200%200%200%202.536-.562l-.006.175c-.802.215-1.848.612-2.021%201.25-.079.295.021.601.274.837.219.203.415.364.598.501-.667.304-1.243.698-1.311%201.179-.02.144-.022.507.393.787.213.144.395.26.564.365-1.285.521-1.361.96-1.381%201.126-.018.142-.011.496.427.746l.854.489c-.473.389-.971.914-.999%201.429-.018.278.095.532.316.713.675.556%201.231.721%201.653.721.059%200%20.104-.014.158-.02.207.707.641%201.64%201.513%201.64h.013c.8-.008%201.236-.345%201.462-.626.173-.216.268-.457.325-.692.424.195.93.374%201.372.374.151%200%20.294-.021.423-.068.732-.27.944-.704.993-1.021.009-.061.003-.119.002-.179.266.086.538.147.789.147.15%200%20.294-.021.423-.069.542-.2.797-.489.914-.754.237.147.478.258.704.288.106.014.205.021.296.021.356%200%20.595-.101.767-.229.438.435%201.094.992%201.656%201.067.106.014.205.021.296.021a1.56%201.56%200%200%200%20.323-.035c.17.575.453%201.289.866%201.605.358.273.665.362.914.362a.99.99%200%200%200%20.421-.093%201.03%201.03%200%200%200%20.245-.164c.168.428.39.846.68%201.068.358.273.665.362.913.362a.99.99%200%200%200%20.421-.093c.317-.148.512-.448.639-.762.251.157.495.257.726.257.127%200%20.25-.024.37-.071.427-.17.706-.617.841-1.314.022-.015.047-.022.068-.038.067-.051.133-.104.196-.159-.443%201.486-1.107%202.761-2.086%203.257zM8.66%209.925a.5.5%200%201%200-1%200c0%20.653-.818%201.205-1.787%201.205s-1.787-.552-1.787-1.205a.5.5%200%201%200-1%200c0%201.216%201.25%202.205%202.787%202.205s2.787-.989%202.787-2.205zm4.4%2015.965l-.208.097c-2.661%201.258-4.708%201.436-6.086.527-1.542-1.017-1.88-3.19-1.844-4.198a.4.4%200%200%200-.385-.414c-.242-.029-.406.164-.414.385-.046%201.249.367%203.686%202.202%204.896.708.467%201.547.7%202.51.7%201.248%200%202.706-.392%204.362-1.174l.185-.086a.4.4%200%200%200%20.205-.527c-.089-.204-.326-.291-.527-.206zM9.547%202.292c.093.077.205.114.317.114a.5.5%200%200%200%20.318-.886L8.817.397a.5.5%200%200%200-.703.068.5.5%200%200%200%20.069.703l1.364%201.124zm-7.661-.065c.086%200%20.173-.022.253-.068l1.523-.893a.5.5%200%200%200-.506-.863l-1.523.892a.5.5%200%200%200-.179.685c.094.158.261.247.432.247z%22%20transform%3D%22matrix%28-1%200%200%201%2058%200%29%22%20fill%3D%22%233bb300%22/%3E%3Cpath%20d%3D%22M.3%2021.86V10.18q0-.46.02-.68.04-.22.18-.5.28-.54%201.34-.54%201.06%200%201.42.28.38.26.44.78.76-1.04%202.38-1.04%201.64%200%203.1%201.54%201.46%201.54%201.46%203.58%200%202.04-1.46%203.58-1.44%201.54-3.08%201.54-1.64%200-2.38-.92v4.04q0%20.46-.04.68-.02.22-.18.5-.14.3-.5.42-.36.12-.98.12-.62%200-1-.12-.36-.12-.52-.4-.14-.28-.18-.5-.02-.22-.02-.68zm3.96-9.42q-.46.54-.46%201.18%200%20.64.46%201.18.48.52%201.2.52.74%200%201.24-.52.52-.52.52-1.18%200-.66-.48-1.18-.48-.54-1.26-.54-.76%200-1.22.54zm14.741-8.36q.16-.3.54-.42.38-.12%201-.12.64%200%201.02.12.38.12.52.42.16.3.18.54.04.22.04.68v11.94q0%20.46-.04.7-.02.22-.18.5-.3.54-1.7.54-1.38%200-1.54-.98-.84.96-2.34.96-1.8%200-3.28-1.56-1.48-1.58-1.48-3.66%200-2.1%201.48-3.68%201.5-1.58%203.28-1.58%201.48%200%202.3%201v-4.2q0-.46.02-.68.04-.24.18-.52zm-3.24%2010.86q.52.54%201.26.54.74%200%201.22-.54.5-.54.5-1.18%200-.66-.48-1.22-.46-.56-1.26-.56-.8%200-1.28.56-.48.54-.48%201.2%200%20.66.52%201.2zm7.833-1.2q0-2.4%201.68-3.96%201.68-1.56%203.84-1.56%202.16%200%203.82%201.56%201.66%201.54%201.66%203.94%200%201.66-.86%202.96-.86%201.28-2.1%201.9-1.22.6-2.54.6-1.32%200-2.56-.64-1.24-.66-2.1-1.92-.84-1.28-.84-2.88zm4.18%201.44q.64.48%201.3.48.66%200%201.32-.5.66-.5.66-1.48%200-.98-.62-1.46-.62-.48-1.34-.48-.72%200-1.34.5-.62.5-.62%201.48%200%20.96.64%201.46zm11.412-1.44q0%20.84.56%201.32.56.46%201.18.46.64%200%201.18-.36.56-.38.9-.38.6%200%201.46%201.06.46.58.46%201.04%200%20.76-1.1%201.42-1.14.8-2.8.8-1.86%200-3.58-1.34-.82-.64-1.34-1.7-.52-1.08-.52-2.36%200-1.3.52-2.34.52-1.06%201.34-1.7%201.66-1.32%203.54-1.32.76%200%201.48.22.72.2%201.06.4l.32.2q.36.24.56.38.52.4.52.92%200%20.5-.42%201.14-.72%201.1-1.38%201.1-.38%200-1.08-.44-.36-.34-1.04-.34-.66%200-1.24.48-.58.48-.58%201.34z%22%20fill%3D%22green%22/%3E%3C/svg%3E"/>
+ </a>
+</div>
+ </nav>
+ <main class="pdoc">
+ <section class="module-info">
+ <h1 class="modulename">
+<a href="./../acmc.html">acmc</a><wbr>.util </h1>
+
+
+ <input id="mod-util-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+
+ <label class="view-source-button" for="mod-util-view-source"><span>View Source</span></label>
+
+ <div class="pdoc-code codehilite"><pre><span></span><span id="L-1"><a href="#L-1"><span class="linenos"> 1</span></a><span class="kn">import</span><span class="w"> </span><span class="nn">yaml</span>
+</span><span id="L-2"><a href="#L-2"><span class="linenos"> 2</span></a>
+</span><span id="L-3"><a href="#L-3"><span class="linenos"> 3</span></a><span class="sd">"""</span>
+</span><span id="L-4"><a href="#L-4"><span class="linenos"> 4</span></a><span class="sd">util.py module</span>
+</span><span id="L-5"><a href="#L-5"><span class="linenos"> 5</span></a>
+</span><span id="L-6"><a href="#L-6"><span class="linenos"> 6</span></a><span class="sd">This module provides general utility functions for the acmc modules</span>
+</span><span id="L-7"><a href="#L-7"><span class="linenos"> 7</span></a>
+</span><span id="L-8"><a href="#L-8"><span class="linenos"> 8</span></a><span class="sd">"""</span>
+</span><span id="L-9"><a href="#L-9"><span class="linenos"> 9</span></a>
+</span><span id="L-10"><a href="#L-10"><span class="linenos">10</span></a>
+</span><span id="L-11"><a href="#L-11"><span class="linenos">11</span></a><span class="k">class</span><span class="w"> </span><span class="nc">QuotedDumper</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">Dumper</span><span class="p">):</span>
+</span><span id="L-12"><a href="#L-12"><span class="linenos">12</span></a><span class="w"> </span><span class="sd">"""Custom Dumper to retain quotes on strings in yaml library"""</span>
+</span><span id="L-13"><a href="#L-13"><span class="linenos">13</span></a>
+</span><span id="L-14"><a href="#L-14"><span class="linenos">14</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">increase_indent</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">flow</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">indentless</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
+</span><span id="L-15"><a href="#L-15"><span class="linenos">15</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">(</span><span class="n">QuotedDumper</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">increase_indent</span><span class="p">(</span><span class="n">flow</span><span class="p">,</span> <span class="n">indentless</span><span class="p">)</span>
+</span></pre></div>
+
+
+ </section>
+ <section id="QuotedDumper">
+ <input id="QuotedDumper-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr class">
+
+ <span class="def">class</span>
+ <span class="name">QuotedDumper</span><wbr>(<span class="base">yaml.dumper.Dumper</span>):
+
+ <label class="view-source-button" for="QuotedDumper-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#QuotedDumper"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="QuotedDumper-12"><a href="#QuotedDumper-12"><span class="linenos">12</span></a><span class="k">class</span><span class="w"> </span><span class="nc">QuotedDumper</span><span class="p">(</span><span class="n">yaml</span><span class="o">.</span><span class="n">Dumper</span><span class="p">):</span>
+</span><span id="QuotedDumper-13"><a href="#QuotedDumper-13"><span class="linenos">13</span></a><span class="w"> </span><span class="sd">"""Custom Dumper to retain quotes on strings in yaml library"""</span>
+</span><span id="QuotedDumper-14"><a href="#QuotedDumper-14"><span class="linenos">14</span></a>
+</span><span id="QuotedDumper-15"><a href="#QuotedDumper-15"><span class="linenos">15</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">increase_indent</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">flow</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">indentless</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
+</span><span id="QuotedDumper-16"><a href="#QuotedDumper-16"><span class="linenos">16</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">(</span><span class="n">QuotedDumper</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">increase_indent</span><span class="p">(</span><span class="n">flow</span><span class="p">,</span> <span class="n">indentless</span><span class="p">)</span>
+</span></pre></div>
+
+
+ <div class="docstring"><p>Custom Dumper to retain quotes on strings in yaml library</p>
+</div>
+
+
+ <div id="QuotedDumper.increase_indent" class="classattr">
+ <input id="QuotedDumper.increase_indent-view-source" class="view-source-toggle-state" type="checkbox" aria-hidden="true" tabindex="-1">
+<div class="attr function">
+
+ <span class="def">def</span>
+ <span class="name">increase_indent</span><span class="signature pdoc-code condensed">(<span class="param"><span class="bp">self</span>, </span><span class="param"><span class="n">flow</span><span class="o">=</span><span class="kc">False</span>, </span><span class="param"><span class="n">indentless</span><span class="o">=</span><span class="kc">False</span></span><span class="return-annotation">):</span></span>
+
+ <label class="view-source-button" for="QuotedDumper.increase_indent-view-source"><span>View Source</span></label>
+
+ </div>
+ <a class="headerlink" href="#QuotedDumper.increase_indent"></a>
+ <div class="pdoc-code codehilite"><pre><span></span><span id="QuotedDumper.increase_indent-15"><a href="#QuotedDumper.increase_indent-15"><span class="linenos">15</span></a> <span class="k">def</span><span class="w"> </span><span class="nf">increase_indent</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">flow</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">indentless</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
+</span><span id="QuotedDumper.increase_indent-16"><a href="#QuotedDumper.increase_indent-16"><span class="linenos">16</span></a> <span class="k">return</span> <span class="nb">super</span><span class="p">(</span><span class="n">QuotedDumper</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">increase_indent</span><span class="p">(</span><span class="n">flow</span><span class="p">,</span> <span class="n">indentless</span><span class="p">)</span>
+</span></pre></div>
+
+
+
+
+ </div>
+ </section>
+ </main>
+<script>
+ function escapeHTML(html) {
+ return document.createElement('div').appendChild(document.createTextNode(html)).parentNode.innerHTML;
+ }
+
+ const originalContent = document.querySelector("main.pdoc");
+ let currentContent = originalContent;
+
+ function setContent(innerHTML) {
+ let elem;
+ if (innerHTML) {
+ elem = document.createElement("main");
+ elem.classList.add("pdoc");
+ elem.innerHTML = innerHTML;
+ } else {
+ elem = originalContent;
+ }
+ if (currentContent !== elem) {
+ currentContent.replaceWith(elem);
+ currentContent = elem;
+ }
+ }
+
+ function getSearchTerm() {
+ return (new URL(window.location)).searchParams.get("search");
+ }
+
+ const searchBox = document.querySelector(".pdoc input[type=search]");
+ searchBox.addEventListener("input", function () {
+ let url = new URL(window.location);
+ if (searchBox.value.trim()) {
+ url.hash = "";
+ url.searchParams.set("search", searchBox.value);
+ } else {
+ url.searchParams.delete("search");
+ }
+ history.replaceState("", "", url.toString());
+ onInput();
+ });
+ window.addEventListener("popstate", onInput);
+
+
+ let search, searchErr;
+
+ async function initialize() {
+ try {
+ search = await new Promise((resolve, reject) => {
+ const script = document.createElement("script");
+ script.type = "text/javascript";
+ script.async = true;
+ script.onload = () => resolve(window.pdocSearch);
+ script.onerror = (e) => reject(e);
+ script.src = "../search.js";
+ document.getElementsByTagName("head")[0].appendChild(script);
+ });
+ } catch (e) {
+ console.error("Cannot fetch pdoc search index");
+ searchErr = "Cannot fetch search index.";
+ }
+ onInput();
+
+ document.querySelector("nav.pdoc").addEventListener("click", e => {
+ if (e.target.hash) {
+ searchBox.value = "";
+ searchBox.dispatchEvent(new Event("input"));
+ }
+ });
+ }
+
+ function onInput() {
+ setContent((() => {
+ const term = getSearchTerm();
+ if (!term) {
+ return null
+ }
+ if (searchErr) {
+ return `<h3>Error: ${searchErr}</h3>`
+ }
+ if (!search) {
+ return "<h3>Searching...</h3>"
+ }
+
+ window.scrollTo({top: 0, left: 0, behavior: 'auto'});
+
+ const results = search(term);
+
+ let html;
+ if (results.length === 0) {
+ html = `No search results for '${escapeHTML(term)}'.`
+ } else {
+ html = `<h4>${results.length} search result${results.length > 1 ? "s" : ""} for '${escapeHTML(term)}'.</h4>`;
+ }
+ for (let result of results.slice(0, 10)) {
+ let doc = result.doc;
+ let url = `../${doc.modulename.replaceAll(".", "/")}.html`;
+ if (doc.qualname) {
+ url += `#${doc.qualname}`;
+ }
+
+ let heading;
+ switch (result.doc.kind) {
+ case "function":
+ if (doc.fullname.endsWith(".__init__")) {
+ heading = `<span class="name">${doc.fullname.replace(/\.__init__$/, "")}</span>${doc.signature}`;
+ } else {
+ heading = `<span class="def">${doc.funcdef}</span> <span class="name">${doc.fullname}</span>${doc.signature}`;
+ }
+ break;
+ case "class":
+ heading = `<span class="def">class</span> <span class="name">${doc.fullname}</span>`;
+ if (doc.bases)
+ heading += `<wbr>(<span class="base">${doc.bases}</span>)`;
+ heading += `:`;
+ break;
+ case "variable":
+ heading = `<span class="name">${doc.fullname}</span>`;
+ if (doc.annotation)
+ heading += `<span class="annotation">${doc.annotation}</span>`;
+ if (doc.default_value)
+ heading += `<span class="default_value"> = ${doc.default_value}</span>`;
+ break;
+ default:
+ heading = `<span class="name">${doc.fullname}</span>`;
+ break;
+ }
+ html += `
+ <section class="search-result">
+ <a href="${url}" class="attr ${doc.kind}">${heading}</a>
+ <div class="docstring">${doc.doc}</div>
+ </section>
+ `;
+
+ }
+ return html;
+ })());
+ }
+
+ if (getSearchTerm()) {
+ initialize();
+ searchBox.value = getSearchTerm();
+ onInput();
+ } else {
+ searchBox.addEventListener("focus", initialize, {once: true});
+ }
+
+ searchBox.addEventListener("keydown", e => {
+ if (["ArrowDown", "ArrowUp", "Enter"].includes(e.key)) {
+ let focused = currentContent.querySelector(".search-result.focused");
+ if (!focused) {
+ currentContent.querySelector(".search-result").classList.add("focused");
+ } else if (
+ e.key === "ArrowDown"
+ && focused.nextElementSibling
+ && focused.nextElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.nextElementSibling.classList.add("focused");
+ focused.nextElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "ArrowUp"
+ && focused.previousElementSibling
+ && focused.previousElementSibling.classList.contains("search-result")
+ ) {
+ focused.classList.remove("focused");
+ focused.previousElementSibling.classList.add("focused");
+ focused.previousElementSibling.scrollIntoView({
+ behavior: "smooth",
+ block: "nearest",
+ inline: "nearest"
+ });
+ } else if (
+ e.key === "Enter"
+ ) {
+ focused.querySelector("a").click();
+ }
+ }
+ });
+</script></body>
+</html>
\ No newline at end of file
diff --git a/docs/api/index.html b/docs/api/index.html
new file mode 100644
index 0000000000000000000000000000000000000000..566bd088aee2d892b13758267a4a298b7b6da730
--- /dev/null
+++ b/docs/api/index.html
@@ -0,0 +1,7 @@
+<!doctype html>
+<html>
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="refresh" content="0; url=./acmc.html"/>
+</head>
+</html>
diff --git a/docs/api/search.js b/docs/api/search.js
new file mode 100644
index 0000000000000000000000000000000000000000..65efab673fa4c5e0141a62ba736a5deb0fbb8f58
--- /dev/null
+++ b/docs/api/search.js
@@ -0,0 +1,46 @@
+window.pdocSearch = (function(){
+/** elasticlunr - http://weixsong.github.io * Copyright (C) 2017 Oliver Nightingale * Copyright (C) 2017 Wei Song * MIT Licensed */!function(){function e(e){if(null===e||"object"!=typeof e)return e;var t=e.constructor();for(var n in e)e.hasOwnProperty(n)&&(t[n]=e[n]);return t}var t=function(e){var n=new t.Index;return n.pipeline.add(t.trimmer,t.stopWordFilter,t.stemmer),e&&e.call(n,n),n};t.version="0.9.5",lunr=t,t.utils={},t.utils.warn=function(e){return function(t){e.console&&console.warn&&console.warn(t)}}(this),t.utils.toString=function(e){return void 0===e||null===e?"":e.toString()},t.EventEmitter=function(){this.events={}},t.EventEmitter.prototype.addListener=function(){var e=Array.prototype.slice.call(arguments),t=e.pop(),n=e;if("function"!=typeof t)throw new TypeError("last argument must be a function");n.forEach(function(e){this.hasHandler(e)||(this.events[e]=[]),this.events[e].push(t)},this)},t.EventEmitter.prototype.removeListener=function(e,t){if(this.hasHandler(e)){var n=this.events[e].indexOf(t);-1!==n&&(this.events[e].splice(n,1),0==this.events[e].length&&delete this.events[e])}},t.EventEmitter.prototype.emit=function(e){if(this.hasHandler(e)){var t=Array.prototype.slice.call(arguments,1);this.events[e].forEach(function(e){e.apply(void 0,t)},this)}},t.EventEmitter.prototype.hasHandler=function(e){return e in this.events},t.tokenizer=function(e){if(!arguments.length||null===e||void 0===e)return[];if(Array.isArray(e)){var n=e.filter(function(e){return null===e||void 0===e?!1:!0});n=n.map(function(e){return t.utils.toString(e).toLowerCase()});var i=[];return n.forEach(function(e){var n=e.split(t.tokenizer.seperator);i=i.concat(n)},this),i}return e.toString().trim().toLowerCase().split(t.tokenizer.seperator)},t.tokenizer.defaultSeperator=/[\s\-]+/,t.tokenizer.seperator=t.tokenizer.defaultSeperator,t.tokenizer.setSeperator=function(e){null!==e&&void 0!==e&&"object"==typeof e&&(t.tokenizer.seperator=e)},t.tokenizer.resetSeperator=function(){t.tokenizer.seperator=t.tokenizer.defaultSeperator},t.tokenizer.getSeperator=function(){return t.tokenizer.seperator},t.Pipeline=function(){this._queue=[]},t.Pipeline.registeredFunctions={},t.Pipeline.registerFunction=function(e,n){n in t.Pipeline.registeredFunctions&&t.utils.warn("Overwriting existing registered function: "+n),e.label=n,t.Pipeline.registeredFunctions[n]=e},t.Pipeline.getRegisteredFunction=function(e){return e in t.Pipeline.registeredFunctions!=!0?null:t.Pipeline.registeredFunctions[e]},t.Pipeline.warnIfFunctionNotRegistered=function(e){var n=e.label&&e.label in this.registeredFunctions;n||t.utils.warn("Function is not registered with pipeline. This may cause problems when serialising the index.\n",e)},t.Pipeline.load=function(e){var n=new t.Pipeline;return e.forEach(function(e){var i=t.Pipeline.getRegisteredFunction(e);if(!i)throw new Error("Cannot load un-registered function: "+e);n.add(i)}),n},t.Pipeline.prototype.add=function(){var e=Array.prototype.slice.call(arguments);e.forEach(function(e){t.Pipeline.warnIfFunctionNotRegistered(e),this._queue.push(e)},this)},t.Pipeline.prototype.after=function(e,n){t.Pipeline.warnIfFunctionNotRegistered(n);var i=this._queue.indexOf(e);if(-1===i)throw new Error("Cannot find existingFn");this._queue.splice(i+1,0,n)},t.Pipeline.prototype.before=function(e,n){t.Pipeline.warnIfFunctionNotRegistered(n);var i=this._queue.indexOf(e);if(-1===i)throw new Error("Cannot find existingFn");this._queue.splice(i,0,n)},t.Pipeline.prototype.remove=function(e){var t=this._queue.indexOf(e);-1!==t&&this._queue.splice(t,1)},t.Pipeline.prototype.run=function(e){for(var t=[],n=e.length,i=this._queue.length,o=0;n>o;o++){for(var r=e[o],s=0;i>s&&(r=this._queue[s](r,o,e),void 0!==r&&null!==r);s++);void 0!==r&&null!==r&&t.push(r)}return t},t.Pipeline.prototype.reset=function(){this._queue=[]},t.Pipeline.prototype.get=function(){return this._queue},t.Pipeline.prototype.toJSON=function(){return this._queue.map(function(e){return t.Pipeline.warnIfFunctionNotRegistered(e),e.label})},t.Index=function(){this._fields=[],this._ref="id",this.pipeline=new t.Pipeline,this.documentStore=new t.DocumentStore,this.index={},this.eventEmitter=new t.EventEmitter,this._idfCache={},this.on("add","remove","update",function(){this._idfCache={}}.bind(this))},t.Index.prototype.on=function(){var e=Array.prototype.slice.call(arguments);return this.eventEmitter.addListener.apply(this.eventEmitter,e)},t.Index.prototype.off=function(e,t){return this.eventEmitter.removeListener(e,t)},t.Index.load=function(e){e.version!==t.version&&t.utils.warn("version mismatch: current "+t.version+" importing "+e.version);var n=new this;n._fields=e.fields,n._ref=e.ref,n.documentStore=t.DocumentStore.load(e.documentStore),n.pipeline=t.Pipeline.load(e.pipeline),n.index={};for(var i in e.index)n.index[i]=t.InvertedIndex.load(e.index[i]);return n},t.Index.prototype.addField=function(e){return this._fields.push(e),this.index[e]=new t.InvertedIndex,this},t.Index.prototype.setRef=function(e){return this._ref=e,this},t.Index.prototype.saveDocument=function(e){return this.documentStore=new t.DocumentStore(e),this},t.Index.prototype.addDoc=function(e,n){if(e){var n=void 0===n?!0:n,i=e[this._ref];this.documentStore.addDoc(i,e),this._fields.forEach(function(n){var o=this.pipeline.run(t.tokenizer(e[n]));this.documentStore.addFieldLength(i,n,o.length);var r={};o.forEach(function(e){e in r?r[e]+=1:r[e]=1},this);for(var s in r){var u=r[s];u=Math.sqrt(u),this.index[n].addToken(s,{ref:i,tf:u})}},this),n&&this.eventEmitter.emit("add",e,this)}},t.Index.prototype.removeDocByRef=function(e){if(e&&this.documentStore.isDocStored()!==!1&&this.documentStore.hasDoc(e)){var t=this.documentStore.getDoc(e);this.removeDoc(t,!1)}},t.Index.prototype.removeDoc=function(e,n){if(e){var n=void 0===n?!0:n,i=e[this._ref];this.documentStore.hasDoc(i)&&(this.documentStore.removeDoc(i),this._fields.forEach(function(n){var o=this.pipeline.run(t.tokenizer(e[n]));o.forEach(function(e){this.index[n].removeToken(e,i)},this)},this),n&&this.eventEmitter.emit("remove",e,this))}},t.Index.prototype.updateDoc=function(e,t){var t=void 0===t?!0:t;this.removeDocByRef(e[this._ref],!1),this.addDoc(e,!1),t&&this.eventEmitter.emit("update",e,this)},t.Index.prototype.idf=function(e,t){var n="@"+t+"/"+e;if(Object.prototype.hasOwnProperty.call(this._idfCache,n))return this._idfCache[n];var i=this.index[t].getDocFreq(e),o=1+Math.log(this.documentStore.length/(i+1));return this._idfCache[n]=o,o},t.Index.prototype.getFields=function(){return this._fields.slice()},t.Index.prototype.search=function(e,n){if(!e)return[];e="string"==typeof e?{any:e}:JSON.parse(JSON.stringify(e));var i=null;null!=n&&(i=JSON.stringify(n));for(var o=new t.Configuration(i,this.getFields()).get(),r={},s=Object.keys(e),u=0;u<s.length;u++){var a=s[u];r[a]=this.pipeline.run(t.tokenizer(e[a]))}var l={};for(var c in o){var d=r[c]||r.any;if(d){var f=this.fieldSearch(d,c,o),h=o[c].boost;for(var p in f)f[p]=f[p]*h;for(var p in f)p in l?l[p]+=f[p]:l[p]=f[p]}}var v,g=[];for(var p in l)v={ref:p,score:l[p]},this.documentStore.hasDoc(p)&&(v.doc=this.documentStore.getDoc(p)),g.push(v);return g.sort(function(e,t){return t.score-e.score}),g},t.Index.prototype.fieldSearch=function(e,t,n){var i=n[t].bool,o=n[t].expand,r=n[t].boost,s=null,u={};return 0!==r?(e.forEach(function(e){var n=[e];1==o&&(n=this.index[t].expandToken(e));var r={};n.forEach(function(n){var o=this.index[t].getDocs(n),a=this.idf(n,t);if(s&&"AND"==i){var l={};for(var c in s)c in o&&(l[c]=o[c]);o=l}n==e&&this.fieldSearchStats(u,n,o);for(var c in o){var d=this.index[t].getTermFrequency(n,c),f=this.documentStore.getFieldLength(c,t),h=1;0!=f&&(h=1/Math.sqrt(f));var p=1;n!=e&&(p=.15*(1-(n.length-e.length)/n.length));var v=d*a*h*p;c in r?r[c]+=v:r[c]=v}},this),s=this.mergeScores(s,r,i)},this),s=this.coordNorm(s,u,e.length)):void 0},t.Index.prototype.mergeScores=function(e,t,n){if(!e)return t;if("AND"==n){var i={};for(var o in t)o in e&&(i[o]=e[o]+t[o]);return i}for(var o in t)o in e?e[o]+=t[o]:e[o]=t[o];return e},t.Index.prototype.fieldSearchStats=function(e,t,n){for(var i in n)i in e?e[i].push(t):e[i]=[t]},t.Index.prototype.coordNorm=function(e,t,n){for(var i in e)if(i in t){var o=t[i].length;e[i]=e[i]*o/n}return e},t.Index.prototype.toJSON=function(){var e={};return this._fields.forEach(function(t){e[t]=this.index[t].toJSON()},this),{version:t.version,fields:this._fields,ref:this._ref,documentStore:this.documentStore.toJSON(),index:e,pipeline:this.pipeline.toJSON()}},t.Index.prototype.use=function(e){var t=Array.prototype.slice.call(arguments,1);t.unshift(this),e.apply(this,t)},t.DocumentStore=function(e){this._save=null===e||void 0===e?!0:e,this.docs={},this.docInfo={},this.length=0},t.DocumentStore.load=function(e){var t=new this;return t.length=e.length,t.docs=e.docs,t.docInfo=e.docInfo,t._save=e.save,t},t.DocumentStore.prototype.isDocStored=function(){return this._save},t.DocumentStore.prototype.addDoc=function(t,n){this.hasDoc(t)||this.length++,this.docs[t]=this._save===!0?e(n):null},t.DocumentStore.prototype.getDoc=function(e){return this.hasDoc(e)===!1?null:this.docs[e]},t.DocumentStore.prototype.hasDoc=function(e){return e in this.docs},t.DocumentStore.prototype.removeDoc=function(e){this.hasDoc(e)&&(delete this.docs[e],delete this.docInfo[e],this.length--)},t.DocumentStore.prototype.addFieldLength=function(e,t,n){null!==e&&void 0!==e&&0!=this.hasDoc(e)&&(this.docInfo[e]||(this.docInfo[e]={}),this.docInfo[e][t]=n)},t.DocumentStore.prototype.updateFieldLength=function(e,t,n){null!==e&&void 0!==e&&0!=this.hasDoc(e)&&this.addFieldLength(e,t,n)},t.DocumentStore.prototype.getFieldLength=function(e,t){return null===e||void 0===e?0:e in this.docs&&t in this.docInfo[e]?this.docInfo[e][t]:0},t.DocumentStore.prototype.toJSON=function(){return{docs:this.docs,docInfo:this.docInfo,length:this.length,save:this._save}},t.stemmer=function(){var e={ational:"ate",tional:"tion",enci:"ence",anci:"ance",izer:"ize",bli:"ble",alli:"al",entli:"ent",eli:"e",ousli:"ous",ization:"ize",ation:"ate",ator:"ate",alism:"al",iveness:"ive",fulness:"ful",ousness:"ous",aliti:"al",iviti:"ive",biliti:"ble",logi:"log"},t={icate:"ic",ative:"",alize:"al",iciti:"ic",ical:"ic",ful:"",ness:""},n="[^aeiou]",i="[aeiouy]",o=n+"[^aeiouy]*",r=i+"[aeiou]*",s="^("+o+")?"+r+o,u="^("+o+")?"+r+o+"("+r+")?$",a="^("+o+")?"+r+o+r+o,l="^("+o+")?"+i,c=new RegExp(s),d=new RegExp(a),f=new RegExp(u),h=new RegExp(l),p=/^(.+?)(ss|i)es$/,v=/^(.+?)([^s])s$/,g=/^(.+?)eed$/,m=/^(.+?)(ed|ing)$/,y=/.$/,S=/(at|bl|iz)$/,x=new RegExp("([^aeiouylsz])\\1$"),w=new RegExp("^"+o+i+"[^aeiouwxy]$"),I=/^(.+?[^aeiou])y$/,b=/^(.+?)(ational|tional|enci|anci|izer|bli|alli|entli|eli|ousli|ization|ation|ator|alism|iveness|fulness|ousness|aliti|iviti|biliti|logi)$/,E=/^(.+?)(icate|ative|alize|iciti|ical|ful|ness)$/,D=/^(.+?)(al|ance|ence|er|ic|able|ible|ant|ement|ment|ent|ou|ism|ate|iti|ous|ive|ize)$/,F=/^(.+?)(s|t)(ion)$/,_=/^(.+?)e$/,P=/ll$/,k=new RegExp("^"+o+i+"[^aeiouwxy]$"),z=function(n){var i,o,r,s,u,a,l;if(n.length<3)return n;if(r=n.substr(0,1),"y"==r&&(n=r.toUpperCase()+n.substr(1)),s=p,u=v,s.test(n)?n=n.replace(s,"$1$2"):u.test(n)&&(n=n.replace(u,"$1$2")),s=g,u=m,s.test(n)){var z=s.exec(n);s=c,s.test(z[1])&&(s=y,n=n.replace(s,""))}else if(u.test(n)){var z=u.exec(n);i=z[1],u=h,u.test(i)&&(n=i,u=S,a=x,l=w,u.test(n)?n+="e":a.test(n)?(s=y,n=n.replace(s,"")):l.test(n)&&(n+="e"))}if(s=I,s.test(n)){var z=s.exec(n);i=z[1],n=i+"i"}if(s=b,s.test(n)){var z=s.exec(n);i=z[1],o=z[2],s=c,s.test(i)&&(n=i+e[o])}if(s=E,s.test(n)){var z=s.exec(n);i=z[1],o=z[2],s=c,s.test(i)&&(n=i+t[o])}if(s=D,u=F,s.test(n)){var z=s.exec(n);i=z[1],s=d,s.test(i)&&(n=i)}else if(u.test(n)){var z=u.exec(n);i=z[1]+z[2],u=d,u.test(i)&&(n=i)}if(s=_,s.test(n)){var z=s.exec(n);i=z[1],s=d,u=f,a=k,(s.test(i)||u.test(i)&&!a.test(i))&&(n=i)}return s=P,u=d,s.test(n)&&u.test(n)&&(s=y,n=n.replace(s,"")),"y"==r&&(n=r.toLowerCase()+n.substr(1)),n};return z}(),t.Pipeline.registerFunction(t.stemmer,"stemmer"),t.stopWordFilter=function(e){return e&&t.stopWordFilter.stopWords[e]!==!0?e:void 0},t.clearStopWords=function(){t.stopWordFilter.stopWords={}},t.addStopWords=function(e){null!=e&&Array.isArray(e)!==!1&&e.forEach(function(e){t.stopWordFilter.stopWords[e]=!0},this)},t.resetStopWords=function(){t.stopWordFilter.stopWords=t.defaultStopWords},t.defaultStopWords={"":!0,a:!0,able:!0,about:!0,across:!0,after:!0,all:!0,almost:!0,also:!0,am:!0,among:!0,an:!0,and:!0,any:!0,are:!0,as:!0,at:!0,be:!0,because:!0,been:!0,but:!0,by:!0,can:!0,cannot:!0,could:!0,dear:!0,did:!0,"do":!0,does:!0,either:!0,"else":!0,ever:!0,every:!0,"for":!0,from:!0,get:!0,got:!0,had:!0,has:!0,have:!0,he:!0,her:!0,hers:!0,him:!0,his:!0,how:!0,however:!0,i:!0,"if":!0,"in":!0,into:!0,is:!0,it:!0,its:!0,just:!0,least:!0,let:!0,like:!0,likely:!0,may:!0,me:!0,might:!0,most:!0,must:!0,my:!0,neither:!0,no:!0,nor:!0,not:!0,of:!0,off:!0,often:!0,on:!0,only:!0,or:!0,other:!0,our:!0,own:!0,rather:!0,said:!0,say:!0,says:!0,she:!0,should:!0,since:!0,so:!0,some:!0,than:!0,that:!0,the:!0,their:!0,them:!0,then:!0,there:!0,these:!0,they:!0,"this":!0,tis:!0,to:!0,too:!0,twas:!0,us:!0,wants:!0,was:!0,we:!0,were:!0,what:!0,when:!0,where:!0,which:!0,"while":!0,who:!0,whom:!0,why:!0,will:!0,"with":!0,would:!0,yet:!0,you:!0,your:!0},t.stopWordFilter.stopWords=t.defaultStopWords,t.Pipeline.registerFunction(t.stopWordFilter,"stopWordFilter"),t.trimmer=function(e){if(null===e||void 0===e)throw new Error("token should not be undefined");return e.replace(/^\W+/,"").replace(/\W+$/,"")},t.Pipeline.registerFunction(t.trimmer,"trimmer"),t.InvertedIndex=function(){this.root={docs:{},df:0}},t.InvertedIndex.load=function(e){var t=new this;return t.root=e.root,t},t.InvertedIndex.prototype.addToken=function(e,t,n){for(var n=n||this.root,i=0;i<=e.length-1;){var o=e[i];o in n||(n[o]={docs:{},df:0}),i+=1,n=n[o]}var r=t.ref;n.docs[r]?n.docs[r]={tf:t.tf}:(n.docs[r]={tf:t.tf},n.df+=1)},t.InvertedIndex.prototype.hasToken=function(e){if(!e)return!1;for(var t=this.root,n=0;n<e.length;n++){if(!t[e[n]])return!1;t=t[e[n]]}return!0},t.InvertedIndex.prototype.getNode=function(e){if(!e)return null;for(var t=this.root,n=0;n<e.length;n++){if(!t[e[n]])return null;t=t[e[n]]}return t},t.InvertedIndex.prototype.getDocs=function(e){var t=this.getNode(e);return null==t?{}:t.docs},t.InvertedIndex.prototype.getTermFrequency=function(e,t){var n=this.getNode(e);return null==n?0:t in n.docs?n.docs[t].tf:0},t.InvertedIndex.prototype.getDocFreq=function(e){var t=this.getNode(e);return null==t?0:t.df},t.InvertedIndex.prototype.removeToken=function(e,t){if(e){var n=this.getNode(e);null!=n&&t in n.docs&&(delete n.docs[t],n.df-=1)}},t.InvertedIndex.prototype.expandToken=function(e,t,n){if(null==e||""==e)return[];var t=t||[];if(void 0==n&&(n=this.getNode(e),null==n))return t;n.df>0&&t.push(e);for(var i in n)"docs"!==i&&"df"!==i&&this.expandToken(e+i,t,n[i]);return t},t.InvertedIndex.prototype.toJSON=function(){return{root:this.root}},t.Configuration=function(e,n){var e=e||"";if(void 0==n||null==n)throw new Error("fields should not be null");this.config={};var i;try{i=JSON.parse(e),this.buildUserConfig(i,n)}catch(o){t.utils.warn("user configuration parse failed, will use default configuration"),this.buildDefaultConfig(n)}},t.Configuration.prototype.buildDefaultConfig=function(e){this.reset(),e.forEach(function(e){this.config[e]={boost:1,bool:"OR",expand:!1}},this)},t.Configuration.prototype.buildUserConfig=function(e,n){var i="OR",o=!1;if(this.reset(),"bool"in e&&(i=e.bool||i),"expand"in e&&(o=e.expand||o),"fields"in e)for(var r in e.fields)if(n.indexOf(r)>-1){var s=e.fields[r],u=o;void 0!=s.expand&&(u=s.expand),this.config[r]={boost:s.boost||0===s.boost?s.boost:1,bool:s.bool||i,expand:u}}else t.utils.warn("field name in user configuration not found in index instance fields");else this.addAllFields2UserConfig(i,o,n)},t.Configuration.prototype.addAllFields2UserConfig=function(e,t,n){n.forEach(function(n){this.config[n]={boost:1,bool:e,expand:t}},this)},t.Configuration.prototype.get=function(){return this.config},t.Configuration.prototype.reset=function(){this.config={}},lunr.SortedSet=function(){this.length=0,this.elements=[]},lunr.SortedSet.load=function(e){var t=new this;return t.elements=e,t.length=e.length,t},lunr.SortedSet.prototype.add=function(){var e,t;for(e=0;e<arguments.length;e++)t=arguments[e],~this.indexOf(t)||this.elements.splice(this.locationFor(t),0,t);this.length=this.elements.length},lunr.SortedSet.prototype.toArray=function(){return this.elements.slice()},lunr.SortedSet.prototype.map=function(e,t){return this.elements.map(e,t)},lunr.SortedSet.prototype.forEach=function(e,t){return this.elements.forEach(e,t)},lunr.SortedSet.prototype.indexOf=function(e){for(var t=0,n=this.elements.length,i=n-t,o=t+Math.floor(i/2),r=this.elements[o];i>1;){if(r===e)return o;e>r&&(t=o),r>e&&(n=o),i=n-t,o=t+Math.floor(i/2),r=this.elements[o]}return r===e?o:-1},lunr.SortedSet.prototype.locationFor=function(e){for(var t=0,n=this.elements.length,i=n-t,o=t+Math.floor(i/2),r=this.elements[o];i>1;)e>r&&(t=o),r>e&&(n=o),i=n-t,o=t+Math.floor(i/2),r=this.elements[o];return r>e?o:e>r?o+1:void 0},lunr.SortedSet.prototype.intersect=function(e){for(var t=new lunr.SortedSet,n=0,i=0,o=this.length,r=e.length,s=this.elements,u=e.elements;;){if(n>o-1||i>r-1)break;s[n]!==u[i]?s[n]<u[i]?n++:s[n]>u[i]&&i++:(t.add(s[n]),n++,i++)}return t},lunr.SortedSet.prototype.clone=function(){var e=new lunr.SortedSet;return e.elements=this.toArray(),e.length=e.elements.length,e},lunr.SortedSet.prototype.union=function(e){var t,n,i;this.length>=e.length?(t=this,n=e):(t=e,n=this),i=t.clone();for(var o=0,r=n.toArray();o<r.length;o++)i.add(r[o]);return i},lunr.SortedSet.prototype.toJSON=function(){return this.toArray()},function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():e.elasticlunr=t()}(this,function(){return t})}();
+ /** pdoc search index */const docs = {"version": "0.9.5", "fields": ["qualname", "fullname", "annotation", "default_value", "signature", "bases", "doc"], "ref": "fullname", "documentStore": {"docs": {"acmc": {"fullname": "acmc", "modulename": "acmc", "kind": "module", "doc": "<p></p>\n"}, "acmc.logging_config": {"fullname": "acmc.logging_config", "modulename": "acmc.logging_config", "kind": "module", "doc": "<p>logging_config.py</p>\n\n<p>This module defines functions to setup logging for acmc across all module.</p>\n"}, "acmc.logging_config.DEFAULT_LOG_FILE": {"fullname": "acmc.logging_config.DEFAULT_LOG_FILE", "modulename": "acmc.logging_config", "qualname": "DEFAULT_LOG_FILE", "kind": "variable", "doc": "<p>The default acmc application log filename.</p>\n", "default_value": "'acmc.log'"}, "acmc.logging_config.setup_logger": {"fullname": "acmc.logging_config.setup_logger", "modulename": "acmc.logging_config", "qualname": "setup_logger", "kind": "function", "doc": "<p>Sets up acmc logger as a singleton outputing to file and sysout syserr.</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">log_level</span><span class=\"p\">:</span> <span class=\"nb\">int</span> <span class=\"o\">=</span> <span class=\"mi\">20</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.logging_config.set_log_level": {"fullname": "acmc.logging_config.set_log_level", "modulename": "acmc.logging_config", "qualname": "set_log_level", "kind": "function", "doc": "<p>Sets the log level for the acmc logger.</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>log_level (int):</strong> log level from the python logging libraru</li>\n</ul>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">log_level</span><span class=\"p\">:</span> <span class=\"nb\">int</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.main": {"fullname": "acmc.main", "modulename": "acmc.main", "kind": "module", "doc": "<p>main.py module</p>\n\n<p>This module provides the functionality for the acmc command line interface</p>\n"}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"fullname": "acmc.main.DEFAULT_WORKSPACE_PATH", "modulename": "acmc.main", "qualname": "DEFAULT_WORKSPACE_PATH", "kind": "variable", "doc": "<p>Default phenotype workspace path</p>\n", "default_value": "PosixPath('workspace')"}, "acmc.main.main": {"fullname": "acmc.main.main", "modulename": "acmc.main", "qualname": "main", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.omop": {"fullname": "acmc.omop", "modulename": "acmc.omop", "kind": "module", "doc": "<p>omop.py module</p>\n\n<p>This module provides functionality to manage OMOP vocabularies.</p>\n"}, "acmc.omop.VOCAB_PATH": {"fullname": "acmc.omop.VOCAB_PATH", "modulename": "acmc.omop", "qualname": "VOCAB_PATH", "kind": "variable", "doc": "<p>Default OMOP vocabulary path</p>\n", "default_value": "PosixPath('vocab/omop')"}, "acmc.omop.OMOP_CDM_Version": {"fullname": "acmc.omop.OMOP_CDM_Version", "modulename": "acmc.omop", "qualname": "OMOP_CDM_Version", "kind": "variable", "doc": "<p>Supported OMOP CDM version number</p>\n", "default_value": "'54'"}, "acmc.omop.OMOP_DB_FILENAME": {"fullname": "acmc.omop.OMOP_DB_FILENAME", "modulename": "acmc.omop", "qualname": "OMOP_DB_FILENAME", "kind": "variable", "doc": "<p>Default OMOP sqllite database filename</p>\n", "default_value": "'omop_54.sqlite'"}, "acmc.omop.DB_PATH": {"fullname": "acmc.omop.DB_PATH", "modulename": "acmc.omop", "qualname": "DB_PATH", "kind": "variable", "doc": "<p>Default OMOP sqllite database path</p>\n", "default_value": "PosixPath('vocab/omop/omop_54.sqlite')"}, "acmc.omop.VERSION_FILE": {"fullname": "acmc.omop.VERSION_FILE", "modulename": "acmc.omop", "qualname": "VERSION_FILE", "kind": "variable", "doc": "<p>Default OMOP version file</p>\n", "default_value": "'omop_version.yml'"}, "acmc.omop.VERSION_PATH": {"fullname": "acmc.omop.VERSION_PATH", "modulename": "acmc.omop", "qualname": "VERSION_PATH", "kind": "variable", "doc": "<p>Default OMOP version path</p>\n", "default_value": "PosixPath('vocab/omop/omop_version.yml')"}, "acmc.omop.EXPORT_FILE": {"fullname": "acmc.omop.EXPORT_FILE", "modulename": "acmc.omop", "qualname": "EXPORT_FILE", "kind": "variable", "doc": "<p>Default OMOP export database filename</p>\n", "default_value": "'omop_54_export.sqlite'"}, "acmc.omop.vocabularies": {"fullname": "acmc.omop.vocabularies", "modulename": "acmc.omop", "qualname": "vocabularies", "kind": "variable", "doc": "<p>Required OMOP vocabularies definition</p>\n", "default_value": "{'source': 'OHDSI Athena', 'url': 'https://athena.ohdsi.org/vocabulary/list', 'cdm_version': '54', 'version': '', 'vocabularies': [{'id': 1, 'name': 'SNOMED'}, {'id': 2, 'name': 'ICD9CM'}, {'id': 17, 'name': 'Readv2'}, {'id': 21, 'name': 'ATC'}, {'id': 55, 'name': 'OPCS4'}, {'id': 57, 'name': 'HES Specialty'}, {'id': 70, 'name': 'ICD10CM'}, {'id': 75, 'name': 'dm+d'}, {'id': 144, 'name': 'UK Biobank'}, {'id': 154, 'name': 'NHS Ethnic Category'}, {'id': 155, 'name': 'NHS Place of Service'}], 'tables': []}"}, "acmc.omop.omop_vocab_types": {"fullname": "acmc.omop.omop_vocab_types", "modulename": "acmc.omop", "qualname": "omop_vocab_types", "kind": "variable", "doc": "<p>Type mappings from acmc medical coding types to OMOP vocabulary types</p>\n", "default_value": "{'read2': 'Read', 'read3': None, 'icd10': 'ICD10CM', 'snomed': 'SNOMED', 'opcs4': 'OPCS4', 'atc': 'ATC', 'med': None, 'cprd': None}"}, "acmc.omop.install": {"fullname": "acmc.omop.install", "modulename": "acmc.omop", "qualname": "install", "kind": "function", "doc": "<p>\"Installs the OMOP release csv files in a file-based sql database</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>omop_zip_file (str):</strong> vocabularies zip file distributed by OHDSI Athena</li>\n<li><strong>version (str):</strong> version of the vocabularies distributed by OHDSI Athena</li>\n</ul>\n\n<h6 id=\"raises\">Raises:</h6>\n\n<ul>\n<li><strong>ValueError:</strong> if the zip file does not exist</li>\n<li><strong>ValueError:</strong> if the file is not a zip file</li>\n<li><strong>Exception:</strong> if error reading omop csv files</li>\n</ul>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">omop_zip_file</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">version</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.omop.write_version_file": {"fullname": "acmc.omop.write_version_file", "modulename": "acmc.omop", "qualname": "write_version_file", "kind": "function", "doc": "<p>Writes the OMOP vocaburaries and version to a file</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>version (str):</strong> version of the vocabularies distributed by OHDSI Athena</li>\n</ul>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">version</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.omop.clear": {"fullname": "acmc.omop.clear", "modulename": "acmc.omop", "qualname": "clear", "kind": "function", "doc": "<p>Clears the OMOP sql database</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>db_path (Path):</strong> the path to the omop sqllite database</li>\n</ul>\n\n<h6 id=\"raises\">Raises:</h6>\n\n<ul>\n<li><strong>FileNotFoundError:</strong> if the omop sqllite database does not exist</li>\n</ul>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">db_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.omop.delete": {"fullname": "acmc.omop.delete", "modulename": "acmc.omop", "qualname": "delete", "kind": "function", "doc": "<p>Deletes the OMOP sql database</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>db_path (Path):</strong> the path to the omop sqllite database</li>\n</ul>\n\n<h6 id=\"raises\">Raises:</h6>\n\n<ul>\n<li><strong>FileNotFoundError:</strong> if the omop sqllite database does not exist</li>\n</ul>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">db_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.omop.table_exists": {"fullname": "acmc.omop.table_exists", "modulename": "acmc.omop", "qualname": "table_exists", "kind": "function", "doc": "<p>Query to check if the table exists</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>cursor (sqlite3.Cursor):</strong> a sqllite database cursor</li>\n<li><strong>table_name (str):</strong> the table name to check</li>\n</ul>\n\n<h6 id=\"returns\">Returns:</h6>\n\n<blockquote>\n <p>bool: true if table exists</p>\n</blockquote>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">cursor</span><span class=\"p\">:</span> <span class=\"n\">sqlite3</span><span class=\"o\">.</span><span class=\"n\">Cursor</span>, </span><span class=\"param\"><span class=\"n\">table_name</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">) -> <span class=\"nb\">bool</span>:</span></span>", "funcdef": "def"}, "acmc.omop.vocab_exists": {"fullname": "acmc.omop.vocab_exists", "modulename": "acmc.omop", "qualname": "vocab_exists", "kind": "function", "doc": "<p>Query to check if the vocabulary exists</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>cursor (sqlite3.Cursor):</strong> a sqllite database cursor</li>\n<li><strong>vocab_id (str):</strong> the vocabulary id to check</li>\n</ul>\n\n<h6 id=\"returns\">Returns:</h6>\n\n<blockquote>\n <p>bool: true if vocabulary id exists</p>\n</blockquote>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">cursor</span><span class=\"p\">:</span> <span class=\"n\">sqlite3</span><span class=\"o\">.</span><span class=\"n\">Cursor</span>, </span><span class=\"param\"><span class=\"n\">vocab_id</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">) -> <span class=\"nb\">bool</span>:</span></span>", "funcdef": "def"}, "acmc.omop.concept_set_exist": {"fullname": "acmc.omop.concept_set_exist", "modulename": "acmc.omop", "qualname": "concept_set_exist", "kind": "function", "doc": "<p>Query to check if the concept set exists</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>cursor (sqlite3.Cursor):</strong> a sqllite database cursor</li>\n<li><strong>concept_set_name (str):</strong> the concept set name to check</li>\n</ul>\n\n<h6 id=\"returns\">Returns:</h6>\n\n<blockquote>\n <p>bool: true if concept set exists</p>\n</blockquote>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">cursor</span><span class=\"p\">:</span> <span class=\"n\">sqlite3</span><span class=\"o\">.</span><span class=\"n\">Cursor</span>, </span><span class=\"param\"><span class=\"n\">concept_set_name</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">) -> <span class=\"nb\">bool</span>:</span></span>", "funcdef": "def"}, "acmc.omop.export": {"fullname": "acmc.omop.export", "modulename": "acmc.omop", "qualname": "export", "kind": "function", "doc": "<p>Export concept sets to omop database in csv format</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>map_path (Path):</strong> path to the acmc map directory containing concept sets in csv format</li>\n<li><strong>export_path (Path):</strong> path to the directory where the omop database csv files are to be written</li>\n<li><strong>version (str):</strong> phenotype version for omop vocabulary version</li>\n<li><strong>omop_metadata (dict):</strong> phenotype omop metadata for omop vocabulary metadata</li>\n</ul>\n\n<h6 id=\"returns\">Returns:</h6>\n\n<blockquote>\n <p>Path: path to the exported sqllite database</p>\n</blockquote>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"n\">map_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>,</span><span class=\"param\">\t<span class=\"n\">export_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>,</span><span class=\"param\">\t<span class=\"n\">version</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">omop_metadata</span><span class=\"p\">:</span> <span class=\"nb\">dict</span></span><span class=\"return-annotation\">) -> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>:</span></span>", "funcdef": "def"}, "acmc.parse": {"fullname": "acmc.parse", "modulename": "acmc.parse", "kind": "module", "doc": "<p>parse.py module</p>\n\n<p>This module provides functionality to set up medical code translation classes</p>\n"}, "acmc.parse.SUPPORTED_CODE_TYPES": {"fullname": "acmc.parse.SUPPORTED_CODE_TYPES", "modulename": "acmc.parse", "qualname": "SUPPORTED_CODE_TYPES", "kind": "variable", "doc": "<p>List of support medical coding types</p>\n", "default_value": "{'snomed', 'atc', 'read2', 'read3', 'icd10', 'opcs4'}"}, "acmc.parse.CodesError": {"fullname": "acmc.parse.CodesError", "modulename": "acmc.parse", "qualname": "CodesError", "kind": "class", "doc": "<p>A class used in InvalidCodesException to report an error if a code parser check fails</p>\n"}, "acmc.parse.CodesError.__init__": {"fullname": "acmc.parse.CodesError.__init__", "modulename": "acmc.parse", "qualname": "CodesError.__init__", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">message</span>, </span><span class=\"param\"><span class=\"n\">codes</span><span class=\"o\">=</span><span class=\"kc\">None</span>, </span><span class=\"param\"><span class=\"n\">codes_file</span><span class=\"o\">=</span><span class=\"kc\">None</span>, </span><span class=\"param\"><span class=\"n\">mask</span><span class=\"o\">=</span><span class=\"kc\">None</span>, </span><span class=\"param\"><span class=\"n\">code_type</span><span class=\"o\">=</span><span class=\"kc\">None</span></span>)</span>"}, "acmc.parse.InvalidCodesException": {"fullname": "acmc.parse.InvalidCodesException", "modulename": "acmc.parse", "qualname": "InvalidCodesException", "kind": "class", "doc": "<p>Custom exception class raised when invalid codes are found that cannot be resolved by processing</p>\n", "bases": "builtins.Exception"}, "acmc.parse.InvalidCodesException.__init__": {"fullname": "acmc.parse.InvalidCodesException.__init__", "modulename": "acmc.parse", "qualname": "InvalidCodesException.__init__", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">error</span></span>)</span>"}, "acmc.parse.InvalidCodesException.error": {"fullname": "acmc.parse.InvalidCodesException.error", "modulename": "acmc.parse", "qualname": "InvalidCodesException.error", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Proto": {"fullname": "acmc.parse.Proto", "modulename": "acmc.parse", "qualname": "Proto", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n"}, "acmc.parse.Proto.__init__": {"fullname": "acmc.parse.Proto.__init__", "modulename": "acmc.parse", "qualname": "Proto.__init__", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">name</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">trud_codes_path</span><span class=\"p\">:</span> <span class=\"n\">Optional</span><span class=\"p\">[</span><span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span><span class=\"p\">]</span> <span class=\"o\">=</span> <span class=\"kc\">None</span></span>)</span>"}, "acmc.parse.Proto.checks": {"fullname": "acmc.parse.Proto.checks", "modulename": "acmc.parse", "qualname": "Proto.checks", "kind": "variable", "doc": "<p></p>\n", "annotation": ": list[tuple[str, typing.Callable[[pandas.core.frame.DataFrame], pandas.core.series.Series], typing.Callable[[pandas.core.frame.DataFrame, pathlib.Path], pandas.core.frame.DataFrame]]]"}, "acmc.parse.Proto.name": {"fullname": "acmc.parse.Proto.name", "modulename": "acmc.parse", "qualname": "Proto.name", "kind": "variable", "doc": "<p></p>\n", "annotation": ": str"}, "acmc.parse.Proto.raise_exception": {"fullname": "acmc.parse.Proto.raise_exception", "modulename": "acmc.parse", "qualname": "Proto.raise_exception", "kind": "function", "doc": "<p>Raises an exception inside a lambda function. Python does not allow using raise statement inside lambda because lambda can only contain expressions, not statements. Using raise_exception not raise_ as it's more explict</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"bp\">self</span>, </span><span class=\"param\"><span class=\"n\">ex</span><span class=\"p\">:</span> <span class=\"ne\">Exception</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.parse.Proto.in_database": {"fullname": "acmc.parse.Proto.in_database", "modulename": "acmc.parse", "qualname": "Proto.in_database", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"bp\">self</span>,</span><span class=\"param\">\t<span class=\"n\">codes</span><span class=\"p\">:</span> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>,</span><span class=\"param\">\t<span class=\"n\">db</span><span class=\"p\">:</span> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>,</span><span class=\"param\">\t<span class=\"n\">col</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">) -> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>:</span></span>", "funcdef": "def"}, "acmc.parse.Proto.process": {"fullname": "acmc.parse.Proto.process", "modulename": "acmc.parse", "qualname": "Proto.process", "kind": "function", "doc": "<p>identify issues that do not pass and fix them with define/d process</p>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"bp\">self</span>,</span><span class=\"param\">\t<span class=\"n\">codes</span><span class=\"p\">:</span> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>,</span><span class=\"param\">\t<span class=\"n\">codes_file</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">) -> <span class=\"n\">Tuple</span><span class=\"p\">[</span><span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span><span class=\"p\">,</span> <span class=\"nb\">list</span><span class=\"p\">]</span>:</span></span>", "funcdef": "def"}, "acmc.parse.Proto.verify": {"fullname": "acmc.parse.Proto.verify", "modulename": "acmc.parse", "qualname": "Proto.verify", "kind": "function", "doc": "<p>verify codes in codes file</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"bp\">self</span>, </span><span class=\"param\"><span class=\"n\">codes</span><span class=\"p\">:</span> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>, </span><span class=\"param\"><span class=\"n\">codes_file</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.parse.Read2": {"fullname": "acmc.parse.Read2", "modulename": "acmc.parse", "qualname": "Read2", "kind": "class", "doc": "<p>This Read2 class extends Proto, adding custom validation checks for a dataset of \"Read2\" codes. It ensures that the dataset is loaded, validates the codes based on several rules, and applies corrections or logs errors when necessary.</p>\n", "bases": "Proto"}, "acmc.parse.Read2.checks": {"fullname": "acmc.parse.Read2.checks", "modulename": "acmc.parse", "qualname": "Read2.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Read3": {"fullname": "acmc.parse.Read3", "modulename": "acmc.parse", "qualname": "Read3", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Read3.checks": {"fullname": "acmc.parse.Read3.checks", "modulename": "acmc.parse", "qualname": "Read3.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Icd10": {"fullname": "acmc.parse.Icd10", "modulename": "acmc.parse", "qualname": "Icd10", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Icd10.checks": {"fullname": "acmc.parse.Icd10.checks", "modulename": "acmc.parse", "qualname": "Icd10.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Snomed": {"fullname": "acmc.parse.Snomed", "modulename": "acmc.parse", "qualname": "Snomed", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Snomed.checks": {"fullname": "acmc.parse.Snomed.checks", "modulename": "acmc.parse", "qualname": "Snomed.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Opcs4": {"fullname": "acmc.parse.Opcs4", "modulename": "acmc.parse", "qualname": "Opcs4", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Opcs4.checks": {"fullname": "acmc.parse.Opcs4.checks", "modulename": "acmc.parse", "qualname": "Opcs4.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Atc": {"fullname": "acmc.parse.Atc", "modulename": "acmc.parse", "qualname": "Atc", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Atc.checks": {"fullname": "acmc.parse.Atc.checks", "modulename": "acmc.parse", "qualname": "Atc.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Med": {"fullname": "acmc.parse.Med", "modulename": "acmc.parse", "qualname": "Med", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Med.checks": {"fullname": "acmc.parse.Med.checks", "modulename": "acmc.parse", "qualname": "Med.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.Cprd": {"fullname": "acmc.parse.Cprd", "modulename": "acmc.parse", "qualname": "Cprd", "kind": "class", "doc": "<p>Define checks as list of 3 tuple: (Message, Condition, Process)</p>\n\n<ul>\n<li>Message = The name of the condition (what is printed and logged)</li>\n<li>Condition = True if Passed, and False if Failed</li>\n<li>Process = Aims to resolve all issues that stop condition from passing (Do not change index!)</li>\n</ul>\n", "bases": "Proto"}, "acmc.parse.Cprd.checks": {"fullname": "acmc.parse.Cprd.checks", "modulename": "acmc.parse", "qualname": "Cprd.checks", "kind": "variable", "doc": "<p></p>\n"}, "acmc.parse.CodeTypeParser": {"fullname": "acmc.parse.CodeTypeParser", "modulename": "acmc.parse", "qualname": "CodeTypeParser", "kind": "class", "doc": "<p>A class used in InvalidCodesException to report an error if a code parser check fails</p>\n"}, "acmc.parse.CodeTypeParser.__init__": {"fullname": "acmc.parse.CodeTypeParser.__init__", "modulename": "acmc.parse", "qualname": "CodeTypeParser.__init__", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">trud_processed_dir</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span> <span class=\"o\">=</span> <span class=\"n\">PosixPath</span><span class=\"p\">(</span><span class=\"s1\">'vocab/trud/processed'</span><span class=\"p\">)</span></span>)</span>"}, "acmc.parse.CodeTypeParser.code_types": {"fullname": "acmc.parse.CodeTypeParser.code_types", "modulename": "acmc.parse", "qualname": "CodeTypeParser.code_types", "kind": "variable", "doc": "<p></p>\n"}, "acmc.phen": {"fullname": "acmc.phen", "modulename": "acmc.phen", "kind": "module", "doc": "<p>phenotype.py module</p>\n\n<p>This module provides functionality for managing phenotypes.</p>\n"}, "acmc.phen.PHEN_DIR": {"fullname": "acmc.phen.PHEN_DIR", "modulename": "acmc.phen", "qualname": "PHEN_DIR", "kind": "variable", "doc": "<p>Default phenotype directory name</p>\n", "default_value": "'phen'"}, "acmc.phen.DEFAULT_PHEN_PATH": {"fullname": "acmc.phen.DEFAULT_PHEN_PATH", "modulename": "acmc.phen", "qualname": "DEFAULT_PHEN_PATH", "kind": "variable", "doc": "<p>Default phenotype directory path</p>\n", "default_value": "PosixPath('workspace/phen')"}, "acmc.phen.CONCEPTS_DIR": {"fullname": "acmc.phen.CONCEPTS_DIR", "modulename": "acmc.phen", "qualname": "CONCEPTS_DIR", "kind": "variable", "doc": "<p>Default concepts directory name</p>\n", "default_value": "'concepts'"}, "acmc.phen.MAP_DIR": {"fullname": "acmc.phen.MAP_DIR", "modulename": "acmc.phen", "qualname": "MAP_DIR", "kind": "variable", "doc": "<p>Default map directory name</p>\n", "default_value": "'map'"}, "acmc.phen.CONCEPT_SET_DIR": {"fullname": "acmc.phen.CONCEPT_SET_DIR", "modulename": "acmc.phen", "qualname": "CONCEPT_SET_DIR", "kind": "variable", "doc": "<p>Default concept set directory name</p>\n", "default_value": "'concept-sets'"}, "acmc.phen.CSV_PATH": {"fullname": "acmc.phen.CSV_PATH", "modulename": "acmc.phen", "qualname": "CSV_PATH", "kind": "variable", "doc": "<p>Default CSV concept set directory path</p>\n", "default_value": "PosixPath('concept-sets/csv')"}, "acmc.phen.OMOP_PATH": {"fullname": "acmc.phen.OMOP_PATH", "modulename": "acmc.phen", "qualname": "OMOP_PATH", "kind": "variable", "doc": "<p>Default OMOP concept set directory path</p>\n", "default_value": "PosixPath('concept-sets/omop')"}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"fullname": "acmc.phen.DEFAULT_PHEN_DIR_LIST", "modulename": "acmc.phen", "qualname": "DEFAULT_PHEN_DIR_LIST", "kind": "variable", "doc": "<p>List of default phenotype directories</p>\n", "default_value": "['concepts', 'map', 'concept-sets']"}, "acmc.phen.CONFIG_FILE": {"fullname": "acmc.phen.CONFIG_FILE", "modulename": "acmc.phen", "qualname": "CONFIG_FILE", "kind": "variable", "doc": "<p>Default configuration filename</p>\n", "default_value": "'config.yml'"}, "acmc.phen.VOCAB_VERSION_FILE": {"fullname": "acmc.phen.VOCAB_VERSION_FILE", "modulename": "acmc.phen", "qualname": "VOCAB_VERSION_FILE", "kind": "variable", "doc": "<p>Default vocabulary version filename</p>\n", "default_value": "'vocab_version.yml'"}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"fullname": "acmc.phen.SEMANTIC_VERSION_TYPES", "modulename": "acmc.phen", "qualname": "SEMANTIC_VERSION_TYPES", "kind": "variable", "doc": "<p>List of semantic version increment types</p>\n", "default_value": "['major', 'minor', 'patch']"}, "acmc.phen.DEFAULT_VERSION_INC": {"fullname": "acmc.phen.DEFAULT_VERSION_INC", "modulename": "acmc.phen", "qualname": "DEFAULT_VERSION_INC", "kind": "variable", "doc": "<p>Default semantic version increment type</p>\n", "default_value": "'patch'"}, "acmc.phen.DEFAULT_GIT_BRANCH": {"fullname": "acmc.phen.DEFAULT_GIT_BRANCH", "modulename": "acmc.phen", "qualname": "DEFAULT_GIT_BRANCH", "kind": "variable", "doc": "<p>Default phenotype repo branch name</p>\n", "default_value": "'main'"}, "acmc.phen.SPLIT_COL_ACTION": {"fullname": "acmc.phen.SPLIT_COL_ACTION", "modulename": "acmc.phen", "qualname": "SPLIT_COL_ACTION", "kind": "variable", "doc": "<p>Split column preprocessing action type</p>\n", "default_value": "'split_col'"}, "acmc.phen.CODES_COL_ACTION": {"fullname": "acmc.phen.CODES_COL_ACTION", "modulename": "acmc.phen", "qualname": "CODES_COL_ACTION", "kind": "variable", "doc": "<p>Codes column preprocessing action type</p>\n", "default_value": "'codes_col'"}, "acmc.phen.DIVIDE_COL_ACTION": {"fullname": "acmc.phen.DIVIDE_COL_ACTION", "modulename": "acmc.phen", "qualname": "DIVIDE_COL_ACTION", "kind": "variable", "doc": "<p>Divide column preprocessing action type</p>\n", "default_value": "'divide_col'"}, "acmc.phen.COL_ACTIONS": {"fullname": "acmc.phen.COL_ACTIONS", "modulename": "acmc.phen", "qualname": "COL_ACTIONS", "kind": "variable", "doc": "<p>List of column preprocessing action types</p>\n", "default_value": "['split_col', 'codes_col', 'divide_col']"}, "acmc.phen.CODE_FILE_TYPES": {"fullname": "acmc.phen.CODE_FILE_TYPES", "modulename": "acmc.phen", "qualname": "CODE_FILE_TYPES", "kind": "variable", "doc": "<p>List of supported source concept coding list file types</p>\n", "default_value": "['.xlsx', '.xls', '.csv']"}, "acmc.phen.CONFIG_SCHEMA": {"fullname": "acmc.phen.CONFIG_SCHEMA", "modulename": "acmc.phen", "qualname": "CONFIG_SCHEMA", "kind": "variable", "doc": "<p>Phenotype config.yml schema definition</p>\n", "default_value": "{'phenotype': {'type': 'dict', 'required': True, 'schema': {'version': {'type': 'string', 'required': True, 'regex': '^\\\\d+\\\\.\\\\d+\\\\.\\\\d+$'}, 'omop': {'type': 'dict', 'required': True, 'schema': {'vocabulary_id': {'type': 'string', 'required': True}, 'vocabulary_name': {'type': 'string', 'required': True}, 'vocabulary_reference': {'type': 'string', 'required': True, 'regex': '^https?://.*'}}}, 'map': {'type': 'list', 'schema': {'type': 'string', 'allowed': ['snomed', 'atc', 'read2', 'read3', 'icd10', 'opcs4']}}, 'concept_sets': {'type': 'list', 'required': True, 'schema': {'type': 'dict', 'schema': {'name': {'type': 'string', 'required': True}, 'file': {'type': 'dict', 'required': False, 'schema': {'path': {'type': 'string', 'required': True}, 'columns': {'type': 'dict', 'required': True}, 'category': {'type': 'string'}, 'actions': {'type': 'dict', 'schema': {'divide_col': {'type': 'string'}}}}}, 'metadata': {'type': 'dict', 'required': True}}}}}}}"}, "acmc.phen.PhenValidationException": {"fullname": "acmc.phen.PhenValidationException", "modulename": "acmc.phen", "qualname": "PhenValidationException", "kind": "class", "doc": "<p>Custom exception class raised when validation errors in phenotype configuration file</p>\n", "bases": "builtins.Exception"}, "acmc.phen.PhenValidationException.__init__": {"fullname": "acmc.phen.PhenValidationException.__init__", "modulename": "acmc.phen", "qualname": "PhenValidationException.__init__", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">message</span>, </span><span class=\"param\"><span class=\"n\">validation_errors</span><span class=\"o\">=</span><span class=\"kc\">None</span></span>)</span>"}, "acmc.phen.PhenValidationException.validation_errors": {"fullname": "acmc.phen.PhenValidationException.validation_errors", "modulename": "acmc.phen", "qualname": "PhenValidationException.validation_errors", "kind": "variable", "doc": "<p></p>\n"}, "acmc.phen.init": {"fullname": "acmc.phen.init", "modulename": "acmc.phen", "qualname": "init", "kind": "function", "doc": "<p>Initial phenotype directory as git repo with standard structure</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">remote_url</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.fork": {"fullname": "acmc.phen.fork", "modulename": "acmc.phen", "qualname": "fork", "kind": "function", "doc": "<p>Forks an upstream phenotype in a remote repo at a specific version to a local director, and optionally sets to a new remote origin\"</p>\n\n<h6 id=\"arguments\">Arguments:</h6>\n\n<ul>\n<li><strong>phen_dir (str):</strong> local directory path where the upstream repo is to be cloned</li>\n<li><strong>upstream_url (str):</strong> url to the upstream repo</li>\n<li><strong>upstream_version (str):</strong> version in the upstream repo to clone</li>\n<li><strong>new_origin_url (str, optional):</strong> url of the remote repo to set as the new origin. Defaults to None.</li>\n</ul>\n\n<h6 id=\"raises\">Raises:</h6>\n\n<ul>\n<li><strong>ValueError:</strong> if the specified version is not in the upstream repo</li>\n<li><strong>ValueError:</strong> if the upstream repo is not a valid phenotype repo</li>\n<li><strong>ValueError:</strong> if there's any other problems with Git</li>\n</ul>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">upstream_url</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">upstream_version</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">new_origin_url</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.validate": {"fullname": "acmc.phen.validate", "modulename": "acmc.phen", "qualname": "validate", "kind": "function", "doc": "<p>Validates the phenotype directory is a git repo with standard structure</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.translate_codes": {"fullname": "acmc.phen.translate_codes", "modulename": "acmc.phen", "qualname": "translate_codes", "kind": "function", "doc": "<p>Translates each source code type the source coding list into a target type and returns all conversions as a concept set</p>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"n\">source_df</span><span class=\"p\">:</span> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>,</span><span class=\"param\">\t<span class=\"n\">target_code_type</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">concept_name</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">) -> <span class=\"n\">pandas</span><span class=\"o\">.</span><span class=\"n\">core</span><span class=\"o\">.</span><span class=\"n\">frame</span><span class=\"o\">.</span><span class=\"n\">DataFrame</span>:</span></span>", "funcdef": "def"}, "acmc.phen.write_vocab_version": {"fullname": "acmc.phen.write_vocab_version", "modulename": "acmc.phen", "qualname": "write_vocab_version", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.map": {"fullname": "acmc.phen.map", "modulename": "acmc.phen", "qualname": "map", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">target_code_type</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.publish": {"fullname": "acmc.phen.publish", "modulename": "acmc.phen", "qualname": "publish", "kind": "function", "doc": "<p>Publishes updates to the phenotype by commiting all changes to the repo directory</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">msg</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">remote_url</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">increment</span><span class=\"p\">:</span> <span class=\"nb\">str</span> <span class=\"o\">=</span> <span class=\"s1\">'patch'</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.export": {"fullname": "acmc.phen.export", "modulename": "acmc.phen", "qualname": "export", "kind": "function", "doc": "<p>Exports a phen repo at a specific tagged version into a target directory</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">version</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.copy": {"fullname": "acmc.phen.copy", "modulename": "acmc.phen", "qualname": "copy", "kind": "function", "doc": "<p>Copys a phen repo at a specific tagged version into a target directory</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">target_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">version</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.extract_concepts": {"fullname": "acmc.phen.extract_concepts", "modulename": "acmc.phen", "qualname": "extract_concepts", "kind": "function", "doc": "<p>Extracts concepts as {name: file_path} dictionary and a name set.</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">config_data</span><span class=\"p\">:</span> <span class=\"nb\">dict</span></span><span class=\"return-annotation\">) -> <span class=\"n\">Tuple</span><span class=\"p\">[</span><span class=\"nb\">dict</span><span class=\"p\">,</span> <span class=\"n\">Set</span><span class=\"p\">[</span><span class=\"nb\">str</span><span class=\"p\">]]</span>:</span></span>", "funcdef": "def"}, "acmc.phen.diff_config": {"fullname": "acmc.phen.diff_config", "modulename": "acmc.phen", "qualname": "diff_config", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">old_config</span><span class=\"p\">:</span> <span class=\"nb\">dict</span>, </span><span class=\"param\"><span class=\"n\">new_config</span><span class=\"p\">:</span> <span class=\"nb\">dict</span></span><span class=\"return-annotation\">) -> <span class=\"nb\">str</span>:</span></span>", "funcdef": "def"}, "acmc.phen.diff_map_files": {"fullname": "acmc.phen.diff_map_files", "modulename": "acmc.phen", "qualname": "diff_map_files", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">old_map_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>, </span><span class=\"param\"><span class=\"n\">new_map_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">) -> <span class=\"nb\">str</span>:</span></span>", "funcdef": "def"}, "acmc.phen.diff_phen": {"fullname": "acmc.phen.diff_phen", "modulename": "acmc.phen", "qualname": "diff_phen", "kind": "function", "doc": "<p>Compare the differences between two versions of a phenotype</p>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"n\">new_phen_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>,</span><span class=\"param\">\t<span class=\"n\">new_version</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">old_phen_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>,</span><span class=\"param\">\t<span class=\"n\">old_version</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">report_path</span><span class=\"p\">:</span> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.phen.diff": {"fullname": "acmc.phen.diff", "modulename": "acmc.phen", "qualname": "diff", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">version</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">old_phen_dir</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">old_version</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud": {"fullname": "acmc.trud", "modulename": "acmc.trud", "kind": "module", "doc": "<p>trud.py module</p>\n\n<p>This module provides functionality to manage installation of the NHS TRUD vocabularies.</p>\n"}, "acmc.trud.FQDN": {"fullname": "acmc.trud.FQDN", "modulename": "acmc.trud", "qualname": "FQDN", "kind": "variable", "doc": "<p>Fully Qualified Domain Name of NHS digital TRUD service API</p>\n", "default_value": "'isd.digital.nhs.uk'"}, "acmc.trud.VOCAB_PATH": {"fullname": "acmc.trud.VOCAB_PATH", "modulename": "acmc.trud", "qualname": "VOCAB_PATH", "kind": "variable", "doc": "<p>Default path to the TRUD vocabulary directory relative to the the acmc execution directory</p>\n", "default_value": "PosixPath('vocab/trud')"}, "acmc.trud.VERSION_FILE": {"fullname": "acmc.trud.VERSION_FILE", "modulename": "acmc.trud", "qualname": "VERSION_FILE", "kind": "variable", "doc": "<p>TRUD version file</p>\n", "default_value": "'trud_version.yml'"}, "acmc.trud.VERSION_PATH": {"fullname": "acmc.trud.VERSION_PATH", "modulename": "acmc.trud", "qualname": "VERSION_PATH", "kind": "variable", "doc": "<p>Default path to the TRUD version file</p>\n", "default_value": "PosixPath('vocab/trud/trud_version.yml')"}, "acmc.trud.DOWNLOADS_PATH": {"fullname": "acmc.trud.DOWNLOADS_PATH", "modulename": "acmc.trud", "qualname": "DOWNLOADS_PATH", "kind": "variable", "doc": "<p>Default path to the TRUD vocabulary downloads directory</p>\n", "default_value": "PosixPath('vocab/trud/downloads')"}, "acmc.trud.PROCESSED_PATH": {"fullname": "acmc.trud.PROCESSED_PATH", "modulename": "acmc.trud", "qualname": "PROCESSED_PATH", "kind": "variable", "doc": "<p>Default path to the processed TRUD mappings directory</p>\n", "default_value": "PosixPath('vocab/trud/processed')"}, "acmc.trud.get_releases": {"fullname": "acmc.trud.get_releases", "modulename": "acmc.trud", "qualname": "get_releases", "kind": "function", "doc": "<p>Retrieve release information for an item from the TRUD API.</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">item_id</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">API_KEY</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">latest</span><span class=\"o\">=</span><span class=\"kc\">False</span></span><span class=\"return-annotation\">) -> <span class=\"nb\">list</span>:</span></span>", "funcdef": "def"}, "acmc.trud.download_release_file": {"fullname": "acmc.trud.download_release_file", "modulename": "acmc.trud", "qualname": "download_release_file", "kind": "function", "doc": "<p>Download specified file type for a given release of an item.</p>\n", "signature": "<span class=\"signature pdoc-code multiline\">(<span class=\"param\">\t<span class=\"n\">item_id</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">release_ordinal</span><span class=\"p\">:</span> <span class=\"nb\">str</span>,</span><span class=\"param\">\t<span class=\"n\">release</span><span class=\"p\">:</span> <span class=\"nb\">dict</span>,</span><span class=\"param\">\t<span class=\"n\">file_json_prefix</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">) -> <span class=\"n\">pathlib</span><span class=\"o\">.</span><span class=\"n\">Path</span>:</span></span>", "funcdef": "def"}, "acmc.trud.validate_download_hash": {"fullname": "acmc.trud.validate_download_hash", "modulename": "acmc.trud", "qualname": "validate_download_hash", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">file_destination</span><span class=\"p\">:</span> <span class=\"nb\">str</span>, </span><span class=\"param\"><span class=\"n\">item_hash</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.unzip_download": {"fullname": "acmc.trud.unzip_download", "modulename": "acmc.trud", "qualname": "unzip_download", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"n\">file_destination</span><span class=\"p\">:</span> <span class=\"nb\">str</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.extract_icd10": {"fullname": "acmc.trud.extract_icd10", "modulename": "acmc.trud", "qualname": "extract_icd10", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.extract_opsc4": {"fullname": "acmc.trud.extract_opsc4", "modulename": "acmc.trud", "qualname": "extract_opsc4", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.extract_nhs_data_migrations": {"fullname": "acmc.trud.extract_nhs_data_migrations", "modulename": "acmc.trud", "qualname": "extract_nhs_data_migrations", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.extract_nhs_read_browser": {"fullname": "acmc.trud.extract_nhs_read_browser", "modulename": "acmc.trud", "qualname": "extract_nhs_read_browser", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.create_map_directories": {"fullname": "acmc.trud.create_map_directories", "modulename": "acmc.trud", "qualname": "create_map_directories", "kind": "function", "doc": "<p>Create map directories.</p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.trud.install": {"fullname": "acmc.trud.install", "modulename": "acmc.trud", "qualname": "install", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"return-annotation\">):</span></span>", "funcdef": "def"}, "acmc.util": {"fullname": "acmc.util", "modulename": "acmc.util", "kind": "module", "doc": "<p></p>\n"}, "acmc.util.QuotedDumper": {"fullname": "acmc.util.QuotedDumper", "modulename": "acmc.util", "qualname": "QuotedDumper", "kind": "class", "doc": "<p>Custom Dumper to retain quotes on strings in yaml library</p>\n", "bases": "yaml.dumper.Dumper"}, "acmc.util.QuotedDumper.increase_indent": {"fullname": "acmc.util.QuotedDumper.increase_indent", "modulename": "acmc.util", "qualname": "QuotedDumper.increase_indent", "kind": "function", "doc": "<p></p>\n", "signature": "<span class=\"signature pdoc-code condensed\">(<span class=\"param\"><span class=\"bp\">self</span>, </span><span class=\"param\"><span class=\"n\">flow</span><span class=\"o\">=</span><span class=\"kc\">False</span>, </span><span class=\"param\"><span class=\"n\">indentless</span><span class=\"o\">=</span><span class=\"kc\">False</span></span><span class=\"return-annotation\">):</span></span>", "funcdef": "def"}}, "docInfo": {"acmc": {"qualname": 0, "fullname": 1, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.logging_config": {"qualname": 0, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 20}, "acmc.logging_config.DEFAULT_LOG_FILE": {"qualname": 3, "fullname": 6, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 9}, "acmc.logging_config.setup_logger": {"qualname": 2, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 24, "bases": 0, "doc": 16}, "acmc.logging_config.set_log_level": {"qualname": 3, "fullname": 6, "annotation": 0, "default_value": 0, "signature": 17, "bases": 0, "doc": 33}, "acmc.main": {"qualname": 0, "fullname": 2, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 18}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 6}, "acmc.main.main": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 3}, "acmc.omop": {"qualname": 0, "fullname": 2, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 16}, "acmc.omop.VOCAB_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 6}, "acmc.omop.OMOP_CDM_Version": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 7}, "acmc.omop.OMOP_DB_FILENAME": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 7}, "acmc.omop.DB_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 7}, "acmc.omop.VERSION_FILE": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 6}, "acmc.omop.VERSION_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 6}, "acmc.omop.EXPORT_FILE": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 8, "signature": 0, "bases": 0, "doc": 7}, "acmc.omop.vocabularies": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 202, "signature": 0, "bases": 0, "doc": 6}, "acmc.omop.omop_vocab_types": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 51, "signature": 0, "bases": 0, "doc": 13}, "acmc.omop.install": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 28, "bases": 0, "doc": 96}, "acmc.omop.write_version_file": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 16, "bases": 0, "doc": 33}, "acmc.omop.clear": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 22, "bases": 0, "doc": 49}, "acmc.omop.delete": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 22, "bases": 0, "doc": 49}, "acmc.omop.table_exists": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 35, "bases": 0, "doc": 55}, "acmc.omop.vocab_exists": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 35, "bases": 0, "doc": 56}, "acmc.omop.concept_set_exist": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 36, "bases": 0, "doc": 59}, "acmc.omop.export": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 71, "bases": 0, "doc": 104}, "acmc.parse": {"qualname": 0, "fullname": 2, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 18}, "acmc.parse.SUPPORTED_CODE_TYPES": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 20, "signature": 0, "bases": 0, "doc": 8}, "acmc.parse.CodesError": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 17}, "acmc.parse.CodesError.__init__": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 51, "bases": 0, "doc": 3}, "acmc.parse.InvalidCodesException": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 2, "doc": 17}, "acmc.parse.InvalidCodesException.__init__": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 9, "bases": 0, "doc": 3}, "acmc.parse.InvalidCodesException.error": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Proto": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 59}, "acmc.parse.Proto.__init__": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 44, "bases": 0, "doc": 3}, "acmc.parse.Proto.checks": {"qualname": 2, "fullname": 4, "annotation": 22, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Proto.name": {"qualname": 2, "fullname": 4, "annotation": 2, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Proto.raise_exception": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 21, "bases": 0, "doc": 36}, "acmc.parse.Proto.in_database": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 93, "bases": 0, "doc": 3}, "acmc.parse.Proto.process": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 85, "bases": 0, "doc": 14}, "acmc.parse.Proto.verify": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 52, "bases": 0, "doc": 7}, "acmc.parse.Read2": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 40}, "acmc.parse.Read2.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Read3": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Read3.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Icd10": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Icd10.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Snomed": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Snomed.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Opcs4": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Opcs4.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Atc": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Atc.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Med": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Med.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.Cprd": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 1, "doc": 59}, "acmc.parse.Cprd.checks": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.parse.CodeTypeParser": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 17}, "acmc.parse.CodeTypeParser.__init__": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 40, "bases": 0, "doc": 3}, "acmc.parse.CodeTypeParser.code_types": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.phen": {"qualname": 0, "fullname": 2, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 15}, "acmc.phen.PHEN_DIR": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 6}, "acmc.phen.DEFAULT_PHEN_PATH": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 6}, "acmc.phen.CONCEPTS_DIR": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 6}, "acmc.phen.MAP_DIR": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 6}, "acmc.phen.CONCEPT_SET_DIR": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.CSV_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 8}, "acmc.phen.OMOP_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 8}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"qualname": 4, "fullname": 6, "annotation": 0, "default_value": 12, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.CONFIG_FILE": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 5}, "acmc.phen.VOCAB_VERSION_FILE": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 6}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 11, "signature": 0, "bases": 0, "doc": 8}, "acmc.phen.DEFAULT_VERSION_INC": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.DEFAULT_GIT_BRANCH": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.SPLIT_COL_ACTION": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.CODES_COL_ACTION": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.DIVIDE_COL_ACTION": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 6, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.COL_ACTIONS": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 14, "signature": 0, "bases": 0, "doc": 8}, "acmc.phen.CODE_FILE_TYPES": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 11, "signature": 0, "bases": 0, "doc": 11}, "acmc.phen.CONFIG_SCHEMA": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 352, "signature": 0, "bases": 0, "doc": 7}, "acmc.phen.PhenValidationException": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 2, "doc": 13}, "acmc.phen.PhenValidationException.__init__": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 20, "bases": 0, "doc": 3}, "acmc.phen.PhenValidationException.validation_errors": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.phen.init": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 28, "bases": 0, "doc": 11}, "acmc.phen.fork": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 55, "bases": 0, "doc": 156}, "acmc.phen.validate": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 17, "bases": 0, "doc": 13}, "acmc.phen.translate_codes": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 76, "bases": 0, "doc": 23}, "acmc.phen.write_vocab_version": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 22, "bases": 0, "doc": 3}, "acmc.phen.map": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 29, "bases": 0, "doc": 3}, "acmc.phen.publish": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 59, "bases": 0, "doc": 15}, "acmc.phen.export": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 27, "bases": 0, "doc": 15}, "acmc.phen.copy": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 38, "bases": 0, "doc": 15}, "acmc.phen.extract_concepts": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 36, "bases": 0, "doc": 14}, "acmc.phen.diff_config": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 31, "bases": 0, "doc": 3}, "acmc.phen.diff_map_files": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 43, "bases": 0, "doc": 3}, "acmc.phen.diff_phen": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 83, "bases": 0, "doc": 11}, "acmc.phen.diff": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 50, "bases": 0, "doc": 3}, "acmc.trud": {"qualname": 0, "fullname": 2, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 20}, "acmc.trud.FQDN": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 8, "signature": 0, "bases": 0, "doc": 12}, "acmc.trud.VOCAB_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 16}, "acmc.trud.VERSION_FILE": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 5}, "acmc.trud.VERSION_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 7, "signature": 0, "bases": 0, "doc": 9}, "acmc.trud.DOWNLOADS_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 10}, "acmc.trud.PROCESSED_PATH": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 5, "signature": 0, "bases": 0, "doc": 10}, "acmc.trud.get_releases": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 41, "bases": 0, "doc": 13}, "acmc.trud.download_release_file": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 62, "bases": 0, "doc": 14}, "acmc.trud.validate_download_hash": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 28, "bases": 0, "doc": 3}, "acmc.trud.unzip_download": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 17, "bases": 0, "doc": 3}, "acmc.trud.extract_icd10": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 3}, "acmc.trud.extract_opsc4": {"qualname": 2, "fullname": 4, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 3}, "acmc.trud.extract_nhs_data_migrations": {"qualname": 4, "fullname": 6, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 3}, "acmc.trud.extract_nhs_read_browser": {"qualname": 4, "fullname": 6, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 3}, "acmc.trud.create_map_directories": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 6}, "acmc.trud.install": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 7, "bases": 0, "doc": 3}, "acmc.util": {"qualname": 0, "fullname": 2, "annotation": 0, "default_value": 0, "signature": 0, "bases": 0, "doc": 3}, "acmc.util.QuotedDumper": {"qualname": 1, "fullname": 3, "annotation": 0, "default_value": 0, "signature": 0, "bases": 3, "doc": 12}, "acmc.util.QuotedDumper.increase_indent": {"qualname": 3, "fullname": 5, "annotation": 0, "default_value": 0, "signature": 31, "bases": 0, "doc": 3}}, "length": 117, "save": true}, "index": {"qualname": {"root": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}}, "df": 5, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 6}}}}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.delete": {"tf": 1}}, "df": 1}}}}}, "b": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}}, "df": 2}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {"acmc.trud.extract_nhs_data_migrations": {"tf": 1}}, "df": 1, "b": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1}}, "df": 1}}}}}}}, "i": {"docs": {}, "df": 0, "r": {"docs": {"acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}}, "df": 5, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.create_map_directories": {"tf": 1}}, "df": 1}}}}}}}}}, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}}, "df": 1}}}}, "f": {"docs": {}, "df": 0, "f": {"docs": {"acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}, "acmc.phen.diff": {"tf": 1}}, "df": 4}}}, "o": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.download_release_file": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}, "acmc.trud.unzip_download": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.trud.DOWNLOADS_PATH": {"tf": 1}}, "df": 1}}}}}}}}}, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2, "g": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}}}, "e": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "l": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}}, "df": 1}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}}, "df": 1}}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 9, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}}, "df": 1}}}}, "s": {"docs": {"acmc.phen.diff_map_files": {"tf": 1}}, "df": 1}}}}, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "k": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}, "q": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}}, "df": 3, "u": {"docs": {}, "df": 0, "p": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}}, "df": 1}}}}}}}, "u": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}}, "df": 1}}}}}}}}, "n": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Snomed.checks": {"tf": 1}}, "df": 2}}}}}, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}}, "df": 1}}}}, "c": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "a": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}, "w": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "k": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.write_version_file": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}}, "df": 2}}}}}, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 11}}}, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Proto.name": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}}, "df": 8}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}}, "df": 4, "v": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.validation_errors": {"tf": 1}}, "df": 3}}}}}}}}}}}}}}}}}}}}}}, "u": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.main.main": {"tf": 1}}, "df": 1}}, "p": {"docs": {"acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.trud.create_map_directories": {"tf": 1}}, "df": 4}}, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Med": {"tf": 1}, "acmc.parse.Med.checks": {"tf": 1}}, "df": 2}}, "i": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.extract_nhs_data_migrations": {"tf": 1}}, "df": 1}}}}}}}}}}, "v": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {"acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}}, "df": 6, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}}}}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}}, "df": 10}}}}, "i": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "y": {"docs": {"acmc.parse.Proto.verify": {"tf": 1}}, "df": 1}}}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PhenValidationException.validation_errors": {"tf": 1}}, "df": 1}}}, "e": {"docs": {"acmc.phen.validate": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}}, "df": 2}}}}}}}}, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}}, "df": 4}}}, "p": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "s": {"4": {"docs": {"acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Opcs4.checks": {"tf": 1}}, "df": 2}, "docs": {}, "df": 0}}, "s": {"docs": {}, "df": 0, "c": {"4": {"docs": {"acmc.trud.extract_opsc4": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}}}}, "c": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "m": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}}, "df": 1}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.clear": {"tf": 1}}, "df": 1}}}}, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}}, "df": 2, "s": {"docs": {"acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 2}}}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1}}, "df": 3}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 2, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodesError.__init__": {"tf": 1}}, "df": 2}}}}}}, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}}, "df": 3}}}}}}}}}}}}, "l": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 4}, "p": {"docs": {}, "df": 0, "y": {"docs": {"acmc.phen.copy": {"tf": 1}}, "df": 1}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "k": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Read2.checks": {"tf": 1}, "acmc.parse.Read3.checks": {"tf": 1}, "acmc.parse.Icd10.checks": {"tf": 1}, "acmc.parse.Snomed.checks": {"tf": 1}, "acmc.parse.Opcs4.checks": {"tf": 1}, "acmc.parse.Atc.checks": {"tf": 1}, "acmc.parse.Med.checks": {"tf": 1}, "acmc.parse.Cprd.checks": {"tf": 1}}, "df": 9}}}}}, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Cprd": {"tf": 1}, "acmc.parse.Cprd.checks": {"tf": 1}}, "df": 2}}}, "s": {"docs": {}, "df": 0, "v": {"docs": {"acmc.phen.CSV_PATH": {"tf": 1}}, "df": 1}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.create_map_directories": {"tf": 1}}, "df": 1}}}}}}, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.export": {"tf": 1}}, "df": 3}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}}, "df": 2}}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}}}, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.extract_concepts": {"tf": 1}, "acmc.trud.extract_icd10": {"tf": 1}, "acmc.trud.extract_opsc4": {"tf": 1}, "acmc.trud.extract_nhs_data_migrations": {"tf": 1}, "acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 5}}}}}}, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.InvalidCodesException.error": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.phen.PhenValidationException.validation_errors": {"tf": 1}}, "df": 1}}}}}}, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 5}}}}, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 1}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}}}}}}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1}}, "df": 1, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.trud.install": {"tf": 1}}, "df": 2}}}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}, "acmc.phen.init": {"tf": 1}}, "df": 6}}, "v": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.error": {"tf": 1}}, "df": 3}}}}}}}}}}}}}}}}}}}, "c": {"docs": {"acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}}, "df": 1, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 1}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {"acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 1}}}}}, "c": {"docs": {}, "df": 0, "d": {"1": {"0": {"docs": {"acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Icd10.checks": {"tf": 1}, "acmc.trud.extract_icd10": {"tf": 1}}, "df": 3}, "docs": {}, "df": 0}, "docs": {}, "df": 0}}}, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.name": {"tf": 1}}, "df": 1}}}, "h": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.extract_nhs_data_migrations": {"tf": 1}, "acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 2}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"2": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read2.checks": {"tf": 1}}, "df": 2}, "3": {"docs": {"acmc.parse.Read3": {"tf": 1}, "acmc.parse.Read3.checks": {"tf": 1}}, "df": 2}, "docs": {"acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 1}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}}}}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "c": {"docs": {"acmc.parse.Atc": {"tf": 1}, "acmc.parse.Atc.checks": {"tf": 1}}, "df": 2}}, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 1}}}}}}}, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 1}}, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}, "b": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 1}}}}, "o": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 1}}}}}}}, "h": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "h": {"docs": {"acmc.trud.validate_download_hash": {"tf": 1}}, "df": 1}}}}, "u": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "z": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "p": {"docs": {"acmc.trud.unzip_download": {"tf": 1}}, "df": 1}}}}}, "q": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 2}}}}}}}}}}}}}}, "fullname": {"root": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}}, "df": 5, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "c": {"docs": {"acmc": {"tf": 1}, "acmc.logging_config": {"tf": 1}, "acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}, "acmc.main": {"tf": 1}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.main.main": {"tf": 1}, "acmc.omop": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.install": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.parse": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.error": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Proto.name": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read2.checks": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Read3.checks": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Icd10.checks": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Snomed.checks": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Opcs4.checks": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Atc.checks": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Med.checks": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.parse.Cprd.checks": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}, "acmc.phen": {"tf": 1}, "acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}, "acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.validation_errors": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}, "acmc.phen.diff": {"tf": 1}, "acmc.trud": {"tf": 1}, "acmc.trud.FQDN": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}, "acmc.trud.unzip_download": {"tf": 1}, "acmc.trud.extract_icd10": {"tf": 1}, "acmc.trud.extract_opsc4": {"tf": 1}, "acmc.trud.extract_nhs_data_migrations": {"tf": 1}, "acmc.trud.extract_nhs_read_browser": {"tf": 1}, "acmc.trud.create_map_directories": {"tf": 1}, "acmc.trud.install": {"tf": 1}, "acmc.util": {"tf": 1}, "acmc.util.QuotedDumper": {"tf": 1}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 117}}, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 1}}}}}}, "t": {"docs": {}, "df": 0, "c": {"docs": {"acmc.parse.Atc": {"tf": 1}, "acmc.parse.Atc.checks": {"tf": 1}}, "df": 2}}}, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 4}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}}}, "e": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "l": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}}, "df": 1}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}}, "df": 1}}}}, "c": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1}}, "df": 7}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}}, "df": 2, "s": {"docs": {"acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 2}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 2, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodesError.__init__": {"tf": 1}}, "df": 2}}}}}}, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}}, "df": 3}}}}}}}}}}}}, "l": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 4}, "p": {"docs": {}, "df": 0, "y": {"docs": {"acmc.phen.copy": {"tf": 1}}, "df": 1}}}, "d": {"docs": {}, "df": 0, "m": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}}, "df": 1}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.clear": {"tf": 1}}, "df": 1}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "k": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Read2.checks": {"tf": 1}, "acmc.parse.Read3.checks": {"tf": 1}, "acmc.parse.Icd10.checks": {"tf": 1}, "acmc.parse.Snomed.checks": {"tf": 1}, "acmc.parse.Opcs4.checks": {"tf": 1}, "acmc.parse.Atc.checks": {"tf": 1}, "acmc.parse.Med.checks": {"tf": 1}, "acmc.parse.Cprd.checks": {"tf": 1}}, "df": 9}}}}}, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Cprd": {"tf": 1}, "acmc.parse.Cprd.checks": {"tf": 1}}, "df": 2}}}, "s": {"docs": {}, "df": 0, "v": {"docs": {"acmc.phen.CSV_PATH": {"tf": 1}}, "df": 1}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.create_map_directories": {"tf": 1}}, "df": 1}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 6}}}}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.delete": {"tf": 1}}, "df": 1}}}}}, "b": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}}, "df": 2}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {"acmc.trud.extract_nhs_data_migrations": {"tf": 1}}, "df": 1, "b": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1}}, "df": 1}}}}}}}, "i": {"docs": {}, "df": 0, "r": {"docs": {"acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}}, "df": 5, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.create_map_directories": {"tf": 1}}, "df": 1}}}}}}}}}, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}}, "df": 1}}}}, "f": {"docs": {}, "df": 0, "f": {"docs": {"acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}, "acmc.phen.diff": {"tf": 1}}, "df": 4}}}, "o": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.download_release_file": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}, "acmc.trud.unzip_download": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.trud.DOWNLOADS_PATH": {"tf": 1}}, "df": 1}}}}}}}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 9, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}}, "df": 1}}}}, "s": {"docs": {"acmc.phen.diff_map_files": {"tf": 1}}, "df": 1}}}}, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "k": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}, "q": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}}, "df": 3, "u": {"docs": {}, "df": 0, "p": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}}, "df": 1}}}}}}}, "u": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}}, "df": 1}}}}}}}}, "n": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Snomed.checks": {"tf": 1}}, "df": 2}}}}}, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}}, "df": 1}}}}, "c": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "a": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.main": {"tf": 1}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.main.main": {"tf": 1.4142135623730951}}, "df": 3}}, "p": {"docs": {"acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.trud.create_map_directories": {"tf": 1}}, "df": 4}}, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Med": {"tf": 1}, "acmc.parse.Med.checks": {"tf": 1}}, "df": 2}}, "i": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.extract_nhs_data_migrations": {"tf": 1}}, "df": 1}}}}}}}}}}, "w": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "k": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.write_version_file": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}}, "df": 2}}}}}, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 11}}, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.error": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Proto.name": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read2.checks": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Read3.checks": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Icd10.checks": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Snomed.checks": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Opcs4.checks": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Atc.checks": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Med.checks": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.parse.Cprd.checks": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}}, "df": 34}}}}, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Proto.name": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}}, "df": 8}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen": {"tf": 1}, "acmc.phen.PHEN_DIR": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1.4142135623730951}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}, "acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.validation_errors": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 1}}, "df": 37, "v": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.validation_errors": {"tf": 1}}, "df": 3}}}}}}}}}}}}}}}}}}}}}}, "u": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}}, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.OMOP_CDM_Version": {"tf": 1.4142135623730951}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1.4142135623730951}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1.4142135623730951}, "acmc.omop.install": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}}, "df": 19}}}, "p": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "s": {"4": {"docs": {"acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Opcs4.checks": {"tf": 1}}, "df": 2}, "docs": {}, "df": 0}}, "s": {"docs": {}, "df": 0, "c": {"4": {"docs": {"acmc.trud.extract_opsc4": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}}}}, "v": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {"acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}}, "df": 6, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}}}}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}}, "df": 10}}}}, "i": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "y": {"docs": {"acmc.parse.Proto.verify": {"tf": 1}}, "df": 1}}}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PhenValidationException.validation_errors": {"tf": 1}}, "df": 1}}}, "e": {"docs": {"acmc.phen.validate": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}}, "df": 2}}}}}}}}, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.export": {"tf": 1}}, "df": 3}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}}, "df": 2}}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}}}, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.extract_concepts": {"tf": 1}, "acmc.trud.extract_icd10": {"tf": 1}, "acmc.trud.extract_opsc4": {"tf": 1}, "acmc.trud.extract_nhs_data_migrations": {"tf": 1}, "acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 5}}}}}}, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.InvalidCodesException.error": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.phen.PhenValidationException.validation_errors": {"tf": 1}}, "df": 1}}}}}}, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 5}}}}, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 1}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}}}}}}}, "u": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud": {"tf": 1}, "acmc.trud.FQDN": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}, "acmc.trud.unzip_download": {"tf": 1}, "acmc.trud.extract_icd10": {"tf": 1}, "acmc.trud.extract_opsc4": {"tf": 1}, "acmc.trud.extract_nhs_data_migrations": {"tf": 1}, "acmc.trud.extract_nhs_read_browser": {"tf": 1}, "acmc.trud.create_map_directories": {"tf": 1}, "acmc.trud.install": {"tf": 1}}, "df": 17}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1}}, "df": 1, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.trud.install": {"tf": 1}}, "df": 2}}}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}, "acmc.phen.init": {"tf": 1}}, "df": 6}}, "v": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1}, "acmc.parse.InvalidCodesException.error": {"tf": 1}}, "df": 3}}}}}}}}}}}}}}}}}}}, "c": {"docs": {"acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}}, "df": 1, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 1}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {"acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 1}}}}}, "c": {"docs": {}, "df": 0, "d": {"1": {"0": {"docs": {"acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Icd10.checks": {"tf": 1}, "acmc.trud.extract_icd10": {"tf": 1}}, "df": 3}, "docs": {}, "df": 0}, "docs": {}, "df": 0}}}, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.name": {"tf": 1}}, "df": 1}}}, "h": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.extract_nhs_data_migrations": {"tf": 1}, "acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 2}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"2": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read2.checks": {"tf": 1}}, "df": 2}, "3": {"docs": {"acmc.parse.Read3": {"tf": 1}, "acmc.parse.Read3.checks": {"tf": 1}}, "df": 2}, "docs": {"acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 1}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}}}}}}, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 1}}, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}, "b": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 1}}}}, "o": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.trud.extract_nhs_read_browser": {"tf": 1}}, "df": 1}}}}}}}, "h": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "h": {"docs": {"acmc.trud.validate_download_hash": {"tf": 1}}, "df": 1}}}}, "u": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "z": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "p": {"docs": {"acmc.trud.unzip_download": {"tf": 1}}, "df": 1}}}}, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {"acmc.util": {"tf": 1}, "acmc.util.QuotedDumper": {"tf": 1}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 3}}}}, "q": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 2}}}}}}}}}}}}}}, "annotation": {"root": {"docs": {"acmc.parse.Proto.checks": {"tf": 1}, "acmc.parse.Proto.name": {"tf": 1}}, "df": 2, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "[": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "[": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.Proto.checks": {"tf": 1}}, "df": 1}}}}}}}}}}}}}}, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.parse.Proto.checks": {"tf": 1.4142135623730951}}, "df": 1}}}}}}, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "[": {"docs": {}, "df": 0, "[": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.checks": {"tf": 1.4142135623730951}}, "df": 1}}}}}}}}}}}}}}}, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.checks": {"tf": 2}}, "df": 1}}}}, "f": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.checks": {"tf": 1.7320508075688772}}, "df": 1}}}}}, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.checks": {"tf": 1.7320508075688772}}, "df": 1}}}}}}}}}, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.checks": {"tf": 1.4142135623730951}}, "df": 1}}}}, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.parse.Proto.checks": {"tf": 1}}, "df": 1, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "b": {"docs": {"acmc.parse.Proto.checks": {"tf": 1}}, "df": 1}}}}}}}, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.checks": {"tf": 1.4142135623730951}}, "df": 1}}}}}, "t": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.Proto.name": {"tf": 1}}, "df": 1}}}}}, "default_value": {"root": {"1": {"4": {"4": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}, "5": {"4": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "5": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}, "7": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "2": {"1": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "5": {"4": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.vocabularies": {"tf": 1}}, "df": 5}, "5": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "7": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}, "7": {"0": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "5": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}, "docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1.4142135623730951}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.OMOP_CDM_Version": {"tf": 1.4142135623730951}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1.4142135623730951}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1.4142135623730951}, "acmc.omop.vocabularies": {"tf": 7.211102550927978}, "acmc.omop.omop_vocab_types": {"tf": 3}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1.4142135623730951}, "acmc.phen.PHEN_DIR": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1.4142135623730951}, "acmc.phen.MAP_DIR": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1.4142135623730951}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1.4142135623730951}, "acmc.phen.CONFIG_FILE": {"tf": 1.4142135623730951}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1.4142135623730951}, "acmc.phen.SPLIT_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.CODES_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.COL_ACTIONS": {"tf": 1.4142135623730951}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1.4142135623730951}, "acmc.phen.CONFIG_SCHEMA": {"tf": 9.327379053088816}, "acmc.trud.FQDN": {"tf": 1.4142135623730951}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 37, "x": {"2": {"7": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1.4142135623730951}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1.4142135623730951}, "acmc.omop.VOCAB_PATH": {"tf": 1.4142135623730951}, "acmc.omop.OMOP_CDM_Version": {"tf": 1.4142135623730951}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1.4142135623730951}, "acmc.omop.DB_PATH": {"tf": 1.4142135623730951}, "acmc.omop.VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.omop.VERSION_PATH": {"tf": 1.4142135623730951}, "acmc.omop.EXPORT_FILE": {"tf": 1.4142135623730951}, "acmc.omop.vocabularies": {"tf": 9.273618495495704}, "acmc.omop.omop_vocab_types": {"tf": 5.0990195135927845}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 3.4641016151377544}, "acmc.phen.PHEN_DIR": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPTS_DIR": {"tf": 1.4142135623730951}, "acmc.phen.MAP_DIR": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1.4142135623730951}, "acmc.phen.CSV_PATH": {"tf": 1.4142135623730951}, "acmc.phen.OMOP_PATH": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 2.449489742783178}, "acmc.phen.CONFIG_FILE": {"tf": 1.4142135623730951}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 2.449489742783178}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1.4142135623730951}, "acmc.phen.SPLIT_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.CODES_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.COL_ACTIONS": {"tf": 2.449489742783178}, "acmc.phen.CODE_FILE_TYPES": {"tf": 2.449489742783178}, "acmc.phen.CONFIG_SCHEMA": {"tf": 12.806248474865697}, "acmc.trud.FQDN": {"tf": 1.4142135623730951}, "acmc.trud.VOCAB_PATH": {"tf": 1.4142135623730951}, "acmc.trud.VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.trud.VERSION_PATH": {"tf": 1.4142135623730951}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1.4142135623730951}, "acmc.trud.PROCESSED_PATH": {"tf": 1.4142135623730951}}, "df": 37}, "docs": {}, "df": 0}, "docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 1, "x": {"docs": {"acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 1}}}}, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "c": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}}, "df": 1}}, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}, "t": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}, "c": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1.4142135623730951}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 4}}, "l": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}}, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}}, "df": 1}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1.4142135623730951}}, "df": 1}}}}, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 11}}}}}}}}, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PHEN_DIR": {"tf": 1}}, "df": 1, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}}, "df": 2}}, "h": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}, "w": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "k": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}}, "df": 1, "/": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}}, "df": 1}}}}}}}}}}}}}}, "v": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {"acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}}, "df": 1, "/": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.VOCAB_PATH": {"tf": 1}}, "df": 1, "/": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}}, "df": 2}}}}}}}}}, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.VOCAB_PATH": {"tf": 1}}, "df": 1, "/": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.VERSION_PATH": {"tf": 1}}, "df": 1}}}}, "d": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud.DOWNLOADS_PATH": {"tf": 1}}, "df": 1}}}}}}}}}, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 1}}}}}}}}}}}}}}}, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}, "y": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1.7320508075688772}}, "df": 1}}}}}}}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.vocabularies": {"tf": 1.4142135623730951}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}}, "df": 7}}}}}}}, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 4}}}, "h": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {"acmc.omop.vocabularies": {"tf": 1.4142135623730951}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "/": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "/": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}}}}}}}}}}}}}, "p": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "s": {"4": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1.4142135623730951}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 4}, "docs": {}, "df": 0}}}, "f": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}, "s": {"docs": {}, "df": 0, "q": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}}, "df": 3}}}}}, "o": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}, "n": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1.4142135623730951}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 4}}}}}, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "y": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}}, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 2}}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}, "t": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 3, "/": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "v": {"docs": {"acmc.phen.CSV_PATH": {"tf": 1}}, "df": 1}}}, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.phen.OMOP_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "c": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "a": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 2.6457513110645907}}, "df": 1}}}}}, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 3}}, "df": 1}}}}}}, "y": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}}, "df": 6}}}, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.EXPORT_FILE": {"tf": 1}}, "df": 1}}}}}, "t": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}, "k": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.trud.FQDN": {"tf": 1}}, "df": 2}}, "h": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1, ":": {"docs": {}, "df": 0, "/": {"docs": {}, "df": 0, "/": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}}}}}}}}, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}, "c": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "m": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "y": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 2}}}}}}}, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}}, "df": 1}}}, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 5, "s": {"docs": {"acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}}, "df": 2}}}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.CONFIG_FILE": {"tf": 1}}, "df": 1}}}}, "l": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1.7320508075688772}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 5, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 2}}}}, "s": {"docs": {}, "df": 0, "v": {"docs": {"acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 1}}}, "i": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.vocabularies": {"tf": 3.3166247903554}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 2}, "c": {"docs": {}, "df": 0, "d": {"1": {"0": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 3, "c": {"docs": {}, "df": 0, "m": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}}, "df": 2}}}, "docs": {}, "df": 0}, "9": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "m": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}, "docs": {}, "df": 0}}, "s": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.vocabularies": {"tf": 3.3166247903554}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1.4142135623730951}}, "df": 2}}}, "h": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.vocabularies": {"tf": 1.4142135623730951}, "acmc.trud.FQDN": {"tf": 1}}, "df": 2}}, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1.7320508075688772}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"2": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 3}, "3": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 3}, "docs": {"acmc.omop.omop_vocab_types": {"tf": 1}}, "df": 1, "v": {"2": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}}}, "q": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 3.4641016151377544}}, "df": 1}}}}}}, "g": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1.4142135623730951}}, "df": 1}}}, "f": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}}}}, "d": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1.7320508075688772}}, "df": 1, "m": {"docs": {}, "df": 0, "+": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}, "i": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 3}}}}, "c": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 2.6457513110645907}}, "df": 1}}, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}}}}, "b": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "k": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}}, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 4.242640687119285}}, "df": 1}}}, "r": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 3.3166247903554}}, "df": 1}, "d": {"docs": {"acmc.trud.VERSION_FILE": {"tf": 1}}, "df": 1}}}}, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}}, "df": 1}, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}}, "a": {"docs": {}, "df": 0, "p": {"docs": {"acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 3}, "j": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}}, "df": 1}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 1}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}}, "df": 1}}}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}}, "signature": {"root": {"2": {"0": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}, "docs": {}, "df": 0}, "3": {"9": {"docs": {"acmc.parse.CodeTypeParser.__init__": {"tf": 1.4142135623730951}, "acmc.phen.publish": {"tf": 1.4142135623730951}}, "df": 2}, "docs": {}, "df": 0}, "docs": {"acmc.logging_config.setup_logger": {"tf": 4.47213595499958}, "acmc.logging_config.set_log_level": {"tf": 3.7416573867739413}, "acmc.main.main": {"tf": 2.6457513110645907}, "acmc.omop.install": {"tf": 4.69041575982343}, "acmc.omop.write_version_file": {"tf": 3.7416573867739413}, "acmc.omop.clear": {"tf": 4.242640687119285}, "acmc.omop.delete": {"tf": 4.242640687119285}, "acmc.omop.table_exists": {"tf": 5.291502622129181}, "acmc.omop.vocab_exists": {"tf": 5.291502622129181}, "acmc.omop.concept_set_exist": {"tf": 5.291502622129181}, "acmc.omop.export": {"tf": 7.483314773547883}, "acmc.parse.CodesError.__init__": {"tf": 6.324555320336759}, "acmc.parse.InvalidCodesException.__init__": {"tf": 2.8284271247461903}, "acmc.parse.Proto.__init__": {"tf": 5.916079783099616}, "acmc.parse.Proto.raise_exception": {"tf": 4.242640687119285}, "acmc.parse.Proto.in_database": {"tf": 8.717797887081348}, "acmc.parse.Proto.process": {"tf": 8.306623862918075}, "acmc.parse.Proto.verify": {"tf": 6.48074069840786}, "acmc.parse.CodeTypeParser.__init__": {"tf": 5.5677643628300215}, "acmc.phen.PhenValidationException.__init__": {"tf": 4}, "acmc.phen.init": {"tf": 4.69041575982343}, "acmc.phen.fork": {"tf": 6.48074069840786}, "acmc.phen.validate": {"tf": 3.7416573867739413}, "acmc.phen.translate_codes": {"tf": 7.681145747868608}, "acmc.phen.write_vocab_version": {"tf": 4.242640687119285}, "acmc.phen.map": {"tf": 4.69041575982343}, "acmc.phen.publish": {"tf": 6.782329983125268}, "acmc.phen.export": {"tf": 4.69041575982343}, "acmc.phen.copy": {"tf": 5.477225575051661}, "acmc.phen.extract_concepts": {"tf": 5.385164807134504}, "acmc.phen.diff_config": {"tf": 4.898979485566356}, "acmc.phen.diff_map_files": {"tf": 5.656854249492381}, "acmc.phen.diff_phen": {"tf": 7.937253933193772}, "acmc.phen.diff": {"tf": 6.164414002968976}, "acmc.trud.get_releases": {"tf": 5.656854249492381}, "acmc.trud.download_release_file": {"tf": 6.928203230275509}, "acmc.trud.validate_download_hash": {"tf": 4.69041575982343}, "acmc.trud.unzip_download": {"tf": 3.7416573867739413}, "acmc.trud.extract_icd10": {"tf": 2.6457513110645907}, "acmc.trud.extract_opsc4": {"tf": 2.6457513110645907}, "acmc.trud.extract_nhs_data_migrations": {"tf": 2.6457513110645907}, "acmc.trud.extract_nhs_read_browser": {"tf": 2.6457513110645907}, "acmc.trud.create_map_directories": {"tf": 2.6457513110645907}, "acmc.trud.install": {"tf": 2.6457513110645907}, "acmc.util.QuotedDumper.increase_indent": {"tf": 5.0990195135927845}}, "df": 45, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2}}, "e": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "l": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.Proto.process": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}}, "df": 2}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2}, "c": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 1}}}}}}}}}, "d": {"docs": {"acmc.omop.vocab_exists": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 3}, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {"acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}}, "df": 3}}}}, "o": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.export": {"tf": 1}}, "df": 2}}}, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.parse.Proto.__init__": {"tf": 1}}, "df": 1}}}}}}}, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}}, "d": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1}}}}}}, "l": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 1.4142135623730951}}, "df": 4}}}, "z": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.install": {"tf": 1}}, "df": 1}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}, "acmc.trud.validate_download_hash": {"tf": 1}, "acmc.trud.unzip_download": {"tf": 1}}, "df": 7}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1.7320508075688772}, "acmc.parse.Proto.process": {"tf": 1.4142135623730951}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}}, "df": 4}}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.get_releases": {"tf": 1}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1.4142135623730951}}, "df": 2}}}}, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "w": {"docs": {"acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 1}}}}, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.phen.init": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 2}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}, "acmc.phen.map": {"tf": 1.4142135623730951}, "acmc.phen.publish": {"tf": 2}, "acmc.phen.export": {"tf": 1.4142135623730951}, "acmc.phen.copy": {"tf": 1.7320508075688772}, "acmc.phen.extract_concepts": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 2}, "acmc.trud.get_releases": {"tf": 1.4142135623730951}, "acmc.trud.download_release_file": {"tf": 1.7320508075688772}, "acmc.trud.validate_download_hash": {"tf": 1.4142135623730951}, "acmc.trud.unzip_download": {"tf": 1}}, "df": 25}}, "q": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"3": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}}, "df": 3}, "docs": {}, "df": 0}}}}}, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 2}, "l": {"docs": {}, "df": 0, "f": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1}}, "df": 5}}}, "o": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}}}}}}, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 1.4142135623730951}}, "df": 8}}}}}}, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {"acmc.omop.vocab_exists": {"tf": 1}}, "df": 1, "/": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "/": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.CodeTypeParser.__init__": {"tf": 1}}, "df": 1}}}}}}}}}}}}}}}}}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.PhenValidationException.__init__": {"tf": 1}}, "df": 1}}}}}}}}}}, "d": {"docs": {}, "df": 0, "b": {"docs": {"acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}}, "df": 3}, "i": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.export": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1.4142135623730951}, "acmc.phen.diff_config": {"tf": 1.4142135623730951}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 4}}, "r": {"docs": {"acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 1.4142135623730951}}, "df": 9}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {"acmc.phen.extract_concepts": {"tf": 1}}, "df": 1, "f": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1.7320508075688772}, "acmc.parse.Proto.process": {"tf": 1.4142135623730951}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}}, "df": 4}}}}}}}}, "f": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.validate_download_hash": {"tf": 1}, "acmc.trud.unzip_download": {"tf": 1}}, "df": 2}}}}}}}}}}}, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.omop.clear": {"tf": 1.4142135623730951}, "acmc.omop.delete": {"tf": 1.4142135623730951}, "acmc.omop.export": {"tf": 2.23606797749979}, "acmc.parse.Proto.__init__": {"tf": 1.4142135623730951}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1.4142135623730951}, "acmc.phen.diff_map_files": {"tf": 2}, "acmc.phen.diff_phen": {"tf": 2.449489742783178}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 11, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "b": {"docs": {"acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.omop.export": {"tf": 1.7320508075688772}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1.4142135623730951}, "acmc.phen.diff_phen": {"tf": 1.7320508075688772}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 11}}}}, "c": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}, "n": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1.7320508075688772}, "acmc.parse.Proto.process": {"tf": 1.4142135623730951}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}}, "df": 4}}}}}, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.CodeTypeParser.__init__": {"tf": 1}}, "df": 1}}}}}}}, "e": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "x": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1}}}}}, "o": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.parse.CodeTypeParser.__init__": {"tf": 1}}, "df": 1}}}}}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.write_vocab_version": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 1.4142135623730951}}, "df": 10}}}}, "c": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.vocab_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}}, "df": 3}}}}}, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 2}}}}, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.extract_concepts": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1.4142135623730951}}, "df": 2}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.map": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1.4142135623730951}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.Proto.in_database": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1.4142135623730951}, "acmc.parse.Proto.verify": {"tf": 1.4142135623730951}}, "df": 5}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1.7320508075688772}, "acmc.parse.Proto.process": {"tf": 1.4142135623730951}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}}, "df": 4}}, "l": {"docs": {"acmc.parse.Proto.in_database": {"tf": 1}}, "df": 1}}}, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 1}}, "df": 1}}}, "r": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.map": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 3}}}}}, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.map": {"tf": 1}}, "df": 3}}}, "r": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Proto.__init__": {"tf": 1}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1}}, "df": 2}}}, "u": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.process": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 2}}}}}, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 4}}}, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 2}, "acmc.parse.Proto.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}}, "df": 3}}}, "e": {"docs": {}, "df": 0, "w": {"docs": {"acmc.phen.fork": {"tf": 1}, "acmc.phen.diff_config": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}}, "df": 4}}}, "b": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}}, "df": 3}}}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.export": {"tf": 1}, "acmc.phen.diff_map_files": {"tf": 1.4142135623730951}}, "df": 2}, "s": {"docs": {}, "df": 0, "k": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}}, "df": 1}}}, "e": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {"acmc.omop.export": {"tf": 1}}, "df": 1}}}}}}, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.CodesError.__init__": {"tf": 1}, "acmc.phen.PhenValidationException.__init__": {"tf": 1}}, "df": 2}}}}}}, "s": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}, "e": {"docs": {}, "df": 0, "x": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.export": {"tf": 1}}, "df": 1}}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}}}}, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.parse.InvalidCodesException.__init__": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.phen.PhenValidationException.__init__": {"tf": 1}}, "df": 1}}}}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.init": {"tf": 1}, "acmc.phen.publish": {"tf": 1}}, "df": 2}}}}, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.diff_phen": {"tf": 1}}, "df": 1}}}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.download_release_file": {"tf": 1.4142135623730951}}, "df": 1}}}}}}}, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "l": {"docs": {"acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1.4142135623730951}, "acmc.phen.publish": {"tf": 1}}, "df": 3}}, "p": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {"acmc.phen.fork": {"tf": 1.4142135623730951}}, "df": 1}}}}}}}}, "a": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "i": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}, "k": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "y": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}, "j": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1}}}}, "h": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "h": {"docs": {"acmc.trud.validate_download_hash": {"tf": 1}}, "df": 1}}}}}}, "bases": {"root": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 2}}}}}}}}, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 2}}}}}}}}}, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}}}, "y": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "l": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}, "d": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.util.QuotedDumper": {"tf": 1.4142135623730951}}, "df": 1}}}}}}}}, "doc": {"root": {"3": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}, "docs": {"acmc": {"tf": 1.7320508075688772}, "acmc.logging_config": {"tf": 2.23606797749979}, "acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1.7320508075688772}, "acmc.logging_config.setup_logger": {"tf": 1.7320508075688772}, "acmc.logging_config.set_log_level": {"tf": 3.7416573867739413}, "acmc.main": {"tf": 2}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1.4142135623730951}, "acmc.main.main": {"tf": 1.7320508075688772}, "acmc.omop": {"tf": 2.23606797749979}, "acmc.omop.VOCAB_PATH": {"tf": 1.4142135623730951}, "acmc.omop.OMOP_CDM_Version": {"tf": 1.4142135623730951}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1.4142135623730951}, "acmc.omop.DB_PATH": {"tf": 1.4142135623730951}, "acmc.omop.VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.omop.VERSION_PATH": {"tf": 1.4142135623730951}, "acmc.omop.EXPORT_FILE": {"tf": 1.4142135623730951}, "acmc.omop.vocabularies": {"tf": 1.4142135623730951}, "acmc.omop.omop_vocab_types": {"tf": 1.4142135623730951}, "acmc.omop.install": {"tf": 6.082762530298219}, "acmc.omop.write_version_file": {"tf": 3.605551275463989}, "acmc.omop.clear": {"tf": 4.795831523312719}, "acmc.omop.delete": {"tf": 4.795831523312719}, "acmc.omop.table_exists": {"tf": 5.0990195135927845}, "acmc.omop.vocab_exists": {"tf": 5.0990195135927845}, "acmc.omop.concept_set_exist": {"tf": 5.0990195135927845}, "acmc.omop.export": {"tf": 6}, "acmc.parse": {"tf": 2}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1.4142135623730951}, "acmc.parse.CodesError": {"tf": 1.4142135623730951}, "acmc.parse.CodesError.__init__": {"tf": 1.7320508075688772}, "acmc.parse.InvalidCodesException": {"tf": 1.4142135623730951}, "acmc.parse.InvalidCodesException.__init__": {"tf": 1.7320508075688772}, "acmc.parse.InvalidCodesException.error": {"tf": 1.7320508075688772}, "acmc.parse.Proto": {"tf": 3.872983346207417}, "acmc.parse.Proto.__init__": {"tf": 1.7320508075688772}, "acmc.parse.Proto.checks": {"tf": 1.7320508075688772}, "acmc.parse.Proto.name": {"tf": 1.7320508075688772}, "acmc.parse.Proto.raise_exception": {"tf": 1.4142135623730951}, "acmc.parse.Proto.in_database": {"tf": 1.7320508075688772}, "acmc.parse.Proto.process": {"tf": 1.4142135623730951}, "acmc.parse.Proto.verify": {"tf": 1.4142135623730951}, "acmc.parse.Read2": {"tf": 1.7320508075688772}, "acmc.parse.Read2.checks": {"tf": 1.7320508075688772}, "acmc.parse.Read3": {"tf": 3.872983346207417}, "acmc.parse.Read3.checks": {"tf": 1.7320508075688772}, "acmc.parse.Icd10": {"tf": 3.872983346207417}, "acmc.parse.Icd10.checks": {"tf": 1.7320508075688772}, "acmc.parse.Snomed": {"tf": 3.872983346207417}, "acmc.parse.Snomed.checks": {"tf": 1.7320508075688772}, "acmc.parse.Opcs4": {"tf": 3.872983346207417}, "acmc.parse.Opcs4.checks": {"tf": 1.7320508075688772}, "acmc.parse.Atc": {"tf": 3.872983346207417}, "acmc.parse.Atc.checks": {"tf": 1.7320508075688772}, "acmc.parse.Med": {"tf": 3.872983346207417}, "acmc.parse.Med.checks": {"tf": 1.7320508075688772}, "acmc.parse.Cprd": {"tf": 3.872983346207417}, "acmc.parse.Cprd.checks": {"tf": 1.7320508075688772}, "acmc.parse.CodeTypeParser": {"tf": 1.4142135623730951}, "acmc.parse.CodeTypeParser.__init__": {"tf": 1.7320508075688772}, "acmc.parse.CodeTypeParser.code_types": {"tf": 1.7320508075688772}, "acmc.phen": {"tf": 2.23606797749979}, "acmc.phen.PHEN_DIR": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPTS_DIR": {"tf": 1.4142135623730951}, "acmc.phen.MAP_DIR": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1.4142135623730951}, "acmc.phen.CSV_PATH": {"tf": 1.4142135623730951}, "acmc.phen.OMOP_PATH": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1.4142135623730951}, "acmc.phen.CONFIG_FILE": {"tf": 1.4142135623730951}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1.4142135623730951}, "acmc.phen.SPLIT_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.CODES_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1.4142135623730951}, "acmc.phen.COL_ACTIONS": {"tf": 1.4142135623730951}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1.4142135623730951}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1.4142135623730951}, "acmc.phen.PhenValidationException": {"tf": 1.4142135623730951}, "acmc.phen.PhenValidationException.__init__": {"tf": 1.7320508075688772}, "acmc.phen.PhenValidationException.validation_errors": {"tf": 1.7320508075688772}, "acmc.phen.init": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 6.928203230275509}, "acmc.phen.validate": {"tf": 1.4142135623730951}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}, "acmc.phen.write_vocab_version": {"tf": 1.7320508075688772}, "acmc.phen.map": {"tf": 1.7320508075688772}, "acmc.phen.publish": {"tf": 1.4142135623730951}, "acmc.phen.export": {"tf": 1.4142135623730951}, "acmc.phen.copy": {"tf": 1.4142135623730951}, "acmc.phen.extract_concepts": {"tf": 1.7320508075688772}, "acmc.phen.diff_config": {"tf": 1.7320508075688772}, "acmc.phen.diff_map_files": {"tf": 1.7320508075688772}, "acmc.phen.diff_phen": {"tf": 1.4142135623730951}, "acmc.phen.diff": {"tf": 1.7320508075688772}, "acmc.trud": {"tf": 2.23606797749979}, "acmc.trud.FQDN": {"tf": 1.4142135623730951}, "acmc.trud.VOCAB_PATH": {"tf": 1.4142135623730951}, "acmc.trud.VERSION_FILE": {"tf": 1.4142135623730951}, "acmc.trud.VERSION_PATH": {"tf": 1.4142135623730951}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1.4142135623730951}, "acmc.trud.PROCESSED_PATH": {"tf": 1.4142135623730951}, "acmc.trud.get_releases": {"tf": 1.7320508075688772}, "acmc.trud.download_release_file": {"tf": 1.7320508075688772}, "acmc.trud.validate_download_hash": {"tf": 1.7320508075688772}, "acmc.trud.unzip_download": {"tf": 1.7320508075688772}, "acmc.trud.extract_icd10": {"tf": 1.7320508075688772}, "acmc.trud.extract_opsc4": {"tf": 1.7320508075688772}, "acmc.trud.extract_nhs_data_migrations": {"tf": 1.7320508075688772}, "acmc.trud.extract_nhs_read_browser": {"tf": 1.7320508075688772}, "acmc.trud.create_map_directories": {"tf": 1.7320508075688772}, "acmc.trud.install": {"tf": 1.7320508075688772}, "acmc.util": {"tf": 1.7320508075688772}, "acmc.util.QuotedDumper": {"tf": 1.4142135623730951}, "acmc.util.QuotedDumper.increase_indent": {"tf": 1.7320508075688772}}, "df": 117, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1.7320508075688772}}, "df": 2, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config": {"tf": 1.4142135623730951}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}}, "df": 2}, "d": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}, "a": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.phen.fork": {"tf": 1.4142135623730951}}, "df": 1}}}}, "e": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "l": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1.7320508075688772}}, "df": 1}}}}, "i": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "u": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}}, "df": 1}, "y": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}}, "n": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main": {"tf": 1}}, "df": 1}}, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1.4142135623730951}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 14}}}, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1.7320508075688772}}, "df": 1}}}}}}, "c": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 2, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 2}}}}}}}}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 2}, "acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 7, "s": {"docs": {"acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 2}}}}}, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.omop.export": {"tf": 1}}, "df": 1}}}}}}}, "d": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto": {"tf": 2}, "acmc.parse.Read3": {"tf": 2}, "acmc.parse.Icd10": {"tf": 2}, "acmc.parse.Snomed": {"tf": 2}, "acmc.parse.Opcs4": {"tf": 2}, "acmc.parse.Atc": {"tf": 2}, "acmc.parse.Med": {"tf": 2}, "acmc.parse.Cprd": {"tf": 2}}, "df": 8}}}}}}, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}}}}}}}}}, "m": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "d": {"docs": {"acmc.main": {"tf": 1}}, "df": 1}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.diff_phen": {"tf": 1}}, "df": 1}}}}}, "d": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 4}}}, "e": {"docs": {"acmc.parse": {"tf": 1}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 4, "s": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1.4142135623730951}, "acmc.parse.Read2": {"tf": 1.4142135623730951}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}}, "df": 4}}}, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}}}}}, "l": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 4}}}}, "p": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.copy": {"tf": 1}}, "df": 1}}}}, "d": {"docs": {}, "df": 0, "m": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}}, "df": 1}}, "s": {"docs": {}, "df": 0, "v": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.export": {"tf": 1.7320508075688772}, "acmc.phen.CSV_PATH": {"tf": 1}}, "df": 3}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.clear": {"tf": 1}}, "df": 1}}}}, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 5, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse": {"tf": 1}}, "df": 1}}}}}, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1, "d": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "k": {"docs": {"acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.vocab_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 5, "s": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 9}}}}, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8, "s": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.table_exists": {"tf": 1.7320508075688772}, "acmc.omop.vocab_exists": {"tf": 1.7320508075688772}, "acmc.omop.concept_set_exist": {"tf": 1.7320508075688772}}, "df": 3}}}}, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}, "acmc.util.QuotedDumper": {"tf": 1}}, "df": 4}}}}}, "a": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1, "n": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}}, "df": 1}}}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.create_map_directories": {"tf": 1}}, "df": 1}}}}}}, "p": {"docs": {}, "df": 0, "y": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.main": {"tf": 1}, "acmc.omop": {"tf": 1}, "acmc.parse": {"tf": 1}, "acmc.phen": {"tf": 1}, "acmc.trud": {"tf": 1}}, "df": 6, "t": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 2}}}}}, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.main": {"tf": 1}, "acmc.omop": {"tf": 1}, "acmc.parse": {"tf": 1}, "acmc.phen": {"tf": 1}, "acmc.trud": {"tf": 1}}, "df": 5}}}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto": {"tf": 1.4142135623730951}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Read3": {"tf": 1.4142135623730951}, "acmc.parse.Icd10": {"tf": 1.4142135623730951}, "acmc.parse.Snomed": {"tf": 1.4142135623730951}, "acmc.parse.Opcs4": {"tf": 1.4142135623730951}, "acmc.parse.Atc": {"tf": 1.4142135623730951}, "acmc.parse.Med": {"tf": 1.4142135623730951}, "acmc.parse.Cprd": {"tf": 1.4142135623730951}}, "df": 9, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}}, "df": 1}}}, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 1}}}}}}, "t": {"docs": {}, "df": 0, "o": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}}}, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 4}}}}}}}}}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.fork": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 3, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.phen": {"tf": 1}, "acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1.4142135623730951}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}}, "df": 14, "s": {"docs": {"acmc.phen": {"tf": 1}}, "df": 1}}}}}}}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.clear": {"tf": 1.7320508075688772}, "acmc.omop.delete": {"tf": 1.7320508075688772}, "acmc.omop.export": {"tf": 2.8284271247461903}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 16}}, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse": {"tf": 1}}, "df": 1, "r": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 2}}}}, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}}}}, "u": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}}}}, "t": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.main": {"tf": 1}, "acmc.omop": {"tf": 1}, "acmc.parse": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.phen": {"tf": 1}, "acmc.trud": {"tf": 1}}, "df": 7}}, "e": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1.7320508075688772}, "acmc.main": {"tf": 1.4142135623730951}, "acmc.omop.install": {"tf": 2}, "acmc.omop.write_version_file": {"tf": 1.4142135623730951}, "acmc.omop.clear": {"tf": 2}, "acmc.omop.delete": {"tf": 2}, "acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.vocab_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}, "acmc.omop.export": {"tf": 2}, "acmc.parse.Proto": {"tf": 1.4142135623730951}, "acmc.parse.Read2": {"tf": 1.4142135623730951}, "acmc.parse.Read3": {"tf": 1.4142135623730951}, "acmc.parse.Icd10": {"tf": 1.4142135623730951}, "acmc.parse.Snomed": {"tf": 1.4142135623730951}, "acmc.parse.Opcs4": {"tf": 1.4142135623730951}, "acmc.parse.Atc": {"tf": 1.4142135623730951}, "acmc.parse.Med": {"tf": 1.4142135623730951}, "acmc.parse.Cprd": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 2.8284271247461903}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.publish": {"tf": 1.4142135623730951}, "acmc.phen.diff_phen": {"tf": 1}, "acmc.trud": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1.7320508075688772}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}}, "df": 31, "m": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1}, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 11}}}, "o": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.logging_config.setup_logger": {"tf": 1}, "acmc.omop": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.vocab_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}, "acmc.omop.export": {"tf": 2.23606797749979}, "acmc.parse": {"tf": 1}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.phen.fork": {"tf": 2.6457513110645907}, "acmc.phen.publish": {"tf": 1.4142135623730951}, "acmc.trud": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1.4142135623730951}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}, "acmc.util.QuotedDumper": {"tf": 1}}, "df": 30}, "y": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 7, "s": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1.4142135623730951}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 5}}}}, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 2}}, "df": 1}}}, "r": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 3}}}}, "g": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 2}}}}}, "r": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 11}, "d": {"docs": {"acmc.trud": {"tf": 1.4142135623730951}, "acmc.trud.FQDN": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}}, "df": 8}}, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse": {"tf": 1}}, "df": 1}}}, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}}}}}}}}}, "u": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}}, "w": {"docs": {}, "df": 0, "o": {"docs": {"acmc.phen.diff_phen": {"tf": 1}}, "df": 1}}}, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.logging_config": {"tf": 1.4142135623730951}, "acmc.main": {"tf": 1.4142135623730951}, "acmc.omop": {"tf": 1.4142135623730951}, "acmc.parse": {"tf": 1.4142135623730951}, "acmc.phen": {"tf": 1.4142135623730951}, "acmc.trud": {"tf": 1.4142135623730951}}, "df": 6}}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.main": {"tf": 1}}, "df": 1}}, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop": {"tf": 1}, "acmc.trud": {"tf": 1}}, "df": 2}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.phen": {"tf": 1}}, "df": 1}}}}}}, "p": {"docs": {"acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.trud.create_map_directories": {"tf": 1}}, "df": 3, "p": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 2}}}}}}}, "e": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}}, "df": 3}}}}}, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {"acmc.omop.export": {"tf": 1.7320508075688772}}, "df": 1}}}}}}, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto": {"tf": 1.4142135623730951}, "acmc.parse.Read3": {"tf": 1.4142135623730951}, "acmc.parse.Icd10": {"tf": 1.4142135623730951}, "acmc.parse.Snomed": {"tf": 1.4142135623730951}, "acmc.parse.Opcs4": {"tf": 1.4142135623730951}, "acmc.parse.Atc": {"tf": 1.4142135623730951}, "acmc.parse.Med": {"tf": 1.4142135623730951}, "acmc.parse.Cprd": {"tf": 1.4142135623730951}}, "df": 8}}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8, "s": {"docs": {"acmc.logging_config": {"tf": 1}}, "df": 1}, "/": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.vocabularies": {"tf": 1}, "acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 2}}}}}}}, "a": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 24, "s": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}}}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.delete": {"tf": 1}}, "df": 1}}}}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.install": {"tf": 1}, "acmc.omop.clear": {"tf": 1.7320508075688772}, "acmc.omop.delete": {"tf": 1.7320508075688772}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1.7320508075688772}}, "df": 10}}}}, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.Read2": {"tf": 1.4142135623730951}}, "df": 1}}}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}}, "df": 2}}}}}}}}}, "r": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1, "y": {"docs": {"acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.DEFAULT_PHEN_PATH": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1.4142135623730951}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}, "acmc.trud.PROCESSED_PATH": {"tf": 1}}, "df": 17}, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.trud.create_map_directories": {"tf": 1}}, "df": 2}}}}}}}}}, "c": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.export": {"tf": 1}}, "df": 1, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "y": {"docs": {"acmc.phen.extract_concepts": {"tf": 1}}, "df": 1}}}}}}}}, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}}, "df": 1}}}}, "f": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.diff_phen": {"tf": 1}}, "df": 1}}}}}}}}}, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}}}, "o": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 9, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 4}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}, "w": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.trud.DOWNLOADS_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "b": {"docs": {"acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}}, "df": 2}, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}}}, "f": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.logging_config": {"tf": 1}}, "df": 1}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "y": {"docs": {"acmc.main": {"tf": 1}, "acmc.omop": {"tf": 1}, "acmc.parse": {"tf": 1}, "acmc.phen": {"tf": 1}, "acmc.trud": {"tf": 1}}, "df": 5}}}}}}}}}}}, "l": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "y": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}, "acmc.main": {"tf": 1}, "acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.parse.Read2": {"tf": 1}, "acmc.phen": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 8, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.export": {"tf": 1.4142135623730951}}, "df": 1}}}, "k": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}, "u": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}}, "df": 1}}}}, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.install": {"tf": 2.449489742783178}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 11, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.phen.CONFIG_FILE": {"tf": 1}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}}, "df": 5}}}, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}}, "df": 2}}}}}}}}}}}}}, "s": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.export": {"tf": 1}}, "df": 2}}}, "x": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1}}, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "m": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}}, "df": 11}}}, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 2}, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}}, "l": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}}}, "s": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.phen.fork": {"tf": 1}}, "df": 2, "e": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.concept_set_exist": {"tf": 2}, "acmc.parse": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.CSV_PATH": {"tf": 1}, "acmc.phen.OMOP_PATH": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 8, "u": {"docs": {}, "df": 0, "p": {"docs": {"acmc.logging_config": {"tf": 1}}, "df": 1}}, "s": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}, "acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 1}}, "df": 4}}, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}}, "df": 2}}}}}}, "r": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}}}}}}, "y": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "t": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "r": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}}}, "u": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}}, "df": 1, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}}, "df": 2}}}}}}}}, "q": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}}, "df": 3, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.clear": {"tf": 1.4142135623730951}, "acmc.omop.delete": {"tf": 1.4142135623730951}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}}, "df": 8}}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"3": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}}, "df": 3}, "docs": {}, "df": 0}}}}}, "t": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.fork": {"tf": 2}}, "df": 7, "u": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.init": {"tf": 1}, "acmc.phen.validate": {"tf": 1}}, "df": 2}}}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "s": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}}, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1, "s": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}}}, "n": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.init": {"tf": 1}, "acmc.phen.validate": {"tf": 1}}, "df": 2}}}}}}}, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}}, "df": 1}}}, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {"acmc.phen.fork": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 3}, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.fork": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 2}}}}}}}}, "o": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}}, "df": 2}}}}}, "c": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "a": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}}}}}, "a": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.parse.CodesError": {"tf": 1.4142135623730951}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 2.23606797749979}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1.4142135623730951}, "acmc.phen.export": {"tf": 1.7320508075688772}, "acmc.phen.copy": {"tf": 1.7320508075688772}, "acmc.phen.extract_concepts": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 18, "c": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "c": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}, "acmc.logging_config.setup_logger": {"tf": 1}, "acmc.logging_config.set_log_level": {"tf": 1}, "acmc.main": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}}, "df": 8}}, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {"acmc.logging_config": {"tf": 1}}, "df": 1}}}}, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.SPLIT_COL_ACTION": {"tf": 1}, "acmc.phen.CODES_COL_ACTION": {"tf": 1}, "acmc.phen.DIVIDE_COL_ACTION": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}}, "df": 4}}}}}, "l": {"docs": {}, "df": 0, "l": {"docs": {"acmc.logging_config": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.publish": {"tf": 1}}, "df": 11, "o": {"docs": {}, "df": 0, "w": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}, "p": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.logging_config.DEFAULT_LOG_FILE": {"tf": 1}}, "df": 1}}}}}}, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}, "i": {"docs": {"acmc.trud.FQDN": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}}, "df": 2}}, "s": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 14}, "n": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 6, "d": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.parse.Proto": {"tf": 1.4142135623730951}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read3": {"tf": 1.4142135623730951}, "acmc.parse.Icd10": {"tf": 1.4142135623730951}, "acmc.parse.Snomed": {"tf": 1.4142135623730951}, "acmc.parse.Opcs4": {"tf": 1.4142135623730951}, "acmc.parse.Atc": {"tf": 1.4142135623730951}, "acmc.parse.Med": {"tf": 1.4142135623730951}, "acmc.parse.Cprd": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1}}, "df": 15}, "y": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}, "r": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "s": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}, "acmc.omop.install": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.fork": {"tf": 1}}, "df": 10}}}}}}}, "e": {"docs": {"acmc.omop.export": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}}, "df": 2}}, "t": {"docs": {"acmc.phen.fork": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 3, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}}, "df": 2}}}}}, "i": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}, "d": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}}, "u": {"docs": {}, "df": 0, "p": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}, "acmc.parse": {"tf": 1}}, "df": 2, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "m": {"docs": {"acmc.phen.fork": {"tf": 2.8284271247461903}}, "df": 1}}}}}}, "d": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.publish": {"tf": 1}}, "df": 1}}}}}}, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 2}}, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1.4142135623730951}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "l": {"docs": {"acmc.phen.fork": {"tf": 2}}, "df": 1}}}, "o": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.logging_config.setup_logger": {"tf": 1}}, "df": 1}}}}}}}}, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop": {"tf": 1.4142135623730951}, "acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.OMOP_DB_FILENAME": {"tf": 1}, "acmc.omop.DB_PATH": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.install": {"tf": 1.7320508075688772}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.omop.clear": {"tf": 1.7320508075688772}, "acmc.omop.delete": {"tf": 1.7320508075688772}, "acmc.omop.export": {"tf": 2.449489742783178}, "acmc.phen.OMOP_PATH": {"tf": 1}}, "df": 16}}}, "h": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}}, "df": 2}}}}, "f": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.parse.SUPPORTED_CODE_TYPES": {"tf": 1}, "acmc.parse.Proto": {"tf": 1.4142135623730951}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read3": {"tf": 1.4142135623730951}, "acmc.parse.Icd10": {"tf": 1.4142135623730951}, "acmc.parse.Snomed": {"tf": 1.4142135623730951}, "acmc.parse.Opcs4": {"tf": 1.4142135623730951}, "acmc.parse.Atc": {"tf": 1.4142135623730951}, "acmc.parse.Med": {"tf": 1.4142135623730951}, "acmc.parse.Cprd": {"tf": 1.4142135623730951}, "acmc.phen.DEFAULT_PHEN_DIR_LIST": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.COL_ACTIONS": {"tf": 1}, "acmc.phen.CODE_FILE_TYPES": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.diff_phen": {"tf": 1}, "acmc.trud": {"tf": 1}, "acmc.trud.FQDN": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 21}, "n": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.util.QuotedDumper": {"tf": 1}}, "df": 2, "l": {"docs": {}, "df": 0, "y": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}, "r": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1, "i": {"docs": {}, "df": 0, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.fork": {"tf": 1.7320508075688772}}, "df": 1}}}}}, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1, "l": {"docs": {}, "df": 0, "y": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}}}}}}}, "t": {"docs": {}, "df": 0, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}}}, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.Proto.verify": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}, "acmc.phen.fork": {"tf": 1.7320508075688772}, "acmc.util.QuotedDumper": {"tf": 1}}, "df": 8, "t": {"docs": {"acmc.logging_config.set_log_level": {"tf": 1}}, "df": 1, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main": {"tf": 1}}, "df": 1}}}}}}, "o": {"docs": {"acmc.phen.translate_codes": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 3}}, "s": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.install": {"tf": 1}}, "df": 1}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud": {"tf": 1}}, "df": 1}}}}}}}}}, "i": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1.4142135623730951}}, "df": 1}}}}, "v": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "d": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}}, "df": 1, "c": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 2}}}}}}}}}}}}}}}}}}}, "d": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "x": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}, "c": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}}, "df": 2}}}}}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "l": {"docs": {"acmc.phen.init": {"tf": 1}}, "df": 1}}}}}, "f": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}}}}}}}}, "f": {"docs": {"acmc.omop.install": {"tf": 1.7320508075688772}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.vocab_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.Proto": {"tf": 1.4142135623730951}, "acmc.parse.Read3": {"tf": 1.4142135623730951}, "acmc.parse.Icd10": {"tf": 1.4142135623730951}, "acmc.parse.Snomed": {"tf": 1.4142135623730951}, "acmc.parse.Opcs4": {"tf": 1.4142135623730951}, "acmc.parse.Atc": {"tf": 1.4142135623730951}, "acmc.parse.Med": {"tf": 1.4142135623730951}, "acmc.parse.Cprd": {"tf": 1.4142135623730951}, "acmc.parse.CodeTypeParser": {"tf": 1}, "acmc.phen.fork": {"tf": 1.7320508075688772}}, "df": 17}, "s": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.phen.fork": {"tf": 1.7320508075688772}, "acmc.phen.validate": {"tf": 1}}, "df": 12, "s": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 9}}}}}, "d": {"docs": {"acmc.omop.vocab_exists": {"tf": 1.7320508075688772}}, "df": 1, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "y": {"docs": {"acmc.parse.Proto.process": {"tf": 1}}, "df": 1}}}}}}}, "t": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}}, "df": 2, "e": {"docs": {}, "df": 0, "m": {"docs": {"acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 2}}}}, "w": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "k": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {"acmc.main.DEFAULT_WORKSPACE_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.write_version_file": {"tf": 1}}, "df": 1}}, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.export": {"tf": 1}}, "df": 1}}}}}}, "h": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.export": {"tf": 1}, "acmc.phen.fork": {"tf": 1}}, "df": 2}}, "n": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 3}}, "a": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8}}}, "i": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "h": {"docs": {"acmc.parse.Proto.process": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}}, "df": 4}}}}, "v": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "b": {"docs": {"acmc.omop.vocab_exists": {"tf": 1}}, "df": 1, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop": {"tf": 1}, "acmc.omop.vocabularies": {"tf": 1}, "acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.trud": {"tf": 1}}, "df": 5}}}, "y": {"docs": {"acmc.omop.VOCAB_PATH": {"tf": 1}, "acmc.omop.omop_vocab_types": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1.7320508075688772}, "acmc.omop.export": {"tf": 1.4142135623730951}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.trud.VOCAB_PATH": {"tf": 1}, "acmc.trud.DOWNLOADS_PATH": {"tf": 1}}, "df": 7}}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.write_version_file": {"tf": 1}}, "df": 1}}}}}}}}}}}, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}, "acmc.omop.VERSION_FILE": {"tf": 1}, "acmc.omop.VERSION_PATH": {"tf": 1}, "acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1.7320508075688772}, "acmc.omop.export": {"tf": 1.7320508075688772}, "acmc.phen.VOCAB_VERSION_FILE": {"tf": 1}, "acmc.phen.SEMANTIC_VERSION_TYPES": {"tf": 1}, "acmc.phen.DEFAULT_VERSION_INC": {"tf": 1}, "acmc.phen.fork": {"tf": 2}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}, "acmc.trud.VERSION_FILE": {"tf": 1}, "acmc.trud.VERSION_PATH": {"tf": 1}}, "df": 14, "s": {"docs": {"acmc.phen.diff_phen": {"tf": 1}}, "df": 1}}}}}, "i": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "y": {"docs": {"acmc.parse.Proto.verify": {"tf": 1}}, "df": 1}}}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.phen.fork": {"tf": 1.7320508075688772}}, "df": 2}}}}}}}, "i": {"docs": {}, "df": 0, "d": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 2}}}, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.phen.validate": {"tf": 1}}, "df": 2}}}}}}}}}, "n": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "b": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.OMOP_CDM_Version": {"tf": 1}}, "df": 1}}}}}, "o": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1.7320508075688772}, "acmc.parse.Proto.process": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.phen.fork": {"tf": 1.4142135623730951}}, "df": 14}, "n": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.fork": {"tf": 1}}, "df": 1}}}, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}, "acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}, "acmc.phen.PHEN_DIR": {"tf": 1}, "acmc.phen.CONCEPTS_DIR": {"tf": 1}, "acmc.phen.MAP_DIR": {"tf": 1}, "acmc.phen.CONCEPT_SET_DIR": {"tf": 1}, "acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}, "acmc.phen.extract_concepts": {"tf": 1.4142135623730951}, "acmc.trud.FQDN": {"tf": 1}}, "df": 17}}}, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "y": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}}}, "w": {"docs": {"acmc.phen.fork": {"tf": 1.7320508075688772}}, "df": 1}}, "h": {"docs": {}, "df": 0, "s": {"docs": {"acmc.trud": {"tf": 1}, "acmc.trud.FQDN": {"tf": 1}}, "df": 2}}}, "e": {"docs": {}, "df": 0, "x": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.EXPORT_FILE": {"tf": 1}, "acmc.omop.export": {"tf": 1.4142135623730951}}, "df": 2, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.export": {"tf": 1}}, "df": 1}}, "s": {"docs": {"acmc.phen.export": {"tf": 1}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}}}}, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "t": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.omop.table_exists": {"tf": 1.4142135623730951}, "acmc.omop.vocab_exists": {"tf": 1.4142135623730951}, "acmc.omop.concept_set_exist": {"tf": 1.4142135623730951}}, "df": 3}}}}, "c": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "p": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1.4142135623730951}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 4}}}}}}}, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "d": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "s": {"docs": {"acmc.phen.extract_concepts": {"tf": 1}}, "df": 1}}}}}}, "e": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.VOCAB_PATH": {"tf": 1}}, "df": 1}}}}}}}}, "r": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "r": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 3, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 2}}}}}, "n": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}}, "a": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.translate_codes": {"tf": 1}}, "df": 1}}}}, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "q": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.vocabularies": {"tf": 1}}, "df": 1}}}}}}, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.trud.get_releases": {"tf": 1}, "acmc.trud.download_release_file": {"tf": 1}}, "df": 3}}}}, "a": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.VOCAB_PATH": {"tf": 1}}, "df": 1}}}}}}, "a": {"docs": {}, "df": 0, "d": {"2": {"docs": {"acmc.parse.Read2": {"tf": 1.4142135623730951}}, "df": 1}, "docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "g": {"docs": {"acmc.omop.install": {"tf": 1}}, "df": 1}}}}}, "t": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "s": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}, "acmc.omop.export": {"tf": 1}, "acmc.phen.translate_codes": {"tf": 1}}, "df": 5}}}}, "r": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {"acmc.trud.get_releases": {"tf": 1}}, "df": 1}}}}}, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "n": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}, "p": {"docs": {}, "df": 0, "o": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}, "acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 2.8284271247461903}, "acmc.phen.validate": {"tf": 1}, "acmc.phen.publish": {"tf": 1}, "acmc.phen.export": {"tf": 1}, "acmc.phen.copy": {"tf": 1}}, "df": 7, "r": {"docs": {}, "df": 0, "t": {"docs": {"acmc.parse.CodesError": {"tf": 1}, "acmc.parse.CodeTypeParser": {"tf": 1}}, "df": 2}}}}, "s": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "v": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto": {"tf": 1}, "acmc.parse.Read3": {"tf": 1}, "acmc.parse.Icd10": {"tf": 1}, "acmc.parse.Snomed": {"tf": 1}, "acmc.parse.Opcs4": {"tf": 1}, "acmc.parse.Atc": {"tf": 1}, "acmc.parse.Med": {"tf": 1}, "acmc.parse.Cprd": {"tf": 1}}, "df": 8, "d": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}}, "df": 1}}}}}}, "m": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {"acmc.phen.fork": {"tf": 1.7320508075688772}}, "df": 1}}}}}, "a": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1.7320508075688772}}, "df": 1, "s": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.omop.clear": {"tf": 1}, "acmc.omop.delete": {"tf": 1}, "acmc.parse.Proto.raise_exception": {"tf": 1}, "acmc.phen.fork": {"tf": 1}}, "df": 5}, "d": {"docs": {"acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.phen.PhenValidationException": {"tf": 1}}, "df": 2}}}}}, "u": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.parse.Read2": {"tf": 1}}, "df": 1}}}}}, "b": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.omop.install": {"tf": 1}, "acmc.parse.Read2": {"tf": 1}}, "df": 2}}}}, "y": {"docs": {"acmc.omop.install": {"tf": 1.4142135623730951}, "acmc.omop.write_version_file": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.phen.publish": {"tf": 1}}, "df": 4}, "o": {"docs": {}, "df": 0, "o": {"docs": {}, "df": 0, "l": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}}, "df": 3}}}, "e": {"docs": {"acmc.omop.export": {"tf": 1}, "acmc.parse.InvalidCodesException": {"tf": 1}, "acmc.phen.fork": {"tf": 1}}, "df": 3, "c": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "s": {"docs": {}, "df": 0, "e": {"docs": {"acmc.parse.Proto.raise_exception": {"tf": 1}}, "df": 1}}}}}, "t": {"docs": {}, "df": 0, "w": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.phen.diff_phen": {"tf": 1}}, "df": 1}}}}}}, "r": {"docs": {}, "df": 0, "a": {"docs": {}, "df": 0, "n": {"docs": {}, "df": 0, "c": {"docs": {}, "df": 0, "h": {"docs": {"acmc.phen.DEFAULT_GIT_BRANCH": {"tf": 1}}, "df": 1}}}}}}, "z": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "p": {"docs": {"acmc.omop.install": {"tf": 2}}, "df": 1}}}, "q": {"docs": {}, "df": 0, "u": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "r": {"docs": {}, "df": 0, "y": {"docs": {"acmc.omop.table_exists": {"tf": 1}, "acmc.omop.vocab_exists": {"tf": 1}, "acmc.omop.concept_set_exist": {"tf": 1}}, "df": 3}}}, "a": {"docs": {}, "df": 0, "l": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "f": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "d": {"docs": {"acmc.trud.FQDN": {"tf": 1}}, "df": 1}}}}}}}, "o": {"docs": {}, "df": 0, "t": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "s": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}}}, "y": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "l": {"docs": {"acmc.phen.CONFIG_SCHEMA": {"tf": 1}}, "df": 1}}, "a": {"docs": {}, "df": 0, "m": {"docs": {}, "df": 0, "l": {"docs": {"acmc.util.QuotedDumper": {"tf": 1}}, "df": 1}}}}, "g": {"docs": {}, "df": 0, "i": {"docs": {}, "df": 0, "t": {"docs": {"acmc.phen.init": {"tf": 1}, "acmc.phen.fork": {"tf": 1}, "acmc.phen.validate": {"tf": 1}}, "df": 3}, "v": {"docs": {}, "df": 0, "e": {"docs": {}, "df": 0, "n": {"docs": {"acmc.trud.download_release_file": {"tf": 1}}, "df": 1}}}}}}}}, "pipeline": ["trimmer"], "_isPrebuiltIndex": true};
+
+ // mirrored in build-search-index.js (part 1)
+ // Also split on html tags. this is a cheap heuristic, but good enough.
+ elasticlunr.tokenizer.setSeperator(/[\s\-.;&_'"=,()]+|<[^>]*>/);
+
+ let searchIndex;
+ if (docs._isPrebuiltIndex) {
+ console.info("using precompiled search index");
+ searchIndex = elasticlunr.Index.load(docs);
+ } else {
+ console.time("building search index");
+ // mirrored in build-search-index.js (part 2)
+ searchIndex = elasticlunr(function () {
+ this.pipeline.remove(elasticlunr.stemmer);
+ this.pipeline.remove(elasticlunr.stopWordFilter);
+ this.addField("qualname");
+ this.addField("fullname");
+ this.addField("annotation");
+ this.addField("default_value");
+ this.addField("signature");
+ this.addField("bases");
+ this.addField("doc");
+ this.setRef("fullname");
+ });
+ for (let doc of docs) {
+ searchIndex.addDoc(doc);
+ }
+ console.timeEnd("building search index");
+ }
+
+ return (term) => searchIndex.search(term, {
+ fields: {
+ qualname: {boost: 4},
+ fullname: {boost: 2},
+ annotation: {boost: 2},
+ default_value: {boost: 2},
+ signature: {boost: 2},
+ bases: {boost: 2},
+ doc: {boost: 1},
+ },
+ expand: true
+ });
+})();
\ No newline at end of file
diff --git a/docs/usage.md b/docs/cli.md
similarity index 57%
rename from docs/usage.md
rename to docs/cli.md
index 1527165ea12c7f7d696f8ba70c011c35eb2e92a5..4b756dcf5c094bdb2825dc614db8947c9be179cd 100644
--- a/docs/usage.md
+++ b/docs/cli.md
@@ -1,4 +1,4 @@
-# ACMC Usage Guide
+# ACMC Command Line Reference
The `acmc` command-line tool provides various commands to interact with TRUD, OMOP, and Phenotype data. Below are the usage details for each command.
@@ -67,16 +67,29 @@ The `omop` command is used for installing OMOP vocabularies.
The `phen` command is used phenotype-related operations.
-- **Initialize Phenotype**
+- **Initialise Phenotype**
- Initialize a phenotype directory locally or from a remote git repository:
+ Initialise a phenotype directory locally or with remote git repository:
```bash
acmc phen init -d <PHENOTYPE_DIRECTORY> -r <REMOTE_URL>
```
- - `-d`, `--phen-dir`: (Optional) Directory to write phenotype configuration (the default is ./build/phen).
- - `-r`, `--remote_url`: (Optional) URL to a remote git repository.
+ - `-d`, `--phen-dir`: (Optional) Directory to write phenotype configuration (the default is ./workspace/phen).
+ - `-r`, `--remote-url`: (Optional) URL to a remote git repository where the phenotype will be published, only supports an empty repo without existing commits.
+
+- **Fork Existing Phenotype**
+
+ Initialise a phenotype an existing phenotype publish in a git repository:
+
+ ```bash
+ acmc phen fork -d <PHENOTYPE_DIRECTORY> -r <REMOTE_URL>
+ ```
+
+ - `-d`, `--phen-dir`: (Optional) Local phenotype workspace directory (default is ./workspace/phen).
+ - `-r`, `--remote-url`: (Optional) URL to a remote git repository where the phenotype will be published, only supports an empty repo without existing commits.
+ - `-u`, `--upstream-url`: (Required) URL to the phenotype repository to fork.
+ - `-v`, `--upstream-version`: (Required) Phenotype version to fork.
- **Validate Phenotype**
@@ -86,7 +99,7 @@ The `phen` command is used phenotype-related operations.
acmc phen validate -d <PHENOTYPE_DIRECTORY>
```
- - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen).
+ - `-d`, `--phen-dir`: (Optional) Local phenotype workspace directory (default is ./workspace/phen).
- **Map Phenotype**
@@ -96,9 +109,8 @@ The `phen` command is used phenotype-related operations.
acmc phen map -d <PHENOTYPE_DIRECTORY> -t <TARGET_CODING> -o <OUTPUT_FORMAT>
```
- - `-t`, `--target-coding`: Specify the target coding (e.g., `read2`, `read3`, `icd10`, `snomed`, `opcs4`).
- - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen).
- - `-o`, `--output`: Output format(s) (`csv`, `omop`, or both), default is 'csv'.
+ - `-t`, `--target-coding`: (Optional) Specify the target coding (e.g., `read2`, `read3`, `icd10`, `snomed`, `opcs4`).
+ - `-d`, `--phen-dir`: (Optional) Local phenotype workspace directory (default is ./workspace/phen).
- **Publish Phenotype Configuration**
@@ -108,7 +120,10 @@ The `phen` command is used phenotype-related operations.
acmc phen publish -d <PHENOTYPE_DIRECTORY>
```
- - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen).
+ - `-d`, `--phen-dir`: (Optional) Local phenotype workspace directory (default is ./workspace/phen).
+ - `-i`, `--increment`: (Optional) Version increment: `major`, `minor`, or `patch`, default is `patch` increment
+ - `-m`, `--msg`: (Optional) Message to include with the published version
+ - `-r`, `--remote_url`: (Optional) URL to a remote git repository where the phenotype will be published, only supports an empty repo without existing commits.
- **Copy Phenotype Configuration**
@@ -118,7 +133,7 @@ The `phen` command is used phenotype-related operations.
acmc phen copy -d <PHENOTYPE_DIRECTORY> -td <TARGET_DIRECTORY> -v <PHENOTYPE_VERSION>
```
- - `-d`, `--phen-dir`: (Optional) Directory of phenotype configuration (the default is ./build/phen).
+ - `-d`, `--phen-dir`: (Optional) Local phenotype workspace directory (default is ./workspace/phen).
- `-td`, `--target-dir`: (Optional) Directory to copy the phenotype configuration to, (the default is ./build).
- `-v`, `--version`: The phenotype version to copy.
@@ -130,5 +145,8 @@ The `phen` command is used phenotype-related operations.
acmc phen diff -d <NEW_PHENOTYPE_DIRECTORY> -old <OLD_PHENOTYPE_DIRECTORY>
```
- - `-d`, `--phen-dir`: (Optional) Directory of current phenotype configuration (the default is ./build/phen).
- - `-old`, `--phen-dir-old`: (Required) Directory of old phenotype version)
\ No newline at end of file
+ - `-d`, `--phen-dir`: (Optional) Local phenotype workspace directory (default is ./workspace/phen).
+ - `-v`, `--version`: (Optional) Directory of changed phenotype version, default is `latest` which is the current files in the changed phen directory.
+ - `-od`, `--old-phen-dir`: (Optional) Directory of old phenotype, default is `./workspace/phen`.
+ - `-ov`, `--old-version`: (Required) Old phenotype version to compare with the chnaged version, default is `latest` which are the current files in the old phen directory.
+
diff --git a/docs/examples/example1.md b/docs/examples/example1.md
new file mode 100644
index 0000000000000000000000000000000000000000..c53aae523d3d96627326d1607d8b3258a3915754
--- /dev/null
+++ b/docs/examples/example1.md
@@ -0,0 +1,3 @@
+# Tutorial 1 Basic local phenotype
+
+TODO
diff --git a/docs/examples/example2.md b/docs/examples/example2.md
new file mode 100644
index 0000000000000000000000000000000000000000..3b3064dffb5fd97094f5f840bc249c6907241597
--- /dev/null
+++ b/docs/examples/example2.md
@@ -0,0 +1,3 @@
+# Tutorial 2 - More complex local phenotype
+
+TODO
\ No newline at end of file
diff --git a/docs/examples/example3.md b/docs/examples/example3.md
new file mode 100644
index 0000000000000000000000000000000000000000..f9c8123bb20b9a8193f726bc24bab95609718b20
--- /dev/null
+++ b/docs/examples/example3.md
@@ -0,0 +1,3 @@
+# Tutorial 3 - Using a remote git repository
+
+TDO
\ No newline at end of file
diff --git a/docs/index.md b/docs/index.md
index ae1a3478131d698e6083c46fa053bbe8f7c83bca..430dfb5dd038853e7420d3139c99f9b281693f2a 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,19 +1,180 @@
-# ACMC Documentation
+# A Tool for Automating the Curation of Medical Concepts derived from Coding Lists (ACMC)
-## Contents
+- [Overview](#overview)
+- [Phenotype Workflow](#phenotype-workflow)
+- [Phenotype Definition](#phenotype-definition)
+- [Phenotype Version Control](#phenotype-version-control)
+- [Supported Medical Coding Standards](#supported-medical-coding-standards)
- - [Installation](./installation.md)
- - [Usage](./usage.md)
- - Tutorials
- - [Example 1 - Basic local phenotype](./tutorials/example1.md)
- - [Example 2 - More complex local phenotype](./tutorials/example2.md)
- - [Example 3 - Using a remote git repository](./tutorials/example3.md)
- - [API Reference](./api.md)
- - [Contributing](./contributing.md)
- - [Change Log](./changelog.md)
- - [Troubleshooting](./troubleshooting.md)
+## Overview
-### Overview
+ACMC is a tool that automates the verification, translation and organisation of medical coding lists defining phenotypes for inclusion criteria in cohort analysis. By processing externally sourced clinical inclusion criteria into actionable code lists, this tool ensures consistent and efficient curation of cohort definitions. These code lists can be subsequently used by data providers to construct study cohorts.
+
+## Phenotype Workflow
+
+ACMC has a five step workflow to create a phenotype including steps initialise, validate, map, publish and export
+
+### Step 1: Initialise
+
+The `initialise` step creates a phenotype directory within the acmc workspace. The outcome will be a directory with all
+required subdirectories and files, see [directory structure](#phenotype-directory-structure)
+
+```bash
+acmc phen init
+```
+
+### Step 2: Validate
+
+The `validate` step checks that the phenotype configuration is valid including verification of the
+configuration file according to schema and consistency between concept sets and source concept coding lists. The outcome will be notifications of the validity of the phenotype configuration.
+
+```bash
+acmc phen validate
+```
+
+### Step 3: Map
+
+The `map` step performs code translations between source and target coding types. The outcome will be a concept sets defined for the target coding types stored in CSV files.
+
+```bash
+acmc phen map
+```
+
+### Step 4: Publish
+
+The `publish` step commits the phenotype to a git repo and increments the version number. The outcome will be a published phenotype at the next version number
+
+```bash
+acmc phen publish
+```
+
+### Step 5: Export
+
+The `export` step creates an OMOP database for the phenotype. The outcome will be an OMOP database including concept sets for all target coding types exported as CSV files
+
+```bash
+acmc phen export
+```
+
+## Phenotype Definition
+
+### **Phenotype directory structure**
+
+Each phenotype has a standard directory structure that includes configuration, source concept coding lists, maps to target coding types and output concept sets.
+
+```
+workspace/ # Default workspace directory
+├── phen/ # Default phenotype directory
+│ ├── concepts/ # Phenotype source concept code lists directory
+│ ├── concept-sets/ # Processed phenotype concept sets
+│ │ ├── csv/ # Processed phenotype concept sets in ACMC CSV format
+│ │ ├── omop/ # Processed phenotype concept sets in OMOP CDM database exported as CSV files
+│ ├── map/ # Process mapping from source to target code types
+│ │ ├── errors/ # Errors recorded during mapping process
+│ ├── config.yaml # Phenotype configuration file
+│ ├── vocab_versions.yaml # Versions file for vocabularies used to generate concept sets
+```
+
+### **Configuration File**
+
+Each phenotype is defined by configuration is stored in the root of the phenotype directory in `config.yaml`. The file is yaml format.
+
+#### **Root Phenotype Element**
+- `phenotype`: **(object)** The root element containing all phenotype-related concept sets and metadata.
+
+#### **Phenotype Attributes**
+- `version`: **(string)** Specifies the version of the phenotype definition.
+- `omop`: **(object)** Metadata related to OMOP vocabulary.
+ - `vocabulary_id`: **(string)** Identifier for the vocabulary.
+ - `vocabulary_name`: **(string)** Human-readable name of the vocabulary.
+ - `vocabulary_reference`: **(string, URL)** A reference URL for the vocabulary source.
+
+#### **Concept Sets**
+- `concept_sets`: **(array)** A list of concept set definitions, where each item has the following attributes:
+ - `name`: **(string)** Unique name of the concept set.
+ - `file`: **(object)** Contains file-related metadata.
+ - `path`: **(string, file path)** Relative path to the source concepts coding list file, relative to `<phen_directory>/concepts`
+ - `columns`: **(object)** Key-value pairs mapping column names in the file to coding list types
+ - `category` **(optional, string)** A categorical identifier for processing files containing multiple concept sets.
+ - `actions` **(optional, object)** Additional transformations on data.
+ - `divide_col`: **(string)** Specifies a column name in the source concept file to group on.
+ - `metadata`: **(object)** Reserved for additional metadata.
+
+## **Phenotype Version Control**
+
+ACMC uses [Git](https://git-scm.com/) to support versioning of phenotypes. Git is a version control system that track changes in documents such as source coding lists, coding maps or configuration files. Using git allows ACMC to track versions and changes.
+
+When a phenotype is initialised the directory is configured as a Git repository. ACMC then provides a simplified interaction with Git through a specific workflow using ACMC commands including integrate with remote Git services such as [GitLab](https://about.gitlab.com/) or [GitHub](https://github.com/).
+
+ACMC does not currently support merging contributions from multiple collaborators on a phenotype through ACMC commands. This has to be done using existing Git tools.
+
+### **Version Numbers**
+
+ACMC uses [semantic versioning](https://semver.org/) to version phenotypes. Semantic versioning uses three numbers MAJOR.MINOR.PATCH where each number is incremented depending on the significance of the change. Although semantic versioning is designed for sofware the idea of major, minor and patch changes is retained for the phenotype as per the following
+
+- MAJOR version when you make changes to concept sets
+- MINOR version when you make changes to coding list concepts
+- PATCH version when you make other minor changes such as documentation
+
+### **Workflows**
+
+ACMC supports local and remote repositories
+
+#### **Local Workflow**
+
+A local phenotype is only stored within a directory on a filesystem. The following command will create a git repository with the initial phenotype directory structure and make a commit to the git repository.
+
+```bash
+acmc phen init
+```
+
+You can then configure your phenotype and generate maps to other coding types as required. When you are finished and happy to publish a version of your phenotype, you run the following command
+
+```bash
+acmc phen publish
+```
+
+This will commit the changes to the git repository and generate a new version number. If this is the first publish the initial version will be `0.0.1`. You can tell ACMC how to increment the version using the `-i` argument with either major, minor or patch. The defaul is a patch change, i.e. incrementing the patch number. Using the following command will create a major release `1.0.0`
+
+```bash
+acmc phen publish -i major
+```
+
+#### **Remote Workflow**
+
+A remote phenotype is stored on a central server that can be accessed remotely by others. Common central services include GitHub or GitLab (public or private). You can connect your local phenotype to a remote repository during initialisation or publication. When connecting to a remote repository it is important and recommended that you connect to an empty repo without any previous commits. Do not initialise it with a readme.md file, which is often the default. If there are commits you will need to resolve the conflicts manually before ACMC will work.
+
+To initiatise a phenotype with a remote Git repository using the following command replacing the git URL with the URL to your remote repo.
+
+```bash
+acmc phen init -r https://git.soton.ac.uk/meldb/remote-phenotype.git
+```
+
+If you have a local phenotype and later want to connect it to a remote phenotype you can do this when it's published
+
+```bash
+acmc phen publish -r https://git.soton.ac.uk/meldb/remote-phenotype.git
+```
+
+#### **Fork Remote Workflow**
+
+If there is an existing published remote phenotype that you want to use as a starting point you can fork the upstream repo and create a new phenotype. To do this you can run the following command to create a fork of the remote repo in a local directory
+
+```bash
+acmc fork -u https://git.soton.ac.uk/meldb/forked-phenotype.git -v 1.0.0
+```
+
+If you want to fork the repo and connect this to a remote repo you can run the following command.
+
+```bash
+acmc fork -u https://git.soton.ac.uk/meldb/forked-phenotype.git -v 1.0.0 -r https://git.soton.ac.uk/meldb/remote-phenotype.git
+```
+
+Alternatively you can connect the remote repo later when you publish
+
+```bash
+acmc phen publish -r https://git.soton.ac.uk/meldb/remote-phenotype.git
+```
### Supported Medical Coding Standards
@@ -38,6 +199,3 @@ The tool supports verification and mapping across diagnostic coding formats belo
Processed resources will be saved in the `build/maps/processed/` directory.
*Note: NHS TRUD provides one-way mappings. To reverse mappings, duplicate the `.parquet` file and reverse the filename (e.g., `read2_code_to_snomed_code.parquet` to `snomed_code_to_read2_code.parquet`).*
-
-
-
diff --git a/docs/installation.md b/docs/installation.md
deleted file mode 100644
index a501c3a69d4782f54cd1a0e87520524416f8bbfc..0000000000000000000000000000000000000000
--- a/docs/installation.md
+++ /dev/null
@@ -1 +0,0 @@
-# ACMC Installation Guide
\ No newline at end of file
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
deleted file mode 100644
index d39abac886728a3d771701ec7da4779f2c4b142a..0000000000000000000000000000000000000000
--- a/docs/mkdocs.yml
+++ /dev/null
@@ -1,28 +0,0 @@
-site_name: ACMC Documentation
-theme:
- name: material
- features:
- - navigation.tabs
- - navigation.expand
- - content.code.copy
-nav:
- - Home: index.md
- - Installation: installation.md
- - Usage: usage.md
- - Contributing: contributing.md
- - API Reference: api.md
- - Tutorials:
- - Example 1 - Basic local phenotype: ./tutorials/example1.md
- - Example 2 - More complex local phenotype: ./tutorials/example2.md
- - Example 3 - Using a remote git repository: ./tutorials/example3.md
- - Contributing: contributing.md
- - Change Log: changelog.md
- - Troubleshooting: troubleshooting.md
-repo_url: https://git.soton.ac.uk/meldb/concepts-processing/
-plugins:
- - search
- - mkdocstrings:
- handlers:
- python:
- options:
- show_source: false
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
deleted file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0000000000000000000000000000000000000000
diff --git a/docs/tutorials/example1.md b/docs/tutorials/example1.md
deleted file mode 100644
index 562e2989c0714a59a0e50176cbdb905d1753e9a1..0000000000000000000000000000000000000000
--- a/docs/tutorials/example1.md
+++ /dev/null
@@ -1 +0,0 @@
-# Example 1 Basic local phenotype
diff --git a/docs/tutorials/example2.md b/docs/tutorials/example2.md
deleted file mode 100644
index 8c68b37e5b78098d5cddb0622728b5a223366315..0000000000000000000000000000000000000000
--- a/docs/tutorials/example2.md
+++ /dev/null
@@ -1 +0,0 @@
-# Example 2 - More complex local phenotype
\ No newline at end of file
diff --git a/docs/tutorials/example3.md b/docs/tutorials/example3.md
deleted file mode 100644
index becc32fe5382e637ff8e682d9d89f1674326934b..0000000000000000000000000000000000000000
--- a/docs/tutorials/example3.md
+++ /dev/null
@@ -1 +0,0 @@
-Example 3 - Using a remote git repository
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/About the source.docx b/examples/codes/clinical-codes-org/About the source.docx
deleted file mode 100644
index acc334d2c3abe0392f83158b763f7b0ba46c9758..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/About the source.docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Behaviours code lists/Alcohol intake/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Behaviours code lists/Alcohol intake/Where this code list comes from .docx
deleted file mode 100644
index 75b25caf91969c25ed0b0cb414e965bb90643c51..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Behaviours code lists/Alcohol intake/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Behaviours code lists/Alcohol intake/res47-alcohol-intake.csv b/examples/codes/clinical-codes-org/Behaviours code lists/Alcohol intake/res47-alcohol-intake.csv
deleted file mode 100644
index 7bac5dd747f1ec61693b5ba5e26827237bfa16d5..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Behaviours code lists/Alcohol intake/res47-alcohol-intake.csv
+++ /dev/null
@@ -1,37 +0,0 @@
-code,coding_system,description,entity,list_name,upload_date
-136..00,Read,Alcohol consumption,test,451 - res47: Alcohol intake,25/07/2016
-1361.11,Read,Non drinker alcohol,test,451 - res47: Alcohol intake,25/07/2016
-1361.12,Read,Non-drinker alcohol,test,451 - res47: Alcohol intake,25/07/2016
-1362.11,Read,Drinks rarely,test,451 - res47: Alcohol intake,25/07/2016
-1362.12,Read,Drinks occasionally,test,451 - res47: Alcohol intake,25/07/2016
-136a.00,Read,Increasing risk drinking,test,451 - res47: Alcohol intake,25/07/2016
-136A.00,Read,Ex-trivial drinker (<1u/day),test,451 - res47: Alcohol intake,25/07/2016
-136b.00,Read,Feels should cut down drinking,test,451 - res47: Alcohol intake,25/07/2016
-136B.00,Read,Ex-light drinker - (1-2u/day),test,451 - res47: Alcohol intake,25/07/2016
-136c.00,Read,Higher risk drinking,test,451 - res47: Alcohol intake,25/07/2016
-136C.00,Read,Ex-moderate drinker - (3-6u/d),test,451 - res47: Alcohol intake,25/07/2016
-136d.00,Read,Lower risk drinking,test,451 - res47: Alcohol intake,25/07/2016
-136D.00,Read,Ex-heavy drinker - (7-9u/day),test,451 - res47: Alcohol intake,25/07/2016
-136e.00,Read,Declines to state current alcohol consumption,test,451 - res47: Alcohol intake,25/07/2016
-136E.00,Read,Ex-very heavy drinker-(>9u/d),test,451 - res47: Alcohol intake,25/07/2016
-136F.00,Read,Spirit drinker,test,451 - res47: Alcohol intake,25/07/2016
-136G.00,Read,Beer drinker,test,451 - res47: Alcohol intake,25/07/2016
-136H.00,Read,Drinks beer and spirits,test,451 - res47: Alcohol intake,25/07/2016
-136I.00,Read,Drinks wine,test,451 - res47: Alcohol intake,25/07/2016
-136J.00,Read,Social drinker,test,451 - res47: Alcohol intake,25/07/2016
-136K.00,Read,Alcohol intake above recommended sensible limits,test,451 - res47: Alcohol intake,25/07/2016
-136L.00,Read,Alcohol intake within recommended sensible limits,test,451 - res47: Alcohol intake,25/07/2016
-136M.00,Read,Current non drinker,test,451 - res47: Alcohol intake,25/07/2016
-136N.00,Read,Light drinker,test,451 - res47: Alcohol intake,25/07/2016
-136O.00,Read,Moderate drinker,test,451 - res47: Alcohol intake,25/07/2016
-136P.00,Read,Heavy drinker,test,451 - res47: Alcohol intake,25/07/2016
-136Q.00,Read,Very heavy drinker,test,451 - res47: Alcohol intake,25/07/2016
-136R.00,Read,Binge drinker,test,451 - res47: Alcohol intake,25/07/2016
-136S.00,Read,Hazardous alcohol use,test,451 - res47: Alcohol intake,25/07/2016
-136T.00,Read,Harmful alcohol use,test,451 - res47: Alcohol intake,25/07/2016
-136V.00,Read,Alcohol units per week,test,451 - res47: Alcohol intake,25/07/2016
-136W.00,Read,Alcohol misuse,test,451 - res47: Alcohol intake,25/07/2016
-136X.00,Read,Alcohol units consumed on heaviest drinking day,test,451 - res47: Alcohol intake,25/07/2016
-136Y.00,Read,Drinks in morning to get rid of hangover,test,451 - res47: Alcohol intake,25/07/2016
-136Z.00,Read,Alcohol consumption NOS,test,451 - res47: Alcohol intake,25/07/2016
-E23..12,Read,Alcohol problem drinking,test,451 - res47: Alcohol intake,25/07/2016
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Behaviours code lists/Physical activity/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Behaviours code lists/Physical activity/Where this code list comes from .docx
deleted file mode 100644
index 5413b668043d248eac2df0368b34ddfedc623e5a..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Behaviours code lists/Physical activity/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Behaviours code lists/Physical activity/res47-physical-activity.csv b/examples/codes/clinical-codes-org/Behaviours code lists/Physical activity/res47-physical-activity.csv
deleted file mode 100644
index 113dbd10c7b6c6f72de0777498838e2803d7bd2b..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Behaviours code lists/Physical activity/res47-physical-activity.csv
+++ /dev/null
@@ -1,35 +0,0 @@
-code,coding_system,description,entity,list_name,upload_date
-1381,Read,Exercise physically impossible,test,450 - res47: Physical activity,25/07/2016
-1382,Read,Avoids even trivial exercise,test,450 - res47: Physical activity,25/07/2016
-1383,Read,Enjoys light exercise,test,450 - res47: Physical activity,25/07/2016
-1384,Read,Enjoys moderate exercise,test,450 - res47: Physical activity,25/07/2016
-1385,Read,Enjoys heavy exercise,test,450 - res47: Physical activity,25/07/2016
-1386,Read,Competitive athlete,test,450 - res47: Physical activity,25/07/2016
-1388,Read,Aerobic exercise 0 times/week,test,450 - res47: Physical activity,25/07/2016
-1389,Read,Aerobic exercise 1 time/week,test,450 - res47: Physical activity,25/07/2016
-138a.00,Read,GPPAQ physical activity index: moderately active,test,450 - res47: Physical activity,25/07/2016
-138A.00,Read,Aerobic exercise 2 times/week,test,450 - res47: Physical activity,25/07/2016
-138b.00,Read,GPPAQ physical activity index: active,test,450 - res47: Physical activity,25/07/2016
-138B.00,Read,Aerobic exercise 3+ times/week,test,450 - res47: Physical activity,25/07/2016
-138C.00,Read,Anaerobic exercise 0 times/wk,test,450 - res47: Physical activity,25/07/2016
-138D.00,Read,Anaerobic exercise 1 time/week,test,450 - res47: Physical activity,25/07/2016
-138E.00,Read,Anaerobic exercise 2 times/wk,test,450 - res47: Physical activity,25/07/2016
-138F.00,Read,Anaerobic exercise 3+ times/wk,test,450 - res47: Physical activity,25/07/2016
-138G.00,Read,Attends exercise classes,test,450 - res47: Physical activity,25/07/2016
-138H.00,Read,Enjoys intermediate exercise,test,450 - res47: Physical activity,25/07/2016
-138I.00,Read,FITT activity level 0; no mod/vig activity of 20 mins duratn,test,450 - res47: Physical activity,25/07/2016
-138J.00,Read,FITT activity level 1; 1-4 occas of mod/vig activit in 4 wks,test,450 - res47: Physical activity,25/07/2016
-138K.00,Read,FITT activity level 2; 5-11 occas of mod/vig activt in 4 wks,test,450 - res47: Physical activity,25/07/2016
-138L.00,Read,FITT activity level 3; 12+ occas of mod activity in 4 weeks,test,450 - res47: Physical activity,25/07/2016
-138M.00,Read,FITT activity level 4; 12+ occas of mod/vig activit in 4 wks,test,450 - res47: Physical activity,25/07/2016
-138N.00,Read,FITT activity level 5; 12+ occas of vig activity in 4 weeks,test,450 - res47: Physical activity,25/07/2016
-138O.00,Read,Takes inadequate exercise,test,450 - res47: Physical activity,25/07/2016
-138P.00,Read,Aerobic exercise three times a week,test,450 - res47: Physical activity,25/07/2016
-138Q.00,Read,Aerobic exercise four times a week,test,450 - res47: Physical activity,25/07/2016
-138R.00,Read,Aerobic exercise five times a week,test,450 - res47: Physical activity,25/07/2016
-138S.00,Read,Declined referral to physical exercise programme,test,450 - res47: Physical activity,25/07/2016
-138T.00,Read,30 mins/day of at least mod intensty physc activity >=5 week,test,450 - res47: Physical activity,25/07/2016
-138V.00,Read,30 minutes a day of at least moderate intensity walk >=5week,test,450 - res47: Physical activity,25/07/2016
-138W.00,Read,Less than 30 mins/day of at least mod int physc act >=5 week,test,450 - res47: Physical activity,25/07/2016
-138X.00,Read,GPPAQ physical activity index: inactive,test,450 - res47: Physical activity,25/07/2016
-138Y.00,Read,GPPAQ physical activity index: moderately inactive,test,450 - res47: Physical activity,25/07/2016
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Behaviours code lists/Smoking status/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Behaviours code lists/Smoking status/Where this code list comes from .docx
deleted file mode 100644
index 1d681412967fb72aada16d97707428c171c538d6..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Behaviours code lists/Smoking status/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Behaviours code lists/Smoking status/res56-smoking-status.csv b/examples/codes/clinical-codes-org/Behaviours code lists/Smoking status/res56-smoking-status.csv
deleted file mode 100644
index 59c19723ae32c5df3501a075fef61659eb0d420a..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Behaviours code lists/Smoking status/res56-smoking-status.csv
+++ /dev/null
@@ -1,63 +0,0 @@
-code,coding_system,description,entity,list_name,upload_date,status
-137..00,Read,Tobacco consumption,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1371,Read,Never smoked tobacco,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-137..11,Read,Smoker - amount smoked,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1371.11,Read,Non-smoker,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-1372,Read,Trivial smoker - < 1 cig/day,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1372.11,Read,Occasional smoker,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1373,Read,Light smoker - 1-9 cigs/day,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1374,Read,Moderate smoker - 10-19 cigs/d,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1375,Read,Heavy smoker - 20-39 cigs/day,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1376,Read,Very heavy smoker - 40+cigs/d,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-1377,Read,Ex-trivial smoker (<1/day),diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-1378,Read,Ex-light smoker (1-9/day),diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-1379,Read,Ex-moderate smoker (10-19/day),diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137a.00,Read,Pipe tobacco consumption,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137A.00,Read,Ex-heavy smoker (20-39/day),diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137b.00,Read,Ready to stop smoking,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137B.00,Read,Ex-very heavy smoker (40+/day),diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137c.00,Read,Thinking about stopping smoking,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137C.00,Read,Keeps trying to stop smoking,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137d.00,Read,Not interested in stopping smoking,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137D.00,Read,Admitted tobacco cons untrue ?,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137e.00,Read,Smoking restarted,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137E.00,Read,Tobacco consumption unknown,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137F.00,Read,Ex-smoker - amount unknown,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137G.00,Read,Trying to give up smoking,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137H.00,Read,Pipe smoker,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137I.00,Read,Passive smoker,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-137j.00,Read,Ex-cigarette smoker,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137J.00,Read,Cigar smoker,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137K.00,Read,Stopped smoking,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137K000,Read,Recently stopped smoking,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137l.00,Read,Ex roll-up cigarette smoker,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137L.00,Read,Current non-smoker,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-137m.00,Read,Failed attempt to stop smoking,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137M.00,Read,Rolls own cigarettes,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137N.00,Read,Ex pipe smoker,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137O.00,Read,Ex cigar smoker,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137P.00,Read,Cigarette smoker,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137P.11,Read,Smoker,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137Q.00,Read,Smoking started,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137Q.11,Read,Smoking restarted,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137R.00,Read,Current smoker,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137S.00,Read,Ex smoker,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137T.00,Read,Date ceased smoking,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-137U.00,Read,Not a passive smoker,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-137V.00,Read,Smoking reduced,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137W.00,Read,Chews tobacco,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137X.00,Read,Cigarette consumption,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137Y.00,Read,Cigar consumption,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-137Z.00,Read,Tobacco consumption NOS,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-8CAL.00,Read,Smoking cessation advice,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-8H7i.00,Read,Referral to smoking cessation advisor,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-8HkQ.00,Read,Referral to NHS stop smoking service,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-8HTK.00,Read,Referral to stop-smoking clinic,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-8IAj.00,Read,Smoking cessation advice declined,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-9km..00,Read,Ex-smoker annual review - enhanced services administration,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-9km..11,Read,Ex-smoker annual review,diagnostic,649 - res56: smoking status,16/03/2017,ex smoker
-9kn..00,Read,Non-smoker annual review - enhanced services administration,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-9kn..11,Read,Non-smoker annual review,diagnostic,649 - res56: smoking status,16/03/2017,never smoker
-9ko..00,Read,Current smoker annual review - enhanced services admin,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-9ko..11,Read,Current smoker annual review,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
-9NS0200,Read,Referral for smoking cessation service offered,diagnostic,649 - res56: smoking status,16/03/2017,current smoker
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Motor neurone disease/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Motor neurone disease/Where this code list comes from .docx
deleted file mode 100644
index 7522774653f1fff251577b19078f026d4b44b352..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Motor neurone disease/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Motor neurone disease/phenotype_PH62_ver_124_concepts_20230719T112819.csv b/examples/codes/clinical-codes-org/Motor neurone disease/phenotype_PH62_ver_124_concepts_20230719T112819.csv
deleted file mode 100644
index 79363ead2fb4c2e8915a3753dae917209f2ce5c7..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Motor neurone disease/phenotype_PH62_ver_124_concepts_20230719T112819.csv
+++ /dev/null
@@ -1,26 +0,0 @@
-code,description,coding_system,concept_id,concept_version_id,concept_name,phenotype_id,phenotype_version_id,phenotype_name,code_attributes
-7Q04100,Amyotrophic lateral sclerosis drugs Band 1,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '109020.0', 'Category': 'History of Motor neuron disease'}"
-F15..00,Anterior horn cell disease,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '21889.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152.00,Motor neurone disease,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '4796.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152000,Amyotrophic lateral sclerosis,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '36433.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152100,Progressive muscular atrophy,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '30987.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152111,Duchenne Aran muscular atrophy,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '7470.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152200,Progressive bulbar palsy,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '27377.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152300,Pseudobulbar palsy,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '18084.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152400,Primary lateral sclerosis,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '20845.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F152z00,Motor neurone disease NOS,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '20120.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F15y.00,Other anterior horn cell disease,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '71400.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-F15z.00,Anterior horn cell disease NOS,Read codes v2,C852,2843,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Medcode': '58729.0', 'Category': 'Diagnosis of Motor neuron disease'}"
-109020,History of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Amyotrophic lateral sclerosis drugs Band 1', 'Readcode': '7Q04100'}"
-18084,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Pseudobulbar palsy', 'Readcode': 'F152300'}"
-20120,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Motor neurone disease NOS', 'Readcode': 'F152z00'}"
-20845,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Primary lateral sclerosis', 'Readcode': 'F152400'}"
-21889,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Anterior horn cell disease', 'Readcode': 'F15..00'}"
-27377,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Progressive bulbar palsy', 'Readcode': 'F152200'}"
-30987,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Progressive muscular atrophy', 'Readcode': 'F152100'}"
-36433,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Amyotrophic lateral sclerosis', 'Readcode': 'F152000'}"
-4796,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Motor neurone disease', 'Readcode': 'F152.00'}"
-58729,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Anterior horn cell disease NOS', 'Readcode': 'F15z.00'}"
-71400,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Other anterior horn cell disease', 'Readcode': 'F15y.00'}"
-7470,Diagnosis of Motor neuron disease,Med codes,C853,2845,Motor neuron disease - Primary care,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'ReadcodeDescr': 'Duchenne Aran muscular atrophy', 'Readcode': 'F152111'}"
-G12.2,Motor neuron disease,ICD10 codes,C854,2847,Motor neuron disease - Secondary care - Diagnoses,PH62,124,Motor neuron disease,"{'Disease': 'Motor neuron disease', 'Category': 'Diagnosis of Motor neuron disease'}"
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Obsessive Compulsive Disorder/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Obsessive Compulsive Disorder/Where this code list comes from .docx
deleted file mode 100644
index d842c7fc2bfb4929366e4e3fa4b5ee79d940a832..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Obsessive Compulsive Disorder/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Obsessive Compulsive Disorder/phenotype_PH223_ver_446_concepts_20230719T114228.csv b/examples/codes/clinical-codes-org/Obsessive Compulsive Disorder/phenotype_PH223_ver_446_concepts_20230719T114228.csv
deleted file mode 100644
index fcbc8da0ed71dabaf29ffb8b987789e27776e1b9..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Obsessive Compulsive Disorder/phenotype_PH223_ver_446_concepts_20230719T114228.csv
+++ /dev/null
@@ -1,28 +0,0 @@
-code,description,coding_system,concept_id,concept_version_id,concept_name,phenotype_id,phenotype_version_id,phenotype_name,code_attributes
-E203.00,Obsessive-compulsive disorders,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '3208.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-E203.11,Anancastic neurosis,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '47365.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-E203000,Compulsive neurosis,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '5678.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-E203100,Obsessional neurosis,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '2030.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-E203z00,Obsessive-compulsive disorder NOS,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '15566.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42.00,[X]Obsessive - compulsive disorder,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '5304.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42.11,[X]Anankastic neurosis,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '24251.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42.12,[X]Obsessive-compulsive neurosis,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '21836.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42000,[X]Predominantly obsessional thoughts or ruminations,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '20634.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42100,[X]Predominantly compulsive acts [obsessional rituals],Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '22019.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42200,[X]Mixed obsessional thoughts and acts,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '18399.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42y00,[X]Other obsessive-compulsive disorders,Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '38809.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-Eu42z00,"[X]Obsessive-compulsive disorder, unspecified",Read codes v2,C1320,3779,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Medcode': '22721.0', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
-15566,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': 'Obsessive-compulsive disorder NOS', 'Readcode': 'E203z00'}"
-18399,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Mixed obsessional thoughts and acts', 'Readcode': 'Eu42200'}"
-2030,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': 'Obsessional neurosis', 'Readcode': 'E203100'}"
-20634,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Predominantly obsessional thoughts or ruminations', 'Readcode': 'Eu42000'}"
-21836,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Obsessive-compulsive neurosis', 'Readcode': 'Eu42.12'}"
-22019,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Predominantly compulsive acts [obsessional rituals]', 'Readcode': 'Eu42100'}"
-22721,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Obsessive-compulsive disorder, unspecified', 'Readcode': 'Eu42z00'}"
-24251,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Anankastic neurosis', 'Readcode': 'Eu42.11'}"
-3208,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': 'Obsessive-compulsive disorders', 'Readcode': 'E203.00'}"
-38809,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Other obsessive-compulsive disorders', 'Readcode': 'Eu42y00'}"
-47365,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': 'Anancastic neurosis', 'Readcode': 'E203.11'}"
-5304,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': '[X]Obsessive - compulsive disorder', 'Readcode': 'Eu42.00'}"
-5678,Diagnosis of Obsessive-compulsive disorder,Med codes,C1321,3781,Obsessive-compulsive disorder - Primary care,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'ReadcodeDescr': 'Compulsive neurosis', 'Readcode': 'E203000'}"
-F42,Obsessive-compulsive disorder,ICD10 codes,C1322,3783,Obsessive-compulsive disorder - Secondary care - Diagnoses,PH223,446,Obsessive-compulsive disorder,"{'Disease': 'Obsessive-compulsive disorder', 'Category': 'Diagnosis of Obsessive-compulsive disorder'}"
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Palliative care codes/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Palliative care codes/Where this code list comes from .docx
deleted file mode 100644
index a29907527c6291410db894218ad8169c5d479544..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Palliative care codes/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Palliative care codes/res176-palliative-and-end-of-life-care.csv b/examples/codes/clinical-codes-org/Palliative care codes/res176-palliative-and-end-of-life-care.csv
deleted file mode 100644
index ba9b96e137be24a89b11793ed25134b5c1905f6a..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Palliative care codes/res176-palliative-and-end-of-life-care.csv
+++ /dev/null
@@ -1,55 +0,0 @@
-code,coding_system,description,entity,list_name,upload_date
-66S3.00,Read,full care by hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-66S4.00,Read,shared care - hospice / gp,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8B2a.00,Read,prescription of palliative care anticipatory medication,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BA2.11,Read,end of life care,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAe.00,Read,anticipatory palliative care,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAN.00,Read,community specialist palliative care,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAP.00,Read,specialist palliative care,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAQ.00,Read,final days pathway,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAR.00,Read,specialist palliative care treatment - inpatient,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAS.00,Read,specialist palliative care treatment - daycare,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BAT.00,Read,specialist palliative care treatment - outpatient,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BJ1.00,Read,palliative treatment,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BMM.00,Read,issue of palliative care anticipatory medication box,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8BMM.11,Read,issue of palliative care just in case box,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8CM1.00,Read,on gold standards palliative care framework,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8CMW300,Read,end of life care pathway,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8H7g.00,Read,referral to palliative care service,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8Hg0.00,Read,discharged from community specialist palliative care team,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8Hga.00,Read,discharge from hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HgX.00,Read,discharge from palliative care service,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HH7.00,Read,referred to community specialist palliative care team,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HX..00,Read,admission to hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HX0.00,Read,urgent admission to hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HX1.00,Read,routine admission to hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HX2.00,Read,admission to hospice for respite,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-8HY..00,Read,referral to hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-949A.00,Read,patient died in hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-949G.00,Read,patient died in hospice community lodge,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9b1B.00,Read,hospice - nhs,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9b1C.00,Read,hospice - independent,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9b74.00,Read,hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9b74000,Read,hospice community lodge,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9b9B.00,Read,palliative medicine,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9c0L000,Read,planned palliative oncology treatment,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9c0P.00,Read,current palliative oncology treatment,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9e02.00,Read,notif to primary care oohs of palliative care plan in place,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9Ng7.00,Read,on end of life care register,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9NgD.00,Read,under care of palliative care service,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9Nh0.00,Read,under the care of community palliative care team,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9NkJ.00,Read,seen in hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9NlJ.00,Read,seen by palliative care service,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9NNd.00,Read,under care of palliative care specialist nurse,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9NNf000,Read,under care of palliative care physician,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-9NNZ.00,Read,has end of life care pathway key worker,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-M270300,Read,hospice acquired pressure ulcer,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-Z172.00,Read,palliative care,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZL18R00,Read,under care of palliative care physician,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZL5AP00,Read,referral to palliative care physician,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZL9AR00,Read,seen by palliative care physician,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZLD3R00,Read,discharge by palliative care physician,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZLE6P00,Read,discharge from palliative care service,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZLF1.00,Read,discharge from hospice,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZLF1100,Read,discharge from hospice day hospital,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
-ZV57C00,Read,[v]palliative care,diagnostic,883 - res176: Palliative and end of life care,18/02/2021
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Peptic ulcer/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Peptic ulcer/Where this code list comes from .docx
deleted file mode 100644
index 5a21e4cd0598ab3ac980a5b767c43f5a605e113f..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Peptic ulcer/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Peptic ulcer/phenotype_PH1091_ver_2385_concepts_20230719T122902.csv b/examples/codes/clinical-codes-org/Peptic ulcer/phenotype_PH1091_ver_2385_concepts_20230719T122902.csv
deleted file mode 100644
index 9b1ed3f02c48e976a56f7332656a52319f06083d..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Peptic ulcer/phenotype_PH1091_ver_2385_concepts_20230719T122902.csv
+++ /dev/null
@@ -1,104 +0,0 @@
-code,description,coding_system,concept_id,concept_version_id,concept_name,phenotype_id,phenotype_version_id,phenotype_name,Medcode
-14C1.00,H/O: peptic ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,2812
-14C1.11,H/O: duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,6321
-14C1.12,H/O: gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,7108
-7612111,Balfour excision of gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,92695
-7612500,Resection of gastric ulcer by cautery,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,64710
-761J.00,Operations on gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,23688
-761J.11,Stomach ulcer operations,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,6865
-761J000,Closure of perforated gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,20677
-761J100,Closure of gastric ulcer NEC,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,64014
-761J111,Suture of ulcer of stomach NEC,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,70005
-761Jy00,Other specified operation on gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,94104
-761Jz00,Operation on gastric ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,33914
-7627,Operations on duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,5928
-7627000,Closure of perforated duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,4741
-7627100,Suture of duodenal ulcer not elsewhere classified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,37268
-7627y00,Other specified operation on duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,56005
-7627z00,Operation on duodenal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,55933
-J102000,Peptic ulcer of oesophagus,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,24021
-J11..00,Gastric ulcer - (GU),Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,1262
-J11..11,Prepyloric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,6333
-J11..12,Pyloric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,3101
-J110.00,Acute gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,24040
-J110000,Acute gastric ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,64165
-J110200,Acute gastric ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,14671
-J110400,Acute gastric ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,109546
-J110y00,Acute gastric ulcer unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,24342
-J110z00,Acute gastric ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,44324
-J111.00,Chronic gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,18654
-J111000,Chronic gastric ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,48946
-J111200,Chronic gastric ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,53336
-J111211,Perforated chronic gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,11104
-J111400,Chronic gastric ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,67356
-J111y00,Chronic gastric ulcer unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,64556
-J111z00,Chronic gastric ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,44309
-J112.00,Anti-platelet induced gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,63001
-J112z00,Anti-platelet induced gastric ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,89227
-J113.00,Non steroidal anti inflammatory drug induced gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,89023
-J11y.00,Unspecified gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,53081
-J11y000,Unspecified gastric ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,73338
-J11y200,Unspecified gastric ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,36461
-J11y400,Unspecified gastric ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,73697
-J11yz00,Unspecified gastric ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,44284
-J11z.00,Gastric ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,29771
-J11z.12,Multiple gastric ulcers,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,52323
-J12..00,Duodenal ulcer - (DU),Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,352
-J120.00,Acute duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,18027
-J120000,Acute duodenal ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,44335
-J120200,Acute duodenal ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,18324
-J120400,Acute duodenal ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,73417
-J120y00,Acute duodenal ulcer unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,53822
-J120z00,Acute duodenal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,53797
-J121.00,Chronic duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,9853
-J121000,Chronic duodenal ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,33438
-J121200,Chronic duodenal ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,37643
-J121211,Perforated chronic duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,23087
-J121400,Chronic duodenal ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,44073
-J121y00,Chronic duodenal ulcer unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,52138
-J121z00,Chronic duodenal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,51406
-J122.00,Duodenal ulcer disease,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,22918
-J123.00,Duodenal erosion,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,3462
-J124.00,Recurrent duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,29317
-J125.00,Anti-platelet induced duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,89234
-J126.00,Non steroidal anti inflammatory drug induced duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,85989
-J12y.00,Unspecified duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,53669
-J12y000,Unspecified duodenal ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,71150
-J12y200,Unspecified duodenal ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,657
-J12y400,Unspecified duodenal ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,71904
-J12yz00,Unspecified duodenal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,65737
-J12z.00,Duodenal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,15175
-J13..00,Peptic ulcer - (PU) site unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,670
-J13..11,Stress ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,15821
-J130.00,Acute peptic ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,32856
-J130000,Acute peptic ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,68661
-J130200,Acute peptic ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,5521
-J130y00,Acute peptic ulcer unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,67711
-J130z00,Acute peptic ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,50048
-J131.00,Chronic peptic ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,40997
-J131000,Chronic peptic ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,99430
-J131200,Chronic peptic ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,37620
-J131400,Chronic peptic ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,52313
-J131y00,Chronic peptic ulcer unspecified,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,70390
-J131z00,Chronic peptic ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,69663
-J13y.00,Unspecified peptic ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,50497
-J13y000,Unspecified peptic ulcer without mention of complication,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,67082
-J13y200,Unspecified peptic ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,64111
-J13y400,Unspecified peptic ulcer with obstruction,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,99670
-J13yz00,Unspecified peptic ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,60249
-J13z.00,Peptic ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,19928
-J14..00,Gastrojejunal ulcer (GJU),Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,23082
-J14..12,Gastrocolic ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,64913
-J14..13,Jejunal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,9981
-J140.00,Acute gastrojejunal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,45184
-J140200,Acute gastrojejunal ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,102177
-J140z00,Acute gastrojejunal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,42274
-J141.00,Chronic gastrojejunal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,96090
-J14y.00,Unspecified gastrojejunal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,63482
-J14y200,Unspecified gastrojejunal ulcer with perforation,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,15979
-J14yz00,Unspecified gastrojejunal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,66092
-J14z.00,Gastrojejunal ulcer NOS,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,15403
-J17y800,Healed gastric ulcer leaving a scar,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,18319
-ZV12711,[V]Personal history of peptic ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,26261
-ZV12712,[V]Personal history of duodenal ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,40489
-ZV12C00,[V] Personal history of gastric ulcer,Read codes v2,C3359,10255,Peptic ulcer disease Read codes - Elixhauser,PH1091,2385,Peptic ulcer disease - Elixhauser primary care,41271
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Personality disorders/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Personality disorders/Where this code list comes from .docx
deleted file mode 100644
index e577b9a9545dba087b0402d039016e1cbc13e1dc..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Personality disorders/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Personality disorders/res38-personality-disorder.csv b/examples/codes/clinical-codes-org/Personality disorders/res38-personality-disorder.csv
deleted file mode 100644
index 9a7ef84a3411b15fb0e17d37d269344c9294eb70..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Personality disorders/res38-personality-disorder.csv
+++ /dev/null
@@ -1,79 +0,0 @@
-code,coding_system,description,entity,list_name,upload_date
-E21..00,Read,Personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-E210.00,Read,Paranoid personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E211.00,Read,Affective personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E211000,Read,Unspecified affective personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21..11,Read,Neurotic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E211100,Read,Hypomanic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E211200,Read,Depressive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E211300,Read,Cyclothymic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E211z00,Read,Affective personality disorder NOS,diagnostic,381 - res38: Personality disorder,20/01/2016
-E212.00,Read,Schizoid personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E212000,Read,Unspecified schizoid personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E212z00,Read,Schizoid personality disorder NOS,diagnostic,381 - res38: Personality disorder,20/01/2016
-E213.00,Read,Explosive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E214.00,Read,Compulsive personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-E214000,Read,Anankastic personality,diagnostic,381 - res38: Personality disorder,20/01/2016
-E214.11,Read,Anancastic personality,diagnostic,381 - res38: Personality disorder,20/01/2016
-E214z00,Read,Compulsive personality disorder NOS,diagnostic,381 - res38: Personality disorder,20/01/2016
-E215.00,Read,Histrionic personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-E215000,Read,Unspecified histrionic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E215.11,Read,Hysterical personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-E215z00,Read,Histrionic personality disorder NOS,diagnostic,381 - res38: Personality disorder,20/01/2016
-E216.00,Read,Inadequate personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E217.00,Read,Antisocial or sociopathic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y.00,Read,Other personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y000,Read,Narcissistic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y100,Read,Avoidant personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y200,Read,Borderline personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y300,Read,Passive-aggressive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y400,Read,Eccentric personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y500,Read,Immature personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y600,Read,Masochistic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21y700,Read,Psychoneurotic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21yz00,Read,Other personality disorder NOS,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21z.00,Read,Personality disorder NOS,diagnostic,381 - res38: Personality disorder,20/01/2016
-E21z.11,Read,Psychopathic personality,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu06000,Read,[X]Organic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu06011,Read,[X]Organic pseudopsychopathic personality,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu21.17,Read,[X]Pseudopsychopathic schizophrenia,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu21.18,Read,[X]Schizotypal personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu34011,Read,[X]Affective personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu34112,Read,[X]Depressive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60.00,Read,[X]Specific personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60000,Read,[X]Paranoid personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60013,Read,[X]Querulant personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60014,Read,[X]Sensitive paranoid personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60100,Read,[X]Schizoid personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60200,Read,[X]Dissocial personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60212,Read,[X]Antisocial personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60213,Read,[X]Asocial personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60214,Read,[X]Psychopathic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60215,Read,[X]Sociopathic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60300,Read,[X]Emotionally unstable personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60311,Read,[X]Aggressive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60312,Read,[X]Borderline personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60313,Read,[X]Explosive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60400,Read,[X]Histrionic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60411,Read,[X]Hysterical personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60412,Read,[X]Psychoinfantile personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60500,Read,[X]Anankastic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60511,Read,[X]Compulsive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60512,Read,[X]Obsessional personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60513,Read,[X]Obsessive-compulsive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60600,Read,[X]Anxious [avoidant] personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60700,Read,[X]Dependent personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60711,Read,[X]Asthenic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60712,Read,[X]Inadequate personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60713,Read,[X]Passive personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60714,Read,[X]Self defeating personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60y00,Read,[X]Other specific personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60y11,Read,[X]Eccentric personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60y12,Read,[X]Haltlose type personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60y13,Read,[X]Immature personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60y14,Read,[X]Narcissistic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60y16,Read,[X]Psychoneurotic personality disorder,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu60z00,Read,[X]Personality disorder; unspecified,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu61.00,Read,[X]Mixed and other personality disorders,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu84511,Read,[X]Autistic psychopathy,diagnostic,381 - res38: Personality disorder,20/01/2016
-Eu94211,Read,[X]Affectionless psychopathy,diagnostic,381 - res38: Personality disorder,20/01/2016
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Self harm/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Self harm/Where this code list comes from .docx
deleted file mode 100644
index 7bc56d6b4ce7206455a5055eeafbf56657ae814f..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Self harm/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Self harm/res41-self-harm.csv b/examples/codes/clinical-codes-org/Self harm/res41-self-harm.csv
deleted file mode 100644
index 8a22c642d18ddc7d8852095b40b6f105f17aecee..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Self harm/res41-self-harm.csv
+++ /dev/null
@@ -1,262 +0,0 @@
-code,coding_system,description,entity,list_name,upload_date
-14K1.00,Read,Intentional overdose of prescription only medication,diagnostic,393 - res41: Self-harm,15/02/2016
-SL...14,Read,Overdose of biological substance,diagnostic,393 - res41: Self-harm,15/02/2016
-SL...15,Read,Overdose of drug,diagnostic,393 - res41: Self-harm,15/02/2016
-SL90.00,Read,Antidepressant poisoning,diagnostic,393 - res41: Self-harm,15/02/2016
-SL90z00,Read,Anti-depressant poisoning NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-SLHz.00,Read,Drug and medicament poisoning NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...00,Read,Suicide and selfinflicted injury,diagnostic,393 - res41: Self-harm,15/02/2016
-TK0..00,Read,Suicide + selfinflicted poisoning by solid/liquid substances,diagnostic,393 - res41: Self-harm,15/02/2016
-TK00.00,Read,Suicide + selfinflicted poisoning by analgesic/antipyretic,diagnostic,393 - res41: Self-harm,15/02/2016
-TK01.00,Read,Suicide + selfinflicted poisoning by barbiturates,diagnostic,393 - res41: Self-harm,15/02/2016
-TK01000,Read,Suicide and self inflicted injury by Amylobarbitone,diagnostic,393 - res41: Self-harm,15/02/2016
-TK01100,Read,Suicide and self inflicted injury by Barbitone,diagnostic,393 - res41: Self-harm,15/02/2016
-TK01400,Read,Suicide and self inflicted injury by Phenobarbitone,diagnostic,393 - res41: Self-harm,15/02/2016
-TK02.00,Read,Suicide + selfinflicted poisoning by oth sedatives/hypnotics,diagnostic,393 - res41: Self-harm,15/02/2016
-TK03.00,Read,Suicide + selfinflicted poisoning tranquilliser/psychotropic,diagnostic,393 - res41: Self-harm,15/02/2016
-TK04.00,Read,Suicide + selfinflicted poisoning by other drugs/medicines,diagnostic,393 - res41: Self-harm,15/02/2016
-TK05.00,Read,Suicide + selfinflicted poisoning by drug or medicine NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK06.00,Read,Suicide + selfinflicted poisoning by agricultural chemical,diagnostic,393 - res41: Self-harm,15/02/2016
-TK07.00,Read,Suicide + selfinflicted poisoning by corrosive/caustic subst,diagnostic,393 - res41: Self-harm,15/02/2016
-TK0z.00,Read,Suicide + selfinflicted poisoning by solid/liquid subst NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK1..00,Read,Suicide + selfinflicted poisoning by gases in domestic use,diagnostic,393 - res41: Self-harm,15/02/2016
-TK10.00,Read,Suicide + selfinflicted poisoning by gas via pipeline,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...11,Read,Cause of overdose - deliberate,diagnostic,393 - res41: Self-harm,15/02/2016
-TK11.00,Read,Suicide + selfinflicted poisoning by liquified petrol gas,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...12,Read,Injury - self-inflicted,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...13,Read,Poisoning - self-inflicted,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...14,Read,Suicide and self harm,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...15,Read,Attempted suicide,diagnostic,393 - res41: Self-harm,15/02/2016
-TK...17,Read,Para-suicide,diagnostic,393 - res41: Self-harm,15/02/2016
-TK1y.00,Read,Suicide and selfinflicted poisoning by other utility gas,diagnostic,393 - res41: Self-harm,15/02/2016
-TK1z.00,Read,Suicide + selfinflicted poisoning by domestic gases NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK2..00,Read,Suicide + selfinflicted poisoning by other gases and vapours,diagnostic,393 - res41: Self-harm,15/02/2016
-TK20.00,Read,Suicide + selfinflicted poisoning by motor veh exhaust gas,diagnostic,393 - res41: Self-harm,15/02/2016
-TK21.00,Read,Suicide and selfinflicted poisoning by other carbon monoxide,diagnostic,393 - res41: Self-harm,15/02/2016
-TK2z.00,Read,Suicide + selfinflicted poisoning by gases and vapours NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK3..00,Read,Suicide + selfinflicted injury by hang/strangulate/suffocate,diagnostic,393 - res41: Self-harm,15/02/2016
-TK30.00,Read,Suicide and selfinflicted injury by hanging,diagnostic,393 - res41: Self-harm,15/02/2016
-TK31.00,Read,Suicide + selfinflicted injury by suffocation by plastic bag,diagnostic,393 - res41: Self-harm,15/02/2016
-TK3y.00,Read,Suicide + selfinflicted inj oth mean hang/strangle/suffocate,diagnostic,393 - res41: Self-harm,15/02/2016
-TK3z.00,Read,Suicide + selfinflicted inj by hang/strangle/suffocate NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK4..00,Read,Suicide and selfinflicted injury by drowning,diagnostic,393 - res41: Self-harm,15/02/2016
-TK5..00,Read,Suicide and selfinflicted injury by firearms and explosives,diagnostic,393 - res41: Self-harm,15/02/2016
-TK51.00,Read,Suicide and selfinflicted injury by shotgun,diagnostic,393 - res41: Self-harm,15/02/2016
-TK52.00,Read,Suicide and selfinflicted injury by hunting rifle,diagnostic,393 - res41: Self-harm,15/02/2016
-TK54.00,Read,Suicide and selfinflicted injury by other firearm,diagnostic,393 - res41: Self-harm,15/02/2016
-TK5z.00,Read,Suicide and selfinflicted injury by firearms/explosives NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK6..00,Read,Suicide and selfinflicted injury by cutting and stabbing,diagnostic,393 - res41: Self-harm,15/02/2016
-TK60.00,Read,Suicide and selfinflicted injury by cutting,diagnostic,393 - res41: Self-harm,15/02/2016
-TK60100,Read,Self inflicted lacerations to wrist,diagnostic,393 - res41: Self-harm,15/02/2016
-TK60111,Read,Slashed wrists self inflicted,diagnostic,393 - res41: Self-harm,15/02/2016
-TK61.00,Read,Suicide and selfinflicted injury by stabbing,diagnostic,393 - res41: Self-harm,15/02/2016
-TK6z.00,Read,Suicide and selfinflicted injury by cutting and stabbing NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TK7..00,Read,Suicide and selfinflicted injury by jumping from high place,diagnostic,393 - res41: Self-harm,15/02/2016
-TK70.00,Read,Suicide+selfinflicted injury-jump from residential premises,diagnostic,393 - res41: Self-harm,15/02/2016
-TK71.00,Read,Suicide+selfinflicted injury-jump from oth manmade structure,diagnostic,393 - res41: Self-harm,15/02/2016
-TK72.00,Read,Suicide+selfinflicted injury-jump from natural sites,diagnostic,393 - res41: Self-harm,15/02/2016
-TK7z.00,Read,Suicide+selfinflicted injury-jump from high place NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx..00,Read,Suicide and selfinflicted injury by other means,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx0.00,Read,Suicide + selfinflicted injury-jump/lie before moving object,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx0000,Read,Suicide + selfinflicted injury-jumping before moving object,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx1.00,Read,Suicide and selfinflicted injury by burns or fire,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx2.00,Read,Suicide and selfinflicted injury by scald,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx3.00,Read,Suicide and selfinflicted injury by extremes of cold,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx4.00,Read,Suicide and selfinflicted injury by electrocution,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx5.00,Read,Suicide and selfinflicted injury by crashing motor vehicle,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx6.00,Read,Suicide and selfinflicted injury by crashing of aircraft,diagnostic,393 - res41: Self-harm,15/02/2016
-TKx7.00,Read,Suicide and selfinflicted injury caustic subst; excl poison,diagnostic,393 - res41: Self-harm,15/02/2016
-TKxy.00,Read,Suicide and selfinflicted injury by other specified means,diagnostic,393 - res41: Self-harm,15/02/2016
-TKxz.00,Read,Suicide and selfinflicted injury by other means NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-TKy..00,Read,Late effects of selfinflicted injury,diagnostic,393 - res41: Self-harm,15/02/2016
-TKz..00,Read,Suicide and selfinflicted injury NOS,diagnostic,393 - res41: Self-harm,15/02/2016
-U2...00,Read,[X]Intentional self-harm,diagnostic,393 - res41: Self-harm,15/02/2016
-U20..00,Read,[X]Intentional self poisoning/exposure to noxious substances,diagnostic,393 - res41: Self-harm,15/02/2016
-U200.00,Read,[X]Intent self poison/exposure to nonopioid analgesic,diagnostic,393 - res41: Self-harm,15/02/2016
-U200000,Read,[X]Int self poison/exposure to nonopioid analgesic at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U200100,Read,[X]Intent self poison nonopioid analgesic at res institut,diagnostic,393 - res41: Self-harm,15/02/2016
-U200.11,Read,[X]Overdose - paracetamol,diagnostic,393 - res41: Self-harm,15/02/2016
-U200.12,Read,[X]Overdose - ibuprofen,diagnostic,393 - res41: Self-harm,15/02/2016
-U200.13,Read,[X]Overdose - aspirin,diagnostic,393 - res41: Self-harm,15/02/2016
-U200400,Read,[X]Intent self pois nonopioid analgesic in street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U200500,Read,[X]Intent self pois nonopioid analgesic trade/service area,diagnostic,393 - res41: Self-harm,15/02/2016
-U200y00,Read,[X]Int self poison nonopioid analgesic other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U200z00,Read,[X]Intent self poison nonopioid analgesic unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U201.00,Read,[X]Intent self poison/exposure to antiepileptic,diagnostic,393 - res41: Self-harm,15/02/2016
-U201000,Read,[X]Int self poison/exposure to antiepileptic at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U20..11,Read,[X]Deliberate drug overdose / other poisoning,diagnostic,393 - res41: Self-harm,15/02/2016
-U201z00,Read,[X]Intent self poison antiepileptic unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.00,Read,[X]Intent self poison/exposure to sedative hypnotic,diagnostic,393 - res41: Self-harm,15/02/2016
-U202000,Read,[X]Int self poison/exposure to sedative hypnotic at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.11,Read,[X]Overdose - sleeping tabs,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.12,Read,[X]Overdose - diazepam,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.13,Read,[X]Overdose - temazepam,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.15,Read,[X]Overdose - nitrazepam,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.16,Read,[X]Overdose - benzodiazepine,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.17,Read,[X]Overdose - barbiturate,diagnostic,393 - res41: Self-harm,15/02/2016
-U202.18,Read,[X]Overdose - amobarbital,diagnostic,393 - res41: Self-harm,15/02/2016
-U202400,Read,[X]Intent self pois sedative hypnotic in street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U202y00,Read,[X]Int self poison sedative hypnotic other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U202z00,Read,[X]Intent self poison sedative hypnotic unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U204.00,Read,[X]Intent self poison/exposure to psychotropic drug,diagnostic,393 - res41: Self-harm,15/02/2016
-U204000,Read,[X]Int self poison/exposure to psychotropic drug at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U204100,Read,[X]Intent self poison psychotropic drug at res institut,diagnostic,393 - res41: Self-harm,15/02/2016
-U204.11,Read,[X]Overdose - antidepressant,diagnostic,393 - res41: Self-harm,15/02/2016
-U204.12,Read,[X]Overdose - amitriptyline,diagnostic,393 - res41: Self-harm,15/02/2016
-U204.13,Read,[X]Overdose - SSRI,diagnostic,393 - res41: Self-harm,15/02/2016
-U204y00,Read,[X]Int self poison psychotropic drug other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U204z00,Read,[X]Intent self poison psychotropic drug unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U205.00,Read,[X]Intent self poison/exposure to narcotic drug,diagnostic,393 - res41: Self-harm,15/02/2016
-U205000,Read,[X]Int self poison/exposure to narcotic drug at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U205y00,Read,[X]Int self poison narcotic drug other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U205z00,Read,[X]Intent self poison narcotic drug unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U206.00,Read,[X]Intent self poison/exposure to hallucinogen,diagnostic,393 - res41: Self-harm,15/02/2016
-U206400,Read,[X]Intent self pois hallucinogen in street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U207.00,Read,[X]Intent self poison/exposure to oth autonomic drug,diagnostic,393 - res41: Self-harm,15/02/2016
-U207000,Read,[X]Int self poison/exposure to oth autonomic drug at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U207z00,Read,[X]Intent self poison oth autonomic drug unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U208.00,Read,[X]Int self poison/exposure to other/unspec drug/medicament,diagnostic,393 - res41: Self-harm,15/02/2016
-U208000,Read,[X]Int self poison/exposure to oth/unsp drug/medicam home,diagnostic,393 - res41: Self-harm,15/02/2016
-U208400,Read,[X]Intent self pois oth/unsp drug/medic in street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U208y00,Read,[X]Int self poison oth/unsp drug/medic other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U208z00,Read,[X]Intent self poison oth/unsp drug/medic unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U209.00,Read,[X]Intent self poison/exposure to alcohol,diagnostic,393 - res41: Self-harm,15/02/2016
-U209y00,Read,[X]Int self poison alcohol other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U209z00,Read,[X]Intent self poison alcohol unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U20A.00,Read,[X]Intentional self poison organ solvent;halogen hydrocarb,diagnostic,393 - res41: Self-harm,15/02/2016
-U20A000,Read,[X]Intent self pois organ solvent;halogen hydrocarb; home,diagnostic,393 - res41: Self-harm,15/02/2016
-U20A.11,Read,[X]Self poisoning from glue solvent,diagnostic,393 - res41: Self-harm,15/02/2016
-U20A400,Read,[X]Int self poison org solvent;halogen hydrocarb;in highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U20Az00,Read,[X]Int self pois org solv;halogen hydrocarb; unspec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U20B.00,Read,[X]Intent self poison/exposure to other gas/vapour,diagnostic,393 - res41: Self-harm,15/02/2016
-U20B000,Read,[X]Int self poison/exposure to other gas/vapour at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U20B.11,Read,[X]Self carbon monoxide poisoning,diagnostic,393 - res41: Self-harm,15/02/2016
-U20B200,Read,[X]Int self poison other gas/vapour school/pub admin area,diagnostic,393 - res41: Self-harm,15/02/2016
-U20By00,Read,[X]Int self poison other gas/vapour other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U20Bz00,Read,[X]Intent self poison other gas/vapour unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U20C.00,Read,[X]Intent self poison/exposure to pesticide,diagnostic,393 - res41: Self-harm,15/02/2016
-U20C000,Read,[X]Int self poison/exposure to pesticide at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U20C.11,Read,[X]Self poisoning with weedkiller,diagnostic,393 - res41: Self-harm,15/02/2016
-U20C.12,Read,[X]Self poisoning with paraquat,diagnostic,393 - res41: Self-harm,15/02/2016
-U20Cy00,Read,[X]Int self poison pesticide other spec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U20y.00,Read,[X]Intent self poison/exposure to unspecif chemical,diagnostic,393 - res41: Self-harm,15/02/2016
-U20y000,Read,[X]Int self poison/exposure to unspecif chemical at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U20y200,Read,[X]Int self poison unspecif chemical school/pub admin area,diagnostic,393 - res41: Self-harm,15/02/2016
-U20yz00,Read,[X]Intent self poison unspecif chemical unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U21..00,Read,[X]Intent self harm by hanging strangulation / suffocation,diagnostic,393 - res41: Self-harm,15/02/2016
-U210.00,Read,[X]Intent self harm by hanging strangulat/suffocat occ home,diagnostic,393 - res41: Self-harm,15/02/2016
-U2...11,Read,[X]Self inflicted injury,diagnostic,393 - res41: Self-harm,15/02/2016
-U211.00,Read,[X]Intent self harm by hangng strangult/suffoct resid instit,diagnostic,393 - res41: Self-harm,15/02/2016
-U2...12,Read,[X]Injury - self-inflicted,diagnostic,393 - res41: Self-harm,15/02/2016
-U212.00,Read,[X]Inten slf harm hang strang/suffc sch oth ins/pub adm area,diagnostic,393 - res41: Self-harm,15/02/2016
-U2...13,Read,[X]Suicide,diagnostic,393 - res41: Self-harm,15/02/2016
-U2...14,Read,[X]Attempted suicide,diagnostic,393 - res41: Self-harm,15/02/2016
-U2...15,Read,[X]Para-suicide,diagnostic,393 - res41: Self-harm,15/02/2016
-U216.00,Read,[X]Intent self harm by hang strangl/suffc indust/constr area,diagnostic,393 - res41: Self-harm,15/02/2016
-U21y.00,Read,[X]Intent self harm by hangng strangul/suffoct oth spec plce,diagnostic,393 - res41: Self-harm,15/02/2016
-U21z.00,Read,[X]Intent self harm by hangng strangul/suffoct unspecif plce,diagnostic,393 - res41: Self-harm,15/02/2016
-U22..00,Read,[X]Intentional self harm by drowning and submersion,diagnostic,393 - res41: Self-harm,15/02/2016
-U220.00,Read,[X]Intent self harm by drowning/submersion occurrn at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U221.00,Read,[X]Intent self harm by drowning/submersn occ resid instit'n,diagnostic,393 - res41: Self-harm,15/02/2016
-U22y.00,Read,[X]Intent self harm by drown/submersn occ oth specif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U22z.00,Read,[X]Intent self harm by drown/submersn occ unspecified place,diagnostic,393 - res41: Self-harm,15/02/2016
-U24..00,Read,[X]Intent self harm by rifle shotgun/larger firearm disch,diagnostic,393 - res41: Self-harm,15/02/2016
-U241.00,Read,[X]Int self harm rifl s'gun/lrg frarm disch occ resid instit,diagnostic,393 - res41: Self-harm,15/02/2016
-U242.00,Read,[X]Int slf hrm rifl s'gun/lrg frarm dis sch/ins/pub adm area,diagnostic,393 - res41: Self-harm,15/02/2016
-U25..00,Read,[X]Intent self harm by other/unspecified firearm discharge,diagnostic,393 - res41: Self-harm,15/02/2016
-U250.00,Read,[X]Intent self harm oth/unspecif firearm disch occ at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U26..00,Read,[X]Intentional self harm by explosive material,diagnostic,393 - res41: Self-harm,15/02/2016
-U27..00,Read,[X]Intentional self harm by smoke; fire and flames,diagnostic,393 - res41: Self-harm,15/02/2016
-U270.00,Read,[X]Intention self harm by smoke fire/flames occurrn at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U274.00,Read,[X]Intent self harm by smoke fire/flame occ street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U27z.00,Read,[X]Intent self harm by smoke fire/flames occ unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U28..00,Read,[X]Intentional self harm by steam hot vapours / hot objects,diagnostic,393 - res41: Self-harm,15/02/2016
-U280.00,Read,[X]Intent self harm by steam hot vapour/hot obj occ at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U28z.00,Read,[X]Intent self harm by steam hot vapour/obj occ unspec place,diagnostic,393 - res41: Self-harm,15/02/2016
-U29..00,Read,[X]Intentional self harm by sharp object,diagnostic,393 - res41: Self-harm,15/02/2016
-U290.00,Read,[X]Intentional self harm by sharp object occurrence at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U291.00,Read,[X]Intent self harm by sharp object occ resident instit'n,diagnostic,393 - res41: Self-harm,15/02/2016
-U294.00,Read,[X]Intention self harm by sharp object occ street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U29y.00,Read,[X]Intention self harm by sharp object occ oth specif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U29z.00,Read,[X]Intentional self harm by sharp object occ unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U2A..00,Read,[X]Intentional self harm by blunt object,diagnostic,393 - res41: Self-harm,15/02/2016
-U2A0.00,Read,[X]Intentional self harm by blunt object occurrence at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U2A1.00,Read,[X]Intent self harm by blunt object occ resident instit'n,diagnostic,393 - res41: Self-harm,15/02/2016
-U2A3.00,Read,[X]Intent self harm by blunt object occ sports/athlet area,diagnostic,393 - res41: Self-harm,15/02/2016
-U2B..00,Read,[X]Intentional self harm by jumping from a high place,diagnostic,393 - res41: Self-harm,15/02/2016
-U2B0.00,Read,[X]Intent self harm by jumping from high place occ at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U2B4.00,Read,[X]Intent self harm by jump from high place occ street/h'way,diagnostic,393 - res41: Self-harm,15/02/2016
-U2B6.00,Read,[X]Int self harm by jump from high place indust/constr area,diagnostic,393 - res41: Self-harm,15/02/2016
-U2By.00,Read,[X]Int self harm by jump from high place occ oth specif plce,diagnostic,393 - res41: Self-harm,15/02/2016
-U2Bz.00,Read,[X]Int self harm by jump from high place occ unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U2C..00,Read,[X]Intent self harm by jumping / lying before moving object,diagnostic,393 - res41: Self-harm,15/02/2016
-U2C1.00,Read,[X]Int self harm jump/lying befr mov obje occ resid instit'n,diagnostic,393 - res41: Self-harm,15/02/2016
-U2C4.00,Read,[X]Int self harm jump/lying befr mov obje occ street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U2Cy.00,Read,[X]Int self harm jump/lying bef mov obje occ oth specif plce,diagnostic,393 - res41: Self-harm,15/02/2016
-U2D..00,Read,[X]Intentional self harm by crashing of motor vehicle,diagnostic,393 - res41: Self-harm,15/02/2016
-U2D0.00,Read,[X]Intent self harm by crash of motor vehicl occurrn at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U2D4.00,Read,[X]Intent self harm by crash motor vehicl occ street/highway,diagnostic,393 - res41: Self-harm,15/02/2016
-U2D6.00,Read,[X]Intent self harm crash motor vehic occ indust/constr area,diagnostic,393 - res41: Self-harm,15/02/2016
-U2E..00,Read,[X]Self mutilation,diagnostic,393 - res41: Self-harm,15/02/2016
-U2y..00,Read,[X]Intentional self harm by other specified means,diagnostic,393 - res41: Self-harm,15/02/2016
-U2y0.00,Read,[X]Intentionl self harm by oth specif means occurrn at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U2y1.00,Read,[X]Intent self harm by oth specif means occ resid instit'n,diagnostic,393 - res41: Self-harm,15/02/2016
-U2yz.00,Read,[X]Intent self harm by oth specif means occ unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U2z..00,Read,[X]Intentional self harm by unspecified means,diagnostic,393 - res41: Self-harm,15/02/2016
-U2z0.00,Read,[X]Intentional self harm by unspecif means occurrn at home,diagnostic,393 - res41: Self-harm,15/02/2016
-U2z2.00,Read,[X]Intent self harm by unspec mean occ sch/ins/pub adm area,diagnostic,393 - res41: Self-harm,15/02/2016
-U2zy.00,Read,[X]Intent self harm by unspecif means occ oth specif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U2zz.00,Read,[X]Intent self harm by unspecif means occ at unspecif place,diagnostic,393 - res41: Self-harm,15/02/2016
-U30..11,Read,[X]Deliberate drug poisoning,diagnostic,393 - res41: Self-harm,15/02/2016
-U41..00,Read,[X]Hanging strangulation + suffocation undetermined intent,diagnostic,393 - res41: Self-harm,15/02/2016
-U44..00,Read,[X]Rifle shotgun+larger firearm discharge undetermin intent,diagnostic,393 - res41: Self-harm,15/02/2016
-U45..00,Read,[X]Other+unspecified firearm discharge undetermined intent,diagnostic,393 - res41: Self-harm,15/02/2016
-U4B..00,Read,[X]Falling jumping/pushed from high place undeterm intent,diagnostic,393 - res41: Self-harm,15/02/2016
-U4Bz.00,Read,[X]Fall jump/push frm high plce undt intnt occ unspecif plce,diagnostic,393 - res41: Self-harm,15/02/2016
-U72..00,Read,[X]Sequel intentn self-harm assault+event of undeterm intent,diagnostic,393 - res41: Self-harm,15/02/2016
-U720.00,Read,[X]Sequelae of intentional self-harm,diagnostic,393 - res41: Self-harm,15/02/2016
-ZRLfC12,Read,HoNOS item 2 - non-accidental self injury,diagnostic,393 - res41: Self-harm,15/02/2016
-ZRn3.00,Read,Suicide intent score subscale - attempt circumstances,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX...00,Read,Self-harm,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1..00,Read,Self-injurious behaviour,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX...11,Read,Self-damage,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX11.00,Read,Biting self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX11.11,Read,Bites self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1..12,Read,SIB - Self-injurious behaviour,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1..13,Read,Deliberate self-harm,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX12.00,Read,Burning self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX13.00,Read,Cutting self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX13100,Read,Cutting own wrists,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX13.11,Read,Cuts self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX15.00,Read,Drowning self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX18.00,Read,Hanging self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX19.00,Read,Hitting self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX19100,Read,Punching self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX19200,Read,Slapping self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1B.00,Read,Jumping from height,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1B100,Read,Jumping from building,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1B200,Read,Jumping from bridge,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1B300,Read,Jumping from cliff,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1C.00,Read,Nipping self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1E.00,Read,Pinching self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1G.00,Read,Scratches self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1H.00,Read,Self-asphyxiation,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1H100,Read,Self-strangulation,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1H200,Read,Self-suffocation,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1I.00,Read,Self-scalding,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1J.00,Read,Self-electrocution,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1K.00,Read,Self-incineration,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1K.11,Read,Setting fire to self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1K.12,Read,Setting self alight,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1L.00,Read,Self-mutilation,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1L100,Read,Self-mutilation of hands,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1L200,Read,Self-mutilation of genitalia,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1L300,Read,Self-mutilation of penis,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1L600,Read,Self-mutilation of ears,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1LD00,Read,[X]Self mutilation,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1M.00,Read,Shooting self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1N.00,Read,Stabbing self,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1Q.00,Read,Throwing self in front of train,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1Q.11,Read,Jumping under train,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1R.00,Read,Throwing self in front of vehicle,diagnostic,393 - res41: Self-harm,15/02/2016
-ZX1S.00,Read,Throwing self onto floor,diagnostic,393 - res41: Self-harm,15/02/2016
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Tuberculosis/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Tuberculosis/Where this code list comes from .docx
deleted file mode 100644
index 6337d1d49d51a77473ddd062daa79c899b3f70a1..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Tuberculosis/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Tuberculosis/phenotype_PH87_ver_174_concepts_20230719T113229.csv b/examples/codes/clinical-codes-org/Tuberculosis/phenotype_PH87_ver_174_concepts_20230719T113229.csv
deleted file mode 100644
index 97f69f9b82e0cfaa3bcb09f4143fdd273db8ab44..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Tuberculosis/phenotype_PH87_ver_174_concepts_20230719T113229.csv
+++ /dev/null
@@ -1,493 +0,0 @@
-code,description,coding_system,concept_id,concept_version_id,concept_name,phenotype_id,phenotype_version_id,phenotype_name,code_attributes
-1411,H/O: tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '5319.0', 'Category': 'History of tuberculosis'}"
-14P9.00,History of tuberculosis drug therapy,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '100469.0', 'Category': 'History of tuberculosis'}"
-65V9.00,Notification of tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '32180.0', 'Category': 'Diagnosis of tuberculosis'}"
-65V9.11,TB - tuberculosis notification,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16582.0', 'Category': 'Diagnosis of tuberculosis'}"
-65Y1.00,On tuberculosis chemoprophylaxis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '47016.0', 'Category': 'Diagnosis of tuberculosis'}"
-65Y2.00,Streptomycin resistant tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '91666.0', 'Category': 'Diagnosis of tuberculosis'}"
-65Y5.00,Isoniazid resistant tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '102673.0', 'Category': 'Diagnosis of tuberculosis'}"
-65Y8.00,Ciprofloxacin resistant tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '97525.0', 'Category': 'Diagnosis of tuberculosis'}"
-65Y9.00,Inactive tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '96477.0', 'Category': 'Diagnosis of tuberculosis'}"
-65Y9.11,Latent tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '95913.0', 'Category': 'Diagnosis of tuberculosis'}"
-8BAD100,TB chemotherapy,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '4247.0', 'Category': 'Diagnosis of tuberculosis'}"
-A1...00,Tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '1840.0', 'Category': 'Diagnosis of tuberculosis'}"
-A10..00,Primary tuberculous infection,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '22011.0', 'Category': 'Diagnosis of tuberculosis'}"
-A100.00,Primary tuberculous complex,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16265.0', 'Category': 'Diagnosis of tuberculosis'}"
-A101.00,Tuberculous pleurisy in primary progressive tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46272.0', 'Category': 'Diagnosis of tuberculosis'}"
-A10y.00,Other primary progressive tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '42630.0', 'Category': 'Diagnosis of tuberculosis'}"
-A10z.00,Primary tuberculous infection NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37694.0', 'Category': 'Diagnosis of tuberculosis'}"
-A11..00,Pulmonary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '635.0', 'Category': 'Diagnosis of tuberculosis'}"
-A11..11,Lung tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '47336.0', 'Category': 'Diagnosis of tuberculosis'}"
-A110.00,Infiltrative lung tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '53701.0', 'Category': 'Diagnosis of tuberculosis'}"
-A111.00,Nodular lung tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '48580.0', 'Category': 'Diagnosis of tuberculosis'}"
-A112.00,Tuberculosis of lung with cavitation,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16331.0', 'Category': 'Diagnosis of tuberculosis'}"
-A113.00,Tuberculosis of bronchus,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '62468.0', 'Category': 'Diagnosis of tuberculosis'}"
-A114.00,Tuberculous fibrosis of lung,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16741.0', 'Category': 'Diagnosis of tuberculosis'}"
-A115.00,Tuberculous bronchiectasis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '15693.0', 'Category': 'Diagnosis of tuberculosis'}"
-A116.00,Tuberculous pneumonia,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '9953.0', 'Category': 'Diagnosis of tuberculosis'}"
-A117.00,Tuberculous pneumothorax,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '66441.0', 'Category': 'Diagnosis of tuberculosis'}"
-A11y.00,Other specified pulmonary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '18950.0', 'Category': 'Diagnosis of tuberculosis'}"
-A11z.00,Pulmonary tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '38110.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12..00,Other respiratory tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '63959.0', 'Category': 'Diagnosis of tuberculosis'}"
-A120.00,Tuberculous pleurisy,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '23472.0', 'Category': 'Diagnosis of tuberculosis'}"
-A120000,Tuberculosis of pleura,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37834.0', 'Category': 'Diagnosis of tuberculosis'}"
-A120100,Tuberculous empyema,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '39512.0', 'Category': 'Diagnosis of tuberculosis'}"
-A120200,Tuberculous hydrothorax,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '14913.0', 'Category': 'Diagnosis of tuberculosis'}"
-A120z00,Tuberculous pleurisy NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '56890.0', 'Category': 'Diagnosis of tuberculosis'}"
-A121.00,Tuberculosis of intrathoracic lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '58827.0', 'Category': 'Diagnosis of tuberculosis'}"
-A121000,Tuberculosis of hilar lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '5145.0', 'Category': 'Diagnosis of tuberculosis'}"
-A121100,Tuberculosis of mediastinal lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '44129.0', 'Category': 'Diagnosis of tuberculosis'}"
-A121200,Tuberculosis of tracheobronchial lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '49503.0', 'Category': 'Diagnosis of tuberculosis'}"
-A121z00,Tuberculosis of intrathoracic lymph nodes NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46926.0', 'Category': 'Diagnosis of tuberculosis'}"
-A122.00,Isolated tracheal or bronchial tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '69260.0', 'Category': 'Diagnosis of tuberculosis'}"
-A122000,Isolated tracheal tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '93015.0', 'Category': 'Diagnosis of tuberculosis'}"
-A122100,Isolated bronchial tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '93948.0', 'Category': 'Diagnosis of tuberculosis'}"
-A122z00,Isolated tracheal or bronchial tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '53473.0', 'Category': 'Diagnosis of tuberculosis'}"
-A123.00,Tuberculous laryngitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '20333.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124.00,Resp TB bacteriologically and histologically confirmed,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31670.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124000,TB lung confirm sputum microscopy with or without culture,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '24413.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124100,"Tuberculosis of lung, confirmed by culture only",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '93071.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124200,"Tuberculosis of lung, confirmed histologically",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '62530.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124300,"Tuberculosis of lung, confirmed by unspecified means",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '58588.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124400,TB intrathoracic lymph nodes confirm bact histologically,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '44655.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124500,"Tuberculosis of larynx, trachea & bronchus conf bact/hist'y",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '44039.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124600,"Tuberculous pleurisy, conf bacteriologically/histologically",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '35443.0', 'Category': 'Diagnosis of tuberculosis'}"
-A124700,Primary respiratory TB confirm bact and histologically,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '24517.0', 'Category': 'Diagnosis of tuberculosis'}"
-A125.00,Respiratory TB not confirmed bact or histologically,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '7133.0', 'Category': 'Diagnosis of tuberculosis'}"
-A125000,"Tuberculosis of lung, bacteriologically & histolog'y neg",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '47832.0', 'Category': 'Diagnosis of tuberculosis'}"
-A125100,Tuberculosis lung bact and histological examin not done,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '41051.0', 'Category': 'Diagnosis of tuberculosis'}"
-A125200,Prim respiratory TB without mention of bact or hist confirm,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '40605.0', 'Category': 'Diagnosis of tuberculosis'}"
-A125X00,"Resp TB unspcf,w'out mention/bacterial or histol confrmtn",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '69471.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12y.00,Other specified respiratory tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '50902.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12y000,Tuberculosis of mediastinum,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37598.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12y100,Tuberculosis of nasopharynx,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '72402.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12y200,Tuberculosis of nasal septum,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '97658.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12y300,Tuberculosis of nasal sinus,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '45861.0', 'Category': 'Diagnosis of tuberculosis'}"
-A12yz00,Other specified respiratory tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '50147.0', 'Category': 'Diagnosis of tuberculosis'}"
-A13..00,Tuberculosis of meninges and central nervous system,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '41208.0', 'Category': 'Diagnosis of tuberculosis'}"
-A130.00,Tuberculous meningitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '11975.0', 'Category': 'Diagnosis of tuberculosis'}"
-A130000,Tuberculosis of cerebral meninges,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '54840.0', 'Category': 'Diagnosis of tuberculosis'}"
-A130100,Tuberculosis of spinal meninges,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46802.0', 'Category': 'Diagnosis of tuberculosis'}"
-A130300,Tuberculous meningoencephalitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '43791.0', 'Category': 'Diagnosis of tuberculosis'}"
-A130z00,Tuberculous meningitis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '69322.0', 'Category': 'Diagnosis of tuberculosis'}"
-A131.00,Tuberculoma of meninges,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '57417.0', 'Category': 'Diagnosis of tuberculosis'}"
-A132.00,Tuberculoma of brain,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '52199.0', 'Category': 'Diagnosis of tuberculosis'}"
-A133.00,Tuberculous abscess of brain,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '57398.0', 'Category': 'Diagnosis of tuberculosis'}"
-A134.00,Tuberculoma of spinal cord,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46039.0', 'Category': 'Diagnosis of tuberculosis'}"
-A135.00,Tuberculous abscess of spinal cord,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46044.0', 'Category': 'Diagnosis of tuberculosis'}"
-A136.00,Tuberculous encephalitis or myelitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '59202.0', 'Category': 'Diagnosis of tuberculosis'}"
-A136000,Tuberculous encephalitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '93360.0', 'Category': 'Diagnosis of tuberculosis'}"
-A136100,Tuberculous myelitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31680.0', 'Category': 'Diagnosis of tuberculosis'}"
-A136z00,Tuberculous encephalitis or myelitis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '72072.0', 'Category': 'Diagnosis of tuberculosis'}"
-A13y.00,Other specified tuberculosis of central nervous system,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '41383.0', 'Category': 'Diagnosis of tuberculosis'}"
-A13z.00,Tuberculosis of central nervous system NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '70140.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14..00,"Tuberculosis of intestines, peritoneum and mesenteric glands",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '2193.0', 'Category': 'Diagnosis of tuberculosis'}"
-A140.00,Tuberculous peritonitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31409.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14y.00,Other gastrointestinal tract tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '54570.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14y100,Tuberculosis of large intestine,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '39279.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14y200,Tuberculosis of small intestine,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31436.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14y300,Tuberculosis of mesenteric lymph glands,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '29482.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14y400,Tuberculosis of rectum,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '59087.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14y500,Tuberculosis of retroperitoneal lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '49433.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14yz00,Other gastrointestinal tract tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '33372.0', 'Category': 'Diagnosis of tuberculosis'}"
-A14z.00,Tuberculosis of gastrointestinal tract NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '54579.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15..00,Tuberculosis of bones and joints,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '3596.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15..11,Tuberculous osteomylelytis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '22572.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15..12,Tuberculous arthritis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '24626.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15..13,Tuberculous synovitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '45318.0', 'Category': 'Diagnosis of tuberculosis'}"
-A150.00,Tuberculosis of vertebral column - Pott's,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '3273.0', 'Category': 'Diagnosis of tuberculosis'}"
-A151.00,Tuberculosis of hip,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '2208.0', 'Category': 'Diagnosis of tuberculosis'}"
-A152.00,Tuberculosis of knee,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '23451.0', 'Category': 'Diagnosis of tuberculosis'}"
-A153.00,Tuberculosis limb bones - Tuberculous dactylitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '4907.0', 'Category': 'Diagnosis of tuberculosis'}"
-A154.00,Tuberculous mastoiditis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '17153.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15x.00,Tuberculosis of other specified bones,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37886.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15y.00,Tuberculosis of other specified joint,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '42201.0', 'Category': 'Diagnosis of tuberculosis'}"
-A15z.00,Tuberculosis of bones or joints NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '44128.0', 'Category': 'Diagnosis of tuberculosis'}"
-A16..00,Tuberculosis of genitourinary system,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37422.0', 'Category': 'Diagnosis of tuberculosis'}"
-A160.00,Tuberculosis of kidney,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '3303.0', 'Category': 'Diagnosis of tuberculosis'}"
-A160.11,Renal tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '23940.0', 'Category': 'Diagnosis of tuberculosis'}"
-A160000,Tuberculous nephropathy,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '52272.0', 'Category': 'Diagnosis of tuberculosis'}"
-A160100,Tuberculous pyelitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '50837.0', 'Category': 'Diagnosis of tuberculosis'}"
-A160200,Tuberculous pyelonephritis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '49235.0', 'Category': 'Diagnosis of tuberculosis'}"
-A160z00,Tuberculosis of kidney NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '67292.0', 'Category': 'Diagnosis of tuberculosis'}"
-A161.00,Tuberculosis of bladder,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '3830.0', 'Category': 'Diagnosis of tuberculosis'}"
-A162.00,Tuberculosis of ureter,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '53548.0', 'Category': 'Diagnosis of tuberculosis'}"
-A163.00,Tuberculosis of other urinary organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '34657.0', 'Category': 'Diagnosis of tuberculosis'}"
-A164.00,Tuberculosis of epididymis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '27399.0', 'Category': 'Diagnosis of tuberculosis'}"
-A165.00,Tuberculosis of other male genital organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '68821.0', 'Category': 'Diagnosis of tuberculosis'}"
-A165000,Tuberculosis of prostate,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '60040.0', 'Category': 'Diagnosis of tuberculosis'}"
-A165100,Tuberculosis seminal vesicle,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '49481.0', 'Category': 'Diagnosis of tuberculosis'}"
-A165200,Tuberculosis of testis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '30945.0', 'Category': 'Diagnosis of tuberculosis'}"
-A165z00,Tuberculosis of other male genital organs NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '69154.0', 'Category': 'Diagnosis of tuberculosis'}"
-A166.00,Tuberculous oophoritis or salpingitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '57466.0', 'Category': 'Diagnosis of tuberculosis'}"
-A166000,Tuberculous oophoritis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '72743.0', 'Category': 'Diagnosis of tuberculosis'}"
-A166100,Tuberculous salpingitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '38920.0', 'Category': 'Diagnosis of tuberculosis'}"
-A166111,Fallopian tube tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31349.0', 'Category': 'Diagnosis of tuberculosis'}"
-A166z00,Tuberculous oophoritis or salpingitis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '63796.0', 'Category': 'Diagnosis of tuberculosis'}"
-A167.00,Tuberculosis of other female genital organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '66584.0', 'Category': 'Diagnosis of tuberculosis'}"
-A167000,Tuberculous cervicitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '49112.0', 'Category': 'Diagnosis of tuberculosis'}"
-A167100,Tuberculous endometritis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '12448.0', 'Category': 'Diagnosis of tuberculosis'}"
-A167z00,Tuberculosis of other female genital organs NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '68973.0', 'Category': 'Diagnosis of tuberculosis'}"
-A168.00,Tuberculosis of urinary tract,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '50489.0', 'Category': 'Diagnosis of tuberculosis'}"
-A16z.00,Genitourinary tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '50261.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17..00,Tuberculosis of other organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '55835.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170.00,Tuberculosis of skin and subcutaneous tissue,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '47881.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170.11,Lupus - tuberculous,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46675.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170000,Tuberculosis - lupus exedens,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '67637.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170100,Tuberculosis - lupus vulgaris,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16367.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170200,Tuberculosis - scrofuloderma,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '47430.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170300,Tuberculosis - lupus NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37492.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170500,Tuberculosis cutis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '63351.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170600,Tuberculosis lichenoides,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '4621.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170700,Tuberculosis papulonecrotica,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '62033.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170800,Tuberculosis verrucosa cutis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31445.0', 'Category': 'Diagnosis of tuberculosis'}"
-A170z00,Tuberculosis of skin and subcutaneous tissue NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '30687.0', 'Category': 'Diagnosis of tuberculosis'}"
-A171.00,Tuberculosis with erythema nodosum hypersensitivity reaction,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '44573.0', 'Category': 'Diagnosis of tuberculosis'}"
-A171000,Bazin's disease - erythema induratum - TB hypersensitivity,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '23962.0', 'Category': 'Diagnosis of tuberculosis'}"
-A171100,Tuberculous erythema nodosum,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '34640.0', 'Category': 'Diagnosis of tuberculosis'}"
-A171z00,Erythema nodosum with tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '56833.0', 'Category': 'Diagnosis of tuberculosis'}"
-A172.00,Tuberculosis of peripheral lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '4256.0', 'Category': 'Diagnosis of tuberculosis'}"
-A172000,Tuberculous - cervical lymphadenitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '3720.0', 'Category': 'Diagnosis of tuberculosis'}"
-A172011,Scrofula - tuberculous cervical lymph nodes,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '40522.0', 'Category': 'Diagnosis of tuberculosis'}"
-A172100,Scrofulous tuberculous abscess,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '51110.0', 'Category': 'Diagnosis of tuberculosis'}"
-A172200,Tuberculous adenitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '5101.0', 'Category': 'Diagnosis of tuberculosis'}"
-A172z00,Tuberculosis of peripheral lymph nodes NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '26344.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173.00,Tuberculosis of eye,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '43976.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173000,Tuberculous chorioretinitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '37329.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173100,Tuberculous episcleritis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '19652.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173200,Tuberculous interstitial keratitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '55629.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173300,Tuberculous chronic iridocyclitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '38342.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173400,Tuberculous keratoconjunctivitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '30358.0', 'Category': 'Diagnosis of tuberculosis'}"
-A173z00,Tuberculosis of eye NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '70491.0', 'Category': 'Diagnosis of tuberculosis'}"
-A174.00,Tuberculosis of ear,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '58673.0', 'Category': 'Diagnosis of tuberculosis'}"
-A175.00,Tuberculosis of thyroid gland,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '73590.0', 'Category': 'Diagnosis of tuberculosis'}"
-A176.00,Tuberculosis of adrenal glands - Addison's disease,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '35760.0', 'Category': 'Diagnosis of tuberculosis'}"
-A177.00,Tuberculosis spleen,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '66976.0', 'Category': 'Diagnosis of tuberculosis'}"
-A178.00,Tuberculosis oesophagus,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '56670.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17y.00,Tuberculosis of other specified organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46727.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17y000,Tuberculosis endocardium,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '96668.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17y100,Tuberculosis myocardium,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '50869.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17y200,Tuberculosis pericardium,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '40231.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17y300,Tuberculosis of stomach,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46019.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17y400,Tuberculosis of liver,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46147.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17yz00,Tuberculosis of other specified organs NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '46383.0', 'Category': 'Diagnosis of tuberculosis'}"
-A17z.00,Tuberculosis of other organs NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '45932.0', 'Category': 'Diagnosis of tuberculosis'}"
-A18..00,Miliary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16414.0', 'Category': 'Diagnosis of tuberculosis'}"
-A180.00,Acute miliary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '72008.0', 'Category': 'Diagnosis of tuberculosis'}"
-A180000,Acute miliary tuberculosis of a single specified site,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '31844.0', 'Category': 'Diagnosis of tuberculosis'}"
-A180100,Acute miliary tuberculosis of multiple sites,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '42479.0', 'Category': 'Diagnosis of tuberculosis'}"
-A18y.00,Other specified miliary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '32459.0', 'Category': 'Diagnosis of tuberculosis'}"
-A18z.00,Miliary tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '53331.0', 'Category': 'Diagnosis of tuberculosis'}"
-A1y..00,Other specified tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '34430.0', 'Category': 'Diagnosis of tuberculosis'}"
-A1z..00,Tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '15158.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE0..00,Late effects of tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '29395.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE00.00,Late effects of respiratory tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '65464.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE01.00,Late effects of central nervous system tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '73273.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE02.00,Late effects of genitourinary system tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '40990.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE03.00,Late effects of tuberculosis of bones and joints,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '73549.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE04.00,Late effects of tuberculosis of other specified organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '70911.0', 'Category': 'Diagnosis of tuberculosis'}"
-AE0z.00,Late effects of tuberculosis NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '61442.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1.00,[X]Tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '73149.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1000,"[X]Other resp tubercul,confirmd bacteriologicly+histologicly",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '73185.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1100,"[X]Resp tuberculos unspcfd,confirmd bacteriolog+histologicly",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '73225.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1300,"[X]Resp TB unspcf,w'out mention/bacterial or histol confrmtn",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '55298.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1400,[X]Other tuberculosis of nervous system,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '106837.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1600,[X]Tuberculosis of other specified organs,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '72680.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1800,[X]Other miliary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '106521.0', 'Category': 'Diagnosis of tuberculosis'}"
-Ayu1900,"[X]Miliary tuberculosis, unspecified",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '97922.0', 'Category': 'Diagnosis of tuberculosis'}"
-AyuJ200,[X]Sequelae of tuberculosis of bones and joints,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '106978.0', 'Category': 'History of tuberculosis'}"
-AyuJ400,[X]Sequelae of respiratory and unspecified tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '93768.0', 'Category': 'History of tuberculosis'}"
-F004.00,Meningitis - tuberculous,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '11976.0', 'Category': 'Diagnosis of tuberculosis'}"
-F033300,Encephalitis due to tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '99925.0', 'Category': 'Diagnosis of tuberculosis'}"
-F033311,Tuberculous encephalitis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '97778.0', 'Category': 'Diagnosis of tuberculosis'}"
-F040600,Tuberculous intracranial abscess,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '54908.0', 'Category': 'Diagnosis of tuberculosis'}"
-F041300,Tuberculous intraspinal abscess,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '105288.0', 'Category': 'Diagnosis of tuberculosis'}"
-F4A5500,Keratitis due to tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '94249.0', 'Category': 'Diagnosis of tuberculosis'}"
-G500300,Acute pericarditis - tuberculous,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '57126.0', 'Category': 'Diagnosis of tuberculosis'}"
-G500311,TB - acute pericarditis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '16996.0', 'Category': 'Diagnosis of tuberculosis'}"
-G520600,Acute myocarditis - tuberculous,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '91847.0', 'Category': 'Diagnosis of tuberculosis'}"
-H450.00,Pneumoconiosis associated with tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '63172.0', 'Category': 'History of tuberculosis'}"
-J550200,Peritonitis - tuberculous,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '32899.0', 'Category': 'Diagnosis of tuberculosis'}"
-K154800,Cystitis in tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '56771.0', 'Category': 'Diagnosis of tuberculosis'}"
-K214300,Prostatitis in tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '109312.0', 'Category': 'Diagnosis of tuberculosis'}"
-K43..00,Female tuberculous pelvic inflammatory disease,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '27611.0', 'Category': 'Diagnosis of tuberculosis'}"
-N018.00,Tuberculous arthritis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '97588.0', 'Category': 'Diagnosis of tuberculosis'}"
-N304.00,Tuberculosis of spine (Pott's),Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '43370.0', 'Category': 'Diagnosis of tuberculosis'}"
-N304.11,Tuberculosis of spine,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '6553.0', 'Category': 'Diagnosis of tuberculosis'}"
-N304000,Tuberculosis of cervical spine,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '70293.0', 'Category': 'Diagnosis of tuberculosis'}"
-N304100,Tuberculosis of thoracic spine,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '67337.0', 'Category': 'Diagnosis of tuberculosis'}"
-N304200,Tuberculosis of lumbar spine,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '65994.0', 'Category': 'Diagnosis of tuberculosis'}"
-N304300,Tuberculosis of sacrum/coccyx,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '97325.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305.00,Tuberculosis of limb bones,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '24372.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305000,Tuberculosis of unspecified limb bone,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '99914.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305100,Tuberculosis of the upper arm bone,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '62963.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305200,Tuberculosis of the forearm bone,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '68154.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305300,Tuberculosis of the pelvic and thigh bones,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '59916.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305400,Tuberculosis of the lower leg bone,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '57587.0', 'Category': 'Diagnosis of tuberculosis'}"
-N305500,Tuberculosis of other limb bones,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '95332.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306.00,Tuberculosis of other bones,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '12338.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306000,"Tuberculosis of bone, site unspecified",Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '99305.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306100,Tuberculosis of the bones of the shoulder region,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '53864.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306200,Tuberculosis of the bones of the hand,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '99783.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306300,Tuberculosis of the bones of the ankle and foot,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '67601.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306400,Tuberculosis of the bones of other sites,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '71138.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306500,Tuberculosis of the bones of multiple sites,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '99593.0', 'Category': 'Diagnosis of tuberculosis'}"
-N306z00,Tuberculosis of bone NOS,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '70862.0', 'Category': 'Diagnosis of tuberculosis'}"
-Q402400,Congenital tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '55957.0', 'Category': 'Diagnosis of tuberculosis'}"
-ZV12A00,[V] Personal history of pulmonary tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '18206.0', 'Category': 'History of tuberculosis'}"
-ZV12B00,[V] Personal history of tuberculosis,Read codes v2,C924,2987,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Medcode': '9158.0', 'Category': 'History of tuberculosis'}"
-100469,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'History of tuberculosis drug therapy', 'Readcode': '14P9.00'}"
-102673,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Isoniazid resistant tuberculosis', 'Readcode': '65Y5.00'}"
-105288,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous intraspinal abscess', 'Readcode': 'F041300'}"
-106521,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Other miliary tuberculosis', 'Readcode': 'Ayu1800'}"
-106837,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Other tuberculosis of nervous system', 'Readcode': 'Ayu1400'}"
-106978,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Sequelae of tuberculosis of bones and joints', 'Readcode': 'AyuJ200'}"
-109312,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Prostatitis in tuberculosis', 'Readcode': 'K214300'}"
-11975,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous meningitis', 'Readcode': 'A130.00'}"
-11976,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Meningitis - tuberculous', 'Readcode': 'F004.00'}"
-12338,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other bones', 'Readcode': 'N306.00'}"
-12448,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous endometritis', 'Readcode': 'A167100'}"
-14913,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous hydrothorax', 'Readcode': 'A120200'}"
-15158,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis NOS', 'Readcode': 'A1z..00'}"
-15693,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous bronchiectasis', 'Readcode': 'A115.00'}"
-16265,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Primary tuberculous complex', 'Readcode': 'A100.00'}"
-16331,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of lung with cavitation', 'Readcode': 'A112.00'}"
-16367,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis - lupus vulgaris', 'Readcode': 'A170100'}"
-16414,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Miliary tuberculosis', 'Readcode': 'A18..00'}"
-16582,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'TB - tuberculosis notification', 'Readcode': '65V9.11'}"
-16741,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous fibrosis of lung', 'Readcode': 'A114.00'}"
-16996,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'TB - acute pericarditis', 'Readcode': 'G500311'}"
-17153,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous mastoiditis', 'Readcode': 'A154.00'}"
-18206,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[V] Personal history of pulmonary tuberculosis', 'Readcode': 'ZV12A00'}"
-1840,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis', 'Readcode': 'A1...00'}"
-18950,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other specified pulmonary tuberculosis', 'Readcode': 'A11y.00'}"
-19652,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous episcleritis', 'Readcode': 'A173100'}"
-20333,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous laryngitis', 'Readcode': 'A123.00'}"
-2193,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of intestines, peritoneum and mesenteric glands', 'Readcode': 'A14..00'}"
-22011,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Primary tuberculous infection', 'Readcode': 'A10..00'}"
-2208,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of hip', 'Readcode': 'A151.00'}"
-22572,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous osteomylelytis', 'Readcode': 'A15..11'}"
-23451,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of knee', 'Readcode': 'A152.00'}"
-23472,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pleurisy', 'Readcode': 'A120.00'}"
-23940,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Renal tuberculosis', 'Readcode': 'A160.11'}"
-23962,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Bazin's disease - erythema induratum - TB hypersensitivity"", 'Readcode': 'A171000'}"
-24372,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of limb bones', 'Readcode': 'N305.00'}"
-24413,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'TB lung confirm sputum microscopy with or without culture', 'Readcode': 'A124000'}"
-24517,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Primary respiratory TB confirm bact and histologically', 'Readcode': 'A124700'}"
-24626,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous arthritis', 'Readcode': 'A15..12'}"
-26344,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of peripheral lymph nodes NOS', 'Readcode': 'A172z00'}"
-27399,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of epididymis', 'Readcode': 'A164.00'}"
-27611,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Female tuberculous pelvic inflammatory disease', 'Readcode': 'K43..00'}"
-29395,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of tuberculosis', 'Readcode': 'AE0..00'}"
-29482,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of mesenteric lymph glands', 'Readcode': 'A14y300'}"
-30358,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous keratoconjunctivitis', 'Readcode': 'A173400'}"
-30687,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of skin and subcutaneous tissue NOS', 'Readcode': 'A170z00'}"
-30945,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of testis', 'Readcode': 'A165200'}"
-31349,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Fallopian tube tuberculosis', 'Readcode': 'A166111'}"
-31409,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous peritonitis', 'Readcode': 'A140.00'}"
-31436,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of small intestine', 'Readcode': 'A14y200'}"
-31445,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis verrucosa cutis', 'Readcode': 'A170800'}"
-31670,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Resp TB bacteriologically and histologically confirmed', 'Readcode': 'A124.00'}"
-31680,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous myelitis', 'Readcode': 'A136100'}"
-31844,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Acute miliary tuberculosis of a single specified site', 'Readcode': 'A180000'}"
-32180,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Notification of tuberculosis', 'Readcode': '65V9.00'}"
-32459,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other specified miliary tuberculosis', 'Readcode': 'A18y.00'}"
-3273,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Tuberculosis of vertebral column - Pott's"", 'Readcode': 'A150.00'}"
-32899,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Peritonitis - tuberculous', 'Readcode': 'J550200'}"
-3303,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of kidney', 'Readcode': 'A160.00'}"
-33372,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other gastrointestinal tract tuberculosis NOS', 'Readcode': 'A14yz00'}"
-34430,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other specified tuberculosis', 'Readcode': 'A1y..00'}"
-34640,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous erythema nodosum', 'Readcode': 'A171100'}"
-34657,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other urinary organs', 'Readcode': 'A163.00'}"
-35443,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pleurisy, conf bacteriologically/histologically', 'Readcode': 'A124600'}"
-35760,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Tuberculosis of adrenal glands - Addison's disease"", 'Readcode': 'A176.00'}"
-3596,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of bones and joints', 'Readcode': 'A15..00'}"
-3720,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous - cervical lymphadenitis', 'Readcode': 'A172000'}"
-37329,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous chorioretinitis', 'Readcode': 'A173000'}"
-37422,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of genitourinary system', 'Readcode': 'A16..00'}"
-37492,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis - lupus NOS', 'Readcode': 'A170300'}"
-37598,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of mediastinum', 'Readcode': 'A12y000'}"
-37694,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Primary tuberculous infection NOS', 'Readcode': 'A10z.00'}"
-37834,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of pleura', 'Readcode': 'A120000'}"
-37886,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other specified bones', 'Readcode': 'A15x.00'}"
-38110,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Pulmonary tuberculosis NOS', 'Readcode': 'A11z.00'}"
-3830,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of bladder', 'Readcode': 'A161.00'}"
-38342,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous chronic iridocyclitis', 'Readcode': 'A173300'}"
-38920,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous salpingitis', 'Readcode': 'A166100'}"
-39279,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of large intestine', 'Readcode': 'A14y100'}"
-39512,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous empyema', 'Readcode': 'A120100'}"
-40231,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis pericardium', 'Readcode': 'A17y200'}"
-40522,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Scrofula - tuberculous cervical lymph nodes', 'Readcode': 'A172011'}"
-40605,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Prim respiratory TB without mention of bact or hist confirm', 'Readcode': 'A125200'}"
-40990,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of genitourinary system tuberculosis', 'Readcode': 'AE02.00'}"
-41051,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis lung bact and histological examin not done', 'Readcode': 'A125100'}"
-41208,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of meninges and central nervous system', 'Readcode': 'A13..00'}"
-41383,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other specified tuberculosis of central nervous system', 'Readcode': 'A13y.00'}"
-42201,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other specified joint', 'Readcode': 'A15y.00'}"
-4247,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'TB chemotherapy', 'Readcode': '8BAD100'}"
-42479,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Acute miliary tuberculosis of multiple sites', 'Readcode': 'A180100'}"
-4256,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of peripheral lymph nodes', 'Readcode': 'A172.00'}"
-42630,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other primary progressive tuberculosis', 'Readcode': 'A10y.00'}"
-43370,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Tuberculosis of spine (Pott's)"", 'Readcode': 'N304.00'}"
-43791,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous meningoencephalitis', 'Readcode': 'A130300'}"
-43976,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of eye', 'Readcode': 'A173.00'}"
-44039,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Tuberculosis of larynx, trachea & bronchus conf bact/hist'y"", 'Readcode': 'A124500'}"
-44128,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of bones or joints NOS', 'Readcode': 'A15z.00'}"
-44129,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of mediastinal lymph nodes', 'Readcode': 'A121100'}"
-44573,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis with erythema nodosum hypersensitivity reaction', 'Readcode': 'A171.00'}"
-44655,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'TB intrathoracic lymph nodes confirm bact histologically', 'Readcode': 'A124400'}"
-45318,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous synovitis', 'Readcode': 'A15..13'}"
-45861,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of nasal sinus', 'Readcode': 'A12y300'}"
-45932,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other organs NOS', 'Readcode': 'A17z.00'}"
-46019,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of stomach', 'Readcode': 'A17y300'}"
-46039,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculoma of spinal cord', 'Readcode': 'A134.00'}"
-46044,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous abscess of spinal cord', 'Readcode': 'A135.00'}"
-46147,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of liver', 'Readcode': 'A17y400'}"
-4621,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis lichenoides', 'Readcode': 'A170600'}"
-46272,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pleurisy in primary progressive tuberculosis', 'Readcode': 'A101.00'}"
-46383,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other specified organs NOS', 'Readcode': 'A17yz00'}"
-46675,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Lupus - tuberculous', 'Readcode': 'A170.11'}"
-46727,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other specified organs', 'Readcode': 'A17y.00'}"
-46802,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of spinal meninges', 'Readcode': 'A130100'}"
-46926,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of intrathoracic lymph nodes NOS', 'Readcode': 'A121z00'}"
-47016,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'On tuberculosis chemoprophylaxis', 'Readcode': '65Y1.00'}"
-47336,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Lung tuberculosis', 'Readcode': 'A11..11'}"
-47430,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis - scrofuloderma', 'Readcode': 'A170200'}"
-47832,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Tuberculosis of lung, bacteriologically & histolog'y neg"", 'Readcode': 'A125000'}"
-47881,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of skin and subcutaneous tissue', 'Readcode': 'A170.00'}"
-48580,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Nodular lung tuberculosis', 'Readcode': 'A111.00'}"
-4907,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis limb bones - Tuberculous dactylitis', 'Readcode': 'A153.00'}"
-49112,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous cervicitis', 'Readcode': 'A167000'}"
-49235,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pyelonephritis', 'Readcode': 'A160200'}"
-49433,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of retroperitoneal lymph nodes', 'Readcode': 'A14y500'}"
-49481,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis seminal vesicle', 'Readcode': 'A165100'}"
-49503,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of tracheobronchial lymph nodes', 'Readcode': 'A121200'}"
-50147,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other specified respiratory tuberculosis NOS', 'Readcode': 'A12yz00'}"
-50261,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Genitourinary tuberculosis NOS', 'Readcode': 'A16z.00'}"
-50489,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of urinary tract', 'Readcode': 'A168.00'}"
-50837,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pyelitis', 'Readcode': 'A160100'}"
-50869,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis myocardium', 'Readcode': 'A17y100'}"
-50902,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other specified respiratory tuberculosis', 'Readcode': 'A12y.00'}"
-5101,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous adenitis', 'Readcode': 'A172200'}"
-51110,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Scrofulous tuberculous abscess', 'Readcode': 'A172100'}"
-5145,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of hilar lymph nodes', 'Readcode': 'A121000'}"
-52199,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculoma of brain', 'Readcode': 'A132.00'}"
-52272,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous nephropathy', 'Readcode': 'A160000'}"
-5319,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'H/O: tuberculosis', 'Readcode': '1411.00'}"
-53331,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Miliary tuberculosis NOS', 'Readcode': 'A18z.00'}"
-53473,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Isolated tracheal or bronchial tuberculosis NOS', 'Readcode': 'A122z00'}"
-53548,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of ureter', 'Readcode': 'A162.00'}"
-53701,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Infiltrative lung tuberculosis', 'Readcode': 'A110.00'}"
-53864,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the bones of the shoulder region', 'Readcode': 'N306100'}"
-54570,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other gastrointestinal tract tuberculosis', 'Readcode': 'A14y.00'}"
-54579,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of gastrointestinal tract NOS', 'Readcode': 'A14z.00'}"
-54840,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of cerebral meninges', 'Readcode': 'A130000'}"
-54908,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous intracranial abscess', 'Readcode': 'F040600'}"
-55298,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""[X]Resp TB unspcf,w'out mention/bacterial or histol confrmtn"", 'Readcode': 'Ayu1300'}"
-55629,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous interstitial keratitis', 'Readcode': 'A173200'}"
-55835,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other organs', 'Readcode': 'A17..00'}"
-55957,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Congenital tuberculosis', 'Readcode': 'Q402400'}"
-56670,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis oesophagus', 'Readcode': 'A178.00'}"
-56771,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Cystitis in tuberculosis', 'Readcode': 'K154800'}"
-56833,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Erythema nodosum with tuberculosis NOS', 'Readcode': 'A171z00'}"
-56890,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pleurisy NOS', 'Readcode': 'A120z00'}"
-57126,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Acute pericarditis - tuberculous', 'Readcode': 'G500300'}"
-57398,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous abscess of brain', 'Readcode': 'A133.00'}"
-57417,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculoma of meninges', 'Readcode': 'A131.00'}"
-57466,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous oophoritis or salpingitis', 'Readcode': 'A166.00'}"
-57587,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the lower leg bone', 'Readcode': 'N305400'}"
-58588,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of lung, confirmed by unspecified means', 'Readcode': 'A124300'}"
-58673,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of ear', 'Readcode': 'A174.00'}"
-58827,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of intrathoracic lymph nodes', 'Readcode': 'A121.00'}"
-59087,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of rectum', 'Readcode': 'A14y400'}"
-59202,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous encephalitis or myelitis', 'Readcode': 'A136.00'}"
-59916,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the pelvic and thigh bones', 'Readcode': 'N305300'}"
-60040,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of prostate', 'Readcode': 'A165000'}"
-61442,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of tuberculosis NOS', 'Readcode': 'AE0z.00'}"
-62033,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis papulonecrotica', 'Readcode': 'A170700'}"
-62468,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of bronchus', 'Readcode': 'A113.00'}"
-62530,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of lung, confirmed histologically', 'Readcode': 'A124200'}"
-62963,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the upper arm bone', 'Readcode': 'N305100'}"
-63172,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Pneumoconiosis associated with tuberculosis', 'Readcode': 'H450.00'}"
-63351,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis cutis', 'Readcode': 'A170500'}"
-635,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Pulmonary tuberculosis', 'Readcode': 'A11..00'}"
-63796,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous oophoritis or salpingitis NOS', 'Readcode': 'A166z00'}"
-63959,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Other respiratory tuberculosis', 'Readcode': 'A12..00'}"
-65464,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of respiratory tuberculosis', 'Readcode': 'AE00.00'}"
-6553,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of spine', 'Readcode': 'N304.11'}"
-65994,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of lumbar spine', 'Readcode': 'N304200'}"
-66441,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pneumothorax', 'Readcode': 'A117.00'}"
-66584,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other female genital organs', 'Readcode': 'A167.00'}"
-66976,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis spleen', 'Readcode': 'A177.00'}"
-67292,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of kidney NOS', 'Readcode': 'A160z00'}"
-67337,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of thoracic spine', 'Readcode': 'N304100'}"
-67601,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the bones of the ankle and foot', 'Readcode': 'N306300'}"
-67637,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis - lupus exedens', 'Readcode': 'A170000'}"
-68154,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the forearm bone', 'Readcode': 'N305200'}"
-68821,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other male genital organs', 'Readcode': 'A165.00'}"
-68973,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other female genital organs NOS', 'Readcode': 'A167z00'}"
-69154,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other male genital organs NOS', 'Readcode': 'A165z00'}"
-69260,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Isolated tracheal or bronchial tuberculosis', 'Readcode': 'A122.00'}"
-69322,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous meningitis NOS', 'Readcode': 'A130z00'}"
-69471,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': ""Resp TB unspcf,w'out mention/bacterial or histol confrmtn"", 'Readcode': 'A125X00'}"
-70140,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of central nervous system NOS', 'Readcode': 'A13z.00'}"
-70293,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of cervical spine', 'Readcode': 'N304000'}"
-70491,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of eye NOS', 'Readcode': 'A173z00'}"
-70862,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of bone NOS', 'Readcode': 'N306z00'}"
-70911,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of tuberculosis of other specified organs', 'Readcode': 'AE04.00'}"
-71138,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the bones of other sites', 'Readcode': 'N306400'}"
-7133,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Respiratory TB not confirmed bact or histologically', 'Readcode': 'A125.00'}"
-72008,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Acute miliary tuberculosis', 'Readcode': 'A180.00'}"
-72072,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous encephalitis or myelitis NOS', 'Readcode': 'A136z00'}"
-72402,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of nasopharynx', 'Readcode': 'A12y100'}"
-72680,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Tuberculosis of other specified organs', 'Readcode': 'Ayu1600'}"
-72743,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous oophoritis', 'Readcode': 'A166000'}"
-73149,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Tuberculosis', 'Readcode': 'Ayu1.00'}"
-73185,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Other resp tubercul,confirmd bacteriologicly+histologicly', 'Readcode': 'Ayu1000'}"
-73225,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Resp tuberculos unspcfd,confirmd bacteriolog+histologicly', 'Readcode': 'Ayu1100'}"
-73273,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of central nervous system tuberculosis', 'Readcode': 'AE01.00'}"
-73549,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Late effects of tuberculosis of bones and joints', 'Readcode': 'AE03.00'}"
-73590,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of thyroid gland', 'Readcode': 'A175.00'}"
-9158,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[V] Personal history of tuberculosis', 'Readcode': 'ZV12B00'}"
-91666,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Streptomycin resistant tuberculosis', 'Readcode': '65Y2.00'}"
-91847,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Acute myocarditis - tuberculous', 'Readcode': 'G520600'}"
-93015,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Isolated tracheal tuberculosis', 'Readcode': 'A122000'}"
-93071,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of lung, confirmed by culture only', 'Readcode': 'A124100'}"
-93360,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous encephalitis', 'Readcode': 'A136000'}"
-93768,History of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Sequelae of respiratory and unspecified tuberculosis', 'Readcode': 'AyuJ400'}"
-93948,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Isolated bronchial tuberculosis', 'Readcode': 'A122100'}"
-94249,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Keratitis due to tuberculosis', 'Readcode': 'F4A5500'}"
-95332,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of other limb bones', 'Readcode': 'N305500'}"
-95913,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Latent tuberculosis', 'Readcode': '65Y9.11'}"
-96477,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Inactive tuberculosis', 'Readcode': '65Y9.00'}"
-96668,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis endocardium', 'Readcode': 'A17y000'}"
-97325,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of sacrum/coccyx', 'Readcode': 'N304300'}"
-97525,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Ciprofloxacin resistant tuberculosis', 'Readcode': '65Y8.00'}"
-97588,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous arthritis', 'Readcode': 'N018.00'}"
-97658,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of nasal septum', 'Readcode': 'A12y200'}"
-97778,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous encephalitis', 'Readcode': 'F033311'}"
-97922,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': '[X]Miliary tuberculosis, unspecified', 'Readcode': 'Ayu1900'}"
-99305,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of bone, site unspecified', 'Readcode': 'N306000'}"
-9953,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculous pneumonia', 'Readcode': 'A116.00'}"
-99593,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the bones of multiple sites', 'Readcode': 'N306500'}"
-99783,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of the bones of the hand', 'Readcode': 'N306200'}"
-99914,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Tuberculosis of unspecified limb bone', 'Readcode': 'N305000'}"
-99925,Diagnosis of tuberculosis,Med codes,C925,2989,Tuberculosis - Primary care,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'ReadcodeDescr': 'Encephalitis due to tuberculosis', 'Readcode': 'F033300'}"
-A15,"Respiratory tuberculosis, bacteriologically and histologically confirmed",ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-A16,"Respiratory tuberculosis, not confirmed bacteriologically or histologically",ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-A17,Tuberculosis of nervous system,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-A18,Tuberculosis of other organs,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-A19,Miliary tuberculosis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-B20.0,HIV disease resulting in mycobacterial infection,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-B90,Sequelae of tuberculosis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-J65,Pneumoconiosis associated with tuberculosis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-K23.0,Tuberculous oesophagitis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-K67.3,Tuberculous peritonitis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-K93.0,"Tuberculous disorders of intestines, peritoneum and mesenteric glands",ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-M01.1,Tuberculous arthritis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-M49.0,Tuberculosis of spine,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-M90.0,Tuberculosis of bone,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-N33.0,Tuberculous cystitis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-N74.0,Tuberculous infection of cervix uteri,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-N74.1,Female tuberculous pelvic inflammatory disease,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
-P37.0,Congenital tuberculosis,ICD10 codes,C926,2991,Tuberculosis - Secondary care - Diagnoses,PH87,174,Tuberculosis,"{'Disease': 'Tuberculosis', 'Category': 'Diagnosis of Tuberculosis'}"
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Urinary tract stones/Where this code list comes from .docx b/examples/codes/clinical-codes-org/Urinary tract stones/Where this code list comes from .docx
deleted file mode 100644
index 7b0cd3b877b74b16f71e2e8843ab6ec1cd3961e5..0000000000000000000000000000000000000000
Binary files a/examples/codes/clinical-codes-org/Urinary tract stones/Where this code list comes from .docx and /dev/null differ
diff --git a/examples/codes/clinical-codes-org/Urinary tract stones/phenotype_PH331_ver_662_concepts_20230719T121545.csv b/examples/codes/clinical-codes-org/Urinary tract stones/phenotype_PH331_ver_662_concepts_20230719T121545.csv
deleted file mode 100644
index 70442539af202b265f44887ca2da6135f5e7c5f7..0000000000000000000000000000000000000000
--- a/examples/codes/clinical-codes-org/Urinary tract stones/phenotype_PH331_ver_662_concepts_20230719T121545.csv
+++ /dev/null
@@ -1,270 +0,0 @@
-code,description,coding_system,concept_id,concept_version_id,concept_name,phenotype_id,phenotype_version_id,phenotype_name,code_attributes
-14D3.00,H/O: urinary stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '24994.0', 'Category': 'History of Urolithiasis'}"
-1A52.00,Renal colic,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '640.0', 'Category': 'Diagnosis of Urolithiasis'}"
-1A52.11,"Renal colic, symptom",Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '6770.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G4..00,O/E: renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '8399.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G4..11,O/E: kidney stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '6979.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G41.00,O/E: oxalate renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '34584.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G42.00,O/E: phosphate -staghorn-stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '24165.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G42.11,Phosphate kidney stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '32858.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G43.00,O/E: uric acid renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '27786.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G44.00,O/E: cystine renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '49783.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G4Z.00,O/E: renal stone NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '38461.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G6..00,O/E - ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '14276.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G7..00,O/E - urethral calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '45245.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G8..00,O/E - bladder calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '6978.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G81.00,O/E - vesical oxalate calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '74021.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G81.11,Oxalate bladder stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '31773.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G82.00,O/E-vesical uric acid calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '93608.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G82.11,Uric acid bladder stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '29242.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G83.00,O/E-vesical phosphate calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '102293.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G83.11,Phosphate bladder stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '65920.0', 'Category': 'Diagnosis of Urolithiasis'}"
-4G8Z.00,O/E - bladder calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '64699.0', 'Category': 'Diagnosis of Urolithiasis'}"
-7B05000,Unspecified open removal of calculus from kidney,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '4139.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07.00,Percutaneous renal stone surgery,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '8190.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07.11,Nephroscopic percutaneous lithotripsy of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '56462.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07.12,Percutaneous lithotripsy of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '2315.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07000,Nephroscopy and ultrasound lithotripsy of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '41619.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07011,Endoscopic ultrasound fragmentation of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '36157.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07100,Nephroscopy & electrohydraulic lithotripsy of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '60234.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07200,Nephroscopy and laser lithotripsy of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '43350.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07211,Endoscopic laser fragmentation of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '8677.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07400,Endoscopic extraction of calculus of kidney nec,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '34139.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07y00,Other specified percutaneous renal stone surgery,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '66743.0', 'Category': 'Procedure for Urolithiasis'}"
-7B07z00,Percutaneous renal stone surgery NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '7119.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0B.00,Extracorporeal shockwave lithotripsy for renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '7682.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0B.11,Extracorporeal fragmentation of renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '21401.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0B000,ESWL for renal calculus of unspecified size,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '17685.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0B100,ESWL for renal calculus less than 2 cm in diameter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '52721.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0B200,ESWL for renal calculus 2 cm or more in diameter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '94219.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0By00,Extracorporeal shockwave lithotripsy for renal calculus OS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '66113.0', 'Category': 'Procedure for Urolithiasis'}"
-7B0Bz00,Extracorporeal shockwave lithotripsy for renal calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '28514.0', 'Category': 'Procedure for Urolithiasis'}"
-7B17000,Nephroscopic laser lithotripsy of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '39511.0', 'Category': 'Procedure for Urolithiasis'}"
-7B17011,Nephroscopic laser fragmentation of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '58004.0', 'Category': 'Procedure for Urolithiasis'}"
-7B17100,Other nephroscopic fragmentation of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '23897.0', 'Category': 'Procedure for Urolithiasis'}"
-7B17111,Other nephroscopic lithotripsy of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '20813.0', 'Category': 'Procedure for Urolithiasis'}"
-7B17200,Nephroscopic extraction of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '45673.0', 'Category': 'Procedure for Urolithiasis'}"
-7B18.00,Ureteroscopic operations for ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '5366.0', 'Category': 'Procedure for Urolithiasis'}"
-7B18000,Ureteroscopic laser lithotripsy of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '36792.0', 'Category': 'Procedure for Urolithiasis'}"
-7B18011,Ureteroscopic laser fragmentation of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '28790.0', 'Category': 'Procedure for Urolithiasis'}"
-7B18100,Other ureteroscopic fragmentation of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '16025.0', 'Category': 'Procedure for Urolithiasis'}"
-7B18200,Ureteroscopic extraction of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '38804.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19.00,Cystoscopic removal of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '4216.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19000,Cystoscopic laser lithotripsy of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '18190.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19100,Other cystoscopic fragmentation of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '41871.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19200,Cystoscopic extraction of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '37073.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19211,Basket extraction of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '10587.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19212,Dormia basket extraction of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '24933.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19300,Cystoscopic catheter drainage for ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '35743.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19400,Cystoscopic dilation of ureter for drainage of calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '39048.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19y00,Other specified cystoscopic removal of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '72447.0', 'Category': 'Procedure for Urolithiasis'}"
-7B19z00,Cystoscopic removal of ureteric calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '58149.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1C.00,Extracorporeal shockwave lithotripsy of ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '3449.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1C000,Extracorp shockwave lithotripsy of unspec ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '51305.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1C100,Extracorporeal shockwave therapy for stone in upper ureter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '61904.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1C200,Extracorporeal shockwave lithotripsy for stone in mid-ureter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '101583.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1C300,Extracorporeal shockwave lithotripsy stone in lower ureter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '71131.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1Cy00,Extracorporeal shockwave lithotripsy of ureteric calculus OS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '29477.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1Cz00,Extracorporeal shockwave lithotripsy ureteric calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '29464.0', 'Category': 'Procedure for Urolithiasis'}"
-7B1F000,Endoscopic extraction of calculus of urinary diversion,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '102036.0', 'Category': 'Procedure for Urolithiasis'}"
-7B25000,Open removal of bladder calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '23381.0', 'Category': 'Procedure for Urolithiasis'}"
-7B29100,Other endoscopic extraction of bladder calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '28595.0', 'Category': 'Procedure for Urolithiasis'}"
-7B29400,Electrokinetic lithotripsy of bladder calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '9323.0', 'Category': 'Procedure for Urolithiasis'}"
-7B2B400,Removal of bladder calculus by urethral catheter suction,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '40272.0', 'Category': 'Procedure for Urolithiasis'}"
-7B42300,Open urethrotomy and removal of calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '34097.0', 'Category': 'Procedure for Urolithiasis'}"
-7B42400,Open extraction of calculus from urethra,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '90777.0', 'Category': 'Procedure for Urolithiasis'}"
-7B43900,Endoscopic removal of urethral calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '28953.0', 'Category': 'Procedure for Urolithiasis'}"
-C341100,Uric acid nephrolithiasis,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '43744.0', 'Category': 'Diagnosis of Urolithiasis'}"
-C341111,Renal stone - uric acid,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '9162.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K100600,Calculous pyelonephritis,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '2939.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K112.00,Hydronephrosis with renal and ureteral calculous obstruction,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '27592.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K12..00,Calculus of kidney and ureter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '9950.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K12..11,Kidney calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '8777.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K12..12,Urinary calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '3669.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K120.00,Calculus of kidney,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '4928.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K120.11,Nephrolithiasis NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '24047.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K120.12,Renal calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '1858.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K120.13,Renal stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '2258.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K120000,Staghorn calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '10282.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K120z00,Renal calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '3906.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K121.00,Calculus of ureter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '3308.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K121.11,Ureteric calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '2410.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K121.12,Ureteric stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '6048.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K121.13,Ureterolithiasis,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '36972.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K122.00,Calculus of kidney with calculus of ureter,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '33746.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K12z.00,Urinary calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '1256.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K14..00,Lower urinary tract calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '47869.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K140.00,Bladder calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '5729.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K140.11,Bladder stone,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '1912.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K140000,Calculus in diverticulum of bladder,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '67386.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K140100,Other calculus in bladder,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '94420.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K140z00,Bladder calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '59834.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K141.00,Calculus in urethra,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '2105.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K14y.00,Other lower urinary tract calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '46291.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K14z.00,Lower urinary tract calculus NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '44648.0', 'Category': 'Diagnosis of Urolithiasis'}"
-K1A..00,Urinary calculus in schistosomiasis,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '52569.0', 'Category': 'Diagnosis of Urolithiasis'}"
-Kyu3.00,[X]Urolithiasis,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '45765.0', 'Category': 'Diagnosis of Urolithiasis'}"
-Kyu3000,[X]Other lower urinary tract calculus,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '106912.0', 'Category': 'Diagnosis of Urolithiasis'}"
-Kyu3100,[X]Calculus of urinary tract in other diseases CE,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '23917.0', 'Category': 'Diagnosis of Urolithiasis'}"
-R080.00,[D]Renal colic,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '6087.0', 'Category': 'Diagnosis of Urolithiasis'}"
-R080000,"[D]Renal colic, unspecified",Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '15201.0', 'Category': 'Diagnosis of Urolithiasis'}"
-R080100,[D]Ureteric colic,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '6147.0', 'Category': 'Diagnosis of Urolithiasis'}"
-R080z00,[D]Renal colic NOS,Read codes v2,C1637,4413,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Medcode': '33747.0', 'Category': 'Diagnosis of Urolithiasis'}"
-101583,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy for stone in mid-ureter', 'Readcode': '7B1C200'}"
-102036,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Endoscopic extraction of calculus of urinary diversion', 'Readcode': '7B1F000'}"
-102293,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E-vesical phosphate calculus', 'Readcode': '4G83.00'}"
-10282,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Staghorn calculus', 'Readcode': 'K120000'}"
-10587,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Basket extraction of ureteric calculus', 'Readcode': '7B19211'}"
-106912,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[X]Other lower urinary tract calculus', 'Readcode': 'Kyu3000'}"
-1256,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Urinary calculus NOS', 'Readcode': 'K12z.00'}"
-14276,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E - ureteric calculus', 'Readcode': '4G6..00'}"
-15201,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[D]Renal colic, unspecified', 'Readcode': 'R080000'}"
-16025,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other ureteroscopic fragmentation of ureteric calculus', 'Readcode': '7B18100'}"
-17685,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'ESWL for renal calculus of unspecified size', 'Readcode': '7B0B000'}"
-18190,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Cystoscopic laser lithotripsy of ureteric calculus', 'Readcode': '7B19000'}"
-1858,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Renal calculus', 'Readcode': 'K120.12'}"
-1912,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Bladder stone', 'Readcode': 'K140.11'}"
-20813,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other nephroscopic lithotripsy of ureteric calculus', 'Readcode': '7B17111'}"
-2105,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculus in urethra', 'Readcode': 'K141.00'}"
-21401,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal fragmentation of renal calculus', 'Readcode': '7B0B.11'}"
-2258,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Renal stone', 'Readcode': 'K120.13'}"
-2315,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Percutaneous lithotripsy of renal calculus', 'Readcode': '7B07.12'}"
-23381,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Open removal of bladder calculus', 'Readcode': '7B25000'}"
-23897,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other nephroscopic fragmentation of ureteric calculus', 'Readcode': '7B17100'}"
-23917,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[X]Calculus of urinary tract in other diseases CE', 'Readcode': 'Kyu3100'}"
-24047,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephrolithiasis NOS', 'Readcode': 'K120.11'}"
-2410,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureteric calculus', 'Readcode': 'K121.11'}"
-24165,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: phosphate -staghorn-stone', 'Readcode': '4G42.00'}"
-24933,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Dormia basket extraction of ureteric calculus', 'Readcode': '7B19212'}"
-24994,History of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'H/O: urinary stone', 'Readcode': '14D3.00'}"
-27592,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Hydronephrosis with renal and ureteral calculous obstruction', 'Readcode': 'K112.00'}"
-27786,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: uric acid renal calculus', 'Readcode': '4G43.00'}"
-28514,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy for renal calculus NOS', 'Readcode': '7B0Bz00'}"
-28595,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other endoscopic extraction of bladder calculus', 'Readcode': '7B29100'}"
-28790,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureteroscopic laser fragmentation of ureteric calculus', 'Readcode': '7B18011'}"
-28953,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Endoscopic removal of urethral calculus', 'Readcode': '7B43900'}"
-29242,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Uric acid bladder stone', 'Readcode': '4G82.11'}"
-2939,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculous pyelonephritis', 'Readcode': 'K100600'}"
-29464,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy ureteric calculus NOS', 'Readcode': '7B1Cz00'}"
-29477,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy of ureteric calculus OS', 'Readcode': '7B1Cy00'}"
-31773,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Oxalate bladder stone', 'Readcode': '4G81.11'}"
-32858,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Phosphate kidney stone', 'Readcode': '4G42.11'}"
-3308,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculus of ureter', 'Readcode': 'K121.00'}"
-33746,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculus of kidney with calculus of ureter', 'Readcode': 'K122.00'}"
-33747,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[D]Renal colic NOS', 'Readcode': 'R080z00'}"
-34097,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Open urethrotomy and removal of calculus', 'Readcode': '7B42300'}"
-34139,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Endoscopic extraction of calculus of kidney nec', 'Readcode': '7B07400'}"
-3449,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy of ureteric calculus', 'Readcode': '7B1C.00'}"
-34584,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: oxalate renal calculus', 'Readcode': '4G41.00'}"
-35743,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Cystoscopic catheter drainage for ureteric calculus', 'Readcode': '7B19300'}"
-36157,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Endoscopic ultrasound fragmentation of renal calculus', 'Readcode': '7B07011'}"
-3669,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Urinary calculus', 'Readcode': 'K12..12'}"
-36792,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureteroscopic laser lithotripsy of ureteric calculus', 'Readcode': '7B18000'}"
-36972,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureterolithiasis', 'Readcode': 'K121.13'}"
-37073,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Cystoscopic extraction of ureteric calculus', 'Readcode': '7B19200'}"
-38461,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: renal stone NOS', 'Readcode': '4G4Z.00'}"
-38804,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureteroscopic extraction of ureteric calculus', 'Readcode': '7B18200'}"
-39048,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Cystoscopic dilation of ureter for drainage of calculus', 'Readcode': '7B19400'}"
-3906,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Renal calculus NOS', 'Readcode': 'K120z00'}"
-39511,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopic laser lithotripsy of ureteric calculus', 'Readcode': '7B17000'}"
-40272,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Removal of bladder calculus by urethral catheter suction', 'Readcode': '7B2B400'}"
-4139,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Unspecified open removal of calculus from kidney', 'Readcode': '7B05000'}"
-41619,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopy and ultrasound lithotripsy of renal calculus', 'Readcode': '7B07000'}"
-41871,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other cystoscopic fragmentation of ureteric calculus', 'Readcode': '7B19100'}"
-4216,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Cystoscopic removal of ureteric calculus', 'Readcode': '7B19.00'}"
-43350,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopy and laser lithotripsy of renal calculus', 'Readcode': '7B07200'}"
-43744,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Uric acid nephrolithiasis', 'Readcode': 'C341100'}"
-44648,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Lower urinary tract calculus NOS', 'Readcode': 'K14z.00'}"
-45245,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E - urethral calculus', 'Readcode': '4G7..00'}"
-45673,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopic extraction of ureteric calculus', 'Readcode': '7B17200'}"
-45765,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[X]Urolithiasis', 'Readcode': 'Kyu3.00'}"
-46291,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other lower urinary tract calculus', 'Readcode': 'K14y.00'}"
-47869,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Lower urinary tract calculus', 'Readcode': 'K14..00'}"
-4928,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculus of kidney', 'Readcode': 'K120.00'}"
-49783,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: cystine renal calculus', 'Readcode': '4G44.00'}"
-51305,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorp shockwave lithotripsy of unspec ureteric calculus', 'Readcode': '7B1C000'}"
-52569,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Urinary calculus in schistosomiasis', 'Readcode': 'K1A..00'}"
-52721,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'ESWL for renal calculus less than 2 cm in diameter', 'Readcode': '7B0B100'}"
-5366,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureteroscopic operations for ureteric calculus', 'Readcode': '7B18.00'}"
-56462,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopic percutaneous lithotripsy of renal calculus', 'Readcode': '7B07.11'}"
-5729,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Bladder calculus', 'Readcode': 'K140.00'}"
-58004,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopic laser fragmentation of ureteric calculus', 'Readcode': '7B17011'}"
-58149,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Cystoscopic removal of ureteric calculus NOS', 'Readcode': '7B19z00'}"
-59834,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Bladder calculus NOS', 'Readcode': 'K140z00'}"
-60234,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Nephroscopy & electrohydraulic lithotripsy of renal calculus', 'Readcode': '7B07100'}"
-6048,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Ureteric stone', 'Readcode': 'K121.12'}"
-6087,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[D]Renal colic', 'Readcode': 'R080.00'}"
-6147,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': '[D]Ureteric colic', 'Readcode': 'R080100'}"
-61904,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave therapy for stone in upper ureter', 'Readcode': '7B1C100'}"
-640,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Renal colic', 'Readcode': '1A52.00'}"
-64699,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E - bladder calculus NOS', 'Readcode': '4G8Z.00'}"
-65920,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Phosphate bladder stone', 'Readcode': '4G83.11'}"
-66113,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy for renal calculus OS', 'Readcode': '7B0By00'}"
-66743,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other specified percutaneous renal stone surgery', 'Readcode': '7B07y00'}"
-67386,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculus in diverticulum of bladder', 'Readcode': 'K140000'}"
-6770,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Renal colic, symptom', 'Readcode': '1A52.11'}"
-6978,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E - bladder calculus', 'Readcode': '4G8..00'}"
-6979,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: kidney stone', 'Readcode': '4G4..11'}"
-71131,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy stone in lower ureter', 'Readcode': '7B1C300'}"
-7119,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Percutaneous renal stone surgery NOS', 'Readcode': '7B07z00'}"
-72447,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other specified cystoscopic removal of ureteric calculus', 'Readcode': '7B19y00'}"
-74021,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E - vesical oxalate calculus', 'Readcode': '4G81.00'}"
-7682,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Extracorporeal shockwave lithotripsy for renal calculus', 'Readcode': '7B0B.00'}"
-8190,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Percutaneous renal stone surgery', 'Readcode': '7B07.00'}"
-8399,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E: renal calculus', 'Readcode': '4G4..00'}"
-8677,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Endoscopic laser fragmentation of renal calculus', 'Readcode': '7B07211'}"
-8777,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Kidney calculus', 'Readcode': 'K12..11'}"
-90777,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Open extraction of calculus from urethra', 'Readcode': '7B42400'}"
-9162,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Renal stone - uric acid', 'Readcode': 'C341111'}"
-9323,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Electrokinetic lithotripsy of bladder calculus', 'Readcode': '7B29400'}"
-93608,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'O/E-vesical uric acid calculus', 'Readcode': '4G82.00'}"
-94219,Procedure for Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'ESWL for renal calculus 2 cm or more in diameter', 'Readcode': '7B0B200'}"
-94420,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Other calculus in bladder', 'Readcode': 'K140100'}"
-9950,Diagnosis of Urolithiasis,Med codes,C1638,4415,Urolithiasis - Primary care,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'ReadcodeDescr': 'Calculus of kidney and ureter', 'Readcode': 'K12..00'}"
-N13.2,Hydronephrosis with renal and ureteral calculous obstruction,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N20.0,Calculus of kidney,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N20.1,Calculus of ureter,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N20.2,Calculus of kidney with calculus of ureter,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N20.9,"Urinary calculus, unspecified",ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N21.0,Calculus in bladder,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N21.1,Calculus in urethra,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N21.8,Other lower urinary tract calculus,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N21.9,"Calculus of lower urinary tract, unspecified",ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N22.8,Calculus of urinary tract in other diseases classified elsewhere,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-N23.X,Unspecified renal colic,ICD10 codes,C1639,4417,Urolithiasis - Secondary care - Diagnoses,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Diagnosis of Urolithiasis'}"
-M06.1,Open removal of calculus from kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09,Therapeutic endoscopic operations on calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09.1,Endoscopic ultrasound fragmentation of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09.2,Endoscopic electrohydraulic shockwave fragmentation of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09.3,Endoscopic laser fragmentation of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09.4,Endoscopic extraction of calculus of kidney NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09.8,Other specified therapeutic endoscopic operations on calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M09.9,Unspecified therapeutic endoscopic operations on calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M14,Extracorporeal fragmentation of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M14.1,Extracorporeal shock wave lithotripsy of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M14.8,Other specified extracorporeal fragmentation of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M14.9,Unspecified extracorporeal fragmentation of calculus of kidney,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M16.4,Percutaneous nephrolithotomy NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M23.1,Open ureterolithotomy,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M26.1,Nephroscopic laser fragmentation of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M26.2,Nephroscopic fragmentation of calculus of ureter NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M26.3,Nephroscopic extraction of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M27.1,Ureteroscopic laser fragmentation of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M27.2,Ureteroscopic fragmentation of calculus of ureter NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M27.3,Ureteroscopic extraction of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28,Other endoscopic removal of calculus from ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.1,Endoscopic laser fragmentation of calculus of ureter NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.2,Endoscopic fragmentation of calculus of ureter NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.3,Endoscopic extraction of calculus of ureter NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.4,Endoscopic catheter drainage of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.5,Endoscopic drainage of calculus of ureter by dilation of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.8,Other specified other endoscopic removal of calculus from ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M28.9,Unspecified other endoscopic removal of calculus from ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M31,Extracorporeal fragmentation of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M31.1,Extracorporeal shockwave lithotripsy of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M31.8,Other specified extracorporeal fragmentation of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M31.9,Unspecified extracorporeal fragmentation of calculus of ureter,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M39.1,Open removal of calculus from bladder,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M44.2,Endoscopic extraction of calculus of bladder NEC,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M75.4,Open extraction of calculus from urethra,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
-M86.1,Endoscopic extraction of calculus of urinary diversion,OPCS4 codes,C1640,4419,Urolithiasis - Secondary care - Procedures,PH331,662,Urolithiasis,"{'Disease': 'Urolithiasis', 'Category': 'Procedure for Urolithiasis'}"
\ No newline at end of file
diff --git a/examples/codes/clinical-codes-org/Cardiovascular events (ICD10)/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Cardiovascular events (ICD10)/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Cardiovascular events (ICD10)/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Cardiovascular events (ICD10)/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Cardiovascular events (ICD10)/res52-cardiovascular-events-icd10.csv b/examples/concepts/clinical-codes-org/Cardiovascular events (ICD10)/res52-cardiovascular-events-icd10.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Cardiovascular events (ICD10)/res52-cardiovascular-events-icd10.csv
rename to examples/concepts/clinical-codes-org/Cardiovascular events (ICD10)/res52-cardiovascular-events-icd10.csv
diff --git a/examples/codes/clinical-codes-org/Non-attendance codes/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Non-attendance codes/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Non-attendance codes/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Non-attendance codes/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Non-attendance codes/res201-did-not-attend-appointment.csv b/examples/concepts/clinical-codes-org/Non-attendance codes/res201-did-not-attend-appointment.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Non-attendance codes/res201-did-not-attend-appointment.csv
rename to examples/concepts/clinical-codes-org/Non-attendance codes/res201-did-not-attend-appointment.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Abdominal pain/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Abdominal pain/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Abdominal pain/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Abdominal pain/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Abdominal pain/res176-abdominal-pain.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Abdominal pain/res176-abdominal-pain.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Abdominal pain/res176-abdominal-pain.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Abdominal pain/res176-abdominal-pain.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Falls/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Falls/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Falls/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Falls/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Falls/res178-fall.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Falls/res178-fall.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Falls/res178-fall.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Falls/res178-fall.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Falls/res202-falls-outcome.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Falls/res202-falls-outcome.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Falls/res202-falls-outcome.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Falls/res202-falls-outcome.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Fatigue/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Fatigue/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Fatigue/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Fatigue/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Fatigue/res175-fatigue.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Fatigue/res175-fatigue.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Fatigue/res175-fatigue.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Fatigue/res175-fatigue.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Headache/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Headache/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Headache/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Headache/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Headache/res175-headache.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Headache/res175-headache.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Headache/res175-headache.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Headache/res175-headache.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Incontinence/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Incontinence/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Incontinence/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Incontinence/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Incontinence/res11-incontinence.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Incontinence/res11-incontinence.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Incontinence/res11-incontinence.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Incontinence/res11-incontinence.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Musculoskeletal pain/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Musculoskeletal pain/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Musculoskeletal pain/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Musculoskeletal pain/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Musculoskeletal pain/res175-msk-pain.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Musculoskeletal pain/res175-msk-pain.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Musculoskeletal pain/res175-msk-pain.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Musculoskeletal pain/res175-msk-pain.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Neuropathic pain/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Neuropathic pain/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Neuropathic pain/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Neuropathic pain/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Neuropathic pain/res55-neuropathic_pain.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Neuropathic pain/res55-neuropathic_pain.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Neuropathic pain/res55-neuropathic_pain.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Neuropathic pain/res55-neuropathic_pain.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Sleep problems/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Sleep problems/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Sleep problems/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Sleep problems/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Sleep problems/res175-sleep-problems.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Sleep problems/res175-sleep-problems.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Sleep problems/res175-sleep-problems.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Sleep problems/res175-sleep-problems.csv
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Stress/Where this code list comes from .docx b/examples/concepts/clinical-codes-org/Symptom code lists/Stress/Where this code list comes from .docx
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Stress/Where this code list comes from .docx
rename to examples/concepts/clinical-codes-org/Symptom code lists/Stress/Where this code list comes from .docx
diff --git a/examples/codes/clinical-codes-org/Symptom code lists/Stress/res175-stress.csv b/examples/concepts/clinical-codes-org/Symptom code lists/Stress/res175-stress.csv
similarity index 100%
rename from examples/codes/clinical-codes-org/Symptom code lists/Stress/res175-stress.csv
rename to examples/concepts/clinical-codes-org/Symptom code lists/Stress/res175-stress.csv
diff --git a/examples/codes/hanlon/Hanlon_drug_read_codes.csv b/examples/concepts/hanlon/Hanlon_drug_read_codes.csv
similarity index 100%
rename from examples/codes/hanlon/Hanlon_drug_read_codes.csv
rename to examples/concepts/hanlon/Hanlon_drug_read_codes.csv
diff --git a/examples/codes/hanlon/MM_code_names.csv b/examples/concepts/hanlon/MM_code_names.csv
similarity index 100%
rename from examples/codes/hanlon/MM_code_names.csv
rename to examples/concepts/hanlon/MM_code_names.csv
diff --git a/examples/codes/hanlon/Read_codes_for_diagnoses.csv b/examples/concepts/hanlon/Read_codes_for_diagnoses.csv
similarity index 100%
rename from examples/codes/hanlon/Read_codes_for_diagnoses.csv
rename to examples/concepts/hanlon/Read_codes_for_diagnoses.csv
diff --git a/examples/codes/hanlon/Read_codes_for_drug_mm_count_diagnoses.csv b/examples/concepts/hanlon/Read_codes_for_drug_mm_count_diagnoses.csv
similarity index 100%
rename from examples/codes/hanlon/Read_codes_for_drug_mm_count_diagnoses.csv
rename to examples/concepts/hanlon/Read_codes_for_drug_mm_count_diagnoses.csv
diff --git a/examples/config1.yaml b/examples/config1.yml
similarity index 70%
rename from examples/config1.yaml
rename to examples/config1.yml
index 2f446aae11cb8c840c64422d15dc36a03ea1ec87..2709d3f97795ee3e8bfea21ff827f0ff26473f6d 100644
--- a/examples/config1.yaml
+++ b/examples/config1.yml
@@ -1,9 +1,12 @@
phenotype:
- version: "v1.0.1"
+ version: "0.0.0"
omop:
- vocabulary_id: "ACMC_Example"
- vocabulary_name: "ACMC example phenotype"
+ vocabulary_id: "ACMC_Example_1"
+ vocabulary_name: "ACMC example 1 phenotype"
vocabulary_reference: "https://git.soton.ac.uk/meldb/concepts-processing/-/tree/main/examples"
+ map:
+ - "read2"
+ - "read3"
concept_sets:
- name: "ABDO_PAIN"
file:
diff --git a/examples/config2.yaml b/examples/config2.yml
similarity index 79%
rename from examples/config2.yaml
rename to examples/config2.yml
index 24acf969f13b58a777978b6fba258dc6e71471e6..4a9ad79b598641a869601329b00a0b016cf1e082 100644
--- a/examples/config2.yaml
+++ b/examples/config2.yml
@@ -1,9 +1,12 @@
phenotype:
- version: "v1.0.4"
+ version: "0.0.0"
omop:
- vocabulary_id: "ACMC_Example"
- vocabulary_name: "ACMC example phenotype"
+ vocabulary_id: "ACMC_Example_2"
+ vocabulary_name: "ACMC example 2 phenotype"
vocabulary_reference: "https://www.it-innovation.soton.ac.uk/projects/meldb/concept-processing/example"
+ map:
+ - "read2"
+ - "read3"
concept_sets:
- name: "CVD_EVENTS"
file:
diff --git a/examples/config3.yaml b/examples/config3.yml
similarity index 86%
rename from examples/config3.yaml
rename to examples/config3.yml
index 411606a1b0390dad52c5e14e1a23ed9490d06c0d..2e07427219e3dc605008007e4c0b9ad66fa8159c 100644
--- a/examples/config3.yaml
+++ b/examples/config3.yml
@@ -1,9 +1,13 @@
phenotype:
- version: "v1.0.4"
+ version: "0.0.0"
omop:
- vocabulary_id: "ACMC_Example"
- vocabulary_name: "ACMC example phenotype"
+ vocabulary_id: "ACMC_Example_3"
+ vocabulary_name: "ACMC example 3 phenotype"
vocabulary_reference: "https://www.it-innovation.soton.ac.uk/projects/meldb/concept-processing/example"
+ map:
+ - "read2"
+ - "read3"
+ - "snomed"
concept_sets:
- name: "CVD_EVENTS"
file:
diff --git a/mkdocs.yml b/mkdocs.yml
index d3b89a064436f44c90ebdb4489f065b39dfa0210..a7689ae1ca0bad0cf23c56f57d0204eab3905e3b 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -1,21 +1,22 @@
site_name: ACMC Documentation
+site_url: https://acmc.readthedocs.io
+
theme:
- name: material
- features:
- - navigation.tabs
- - navigation.expand
- - content.code.copy
+ name: 'readthedocs'
+
nav:
- Home: index.md
- - Installation: installation.md
- - Usage: usage.md
- - Contributing: contributing.md
- - API Reference: api.md
- - Tutorials:
- - Example 1 - Basic local phenotype: ./tutorials/example1.md
- - Example 2 - More complex local phenotype: ./tutorials/example2.md
- - Example 3 - Using a remote git repository: ./tutorials/example3.md
- - Contributing: contributing.md
+ - Example 1 Basic local phenotype: examples/example1.md
+ - Example 2 More complex local phenotype: examples/example2.md
+ - Example 3 Using a remote git repository: examples/example3.md
+ - Comand Line Reference: cli.md
+ - API Reference: api/acmc.html
- Change Log: changelog.md
- - Troubleshooting: troubleshooting.md
-repo_url: https://git.soton.ac.uk/meldb/concepts-processing/
+
+markdown_extensions:
+ - toc:
+ permalink: true
+
+plugins:
+ - search
+
diff --git a/pyproject.toml b/pyproject.toml
index a6909823668464f612bc9eb88efa25c2c845b4dd..f98296456fff104a5f899cbfafb52d04f19ff4ef 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "acmc"
-version = "0.0.3"
+version = "0.1.0"
authors = [
{ name = "Jakub Dylag", email = "j.j.dylag@soton.ac.uk" },
{ name = "Michael Boniface", email = "m.j.boniface@soton.ac.uk" }
@@ -16,8 +16,10 @@ requires-python = ">=3.9"
dependencies = [
"aiosqlite",
+ "cerberus",
"click",
"cramjam",
+ "deepdiff",
"et-xmlfile",
"fastparquet",
"fsspec",
@@ -35,7 +37,8 @@ dependencies = [
"tables",
"pytest",
"pyyaml",
- "requests",
+ "requests",
+ "semver",
"simpledbf",
"smmap",
"sqlalchemy",
@@ -48,9 +51,10 @@ acmc = "acmc.main:main" # Entry point for CLI command
[project.urls]
Repository = "https://git.soton.ac.uk/meldb/concepts-processing"
-Documentation = "https://git.soton.ac.uk/meldb/concepts-processing/docs"
+Documentation = "https://acmc.readthedocs.io/en/latest/"
Issues = "https://git.soton.ac.uk/meldb/concepts-processing/-/issues"
+
[tool.hatch.build.targets.wheel]
packages = ["acmc"]
@@ -59,20 +63,28 @@ dependencies = [
"hatch",
"pytest",
]
+
+[tool.hatch.envs.docs]
+dependencies = [
+ "mkdocs",
+ "mkdocs-material"
+]
[tool.hatch.envs.dev]
dependencies = [
- "pydocstyle",
"pytest",
"black",
"mypy",
- "mkdocs",
- "mkdocs-material",
- "mkdocstrings"
+ "pdoc",
+ "pydoc-markdown",
+ "mkdocs"
]
[tool.hatch.envs.dev.scripts]
-check = "black . && mypy ."
+format = "black ."
+type = "mypy ."
+docs = "pdoc -o docs/api -d google --docformat google --include-undocumented acmc"
+precommit = "hatch run format && hatch run type && hatch run docs"
[tool.hatch.build]
include = ["acmc/**"] # Ensure only the acmc package is included
diff --git a/readthedocs.yml b/readthedocs.yml
new file mode 100644
index 0000000000000000000000000000000000000000..8b758a8db0a8ae8e7b563a7f4d15a73f5efbced3
--- /dev/null
+++ b/readthedocs.yml
@@ -0,0 +1,17 @@
+# Read the Docs configuration file for MkDocs projects
+
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+ os: ubuntu-24.04
+ tools:
+ python: "3.12"
+
+mkdocs:
+ configuration: mkdocs.yml
+
+
+
diff --git a/tests/test_acmc.py b/tests/test_acmc.py
index 0ad6f863f595dbe9c78acc368ff4a0bd7307648f..5c8ecd0b5820c21ba5eb544b7b49aedbd700e046 100644
--- a/tests/test_acmc.py
+++ b/tests/test_acmc.py
@@ -5,7 +5,7 @@ import shutil
import logging
from pathlib import Path
-from acmc import trud, omop, main, logging_config as lc
+from acmc import trud, omop, main, phen, logging_config as lc
# setup logging
lc.setup_logger()
@@ -49,9 +49,9 @@ def test_phen_init_local_specified(tmp_dir, monkeypatch, caplog):
@pytest.mark.parametrize(
"config_file",
[
- ("config1.yaml"), # config.yaml test case
- ("config2.yaml"), # config.yaml test case
- ("config3.yaml"), # config.yaml test case
+ ("config1.yml"), # config.yml test case
+ ("config2.yml"), # config.yml test case
+ ("config3.yml"), # config.yml test case
],
)
def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
@@ -71,7 +71,7 @@ def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
with caplog.at_level(logging.DEBUG):
# validate phenotype
# copy examples across
- shutil.rmtree(phen_path / "codes")
+ shutil.rmtree(phen_path / phen.CONCEPTS_DIR)
ex_path = Path("./examples").resolve()
for item in ex_path.iterdir():
source = ex_path / item.name
@@ -82,7 +82,7 @@ def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
shutil.copy(source, destination)
# copy the test file to configuration
- shutil.copy(phen_path / config_file, phen_path / "config.yaml")
+ shutil.copy(phen_path / config_file, phen_path / "config.yml")
monkeypatch.setattr(
sys, "argv", ["main.py", "phen", "validate", "-d", str(phen_path.resolve())]
@@ -91,7 +91,7 @@ def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
assert "Phenotype validated successfully" in caplog.text
# map phenotype
- for code_type in ["read2", "read3", "snomed"]:
+ for code_type in ["read2", "read3"]:
with caplog.at_level(logging.DEBUG):
monkeypatch.setattr(
sys,
@@ -109,6 +109,109 @@ def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
main.main()
assert "Phenotype processed successfully" in caplog.text
+ # publish phenotype with version message
+ with caplog.at_level(logging.DEBUG):
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "publish",
+ "-d",
+ str(phen_path.resolve()),
+ "-m",
+ "version message",
+ ],
+ )
+ main.main()
+ assert "Phenotype published successfully" in caplog.text
+
+ # copy phenotype'
+ with caplog.at_level(logging.DEBUG):
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "copy",
+ "-d",
+ str(phen_path.resolve()),
+ "-td",
+ str(tmp_dir.resolve()),
+ "-v",
+ "0.0.1",
+ ],
+ )
+ main.main()
+ assert "Phenotype copied successfully" in caplog.text
+
+ # diff phenotype
+ with caplog.at_level(logging.DEBUG):
+ old_path = tmp_dir / "0.0.1"
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "diff",
+ "-d",
+ str(phen_path.resolve()),
+ "-ov",
+ "0.0.1",
+ ],
+ )
+ main.main()
+ assert "Phenotypes diff'd successfully" in caplog.text
+
+
+# TODO: Refactor this test so it is shortened, there's lots of duplicate codes
+def test_diff(tmp_dir, monkeypatch, caplog):
+ print(f"Temporary directory: {tmp_dir}") # Prints path for debugging
+
+ # init phenotype
+ phen_path = tmp_dir / "phen"
+ phen_path = phen_path.resolve()
+ monkeypatch.setattr(
+ sys, "argv", ["main.py", "phen", "init", "-d", str(phen_path.resolve())]
+ )
+ # Mock input() to return "yes" to the question about reinitialising the directory
+ monkeypatch.setattr("builtins.input", lambda _: "y")
+ main.main()
+
+ # copy example codes
+ shutil.rmtree(phen_path / phen.CONCEPTS_DIR)
+ ex_path = Path("./examples").resolve()
+ for item in ex_path.iterdir():
+ source = ex_path / item.name
+ destination = phen_path / item.name
+ if source.is_dir():
+ shutil.copytree(source, destination)
+ else:
+ shutil.copy(source, destination)
+
+ # map phenotype for config 1
+ shutil.copy(phen_path / "config1.yml", phen_path / "config.yml")
+ for code_type in ["read2"]:
+ with caplog.at_level(logging.DEBUG):
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "map",
+ "-d",
+ str(phen_path.resolve()),
+ "-t",
+ code_type,
+ ],
+ )
+ main.main()
+ assert "Phenotype processed successfully" in caplog.text
+
# publish phenotype
with caplog.at_level(logging.DEBUG):
monkeypatch.setattr(
@@ -131,15 +234,35 @@ def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
"-td",
str(tmp_dir.resolve()),
"-v",
- "v1.0.3",
+ "0.0.1",
],
)
main.main()
assert "Phenotype copied successfully" in caplog.text
- # diff phenotype
+ # map phenotype example 2
+ shutil.copy(phen_path / "config2.yml", phen_path / "config.yml")
+ for code_type in ["read2", "read3"]:
+ with caplog.at_level(logging.DEBUG):
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "map",
+ "-d",
+ str(phen_path.resolve()),
+ "-t",
+ code_type,
+ ],
+ )
+ main.main()
+ assert "Phenotype processed successfully" in caplog.text
+
+ # diff phenotype with 0.0.1
with caplog.at_level(logging.DEBUG):
- old_path = tmp_dir / "v1.0.3"
+ old_path = tmp_dir / "0.0.1"
monkeypatch.setattr(
sys,
"argv",
@@ -149,9 +272,76 @@ def test_phen_workflow(tmp_dir, monkeypatch, caplog, config_file):
"diff",
"-d",
str(phen_path.resolve()),
- "-old",
- str(old_path.resolve()),
+ "-ov",
+ "0.0.1",
],
)
main.main()
assert "Phenotypes diff'd successfully" in caplog.text
+
+ # check changes
+ with open(phen_path / "latest_0.0.1_diff.md", "r") as file:
+ content = file.read()
+ assert "Removed concepts ['ABDO_PAIN']" in content
+ assert "Added concepts ['DID_NOT_ATTEND']" in content
+ assert "Added outputs: ['read3.csv']" in content
+
+ # map phenotype with example 3
+ shutil.copy(phen_path / "config3.yml", phen_path / "config.yml")
+ for code_type in ["read2", "read3", "snomed"]:
+ with caplog.at_level(logging.DEBUG):
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "map",
+ "-d",
+ str(phen_path.resolve()),
+ "-t",
+ code_type,
+ ],
+ )
+ main.main()
+ assert "Phenotype processed successfully" in caplog.text
+
+ # diff phenotype with 0.0.1
+ with caplog.at_level(logging.DEBUG):
+ old_path = tmp_dir / "0.0.1"
+ monkeypatch.setattr(
+ sys,
+ "argv",
+ [
+ "main.py",
+ "phen",
+ "diff",
+ "-d",
+ str(phen_path.resolve()),
+ "-ov",
+ "0.0.1",
+ ],
+ )
+ main.main()
+ assert "Phenotypes diff'd successfully" in caplog.text
+
+ with open(phen_path / "latest_0.0.1_diff.md", "r") as file:
+ content = file.read()
+ assert "Removed concepts ['ABDO_PAIN']" in content
+ assert "Added concepts ['DEPRESSION', 'DID_NOT_ATTEND', 'HYPERTENSION']" in content
+ assert "Added outputs: ['read3.csv', 'snomed.csv']" in content
+
+
+# TEST REPO NEEDS TO BE AUTOMATED
+
+# Create remote repo acmc-test1 (https://git.soton.ac.uk/mjbonifa/acmc-test1.git) and acmc-test2 (https://git.soton.ac.uk/mjbonifa/acmc-test2.git)
+
+# Init repo from the remote acmc-test1
+
+# Copy example and run map
+
+# Publish creating a version on the remote repo
+
+# Fork repo from acmc-test1 with remote acmc-test2
+
+# Publish repo creating a version on the new repo