App settings management guide

Settings for the redaction app can be set from outside by changing values in the .env file stored in your local config folder, or in S3 if running on AWS. This guide provides an overview of how to configure the application using environment variables. The application loads configurations using os.environ.get(). It first attempts to load variables from the file specified by APP_CONFIG_PATH (which defaults to config/app_config.env). If AWS_CONFIG_PATH is also set (e.g., to config/aws_config.env), variables are loaded from that file as well. Environment variables set directly in the system will always take precedence over those defined in these .env files.

Value Format Notes

  • Boolean Values: Boolean environment variables accept multiple formats: "True", "1", "true", or "TRUE" for true; "False", "0", "false", or "FALSE" for false.

  • List Values: List environment variables should be provided as comma-separated strings within square brackets, e.g., "['item1', 'item2', 'item3']". The application will automatically parse these into Python lists.

  • Temporary Folders: Setting OUTPUT_FOLDER or INPUT_FOLDER to "TEMP" will create a temporary directory that is automatically cleaned up when the application exits. This is useful for increased security in some deployment scenarios.

App Configuration File (config.env)

This section details variables related to the main application configuration file.

  • CONFIG_FOLDER
    • Description: The folder where configuration files are stored.
    • Default Value: config/
  • APP_CONFIG_PATH
    • Description: Specifies the path to the application configuration .env file. This file contains various settings that control the application’s behavior.
    • Default Value: config/app_config.env

AWS Options

This section covers configurations related to AWS services used by the application.

  • AWS_CONFIG_PATH
    • Description: Specifies the path to the AWS configuration .env file. This file is intended to store AWS credentials and specific settings.
    • Default Value: '' (empty string)
  • RUN_AWS_FUNCTIONS
    • Description: Enables or disables AWS-specific functionalities within the application. Set to "True" to enable.
    • Default Value: "False"
  • AWS_REGION
    • Description: Defines the AWS region where services like S3, Cognito, and Textract are located.
    • Default Value: ''
  • AWS_CLIENT_ID
    • Description: The client ID for AWS Cognito, used for user authentication.
    • Default Value: ''
  • AWS_CLIENT_SECRET
    • Description: The client secret for AWS Cognito, used in conjunction with the client ID for authentication.
    • Default Value: ''
  • AWS_USER_POOL_ID
    • Description: The user pool ID for AWS Cognito, identifying the user directory.
    • Default Value: ''
  • AWS_ACCESS_KEY
    • Description: The AWS access key ID for programmatic access to AWS services.
    • Default Value: ''
  • AWS_SECRET_KEY
    • Description: The AWS secret access key corresponding to the AWS access key ID.
    • Default Value: ''
  • DOCUMENT_REDACTION_BUCKET
    • Description: The name of the S3 bucket used for storing documents related to the redaction process.
    • Default Value: ''
  • PRIORITISE_SSO_OVER_AWS_ENV_ACCESS_KEYS
    • Description: If set to "True", the app will prioritize using AWS SSO credentials over access keys stored in environment variables.
    • Default Value: "True"
  • CUSTOM_HEADER
    • Description: Specifies a custom header name to be included in requests, often used for services like AWS CloudFront.
    • Default Value: ''
  • CUSTOM_HEADER_VALUE
    • Description: The value for the custom header specified by CUSTOM_HEADER.
    • Default Value: ''

Image Options

Settings related to image processing within the application.

  • IMAGES_DPI
    • Description: Dots Per Inch (DPI) setting for image processing, affecting the resolution and quality of processed images.
    • Default Value: '300.0'
  • LOAD_TRUNCATED_IMAGES
    • Description: Controls whether the application attempts to load truncated images. Set to 'True' to enable.
    • Default Value: 'True'
  • MAX_IMAGE_PIXELS
    • Description: Sets the maximum number of pixels for an image that the application will process. Leave blank for no limit. This can help prevent issues with very large images.
    • Default Value: ''
  • MERGE_BOUNDING_BOXES
    • Description: When enabled ('True'), merges nearby bounding boxes (reconstruction, grouping, and horizontal merge). Affects OCR/Textract result processing.
    • Default Value: 'True'
  • MAX_SPACES_GPU_RUN_TIME
    • Description: Maximum number of seconds to run the GPU on Hugging Face Spaces. Used to limit long-running VLM/OCR jobs when deployed on Spaces.
    • Default Value: 60

File I/O Options

Configuration for input and output file handling.

  • SESSION_OUTPUT_FOLDER
    • Description: If set to 'True', the application will save output and input files into session-specific subfolders.
    • Default Value: 'False'
  • OUTPUT_FOLDER (environment variable: GRADIO_OUTPUT_FOLDER)
    • Description: Specifies the default output folder for generated files. Can be set to "TEMP" to use a temporary directory.
    • Default Value: 'output/'
  • INPUT_FOLDER (environment variable: GRADIO_INPUT_FOLDER)
    • Description: Specifies the default input folder for files. Can be set to "TEMP" to use a temporary directory.
    • Default Value: 'input/'
  • GRADIO_TEMP_DIR
    • Description: Defines the path for Gradio’s temporary file storage.
    • Default Value: ''
  • MPLCONFIGDIR
    • Description: Specifies the cache directory for the Matplotlib library.
    • Default Value: ''

Logging Options

Settings for configuring application logging.

Note: By default, logs are stored in subfolders based on today’s date and the hostname of the instance running the app (e.g., logs/20240101/hostname/). This prevents log files from one instance overwriting logs from another instance, which is especially important when running on S3 or in multi-instance deployments. If you’re always running the app on a single system or just locally, you can disable this behavior by setting USE_LOG_SUBFOLDERS to "False".

  • SAVE_LOGS_TO_CSV
    • Description: Enables or disables saving logs to CSV files. Set to 'True' to enable.
    • Default Value: 'True'
  • USE_LOG_SUBFOLDERS
    • Description: If enabled ('True'), logs will be stored in subfolders based on date and hostname.
    • Default Value: 'True'
  • FEEDBACK_LOGS_FOLDER, ACCESS_LOGS_FOLDER, USAGE_LOGS_FOLDER
    • Description: Base folders for feedback, access, and usage logs respectively.
    • Default Values: 'feedback/', 'logs/', 'usage/'
  • S3_FEEDBACK_LOGS_FOLDER, S3_ACCESS_LOGS_FOLDER, S3_USAGE_LOGS_FOLDER
    • Description: S3 paths where feedback, access, and usage logs will be stored if RUN_AWS_FUNCTIONS is enabled.
    • Default Values: Dynamically generated based on date and hostname, e.g., 'feedback/YYYYMMDD/hostname/'.
  • LOG_FILE_NAME, USAGE_LOG_FILE_NAME, FEEDBACK_LOG_FILE_NAME
    • Description: Specifies the name for log files. USAGE_LOG_FILE_NAME and FEEDBACK_LOG_FILE_NAME default to the value of LOG_FILE_NAME.
    • Default Value: 'log.csv'
  • DISPLAY_FILE_NAMES_IN_LOGS
    • Description: If set to 'True', file names will be included in log entries.
    • Default Value: 'False'
  • CSV_ACCESS_LOG_HEADERS, CSV_FEEDBACK_LOG_HEADERS, CSV_USAGE_LOG_HEADERS
    • Description: Defines custom headers for the respective CSV logs as a string representation of a list. If blank, component labels are used.
    • Default Value: Varies; see script for CSV_USAGE_LOG_HEADERS default.
  • SAVE_LOGS_TO_DYNAMODB
    • Description: Enables or disables saving logs to AWS DynamoDB. Set to 'True' to enable.
    • Default Value: 'False'
  • ACCESS_LOG_DYNAMODB_TABLE_NAME, FEEDBACK_LOG_DYNAMODB_TABLE_NAME, USAGE_LOG_DYNAMODB_TABLE_NAME
    • Description: Names of the DynamoDB tables for storing access, feedback, and usage logs.
    • Default Values: 'redaction_access_log', 'redaction_feedback', 'redaction_usage'
  • DYNAMODB_ACCESS_LOG_HEADERS, DYNAMODB_FEEDBACK_LOG_HEADERS, DYNAMODB_USAGE_LOG_HEADERS
    • Description: Specifies the headers (attributes) for the respective DynamoDB log tables.
    • Default Value: ''
  • LOGGING
    • Description: Enables or disables general console logging. Set to 'True' to enable.
    • Default Value: 'False'

Gradio & General App Options

Configurations for the Gradio UI, server behavior, and application limits.

  • FAVICON_PATH
    • Description: Path to the favicon icon file for the web interface.
    • Default Value: "favicon.png"
  • RUN_FASTAPI
    • Description: If set to "True", the application will be served via FastAPI, allowing for API endpoint integration.
    • Default Value: "False"
  • RUN_MCP_SERVER
    • Description: If set to "True", the application will run as an MCP (Model Context Protocol) server.
    • Default Value: "False"
  • GRADIO_SERVER_NAME
    • Description: The IP address the Gradio server will bind to. Use "0.0.0.0" to allow external access.
    • Default Value: "127.0.0.1"
  • GRADIO_SERVER_PORT
    • Description: The network port on which the Gradio server will listen.
    • Default Value: 7860
  • ALLOWED_ORIGINS
    • Description: A comma-separated list of allowed origins for Cross-Origin Resource Sharing (CORS).
    • Default Value: ''
  • ALLOWED_HOSTS
    • Description: A comma-separated list of allowed hostnames.
    • Default Value: ''
  • ROOT_PATH
    • Description: The root path for the application, useful if running behind a reverse proxy (e.g., /app).
    • Default Value: ''
  • FASTAPI_ROOT_PATH
    • Description: The root path for the FastAPI application, used when RUN_FASTAPI is true.
    • Default Value: "/"
  • MAX_QUEUE_SIZE
    • Description: The maximum number of requests that can be queued in the Gradio interface.
    • Default Value: 20
  • MAX_FILE_SIZE
    • Description: Maximum file size allowed for uploads (e.g., “250mb”, “1gb”).
    • Default Value: '250mb'
  • DEFAULT_CONCURRENCY_LIMIT
    • Description: The default concurrency limit for Gradio event handlers, controlling how many requests can be processed simultaneously.
    • Default Value: 3
  • MAX_SIMULTANEOUS_FILES
    • Description: The maximum number of files that can be processed at once.
    • Default Value: 10
  • MAX_DOC_PAGES
    • Description: The maximum number of pages a document can have.
    • Default Value: 3000
  • MAX_TABLE_ROWS / MAX_TABLE_COLUMNS
    • Description: Maximum number of rows and columns for tabular data processing.
    • Default Values: 250000 / 100
  • MAX_OPEN_TEXT_CHARACTERS
    • Description: Maximum number of characters for open text input.
    • Default Value: 50000
  • PAGE_BREAK_VALUE
    • Description: Number of pages to process before breaking and restarting from the last finished page.
    • Default Value: 99999
    • Note: This feature is not currently activated in the application.
  • MAX_TIME_VALUE
    • Description: Maximum time value for processing operations.
    • Default Value: 999999
  • TLDEXTRACT_CACHE
    • Description: Path to the cache directory used by the tldextract library.
    • Default Value: 'tmp/tld/'
  • COGNITO_AUTH
    • Description: Enables or disables AWS Cognito authentication. Set to 'True' to enable.
    • Default Value: 'False'
  • SHOW_FEEDBACK_BUTTONS
    • Description: If set to "True", displays feedback buttons in the Gradio interface.
    • Default Value: "False"
  • USER_GUIDE_URL
    • Description: A safe URL pointing to the user guide. The URL is validated against a list of allowed domains.
    • Default Value: "https://seanpedrick-case.github.io/doc_redaction"
  • INTRO_TEXT
    • Description: Custom introduction text for the app. Should be in Markdown format, html is stripped out. Can also be set to a path to a .txt file (e.g., "intro.txt"), which will be read and used as the intro text. The text is automatically sanitized to remove dangerous HTML/scripts while preserving safe markdown syntax.
    • Default Value: "Too long to display here, see tools/config.py"
  • SHOW_EXAMPLES
    • Description: If set to "True", displays example files in the Gradio interface.
    • Default Value: "True"
  • SHOW_AWS_EXAMPLES
    • Description: If set to "True", includes AWS-specific examples.
    • Default Value: "False"
  • SHOW_DIFFICULT_OCR_EXAMPLES
    • Description: If set to "True", includes examples that demonstrate difficult OCR scenarios.
    • Default Value: "False"
  • SHOW_ALL_OUTPUTS_IN_OUTPUT_FOLDER
    • Description: If set to "True", displays all output files in the output folder interface.
    • Default Value: "False"
  • FILE_INPUT_HEIGHT
    • Description: Sets the height (in pixels) of the file input component in the Gradio UI.
    • Default Value: 200
  • SHOW_QUICKSTART
    • Description: If set to "True", displays a quickstart/walkthrough component in the Gradio interface to guide new users.
    • Default Value: "False"
  • SHOW_SUMMARISATION
    • Description: If set to "True", shows the summarisation tab in the Gradio interface.
    • Default Value: "False"
  • APPLY_DUPLICATES_TO_FILE_AUTOMATICALLY
    • Description: If set to "True", duplicate detection results can be applied to the file automatically in the UI workflow.
    • Default Value: "False"
  • RUN_ALL_EXAMPLES_THROUGH_AWS
    • Description: If set to "True", example documents will be run through AWS (e.g. Textract) when examples are used.
    • Default Value: "False"

Redaction & PII Options

Configurations related to text extraction, PII detection, and the redaction process.

UI and Model Selection

  • EXTRACTION_AND_PII_OPTIONS_OPEN_BY_DEFAULT
    • Description: If set to "True", the “Extraction and PII Options” accordion in the UI will be open by default.
    • Default Value: "True"
  • SHOW_LOCAL_TEXT_EXTRACTION_OPTIONS / SHOW_AWS_TEXT_EXTRACTION_OPTIONS
    • Description: Controls whether local (Tesseract) or AWS (Textract) text extraction options are shown in the UI.
    • Default Value: "True" for both.
    • Note: If both are set to "False", the application will automatically enable SHOW_LOCAL_TEXT_EXTRACTION_OPTIONS to ensure at least one option is available.
  • SELECTABLE_TEXT_EXTRACT_OPTION, LOCAL_OCR_MODEL_TEXT_EXTRACT_OPTION, TEXTRACT_TEXT_EXTRACT_OPTION
    • Description: Labels for text extraction model options displayed in the UI. Customize the display names for “Local model - selectable text”, “Local OCR model - PDFs without selectable text”, and “AWS Textract service - all PDF types” respectively.
    • Default Values: "Local model - selectable text", "Local OCR model - PDFs without selectable text", "AWS Textract service - all PDF types"
  • BEDROCK_VLM_TEXT_EXTRACT_OPTION, GEMINI_VLM_TEXT_EXTRACT_OPTION, AZURE_OPENAI_VLM_TEXT_EXTRACT_OPTION
    • Description: Labels for cloud VLM text extraction options (AWS Bedrock, Google Gemini, Azure/OpenAI) displayed in the UI when the corresponding show options are enabled.
    • Default Values: "AWS Bedrock VLM OCR - all PDF types", "Google Gemini VLM OCR - all PDF types", "Azure/OpenAI VLM OCR - all PDF types"
  • EFFICIENT_OCR
    • Description: If set to "True", uses a two-step OCR process for PDFs: try selectable text extraction per page first; only run OCR (Tesseract/Textract/VLM) on pages where no text could be extracted. Saves cost and time. Affects both the Gradio app and the CLI.
    • Default Value: "False"
  • EFFICIENT_OCR_MIN_WORDS
    • Description: Minimum number of extractable words on a page to use the text-only route when EFFICIENT_OCR is enabled; below this threshold the page uses OCR.
    • Default Value: 20
  • EFFICIENT_OCR_MIN_IMAGE_COVERAGE_FRACTION
    • Description: When EFFICIENT_OCR is enabled, minimum fraction of the page area (MediaBox) that a single embedded image must cover to route that page through OCR (redact_image_pdf) in addition to the word-count rule. This catches pages with plenty of selectable text but meaningful raster content (e.g. photos) where text may appear only inside the image. Set to 0 to disable image-based routing and rely on word count only.
    • Default Value: 0.01 (1% of page area)
  • MAX_WORKERS
    • Description: Default maximum number of worker threads for parallel processing across the app. Used as the default or cap for thread pools in review (saving redacted pages, loading page images), file conversion (PDF-to-images, input file listing, annotation processing), file redaction (text extraction, merge/summary, pikepdf annotations), custom image analyser (mapping analyzer results to bounding boxes), and data anonymisation (initial column cleaning). When not set, OCR_FIRST_PASS_MAX_WORKERS and SUMMARY_PAGE_GROUP_MAX_WORKERS default to this value; set those explicitly to override for OCR or summarisation only.
    • Default Value: 8
  • OCR_FIRST_PASS_MAX_WORKERS
    • Description: Maximum number of threads for the OCR first pass in image-based PDF redaction (e.g. when using AWS Textract). Set to 1 for sequential processing; use a value greater than 1 to run OCR on multiple pages in parallel. Only applies when the OCR method supports parallel calls (e.g. Textract). Defaults to MAX_WORKERS if not set. Can be overridden via CLI --ocr_first_pass_max_workers or direct-mode/Lambda env.
    • Default Value: Same as MAX_WORKERS (e.g. 8)
  • TESSERACT_MAX_WORKERS
    • Description: Maximum number of worker threads used to run local Tesseract OCR on multiple pages in parallel (page-level parallelism). Keep this lower than MAX_WORKERS to avoid saturating CPU/RAM; increase gradually if you have spare CPU and want faster OCR.
    • Default Value: 4
  • PADDLE_MAX_WORKERS
    • Description: Maximum number of worker threads used to run local PaddleOCR on multiple pages in parallel when Paddle is the chosen local OCR model. Paddle is often GPU-bound and shares one model instance; keep this low (e.g. 1–2) to avoid VRAM contention or errors. Set to 1 for effectively sequential Paddle OCR per document.
    • Default Value: 2
  • LINE_TO_WORD_SEGMENT_MAX_WORKERS
    • Description: Maximum number of worker threads used to run the line-to-word segmentation process. This is a CPU/memory intensive process and can slow down when matching MAX_WORKERS on busy documents.
    • Default Value: 4
  • SUMMARY_PAGE_GROUP_MAX_WORKERS
    • Description: Maximum number of threads for page-group summarisation in document summarisation. Set to 1 for sequential processing (recommended for local LLM models); use a value greater than 1 to summarise multiple page groups in parallel (typically for API-based models). Defaults to MAX_WORKERS if not set. Can be overridden via CLI --summary_page_group_max_workers or direct-mode/Lambda env.
    • Default Value: Same as MAX_WORKERS (e.g. 8)

Hybrid Textract + Bedrock VLM

When AWS Textract is selected for text extraction, these options allow lines with low confidence to be re-extracted using Bedrock VLM and the result to replace the Textract output for those lines.

  • SHOW_HYBRID_TEXTRACT_BEDROCK_CHECKBOX
    • Description: When set to 'True', the “High-quality Textract OCR (re-run low-confidence lines with Bedrock VLM)” checkbox is shown in the GUI when AWS Textract options are visible. Set to 'False' to hide this option from the UI.
    • Default Value: 'False'
  • HYBRID_TEXTRACT_BEDROCK_VLM
    • Description: When set to 'True' and AWS Textract is selected, lines whose average confidence is below HYBRID_TEXTRACT_BEDROCK_VLM_CONFIDENCE_THRESHOLD are re-analyzed with Bedrock VLM and the extracted text replaces the Textract result for those lines.
    • Default Value: 'False'
  • HYBRID_TEXTRACT_BEDROCK_VLM_CONFIDENCE_THRESHOLD
    • Description: Line average confidence (0–100) below this value triggers Bedrock VLM re-extraction when HYBRID_TEXTRACT_BEDROCK_VLM is enabled.
    • Default Value: 97
  • HYBRID_TEXTRACT_BEDROCK_VLM_PADDING
    • Description: Padding in pixels around the line crop when calling Bedrock VLM for re-extraction.
    • Default Value: 5
  • SAVE_TEXTRACT_BEDROCK_HYBRID_EXAMPLES
    • Description: If set to "True", saves example images or artefacts from the Textract + Bedrock VLM hybrid re-extraction path for debugging (see tools/config.py / hybrid Textract–Bedrock implementation).
    • Default Value: "False"

Bedrock VLM and LLM cost estimation

These variables are used by the app to estimate AWS cost when “AWS Bedrock VLM OCR - all PDF types” or “LLM (AWS Bedrock)” PII detection is selected. Estimates are shown in the UI (e.g. estimated cost and time).

  • BEDROCK_VLM_INPUT_COST
    • Description: USD per million input tokens for Bedrock VLM OCR cost estimate.
    • Default Value: 3.0
  • BEDROCK_VLM_OUTPUT_COST
    • Description: USD per million output tokens for Bedrock VLM OCR cost estimate.
    • Default Value: 15.0
  • BEDROCK_VLM_PIXELS_PER_INPUT_TOKEN
    • Description: Pixels (width × height) per input token; used with VLM_MAX_IMAGE_SIZE to estimate input tokens per page for VLM cost calculation.
    • Default Value: 2500
  • BEDROCK_LLM_INPUT_COST
    • Description: USD per million input tokens for Bedrock LLM (e.g. PII detection) cost estimate.
    • Default Value: 3.0
  • BEDROCK_LLM_OUTPUT_COST
    • Description: USD per million output tokens for Bedrock LLM cost estimate.
    • Default Value: 15.0
  • BEDROCK_LLM_INPUT_TOKENS_PER_PAGE
    • Description: Estimated input tokens per page used for Bedrock LLM cost calculation.
    • Default Value: 2000
  • BEDROCK_LLM_OUTPUT_TOKENS_PER_PAGE
    • Description: Estimated output tokens per page used for Bedrock LLM cost calculation.
    • Default Value: 250
  • SHOW_BEDROCK_VLM_MODELS, SHOW_GEMINI_VLM_MODELS, SHOW_AZURE_OPENAI_VLM_MODELS
    • Description: Controls whether AWS Bedrock, Google Gemini, or Azure/OpenAI VLM text extraction options are shown in the UI.
    • Default Value: "False" for all.
  • NO_REDACTION_PII_OPTION, LOCAL_PII_OPTION, AWS_PII_OPTION
    • Description: Labels for PII detection model options displayed in the UI. Customize the display names for “Only extract text (no redaction)”, “Local”, and “AWS Comprehend” respectively.
    • Default Values: "Only extract text (no redaction)", "Local", "AWS Comprehend"
  • AWS_LLM_PII_OPTION, INFERENCE_SERVER_PII_OPTION, LOCAL_TRANSFORMERS_LLM_PII_OPTION
    • Description: Labels for LLM-based PII detection options: AWS Bedrock, local inference server, and local transformers LLM.
    • Default Values: "LLM (AWS Bedrock)", "Local inference server", "Local transformers LLM"
  • SHOW_INFERENCE_SERVER_PII_OPTIONS, SHOW_TRANSFORMERS_LLM_PII_DETECTION_OPTIONS, SHOW_AWS_BEDROCK_LLM_MODELS
    • Description: Controls whether inference-server, local transformers LLM, or AWS Bedrock LLM PII detection options are shown in the UI.
    • Default Value: "False" for inference-server and transformers; "False" for AWS Bedrock LLM.
  • CHOSEN_LLM_PII_INFERENCE_METHOD
    • Description: Default LLM inference method for PII detection. Options include "aws-bedrock", "local", "inference-server", "azure-openai", "gemini". Only methods enabled via the corresponding SHOW_*_LLM_PII_OPTIONS appear in the UI.
    • Default Value: "aws-bedrock"
  • SHOW_LOCAL_LLM_PII_OPTIONS, SHOW_INFERENCE_SERVER_LLM_PII_OPTIONS, SHOW_AZURE_LLM_PII_OPTIONS, SHOW_GEMINI_LLM_PII_OPTIONS
    • Description: Controls whether local, inference-server, Azure/OpenAI, or Gemini LLM PII options are shown in the UI.
    • Default Value: "False" for all.
  • SHOW_PII_IDENTIFICATION_OPTIONS
    • Description: If set to "True", shows the PII entity identification/selection options in the UI.
    • Default Value: "True"
  • SHOW_GEMINI_LLM_MODELS, SHOW_AZURE_LLM_MODELS, SHOW_INFERENCE_SERVER_LLM_MODELS
    • Description: Controls whether Google Gemini, Azure/OpenAI, or inference-server LLM options appear in the PII detection model list. When enabled, the corresponding models (e.g. gemini-2.5-flash, gpt-4o-mini) can be selected for PII detection.
    • Default Value: "False" for all.
  • SHOW_LOCAL_PII_DETECTION_OPTIONS / SHOW_AWS_PII_DETECTION_OPTIONS
    • Description: Controls whether local or AWS (Comprehend) PII detection options are shown in the UI.
    • Default Value: "True" for both.
    • Note: If both are set to "False", the application will automatically enable SHOW_LOCAL_PII_DETECTION_OPTIONS to ensure at least one option is available.
  • DEFAULT_TEXT_EXTRACTION_MODEL
    • Description: Sets the default text extraction model selected in the UI.
    • Default Value: Automatically defaults to AWS Textract if SHOW_AWS_TEXT_EXTRACTION_OPTIONS is enabled, otherwise defaults to the local selectable text option.
  • DEFAULT_PII_DETECTION_MODEL
    • Description: Sets the default PII detection model selected in the UI.
    • Default Value: Automatically defaults to AWS Comprehend if SHOW_AWS_PII_DETECTION_OPTIONS is enabled, otherwise defaults to the local model.
  • LOAD_REDACTION_ANNOTATIONS_FROM_PDF
    • Description: If set to "True", the application will load existing redaction annotations from PDFs during the review step.
    • Default Value: "True"

External Tool Paths

  • TESSERACT_FOLDER
    • Description: Path to the local Tesseract OCR installation folder.
    • Default Value: ''
    • Installation Note: For Windows, install Tesseract 5.5.0 from UB-Mannheim/tesseract. This environment variable should point to the Tesseract folder (e.g., tesseract/).
  • TESSERACT_DATA_FOLDER
    • Description: Path to the Tesseract trained data files (tessdata).
    • Default Value: "/usr/share/tessdata"
  • POPPLER_FOLDER
    • Description: Path to the local Poppler installation’s bin folder.
    • Default Value: ''
    • Installation Note: For Windows, install Poppler from oschwartz10612/poppler-windows. This variable needs to point to the Poppler bin folder (e.g., poppler/poppler-24.02.0/Library/bin/).
  • PADDLE_MODEL_PATH / SPACY_MODEL_PATH
    • Description: Custom directory for PaddleOCR and spaCy model storage, useful for environments like AWS Lambda.
    • Default Value: '' (uses default location).
  • PADDLE_FONT_PATH
    • Description: Custom font path for PaddleOCR. If empty, the application will attempt to use system fonts to avoid downloading simfang.ttf or PingFang-SC-Regular.ttf. Set this if you want to use a specific font file for PaddleOCR text rendering.
    • Default Value: '' (uses system fonts).
  • GEMINI_API_KEY
    • Description: API key for Google Gemini. Required when using Gemini for PII detection or VLM OCR (SHOW_GEMINI_LLM_MODELS or SHOW_GEMINI_VLM_MODELS).
    • Default Value: ''
  • AZURE_OPENAI_API_KEY
    • Description: API key for Azure or OpenAI. Required when using Azure/OpenAI for PII detection or VLM OCR (SHOW_AZURE_LLM_MODELS or SHOW_AZURE_OPENAI_VLM_MODELS).
    • Default Value: ''
  • AZURE_OPENAI_INFERENCE_ENDPOINT
    • Description: Inference endpoint URL for Azure/OpenAI. Used with AZURE_OPENAI_API_KEY when Azure/OpenAI LLM or VLM options are enabled.
    • Default Value: ''
  • HF_TOKEN
    • Description: Hugging Face token. May be required for downloading gated or private models (e.g. some local transformers LLM or VLM models).
    • Default Value: ''

Local OCR (Tesseract & PaddleOCR)

  • SHOW_OCR_GUI_OPTIONS

    • Description: If set to "True", OCR-related options (e.g. local OCR model, Paddle options) are shown in the UI.
    • Default Value: "True"

  • DEFAULT_LOCAL_OCR_MODEL

    • Description: Choose the engine for local OCR: "tesseract", "paddle", "hybrid-paddle", "hybrid-vlm", "hybrid-paddle-vlm", "hybrid-paddle-inference-server", "vlm", or "inference-server".
    • Default Value: "tesseract"

  • SHOW_LOCAL_OCR_MODEL_OPTIONS

    • Description: If set to "True", allows the user to select the local OCR model from the UI.
    • Default Value: "False"

  • HYBRID_OCR_CONFIDENCE_THRESHOLD

    • Description: In “hybrid-paddle” mode, this is the Tesseract confidence score below which PaddleOCR will be used for re-extraction.
    • Default Value: 95

  • HYBRID_OCR_PADDING

    • Description: In “hybrid-paddle” mode, padding added to the word’s bounding box before re-extraction.
    • Default Value: 1

  • PADDLE_USE_TEXTLINE_ORIENTATION

    • Description: Toggles textline orientation detection for PaddleOCR.
    • Default Value: "False"

  • PADDLE_DET_DB_UNCLIP_RATIO

    • Description: Controls the expansion ratio of the detected text region in PaddleOCR.
    • Default Value: 1.2

  • SAVE_EXAMPLE_HYBRID_IMAGES

    • Description: Saves comparison images when using “hybrid-paddle” OCR mode.
    • Default Value: "False"

  • SAVE_PAGE_OCR_VISUALISATIONS

    • Description: If set to "True", saves visualisations of Tesseract, PaddleOCR, and Textract bounding boxes overlaid on the page images.
    • Default Value: "False"

  • INCLUDE_OCR_VISUALISATION_IN_OUTPUT_FILES

    • Description: If set to "True", OCR visualisation output files (created when SAVE_PAGE_OCR_VISUALISATIONS is enabled) will be included in the final output file list returned by choose_and_run_redactor. This makes the visualisation files available in the Gradio output interface.
    • Default Value: "False"

  • SAVE_WORD_SEGMENTER_OUTPUT_IMAGES

    • Description: If set to "True", saves output images from the word segmenter for debugging purposes.
    • Default Value: "False"

  • PREPROCESS_LOCAL_OCR_IMAGES

    • Description: If set to "True", images will be preprocessed before local OCR. Can slow down processing.
    • Default Value: "True"
    • Note: Testing has shown that preprocessing doesn’t necessarily improve OCR results and can significantly slow down extraction. Consider setting this to "False" if processing speed is a priority.

  • SAVE_PREPROCESS_IMAGES

    • Description: If set to "True", saves the preprocessed images for debugging purposes.
    • Default Value: "False"

  • SHOW_PADDLE_MODEL_OPTIONS

    • Description: If set to "True", allows the user to select PaddleOCR-related options (paddle, hybrid-paddle) from the UI.
    • Default Value: "False"

  • MODEL_CACHE_PATH

    • Description: Path to the directory where models are cached.
    • Default Value: "./model_cache"

  • TESSERACT_SEGMENTATION_LEVEL

    • Description: Tesseract PSM (Page Segmentation Mode) level to use for OCR. Valid values are 0-13.
    • Default Value: 11

  • TESSERACT_WORD_LEVEL_OCR

    • Description: If set to "True", uses Tesseract word-level OCR instead of line-level.
    • Default Value: "True"

  • CONVERT_LINE_TO_WORD_LEVEL

    • Description: If set to "True", converts PaddleOCR line-level OCR results to word-level for better precision.
    • Default Value: "False"

  • LOAD_PADDLE_AT_STARTUP

    • Description: If set to "True", loads the PaddleOCR model at application startup.
    • Default Value: "False"

  • USE_TRANSFORMERS_VLM_MODEL_AS_LLM

    • Description: When set to "True", use the same local transformers VLM model (e.g. Qwen3-VL-4B-Instruct) for LLM tasks (e.g. PII entity detection in tools/llm_entity_detection.py) as for VLM/OCR. Overrides LOCAL_TRANSFORMERS_LLM_PII_MODEL_CHOICE when local transformers LLM is selected. The VLM weights must be loaded first (e.g. LOAD_TRANSFORMERS_VLM_MODEL_AT_START=True, or by running a VLM OCR task first); otherwise the standard PII model is used.
    • Default Value: "False"

  • LOCAL_TRANSFORMERS_LLM_PII_MODEL_CHOICE

    • Description: Identifier for the local transformers LLM used for PII detection (e.g. "gemma-3-4b", "qwen-3-4b"). Used for display and to select the correct repo/file/folder. Supported values include gemma-3-4b, gemma-3-12b, gemma-2, qwen-3-4b, GPT-OSS 20B, granite-4-tiny, granite-4-micro. Ignored for local LLM when USE_TRANSFORMERS_VLM_MODEL_AS_LLM is "True" (the VLM model is used instead).
    • Default Value: "gemma-3-4b"

  • LOCAL_TRANSFORMERS_LLM_PII_REPO_ID, LOCAL_TRANSFORMERS_LLM_PII_MODEL_FILE, LOCAL_TRANSFORMERS_LLM_PII_MODEL_FOLDER

    • Description: Hugging Face repo ID, for the local transformers PII detection model. Can be overridden automatically based on LOCAL_TRANSFORMERS_LLM_PII_MODEL_CHOICE.
    • Default Values: e.g. "unsloth/gemma-3-4b-it-bnb-4bit".

  • OVERRIDE_LLM_TRANSFORMERS_REPO_ID

    • Description: If non-empty, overrides LOCAL_TRANSFORMERS_LLM_PII_REPO_ID (the Hugging Face repo ID or local path used for the local transformers LLM PII model). Use to point at a different checkpoint or a local folder without changing LOCAL_TRANSFORMERS_LLM_PII_MODEL_CHOICE. Leave empty for no override.
    • Default Value: '' (empty)

  • LOAD_TRANSFORMERS_LLM_PII_MODEL_AT_START

    • Description: If set to "True", loads the local transformers LLM PII model at application startup (reduces latency on first use).
    • Default Value: "False"

  • MULTIMODAL_PROMPT_FORMAT

    • Description: If set to "True", uses multimodal prompt format for the local LLM. Set automatically for some model choices (e.g. Gemma 3 4B/12B).
    • Default Value: "False"

  • REASONING_SUFFIX

    • Description: Suffix added to prompts for reasoning-capable models (e.g. "Reasoning: low" for GPT-OSS 20B, "/nothink" for Qwen 3 4B to disable chain-of-thought). Model-specific; see config for defaults.
    • Default Value: Model-dependent (e.g. "Reasoning: low" or "/nothink" or '').

  • PRINT_TRANSFORMERS_USER_PROMPT

    • Description: If set to "True", prints the system and user prompts to the console when calling the local transformers LLM (tools/llm_funcs.py). Useful for debugging; avoid in production if logs are sensitive.
    • Default Value: "False"

  • LLM generation parameters: LLM_TEMPERATURE (default 0.1), LLM_TOP_K (64), LLM_MIN_P (0), LLM_TOP_P (0.95), LLM_REPETITION_PENALTY (1.0), LLM_MAX_NEW_TOKENS (4096), LLM_CONTEXT_LENGTH (32768), LLM_SEED (42), LLM_RESET (False), LLM_STREAM (True), LLM_STOP_STRINGS, LLM_THREADS (-1), SPECULATIVE_DECODING (False), ASSISTANT_MODEL, LLM_MODEL_DTYPE (bfloat16), and other LLM_* / related variables in tools/config.py control local/transformers LLM generation in tools/llm_funcs.py. LLM_TEMPERATURE and LLM_MAX_NEW_TOKENS also control PII detection LLM calls and retries.

  • SHOW_INFERENCE_SERVER_VLM_OPTIONS

    • Description: If set to "True", allows the user to select inference-server-related options from the UI.
    • Default Value: "False"

  • SHOW_HYBRID_MODELS

    • Description: If set to "True", enables hybrid model options (e.g., hybrid-paddle-vlm, hybrid-paddle-inference-server) in the UI.
    • Default Value: "False"

  • INFERENCE_SERVER_API_URL

    • Description: Base URL of the inference-server API for remote OCR processing and PII detection.
    • Default Value: "http://localhost:8080"

  • INFERENCE_SERVER_MODEL_NAME

    • Description: Optional model name to use for inference-server API. If empty, uses the default model on the server.
    • Default Value: ''

  • INFERENCE_SERVER_TIMEOUT

    • Description: Timeout in seconds for inference-server API requests.
    • Default Value: 300

  • DEFAULT_INFERENCE_SERVER_VLM_MODEL

    • Description: Default model name for inference-server VLM API calls. If empty, uses INFERENCE_SERVER_MODEL_NAME or the server default.
    • Default Value: "qwen_3_vl_30b_a3b_it"

  • DEFAULT_INFERENCE_SERVER_PII_MODEL

    • Description: Default model name for inference-server PII detection API calls. If empty, uses INFERENCE_SERVER_MODEL_NAME, CHOSEN_INFERENCE_SERVER_PII_MODEL, or the server default.
    • Default Value: "gemma_3_12b"

  • CHOSEN_INFERENCE_SERVER_PII_MODEL

    • Description: Default inference-server PII model selected when SHOW_INFERENCE_SERVER_LLM_MODELS is enabled. Must be one of the registered inference-server models.
    • Default Value: First model in the inference-server models list (e.g. "unnamed-inference-server-model").

  • INFERENCE_SERVER_LLM_PII_MODEL_CHOICE

    • Description: Primary config variable for the inference-server PII model used at runtime. Defaults to DEFAULT_INFERENCE_SERVER_PII_MODEL, then CHOSEN_INFERENCE_SERVER_PII_MODEL, if not set.
    • Default Value: Inherits from DEFAULT_INFERENCE_SERVER_PII_MODEL or CHOSEN_INFERENCE_SERVER_PII_MODEL.

  • SHOW_INFERENCE_SERVER_VLM_MODEL_OPTIONS

    • Description: If set to "True", allows the user to select from inference-server VLM model options in the UI.
    • Default Value: "False"

Vision Language Model (VLM) Options

  • SHOW_VLM_MODEL_OPTIONS
    • Description: If set to "True", VLM (Vision Language Model) options will be shown in the UI.
    • Default Value: "False"
  • LOAD_TRANSFORMERS_VLM_MODEL_AT_START
    • Description: When SHOW_VLM_MODEL_OPTIONS is "True", if set to "True", loads local transformers VLM weights at application startup (same as historical behavior). If "False", weights load on the first VLM OCR call (tools/run_vlm.py), reducing startup memory at the cost of a slower first run. Compare with LOAD_TRANSFORMERS_LLM_PII_MODEL_AT_START for the PII LLM (which defaults to "False").
    • Default Value: "True"
  • SELECTED_LOCAL_TRANSFORMERS_VLM_MODEL
    • Description: Selected vision model for OCR. Choose from: "Nanonets-OCR2-3B", "Dots.OCR", "Qwen3-VL-2B-Instruct", "Qwen3-VL-4B-Instruct", "Qwen3-VL-8B-Instruct", "Qwen3-VL-30B-A3B-Instruct", "Qwen3-VL-235B-A22B-Instruct", "PaddleOCR-VL".
    • Default Value: "Qwen3-VL-8B-Instruct"
  • QUANTISE_VLM_MODELS
    • Description: If set to "True", the VLM models will be quantized using 4-bit quantisation (bitsandbytes).
    • Default Value: "False"
  • OVERRIDE_VLM_REPO_ID
    • Description: If non-empty, overrides the Hugging Face repo ID or local path used for the selected local transformers VLM (SELECTED_LOCAL_TRANSFORMERS_VLM_MODEL). Use to point at a different checkpoint or a local folder. Leave empty for no override.
    • Default Value: '' (empty)
  • USE_FLASH_ATTENTION
    • Description: If set to "True", uses flash attention for the VLM, which can improve performance (not possible on Windows).
    • Default Value: "False"
  • VLM_MAX_IMAGE_SIZE
    • Description: Upper bound on total pixels (width × height) for images sent to the VLM after preparation (_prepare_image_for_vlm). Larger images are downscaled (aspect ratio preserved). For AWS Bedrock VLM paths, the effective cap is raised automatically (much larger budget) so this variable mainly limits local/inference-server/Gemini/Azure full-page and hybrid crops unless Bedrock is used. Multiples of 32×32 align with Qwen3-VL-style tiling.
    • Default Value: 819200
  • VLM_MIN_IMAGE_SIZE
    • Description: Minimum total pixels for full-page VLM OCR (local transformers, inference-server, Gemini, Azure/OpenAI, Bedrock page). Smaller pages are upscaled (aspect preserved). Does not apply to hybrid line/crop VLM passes—those use VLM_HYBRID_MIN_IMAGE_SIZE. Preparation combines this with VLM_MIN_DPI / VLM_MAX_DPI; if constraints conflict, max pixel/DPI caps win (see console warning).
    • Default Value: 614400
  • VLM_HYBRID_MIN_IMAGE_SIZE
    • Description: Minimum total pixels for hybrid VLM crops (second pass on line regions: local VLM, inference-server, Bedrock, Gemini, Azure in hybrid modes). Smaller crops are upscaled. Full-page VLM uses VLM_MIN_IMAGE_SIZE instead. Subject to the same DPI and max-pixel logic as other VLM images.
    • Default Value: 153600
  • VLM_MIN_DPI
    • Description: Minimum effective DPI after preparation. Uses the image’s reported DPI from metadata (image.info["dpi"]), defaulting to 72 if absent. If reported DPI is below this value, the image is upscaled so that reported_dpi × scale falls within [VLM_MIN_DPI, VLM_MAX_DPI] (together with min/max pixel limits).
    • Default Value: 200.0
  • VLM_MAX_DPI
    • Description: Maximum effective DPI after preparation. High-DPI scans are downscaled so effective DPI (see VLM_MIN_DPI) does not exceed this value, subject to VLM_MAX_IMAGE_SIZE (and hybrid/full-page min pixel settings).
    • Default Value: 300.0
  • VLM_MAX_ASPECT_RATIO
    • Description: Upper bound on image aspect ratio after white-padding for every VLM path in custom_image_analyser_engine: local transformers (extract_text_from_image_vlm), inference-server (_call_inference_server_vlm_api), AWS Bedrock (_call_bedrock_vlm_api), Gemini, and Azure/OpenAI. The code pads very long/thin crops (typical hybrid line regions) so max(width/height, height/width) does not exceed this value. Lower values are stricter (more padding, smaller embedded text relative to canvas); higher values send nearer-native crops but may hit provider limits. Values below 1.0 are clamped to 1.0. For Bedrock, if the image is still above the AWS Converse vision limit (20:1) after this step, a second stricter pad is applied automatically.
    • Default Value: 10.0
  • VLM_DISABLE_QWEN3_5_THINKING
    • Description: If set to "True", disables Qwen3.5 “thinking” mode for local transformers VLM calls. Implemented by appending <think></think> to the generation prompt after apply_chat_template, so the model skips its reasoning chain and outputs the answer directly.
    • Default Value: "False"
  • INFERENCE_SERVER_DISABLE_THINKING
    • Description: If set to "True", disables thinking for inference-server VLM calls (the server-side equivalent of VLM_DISABLE_QWEN3_5_THINKING). Adds chat_template_kwargs: {"enable_thinking": false} to every request payload; vLLM applies the Qwen3/Qwen3.5 chat template server-side and honours this flag to suppress <think>...</think> generation entirely. Useful when the inference server is running a Qwen3.5 reasoning model and thinking tokens are unnecessary (e.g. for simple line-crop OCR in hybrid-paddle-inference-server mode). Eliminates thinking overhead and removes the need to raise HYBRID_OCR_MAX_NEW_TOKENS to accommodate the reasoning budget.
    • Default Value: "False"
  • MAX_NEW_TOKENS
    • Description: Maximum number of tokens to generate for VLM responses.
    • Default Value: 4096
  • DEFAULT_MAX_NEW_TOKENS
    • Description: Default maximum number of tokens to generate for VLM responses.
    • Default Value: 4096
  • MAX_INPUT_TOKEN_LENGTH
    • Description: Maximum number of tokens that can be input to the VLM.
    • Default Value: 8192
  • OVERWRITE_EXISTING_OCR_RESULTS
    • Description: If set to "True", always creates new OCR results instead of loading from existing JSON files.
    • Default Value: "False"
  • SAVE_VLM_INPUT_IMAGES
    • Description: If set to "True", saves input images sent to VLM OCR for debugging purposes.
    • Default Value: "False"
  • ADD_VLM_BOUNDING_BOX_RULES
    • Description: If set to "True", adds bounding box rules to the VLM prompt (e.g. coordinate format or output constraints for OCR).
    • Default Value: "False"
  • HYBRID_OCR_MAX_NEW_TOKENS
    • Description: Maximum number of tokens the inference server (or local VLM) may generate per hybrid line-crop OCR call. This is the generation budget passed directly to the model via max_tokens. For reasoning models such as Qwen3.5, thinking tokens (<think>...</think>) count against this budget before any answer is produced, so the value must be large enough to cover both the reasoning chain and the short JSON answer. Setting this too low (e.g. 50) causes the model to exhaust the budget during thinking and return empty results for every line. Increase to 2048 or higher if you observe “Inference server returned no results” log lines when using hybrid-paddle-inference-server with a reasoning model.
    • Default Value: 1024
  • HYBRID_OCR_MAX_WORDS
    • Description: Maximum number of words allowed in a hybrid OCR result for a single text line. Results whose word count exceeds this threshold are discarded as likely hallucinations or runaway model output, and the original OCR result (e.g. from Paddle) is kept instead. This is a separate sanity check from HYBRID_OCR_MAX_NEW_TOKENS — it operates on the parsed response text, not on the token budget. A single line of text rarely contains more than ~50 words, so the default is conservative.
    • Default Value: 50
  • VLM_OCR_INTRO_TEXT
    • Description: UI intro text when the local OCR option vlm (full-page vision-language OCR) is shown. Customise for your deployment.
    • Default Value: "vlm" will call the chosen vision model (VLM) to return a structured json output that is then parsed into word-level bounding boxes.
  • VLM_OCR_HYBRID_INTRO_TEXT
    • Description: UI intro text for hybrid-vlm (Tesseract first pass + VLM on selected lines/regions).
    • Default Value: "hybrid-vlm" is a combination of Tesseract for OCR, and a second pass with the chosen vision model (VLM).
  • REPORT_VLM_OUTPUTS_TO_GUI
    • Description: If set to "True", reports VLM outputs to the GUI with info boxes as they are processed.
    • Default Value: "False"
  • REPORT_LLM_OUTPUTS_TO_GUI
    • Description: If set to "True", reports streamed LLM outputs (local transformers or local inference-server PII detection) to the GUI with gr.Info as they are processed.
    • Default Value: "False"
  • CLOUD_VLM_MODEL_CHOICE
    • Description: Default model choice for cloud VLM OCR (AWS Bedrock, Gemini, or Azure/OpenAI). Used when cloud VLM options are enabled.
    • Default Value: "qwen.qwen3-vl-235b-a22b" (or first available cloud model if empty)
  • SHOW_CUSTOM_VLM_ENTITIES
    • Description: If set to "True", appends CUSTOM_VLM_FACES and CUSTOM_VLM_SIGNATURE to the redaction entity lists when at least one of SHOW_VLM_MODEL_OPTIONS, SHOW_INFERENCE_SERVER_VLM_OPTIONS, or SHOW_BEDROCK_VLM_MODELS is enabled—so users can redact detected faces and signatures from VLM-capable pipelines.
    • Default Value: "False"
  • CUSTOM_VLM_BACKEND
    • Description: Backend used for CUSTOM_VLM_FACES and CUSTOM_VLM_SIGNATURE (face and signature detection) when those entities are selected for redaction. One of: "transformers_vlm" (local transformers VLM), "inference_vlm" (inference-server VLM), or "bedrock_vlm" (AWS Bedrock VLM). Applies regardless of the PII identification method (Local, AWS Comprehend, or LLM). Value is case-insensitive; invalid values fall back to "bedrock_vlm".
    • Default Value: "bedrock_vlm"
  • CUSTOM_VLM_MIN_CONFIDENCE
    • Description: Minimum confidence required for VLM face/signature detections (CUSTOM_VLM_FACES / CUSTOM_VLM_SIGNATURE) to be kept for downstream outputs (annotations, decision tables, and PDF redaction). Values can be provided as 0..1 (e.g. 0.5) or 0..100 (e.g. 50).
    • Default Value: 0.5
  • CLOUD_LLM_PII_MODEL_CHOICE
    • Description: Default AWS Bedrock (or cloud) model for LLM-based PII detection when SHOW_AWS_BEDROCK_LLM_MODELS (or other cloud LLM options) is enabled. Used by tools/llm_entity_detection.py.
    • Default Value: "amazon.nova-pro-v1:0"
  • CLOUD_LLM_PII_CUSTOM_INSTRUCTIONS_MODEL_CHOICE
    • Description: Optional “upgraded” cloud model used for LLM-based PII detection when custom instructions are provided (e.g. allow-list or user-specific rules). If set and non-empty, it overrides CLOUD_LLM_PII_MODEL_CHOICE whenever custom_instructions are passed to the entity-detection LLM. Leave empty to always use CLOUD_LLM_PII_MODEL_CHOICE.
    • Default Value: '' (empty; no override)
  • CLOUD_SUMMARISATION_MODEL_CHOICE
    • Description: Default AWS Bedrock (or cloud) model for LLM-based summarisation when using cloud inference (e.g. AWS Bedrock). Used by tools/summaries.py. Separate from CLOUD_LLM_PII_MODEL_CHOICE so you can assign a different model for summarisation than for entity detection.
    • Default Value: "amazon.nova-pro-v1:0"
  • VLM_SEED
    • Description: Random seed for VLM generation. If empty, no seed is set (non-deterministic). If set to an integer, generation will be deterministic.
    • Default Value: '' (empty; no seed set)
  • VLM_DEFAULT_TEMPERATURE
    • Description: Default temperature for VLM generation. If empty, model-specific defaults are used.
    • Default Value: ''
  • VLM_DEFAULT_TOP_P
    • Description: Default top_p (nucleus sampling) for VLM generation. If empty, model-specific defaults are used.
    • Default Value: ''
  • VLM_DEFAULT_MIN_P
    • Description: Default min_p (minimum probability threshold) for VLM generation. If empty, model-specific defaults are used.
    • Default Value: ''
  • VLM_DEFAULT_TOP_K
    • Description: Default top_k for VLM generation. If empty, model-specific defaults are used.
    • Default Value: ''
  • VLM_DEFAULT_REPETITION_PENALTY
    • Description: Default repetition penalty for VLM generation. If empty, model-specific defaults are used.
    • Default Value: ''
  • VLM_DEFAULT_DO_SAMPLE
    • Description: Default do_sample setting for VLM generation. "True" means use sampling (do_sample=True), "False" means use greedy decoding (do_sample=False). If empty, model-specific defaults are used.
    • Default Value: ''
  • VLM_DEFAULT_PRESENCE_PENALTY
    • Description: Default presence penalty for VLM generation. If empty, model-specific defaults are used.
    • Default Value: ''

Entity and Search Options

  • CHOSEN_COMPREHEND_ENTITIES / FULL_COMPREHEND_ENTITY_LIST
    • Description: The selected and available PII entity types for AWS Comprehend.
    • Default Value: Predefined lists of entities (see script).
  • CHOSEN_LLM_ENTITIES / FULL_LLM_ENTITY_LIST
    • Description: The selected and available PII entity types for LLM-based PII detection (e.g. AWS Bedrock, local transformers, inference server).
    • Default Value: Predefined lists (e.g. ['EMAIL_ADDRESS','ADDRESS','NAME','PHONE_NUMBER', 'CUSTOM'] for chosen; see script for full list).
  • CHOSEN_REDACT_ENTITIES / FULL_ENTITY_LIST
    • Description: The selected and available PII entity types for the local model.
    • Default Value: Predefined lists of entities (see script).
  • CUSTOM_ENTITIES
    • Description: A list of entities that are considered “custom” and may have special handling.
    • Default Value: ['TITLES', 'UKPOSTCODE', 'STREETNAME', 'CUSTOM']
  • DEFAULT_SEARCH_QUERY
    • Description: The default text for the custom search/redact input box.
    • Default Value: ''
  • DEFAULT_FUZZY_SPELLING_MISTAKES_NUM
    • Description: Default number of allowed spelling mistakes for fuzzy searches.
    • Default Value: 0
  • DEFAULT_PAGE_MIN / DEFAULT_PAGE_MAX
    • Description: Default start and end pages for processing. 0 for max means process all pages.
    • Default Value: 0 for both.

Textract Feature Selection

  • DEFAULT_HANDWRITE_SIGNATURE_CHECKBOX
    • Description: The default options selected for Textract’s handwriting and signature detection. Provide as a list string, e.g. "['Extract handwriting', 'Extract signatures']".
    • Default Value: []
  • HANDWRITE_SIGNATURE_TEXTBOX_FULL_OPTIONS
    • Description: Full list of available options for Textract’s handwriting and signature detection. Can include 'Extract handwriting', 'Extract signatures', and optionally 'Extract forms', 'Extract layout', 'Extract tables', and 'Face detection' if the corresponding include options are enabled.
    • Default Value: ['Extract handwriting', 'Extract signatures']
  • INCLUDE_FORM_EXTRACTION_TEXTRACT_OPTION
  • INCLUDE_LAYOUT_EXTRACTION_TEXTRACT_OPTION
  • INCLUDE_TABLE_EXTRACTION_TEXTRACT_OPTION
    • Description: Booleans ("True"/"False") to include Forms, Layout, and Tables as selectable options for Textract analysis.
    • Default Value: "False" for all.
  • INCLUDE_FACE_IDENTIFICATION_TEXTRACT_OPTION
    • Description: When set to "True", adds “Face detection” as a selectable option in the handwriting/signature checkbox group. When enabled with AWS Bedrock VLM OCR, a second VLM pass per page is run to identify faces (cost and time estimates account for this).
    • Default Value: "False"

Tabular Data Options

  • DO_INITIAL_TABULAR_DATA_CLEAN
    • Description: If "True", performs an initial cleaning step on tabular data.
    • Default Value: "True"
  • DEFAULT_TEXT_COLUMNS / DEFAULT_EXCEL_SHEETS
    • Description: Default values for specifying which columns or sheets to process in tabular files.
    • Default Value: [] (empty list)
  • DEFAULT_TABULAR_ANONYMISATION_STRATEGY
    • Description: The default method for anonymizing tabular data (e.g., “redact completely”).
    • Default Value: "redact completely"

Language Options

Settings for multi-language support.

  • SHOW_LANGUAGE_SELECTION
    • Description: If set to "True", a language selection dropdown will be visible in the UI.
    • Default Value: "False"
  • DEFAULT_LANGUAGE_FULL_NAME / DEFAULT_LANGUAGE
    • Description: The default language’s full name (e.g., “english”) and its short code (e.g., “en”).
    • Default Values: "english", "en"
    • Language Support Notes:
      • Tesseract: Ensure the Tesseract language data (e.g., fra.traineddata) is installed. Find language packs at tesseract-ocr/tessdata.
      • PaddleOCR: Ensure the PaddleOCR language data is installed. See supported languages at PaddleOCR documentation.
      • AWS Comprehend: Only English (en) and Spanish (es) are supported. See AWS Comprehend PII documentation.
      • AWS Textract: Automatically detects language and supports English, Spanish, Italian, Portuguese, French, and German. Note that handwriting, invoices, receipts, identity documents, and queries processing are in English only. See AWS Textract FAQs.
  • textract_language_choices / aws_comprehend_language_choices
    • Description: Lists of supported language codes for Textract and Comprehend.
    • Default Value: ['en', 'es', 'fr', 'de', 'it', 'pt'] and ['en', 'es']
  • SPLIT_PUNCTUATION_FROM_WORDS
    • Description: If set to "True", splits punctuation from words in Textract output.
    • Default Value: "False"
  • MAPPED_LANGUAGE_CHOICES / LANGUAGE_CHOICES
    • Description: Paired lists of full language names and their corresponding short codes for the UI dropdown.
    • Default Value: Predefined lists (see script).

Duplicate Detection Settings

  • DEFAULT_DUPLICATE_DETECTION_THRESHOLD
    • Description: The similarity score (0.0 to 1.0) above which documents/pages are considered duplicates.
    • Default Value: 0.95
  • DEFAULT_MIN_CONSECUTIVE_PAGES
    • Description: Minimum number of consecutive pages that must be duplicates to be flagged.
    • Default Value: 1
  • USE_GREEDY_DUPLICATE_DETECTION
    • Description: If "True", uses a greedy algorithm that may find more duplicates but can be less precise.
    • Default Value: "True"
  • DEFAULT_COMBINE_PAGES
    • Description: If "True", text from the same page number across different files is combined before checking for duplicates. If "False", line-level duplicate detection will be enabled instead.
    • Default Value: "True"
  • DEFAULT_MIN_WORD_COUNT
    • Description: Pages with fewer words than this value will be ignored by the duplicate detector.
    • Default Value: 10
  • REMOVE_DUPLICATE_ROWS
    • Description: If "True", enables duplicate row detection in tabular data.
    • Default Value: "False"

File Output Options

  • USE_GUI_BOX_COLOURS_FOR_OUTPUTS
    • Description: If "True", the final redacted PDF will use the same redaction box colors as shown in the review UI.
    • Default Value: "False"
  • CUSTOM_BOX_COLOUR
    • Description: Specifies the color for redaction boxes as an RGB tuple string, e.g., "(0, 0, 0)" for black. Alternatively, you can use the named color "grey" (which maps to RGB (128, 128, 128)).
    • Default Value: "(0, 0, 0)"
  • APPLY_REDACTIONS_IMAGES, APPLY_REDACTIONS_GRAPHICS, APPLY_REDACTIONS_TEXT
    • Description: Advanced control over how redactions are applied to underlying images, vector graphics, and text in the PDF, based on PyMuPDF options. 0 is the default for a standard redaction workflow.
    • Default Value: 0 for all.
    • Detailed Options:
      • APPLY_REDACTIONS_IMAGES: 0 = ignore (default), 1 = completely remove images overlapping redaction annotations, 2 = blank out overlapping pixels, 3 = only remove images that are actually visible. Note: Text in images is effectively removed by the overlapping rectangle shape.
      • APPLY_REDACTIONS_GRAPHICS: 0 = ignore (default), 1 = remove graphics fully contained in redaction annotation, 2 = remove any overlapping vector graphics.
      • APPLY_REDACTIONS_TEXT: 0 = remove all characters whose boundary box overlaps any redaction rectangle (default, complies with legal/data protection intentions), 1 = keep text while redacting graphics/images (does NOT comply with data protection intentions - use at your own risk).
  • RETURN_PDF_FOR_REVIEW
    • Description: If set to "True", a PDF with redaction boxes drawn on it (but text not removed) is generated for the “Review” tab.
    • Default Value: "True"
  • RETURN_REDACTED_PDF
    • Description: If set to 'True', the application will return a fully redacted PDF at the end of the main task.
    • Default Value: "True"
  • COMPRESS_REDACTED_PDF
    • Description: If set to 'True', the redacted PDF output will be compressed.
    • Default Value: "False"
    • Warning: On low memory systems, the compression options in PyMuPDF can cause the app to crash if the PDF is longer than 500 pages or so. Setting this to "False" will save the PDF only with a basic cleaning option enabled, which is more memory-efficient.
  • SAVE_OUTPUTS_TO_S3
    • Description: If set to 'True', the application will automatically upload redaction outputs (PDFs, text/tabular outputs, duplicate-analysis files, and Adobe XFDF review files) to Amazon S3 when RUN_AWS_FUNCTIONS is also enabled. Uploads use the S3_OUTPUTS_FOLDER prefix within the DOCUMENT_REDACTION_BUCKET, optionally with a per-session subfolder when SESSION_OUTPUT_FOLDER is enabled.
    • Default Value: 'False'
  • S3_OUTPUTS_FOLDER
    • Description: Base S3 key prefix (folder path) within DOCUMENT_REDACTION_BUCKET where redaction outputs are stored, for example 'outputs/' or 'redaction/outputs/'. When SESSION_OUTPUT_FOLDER is 'True', a session-specific subfolder (based on session_hash) is appended to this path so each user/session writes to its own S3 subdirectory. If left blank, outputs will not be uploaded to S3 even if SAVE_OUTPUTS_TO_S3 is 'True'.
    • Default Value: ''
  • S3_OUTPUTS_BUCKET
    • Description: Name of the S3 bucket where redaction outputs are stored.
    • Default Value: 'DOCUMENT_REDACTION_BUCKET'

Direct Mode & Lambda Configuration

Settings for running the application from the command line (Direct Mode) or as an AWS Lambda function.

Note: Many DIRECT_MODE_* variables inherit their default values from their corresponding non-direct-mode variables if not explicitly set. For example, DIRECT_MODE_LANGUAGE defaults to DEFAULT_LANGUAGE, DIRECT_MODE_IMAGES_DPI defaults to IMAGES_DPI, etc.

Direct Mode

  • RUN_DIRECT_MODE
    • Description: Set to 'True' to enable direct command-line mode.
    • Default Value: 'False'
  • DIRECT_MODE_DEFAULT_USER
    • Description: Default username for CLI requests.
    • Default Value: ''
  • DIRECT_MODE_TASK
    • Description: The task to perform: 'redact' or 'deduplicate'.
    • Default Value: 'redact'
  • DIRECT_MODE_INPUT_FILE / DIRECT_MODE_OUTPUT_DIR
    • Description: Path to the input file and output directory for the task.
    • Default Values: '', output/
  • DIRECT_MODE_DUPLICATE_TYPE
    • Description: Type of duplicate detection for direct mode: 'pages' or 'tabular'.
    • Default Value: 'pages'
  • DIRECT_MODE_LANGUAGE
    • Description: Language for document processing in direct mode.
    • Default Value: Inherits from DEFAULT_LANGUAGE
  • DIRECT_MODE_PII_DETECTOR
    • Description: PII detection method for direct mode.
    • Default Value: Inherits from LOCAL_PII_OPTION
  • DIRECT_MODE_OCR_METHOD
    • Description: OCR method for PDF/image processing in direct mode.
    • Default Value: "Local OCR"
  • DIRECT_MODE_OCR_FIRST_PASS_MAX_WORKERS
    • Description: Maximum number of threads for the OCR first pass in direct mode. Inherits from OCR_FIRST_PASS_MAX_WORKERS if not set.
    • Default Value: Same as OCR_FIRST_PASS_MAX_WORKERS (e.g. 3)
  • DIRECT_MODE_SUMMARY_PAGE_GROUP_MAX_WORKERS
    • Description: Maximum number of threads for page-group summarisation in direct mode (when task is summarise). Inherits from SUMMARY_PAGE_GROUP_MAX_WORKERS if not set.
    • Default Value: Same as SUMMARY_PAGE_GROUP_MAX_WORKERS (e.g. 1)
  • DIRECT_MODE_PAGE_MIN / DIRECT_MODE_PAGE_MAX
    • Description: First and last page to process in direct mode. 0 for max means process all pages.
    • Default Values: Inherit from DEFAULT_PAGE_MIN / DEFAULT_PAGE_MAX
  • DIRECT_MODE_IMAGES_DPI
    • Description: DPI for image processing in direct mode.
    • Default Value: Inherits from IMAGES_DPI
  • DIRECT_MODE_CHOSEN_LOCAL_OCR_MODEL
    • Description: Local OCR model choice for direct mode.
    • Default Value: Inherits from DEFAULT_LOCAL_OCR_MODEL
  • DIRECT_MODE_PREPROCESS_LOCAL_OCR_IMAGES
    • Description: If set to "True", preprocesses images before OCR in direct mode.
    • Default Value: Inherits from PREPROCESS_LOCAL_OCR_IMAGES
  • DIRECT_MODE_COMPRESS_REDACTED_PDF
    • Description: If set to "True", compresses the redacted PDF output in direct mode.
    • Default Value: Inherits from COMPRESS_REDACTED_PDF
  • DIRECT_MODE_RETURN_PDF_END_OF_REDACTION
    • Description: If set to "True", returns a PDF at the end of redaction in direct mode.
    • Default Value: Inherits from RETURN_REDACTED_PDF
  • DIRECT_MODE_EXTRACT_FORMS
    • Description: If set to "True", extracts forms during Textract analysis in direct mode.
    • Default Value: "False"
  • DIRECT_MODE_EXTRACT_TABLES
    • Description: If set to "True", extracts tables during Textract analysis in direct mode.
    • Default Value: "False"
  • DIRECT_MODE_EXTRACT_LAYOUT
    • Description: If set to "True", extracts layout during Textract analysis in direct mode.
    • Default Value: "False"
  • DIRECT_MODE_EXTRACT_SIGNATURES
    • Description: If set to "True", extracts signatures during Textract analysis in direct mode.
    • Default Value: "False"
  • DIRECT_MODE_MATCH_FUZZY_WHOLE_PHRASE_BOOL
    • Description: If set to "True", matches fuzzy whole phrases in direct mode.
    • Default Value: "True"
  • DIRECT_MODE_ANON_STRATEGY
    • Description: Anonymisation strategy for tabular data in direct mode.
    • Default Value: Inherits from DEFAULT_TABULAR_ANONYMISATION_STRATEGY
  • DIRECT_MODE_FUZZY_MISTAKES
    • Description: Number of fuzzy spelling mistakes allowed in direct mode.
    • Default Value: Inherits from DEFAULT_FUZZY_SPELLING_MISTAKES_NUM
  • DIRECT_MODE_SIMILARITY_THRESHOLD
    • Description: Similarity threshold for duplicate detection in direct mode.
    • Default Value: Inherits from DEFAULT_DUPLICATE_DETECTION_THRESHOLD
  • DIRECT_MODE_MIN_WORD_COUNT
    • Description: Minimum word count for duplicate detection in direct mode.
    • Default Value: Inherits from DEFAULT_MIN_WORD_COUNT
  • DIRECT_MODE_MIN_CONSECUTIVE_PAGES
    • Description: Minimum consecutive pages for duplicate detection in direct mode.
    • Default Value: Inherits from DEFAULT_MIN_CONSECUTIVE_PAGES
  • DIRECT_MODE_GREEDY_MATCH
    • Description: If set to "True", uses greedy matching for duplicate detection in direct mode.
    • Default Value: Inherits from USE_GREEDY_DUPLICATE_DETECTION
  • DIRECT_MODE_COMBINE_PAGES
    • Description: If set to "True", combines pages for duplicate detection in direct mode.
    • Default Value: Inherits from DEFAULT_COMBINE_PAGES
  • DIRECT_MODE_REMOVE_DUPLICATE_ROWS
    • Description: If set to "True", removes duplicate rows in tabular data in direct mode.
    • Default Value: Inherits from REMOVE_DUPLICATE_ROWS
  • DIRECT_MODE_TEXTRACT_ACTION
    • Description: Textract action for batch operations in direct mode.
    • Default Value: ''
  • DIRECT_MODE_JOB_ID
    • Description: Job ID for Textract operations in direct mode.
    • Default Value: ''
  • DIRECT_MODE_INFERENCE_SERVER_MODEL
    • Description: Inference server model to use for PII detection in direct mode. If empty, uses CHOSEN_INFERENCE_SERVER_PII_MODEL.
    • Default Value: '' (inherits from CHOSEN_INFERENCE_SERVER_PII_MODEL when set)

Lambda Configuration

  • LAMBDA_POLL_INTERVAL
    • Description: Polling interval in seconds for checking Textract job status.
    • Default Value: 30
  • LAMBDA_MAX_POLL_ATTEMPTS
    • Description: Maximum number of polling attempts before timeout.
    • Default Value: 120
  • LAMBDA_PREPARE_IMAGES
    • Description: If "True", prepares images for OCR processing within the Lambda environment.
    • Default Value: "True"
  • LAMBDA_EXTRACT_SIGNATURES
    • Description: Enables signature extraction during Textract analysis in Lambda.
    • Default Value: "False"
  • LAMBDA_DEFAULT_USERNAME
    • Description: Default username for operations initiated by Lambda.
    • Default Value: "lambda_user"

Allow, Deny, & Whole Page Redaction Lists

  • GET_DEFAULT_ALLOW_LIST, GET_DEFAULT_DENY_LIST, GET_DEFAULT_WHOLE_PAGE_REDACTION_LIST
    • Description: Booleans ("True"/"False") to enable the use of allow, deny, or whole-page redaction lists.
    • Default Value: "False"
    • Note: GET_DEFAULT_WHOLE_PAGE_REDACTION_LIST is stored as a string value, not converted to boolean (unlike the other two variables).
  • ALLOW_LIST_PATH, DENY_LIST_PATH, WHOLE_PAGE_REDACTION_LIST_PATH
    • Description: Local paths to the respective CSV list files.
    • Default Value: ''
  • S3_ALLOW_LIST_PATH, S3_DENY_LIST_PATH, S3_WHOLE_PAGE_REDACTION_LIST_PATH
    • Description: Paths to the respective list files within the DOCUMENT_REDACTION_BUCKET.
    • Default Value: ''

Cost Code Options

  • SHOW_COSTS
    • Description: If set to 'True', cost-related information will be displayed in the UI.
    • Default Value: 'False'
  • GET_COST_CODES
    • Description: Enables fetching and using cost codes. Set to 'True' to enable.
    • Default Value: 'False'
  • DEFAULT_COST_CODE
    • Description: Specifies a default cost code.
    • Default Value: ''
  • COST_CODES_PATH / S3_COST_CODES_PATH
    • Description: Local or S3 path to a CSV file containing available cost codes.
    • Default Value: ''
    • File Format: The CSV file should contain a single table with two columns and a header row. The first column should contain cost codes, and the second column should contain a name or description for each cost code.
  • ENFORCE_COST_CODES
    • Description: If set to 'True', makes the selection of a cost code mandatory.
    • Default Value: 'False'

Session default cost code (no extra environment variables)

When cost codes are enabled (GET_COST_CODES or ENFORCE_COST_CODES), users can save a default cost code for their session via the “Set default cost code” button. This uses the existing cost code and AWS settings; no additional environment variables are required.

  • Behaviour:
    • The chosen cost code is stored in a CSV file (session_default_cost_codes.csv) with columns session_hash, default_cost_code, and saved_at (ISO timestamp). Duplicate session_hash rows are deduplicated on save, keeping only the latest.
    • Local save location: The CSV is written under the app’s input folder (the value of the input folder at save time, e.g. from INPUT_FOLDER or the session’s input_folder_textbox).
    • S3 (optional): If S3_COST_CODES_PATH is set and RUN_AWS_FUNCTIONS and DOCUMENT_REDACTION_BUCKET are set, the same file is also uploaded to DOCUMENT_REDACTION_BUCKET in the same key “folder” as the cost codes file (e.g. config/session_default_cost_codes.csv).
    • Email-only: Saves and S3 upload only run when the session identifier is a valid email address (e.g. when users are signed in via Cognito). If the session identifier is not an email (e.g. when running locally with a Gradio session hash), the default is not written to the CSV and not uploaded to S3. On load, the app does not download the session default cost codes file from S3 when the session identifier is not an email.
  • Relevant existing variables: GET_COST_CODES, ENFORCE_COST_CODES, COST_CODES_PATH, S3_COST_CODES_PATH, DOCUMENT_REDACTION_BUCKET, RUN_AWS_FUNCTIONS, and the app input folder (e.g. INPUT_FOLDER / GRADIO_INPUT_FOLDER).

Whole Document API Options (Textract Async)

  • SHOW_WHOLE_DOCUMENT_TEXTRACT_CALL_OPTIONS
    • Description: Controls whether UI options for asynchronous whole document Textract calls are displayed.
    • Default Value: 'False'
    • Note: This feature is not currently fully implemented in the application.
  • TEXTRACT_WHOLE_DOCUMENT_ANALYSIS_BUCKET
    • Description: The S3 bucket used for asynchronous Textract analysis.
    • Default Value: ''
  • TEXTRACT_WHOLE_DOCUMENT_ANALYSIS_INPUT_SUBFOLDER / ..._OUTPUT_SUBFOLDER
    • Description: Input and output subfolders within the analysis bucket.
    • Default Values: 'input', 'output'
  • LOAD_PREVIOUS_TEXTRACT_JOBS_S3
    • Description: If set to 'True', the application will load data from previous Textract jobs stored in S3.
    • Default Value: 'False'
  • TEXTRACT_JOBS_S3_LOC / TEXTRACT_JOBS_S3_INPUT_LOC
    • Description: S3 subfolders where Textract job output and input are stored.
    • Default Value: 'output', 'input'
  • TEXTRACT_JOBS_LOCAL_LOC
    • Description: The local subfolder for storing Textract job data.
    • Default Value: 'output'
  • DAYS_TO_DISPLAY_WHOLE_DOCUMENT_JOBS
    • Description: Specifies the number of past days for which to display whole document Textract jobs.
    • Default Value: 7
Back to top