Skip to main content
This workflow requires network access to Permitech APIs. If you get an access denied error or cannot reach the Permitech endpoints, talk to your DevOps or Permitech administrator before proceeding.

​
Prerequisites

Set these environment variables:
  • PERMITECH_API_URL
  • PERMITECH_API_KEY

​
Steps to register a metric in Permitech

There are two steps to register a metric in Permitech:

​
1. Upload LoRA weights

Use the SDK helper to upload your fine-tuned LoRA artifacts to Permitech via the Permitech API:
upload_finetuned_model_weights(...) reads PERMITECH_API_URL and PERMITECH_API_KEY from the environment, requests upload URLs from Permitech, uploads the required LoRA files from lora_files_path, and returns both the resolved lora_task_id and the Permitech-side lora_weights_path. If you do not provide lora_task_id, the SDK automatically picks the first free task id at or above lora_task_id_min. The directory passed in lora_files_path should contain:
  • model.lora_weights.npy
  • model.lora_config.npy
  • adapter_config.json
  • adapter_model.safetensors

​
2. Register the metric in Permitech

Once your weights are uploaded and you have the returned weights_path, register (or reuse) a scorer and create a new Luna scorer version using the SDK:

​
How to set the upload arguments

  • lora_task_id [optional]: provide this only if you want to force a specific LoRA task id. If omitted, the SDK auto-selects one for you.
  • lora_task_id_min [optional]: minimum LoRA task id to consider when auto-picking. In most cases, leave the default.
  • lora_files_path: the local directory that contains the LoRA artifact files produced by training.
  • timeout_seconds [optional]: request timeout used for the prepare-upload call and the file uploads. In most cases, the default is fine.

​
How to set the registration arguments

  • metric_name: the name of the scorer in Permitech. Use a stable, human-readable metric name.
  • prompt_template: the prompt template used for the metric. Make sure to use the same template shape you trained with. It is also available in the output artifact of the training run.
  • luna_input_type: the Permitech Luna input type enum that matches the metric input shape. Available options are: span, trace_object, trace_input_output_only.
  • luna_output_type: the Permitech Luna output type enum that matches the metric output shape. Available options are: float (for boolean metrics), string (for categorical metrics).
  • lora_task_id: the LoRA task id associated with the uploaded fine-tuned weights. Pass the lora_task_id returned by upload_finetuned_model_weights(...).
  • weights_path: the Permitech-side storage path returned by upload_finetuned_model_weights(...). Pass the lora_weights_path value from the upload response.

​
Use the registered metric

After registration, the metric becomes available inside Permitech (UI / API) as a Luna scorer.