Private Ai Model Hosting: When Bedrock Alone Cannot Meet Constraints

1. Need weights you train/own that Bedrock Custom Model Import cannot serve?
   → SageMaker (or self-managed GPU) with your artifact + CMK
2. Data residency / sovereign region without Bedrock model access?
   → Private hosting in that region; deny cross-region invokes via SCP
3. Hard p95 < Bedrock floor for your token shape (measure, don't guess)?
   → SageMaker real-time + provisioned concurrency / Triton
4. Otherwise
   → Bedrock (+ Prompt Management, Guardrails, inference profiles)

# policy/hosting_choice.py
from dataclasses import dataclass

@dataclass
class Constraints:
    custom_weights: bool
    residency_ok_on_bedrock: bool
    p95_ms_budget: int
    bedrock_p95_ms: int  # measured on canary

def choose(c: Constraints) -> str:
    if c.custom_weights:
        return "sagemaker_realtime"
    if not c.residency_ok_on_bedrock:
        return "sagemaker_or_onprem"
    if c.p95_ms_budget < c.bedrock_p95_ms:
        return "sagemaker_realtime"
    return "bedrock"  # ✅ default

# infra/sagemaker-endpoint.yaml (conceptual)
EndpointConfig:
  ProductionVariants:
    - ModelName: coding-assistant-v3
      InitialInstanceCount: 2
      InstanceType: ml.g5.xlarge
VpcConfig:
  Subnets: [subnet-private-a, subnet-private-b]
  SecurityGroupIds: [sg-model-only]
# ✅ No public internet; PrivateLink to caller VPC
# ✅ Invoke via IAM authz, not long-lived API keys