MLServer Settings

MLServer can be configured through a settings.json file on the root folder from where MLServer is started. Note that these are server-wide settings (e.g. gRPC or HTTP port) which are separate from the invidual model settings <./model-settings>. Alternatively, this configuration can also be passed through environment variables prefixed with MLSERVER_ (e.g. MLSERVER_GRPC_PORT).


pydantic settings mlserver.settings.Settings
  • env_prefix: str = MLSERVER_

field debug: bool = True

Root of the model repository, where we will search for models.

field extensions: List[str] = []

Host where to listen for connections.

field grpc_max_message_length: Optional[int] = None
field grpc_port: int = 8081

Maximum length (i.e. size) of gRPC payloads.

field host: str = ''

Port where to listen for HTTP / REST connections.

field http_port: int = 8080

Port where to listen for gRPC connections.

field load_models_at_startup: bool = True

Name of the server.

field model_repository_root: str = '.'

Flag to load all available models automatically at startup.

field server_name: str = 'mlserver'

Version of the server.

field server_version: str = '0.6.0.dev0'

Server extensions loaded.