Codecs

Codecs are used to encapsulate the logic required to encode / decode payloads following the Open Inference Protocol into high-level Python types. You can read more about the high-level concepts behind codecs in the Content Types (and Codecs) section of the docs, as well as how to use them.

Base Codecs

All the codecs within MLServer extend from either the InputCodec or the RequestCodec base classes. These define the interface to deal with input (outputs) and request (responses) respectively.

class mlserver.codecs.InputCodec

The InputCodec interface lets you define type conversions of your raw input data to / from the Open Inference Protocol. Note that this codec applies at the individual input (output) level.

For request-wide transformations (e.g. dataframes), use the RequestCodec interface instead.

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_input(request_input: RequestInput) → Any

Decode a request input into a high-level Python type.

classmethod decode_output(response_output: ResponseOutput) → Any

Decode a response output into a high-level Python type.

classmethod encode_input(name: str, payload: Any, **kwargs) → RequestInput

Encode the given payload into a RequestInput.

classmethod encode_output(name: str, payload: Any, **kwargs) → ResponseOutput

Encode the given payload into a response output.

class mlserver.codecs.RequestCodec

The RequestCodec interface lets you define request-level conversions between high-level Python types and the Open Inference Protocol. This can be useful where the encoding of your payload encompases multiple input heads (e.g. dataframes, where each column can be thought as a separate input head).

For individual input-level encoding / decoding, use the InputCodec interface instead.

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_request(request: InferenceRequest) → Any

Decode an inference request into a high-level Python object.

classmethod decode_response(response: InferenceResponse) → Any

Decode an inference response into a high-level Python object.

classmethod encode_request(payload: Any, **kwargs) → InferenceRequest

Encode the given payload into an inference request.

classmethod encode_response(model_name: str, payload: Any, model_version: str | None = None, **kwargs) → InferenceResponse

Encode the given payload into an inference response.

Built-in Codecs

The mlserver package will include a set of built-in codecs to cover common conversions. You can learn more about these in the Available Content Types section of the docs.

class mlserver.codecs.Base64Codec

Codec that convers to / from a base64 input.

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_input(request_input: RequestInput) → List[bytes]

Decode a request input into a high-level Python type.

classmethod decode_output(response_output: ResponseOutput) → List[bytes]

Decode a response output into a high-level Python type.

classmethod encode_input(name: str, payload: List[bytes], use_bytes: bool = True, **kwargs) → RequestInput

Encode the given payload into a RequestInput.

classmethod encode_output(name: str, payload: List[bytes], use_bytes: bool = True, **kwargs) → ResponseOutput

Encode the given payload into a response output.

class mlserver.codecs.DatetimeCodec

Codec that convers to / from a datetime input.

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_input(request_input: RequestInput) → List[datetime]

Decode a request input into a high-level Python type.

classmethod decode_output(response_output: ResponseOutput) → List[datetime]

Decode a response output into a high-level Python type.

classmethod encode_input(name: str, payload: List[str | datetime], use_bytes: bool = True, **kwargs) → RequestInput

Encode the given payload into a RequestInput.

classmethod encode_output(name: str, payload: List[str | datetime], use_bytes: bool = True, **kwargs) → ResponseOutput

Encode the given payload into a response output.

class mlserver.codecs.NumpyCodec

Decodes an request input (response output) as a NumPy array.

TypeHint

alias of ndarray

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_input(request_input: RequestInput) → ndarray

Decode a request input into a high-level Python type.

classmethod decode_output(response_output: ResponseOutput) → ndarray

Decode a response output into a high-level Python type.

classmethod encode_input(name: str, payload: ndarray, **kwargs) → RequestInput

Encode the given payload into a RequestInput.

classmethod encode_output(name: str, payload: ndarray, **kwargs) → ResponseOutput

Encode the given payload into a response output.

class mlserver.codecs.NumpyRequestCodec

Decodes the first input (output) of request (response) as a NumPy array. This codec can be useful for cases where the whole payload is a single NumPy tensor.

InputCodec

alias of NumpyCodec

class mlserver.codecs.PandasCodec

Decodes a request (response) into a Pandas DataFrame, assuming each input (output) head corresponds to a column of the DataFrame.

TypeHint

alias of DataFrame

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_request(request: InferenceRequest) → DataFrame

Decode an inference request into a high-level Python object.

classmethod decode_response(response: InferenceResponse) → DataFrame

Decode an inference response into a high-level Python object.

classmethod encode_request(payload: DataFrame, use_bytes: bool = True, **kwargs) → InferenceRequest

Encode the given payload into an inference request.

classmethod encode_response(model_name: str, payload: DataFrame, model_version: str | None = None, use_bytes: bool = True, **kwargs) → InferenceResponse

Encode the given payload into an inference response.

class mlserver.codecs.StringCodec

Encodes a list of Python strings as a BYTES input (output).

classmethod can_encode(payload: Any) → bool

Evaluate whether the codec can encode (decode) the payload.

classmethod decode_input(request_input: RequestInput) → List[str]

Decode a request input into a high-level Python type.

classmethod decode_output(response_output: ResponseOutput) → List[str]

Decode a response output into a high-level Python type.

classmethod encode_input(name: str, payload: List[str], use_bytes: bool = True, **kwargs) → RequestInput

Encode the given payload into a RequestInput.

classmethod encode_output(name: str, payload: List[str], use_bytes: bool = True, **kwargs) → ResponseOutput

Encode the given payload into a response output.

class mlserver.codecs.StringRequestCodec

Decodes the first input (output) of request (response) as a list of strings. This codec can be useful for cases where the whole payload is a single list of strings.

InputCodec

alias of StringCodec