Introduction
pyserde
is a simple yet powerful serialization library on top of dataclasses. It allows you to convert Python objects to and from JSON, YAML, and other formats easily and efficiently.
Declare your class with @serde
decorator and annotate fields using PEP484 as below.
from serde import serde
@serde
class Foo:
i: int
s: str
f: float
b: bool
You can serialize Foo
object into JSON.
>>> from serde.json import to_json
>>> to_json(Foo(i=10, s='foo', f=100.0, b=True))
'{"i":10,"s":"foo","f":100.0,"b":true}'
You can deserialize JSON into Foo
object.
>>> from serde.json import from_json
>>> from_json(Foo, '{"i": 10, "s": "foo", "f": 100.0, "b": true}')
Foo(i=10, s='foo', f=100.0, b=True)
Next Steps
Getting started
Installation
Install pyserde from PyPI. pyserde requires Python>=3.9.
pip install pyserde
If you're using poetry, run this command.
poetry add pyserde
Additional data formats besides JSON and Pickle need additional dependencies installed. Install msgpack
, toml
or yaml
extras to work with the appropriate data formats; you can skip formats that you don't plan to use. For example, if you want to use Toml and YAML:
pip install "pyserde[toml,yaml]"
With poetry
poetry add pyserde -E toml -E yaml
Or all at once:
pip install "pyserde[all]"
With poetry
poetry add pyserde -E all
Here are the available extras
all
: Installmsgpack
,toml
,yaml
,numpy
,orjson
, andsqlalchemy
extrasmsgpack
: Install msgpacktoml
: Install tomli and tomli-w- NOTE: tomllib is used for python 3.11 onwards
yaml
: Install pyyamlnumpy
: Install numpyorjson
: Install orjsonsqlalchemy
: Install sqlalchemy
Define your first pyserde class
Define your class with pyserde's @serde
decorators. Be careful that module name is serde
, not pyserde
. pyserde
heavily depends on the standard library's dataclasses
module. If you are new to dataclass, I would recommend to read dataclasses documentation first.
from serde import serde
@serde
class Foo:
i: int
s: str
f: float
b: bool
pyserde generates methods necessary for (de)serialization by @serde
when a class is loaded into python interpreter. The code generation occurs only once and there is no overhead when you use the class. Now your class is serializable and deserializable in all the data formats supported by pyserde.
NOTE: If you need only either serialization or deserialization functionality, you can use
@serialize
or@deserialize
instead of@serde
decorator.e.g. If you do only serialization, you can use
@serialize
decorator. But calling deserialize API e.g.from_json
forFoo
will raise an error.from serde import serialize @serialize class Foo: i: int s: str f: float b: bool
PEP585 and PEP604
PEP585 style annotation is supported for python>=3.9. PEP604 Union operator is also supported for python>=3.10. With PEP585 and PEP604, you can write a pyserde class pretty neatly.
@serde
class Foo:
a: int
b: list[str]
c: tuple[int, float, str, bool]
d: dict[str, list[tuple[str, int]]]
e: str | None
Use pyserde class
Next, import pyserde (de)serialize APIs. For JSON:
from serde.json import from_json, to_json
Use to_json
to serialize the object into JSON.
f = Foo(i=10, s='foo', f=100.0, b=True)
print(to_json(f))
Pass Foo
class and JSON string in from_json
to deserialize JSON into the object.
s = '{"i": 10, "s": "foo", "f": 100.0, "b": true}'
print(from_json(Foo, s))
That's it! pyserde offers many more features. If you're interested, please read the rest of the documentation.
💡 Tip: which type checker should I use? pyserde depends on PEP681 dataclass_transform. mypy does not fully support dataclass_transform as of Jan. 2024. My personal recommendation is pyright.
Data Formats
pyserde supports several data formats for serialization and deserialization, including dict
, tuple
, JSON
, YAML
, TOML
, MsgPack
, and Pickle
. Each API can take additional keyword arguments, which are forwarded to the underlying packages used by pyserde.
e.g. If you want to preserve the field order in YAML, you can pass sort_key=True
in serde.yaml.to_yaml
serde.yaml.to_yaml(foo, sort_key=True)
sort_key=True
will be passed in the yaml.safedump
dict
>>> from serde import to_dict, from_dict
>>> to_dict(Foo(i=10, s='foo', f=100.0, b=True))
from_dict(Foo, {"i": 10, "s": "foo", "f": 100.0, "b": True})
>>> from_dict(Foo, {"i": 10, "s": "foo", "f": 100.0, "b": True})
Foo(i=10, s='foo', f=100.0, b=True)
See serde.to_dict / serde.from_dict for more information.
tuple
>>> from serde import to_tuple, from_tuple
>>> to_tuple(Foo(i=10, s='foo', f=100.0, b=True))
(10, 'foo', 100.0, True)
>>> from_tuple(Foo, (10, 'foo', 100.0, True))
Foo(i=10, s='foo', f=100.0, b=True)
See serde.to_tuple / serde.from_tuple for more information.
JSON
>>> from serde.json import to_json, from_json
>>> to_json(Foo(i=10, s='foo', f=100.0, b=True))
'{"i":10,"s":"foo","f":100.0,"b":true}'
>>> from_json(Foo, '{"i": 10, "s": "foo", "f": 100.0, "b": true}')
Foo(i=10, s='foo', f=100.0, b=True)
See serde.json.to_json / serde.json.from_json for more information.
Yaml
>>> from serde.yaml import from_yaml, to_yaml
>>> to_yaml(Foo(i=10, s='foo', f=100.0, b=True))
b: true
f: 100.0
i: 10
s: foo
>>> from_yaml(Foo, 'b: true\nf: 100.0\ni: 10\ns: foo')
Foo(i=10, s='foo', f=100.0, b=True)
See serde.yaml.to_yaml / serde.yaml.from_yaml for more information.
Toml
>>> from serde.toml import from_toml, to_toml
>>> to_toml(Foo(i=10, s='foo', f=100.0, b=True))
i = 10
s = "foo"
f = 100.0
b = true
>>> from_toml(Foo, 'i = 10\ns = "foo"\nf = 100.0\nb = true')
Foo(i=10, s='foo', f=100.0, b=True)
See serde.toml.to_toml / serde.toml.from_toml for more information.
MsgPack
>>> from serde.msgpack import from_msgpack, to_msgpack
>>> to_msgpack(Foo(i=10, s='foo', f=100.0, b=True))
b'\x84\xa1i\n\xa1s\xa3foo\xa1f\xcb@Y\x00\x00\x00\x00\x00\x00\xa1b\xc3'
>>> from_msgpack(Foo, b'\x84\xa1i\n\xa1s\xa3foo\xa1f\xcb@Y\x00\x00\x00\x00\x00\x00\xa1b\xc3')
Foo(i=10, s='foo', f=100.0, b=True)
See serde.msgpack.to_msgpack / serde.msgpack.from_msgpack for more information.
Pickle
New in v0.9.6
>>> from serde.pickle import from_pickle, to_pickle
>>> to_pickle(Foo(i=10, s='foo', f=100.0, b=True))
b"\x80\x04\x95'\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\x01i\x94K\n\x8c\x01s\x94\x8c\x03foo\x94\x8c\x01f\x94G@Y\x00\x00\x00\x00\x00\x00\x8c\x01b\x94\x88u."
>>> from_pickle(Foo, b"\x80\x04\x95'\x00\x00\x00\x00\x00\x00\x00}\x94(\x8c\x01i\x94K\n\x8c\x01s\x94\x8c\x03foo\x94\x8c\x01f\x94G@Y\x00\x00\x00\x00\x00\x00\x8c\x01b\x94\x88u.")
Foo(i=10, s='foo', f=100.0, b=True)
See serde.pickle.to_pickle / serde.pickle.from_pickle for more information.
Needs a new data format support?
We don't plan to supprot a data format such as XML to keep pyserde as simple as possible. If you need a new data format support, we recommend to create a separate python package. If the data format is interchangable to dict or tuple, implementing the serialize/desrialize API is not that difficult. See YAML's implementation to know how to implement.
Types
Here is the list of the supported types. See the simple example for each type in the footnotes
- Primitives (int, float, str, bool) 1
- Containers
list
,set
,tuple
,dict
2frozenset
, 3defaultdict
4
typing.Optional
5typing.Union
6 7 8- User defined class with
@dataclass
9 10 typing.NewType
for primitive types 11typing.Any
12typing.Literal
13typing.Generic
14typing.ClassVar
15typing.InitVar
16Enum
andIntEnum
17- Standard library
- PyPI library
numpy
types 23SQLAlchemy
Declarative Dataclass Mapping (experimental) 24
You can write pretty complex class like this:
@serde
class bar:
i: int
@serde
class Foo:
a: int
b: list[str]
c: tuple[int, float, str, bool]
d: dict[str, list[tuple[str, int]]]
e: str | None
f: Bar
Numpy
All of the above (de)serialization methods can transparently handle most numpy types with the "numpy" extras package.
import numpy as np
import numpy.typing as npt
@serde
class NPFoo:
i: np.int32
j: np.int64
f: np.float32
g: np.float64
h: np.bool_
u: np.ndarray
v: npt.NDArray
w: npt.NDArray[np.int32]
x: npt.NDArray[np.int64]
y: npt.NDArray[np.float32]
z: npt.NDArray[np.float64]
npfoo = NPFoo(
np.int32(1),
np.int64(2),
np.float32(3.0),
np.float64(4.0),
np.bool_(False),
np.array([1, 2]),
np.array([3, 4]),
np.array([np.int32(i) for i in [1, 2]]),
np.array([np.int64(i) for i in [3, 4]]),
np.array([np.float32(i) for i in [5.0, 6.0]]),
np.array([np.float64(i) for i in [7.0, 8.0]]),
)
>>> from serde.json import to_json, from_json
>>> to_json(npfoo)
'{"i": 1, "j": 2, "f": 3.0, "g": 4.0, "h": false, "u": [1, 2], "v": [3, 4], "w": [1, 2], "x": [3, 4], "y": [5.0, 6.0], "z": [7.0, 8.0]}'
>>> from_json(NPFoo, to_json(npfoo))
NPFoo(i=1, j=2, f=3.0, g=4.0, h=False, u=array([1, 2]), v=array([3, 4]), w=array([1, 2], dtype=int32), x=array([3, 4]), y=array([5., 6.], dtype=float32), z=array([7., 8.]))
SQLAlchemy Declarative Dataclass Mapping (experimental)
While experimental support for SQLAlchemy Declarative Dataclass Mapping integration has been added, certain features such as @serde(type_check=strict)
and serde.field()
are not currently supported.
It's recommended to exercise caution when relying on experimental features in production environments. It's also advisable to thoroughly test your code and be aware of potential limitations or unexpected behavior.
Needs a new type support?
If you need to use a type which is currently not supported in the standard library, please creat a Github issue to request. For types in third party python packages, unless it's polular like numpy, we don't plan to support it to keep pyserde as simple as possible. We recommend to use custom class or field serializer.
See examples/any.py
Decorators
@serde
@serde
is a wrapper of @serialize
and @deserialize
decorators.
This code
@serde
class Foo:
...
is equivalent to the following code.
@deserialize
@serialize
@dataclass
class Foo:
...
@serde
decorator does these for you:
- Add
@serialize
and@deserialize
decorators to a class - Add
@dataclass
decorator to a class if a class doesn't have@dataclass
- You can pass both (de)serialize attributes to the decorator
serializer
attribute is ignored in@deserialize
anddeserializer
attribute is ignored in@serialize
@serde(serializer=serializer, deserializer=deserializer)
@dataclass
class Foo:
...
NOTE:
@serde
actually works without @dataclass decorator, because it detects and add @dataclass to the declared class automatically. However, mypy will produceToo many arguments
orUnexpected keyword argument
error. This is due to the mypy limitation.@serde class Foo: ...
But if you use a PEP681 compliant type checker (e.g. pyright), you don't get the type error because pyserde supports PEP681 dataclass_transform
@serialize
/@deserialize
@serialize
and @deserialize
are used by @serde
under the hood. We recommend to use those two decorators in the following cases. Otherwise @serde
is recommended.
- You only need either serialization or deserialization functionality
- You want to have different class attributes as below
@deserialize(rename_all = "snakecase")
@serialize(rename_all = "camelcase")
class Foo:
...
(de)serializing class without @serde
pyserde can (de)serialize dataclasses without @serde
since v0.10.0. This feature is convenient when you want to use classes declared in external libraries or a type checker doesn't work with @serde
decorator. See this example.
How does it work? It's quite simple that pyserde add @serde
decorator if a class doesn't has it. It may take for a while for the first API call, but the generated code will be cached internally. So it won't be a problem. See the following example to deserialize a class in a third party package.
@dataclass
class External:
...
to_dict(External()) # works without @serde
Note that you can't specify class attributes e.g. rename_all
in this case. If you want to add a class attribute to an external dataclass, there is a technique to do that by extending dataclass. See this example.
@dataclass
class External:
some_value: int
@serde(rename_all="kebabcase")
@dataclass
class Wrapper(External):
pass
How can I use forward references?
pyserde supports forward references. If you replace a nested class name with with string, pyserde looks up and evaluate the decorator after nested class is defined.
from __future__ import annotations # make sure to import annotations
@dataclass
class Foo:
i: int
s: str
bar: Bar # Bar can be specified although it's declared afterward.
@serde
@dataclass
class Bar:
f: float
b: bool
# Evaluate pyserde decorators after `Bar` is defined.
serde(Foo)
PEP563 Postponed Evaluation of Annotations
pyserde supports PEP563 Postponed evaluation of annotation.
from __future__ import annotations
from serde import serde
@serde
class Foo:
i: int
s: str
f: float
b: bool
def foo(self, cls: Foo): # You can use "Foo" type before it's defined.
print('foo')
See examples/lazy_type_evaluation.py for complete example.
Class Attributes
Class attributes can be specified as arguments in the serialize
/deserialize
decorators in order to customize the (de)serialization behaviour of the class entirely. If you want to customize a field, please consider using Field Attributes.
Attributes offered by dataclasses
frozen
dataclass frozen
class attribute works as expected.
kw_only
New in v0.12.2. dataclass kw_only
class attribute works as expected.
@serde
@dataclass(kw_only=True)
class Foo:
i: int
s: str
f: float
b: bool
See examples/kw_only.py for the complete example.
Attributes offered by pyserde
rename_all
rename_all
can converts field names into the specified string case. The following example converts camel case field names into snake case names. Case coversion depends on python-casefy. You can find the list of supported cases in the python-casefy's docs.
@serde(rename_all = 'camelcase')
class Foo:
int_field: int
str_field: str
f = Foo(int_field=10, str_field='foo')
print(to_json(f))
"int_field" is converted to "intField" and "str_field" is converted to "strField".
{"intField": 10, "strField": "foo"}
NOTE: If
rename_all
class attribute andrename
field attribute are used at the same time,rename
will be prioritized.@serde(rename_all = 'camelcase') class Foo: int_field: int str_field: str = field(rename='str-field') f = Foo(int_field=10, str_field='foo') print(to_json(f))
The above code prints
{"intField": 10, "str-field": "foo"}
See examples/rename_all.py for the complete example.
tagging
New in v0.7.0. See Union.
class_serializer
/ class_deserializer
If you want to use a custom (de)serializer at class level, you can pass your (de)serializer object in class_serializer
and class_deserializer
class attributes. Class custom (de)serializer depends on a python library plum which allows multiple method overloading like C++. With plum, you can write robust custom (de)serializer in a quite neat way.
class MySerializer:
@dispatch
def serialize(self, value: datetime) -> str:
return value.strftime("%d/%m/%y")
class MyDeserializer:
@dispatch
def deserialize(self, cls: Type[datetime], value: Any) -> datetime:
return datetime.strptime(value, "%d/%m/%y")
@serde(class_serializer=MySerializer(), class_deserializer=MyDeserializer())
class Foo:
v: datetime
One big difference from legacy serializer
and deserializer
is the fact that new class_serializer
and class_deserializer
are more deeply integrated at the pyserde's code generator level. You no longer need to handle Optional, List and Nested dataclass by yourself. Custom class (de)serializer will be used at all level of (de)serialization so you can extend pyserde to support third party types just like builtin types.
Also,
- If both field and class serializer specified, field serializer is prioritized
- If both legacy and new class serializer specified, new class serializer is prioritized
💡 Tip: If you implements multiple
serialize
methods, you will receive "Redefinition of unusedserialize
" warning from type checker. In such case, try usingplum.overload
andplum.dispatch
to workaround it. See plum's documentation for more information.from plum import dispatch, overload class Serializer: # use @overload @overload def serialize(self, value: int) -> Any: return str(value) # use @overload @overload def serialize(self, value: float) -> Any: return int(value) # Add method time and make sure to add @dispatch. Plum will do all the magic to erase warnings from type checker. @dispatch def serialize(self, value: Any) -> Any: ...
See examples/custom_class_serializer.py for complete example.
New in v0.13.0.
serializer
/ deserializer
NOTE: Deprecated since v0.13.0. Consider using
class_serializer
andclass_deserializer
.
If you want to use a custom (de)serializer at class level, you can pass your (de)serializer methods n serializer
and deserializer
class attributes.
def serializer(cls, o):
if cls is datetime:
return o.strftime('%d/%m/%y')
else:
raise SerdeSkip()
def deserializer(cls, o):
if cls is datetime:
return datetime.strptime(o, '%d/%m/%y')
else:
raise SerdeSkip()
@serde(serializer=serializer, deserializer=deserializer)
class Foo:
a: datetime
See examples/custom_legacy_class_serializer.py for complete example.
type_check
New in v0.9.0. See Type Check.
serialize_class_var
New in v0.9.8. Since dataclasses.fields
doesn't include a class variable 1, pyserde doesn't serialize class variable as default. This option allows a field of typing.ClassVar
to be serialized.
@serde(serialize_class_var=True)
class Foo:
a: ClassVar[int] = 10
See examples/class_var.py for complete example.
deny_unknown_fields
New in v0.22.0, the deny_unknown_fields
option in the pyserde decorator allows you to enforce strict field validation during deserialization. When this option is enabled, any fields in the input data that are not defined in the target class will cause deserialization to fail with a SerdeError
.
Consider the following example:
@serde(deny_unknown_fields=True)
class Foo:
a: int
b: str
With deny_unknown_fields=True
, attempting to deserialize data containing fields beyond those defined (a and b in this case) will raise an error. For instance:
from_json(Foo, '{"a": 10, "b": "foo", "c": 100.0, "d": true}')
This will raise a SerdeError
since fields c and d are not recognized members of Foo.
See examples/deny_unknown_fields.py for complete example.
Field Attributes
Field attributes are options to customize (de)serialization behaviour for a field of a dataclass.
Attributes offered by dataclasses
default
/ default_factory
default
and default_factory
work as expected. If a field has default
or default_factory
attribute, it behaves like an optional field. If the field is found in the data, the value is fetched from the data and set in the deserialized object. If the field is not found in the data, the specified default value is set in the deserialized object.
class Foo:
a: int = 10
b: int = field(default=10) # same as field "a"
c: dict[str, int] = field(default_factory=dict)
print(from_dict(Foo, {})) # prints Foo(a=10, b=10, c={})
See examples/default.py for the complete example.
ClassVar
dataclasses.ClassVar
is a class variable for the dataclasses. Since dataclass treats ClassVar as pseudo-field and dataclasses.field
doesn't pick ClassVar, pyserde doesn't (de)serialize ClassVar fields as a default behaviour. If you want to serialize ClassVar fields, consider using serialize_class_var class attribute.
See examples/class_var.py for the complete example.
Attributes offered by pyserde
Field attributes can be specified through serde.field
or dataclasses.field
. We recommend to use serde.field
because it's shorter and type check works.
Here is an example specifying rename
attribute in both serde.field
and dataclasses.field
.
@serde.serde
class Foo:
a: str = serde.field(rename="A")
b: str = dataclasses.field(metadata={"serde_rename"="B"})
rename
rename
is used to rename field name during (de)serialization. This attribute is convenient when you want to use a python keyword in field name. For example, this code renames field name id
to ID
.
@serde
class Foo:
id: int = field(rename="ID")
See examples/rename.py for the complete example.
skip
skip
is used to skip (de)serialization of the field with this attribute.
@serde
class Resource:
name: str
hash: str
metadata: dict[str, str] = field(default_factory=dict, skip=True)
See examples/skip.py for the complete example.
skip_if
skip
is used to skip (de)serialization of the field if the predicate function returns True
.
@serde
class World:
buddy: str = field(default='', skip_if=lambda v: v == 'Pikachu')
See examples/skip.py for the complete example.
skip_if_false
skip
is used to skip (de)serialization of the field if the field evaluates to False
. For example, this code skip (de)serializing if enemies
is empty.
@serde
class World:
enemies: list[str] = field(default_factory=list, skip_if_false=True)
See examples/skip.py for the complete example.
skip_if_default
skip
is used to skip (de)serialization of the field if the field is equivalent to its default value. For example, this code skip (de)serializing if town
is Masara Town
.
@serde
class World:
town: str = field(default='Masara Town', skip_if_default=True)
See examples/skip.py for the complete example.
alias
You can set aliases for field names. Alias only works for deserialization.
@serde
class Foo:
a: str = field(alias=["b", "c"])
Foo
can be deserialized from either {"a": "..."}
, {"b": "..."}
or {"c": "..."}
.
See examples/alias.py for complete example.
serializer
/deserializer
Sometimes you want to customize (de)serializer for a particular field, such as
- You want to serialize datetime into a different format
- You want to serialize a type in a third party package
In the following example, field a
is serialized into "2021-01-01T00:00:00"
by the default serializer for datetime
, whereas field b
is serialized into "01/01/21"
by the custom serializer.
@serde
class Foo:
a: datetime
b: datetime = field(serializer=lambda x: x.strftime('%d/%m/%y'), deserializer=lambda x: datetime.strptime(x, '%d/%m/%y'))
See examples/custom_field_serializer.py for the complete example.
flatten
You can flatten the fields of the nested structure.
@serde
class Bar:
c: float
d: bool
@serde
class Foo:
a: int
b: str
bar: Bar = field(flatten=True)
Bar's c, d fields are deserialized as if they are defined in Foo. So you will get {"a":10,"b":"foo","c":100.0,"d":true}
if you serialize Foo
into JSON.
See examples/flatten.py for complete example.
Union Representation
pyserde
>=0.7 offers attributes to control how Union is (de)serialized. This concept is the very same as the one in serde-rs. Note these representations only apply to the dataclass, non dataclass objects are (de)serialized with Untagged
always.
Untagged
This is the default Union representation for pyserde<0.7. Given these dataclasses,
@serde
class Bar:
b: int
@serde
class Baz:
b: int
@serde(tagging=Untagged)
class Foo:
a: Union[Bar, Baz]
Note that Bar
and Baz
have the same field name and type. If you serialize Foo(Baz(10))
into dict, you get {"a": {"b": 10}}
. But if you deserialize {"a": {"b": 10}}
, you get Foo(Bar(10))
instead of Foo(Baz(10))
. This means pyserde can't correctly (de)serialize dataclasses of Union with Untagged
. This is why pyserde offers other kinds of union representation options.
ExternalTagging
This is the default Union representation since 0.7. A class declaration with ExternalTagging
looks like below. If you serialize Foo(Baz(10))
into dict, you get {"a": {"Baz": {"b": 10}}}
and you can deserialize it back to Foo(Baz(10))
.
@serde(tagging=ExternalTagging)
class Foo:
a: Union[Bar, Baz]
NOTE: Non dataclass objects are alreays (de)serialized with
Untagged
regardless oftagging
attribute because there is no information which can be used for tag. The drawback ofUntagged
is pyserde can't correctly deserialize certain types. For example,Foo({1, 2, 3})
of below class is serialized into{"a": [1, 2, 3]}
, but you getFoo([1, 2, 3])
by deserializing.@serde(tagging=ExternalTagging) class Foo: a: Union[list[int], set[int]]
InternalTagging
A class declaration with InternalTagging
looks like below. If you serialize Foo(Baz(10))
into dict, you will get {"a": {"type": "Baz", "b": 10}}
and you can deserialize it back to Foo(Baz(10))
. type
tag is encoded inside the Baz
's dictionary.
@serde(tagging=InternalTagging("type"))
class Foo:
a: Union[Bar, Baz]
AdjacentTagging
A class declaration with AdjacentTagging
looks like below. If you serialize Foo(Baz(10))
into dict, you will get {"a": {"type": "Baz", "content": {"b": 10}}}
and you can deserialize it back to Foo(Baz(10))
. type
tag is encoded inside Baz
's dictionary and Baz
s fields are encoded inside content
.
@serde(tagging=AdjacentTagging("type", "content"))
class Foo:
a: Union[Bar, Baz]
(de)serializing Union types directly
New in v0.12.0.
Passing Union types directly in (de)serialize APIs (e.g. to_json, from_json) was partially supported prior to v0.12, but the union type was always treated as untagged. Users had no way to change the union tagging. The following example code wasn't able to correctly deserialize into Bar
due to untagged.
@serde
class Foo:
a: int
@serde
class Bar:
a: int
bar = Bar(10)
s = to_json(bar)
print(s)
# prints {"a": 10}
print(from_json(Union[Foo, Bar], s))
# prints Foo(10)
Since v0.12.0, pyserde can handle union that's passed in (de)serialize APIs a bit nicely. The union type is treated as externally tagged as that is the default tagging in pyserde. So the above example can correctly (de)serialize as Bar
.
@serde
class Foo:
a: int
@serde
class Bar:
a: int
bar = Bar(10)
s = to_json(bar, cls=Union[Foo, Bar])
print(s)
# prints {"Bar" {"a": 10}}
print(from_json(Union[Foo, Bar], s))
# prints Bar(10)
Also you can change the tagging using serde.InternalTagging
, serde.AdjacentTagging
and serde.Untagged
.
Now try to change the tagging for the above example. You need to pass a new argument cls
in to_json
. Also union class must be wrapped in either InternalTagging
, AdjacentTaging
or Untagged
with required parameters.
- InternalTagging
from serde import InternalTagging s = to_json(bar, cls=InternalTagging("type", Union[Foo, Bar])) print(s) # prints {"type": "Bar", "a": 10} print(from_json(InternalTagging("type", Union[Foo, Bar]), s)) # prints Bar(10)
- AdjacentTagging
from serde import AdjacentTagging s = to_json(bar, cls=AdjacentTagging("type", "content", Union[Foo, Bar])) print(s) # prints {"type": "Bar", "content": {"a": 10}} print(from_json(AdjacentTagging("type", "content", Union[Foo, Bar]), s)) # prints Bar(10)
- Untagged
from serde import Untagged s = to_json(bar, cls=Untagged(Union[Foo, Bar])) print(s) # prints {"a": 10} print(from_json(Untagged(Union[Foo, Bar]), s)) # prints Foo(10)
Type Checking
pyserde offers runtime type checking since v0.9. It was completely reworked at v0.14 using beartype and it became more sophisticated and reliable. It is highly recommended to enable type checking always as it helps writing type-safe and robust programs.
strict
Strict type checking is to check every field value against the declared type during (de)serialization and object construction. This is the default type check mode since v0.14. What will happen with this mode is if you declare a class with @serde
decorator without any class attributes, @serde(type_check=strict)
is assumed and strict type checking is enabled.
@serde
class Foo:
s: str
If you call Foo
with wrong type of object,
foo = Foo(10)
you get an error
beartype.roar.BeartypeCallHintParamViolation: Method __main__.Foo.__init__() parameter s=10 violates type hint <class 'str'>, as int 10 not instance of str.
NOTE: beartype exception instead of SerdeError is raised from constructor because beartype does not provide post validation hook as of Feb. 2024.
similarly, if you call (de)serialize APIs with wrong type of object,
print(to_json(foo))
again you get an error
serde.compat.SerdeError: Method __main__.Foo.__init__() parameter s=10 violates type hint <class 'str'>, as int 10 not instance of str.
NOTE: There are several caveats regarding type checks by beartype.
- beartype can not validate on mutated properties
The following code mutates the property "s" at the bottom. beartype can not detect this case.
@serde class Foo s: str f = Foo("foo") f.s = 100
- beartype can not validate every one of elements in containers. This is not a bug. This is desgin principle of beartype. See [Does beartype actually do anything?](https://beartype.readthedocs.io/en/latest/faq/#faq-o1].
coerce
Type coercing automatically converts a value into the declared type during (de)serialization. If the value is incompatible e.g. value is "foo" and type is int, pyserde raises an SerdeError
.
@serde(type_check=Coerce)
class Foo
s: str
foo = Foo(10)
# pyserde automatically coerce the int value 10 into "10".
# {"s": "10"} will be printed.
print(to_json(foo))
disabled
This is the default behavior until pyserde v0.8.3 and v0.9.x. No type coercion or checks are run. Even if a user puts a wrong value, pyserde doesn't complain anything.
@serde
class Foo
s: str
foo = Foo(10)
# pyserde doesn't complain anything. {"s": 10} will be printed.
print(to_json(foo))
Extending pyserde
pyserde offers three ways to extend pyserde to support non builtin types.
Custom field (de)serializer
💡 Tip: wrapping
serde.field
with your own field function makesimport serde def field(*args, **kwargs): serde.field(*args, **kwargs, serializer=str) @serde class Foo: a: int = field(default=0) # Configuring field serializer
Custom class (de)serializer
Custom global (de)serializer
You apply the custom (de)serialization for entire codebase by registering class (de)serializer by add_serializer
and add_deserializer
. Registered class (de)serializers are stacked in pyserde's global space and automatically used for all the pyserde classes.
e.g. Implementing custom (de)serialization for datetime.timedelta
using isodate package.
Here is the code of registering class (de)serializer for datetime.timedelta
. This package is actually published in PyPI as pyserde-timedelta.
from datetime import timedelta
from plum import dispatch
from typing import Type, Any
import isodate
import serde
class Serializer:
@dispatch
def serialize(self, value: timedelta) -> Any:
return isodate.duration_isoformat(value)
class Deserializer:
@dispatch
def deserialize(self, cls: Type[timedelta], value: Any) -> timedelta:
return isodate.parse_duration(value)
def init() -> None:
serde.add_serializer(Serializer())
serde.add_deserializer(Deserializer())
Users of this package can reuse custom (de)serialization functionality for datetime.timedelta
just by calling serde_timedelta.init()
.
import serde_timedelta
from serde import serde
from serde.json import to_json, from_json
from datetime import timedelta
serde_timedelta.init()
@serde
class Foo:
a: timedelta
f = Foo(timedelta(hours=10))
json = to_json(f)
print(json)
print(from_json(Foo, json))
and you get datetime.timedelta
to be serialized in ISO 8601 duration format!
{"a":"PT10H"}
Foo(a=datetime.timedelta(seconds=36000))
💡 Tip: You can register as many class (de)serializer as you want. This means you can use as many pyserde extensions as you want. Registered (de)serializers are stacked in the memory. A (de)serializer can be overridden by another (de)serializer.
e.g. If you register 3 custom serializers in this order, the first serializer will completely overridden by the 3rd one. 2nd one works because it is implemented for a different type.
- Register Serializer for
int
- Register Serializer for
float
- Register Serializer for
int
New in v0.13.0.
FAQ
How can I see the code generated by pyserde?
pyserde provides inspect
submodule that works as commandline:
python -m serde.inspect <PATH_TO_FILE> <CLASS>
e.g. in pyserde project
cd pyserde
poetry shell
python -m serde.inspect examples/simple.py Foo
Output
Loading simple.Foo from examples.
==================================================
Foo
==================================================
--------------------------------------------------
Functions generated by pyserde
--------------------------------------------------
def to_iter(obj, reuse_instances=True, convert_sets=False):
if reuse_instances is Ellipsis:
reuse_instances = True
if convert_sets is Ellipsis:
convert_sets = False
if not is_dataclass(obj):
return copy.deepcopy(obj)
Foo = serde_scope.types["Foo"]
res = []
res.append(obj.i)
res.append(obj.s)
res.append(obj.f)
res.append(obj.b)
return tuple(res)
...