Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
6055f74
cbor: Scapy-native CBOR fields, codec hardening, and cbor2 CI
Sep 1, 2026
1c7c549
cbor: avoid Generic cls= kwargs for Python 3.7
Sep 1, 2026
ebd8e99
cbor: add RFC and lifecycle regression tests for PR #5125 refactor
Sep 9, 2026
5c2964d
cbor: fix RFC map-key equivalence, deterministic floats, and tag fields
Sep 9, 2026
810b262
cbor: store CBORF_ANY as CBOR_Object and drop islist workaround
Sep 9, 2026
077bc07
cbor: store unknown map extensions in per-map packet fields
Sep 9, 2026
ec71dff
cbor: restore standard Packet.do_build and slim copy()
Sep 9, 2026
8b61a2a
cbor: require explicit SEQUENCE_OF framing and Scapy list limits
Sep 9, 2026
3f89ec9
cbor: trim exports, tighten deterministic defaults, simplify byte-str…
Sep 9, 2026
c4c488a
cbor: remove legacy Float/Tag/Simple/UNDEFINED_VALUE wrappers
Sep 9, 2026
8fb9edb
cbor: compare NaN map keys by sign and significand
Sep 9, 2026
6d9d8d6
cbor: preserve NaN sign and significand in deterministic encoding
Sep 9, 2026
4e07f8b
cbor: drop dead map key-identity helpers and stale PR noise
Sep 9, 2026
6272f40
cbor: invalidate CBOR_FLOAT wire cache when val is assigned
Sep 9, 2026
1175485
cbor: reject nonterminal SEQUENCE_OF hidden by optional/conditional
Sep 9, 2026
b0700cc
cbor: speed map-key checks and reject collection pkt instances
Sep 9, 2026
ffe5f1c
cbor: track CBOR_ANY float wire-cache mutation
Sep 9, 2026
1c0a02b
cbor: require Packet subclasses with CBOR_root for collections
Sep 9, 2026
7fc8b1a
cbor: raise ValueError for unhashable as_dict keys
Sep 9, 2026
9c04aab
cbor: compare CBORMapData equality independent of pair order
Sep 9, 2026
1d05db2
cbor: share homogeneous SEQUENCE_OF/ARRAY_OF via _CBORF_HOMOGENEOUS
Sep 9, 2026
bcc5734
cbor: own SEQUENCE/ARRAY schema checks on _CBORF_compound
Sep 9, 2026
eb37dbd
cbor: make item encode/decode real CBORcodec_Object methods
Sep 9, 2026
49a722f
cbor: inline cache helpers and tighten packet field ownership
Sep 9, 2026
a63aad8
cbor: add AdditionalInfo/SimpleValue enums and UINT64_MAX
Sep 9, 2026
49ae42b
cbor: simplify float decode and indefinite child budgeting
Sep 9, 2026
6142d78
cbor: tighten SEMANTIC_TAG API and map unknown_field defaults
Sep 9, 2026
ace88d6
cbor: wrap long enum expressions and raise ARRAY_OF interop limit
Sep 9, 2026
60bf1d2
cbor: add CBOR_FloatAI for half/single/double encodings
Sep 9, 2026
359e020
cbor: reject duplicate map keys when encoding
Sep 9, 2026
5e43704
cbor: store CBOR_MAP as CBORMapData for key equivalence
Sep 9, 2026
99ee64a
cbor: count SEQUENCE items with skip instead of double decode
Sep 9, 2026
8769334
cbor: restore SEQUENCE build/dissect after skip-count change
Sep 9, 2026
f8734b4
cbor: fold SEQUENCE_OF next_cls_cb into homogeneous init
Sep 9, 2026
a58dfc6
cbor: thin CBORF_PACKET and inline single-use helpers
Sep 9, 2026
432a291
cbor: use MajorTypes and reserved AI in non-det scanner
Sep 9, 2026
1f519e5
cbor: add CBOR_encode_initial and fold indefinite helpers into fields
Sep 9, 2026
eb19bbd
cbor: bound skip/count nesting and finish scanner cleanup
Sep 9, 2026
2261aad
cbor: share structural skipper and harden indefinite string walks
Sep 9, 2026
28f5bb0
cbor: restore raw well-formed decode and house item span in codec
Sep 9, 2026
255606a
cbor: move non-det scanner to tests and trim field/API scaffolding
Sep 9, 2026
5e07ea7
cbor: reject unknown map keys that collide with schema names
Sep 9, 2026
ef2c5a9
cbor: optional semantic tags treat None defaults as empty
Sep 9, 2026
7e78ba2
cbor: copy ismutable defaults in Packet, drop getter materialization
Sep 9, 2026
dcf170a
cbor: share Packet raw-cache validity with CBOR_Packet
Sep 9, 2026
77a3a22
cbor: make counted build/dissect private compound bookkeeping
Sep 9, 2026
ee29b4d
cbor: drop dead field scaffolding and tighten nested encode/decode
Sep 9, 2026
4a6475e
cbor: fix Packet self_build typing after cache helper extract
Sep 9, 2026
a3300a6
cbor: honor debug_dissector and slim CBOR packet init
Sep 10, 2026
c4270d7
cbor: compose nested and collection raw-cache fingerprints
Sep 10, 2026
00a2bf0
cbor: separate structural parse budgets from live cardinality
Sep 10, 2026
76f0b7d
packet: keep FlagsField defaults out of fields for bind overloads
Sep 10, 2026
b4dc634
cbor: nested encode via bytes(); ANY counted build honors build()
Sep 10, 2026
6886f03
cbor: inherit BYTE_STRING_PACKET; trim absent/budget/float helpers
Sep 10, 2026
4045115
Split/cbor review budgets (#48)
polybassa Sep 10, 2026
73c774f
cbor: isolate nested TypeError test from debug_dissector pollution
Sep 10, 2026
3d6955d
cbor: terminal SEQUENCE_OF and trim review abstractions
Sep 10, 2026
e349446
cbor: reserve optionals with live min_items
Sep 10, 2026
df6b204
cbor: fix absent cache and greedy unframed sequences
Sep 10, 2026
bf30440
cbor: reject tagged SEQUENCE_OF; slim optional dissect
Sep 10, 2026
19841ec
cbor: typed key/fingerprint kinds and enum-safe compares
Sep 11, 2026
02f65fd
cbor: close SEQUENCE_OF holes and rename unframed fields
Sep 11, 2026
8daace9
cbor: allow nested framed REMAINDER_OF; bound MAP dissect
Sep 11, 2026
94bb572
cbor: prefer ITEMS isinstance for remainder recursion
Sep 11, 2026
de7b468
cbor: add MAP at-limit tests and nested indefinite remainder
Sep 11, 2026
9f3c0ed
cbor: keep unknown maps through optional tags
Sep 12, 2026
d70362d
cbor: reject unused ITEMS kwargs in UTS
Sep 14, 2026
6d0a4d4
cbor: fix array budget, float fingerprints, nested packets
Sep 14, 2026
e278abf
cbor: flatten ITEMS under ARRAY and fix conditional reserve
Sep 14, 2026
af0ca70
cbor: reserve conditionals from decoded dependencies
Sep 14, 2026
2468711
cbor: harden UTS keyword, float, and scanner coverage
Sep 14, 2026
90f71d6
cbor: support Packet APIs in conditional predicates
Sep 14, 2026
9fc6c38
cbor: evaluate array conditionals on the real packet
Sep 14, 2026
79448e3
cbor: evaluate array conditionals with packet defaults
Sep 14, 2026
92e1388
cbor: reserve array optionals with explicit depends_on
Sep 14, 2026
1c7a70f
cbor: default depends_on does not reserve array suffix items
Sep 14, 2026
71caaf0
cbor: reserve array suffix from resolved fields, not pkt.fields
Sep 15, 2026
a0e8846
cbor: document depends_on as optional-tail reservation only
Sep 15, 2026
8ebd71d
cbor: drop unused ITEMS dissect helper and type-check depends_on
Sep 15, 2026
d8e42ec
cbor: type depends_on as Sequence[str] to match runtime
Sep 15, 2026
be4a3ca
cbor: include nested child raw_packet_cache in parent fingerprint
Sep 16, 2026
17b1928
cbor: integrate cbor2 checks into the regular test suite
Oct 30, 2026
861299b
cbor: update packet and codec documentation
Nov 9, 2026
22e9e65
cbor: use None for absent packet fields
Nov 12, 2026
b55a46b
cbor: rebuild packet values without raw-cache tracking
Nov 12, 2026
188ad28
cbor: route counted operations through public field hooks
Nov 12, 2026
8146435
cbor: clarify collection names and optional-tail reservation
Nov 12, 2026
7ec4a18
cbor: move deterministic NaN encoding into the float codec
Nov 12, 2026
be821aa
cbor: use None for indefinite lengths and trim module scaffolding
Nov 12, 2026
9761613
cbor: update guide for optional values and packet rebuilds
Nov 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/unittests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,6 @@ jobs:
continue-on-error: true
with:
token: ${{ secrets.CODECOV_TOKEN }}

cryptography:
name: pyca/cryptography test
runs-on: ubuntu-latest
Expand Down
296 changes: 144 additions & 152 deletions doc/scapy/advanced_usage/cbor.rst
Original file line number Diff line number Diff line change
@@ -1,167 +1,159 @@
CBOR
====

What is CBOR?
-------------

.. note::

This section provides a practical introduction to CBOR from Scapy's perspective. For the complete specification, see RFC 8949.

CBOR (Concise Binary Object Representation) is a data format whose goal is to provide a compact, self-describing binary data interchange format based on the JSON data model. It is defined in RFC 8949 and is designed to be small in code size, reasonably small in message size, and extensible without the need for version negotiation.

CBOR provides basic data types including:

* **Unsigned integers** (major type 0): Non-negative integers
* **Negative integers** (major type 1): Negative integers
* **Byte strings** (major type 2): Raw binary data
* **Text strings** (major type 3): UTF-8 encoded strings
* **Arrays** (major type 4): Ordered sequences of values
* **Maps** (major type 5): Unordered key-value pairs
* **Semantic tags** (major type 6): Tagged values with additional semantics
* **Simple values and floats** (major type 7): Booleans, null, undefined, and floating-point numbers

Each CBOR data item begins with an initial byte that encodes the major type (in the top 3 bits) and additional information (in the low 5 bits). This design allows for compact encoding while maintaining self-describing properties.

Scapy and CBOR
--------------


Creating CBOR objects
^^^^^^^^^^^^^^^^^^^^^

CBOR objects can be easily created and composed::

>>> from scapy.cbor import CBOR_UNSIGNED_INTEGER, CBOR_TEXT_STRING, CBOR_BYTE_STRING, CBOR_ARRAY
>>> # Create basic types
>>> num = CBOR_UNSIGNED_INTEGER(42)
>>> text = CBOR_TEXT_STRING("Hello, CBOR!")
>>> data = CBOR_BYTE_STRING(b'\x01\x02\x03')
>>>
>>> # Create collections
>>> arr = CBOR_ARRAY([CBOR_UNSIGNED_INTEGER(1),
... CBOR_UNSIGNED_INTEGER(2),
... CBOR_TEXT_STRING("three")])
>>> arr
<CBOR_ARRAY[[<CBOR_UNSIGNED_INTEGER[1]>, <CBOR_UNSIGNED_INTEGER[2]>, <CBOR_TEXT_STRING['three']>]]>
>>>
>>> # Create maps
>>> from scapy.cbor.cborcodec import CBORcodec_MAP
>>> mapping = {"name": "Alice", "age": 30, "active": True}
CBOR (Concise Binary Object Representation) is a binary data format defined
in `RFC 8949 <https://www.rfc-editor.org/rfc/rfc8949.html>`_. Scapy provides
CBOR objects and codecs for individual values, and ``CBOR_Packet`` with
``CBORF_*`` fields for describing packet formats. No external CBOR library
is required.
Comment on lines +4 to +8

Encoding and decoding
^^^^^^^^^^^^^^^^^^^^^
---------------------

CBOR objects are encoded using their ``.enc()`` method. All codecs are referenced in the ``CBOR_Codecs`` object. The default codec is ``CBOR_Codecs.CBOR``::
Use ``bytes()`` to encode a CBOR object. ``CBORcodec_Object.dec()`` returns
the decoded object and the remaining bytes. The object's value is in ``.val``::

>>> num = CBOR_UNSIGNED_INTEGER(42)
>>> encoded = bytes(num)
>>> from scapy.cbor import CBOR_UNSIGNED_INTEGER, CBORcodec_Object
>>> encoded = bytes(CBOR_UNSIGNED_INTEGER(42))
>>> encoded.hex()
'182a'
>>>
>>> # Decode back
>>> decoded, remainder = CBOR_Codecs.CBOR.dec(encoded)
>>> decoded.val
42
>>> isinstance(decoded, CBOR_UNSIGNED_INTEGER)
>>> obj, remainder = CBORcodec_Object.dec(encoded)
>>> obj.val, remainder
(42, b'')

The codec also encodes Python values, including nested lists and dictionaries::

>>> CBORcodec_Object.encode_cbor_item([1, "hello", b"data"]).hex()
'83016568656c6c6f4464617461'

Use ``encode_cbor_item_deterministic()`` for RFC 8949 core-deterministic
encoding, including recursive map-key sorting by encoded bytes. Ordinary
encoding preserves map pair order instead.

Defining packets
----------------

Define a ``CBOR_root`` instead of ``fields_desc``. ``CBORF_ARRAY`` places
fields in a CBOR array. Integer, string, Boolean, and float fields expose
Python values::

>>> from scapy.cborpacket import CBOR_Packet
>>> from scapy.cbor import CBORF_ARRAY, CBORF_TEXT_STRING, CBORF_UNSIGNED_INTEGER
>>> class Message(CBOR_Packet):
... CBOR_root = CBORF_ARRAY(
... CBORF_UNSIGNED_INTEGER("version", 1),
... CBORF_TEXT_STRING("label", ""),
... )
...
>>> pkt = Message(label="hi")
>>> bytes(pkt).hex()
'8201626869'
>>> decoded = Message(bytes(pkt))
>>> decoded.version, decoded.label
(1, 'hi')
>>> decoded.version = 2
>>> bytes(decoded).hex()
'8202626869'

Both definite and indefinite arrays can be decoded. Use
``CBORF_ARRAY_INDEFINITE`` to build an indefinite array, or ``CBORF_ITEMS``
to concatenate fields without an array header.

For an array whose elements use the same field or packet decoder, use
``CBORF_ARRAY_OF("values", [],
pkt_cls=CBORF_UNSIGNED_INTEGER)``. ``CBORF_PACKET("child", None,
pkt_cls=Message)`` embeds a packet as one CBOR item;
``CBORF_BYTE_STRING_PACKET`` wraps the packet bytes in a CBOR byte string
instead. ``CBORF_SEMANTIC_TAG(tag_number, field)`` adds a semantic tag to a
field.

Optional fields
---------------

Wrap a field in ``CBORF_optional`` to allow it to be omitted. Missing decoded
fields contain ``None``. Use ``None`` as the default or assign it to omit an
optional field when building.

``CBORF_ANY`` keeps CBOR objects rather than converting them to Python values.
Use ``CBOR_NULL()`` or ``CBOR_UNDEFINED()`` for a present null or undefined
value::

>>> from scapy.cbor import CBOR_NULL, CBOR_UNDEFINED, CBORF_ANY, CBORF_optional
>>> class OptionalValue(CBOR_Packet):
... CBOR_root = CBORF_ARRAY(
... CBORF_optional(CBORF_ANY("value", None)),
... )
...
>>> bytes(OptionalValue()).hex()
'80'
>>> bytes(OptionalValue(value=None)).hex()
'80'
>>> bytes(OptionalValue(value=CBOR_NULL())).hex()
'81f6'
>>> bytes(OptionalValue(value=CBOR_UNDEFINED())).hex()
'81f7'
>>> OptionalValue(b"\x80").value is None
True

Encoding collections::

>>> from scapy.cbor import CBORcodec_ARRAY, CBORcodec_MAP
>>> # Encode an array
>>> encoded = CBORcodec_ARRAY.enc([1, 2, 3, 4, 5])
>>> encoded.hex()
'850102030405'
>>>
>>> # Decode the array
>>> decoded, _ = CBOR_Codecs.CBOR.dec(encoded)
>>> [item.val for item in decoded.val]
[1, 2, 3, 4, 5]
>>>
>>> # Encode a map
>>> encoded = CBORcodec_MAP.enc({"x": 100, "y": 200})
>>> decoded, _ = CBOR_Codecs.CBOR.dec(encoded)
>>> isinstance(decoded, CBOR_MAP)
>>> isinstance(OptionalValue(b"\x81\xf6").value, CBOR_NULL)
True

Working with different types
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

CBOR supports various data types::

>>> # Booleans
>>> true_val = CBOR_TRUE()
>>> false_val = CBOR_FALSE()
>>> bytes(true_val).hex()
'f5'
>>> bytes(false_val).hex()
'f4'
>>>
>>> # Null and undefined
>>> null_val = CBOR_NULL()
>>> undef_val = CBOR_UNDEFINED()
>>> bytes(null_val).hex()
'f6'
>>> bytes(undef_val).hex()
'f7'
>>>
>>> # Floating point
>>> float_val = CBOR_FLOAT(3.14159)
>>> bytes(float_val).hex()
'fb400921f9f01b866e'
>>>
>>> # Negative integers
>>> neg = CBOR_NEGATIVE_INTEGER(-100)
>>> bytes(neg).hex()
'3863'

Complex structures
^^^^^^^^^^^^^^^^^^

CBOR supports nested structures::

>>> # Nested arrays
>>> nested = CBORcodec_ARRAY.enc([1, [2, 3], [4, [5, 6]]])
>>> decoded, _ = CBOR_Codecs.CBOR.dec(nested)
>>> isinstance(decoded, CBOR_ARRAY)
True
>>>
>>> # Complex maps with mixed types
>>> data = {
... "name": "Bob",
... "age": 25,
... "active": True,
... "tags": ["user", "admin"]
... }
>>> encoded = CBORcodec_MAP.enc(data)
>>> decoded, _ = CBOR_Codecs.CBOR.dec(encoded)
>>> len(decoded.val)
4

Semantic tags
^^^^^^^^^^^^^

CBOR supports semantic tags (major type 6) for providing additional meaning to data items::

>>> # Tag 1 is for Unix epoch timestamps
>>> import time
>>> timestamp = int(time.time())
>>> tagged = CBOR_SEMANTIC_TAG((1, CBOR_UNSIGNED_INTEGER(timestamp)))
>>> encoded = bytes(tagged)
>>> decoded, _ = CBOR_Codecs.CBOR.dec(encoded)
>>> decoded.val[0] # Tag number
1


Error handling
^^^^^^^^^^^^^^

Scapy provides safe decoding with error handling::

>>> # Safe decoding returns error objects for invalid data
>>> invalid_data = b'\xff\xff\xff'
>>> obj, remainder = CBOR_Codecs.CBOR.safedec(invalid_data)
``CBORF_NULL`` and ``CBORF_UNDEFINED`` also retain explicit CBOR objects for
present values. Within Python lists and dictionaries passed to the codec or
``CBORF_ANY``, ``None`` still encodes CBOR null.

Maps
----

``CBORF_MAP`` uses field names as text keys and builds them in encoded-key
order. Unknown members are preserved in ``_cbor_unknown`` when dissecting::

>>> from scapy.cbor import CBORF_MAP
>>> class Metadata(CBOR_Packet):
... CBOR_root = CBORF_MAP(
... CBORF_UNSIGNED_INTEGER("b", 1),
... CBORF_UNSIGNED_INTEGER("a", 2),
... )
...
>>> bytes(Metadata()).hex()
'a2616102616201'

Use distinct ``unknown_field=`` names if a schema contains multiple maps.
For arbitrary map keys, use ``CBOR_MAP`` objects or ``CBORF_ANY`` instead.
``CBOR_MAP.val`` is a ``CBORMapData`` container of ordered pairs: it keeps
keys such as ``1`` and ``True`` distinct. ``cbor_pairs()`` returns the pairs;
``as_dict()`` raises ``ValueError`` if conversion would lose key distinctions
or a key cannot be used in a Python dictionary.

Rebuilding packets
------------------

Decoded packets are rebuilt from their fields when encoded. The schema
determines the container form and map-key order; typed floats use the shortest
exact encoding. Non-preferred input can therefore change even when no field
has been edited. The received input remains available in ``pkt.original``::

>>> decoded = Message(bytes.fromhex("9f1801626869ff"))
>>> decoded.version, decoded.label
(1, 'hi')
>>> decoded.original.hex()
'9f1801626869ff'
>>> bytes(decoded).hex()
'8201626869'

``CBORF_ANY`` retains its CBOR object values. Individual objects such as
``CBOR_FLOAT`` can preserve their received float encoding until their ``.val``
is assigned. This does not guarantee an exact-byte round trip for the packet.

Invalid data
------------

``dec()`` raises on invalid data. ``safedec()`` returns a
``CBOR_DECODING_ERROR`` object instead::

>>> from scapy.cbor import CBOR_DECODING_ERROR
>>> obj, remainder = CBORcodec_Object.safedec(b"\xff")
>>> isinstance(obj, CBOR_DECODING_ERROR)
True

See :mod:`scapy.cbor.cborfields` and :mod:`scapy.cbor.cborcodec` for the
field and codec APIs.
Loading
Loading