binfield

Python binfield implementation for binary data manipulation.

Why? Python supports binary data manipulation via binary operations out of the box and it’s fast, but it’s hard to read and painful during prototyping, especially for complex (nested) structures.

This library is designed to fix this issue: it allows to operate with binary data like dict with constant indexes: you just need to define structure class and create an instance with start data. Now you can use indexes for reading and writing data

Pros:

• Free software: Apache license
• Open Source: https://github.com/penguinolog/binfield
• Self-documented code: docstrings with types in comments
• Tested: see badges on top
• Support multiple Python versions:

Python 3.6
Python 3.7
PyPy3

Usage

Not mapped objects can be created simply from BinField class:

bf = BinField(42)

Data with fixed size should be created as a new class (type): Example on real data (ZigBee frame control field):

# Describe
class ZBFrameControl(binfield.BinField):
    _size_ = 16  # Optional, used as source for mask, if mask is not defined
    _mask_ = 0xFF7F  # Optional, used as source for size, if size is not defined
    FrameType = [0, 3]  # Enum
    Security = 3
    FramePending = 4
    AckRequest = 5
    PAN_ID_Compression = 6
    SecurityNumberSuppress = 8
    InformationPresent = 9
    DstAddrMode = [10, 12]
    FrameVersion =  [12, 14]
    SrcAddrMode = [14, 16]

# Construct from frame
# (limitation: endian conversion is not supported, make it using another tools)
frame = frame = ZBFrameControl(0x0803)  # Beacon request

>>> print(frame)
<2051 == 0x0803 == (0b0000100000000011 & 0b1111111111111111)
  FrameType             = <3 == 0x03 == (0b011 & 0b111)>
  Security               = <0 == 0x00 == (0b0 & 0b1)>
  FramePending           = <0 == 0x00 == (0b0 & 0b1)>
  AckRequest             = <0 == 0x00 == (0b0 & 0b1)>
  PAN_ID_Compression     = <0 == 0x00 == (0b0 & 0b1)>
  SecurityNumberSuppress = <0 == 0x00 == (0b0 & 0b1)>
  InformationPresent     = <0 == 0x00 == (0b0 & 0b1)>
  DstAddrMode            = <2 == 0x02 == (0b10 & 0b11)>
  FrameVersion           = <0 == 0x00 == (0b00 & 0b11)>
  SrcAddrMode            = <0 == 0x00 == (0b00 & 0b11)>

>>> repr(frame)
'ZBFrameControl(x=0x0803, base=16)'

>>> print(frame.FrameType)
<3 == 0x03 == (0b011 & 0b111)>  # Get nested structure: current is flat, so we have single value

# We can use slice to get bits from value: result type is always subclass of BinField
>>> repr(frame.FrameType[: 2])
'<FrameType_slice_0_2(x=0x03, base=16) at 0x7FD0ACA57408>'

>>> frame.FrameType == 3  # Transparent comparision with integers
True

>>> int(frame.FrameType)  # Painless conversion to int
3

>>> bool(frame.AckRequest)  # And bool
False

>>> print(frame[1: 5])  # Ignore indexes and just get few bits using slice
<1 == 0x01 == (0b0001 & 0b1111)>

>>> print(ZBFrameControl.AckRequest)  # Request indexes from created data type
5

>>> print(ZBFrameControl.DstAddrMode)  # Multiple bits too
slice(10, 12, None)

# Modification of nested data (if no type conversion was used) changes original object:
>>> frame.AckRequest = 1
>>> print(frame)
<2083 == 0x0823 == (0b0000100000100011 & 0b1111111101111111)
  FrameType              = <3 == 0x03 == (0b011 & 0b111)>
  Security               = <0 == 0x00 == (0b0 & 0b1)>
  FramePending           = <0 == 0x00 == (0b0 & 0b1)>
  AckRequest             = <1 == 0x01 == (0b1 & 0b1)>
  PAN_ID_Compression     = <0 == 0x00 == (0b0 & 0b1)>
  SecurityNumberSuppress = <0 == 0x00 == (0b0 & 0b1)>
  InformationPresent     = <0 == 0x00 == (0b0 & 0b1)>
  DstAddrMode            = <2 == 0x02 == (0b10 & 0b11)>
  FrameVersion           = <0 == 0x00 == (0b00 & 0b11)>
  SrcAddrMode            = <0 == 0x00 == (0b00 & 0b11)>
>

# But remember, that nested blocks has it's own classes
>>> repr(frame.DstAddrMode)
'<DstAddrMode(x=0x02, base=16) at 0x7FD0AD139548>'

>>> fr2 = ZBFrameControl(0xFFFF)
>>> repr(fr2)
'ZBFrameControl(x=0xFF7F, base=16)'  # Mask if applied, if defined

# Fields can be set only from integers
>>> frame.SrcAddrMode = fr2.SrcAddrMode
Traceback (most recent call last):
...
TypeError: BinField value could be set only as int

>>> repr(frame['FramePending'])  # __getitem__ and __setitem__ is supported
'<FramePending(x=0x00, base=16) at 0x7FD0ACAD3188>'

Nested structures are supported, if required. Definition example (not aligned with any real data):

class NestedMappedBinField(BinField):
    test_index = 0
    nested_block = {
        '_index_': (1, 6),
        'single_bit': 0,
        'multiple': (1, 3)
    }

>>> bf = NestedMappedBinField(0xFF)
# No _size_ and no _mask_ -> size is not limited,
# but indexes can not be changed after class creation
>>> print(bf)
<255 == 0xFF == (0b11111111)
  test_index   = <1 == 0x01 == (0b1 & 0b1)>
  nested_block =
    <31 == 0x1F == (0b11111 & 0b11111)
      single_bit = <1 == 0x01 == (0b1 & 0b1)>
      multiple   = <3 == 0x03 == (0b11 & 0b11)>
    >
>

# Get nested block: nested block is structured.
>>> print(bf.nested_block)
<31 == 0x1F == (0b11111 & 0b11111)
  single_bit = <1 == 0x01 == (0b1 & 0b1)>
  multiple   = <3 == 0x03 == (0b11 & 0b11)>
>

Note: negative indexes are not supported by design!

Testing

Main test mechanism for the package binfield uses tox. Test environments available:

pep8
py36
pypy3
pylint
docs

CI systems

For code checking several CI systems are used in parallel:

1. Travis CI: is used for checking: PEP8, pylint, bandit, installation possibility and unit tests. Also it publishes coverage on coveralls.
2. coveralls: is used for coverage display.

CD system

Travis CI: is used for package delivery on PyPI.

Contents:

API: BinField class.

class binfield.BinField(x=0, base=10, _parent=None)

BinField representation.

Parameters:

• x (typing.Union[int, str, bytes]) – Start value
• base (int) – base for start value
• _parent (typing.Optional[typing.Tuple[BinField, int]]) – Parent link. For internal usage only.

Note

Subclasses have getters for mapping indexes.

Note

Subclasses instances have getters and setters for mapping records.

_bit_size_

int - Number of bits necessary to represent in binary.

_value_

int - Internal value.

__int__()

Convert to integer.

Return type:int

__index__()

Special method used for bin()/hex/oct/slicing support.

Return type:int

__abs__()

int mimic.

Return type:int

__gt__(other)

Comparing logic.

Return type:bool

__ge__(other)

Comparing logic.

Return type:bool

__lt__(other)

Comparing logic.

Return type:bool

__le__(other)

Comparing logic.

Return type:bool

__eq__(other)

Comparing logic.

Return type:bool

__ne__(other)

Comparing logic.

Return type:bool

__iand__(other)

int mimic.

__ior__(other)

int mimic.

__ixor__(other)

int mimic.

__and__(other)

int mimic.

Return type:BinField

__rand__(other)

int mimic

Allows to call other & self

__or__(other)

int mimic.

Return type:BinField

__ror__(other)

int mimic

Allows to call other | self

__xor__(other)

int mimic.

Return type:BinField

__rxor__(other)

int mimic

Allows to call other ^ self

__iadd__(other)

int mimic.

Raises:

• OverflowError – Result not fills in data length
• ValueError – negative result

__isub__(other)

int mimic.

__add__(other)

int mimic.

Return type:typing.Union[int, BinField] Raises:ValueError – negative result

__radd__(other)

int mimic

Allows to call other + self

__sub__(other)

int mimic.

Return type:typing.Union[int, BinField]

__rsub__(other)

int mimic

Allows to call other - self

__mul__(other)

int mimic.

Return type:int

__rmul__(other)

int mimic

Allows to call other * self

__lshift__(other)

int mimic.

Return type:int

__rlshift__(other)

int mimic

Allows to call other << self

__rshift__(other)

int mimic.

Return type:int

__rrshift__(other)

int mimic

Allows to call other >> self

__bool__(other)

int mimic.

Return type:bool

__hash__()

Hash.

__copy__()

Copy logic.

Return type:BinField

Note

Uplink is destroyed on copy.

__getstate__()

Pickling.

Return type:typing.Dict[str: int] Raises:ValueError – Pickle of linked instance

__getnewargs__()

required for pickle.

Return type:typing.Tuple

__setstate__(state)

Restore from pickle.

__getitem__(item)

Extract bits.

Return type:BinField Raises:IndexError – Mapping is not available

__setitem__(key, value)

Indexed setter

Raises:

• TypeError – value type is not int
• IndexError – key not found (or key is not string, no mapping)

Indices and tables

• Index
• Module Index
• Search Page