epicsarchiver.retrieval.pb
==========================

.. py:module:: epicsarchiver.retrieval.pb

.. autoapi-nested-parse::

   Handle retrieval of data in the Archiver Appliance PB file format.

   The file format is described on this page:
   https://slacmshankar.github.io/epicsarchiver_docs/pb_pbraw.html

   The data can be parsed in the same way whether retrieved using the
   Rest API or whether reading files directly from disk. In either
   case, it is important to treat the data as binary data - a stream of
   bytes. The Google Protobuf library handles converting the stream of
   bytes into the objects defined by the EPICSEvent.proto file.

   The Archiver Appliance escapes certain characters as described on the
   page above, which allows one to deduce the number of events in the
   binary file using tools such as wc.

   The unescape_bytes() method handles unescaping these characters before
   handing the interpretation over to the Google Protobuf library.

   Note: due to the way the protobuf objects are constructed, pylint can't
   correctly deduce some properties, so I have manually disabled some warnings.



Attributes
----------

.. autoapisummary::

   epicsarchiver.retrieval.pb.LOG
   epicsarchiver.retrieval.pb.TYPE_MAPPINGS
   epicsarchiver.retrieval.pb.INVERSE_TYPE_MAPPINGS
   epicsarchiver.retrieval.pb.ESC_BYTE
   epicsarchiver.retrieval.pb.NL_BYTE
   epicsarchiver.retrieval.pb.CR_BYTE
   epicsarchiver.retrieval.pb.PB_REPLACEMENTS_ESCAPING
   epicsarchiver.retrieval.pb.PB_REPLACEMENTS_UNESCAPING
   epicsarchiver.retrieval.pb.EeScalarEvent
   epicsarchiver.retrieval.pb.EeVectorEvent
   epicsarchiver.retrieval.pb.EeEvent


Functions
---------

.. autoapisummary::

   epicsarchiver.retrieval.pb.unescape_bytes
   epicsarchiver.retrieval.pb.escape_bytes
   epicsarchiver.retrieval.pb.event_pd_timestamp
   epicsarchiver.retrieval.pb.event_timestamp
   epicsarchiver.retrieval.pb.get_timestamp_from_line_function
   epicsarchiver.retrieval.pb._break_up_chunks
   epicsarchiver.retrieval.pb._event_from_line
   epicsarchiver.retrieval.pb.parse_pb_data
   epicsarchiver.retrieval.pb.get_iso_timestamp_for_event
   epicsarchiver.retrieval.pb.read_pb_file
   epicsarchiver.retrieval.pb.to_field_value


Module Contents
---------------

.. py:data:: LOG
   :type:  logging.Logger

.. py:data:: TYPE_MAPPINGS
   :type:  dict[int, type]

.. py:data:: INVERSE_TYPE_MAPPINGS

.. py:data:: ESC_BYTE
   :value: b'\x1b'


.. py:data:: NL_BYTE
   :value: b'\n'


.. py:data:: CR_BYTE
   :value: b'\r'


.. py:data:: PB_REPLACEMENTS_ESCAPING

.. py:data:: PB_REPLACEMENTS_UNESCAPING

.. py:data:: EeScalarEvent

.. py:data:: EeVectorEvent

.. py:data:: EeEvent

.. py:function:: unescape_bytes(byte_seq: bytes) -> bytes

   Replace specific sub-sequences in a bytes sequence.

   This escaping is defined as part of the Archiver Appliance raw file
   format: https://slacmshankar.github.io/epicsarchiver_docs/pb_pbraw.html

   :param byte_seq: any byte sequence

   :returns: the byte sequence unescaped according to the AA file format rules


.. py:function:: escape_bytes(byte_seq: bytes) -> bytes

   Replace specific sub-sequences in a bytes sequence.

   This escaping is defined as part of the Archiver Appliance raw file
   format: https://slacmshankar.github.io/epicsarchiver_docs/pb_pbraw.html

   :param byte_seq: any byte sequence

   :returns: the byte sequence escaped according to the AA file format rules


.. py:function:: event_pd_timestamp(year: int, event: EeEvent) -> pandas.Timestamp

   Converts from protobuf event time format to python datetime.

   :param year: year of event
   :type year: int
   :param event: input event
   :type event: EeEvent

   :returns: Output datetime
   :rtype: pydt


.. py:function:: event_timestamp(year: int, event: EeEvent) -> datetime.datetime

   Converts from protobuf event time format to python datetime.

   :param year: year of event
   :type year: int
   :param event: input event
   :type event: EeEvent

   :returns: Output datetime
   :rtype: pydt


.. py:function:: get_timestamp_from_line_function(chunk_info: epicsarchiver.retrieval.EPICSEvent_pb2.PayloadInfo) -> collections.abc.Callable[[bytes], datetime.datetime]

   From a unescaped protobuf line create a function to get datetime.

   :param chunk_info: Payload info of protobuf file
   :type chunk_info: ee.PayloadInfo

   :returns: Function to provide event time
   :rtype: Callable[[bytes], pydt]


.. py:function:: _break_up_chunks(raw_data: bytes) -> collections.OrderedDict[int, tuple[epicsarchiver.retrieval.EPICSEvent_pb2.PayloadInfo, list[bytes]]]

   Break up raw data into chunks by year.

   :param raw_data: Raw data from file

   :returns: keys are years; values are lists of chunks
   :rtype: collections.OrderedDict


.. py:function:: _event_from_line(line: bytes, pv: str, year: int, event_type: int) -> epicsarchiver.retrieval.archive_event.ArchiveEvent

   Get an ArchiveEvent from this line.

   :param line: A line of chunks of data
   :param pv: Name of the PV
   :param year: Year of interest
   :param event_type: Need to know the type of the event as key of TYPE_MAPPINGS

   :returns: ArchiveEvent


.. py:function:: parse_pb_data(raw_data: bytes) -> list[epicsarchiver.retrieval.archive_event.ArchiveEvent]

   Turn raw PB data into an ArchiveData object.

   :param raw_data: The raw data

   :returns: An ArchiveData object


.. py:function:: get_iso_timestamp_for_event(year: int, event: EeEvent) -> str

   Returns an ISO-formatted timestamp string for the given event.


.. py:function:: read_pb_file(filename: str) -> list[epicsarchiver.retrieval.archive_event.ArchiveEvent]

   Read an unescaped protobuf file and produce a list of events from file.

   :param filename: location of file
   :type filename: str

   :returns: list of events in file
   :rtype: list[ArchiveEvent]


.. py:function:: to_field_value(f: epicsarchiver.retrieval.EPICSEvent_pb2.FieldValue) -> epicsarchiver.retrieval.archive_event.FieldValue

   From the protobuf variant.

   :param f: protobuf field value
   :type f: ee.FieldValue

   :returns: Basic Field Value
   :rtype: FieldValue