Jupyter Notebook - Retrieve Data
SDS & Archiver Data Retrieval
Get data from two sources, specify a time range.
Import modules
Need a data reading function from ops_data_client:
from ops_data_client import get_data
import polars as pl
Get PV data
Using the full PV names, get data from the SDS, which is returned as a Polars Dataframe.
df = get_data(
pvs=["RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData",
"PBI-APTM02:Ctrl-ECAT-100:WF-Temp11"],
start="2025-06-11 04:06:25.925641123",
end="2025-06-11 04:06:26.782784123456"
)
Display data
Simply print the dataframe in its default table format.
df
shape: (16, 36)
| PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_data_array | PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_severity | PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_status | PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_field_values | PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_data_ts | cycle_id | cycle_ts | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_data_ts | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_stored_at | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_collector | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_ctype | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_pv | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_data_array | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_event_id | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_event_count | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_event_code | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_event_ts | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_sds_cycle_id | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_sds_cycle_ts | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_mode | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_state | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_present | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_len | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_energy | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_current | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_beam_dest | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_buffer_size | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_buffer_index | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_array_sampling | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_array_decimation | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_array_size | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_trig_name | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_trig_evr | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_trig_delay | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_trig_code | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_trig_ts |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| list[f64] | i64 | i64 | list[null] | datetime[ns, UTC] | i64 | datetime[ns, UTC] | datetime[ns, UTC] | datetime[μs, UTC] | str | str | str | list[f32] | u64 | u64 | u64 | datetime[ns, UTC] | u64 | datetime[ns, UTC] | str | str | bool | u32 | u32 | u32 | str | u32 | u32 | f32 | u32 | u32 | str | str | f32 | u64 | datetime[ns, UTC] |
| null | null | null | null | null | 15658604172 | 2025-06-11 04:06:25.854212373 UTC | 2025-06-11 04:06:25.854212352 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [4.908, 4.898, … 4.918] | 15658604183 | 18819282 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604172 | 2025-06-11 04:06:25.854212352 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 0 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:25.925640960 UTC |
| [21.791, 23.152, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:25.925640941 UTC | 15658604173 | 2025-06-11 04:06:25.925640941 UTC | 2025-06-11 04:06:25.925640960 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [4.898, 4.918, … 4.778] | 15658604183 | 18819284 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604173 | 2025-06-11 04:06:25.925640960 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 1 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:25.997069568 UTC |
| [20.950001, 23.01, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:25.997069509 UTC | 15658604174 | 2025-06-11 04:06:25.997069509 UTC | 2025-06-11 04:06:25.997069568 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [5.028, 4.998, … 4.988] | 15658604183 | 18819284 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604174 | 2025-06-11 04:06:25.997069568 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 2 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.068498176 UTC |
| [20.929001, 19.315001, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.068498077 UTC | 15658604175 | 2025-06-11 04:06:26.068498077 UTC | 2025-06-11 04:06:26.068498176 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [4.898, 4.888, … 4.888] | 15658604183 | 18819284 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604175 | 2025-06-11 04:06:26.068498176 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 3 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.139926528 UTC |
| [18.632, 20.610001, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.139926646 UTC | 15658604176 | 2025-06-11 04:06:26.139926646 UTC | 2025-06-11 04:06:26.139926528 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [4.918, 4.828, … 4.878] | 15658604183 | 18819286 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604176 | 2025-06-11 04:06:26.139926528 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 4 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.211355136 UTC |
| … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … | … |
| [18.878, 22.485001, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.639926623 UTC | 15658604183 | 2025-06-11 04:06:26.639926623 UTC | 2025-06-11 04:06:26.639926528 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [5.018, 4.998, … 4.828] | 15658604183 | 18819292 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604183 | 2025-06-11 04:06:26.639926528 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 11 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.711355136 UTC |
| [18.878, 22.485001, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.639926623 UTC | 15658604183 | 2025-06-11 04:06:26.639926623 UTC | 2025-06-11 04:06:26.639926528 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [4.898, 4.858, … 4.958] | 15658604183 | 18819292 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604183 | 2025-06-11 04:06:26.639926528 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 12 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.782783744 UTC |
| [22.555, 22.573999, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.711355191 UTC | 15658604184 | 2025-06-11 04:06:26.711355191 UTC | 2025-06-11 04:06:26.711355136 UTC | 2025-06-11 04:10:03 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [5.168, 5.148, … 4.928] | 15658604183 | 18819294 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604184 | 2025-06-11 04:06:26.711355136 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 13 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.782783744 UTC |
| [22.555, 22.573999, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.711355191 UTC | 15658604184 | 2025-06-11 04:06:26.711355191 UTC | 2025-06-11 04:06:26.711355136 UTC | 2025-06-11 09:33:09 UTC | "RFQ-010-FIM-DOD" | "DD" | "RFQ-010:RFS-Cav-110:Fld-Wave_S… | [5.168, 5.148, … 4.928] | 15658604183 | 18819294 | 42 | 2025-06-11 04:06:26.668351744 UTC | 15658604184 | 2025-06-11 04:06:26.711355136 UTC | "Probe" | "BEAM_PRESENT_YES_FAKE" | false | 0 | 0 | 59300 | "LebtFc" | 14 | 13 | 1e6 | 1 | 8000 | "RFQ-010:RFS-EVR-101" | "DlyGen-6" | -3000.0 | 14 | 2025-06-11 04:06:26.782783744 UTC |
| [20.827, 21.809, … 0.0] | 0 | 0 | [] | 2025-06-11 04:06:26.782783759 UTC | 15658604185 | 2025-06-11 04:06:26.782783759 UTC | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Simple calculations
Find the mean values in the data array for each PV in each cycle, then display just some of the dataframe columns.
df_partial = df.with_columns(
(pl.col("RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_data_array").list.mean()).alias("sds_mean_value"),
(pl.col("PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_data_array").list.mean()).alias("arch_mean_value"),
).select(
"cycle_id", "cycle_ts", "RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_data_array","PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_data_array", "sds_mean_value", "arch_mean_value"
)
df_partial
shape: (16, 6)
| cycle_id | cycle_ts | RFQ-010:RFS-Cav-110:Fld-Wave_SDS-DODData_data_array | PBI-APTM02:Ctrl-ECAT-100:WF-Temp11_data_array | sds_mean_value | arch_mean_value |
|---|---|---|---|---|---|
| i64 | datetime[ns, UTC] | list[f32] | list[f64] | f32 | f64 |
| 15658604172 | 2025-06-11 04:06:25.854212373 UTC | [4.908, 4.898, … 4.918] | null | 4.888259 | null |
| 15658604173 | 2025-06-11 04:06:25.925640941 UTC | [4.898, 4.918, … 4.778] | [21.791, 23.152, … 0.0] | 4.854619 | 6.180347 |
| 15658604174 | 2025-06-11 04:06:25.997069509 UTC | [5.028, 4.998, … 4.988] | [20.950001, 23.01, … 0.0] | 4.984537 | 6.302639 |
| 15658604175 | 2025-06-11 04:06:26.068498077 UTC | [4.898, 4.888, … 4.888] | [20.929001, 19.315001, … 0.0] | 4.881917 | 6.740958 |
| 15658604176 | 2025-06-11 04:06:26.139926646 UTC | [4.918, 4.828, … 4.878] | [18.632, 20.610001, … 0.0] | 4.893318 | 6.561056 |
| … | … | … | … | … | … |
| 15658604183 | 2025-06-11 04:06:26.639926623 UTC | [5.018, 4.998, … 4.828] | [18.878, 22.485001, … 0.0] | 4.925144 | 6.506083 |
| 15658604183 | 2025-06-11 04:06:26.639926623 UTC | [4.898, 4.858, … 4.958] | [18.878, 22.485001, … 0.0] | 4.911372 | 6.506083 |
| 15658604184 | 2025-06-11 04:06:26.711355191 UTC | [5.168, 5.148, … 4.928] | [22.555, 22.573999, … 0.0] | 4.955574 | 6.503056 |
| 15658604184 | 2025-06-11 04:06:26.711355191 UTC | [5.168, 5.148, … 4.928] | [22.555, 22.573999, … 0.0] | 4.955574 | 6.503056 |
| 15658604185 | 2025-06-11 04:06:26.782783759 UTC | null | [20.827, 21.809, … 0.0] | null | 6.178431 |