Folder Structure
Directory Layout
Both the old and new Parquet exporters are currently active and writing to the same S3 bucket under different prefixes. During the transition period, files for the same time period will exist in both locations. See Schema Versioning for details.
New path format (current)
parquet/
└── v1/
└── {organizationId}/
├── devices/
│ └── {year}/
│ └── {month}/
│ └── {YYYY_MM_DD_HHMM}_{YYYY_MM_DD_HHMM}_devices.parquet
├── organizations/
├── spaces/
├── zones/
├── deviceGroups/
├── deviceGroupDevices/
├── deviceKeywords/
├── deviceNotes/
├── measurementAmbientTemperature/
├── measurementBatteryPercentage/
├── measurementBatteryVoltage/
├── measurementBootCount/
├── measurementCO2/
├── measurementDirectionalMovement/
├── measurementDistance/
├── measurementOccupancySeconds/
├── measurementOccupancyStatus/
├── measurementOccupantsCount/
├── measurementPosition/
├── measurementPositionZone/
├── measurementSystemTemperature/
└── measurementTVOC/
Old path format (legacy)
parquet/
└── {organizationId}/
├── device/
│ └── {year}/
│ └── {month}/
│ └── {YYYY_MM_DD_HH}_{dataType}.parquet
├── deviceGroup/
├── deviceGroupDevices/
├── deviceKeyword/
├── measurementAmbientTemperature/
├── measurementBatteryPercentage/
├── measurementBatteryVoltage/
├── measurementBootCount/
├── measurementCO2/
├── measurementDirectionalMovement/
├── measurementDistance/
├── measurementOccupancySeconds/
├── measurementOccupancyStatus/
├── measurementOccupantsCount/
├── measurementPosition/
├── measurementPositionZone/
├── measurementTVOC/
├── organization/
├── space/
└── zone/
Entity Folders
Each entity type has its own top-level folder. Folder names differ slightly between the old and new path formats (singular vs. plural):
| New folder | Old folder | Content type | Description |
|---|---|---|---|
organizations/ | organization/ | Metadata | Organization details |
devices/ | device/ | Metadata | Device inventory and properties |
deviceGroups/ | deviceGroup/ | Metadata | Device group definitions |
deviceGroupDevices/ | deviceGroupDevices/ | Metadata | Device-to-group mappings |
deviceKeywords/ | deviceKeyword/ | Metadata | Device tags and labels |
deviceNotes/ | (not exported) | Metadata | Device notes |
spaces/ | space/ | Metadata | Locations, buildings, floors |
zones/ | zone/ | Metadata | Areas within spaces |
measurementXxx/ | measurementXxx/ | Time-series | Sensor readings by type |
File Naming Convention
New format
Files include both the start and end timestamp of the export window:
{YYYY_MM_DD_HHMM}_{YYYY_MM_DD_HHMM}_{dataType}.parquet
| Component | Format | Description |
|---|---|---|
YYYY_MM_DD_HHMM | datetime | Start of the export window |
YYYY_MM_DD_HHMM | datetime | End of the export window |
dataType | string | Entity or measurement type |
Examples:
2026_03_20_1400_2026_03_20_1500_measurementCO2.parquet
2026_03_20_0000_2026_03_20_0100_devices.parquet
For measurement files, start and end mark the data window. For metadata files, the timestamps reflect the export interval window — the data is always a full snapshot, not scoped to that window.
Old format
Files use only the start hour of the export window:
{YYYY_MM_DD_HH}_{dataType}.parquet
Examples:
2026_01_28_08_device.parquet
2026_01_28_14_measurementOccupancyStatus.parquet
Time-Based Organization
Files are partitioned by year and month within each entity folder:
measurementOccupancyStatus/
├── 2025/
│ ├── 11/
│ │ ├── 2025_11_01_0000_2025_11_01_0100_measurementOccupancyStatus.parquet
│ │ ├── 2025_11_01_0100_2025_11_01_0200_measurementOccupancyStatus.parquet
│ │ └── ...
│ └── 12/
└── 2026/
└── 03/
├── 2026_03_20_1400_2026_03_20_1500_measurementOccupancyStatus.parquet
└── ...
Accessing Files
AWS CLI Example
List available files (new format):
aws s3 ls s3://<BUCKET>/parquet/v1/{organizationId}/devices/2026/03/
Download all device data for March 2026 (new format):
aws s3 cp s3://<BUCKET>/parquet/v1/{organizationId}/devices/2026/03/ ./devices/ --recursive
Programmatic Access
Most data platforms support direct S3 access. See Integration Examples for platform-specific code samples.
Storage Considerations
| Aspect | Details |
|---|---|
| Compression | Zstandard (zstd) in the new format; Snappy in the old format |
| File size | Varies by data volume; typically 1KB–100MB |
| Retention | Contact Haltian for retention policy details |
| Updates | New files each export cycle; historical files unchanged |
| Schema version | Embedded in each file’s Parquet footer metadata (schemaVersion key, only applies to new format) |