Haystack-Lua: Lua Bindings for Zinc Data Processing

I've been working on Lua bindings for my haystack library announced here and wanted to share what I've built so far. Haystack-Lua provides parsing and manipulation of Haystack data types through a pipeline-friendly Lua interface.

What it does

The library handles Zinc format parsing for grids, lists, and dictionaries. It's designed to work in command-line workflows where you pipe zinc data into lua scripts for transformation - similar to how you might use AWK/GAWK for text processing.

Built on top of Rust for performance, but exposes a straightforward Lua API for data operations.

Basic example

local hs = require('haystack')

-- Parse zinc from stdin
io.input(io.stdin)
local result = io.read("*all")
local grid = hs.io.parse.zinc.grid(result)

-- Access first row
local rec = grid:first()
print("ID:", rec["id"].id)
print("Dis:", rec["dis"])

-- Iterate over rows
for i, row in ipairs(grid) do
    if row:has("dis") then
        print("Site:", row.dis)
    end
end

Pipeline workflows

Works well with haystack-client linked earlier for end-to-end data processing:

# Get server info
haystack-client {{server}} about | lua haystack_about.lua

# Extract record IDs
haystack-client {{server}} read --filter "point and energy" | lua haystack_to_rec_ids.lua

# Generate plots from historical data. Output is gnuplot script
HIS=haystack-client {{server}} hisRead @elec-energy-point @elec-energy-point-2 | lua plot_history.lua

The examples directory includes scripts for server info display, record ID extraction, and gnuplot visualisation generation (shown below). See comment block at the heading of each script for a description and example usage.

Current state

This is actively under development. The core Zinc parsing is stable and ready for use, but there are some limitations but expanded support is planned:

• Read operations only (no serialisation yet)
• REST API not supported yet but is planned
• Zinc format exclusively
• Only tested on Lua 5.4
• API will evolve. I will do my best to limit breaking changes but cannot make guarantees

Installation is via LuaRocks with a rockspec, or building from source with cargo. Supports Lua 5.1+ including LuaJIT and Luau. Detailed instructions are in the github readme linked below.

Repository

Source and documentation: 'https://github.com/candronikos/haystack/tree/master/haystack-lua'

Looking for feedback on the API design and use cases. The goal is to make Haystack data processing accessible for ad-hoc processing, automation scripts, and analysis workflows.