Reductions
Scalar and grouped reductions with predicates, logical any/all, and row-aligned running totals, neighbors and windows.
with hm.Calculate(share=hm.col.sales / hm.sum("sales", by="region")):
hm.Points(x="month", y="share", fill="region")Usage
A reduction collapses a column (or an expression) to one value, which every row then gets: divide("sales", sum("sales")) is each row’s share of the total, and subtract("x", mean("x")) centers a column. With by, a categorical column or a list of them, each row gets its own group’s value instead, as pandas’ transform does: sum("sales", { by: "region" }) (Python hm.sum("sales", by="region")). Any number of groups works; a Group transform gives one row per group instead.
Missing and nonfinite values are skipped. With no value left, a reduction is NaN, except count and sum, which are 0. A reduction runs over the rows shown where it is evaluated: under a Filter or a brush it follows the kept rows, and it runs again when the data changes, not when the view moves.
quantile("x", p) (Python hm.quantile("x", p)) is the p quantile, 0 to 1, interpolated between the two nearest values as NumPy’s default does, so quantile("x", 0.5) is the median: Group with columns={{ p90: quantile("delay", 0.9) }} gives each group’s 90th percentile. For a whole quantile curve, sort the rows and give each its share of the group: divide(cumsum(is_finite("x"), {by: "g", orderBy: "x"}), count("x", {by: "g"})), which is the empirical distribution Ecdf draws when the values are distinct (Ecdf puts tied values at their last rank).
Python spells variance as hm.var. hm.min, hm.max, hm.sum, hm.any and hm.all are left out of from hmplot import *; use the hm. prefix.
Participation and booleans
sum("sales", {where: "@selected"}) (Python hm.sum("sales", where="@selected"))
uses selected visible rows. A predicate is an ordinary expression: changing it updates the
reduction without changing table membership. False, missing and nonfinite predicates exclude
rows. count(undefined, {where: predicate}) counts eligible rows; count("sales", ...) counts
eligible nonmissing sales. Python uses hm.count(where=predicate).
Highlight all observations of selected entities
any("@selected", {by: "country"}) (Python hm.any("@selected", by="country"))
broadcasts selection membership to every visible row with the same country. Use it in Calculate to name the predicate, in Filter(where=...) to keep matching observations,
or in an opacity expression to highlight them. Replace @selected with @hovered for
hover-linked highlighting. This derives a predicate; it does not expand the underlying
selection resource. Evaluate it before a filter that removes the peer observations you want
included. No Group or Join is needed, and grouping fields must be categorical.
Domains from the data
An axis domain can be read from the data: extent("x", {where}) is [min, max] of the rows where matches (Python hm.extent("x", where=...)), so x={{ domain: extent("x", { where: "@brushed" }) }} zooms a plot to the rows brushed in another plot of the same data and follows the brush. Each
end can be any reduction. It is read again when the data or the selection changes, and the
domain is automatic while no row matches.
any and all accept logical booleans or finite numbers (zero is false). Dictionary-encoded
booleans are decoded by value; category codes are never truth values. Compare string categories
first. Missing observations are skipped: no eligible observations means any=false, all=true.
Both reducers also work in Group’s columns, GraphData’s reduce and Window.
first and last accept orderBy/order (order_by in Python). Other scalar reducers have
no order option. Grouping fields must be categorical, as with existing reductions.
Group keys
| Expression | A group of rows per |
|---|---|
bin(field, bins, {interval, tz}) | Bin of field, as Bin bins it; writes its edges and center (x0, x1, x). |
hexbin(x, y, {radius, bins, orientation}) | Hexagon of x and y, as Hexbin’s; writes its center (x, y). |
These go in Group’s by, alone or beside columns: by={["carrier", bin("delay", 20)]} (Python by=["carrier", hm.bin("delay", 20)]). They are not values, so Calculate rejects them. Like hm.min, hm.bin is left out of from hmplot import *.
Row operations
| Expression | Result for each visible row |
|---|---|
cumsum(value, options) | Inclusive running total; missing contributes zero. |
lag(value, {periods: 1, fill: null, ...}) | Previous observation; never wraps across a group. |
lead(value, {periods: 1, fill: null, ...}) | Next observation; zero periods is identity. |
window(value, {size: 5, reduce: "mean", align: "trailing", ...}) | Window’s clipped rolling reduction. |
These accept by, orderBy and order: "asc" | "desc". Without an order field, they use
visible table order. They calculate stably within groups and restore results to the original
rows. Sort reorders the whole table instead. Ordered grid expressions are unsupported;
use a one-dimensional table. A standalone column-sort expression is deferred.
Totals and arithmetic window outputs use float32, with its normal rounding/overflow limits.
Neighbor gathers preserve categorical dictionaries, datetime origins and exact int32 values.
Missing floats are NaN; missing categories use their existing missing code. Int32 buffers have
no nullable representation: pass an integer fill, such as {fill: 0}, instead of losing precision.
Boolean device buffers likewise need an explicit boolean fill. Datetime fills use Date in
JavaScript and a datetime in Python. An explicit categorical fill must already be in the dictionary.
A row whose by key is missing belongs to no group: it is missing (NaN or the missing category), or
for int32 and boolean buffers takes the fill; a rolling any is false and all true there.
Window size can read a scalar input; it must resolve to a positive integer. Its reducer set and
trailing/center/leading alignment match Window, including any/all identities.
Another row’s value
lookup(value, row, {fill: null}) (Python hm.lookup(value, row, fill=None)) gives each row value at the row numbered row (0-based) of the visible rows in their order, as x[parent]: lookup("x", "parent") puts each node’s parent’s x beside it, and lookup("x", 0) the first
row’s on every row. A row number that is missing, fractional or past the last visible row reads
the fill, typed as for lag. Row numbers count the visible rows, so after a Filter they count what it keeps. Float row numbers are exact up to 2^24 rows; an
int32 column of row numbers is exact throughout. A drag cannot move a value read through lookup, and a Group’s columns reject it, as they reject the row operations above.
Running totals, previous values and rolling predicates
<script>
import { Plot, Calculate, Points, cumsum, lag, window, greater, subtract, sum } from "hmplot";
const data = { day: [0, 1, 2, 3, 4, 5], sales: [4, 2, 1, 3, 5, 2], team: ["A", "B", "A", "B", "A", "B"] };
const order = { by: "team", orderBy: "day" };
</script>
<Plot {data} height={320}>
<Calculate columns={{
total: cumsum("sales", order),
change: subtract("sales", lag("sales", { ...order, fill: 0 })),
recent: window(greater("sales", 3), { ...order, size: 2, reduce: "any" }),
selectedTotal: sum("sales", { where: "@selected" }),
}}>
<Points x="day" y="total" fill="recent" select respond={false}
tooltip={["team", "sales", "total", "change", "selectedTotal"]} />
</Calculate>
</Plot> import hmplot as hm
data = {"day": [0, 1, 2, 3, 4, 5], "sales": [4, 2, 1, 3, 5, 2],
"team": ["A", "B", "A", "B", "A", "B"]}
order = {"by": "team", "order_by": "day"}
with hm.Plot(data=data, height=320) as plot:
with hm.Calculate(
total=hm.cumsum("sales", **order),
change=hm.col.sales - hm.lag("sales", fill=0, **order),
recent=hm.window(hm.col.sales > 3, size=2, reduce="any", **order),
selectedTotal=hm.sum("sales", where="@selected"),
):
hm.Points(x="day", y="total", fill="recent", select=True, respond=False,
tooltip=["team", "sales", "total", "change", "selectedTotal"])