---
name: eocube-dask
description: "Scale notebook computation on the EOCube Dask Gateway: start or reuse a personal cluster with setup_personal_cluster, size it, ship presigned hrefs rather than filesystem objects to workers, chunk COG reads sensibly, watch the dashboard, and close the cluster when done."
license: Apache-2.0
compatibility: "EOCube.RO JupyterHub notebooks (notebooks.eocube.ro); the Dask Gateway is reachable only from inside the hub (DASK_GATEWAY__* variables are injected into notebook pods)."
---

## Start or reuse a cluster

```python
from eocube.integrations.notebooks import setup_notebook, setup_personal_cluster
setup_notebook()
cluster, client = setup_personal_cluster(4)   # workers; omit to keep the current size
client.dashboard_link                          # open it in a browser tab
```

`setup_personal_cluster` connects to the user's existing gateway cluster if there is
exactly one, creates one if there is none, and raises if there are several (close the
extras with `dask_gateway.Gateway().list_clusters()` and `.connect(name).shutdown()`).
Scale later with `cluster.scale(n)` or `cluster.adapt(minimum=1, maximum=8)`.

## What workers can and cannot do

- Workers run the same `rocs-base-notebook` image as the notebook, so imports match.
- Workers have **no sidecars**: no LLM proxy, no MCP server, no `eocube` presigning
  helper, and no hub-injected `AWS_*` credentials beyond what you pass.
- Therefore: resolve, sign and filter in the notebook; ship **hrefs** (presigned URLs or
  public HTTPS URLs) to workers. Never send `fs`, `user_fs`, sessions or sockets through
  `client.submit` or inside a graph.

```python
signed_items = ...  # see eocube-data-access
import odc.stac
cube = odc.stac.load(signed_items, bands=["B04", "B08"], chunks={"x": 2048, "y": 2048},
                     crs="EPSG:32634", resolution=10)
ndvi = ((cube.B08 - cube.B04) / (cube.B08 + cube.B04)).mean("time").compute()
```

## Chunking rules of thumb

- COG internal tiles are 512 px; chunks of 1024 to 2048 px per band keep requests few
  and tasks around 10 to 100 MB.
- One time step per chunk along `time` unless you reduce over time immediately.
- Use `overview_level` or a coarser `resolution` for previews; compute at full
  resolution only on the clipped AOI.
- `persist()` intermediate cubes you reuse; `compute()` only final, small results.

## Watch and debug

- Dashboard: task stream, memory per worker, spilled bytes. Red memory means chunks are
  too big or too many are held; reduce chunk size or persist less.
- `client.restart()` clears worker memory after a failed experiment.
- Worker errors about credentials or 403 mean an href was not signed or has expired:
  re-sign from the unsigned items in the notebook (`eocube-data-access`).

## Finish

```python
cluster.close()
```

Idle clusters hold GPU-free but finite worker slots for everybody. Close when the
notebook work is done; the gateway also reaps idle clusters after a timeout.

## Pitfalls

- Signed URLs expire with the signing session: long jobs that outlive the notebook's
  credentials need re-signing, not caching.
- Importing from local project files on workers fails unless the module is on the
  workers too (`client.upload_file`).
- The MCP tools and the `eocube` CLI are notebook-side helpers; orchestrate from the
  notebook, compute on the workers.