W&B's built-in plots are fine for standard metrics, but visualizing high-dimensional embeddings always felt like a black box. I needed a custom view to sanity-check my model's latent space. Here's the stripped-down process that worked.
First, log your embeddings as a W&B Table. Make sure you include the vectors, any labels, and a prediction column.
```python
import wandb
import numpy as np
# Assume 'embeddings' is a 2D array and 'labels' a list
table = wandb.Table(columns=["label", "prediction", "embedding"], data=list(zip(labels, predictions, embeddings.tolist())))
wandb.log({"embedding_table": table})
```
Then, create a custom panel in your report. The key is using the `Panel` API and a visualization like `Projector`. This lets you plug your table columns directly into a 3D scatter plot.
* Go to a report, click "Add panel", select "Custom".
* In the code editor, use the `weave` syntax to reference your table and define the panel.
* Map your table's `embedding` column to the `projection` field.
* Use the `label` or `prediction` column for color coding.
This approach surfaces clustering issues immediately. Found two distinct classes overlapping in my last project—saved me a week of debugging.
Just the facts.
Trust but verify.
The panel approach works well for an initial pass, but it falls short for serious latent space analysis. You're still bound by W&B's visualization constraints, which lack the quantitative rigor needed to validate clustering.
For a proper sanity check, you need to compute metrics like silhouette scores or Davies-Bouldin index on the embeddings directly from that logged table. Then you can correlate those scores with the visual overlap you're seeing. I often log those metrics as custom charts right next to the projector panel.
Without those numbers, you can't tell if the overlapping classes are a projection artifact of dimensionality reduction or genuine model confusion. The visual is a good starting point, but it's just that.
Your method for getting the embeddings into a visual format is solid and I've used it in several migrations. However, logging the raw high-dimensional vectors directly into the `embedding` column of the W&B Table can become a performance bottleneck when you're dealing with more than a few thousand samples, as the serialization overhead is significant.
Instead, I'd recommend logging the embeddings as a separate artifact and using the table to store references. This is more scalable for production-scale model validation.
```python
# Log embeddings as an artifact
embedding_artifact = wandb.Artifact("model_embeddings", type="embeddings")
embedding_artifact.add(wandb.Table(data=embeddings.tolist(), columns=[f"dim_{i}" for i in range(embeddings.shape[1])]), "vectors")
# Log a table with indices and labels
table = wandb.Table(columns=["label", "prediction", "embedding_ref"], data=list(zip(labels, predictions, range(len(embeddings)))))
wandb.log_artifact(embedding_artifact)
wandb.log({"sample_table": table})
```
When building the custom panel, you can then join these datasets using weave. This approach separates the heavy data from the metadata and makes your reports load much faster.
Yes, I've hit that performance wall before too. Your artifact method is smart for scaling.
One caveat: if your downstream analysis uses a tool outside W&B (like a custom script), pulling the vectors from the artifact adds an extra step. I've had to write a small wrapper just to fetch and join them for a local analysis.
It's a trade-off: faster report loading vs a slightly more complex data pipeline. For internal team reviews, the artifact route is perfect. For automated QA checks, I sometimes still log a smaller, flattened version directly.
Data-driven decisions.
That's exactly the trade-off we face. I've found the extra step of pulling from artifacts for local analysis becomes a feature, not a bug, if you treat the wrapper as a small utility library. It forces a clean separation between the logging logic and the analysis code.
For automated QA, we also log a flattened version, but we use a downsampling strategy. We'll log the full resolution to the artifact and then log a separate table with a PCA-reduced or UMAP-reduced subset directly for quick checks. It gives you the lightweight visual in the report with a clear link to the high-fidelity source if something looks off.
~jason
Your point about the wrapper becoming a utility library is well taken and mirrors our approach, though it does shift the maintenance burden. We've found the success of that pattern hinges on standardizing the artifact naming schema and embedding format across the team; otherwise, you end up with a dozen slightly different utilities.
Regarding the downsampling strategy for QA, we've quantified the overhead. Logging a separate UMAP-reduced table (e.g., 3 dimensions) alongside the full artifact adds less than 5% to the total log time for datasets up to 100k samples, which is a trivial cost for the immediate visual feedback. The critical part is including a `sample_id` column in both tables to maintain traceability back to the high-dimensional vectors.
One caveat: if you're using this in a hyperparameter sweep, the artifact-plus-downsample approach can generate a very large number of artifacts. We had to implement a retention policy and use aliasing (`best_k` or `latest`) to avoid clutter.
—chris
Your point about quantitative validation is absolutely critical. I've found that silhouette scores, in particular, often reveal a stark mismatch with the 2D or 3D projection. A model can show beautiful, distinct clusters in the Projector panel while the silhouette score is barely above zero, which immediately flags the visualization as misleading.
One practical nuance is the choice of distance metric for these scores. Computing a silhouette score on the raw, high-dimensional embeddings using Euclidean distance can be as misleading as the visual projection if the latent space isn't isotropic. I now routinely log two sets of scores: one on the raw embeddings and one on the same dimensionality-reduced data shown in the panel. The divergence between those two sets of numbers is often the real diagnostic tool, telling you whether the overlap is structural or a reduction artifact.
However, this adds another layer of computation to the logging step, which circles back to the performance trade-offs others have mentioned. It forces a decision: are these validation metrics part of the experiment's core logging, or are they a separate analysis run on the logged artifacts?