Narwhals Deep Intuition

1. One-Sentence Essence

Narwhals is a thin, dependency-free translation layer that lets you write code once against a subset of the Polars API and have it execute natively on whatever dataframe the caller hands you — pandas, Polars, PyArrow, DuckDB, PySpark, and more — without ever converting between them.

That sentence has two load-bearing words. Translation: Narwhals does not compute anything itself — it forwards your operations to the underlying library’s own engine. Native: if you pass in Polars, the work happens in Polars; if you pass in cuDF, the work happens on a GPU. Narwhals is the universal adapter, not the appliance. Once that clicks, almost everything else about the library follows.

---

2. The Problem It Solved

Picture you maintain a library — say a plotting tool, or a collection of scikit-learn-style estimators. Your users show up with data. Five years ago, “data” meant a pandas DataFrame, and life was simple: you wrote df.groupby(...), called it a day.

Then Polars happened. Then PyArrow tables became common. Then someone wanted to push computation to DuckDB or PySpark or run it on a GPU via cuDF. Suddenly your users are arriving with five different kinds of dataframe, each with its own API, its own quirks, and its own idea of what basic operations even mean.

And the libraries are not subtly different — they are treacherously different. The Narwhals docs lead with a perfect gotcha: 3 in pd.Series([1, 2, 3]) checks the index, while 3 in pl.Series([1, 2, 3]) checks the values. Same syntax, different universe. Or: do a left-join in pandas vs. Polars with overlapping column names and you get different output columns. These aren’t bugs; they’re genuine design divergences. Writing code that’s correct across all of them by hand is a nightmare of if isinstance(df, pd.DataFrame) branches multiplied across every library and every version.

The naive fix — “just convert everything to pandas at the door” — is what the ecosystem actually did for years, and it’s terrible. It throws away Polars’ speed, forces a full in-memory materialization of a DuckDB query that could have stayed lazy, and drops you onto the GPU’s slowest possible path by yanking data off a cuDF frame. You pay a conversion tax and you lose every advantage the user chose their library for.

Narwhals came from Marco Gorelli — a core contributor to both pandas and Polars, working at Quansight Labs — who started it as a weekend project in 2024. His key insight, the one that makes the whole thing tractable: use a strict subset of the Polars API, because Polars expressions transpile cleanly down to pandas, whereas the reverse (pandas’ messier, index-laden API) does not transpile cleanly up to Polars. Pick the cleaner language as your interface and translation becomes feasible. That single decision is why Narwhals works and why earlier attempts at “dataframe-agnostic code” mostly didn’t.

It worked spectacularly. By 2025, Narwhals had something like 30 million downloads a month and had become a required dependency of Altair, Plotly (since v6), Bokeh, Marimo, Shiny, and scikit-lego — almost none of whose users have ever heard of it. That’s the tell of good glue: it’s everywhere and invisible.

---

3. The Concepts You Need

Narwhals borrows its vocabulary almost entirely from Polars, so learning Narwhals is learning the core of the Polars mental model. Here’s the vocabulary the rest of this document leans on.

The four public classes. Narwhals exposes exactly four user-facing types, and the distinction between them is the spine of the whole library:

• DataFrame — an eager, in-memory table with a well-defined row order that is preserved across operations. Backed by pandas, Polars (eager), PyArrow, cuDF, or Modin.
• LazyFrame — a table that makes no assumption about row order and computes nothing until you call .collect(). This is what lets Narwhals sit on top of SQL engines (DuckDB, PySpark, Dask, Ibis, SQLFrame, Daft), where there is no inherent “row 3.”
• Series — a single, 1-dimensional, ordered, in-memory column. There is no such thing as a lazy Series (Polars doesn’t have one, so Narwhals doesn’t either — a recurring pattern).
• Expr (expression) — the most important concept, and the one that makes Narwhals more than a wrapper. We’ll define it precisely below.

Native frame vs. Narwhals frame. Your actual pandas/Polars/etc. object is the native frame. When you wrap it with nw.from_native(df), you get a Narwhals object that holds the native frame and forwards operations to it. nw.to_native(df) unwraps it again. The Narwhals object is a thin shell; the data never moves.

Expression. This is the concept that separates people who use Narwhals from people who understand it. The official, literal definition:

An expression is a function from a DataFrame to a sequence of Series.

nw.col("a") is not a column. It is a recipe that says “given some dataframe, hand me back its column a.” It produces nothing on its own. It only produces a value when you give it to a context — select, with_columns, filter, group_by().agg() — that knows how to evaluate it against a real frame. Hold onto this; Section 5 builds the entire mental model on top of it.

Context. A method that takes expressions and actually runs them: select (produce a new frame with only the expression results), with_columns (produce the current frame plus the results), filter (keep rows where the expression is true), group_by().agg() (evaluate per group).

Eager vs. lazy. Eager = compute immediately, row order is real. Lazy = build up a query plan, compute only at .collect(), row order is undefined until you impose one. The eager/lazy split is not a Narwhals invention — it’s inherited from Polars and from the reality that SQL engines are lazy by nature.

Order-dependence. Some operations only have meaning if rows have a defined order: diff, shift, cum_sum, rolling_mean, is_first_distinct, first, last. These are fine on a DataFrame (order is real) but forbidden on a bare LazyFrame unless you explicitly supply an ordering via .over(order_by=...). This is the single biggest conceptual hurdle for newcomers and gets its own treatment in Sections 5 and 7.

Broadcasting. When you mix a full-length column with a scalar or aggregation — e.g. nw.col("a") - nw.col("a").mean() — the scalar is conceptually stretched to the column’s length so the subtraction works element-wise. Every backend does this differently under the hood; Narwhals normalizes the behavior.

The Narwhals dtype system. Every backend has its own idea of types — pandas has int64 and the newer nullable Int64 and Arrow-backed types; Polars has pl.Int64; PyArrow has its own. For agnostic code to be possible at all, Narwhals must impose a single, canonical type vocabulary that all backends map into and out of. That vocabulary is nw.Int8/16/32/64/128, the unsigned nw.UInt*, nw.Float32/64, nw.String, nw.Boolean, the temporal nw.Date, nw.Datetime(time_unit, time_zone), nw.Duration, nw.Time, the nested nw.List, nw.Array, nw.Struct, and the categorical pair nw.Categorical / nw.Enum. You .cast(nw.Int64) to one of these and Narwhals translates it to whatever the backend calls that type. Crucially, dtypes compare structurally: nw.Int64() == nw.Int64 is True, and nw.Datetime("us") == nw.Datetime is True (a parameterized type matches its base type under ==). This is the dtype layer’s whole reason for existing — it’s the type Rosetta Stone, and Section 4 shows how you actually use it.

Schema. A Schema is an ordered mapping of column name → Narwhals dtype, the structural description of a frame. You read it with df.schema (eager) or — importantly — df.collect_schema(), which is the lazy-safe form that works even on a LazyFrame without triggering computation. Inspecting the schema is how you write truly generic code: loop the columns, branch on dtype.is_numeric() / is_temporal() / is_integer(), act accordingly (Section 4).

Support tiers (full / lazy-only / interchange-level). Not every backend is supported equally, and this distinction is load-bearing when you design a library. There are three tiers. Full API (pandas, Polars, PyArrow, cuDF, Modin) — the complete eager Narwhals surface works. Lazy-only (DuckDB, PySpark, Dask, Ibis, SQLFrame, Daft) — supported as LazyFrames; the lazy expression API works but eager-only operations (extracting a Series, indexing a row) don’t. Interchange-level — for some inputs Narwhals can do little more than read the schema and shuttle data via the Arrow interchange protocol; enough for a plotting library to inspect columns, not enough to run a pipeline. When you say “my library supports backend X,” you must know which tier X sits in, or you’ll promise capabilities that don’t exist.

The stable API (narwhals.stable.v1, narwhals.stable.v2). A versioned, frozen-forever snapshot of the Narwhals API. Code written against a stable namespace is promised never to break. This is Narwhals’ answer to “what happens when Polars or pandas changes underneath us?” — covered fully in Section 8.

Compliant objects (internal). Under the hood, each backend has a PandasLikeDataFrame, ArrowDataFrame, PolarsDataFrame, etc. — internal “Narwhals-compliant” wrappers that know how to translate Narwhals calls into that backend’s native calls. You’ll never touch these directly, but knowing they exist demystifies the architecture (Section 6).

---

4. The Distilled Introduction

This section is everything you’d get from a tutorial, minus the padding. By the end you can write real dataframe-agnostic code.

Installation

pip install narwhals

That’s the whole story, and it’s deliberately remarkable: Narwhals has zero required dependencies. It does not install pandas, Polars, or anything else. It works only with what the caller already has installed, because it only ever touches dataframes the caller passes in. This is the property that lets a library depend on Narwhals without bloating its own dependency tree — the thing that made Plotly and Altair willing to adopt it.

The fundamental loop

Almost all Narwhals code follows the same three-step shape:

import narwhals as nw

def my_function(df_native):
    df = nw.from_native(df_native)      # 1. wrap whatever came in
    result = df.with_columns(...)       # 2. operate using the Narwhals/Polars API
    return result.to_native()           # 3. hand back the same flavor it came in

Wrap, operate, unwrap. If the user gave you pandas, step 3 returns pandas. Polars in, Polars out. cuDF in, cuDF out (and the compute happened on the GPU). You wrote the logic once.

What this replaces (the pain made concrete)

To feel why this matters, here’s the hand-rolled version of a trivial “add a category column” function that supports just two backends:

# The nightmare you're escaping
def add_category(df):
    if isinstance(df, pd.DataFrame):
        df = df.copy()
        df["category"] = df["animal"].str.contains("whale").map(
            {True: "whale", False: "other"})
        return df
    elif isinstance(df, pl.DataFrame):
        return df.with_columns(
            category=pl.when(pl.col("animal").str.contains("whale"))
                       .then(pl.lit("whale")).otherwise(pl.lit("other")))
    else:
        raise TypeError(f"Unsupported: {type(df)}")

Every backend is a new branch. Every API divergence (note pandas needs .map, Polars needs when/then) is a fresh bug surface. Add PyArrow or DuckDB and it grows again. The Narwhals version is the entire thing, for all backends at once:

@nw.narwhalify
def add_category(df):
    return df.with_columns(
        category=nw.when(nw.col("animal").str.contains("whale"))
                   .then(nw.lit("whale")).otherwise(nw.lit("other")))

Pass it pandas, Polars (eager or lazy), PyArrow, DuckDB, PySpark — it works, and each gets its own flavor back. That collapse from “N branches that rot independently” to “one expression” is the whole product.

The @nw.narwhalify shortcut

The wrap/unwrap ceremony is so common that Narwhals offers a decorator that does it for you:

@nw.narwhalify
def my_function(df):
    return df.with_columns(nw.col("a") + 1)

narwhalify automatically calls from_native on the arguments and to_native on the return. Use it for clean leaf functions; use explicit from_native/to_native when you need more control (e.g. mixing eager and lazy paths). Both styles appear constantly in real Narwhals code and the docs present them side-by-side everywhere.

Selecting and transforming columns

The API is Polars’. If you know Polars, you already know this. If you don’t, here’s the core:

# Select specific columns / computed columns
df.select(nw.col("a"), nw.col("b") * 2)

# Add or replace columns, keeping the rest
df.with_columns(
    b_doubled=nw.col("b") * 2,
    a_plus_b=nw.col("a") + nw.col("b"),
)

# Filter rows
df.filter(nw.col("a") > 5)

Note the style: you describe what you want with expressions (nw.col("a") + nw.col("b")), and hand them to a context (select, with_columns, filter). You almost never index or loop. This is the expression-oriented idiom and it’s the entire point — it’s what lets the same code compile down to pandas vectorized ops or a SQL query plan.

Aggregation and group-by

df.group_by("category").agg(
    nw.col("price").mean().alias("avg_price"),
    nw.col("price").max().alias("max_price"),
    nw.len().alias("n"),
)

Group-by is where Polars’ expression model shines hardest over pandas, and Narwhals inherits that. You can even write conditional aggregations like (nw.col("c") > nw.col("b").mean()).max() inside an .agg() — something that’s painful and slow in raw pandas. (There’s an important performance caveat here; see Section 7.)

Common expression operations you’ll reach for daily

nw.col("a").alias("b")            # rename
nw.col("a").cast(nw.Int64)        # change dtype
nw.col("a").is_null()             # null mask
nw.col("a").fill_null(0)
nw.col("a").mean() / .sum() / .std() / .min() / .max()
nw.col("a").abs() / .round(2)
nw.col("name").str.to_uppercase()       # string namespace
nw.col("ts").dt.year()                  # datetime namespace
nw.when(nw.col("a") > 0).then(1).otherwise(-1)   # conditional
nw.sum_horizontal("a", "b", "c")        # across columns in a row

Expressions have namespaces for typed operations: .str for strings, .dt for datetimes, .cat for categoricals, .list and .struct for nested data. This mirrors Polars exactly.

Series, when you need an actual value

When you need to pull a concrete value out — say, to compute and store a mean during a fit step — you need eager evaluation and real Series:

df = nw.from_native(df_native, eager_only=True)   # refuse lazy input
mean_a = df["a"].mean()                            # a real Python float

Passing eager_only=True to from_native (or narwhalify) says “I need to materialize values; reject anything lazy.” Use this consciously — it’s the boundary where you give up the ability to stay lazy.

Types and schema — writing genuinely generic code

Sooner or later your agnostic code needs to know about the data’s structure: cast a column, or behave differently for numeric vs string columns. This is where the dtype system (Section 3) earns its keep.

Casting is uniform across every backend:

df.with_columns(nw.col("id").cast(nw.Int64), nw.col("ts").cast(nw.Datetime("us", "UTC")))

And the schema lets you write code that adapts to whatever frame arrives — the essence of “generic”:

df = nw.from_native(df_native)
schema = df.collect_schema()           # lazy-safe; works on LazyFrame too

# Operate only on numeric columns, agnostically:
numeric = [name for name, dtype in schema.items() if dtype.is_numeric()]
df = df.with_columns(nw.col(numeric).fill_null(0))

Two things to absorb here. First, use collect_schema(), not .schema, in code that might see a lazy frame — .schema can force computation or be unavailable, while collect_schema() is the portable choice. Second, branch on dtype predicates (is_numeric(), is_temporal(), is_integer()), not on exact-type equality — they’re the robust, future-proof way to ask “what kind of column is this?” For column selection by type you can also skip the manual loop entirely with selectors: df.select(nw.selectors.numeric()).

A subtle dtype gotcha worth flagging now: types compare structurally, and a parameterized temporal type is not interchangeable with its base in a set. nw.Datetime("us") == nw.Datetime is True (good for “is this any datetime?”), but nw.Datetime("us") in {nw.Datetime} is False in the modern API. When you mean “is this a datetime of any unit?”, use ==, not set membership.

Staying lazy when you can

The flip side: if your transformation doesn’t need concrete values, don’t force eagerness, and your code will stay lazy all the way through for backends that support it. The classic example is a scikit-learn-style transformer: the fit step must be eager (it computes and stores means/std-devs), but transform can stay lazy:

class StandardScaler:
    @nw.narwhalify(eager_only=True)
    def fit(self, df):
        self._means = {c: df[c].mean() for c in df.columns}
        self._stds  = {c: df[c].std()  for c in df.columns}
        self._cols  = df.columns
        return self

    @nw.narwhalify   # no eager_only -> stays lazy if input was lazy
    def transform(self, df):
        return df.with_columns(
            (nw.col(c) - self._means[c]) / self._stds[c]
            for c in self._cols
        )

Pass this a Polars LazyFrame and transform returns a LazyFrame — the scaling is fused into the query plan and nothing computes until .collect(). Pass it pandas and it runs eagerly. Same code. This example is the Rosetta Stone of why Narwhals exists; reread it until the lazy-preservation clicks.

Converting between libraries (when you truly must)

Narwhals’ philosophy is “don’t convert,” but it does provide escape hatches: df.to_pandas(), df.to_arrow(), df.to_polars(), and nw.from_native(..., backend=...). Reach for these only at genuine boundaries (e.g. handing data to a library that only speaks pandas). Every conversion materializes data and forfeits the native-execution advantage.

Which import to use

• import narwhals as nw — the main namespace, latest API. Use while prototyping.
• import narwhals.stable.v2 as nw — the frozen stable API. Use for anything you ship.

The two are nearly identical in surface; the difference is the promise attached, which Section 8 explains.

---

5. The Mental Model

Three ideas. Internalize these and you can predict almost any Narwhals behavior without the docs.

Core Idea 1: Narwhals is a transpiler, not an engine.

Narwhals computes nothing. Every operation you express is forwarded to the native library’s own implementation. nw.col("a") + 1 against a pandas frame becomes, roughly, df.loc[:, "a"] + 1 executed by pandas; against a DuckDB relation it becomes part of a SQL query; against cuDF it becomes a CUDA kernel.

What this predicts:

• Performance is the backend’s performance. Narwhals doesn’t make pandas faster or Polars slower. If your pandas code is slow, Narwhals-wrapped pandas is exactly as slow (the docs note overhead is negligible, sometimes even negative because Narwhals steers you onto faster native paths).
• Behavior is mostly the backend’s behavior, except where Narwhals deliberately normalizes a known divergence (like the in operator or join-column semantics). Narwhals smooths the documented sharp edges; it does not, and cannot, paper over every numerical quirk of every engine.
• If a backend can’t do something, Narwhals can’t fake it. There’s no lazy Series because Polars has none. Narwhals’ API is constrained by the intersection of what its backends can natively express.

Core Idea 2: An expression is a function from a DataFrame to a sequence of Series — and it does nothing until a context runs it.

This is the literal internal definition, and treating it literally is the key. nw.col("a") is lambda df: [df["a"]]. nw.col("a") + 1 is lambda df: [s + 1 for s in inner(df)]. Expressions are lazy little function objects that get composed and only executed when handed to select, with_columns, filter, or agg.

What this predicts:

• You can build expressions before you have data, store them, pass them around, compose them. They’re just deferred functions.
• The same expression works in eager and lazy contexts, because it’s defined as a transformation, not a computation. This is why your transform method above didn’t care whether it got a DataFrame or a LazyFrame.
• Expressions can fan out. nw.col("a", "b") returns two Series (hence “a sequence of Series”). nw.selectors.numeric() returns however many numeric columns exist — a count that depends on the input frame. This is why Narwhals internally tracks an expression’s “expansion kind.”
• All columns in one expression must come from the same frame. You can’t blend columns from two different dataframes inside a single expression — there’s no defined alignment. (Reread that; it’s a common early mistake for people coming from pandas, where the index silently aligns things.)

Core Idea 3: Row order is a capability, not a given — and lazy frames don’t have it.

A DataFrame is an ordered, in-memory thing: “row 3” means something, so diff, shift, and cum_sum are well-defined. A LazyFrame may be backed by a SQL engine where rows are an unordered set — “row 3” is meaningless. Therefore Narwhals forbids order-dependent operations on lazy frames unless you explicitly say what defines the order, via .over(order_by="some_column").

# Fine on a DataFrame (order is real):
df.with_columns(nw.col("a").cum_sum())

# Required form on a LazyFrame (you must declare the order):
lf.with_columns(nw.col("a").cum_sum().over(order_by="i"))

What this predicts:

• Code that works on pandas may error on DuckDB — not because of a bug, but because you relied on implicit row order that doesn’t exist in a SQL engine. Narwhals catches this at expression-build time with a loud, clear error rather than silently returning garbage.
• Writing for the lazy case is stricter but more portable. If you always supply order_by for order-dependent ops, your code runs on everything — eager and lazy alike.
• This is a feature, not a nuisance. The strictness is Narwhals refusing to let you write code that’s silently wrong on half its backends. The annoyance you feel is the bug you didn’t ship.

---

6. The Architecture in Plain English

Let’s trace exactly what happens when you run df.select(nw.col("a") + 1).

Layer 0 — the native frame. You start with, say, a real pandas.DataFrame. Narwhals never loses sight of this object; it’s stored and ultimately does all the actual work.

Layer 1 — the Narwhals public object. nw.from_native(df_pd) returns a narwhals.DataFrame. This is a thin shell. Internally it holds a compliant frame (._compliant_frame), which for pandas input is a PandasLikeDataFrame — a Narwhals-internal object that wraps the real pandas frame (accessible as ._native_frame) and knows how to speak both Narwhals and pandas.

Layer 2 — the expression gets parsed for this backend. nw.col("a") + 1 is a backend-agnostic narwhals.Expr. Before it can run, .select calls expr._to_compliant_expr(namespace), passing the backend’s compliant namespace — for pandas, a PandasLikeNamespace. This namespace is the catalog of “how does this backend implement each Narwhals operation.” The result is a PandasLikeExpr: a concrete object that knows how to evaluate against a pandas-backed frame.

Layer 3 — evaluation. Recall an expression is a function from a frame to a sequence of Series. The compliant expression has a _call method that is that function. select hands it the compliant frame; _call returns a list of compliant Series; select assembles them into a new PandasLikeDataFrame.

Layer 4 — unwrap. .to_native() reaches into the compliant frame’s ._native_frame and hands you back a real pandas DataFrame. Done. The data lived in pandas the entire time; Narwhals just choreographed the calls.

The crucial architectural fact: each backend lives in its own subpackage (narwhals._pandas_like, narwhals._arrow, narwhals._polars, narwhals._duckdb, …), each implementing the same internal CompliantExpr / compliant-frame protocols. The top-level narwhals.dataframe and narwhals.expr modules are pure dispatch — they decide which backend to route to and let that backend do the translation. Adding a new backend means writing a new compliant subpackage that satisfies the protocol; the public API doesn’t change. This plugin architecture is why the backend list keeps growing without the user-facing surface shifting.

Two genuinely clever architectural touches worth knowing because they explain otherwise-baffling behavior:

Expression nodes and metadata. Internally an expression is a tuple of nodes (expr._nodes), each a single operation: nw.col("a").abs().std(ddof=1) is the nodes (col(a), abs(), std(ddof=1)). Alongside the nodes, Narwhals tracks metadata: is this elementwise? does it preserve length? is it scalar-like? how many order-dependent ops does it contain (n_orderable_ops)? This metadata is how Narwhals enforces the order-dependence rules before execution and how it knows when to broadcast.

Elementwise push-down. SQL is picky: abs(sum(a)) over (partition by b) is illegal, but abs(sum(a) over (partition by b)) is fine. Polars happily accepts both orderings. To bridge this, Narwhals automatically rewrites your expression, pushing over nodes back past elementwise operations — so nw.col("a").sum().abs().over("b") is silently reordered to apply over before abs. This is the one place Narwhals does “query optimization,” and it does it only because otherwise valid Polars-style code would be rejected by SQL backends. They keep this deliberately minimal; general query optimization is explicitly out of scope (that’s the backend engine’s job).

---

7. The Things That Bite You

Each of these connects to a mental-model idea. None are bugs; all are consequences of what Narwhals is.

1. Order-dependent ops blow up on lazy frames. You’d expect: lf.with_columns(nw.col("a").cum_sum()) to just work, like it does on pandas. What happens: a loud error, because a LazyFrame (Core Idea 3) has no defined row order. Fix: add .over(order_by="some_col"). Write your library code in the strict lazy-compatible form from the start and you sidestep this entirely.

2. You can’t mix columns from two different frames in one expression. You’d expect (coming from pandas): index alignment to silently make df1_col + df2_col work. What happens: it’s undefined and disallowed — an expression is a function of one frame (Core Idea 2). Fix: join the frames first, then operate on the single resulting frame.

3. Complex group-by aggregations silently fall off the fast path in pandas. You’d expect: any .agg() you can write to run efficiently. What happens: simple aggregations (nw.col("b").mean()) translate to fast native df.groupby("a").agg({"b": ["mean"]}). But a complex aggregation falls back to pandas’ GroupBy.apply, which is slow — and Narwhals raises a UserWarning telling you so. Fix: heed the warning and refactor toward a simple aggregation. The warning is Narwhals doing you a favor, not nagging.

4. eager_only is a one-way door you sometimes forget to open — or forget to close. You’d expect: to grab a Series value whenever. What happens: try to index out a Series or pull a scalar from a lazy frame and it won’t work — there’s no lazy Series (Core Idea 1). Conversely, slapping eager_only=True everywhere kills your ability to stay lazy on SQL backends. Fix: be deliberate. Eager only at the boundaries where you genuinely materialize values (like a fit step).

5. Narwhals is a subset — the method you want may not exist. You’d expect: full Polars (or pandas) API coverage. What happens: Narwhals intentionally implements only the subset expressible across its backends. Some niche Polars method may simply be absent. Fix: check the API Completeness tables in the docs before designing around a method; restructure, or drop to to_native() for a backend-specific escape hatch (at the cost of agnosticism).

6. The pandas Index is deliberately ignored — and that surprises pandas natives. You’d expect: index-based behavior (alignment, 3 in series checking the index, etc.). What happens: Narwhals adopts the Polars worldview, where there is no index. It neither relies on nor preserves pandas’ index semantics. Fix: stop thinking in indexes. If you need the row identity, make it an explicit column.

7. strict= was renamed to pass_through= — old tutorials will mislead you. You’d expect: from_native(df, strict=False) from a 2024 blog post to be current. What happens: since Narwhals 1.13 the parameter is pass_through=True (because strict=False confused everyone about what it did). Fix: use pass_through; it works consistently across namespaces.

8. .over(order_by=...) vs .first(order_by=...) produce different shapes. You’d expect: one canonical way to make an order-dependent aggregation lazy-safe. What happens: nw.col("a").first().over(order_by="i") yields a full-length column (the value broadcast), while nw.col("a").first(order_by="i") yields a scalar. In a group-by .agg() you want the scalar form. Fix: in group-by aggregations prefer passing order_by directly to the reduction.

9. A backend you “support” turns out to be interchange-level only. You’d expect: if from_native accepts it, the full API works. What happens: some inputs are supported only at the interchange tier (Section 3) — you can read the schema and shuttle Arrow data, but a real transformation pipeline raises. This historically bit people with Ibis and older DuckDB integration. Fix: know each backend’s tier before designing around it; if you only need schema/columns (e.g. a plotting tool), interchange-level is plenty — if you need to transform, confirm the backend is tier-1 or tier-2.

10. Empty or all-null columns infer a type you didn’t expect. You’d expect: an empty object-dtype pandas Series to stay Object. What happens: since Narwhals 1.24.1 an empty/all-null object-dtype pandas Series is inferred as String, not Object — a deliberate normalization that nonetheless surprises. More broadly, dtype inference of degenerate columns is one of the genuinely backend-divergent corners (Section 11.3). Fix: cast explicitly when a column’s type matters and the data might be empty; don’t rely on inference at boundaries.

---

8. The Judgment Calls

1. import narwhals as nw vs import narwhals.stable.v2 as nw. Main namespace = latest API, may change. Stable = frozen, promised never to break. Use main while prototyping, where iteration speed matters and breakage is cheap. Use stable.v2 for anything you ship — especially a library, where your users’ code breaking because you upgraded Narwhals is unacceptable. The signal: “am I publishing this?” → stable. The whole reason the stable API exists is so a library author can pin behavior forever; not using it in shipped code throws away Narwhals’ headline guarantee.

2. stable.v1 vs stable.v2. v1 is the original frozen API; v2 is the newer one (Narwhals ≥ 2.0). Both are maintained indefinitely and can coexist in one project. Stay on v1 if your existing project works and you don’t need new features — moving costs you a higher minimum Narwhals version for no gain. Choose v2 for new projects. The signal: greenfield → v2; working legacy → leave it.

3. @nw.narwhalify vs explicit from_native/to_native. The decorator is concise; the explicit form gives control. Use the decorator for clean leaf functions with uniform input/output. Use explicit calls when you need to pass options like eager_only on some paths but not others, when only some arguments are frames, or when control flow branches on frame type. The signal: if you find yourself fighting the decorator’s all-or-nothing wrapping, go explicit.

4. Stay lazy vs force eager. Stay lazy (omit eager_only) whenever your logic is pure transformation — it lets SQL/Spark/Dask backends fuse your work into one optimized plan. Force eager only when you must read concrete values back into Python (computing a threshold, a mean to store, a row count to branch on). The signal: “do I need a Python value right now, or just a transformed frame?” Defaulting to eager out of habit is the most common way people leave performance on the table.

5. Should your library even depend on Narwhals? Yes if you consume dataframes and want to support more than pandas without a conversion tax or a dependency explosion — its zero-dependency design means near-zero cost to your users. No if you’re an end user writing a one-off script against a single known library: then just use that library directly. Narwhals is explicitly aimed at library maintainers, not end users. The signal: “will code I don’t control hand me dataframes of types I don’t know in advance?” → Narwhals.

6. Normalize a behavior via Narwhals, or drop to to_native()? When you hit something Narwhals doesn’t cover, you can unwrap to the native frame and do backend-specific work. Stay in Narwhals for anything you want to remain agnostic. Drop to native only at a deliberate, isolated boundary, accepting that that code path is now backend-specific. The signal: every to_native() mid-pipeline is a small surrender of agnosticism — fine occasionally, a smell if frequent.

7. Which support tier does a backend actually have — and does your feature need more? Backends fall into three tiers (Section 3): full eager API (pandas, Polars, PyArrow, cuDF, Modin), lazy-only (DuckDB, PySpark, Dask, Ibis, SQLFrame, Daft), and interchange-level (schema inspection and Arrow data-shuttling only). Claim full support only for the tier-1 backends if your code extracts Series, indexes rows, or otherwise needs eager features. Claim lazy support for tier-2 only if your whole pipeline stays lazy-clean (no order-dependence without over, no Series extraction). The signal: match the minimum tier your feature requires against the tier each backend offers — a plotting tool that only reads a schema can support far more backends (interchange-level) than a transformer that must compute a Series (full API only). Misjudging this is how you ship “supports DuckDB” and then crash the first time someone passes one.

8. Which backends do you actually test against? Narwhals supports many backends, but your library should test the ones your users use. Test pandas + Polars at minimum (the two dominant eager backends with the nastiest divergences). Add DuckDB or PySpark if you advertise lazy/SQL support — because the order-dependence rules mean lazy backends exercise genuinely different code paths. The signal: “have I claimed support for a lazy backend?” If yes, you must test a lazy backend, or you haven’t really tested the hard part.

9. Handle a deprecation yourself, or let Narwhals absorb it? A core selling point: Narwhals tests against pandas and Polars nightly builds and absorbs their churn internally. Let Narwhals handle it — that’s the entire value proposition; do not write version-sniffing if parse_version(...) branches against the underlying libraries. Handle it yourself essentially never, unless you’ve hit a genuine gap. The signal: if you’re writing version checks against pandas/Polars inside Narwhals code, you’re fighting the tool.

10. Modin/Dask for scale, or DuckDB/PySpark? All are Narwhals-supported paths to bigger-than-pandas data, but they’re different beasts: Modin is a near-drop-in parallel pandas (full eager API), while DuckDB/PySpark are lazy SQL engines (lazy-only support). The signal: if your code is eager and pandas-shaped, Modin/cuDF slot in with least friction; if you’re willing to write lazy-clean code, DuckDB/PySpark scale further. Narwhals lets you keep your options open, which is itself the point.

---

9. The APIs That Actually Matter

Grouped by what you’re trying to do. This is the 80/20.

Entering and leaving Narwhals:

• nw.from_native(obj, *, eager_only=False, series_only=False, pass_through=False, backend=...) — wrap. eager_only rejects lazy input; series_only expects a Series; pass_through=True returns unrecognized objects untouched instead of erroring.
• nw.to_native(obj) / obj.to_native() — unwrap to the original flavor.
• @nw.narwhalify(**kwargs) — auto-wrap/unwrap a function; forwards kwargs to from_native.

Building expressions:

• nw.col("a", "b", ...) — select columns. The foundational expression.
• nw.lit(value) — a literal/constant as an expression.
• nw.nth(0, 1) — columns by position.
• nw.selectors.numeric() / .string() / .by_dtype(...) — select columns by property.
• nw.when(cond).then(x).otherwise(y) — conditional logic.
• nw.sum_horizontal(...), nw.min_horizontal(...), nw.max_horizontal(...), nw.all_horizontal(...), nw.any_horizontal(...) — operate across columns within a row.

Contexts (where expressions run):

• df.select(*exprs) — new frame, only the results.
• df.with_columns(*exprs, **named_exprs) — current frame plus results.
• df.filter(expr) — keep rows where true.
• df.group_by(*keys).agg(*exprs) — aggregate per group.
• df.sort(by, descending=...), df.join(other, on=, how=), df.unique(...), df.head(n) / df.tail(n).

Expression methods you’ll use constantly:

• Reductions: .mean() .sum() .min() .max() .std() .median() .count() .n_unique()
• Element-wise: .abs() .round(n) .cast(dtype) .fill_null(v) .is_null() .is_in([...]) .clip(lo, hi)
• Naming: .alias("name"), .name.suffix("_x")
• Order-dependent (need over(order_by=...) when lazy): .diff() .shift(n) .cum_sum() .rolling_mean(window) .first() .last()
• Windowed: .over("partition_col") / .over(order_by="ts") / .over("group", order_by="ts")

Typed namespaces on expressions and Series:

• .str — .to_uppercase(), .contains(pat), .replace(...), .strip_chars(), .len_chars(), .slice(...)
• .dt — .year() .month() .day() .hour(), .to_string(fmt), .total_seconds()
• .cat, .list, .struct — categorical and nested-data operations.

Lazy control:

• lf.collect() — execute the plan, materialize a DataFrame.
• lf.collect(backend=...) — execute and convert in one step.

Inspection:

• df.columns, df.collect_schema() (lazy-safe schema), df.schema (eager), df.shape (eager), nw.get_native_namespace(df), df.to_pandas() / .to_arrow() / .to_polars().

Types:

• Canonical dtypes: nw.Int8/16/32/64/128, nw.UInt*, nw.Float32/64, nw.String, nw.Boolean, nw.Date, nw.Datetime(unit, tz), nw.Duration, nw.List, nw.Array, nw.Struct, nw.Categorical, nw.Enum.
• Cast with nw.col("x").cast(nw.Int64).
• Branch with dtype predicates: dtype.is_numeric() / is_integer() / is_temporal() / is_float() / is_nested().
• Compare with == (structural, base-type-aware), not set membership, for parameterized temporal types.

---

10. How It Breaks

Symptom: OrderDependentExprError (or similar) on a lazy frame. Root cause: you used diff/shift/cum_sum/first on a LazyFrame without declaring an order (Core Idea 3). Diagnose: look for order-dependent methods not followed by .over(order_by=...). Fix: add the order_by, or operate on an eager DataFrame if order is genuinely physical.

Symptom: UserWarning about slow group-by during a pandas run. Root cause: a complex aggregation fell back to GroupBy.apply (Section 7.3). Diagnose: read the warning — it names the penalty. Fix: refactor the aggregation into a simple per-column reduction.

Symptom: An AttributeError for a method you “know” exists. Root cause: it’s outside Narwhals’ supported subset, or you’re on a stable namespace that doesn’t expose it (Section 7.5). Diagnose: check the API Completeness tables and confirm which namespace you imported. Fix: find the supported equivalent, or drop to to_native() for that step.

Symptom: Results differ between pandas and Polars backends. Root cause: a behavioral divergence Narwhals doesn’t normalize — null vs NaN handling, default sort stability, dtype inference of empty/all-null columns. Diagnose: isolate the operation, run it on each native backend directly, compare. Fix: make the intent explicit (cast dtypes, specify null handling, sort with a tiebreaker). Remember Core Idea 1: Narwhals normalizes documented sharp edges, not every numeric quirk.

Symptom: A scalar broadcast didn’t happen the way you expected. Root cause: mixing length-preserving and scalar-like expressions where the broadcasting rules (and the elementwise push-down rewrite) interact. Diagnose: inspect expr._metadata to see is_scalar_like / preserves_length, and expr._nodes to see the operation order after any push-down. Fix: restructure so the broadcast intent is unambiguous; wrap aggregations explicitly.

General debugging workflow:

1. nw.to_native(df) and inspect the real underlying object — the bug is usually in the backend, not Narwhals.
2. Run the same operation directly in the native library to see if Narwhals or the backend is responsible.
3. For expression confusion, print expr._nodes and expr._metadata — they reveal exactly what Narwhals thinks your expression does.
4. Check whether you’re eager or lazy (type(df)), and whether an order-dependence rule applies.
5. Confirm your import (narwhals vs narwhals.stable.v1 vs v2) — subtle behavior differences live there.
6. Reproduce on the smallest possible frame across two backends; divergence localizes the cause instantly.

---

11. The Downsides / Disadvantages

Honest accounting. Narwhals is excellent at its job, but adopting it signs you up for real, durable costs.

1. You’re permanently constrained to the intersection of what every backend can do. Where it comes from: Core Idea 1 — Narwhals can only expose operations it can faithfully forward to all (or a declared subset of) its backends. A brilliant Polars-only feature can’t be in the agnostic API if DuckDB can’t express it. What it costs: you write to a deliberately smaller API than any single library offers. Power users of Polars will occasionally feel handcuffed. Dealbreaker when: your application is single-backend and leans hard on that backend’s unique features — then Narwhals is pure overhead and constraint for no benefit. Live with it when: breadth of input matters more than depth on any one engine.

2. It’s a translation layer, so debugging now spans two libraries. Where it comes from: the architecture (Section 6) — your call passes through Narwhals’ dispatch, a compliant wrapper, and the native engine. What it costs: when something’s wrong, you have to figure out whether the fault is yours, Narwhals’, or the backend’s. Stack traces traverse internal _pandas_like/_arrow machinery. The mental overhead of “which layer is lying to me” is real, especially for contributors. Dealbreaker when: your team has no appetite for occasionally reading a stranger’s library internals. Live with it when: the alternative (hand-maintaining N backends) is obviously worse — which it usually is.

3. The agnosticism is leaky exactly where libraries differ most. Where it comes from: Narwhals normalizes known, documented divergences but cannot erase every numerical and semantic difference between independently-built engines. What it costs: you can write code that passes on pandas and subtly misbehaves on Polars (null vs NaN, sort stability, float edge cases). “Write once, run anywhere” is true for most operations and aspirational for the long tail. What people think mitigates it but doesn’t: “I tested it on pandas.” Testing one backend tells you almost nothing about the others — the whole point is that they differ. You must test across backends or you haven’t tested agnosticism at all.

4. The order-dependence model imposes a real cognitive tax. Where it comes from: Core Idea 3 — supporting orderless SQL engines means the API must forbid implicit row order on lazy frames. What it costs: developers steeped in pandas, where row order is always implicitly there, must internalize a stricter discipline (over(order_by=...) everywhere it matters). It’s a genuine relearning, and it generates errors during the learning curve. Dealbreaker when: essentially never — but budget for the ramp-up. This is the price of lazy/SQL support, and if you only ever use eager backends you’re paying conceptual rent on a room you don’t enter.

5. You inherit the dependency-version politics of every backend. Where it comes from: Narwhals tracks pandas and Polars nightly and periodically must bump minimum supported backend versions (an explicit exception to its stability promise). What it costs: over time, staying current with Narwhals can force you to raise the floor on pandas/Polars versions, which can ripple through your users’ environments. The stability promise covers the Narwhals API surface, not the version matrix beneath it. Live with it when: you’re already keeping reasonably current — which you should be.

6. It’s young, and its world moves fast. Where it comes from: a 2024 project riding two libraries (pandas, Polars) that themselves change quickly, plus a growing roster of backends. What it costs: documentation, tutorials, and blog posts go stale fast (the strict→pass_through rename is already a trap for anyone reading 2024 material). Behavior and supported-backend lists shift between versions in the main namespace. What genuinely mitigates it: the stable API — if you pin to stable.v2, most of this churn stops mattering. That mitigation is real, which is why it’s the headline feature rather than a footnote.

7. The benefit is invisible, which makes it politically hard to justify. Where it comes from: good glue disappears. ~30M downloads a month and most consumers have never heard of it. What it costs: convincing a team to adopt a dependency whose entire value is “things you’ll never notice keep working” is a harder sell than adopting something with a flashy demo. There’s no dashboard that lights up to show Narwhals earning its keep. Live with it when: you’ve felt the pain of the alternative — anyone who has hand-maintained multi-backend support needs no convincing.

---

12. The Taste Test

Good vs. bad, at a glance.

Bad — converts at the door, throwing away everything:

def process(df):
    df = pd.DataFrame(df)        # forces materialization, kills laziness, GPU, speed
    return df.groupby("k")["v"].mean().reset_index()

Good — wraps, stays native, stays lazy:

@nw.narwhalify
def process(df):
    return df.group_by("k").agg(nw.col("v").mean())

Bad — version-sniffing the underlying library (the exact thing Narwhals exists to delete):

if parse_version(pd.__version__) < parse_version("2.0"):
    ...

Good — let Narwhals absorb backend churn; pin the Narwhals stable API instead.

Bad — eager_only=True slapped on everything out of habit, silently forfeiting lazy execution on SQL backends. Good — eager only at genuine materialization boundaries (a fit step), lazy everywhere else.

Bad — order-dependent op with no declared order, shipped without testing a lazy backend:

lf.with_columns(running=nw.col("x").cum_sum())   # explodes on DuckDB

Good — order made explicit, works eager and lazy:

lf.with_columns(running=nw.col("x").cum_sum().over(order_by="ts"))

Bad — to_native() scattered through the middle of a pipeline to reach for backend-specific methods, quietly destroying agnosticism. Good — agnostic from wrap to unwrap, with native escape hatches isolated to one clearly-commented boundary if truly needed.

Red flags in a code review: any isinstance(df, pd.DataFrame) branching; conversions in the hot path; import narwhals as nw (not stable) in shipped library code; no test matrix across at least pandas + Polars; order-dependent expressions with no over(order_by=...) in code that claims lazy support; reliance on the pandas index.

Green flags: a clean wrap/operate/unwrap shape; stable.v2 in published code; an explicit multi-backend test matrix; expressions composed and passed around as values; conscious, sparing use of eager_only; zero version-sniffing of the underlying libraries.

---

13. Where to Go Deeper

• The official docs — “How it works” page. The single highest-value page in the project. It walks the expression-as-function definition, the compliant-object dispatch, expression nodes and metadata, broadcasting, and the elementwise push-down. Read it once you’re comfortable using Narwhals and want to understand why it behaves as it does. (narwhals-dev.github.io/narwhals/how_it_works/)

• The “Perfect backwards compatibility policy” page. Short, and essential before you ship anything. Explains the stable-API model (the Rust-editions analogy) and lists every main vs stable.v1/v2 difference — including the traps like strict→pass_through. (narwhals-dev.github.io/narwhals/backcompat/)

• The “Order-dependence” concept page. The clearest treatment of the one model that trips everyone up. Read it the first time a lazy frame rejects your expression. (narwhals-dev.github.io/narwhals/concepts/order_dependence/)

• The “Complete example” tutorial (dataframe-agnostic StandardScaler). The best single worked example — it shows eager fit + lazy transform in one class, which is the whole philosophy in miniature. Type it out yourself. (narwhals-dev.github.io/narwhals/basics/complete_example/)

• Marco Gorelli’s conference talks (PyData Berlin/London 2025, PyCon Italy/Lithuania 2024). From the author. Best for the why it exists and the ecosystem story — how Altair, Plotly, scikit-lego, and others adopted it. Search his name plus “Narwhals.”

• The Wes McKinney interview with Marco Gorelli (Dec 2025). Excellent for design rationale, especially the decision to use a subset of Polars because Polars transpiles cleanly down to pandas but not vice versa. The closest thing to the design diary.

• A hands-on project that builds real understanding: take any small single-backend pandas utility you’ve written and make it dataframe-agnostic with Narwhals, then add a test that runs it against pandas, Polars (eager), Polars LazyFrame, and DuckDB. The moment your lazy test forces you to add over(order_by=...), the whole model lands permanently.

---

14. The Final Verdict

Narwhals is the rare piece of infrastructure that is both unglamorous and genuinely important, and the reason it’s important is that it solved a problem the right way instead of the easy way. The easy way — “convert everything to pandas” — was what the ecosystem did for years, and it was a slow, lossy tax that punished anyone for choosing a better dataframe library. Narwhals’ founding insight, that you should pick the cleaner API (Polars’) as your lingua franca because clean transpiles down to messy but not the reverse, is the kind of decision that looks obvious only after someone makes it. Everything good about the library descends from that one call.

What it gets profoundly right: it computes nothing. By being a transpiler rather than an engine, it stays honest — it can’t be slower than the backend, can’t silently materialize your lazy query, can’t quietly drag your GPU data onto the CPU. The zero-dependency design is the second thing it nails, and it’s not a vanity metric: it’s precisely why Plotly and Altair were willing to make it a required dependency, and why ~30 million downloads a month flow through code whose authors have never heard the name. The third triumph is the stable API — a credible, Rust-editions-style promise that lets a library author write once and not get woken up by someone else’s breaking change. For a tool aimed at maintainers, that promise is the product.

What it costs you is the shadow of those same strengths. You’re confined to the intersection of what every backend can express, so you trade depth for breadth. The agnosticism is leaky exactly where engines genuinely differ, so “test on pandas and ship” is a quiet lie you must resist. And the order-dependence discipline that makes SQL backends possible is a real cognitive tax that eager-only users pay without collecting the benefit. None of these are flaws to fix; they’re the price of admission, and the price is fair.

Who should reach for it: library and tool maintainers who consume dataframes they don’t control and want to support more than one backend without a conversion tax or a dependency explosion. That’s the bullseye, and within it Narwhals is close to a no-brainer. Who shouldn’t: end users writing application code against a single, known dataframe library. If you’ve already chosen Polars and you’re building an app, use Polars — Narwhals would only constrain you for a portability you don’t need.

What to believe walking away: believe that Narwhals is not a dataframe library and will never make your code faster than its backend — it’s an adapter, full stop. Don’t believe that wrapping in Narwhals makes your code automatically correct across backends; it makes it expressible across backends, and correctness still demands a real test matrix. And when someone says “we made our library dataframe-agnostic,” what they almost certainly mean, in 2026, is “we added Narwhals” — because for this specific problem, it has quietly become the answer.

One last calibration, because you’ll inevitably be asked “why not Ibis?” They solve adjacent but different problems. Ibis has its own expression API that compiles to backend-native SQL across 20+ database engines — it’s for analysts and engineers who want one query language over many warehouses, and it replaces what you’d otherwise write. Fugue is for scaling Python/SQL workloads onto Spark, Dask, or Ray. Narwhals is the lightweight one that exposes a subset of an API you may already know (Polars’), adds zero dependencies, and hands you back your original dataframe flavor — built specifically so a library author can accept whatever a user brings. If you’re choosing a query language, look at Ibis; if you’re scaling a pipeline, look at Fugue; if you’re a tool builder who wants to accept anyone’s dataframe without a conversion tax, Narwhals is the one.

The hard-won line: the best infrastructure is the kind nobody notices, and Narwhals is so good at disappearing that its biggest practical risk is a maintainer ripping it out because they can’t see what it’s doing for them. Don’t be that maintainer. The silence is the feature working.

---

The ideas are mine. The writing is AI assisted