Add support for source maps (#17775)

Closes #13694
Closes #13591

# Source Maps Support for Tailwind CSS

This PR adds support for source maps to Tailwind CSS v4 allowing us to
track where styles come from whether that be user CSS, imported
stylesheets, or generated utilities. This will improve debuggability in
browser dev tools and gives us a good foundation for producing better
error messages. I'll go over the details on how end users can enable
source maps, any limitations in our implementation, changes to the
internal `compile(…)` API, and some details and reasoning around the
implementation we chose.

## Usage

### CLI

Source maps can be enabled in the CLI by using the command line argument
`--map` which will generate an inline source map comment at the bottom
of your CSS. A separate file may be generated by passing a file name to
`--map`:

```bash
# Generates an inline source map
npx tailwindcss -i input.css -o output.css --map

# Generates a separate source map file
npx tailwindcss -i input.css -o output.css --map output.css.map
```

### PostCSS

Source maps are supported when using Tailwind as a PostCSS plugin *in
development mode only*. They may or may not be enabled by default
depending on your build tool. If they are not you may be able to
configure them within your PostCSS config:

```jsonc
// package.json
{
  // …
  "postcss": {
    "map": { "inline": true },
    "plugins": {
      "@tailwindcss/postcss": {},
    },
  }
}
```

### Vite

Source maps are supported when using the Tailwind CSS Vite plugin in
*development mode only* by enabling the `css.devSourcemap` setting:

```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";

export default defineConfig({
  plugins: [tailwindcss()],
  css: {
    devSourcemap: true,
  },
})
```

Now when a CSS file is requested by the browser it'll have an inline
source map comment that the browser can use.

## Limitations

- Production build source maps are currently disabled due to a bug in
Lightning CSS. See
parcel-bundler/lightningcss#971 for more
details.
- In Vite, minified CSS build source maps are not supported at all. See
vitejs/vite#2830 for more details.
- In PostCSS, minified CSS source maps are not supported. This is due to
the complexity required around re-associating every AST node with a
location in the generated, optimized CSS. This complexity would also
have a non-trivial performance impact.

## Testing

Here's how to test the source map functionality in different
environments:

### Testing the CLI

1. Setup typical project that the CLI can use and with sources to scan.

```css
@import "tailwindcss";

@utilty my-custom-utility {
  color: red;
}

/* to test `@apply` */
.card {
  @apply bg-white text-center shadow-md;
}
```

2. Build with source maps:
```bash
bun /path/to/tailwindcss/packages/@tailwindcss-cli/src/index.ts --input input.css -o output.css --map
```

3. Open Chrome DevTools, inspect an element with utility classes, and
you should see rules pointing to `input.css` or
`node_modules/tailwindcss/index.css`

### Testing with Vite

Testing in Vite will require building and installing necessary files
under `dist/*.tgz`.

1. Create a Vite project and enable source maps in `vite.config.js`:
```js
import tailwindcss from "@tailwindcss/vite";
import { defineConfig } from "vite";

export default defineConfig({
  plugins: [tailwindcss()],
  css: {
    // This line is required for them to work
    devSourcemap: true,
  },
})
```

2. Add a component that uses Tailwind classes and custom CSS:
```jsx
// ./src/app.jsx
export default function App() {
  return (
    <div className="bg-blue-500 my-custom-class">
      Hello World
    </div>
  )
}
```

```css
/* ./src/styles.css */
@import "tailwindcss";

@utilty my-custom-utility {
  color: red;
}

/* to test `@apply` */
.card {
  @apply bg-white text-center shadow-md;
}
```

3. Run `npm run dev`, open DevTools, and inspect elements to verify
source mapping works for both utility classes and custom CSS.

### Testing with PostCSS CLI

1. Create a test file and update your PostCSS config:
```css
/* input.css */
@import "tailwindcss";

@layer components {
  .card {
    @apply p-6 rounded-lg shadow-lg;
  }
}
```

```jsonc
// package.json
{
  // …
  "postcss": {
    "map": {
      "inline": true
    },
    "plugins": {
      "/path/to/tailwindcss/packages/packages/@tailwindcss-postcss/src/index.ts": {}
    }
  }
}
```

2. Run PostCSS through Bun:
```bash
bunx --bun postcss ./src/index.css -o out.css
```

3. Inspect the output CSS - it should include an inline source map
comment at the bottom.

### Testing with PostCSS + Next.js

Testing in Next.js will require building and installing necessary files
under `dist/*.tgz`. However, I've not been able to get CSS source maps
to work in Next.js without this hack:

```js
const nextConfig: NextConfig = {
  // next.js overwrites config.devtool so we prevent it from doing so
  // please don't actually do this…
  webpack: (config) =>
    Object.defineProperty(config, "devtool", {
      get: () => "inline-source-map",
      set: () => {},
    }),
};
```

This is definitely not supported and also doesn't work with turbopack.
This can be used to test them temporarily but I suspect that they just
don't work there.

### Manual source map analysis

You can analyze source maps using Evan Wallace's [Source Map
Visualization](https://evanw.github.io/source-map-visualization/) tool
which will help to verify the accuracy and quality of source maps. This
is what I used extensively while developing this implementation.

It'll help verify that custom, user CSS maps back to itself in the
input, that generated utilities all map back to `@tailwind utilities;`,
that source locations from imported files are also handled correctly,
etc… It also highlights the ranges of stuff so it's easy to see if there
are off-by-one errors.

It's easiest to use inline source maps with this tool because you can
take the CSS file and drop it on the page and it'll analyze it while
showing the file content.

If you're using Vite you'll want to access the CSS file with `?direct`
at the end so you don't get a JS module back.

## Implementation

The source map implementation follows the ECMA-426 specification and
includes several key components to aid in that goal:

### Source Location Tracking

Each emittable AST node in the compilation pipeline tracks two types of
source locations:
- `src`: Original source location - [source file, start offset, end
offset]
- `dst`: Generated source location - [output file, start offset, end
offset]

This dual tracking allows us to maintain mappings between the original
source and generated output for things like user CSS, generated
utilities, uses of `@apply`, and tracking theme variables.

It is important to note that source locations for nodes _never overlap_
within a file which helps simplify source map generation. As such each
type of node tracks a specific piece of itself rather than its entire
"block":

| Node | What a `SourceLocation` represents |
| ----------- |
---------------------------------------------------------------- |
| Style Rule | The selector |
| At Rule | Rule name and params, includes the `@` |
| Declaration | Property name and value, excludes the semicolon |
| Comment | The entire comment, includes the start `/*` and end `*/`
markers |

### Windows line endings when parsing CSS

Because our AST tracks nodes through offsets we must ensure that any
mutations to the file do *not* change the lenth of the string. We were
previously replacing `\r\n` with `\n` (see [filter code
points](https://drafts.csswg.org/css-syntax/#css-filter-code-points)
from the spec) — which changes the length of the string and all offsets
may end up incorrect. The CSS parser was updated to handle the CRLF
token directly by skipping over the `\r` and letting remaining code
handle `\n` as it did previously. Some additional tweaks were required
when "peeking" the input but those changes were fairly small.

### Tracking of imports

Source maps need paths to the actual imported stylesheets but the
resolve step for stylesheets happens inside the call to `loadStylesheet`
which make the file path unavailable to us. Because of this the
`loadStylesheet` API was augmented such that it has to return a `path`
property that we can then use to identify imported sources. I've also
made the same change to the `loadModule` API for consistency but nothing
currently uses this property.

The `path` property likely makes `base` redundant but elminating that
(if we even want to) is a future task.

### Optimizing the AST

Our optimization pass may intoduce some nodes, for example, fallbacks we
create for `@property`. These nodes are linked back to `@tailwind
utilities` as ultimately that is what is responsible for creating them.

### Line Offset Tables

A key component to our source map generation is the line offset table,
which was inspired by some ESBuild internals. It stores a sorted list of
offsets for the start of each line allowing us to translate offsets to
line/column `Position`s in `O(log N)` time and from `Position`s to
offsets in `O(1)` time. Creation of the table takes `O(N)` time.

This means that we can store code point offsets for source locations and
not have to worry about computing or tracking line/column numbers during
parsing and serialization. Only when a source map is generated do these
offsets need to be computed. This ensures the performance penalty when
not using source maps is minimal.

### Source Map Generation

The source map returned by `buildSourceMap()` is designed to follow the
[ECMA-426 spec](https://tc39.es/ecma426). Because that spec is not
completely finalized we consider the result of `buildSourceMap()` to be
internal API that may change as the spec chamges.

The produces source map is a "decoded" map such that all sources and
mappings are in an object graph. A library like `source-map-js` must be
used to convert this to an encoded source map of the right version where
mappings are encoded with base 64 VLQs.

Any specific integration (Vite, PostCSS, etc…) can then use
`toSourceMap()` from `@tailwindcss/node` to convert from the internal
source map to an spec-compliant encoded source map that can be
understood by other tools.

### Handling minification in Lightning

Since we use Lightning CSS for optimization, and it takes in an input
map, we generate an encoded source map that we then pass to lightning.
The output source map *from lighting itself* is then passed back in
during the second optimization pass. The final map is then passed from
lightning to the CLI (but not Vite or PostCSS — see the limitations
section for details).

In some cases we have to "fix up" the output CSS. When this happens we
use `magic-string` to do the replacement in a way that is trackable and
`@amppproject/remapping` to map that change back onto the original
source map. Once the need for these fix ups disappear these dependencies
can go away.

Notes:
- The accuracy of source maps run though lightning is reduced as it only
tracks on a per-rule level. This is sufficient enough for browser dev
tools so should be fine.
- Source maps during optimization do not function properly at this time
because of a bug in Lightning CSS regarding license comments. Once this
bug is fixed they will start working as expected.

### How source locations flow through the system

1. During initial CSS parsing, source locations are preserved.
2. During parsing these source locations are also mapped to the
destinations which supports an optimization for when no utilities are
generated.
3. Throughout the compilation process, transformations maintain source
location data
4. Generated utilities are explicitly pointed to `@tailwind utilities`
unless generated by `@apply`.
5. When optimization is enabled, source maps are remapped through
lightningcss
6. Final source maps are written in the requested format (inline or
separate file)

Author: thecrypticace

CHANGELOG.md

@@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Upgrade: Automatically convert candidates with arbitrary values to their utilities ([#17831](https://github.com/tailwindlabs/tailwindcss/pull/17831), [#17854](https://github.com/tailwindlabs/tailwindcss/pull/17854))
 - Write to log file when using `DEBUG=*` ([#17906](https://github.com/tailwindlabs/tailwindcss/pull/17906))
+- Add support for source maps in development ([#17775](https://github.com/tailwindlabs/tailwindcss/pull/17775))
 
 ### Fixed
 

integrations/cli/index.test.ts

@@ -23,7 +23,7 @@ describe.each([
     'Standalone CLI',
     path.resolve(__dirname, `../../packages/@tailwindcss-standalone/dist/${STANDALONE_BINARY}`),
   ],
-])('%s', (_, command) => {
+])('%s', (kind, command) => {
   test(
     'production build',
     {
@@ -628,6 +628,194 @@ describe.each([
       ])
     },
   )
+
+  test(
+    'production build + inline source maps',
+    {
+      fs: {
+        'package.json': json`
+          {
+            "dependencies": {
+              "tailwindcss": "workspace:^",
+              "@tailwindcss/cli": "workspace:^"
+            }
+          }
+        `,
+        'ssrc/index.html': html`
+          <div class="flex"></div>
+        `,
+        'src/index.css': css`
+          @import 'tailwindcss/utilities';
+          /*  */
+        `,
+      },
+    },
+    async ({ exec, expect, fs, parseSourceMap }) => {
+      await exec(`${command} --input src/index.css --output dist/out.css --map`)
+
+      await fs.expectFileToContain('dist/out.css', [candidate`flex`])
+
+      // Make sure we can find a source map
+      let map = parseSourceMap(await fs.read('dist/out.css'))
+
+      expect(map.at(1, 0)).toMatchObject({
+        source: null,
+        original: '(none)',
+        generated: '/*! tailwi...',
+      })
+
+      expect(map.at(2, 0)).toMatchObject({
+        source:
+          kind === 'CLI'
+            ? expect.stringContaining('node_modules/tailwindcss/utilities.css')
+            : expect.stringMatching(/\/utilities-\w+\.css$/),
+        original: '@tailwind...',
+        generated: '.flex {...',
+      })
+
+      expect(map.at(3, 2)).toMatchObject({
+        source:
+          kind === 'CLI'
+            ? expect.stringContaining('node_modules/tailwindcss/utilities.css')
+            : expect.stringMatching(/\/utilities-\w+\.css$/),
+        original: '@tailwind...',
+        generated: 'display: f...',
+      })
+
+      expect(map.at(4, 0)).toMatchObject({
+        source: null,
+        original: '(none)',
+        generated: '}...',
+      })
+    },
+  )
+
+  test(
+    'production build + separate source maps',
+    {
+      fs: {
+        'package.json': json`
+          {
+            "dependencies": {
+              "tailwindcss": "workspace:^",
+              "@tailwindcss/cli": "workspace:^"
+            }
+          }
+        `,
+        'ssrc/index.html': html`
+          <div class="flex"></div>
+        `,
+        'src/index.css': css`
+          @import 'tailwindcss/utilities';
+          /*  */
+        `,
+      },
+    },
+    async ({ exec, expect, fs, parseSourceMap }) => {
+      await exec(`${command} --input src/index.css --output dist/out.css --map dist/out.css.map`)
+
+      await fs.expectFileToContain('dist/out.css', [candidate`flex`])
+
+      // Make sure we can find a source map
+      let map = parseSourceMap({
+        map: await fs.read('dist/out.css.map'),
+        content: await fs.read('dist/out.css'),
+      })
+
+      expect(map.at(1, 0)).toMatchObject({
+        source: null,
+        original: '(none)',
+        generated: '/*! tailwi...',
+      })
+
+      expect(map.at(2, 0)).toMatchObject({
+        source:
+          kind === 'CLI'
+            ? expect.stringContaining('node_modules/tailwindcss/utilities.css')
+            : expect.stringMatching(/\/utilities-\w+\.css$/),
+        original: '@tailwind...',
+        generated: '.flex {...',
+      })
+
+      expect(map.at(3, 2)).toMatchObject({
+        source:
+          kind === 'CLI'
+            ? expect.stringContaining('node_modules/tailwindcss/utilities.css')
+            : expect.stringMatching(/\/utilities-\w+\.css$/),
+        original: '@tailwind...',
+        generated: 'display: f...',
+      })
+
+      expect(map.at(4, 0)).toMatchObject({
+        source: null,
+        original: '(none)',
+        generated: '}...',
+      })
+    },
+  )
+
+  // Skipped because Lightning CSS has a bug with source maps containing
+  // license comments and it breaks stuff.
+  test.skip(
+    'production build + minify + source maps',
+    {
+      fs: {
+        'package.json': json`
+          {
+            "dependencies": {
+              "tailwindcss": "workspace:^",
+              "@tailwindcss/cli": "workspace:^"
+            }
+          }
+        `,
+        'ssrc/index.html': html`
+          <div class="flex"></div>
+        `,
+        'src/index.css': css`
+          @import 'tailwindcss/utilities';
+          /*  */
+        `,
+      },
+    },
+    async ({ exec, expect, fs, parseSourceMap }) => {
+      await exec(`${command} --input src/index.css --output dist/out.css --minify --map`)
+
+      await fs.expectFileToContain('dist/out.css', [candidate`flex`])
+
+      // Make sure we can find a source map
+      let map = parseSourceMap(await fs.read('dist/out.css'))
+
+      expect(map.at(1, 0)).toMatchObject({
+        source: null,
+        original: '(none)',
+        generated: '/*! tailwi...',
+      })
+
+      expect(map.at(2, 0)).toMatchObject({
+        source:
+          kind === 'CLI'
+            ? expect.stringContaining('node_modules/tailwindcss/utilities.css')
+            : expect.stringMatching(/\/utilities-\w+\.css$/),
+        original: '@tailwind...',
+        generated: '.flex {...',
+      })
+
+      expect(map.at(3, 2)).toMatchObject({
+        source:
+          kind === 'CLI'
+            ? expect.stringContaining('node_modules/tailwindcss/utilities.css')
+            : expect.stringMatching(/\/utilities-\w+\.css$/),
+        original: '@tailwind...',
+        generated: 'display: f...',
+      })
+
+      expect(map.at(4, 0)).toMatchObject({
+        source: null,
+        original: '(none)',
+        generated: '}...',
+      })
+    },
+  )
 })
 
 test(

integrations/package.json

@@ -4,6 +4,7 @@
   "private": true,
   "devDependencies": {
     "dedent": "1.5.3",
-    "fast-glob": "^3.3.3"
+    "fast-glob": "^3.3.3",
+    "source-map-js": "^1.2.1"
   }
 }

integrations/postcss/index.test.ts

@@ -712,3 +712,68 @@ test(
     await retryAssertion(async () => expect(await fs.read('dist/out.css')).toEqual(''))
   },
 )
+
+test(
+  'dev mode + source maps',
+  {
+    fs: {
+      'package.json': json`
+        {
+          "dependencies": {
+            "postcss": "^8",
+            "postcss-cli": "^11",
+            "tailwindcss": "workspace:^",
+            "@tailwindcss/postcss": "workspace:^"
+          }
+        }
+      `,
+      'postcss.config.js': js`
+        module.exports = {
+          map: { inline: true },
+          plugins: {
+            '@tailwindcss/postcss': {},
+          },
+        }
+      `,
+      'src/index.html': html`
+        <div class="flex"></div>
+      `,
+      'src/index.css': css`
+        @import 'tailwindcss/utilities';
+        @source not inline("inline");
+        /*  */
+      `,
+    },
+  },
+  async ({ fs, exec, expect, parseSourceMap }) => {
+    await exec('pnpm postcss src/index.css --output dist/out.css')
+
+    await fs.expectFileToContain('dist/out.css', [candidate`flex`])
+
+    let map = parseSourceMap(await fs.read('dist/out.css'))
+
+    expect(map.at(1, 0)).toMatchObject({
+      source: '<no source>',
+      original: '(none)',
+      generated: '/*! tailwi...',
+    })
+
+    expect(map.at(2, 0)).toMatchObject({
+      source: expect.stringContaining('node_modules/tailwindcss/utilities.css'),
+      original: '@tailwind...',
+      generated: '.flex {...',
+    })
+
+    expect(map.at(3, 2)).toMatchObject({
+      source: expect.stringContaining('node_modules/tailwindcss/utilities.css'),
+      original: '@tailwind...',
+      generated: 'display: f...',
+    })
+
+    expect(map.at(4, 0)).toMatchObject({
+      source: expect.stringContaining('node_modules/tailwindcss/utilities.css'),
+      original: ';...',
+      generated: '}...',
+    })
+  },
+)