Source Maps

Parcel utilizes the @parcel/source-maps package for processing source maps to ensure performance and reliability when manipulating source maps across plugins and Parcel's core. This library has been written from the ground up in Rust, and gave us a 20x performance improvement over the previous JavaScript-based implementation. This improvement in performance is mainly due to optimizations in the data structures and the way in which we cache source maps.

How to use the library

#

To use @parcel/source-maps, create an instance of the exported SourceMap class, on which you can call various functions to add and edit source mappings. A projectRoot directory path should be passed as an argument. All paths within the source map are converted to be relative to this.

Below is an example covering all ways of adding mappings to a SourceMap instance:

import SourceMap from '@parcel/source-map';

let sourcemap = new SourceMap(projectRoot);

// Each function that adds mappings has optional offset arguments.
// These can be used to offset the generated mappings by a certain amount.
let lineOffset = 0;
let columnOffset = 0;

// Add indexed mappings
// These are mappings that can sometimes be extracted from a library even before they get converted into VLQ Mappings
sourcemap.addIndexedMappings(
[
{
generated: {
// line index starts at 1
line: 1,
// column index starts at 0
column: 4,
},
original: {
// line index starts at 1
line: 1,
// column index starts at 0
column: 4,
},
source: "index.js",
// Name is optional
name: "A",
},
],
lineOffset,
columnOffset
);

// Add vlq mappings. This is what would be outputted into a vlq encoded source map
sourcemap.addVLQMap(
{
file: "min.js",
names: ["bar", "baz", "n"],
sources: ["one.js", "two.js"],
sourceRoot: "/the/root",
mappings:
"CAAC,IAAI,IAAM,SAAUA,GAClB,OAAOC,IAAID;CCDb,IAAI,IAAM,SAAUE,GAClB,OAAOA",
},
lineOffset,
columnOffset
);

// Source maps can be serialized to buffers, which is what we use for caching in Parcel.
// You can instantiate a SourceMap with these buffer values by passing it to the constructor
let map = new SourceMap(projectRoot, mapBuffer);

// You can also add a buffer to an existing source map using the addBuffer method.
sourcemap.addBuffer(originalMapBuffer, lineOffset);

// One SourceMap object may be added to another using the addSourceMap method.
sourcemap.addSourceMap(map, lineOffset);

Transformations/Manipulations

#

If your plugin does any code manipulations, you should ensure that it creates correct mappings to the original source code to guarantee that we still end up creating an accurate source map at the end of the bundling process. You are expected to return a SourceMap instance at the end of a transform in a Transformer plugin.

We also provide the source map from the previous transform to ensure you map to the original source code and not just the output of the previous transform. If a compiler doesn't have a way to pass in an input source map, you can use the extends method of a SourceMap to map the original mappings to the compiled ones.

The asset value that gets passed in the parse, transform and generate functions of a transformer plugin contains a function called getMap() and getMapBuffer(). These functions can be used to get a SourceMap instance (getMap()) and the cached SourceMap Buffer (getMapBuffer()).

You are free to manipulate the source map at any of these steps in the transformer as long as you ensure the source map that gets returned in generate maps to the original sourcefile correctly.

Below is an example on how to manipulate sourcemaps in a transformer plugin:

import {Transformer} from '@parcel/plugin';
import SourceMap from '@parcel/source-map';

export default new Transformer({
// ...

async generate({asset, ast, resolve, options}) {
let compilationResult = someCompiler(await asset.getAST());

let map = null;
if (compilationResult.map) {
// If the compilationResult returned a map we convert
// it to a Parcel SourceMap instance.
map = new SourceMap(options.projectRoot);

// The compiler returned a full, encoded sourcemap with vlq mappings.
// Some compilers might have the possibility of returning
// indexedMappings which might improve performance (like Babel does).
// In general, every compiler is able to return rawMappings, so
// it's always a safe bet to use this.
map.addVLQMap(compilationResult.map);

// We get the original source map from the asset to extend our mappings
// on top of it. This ensures we are mapping to the original source
// instead of the previous transformation.
let originalMap = await asset.getMap();
if (originalMap) {
// The `extends` function uses the provided map to remap the original
// source positions of the map it is called on. In this case, the
// original source positions of `map` get remapped to the positions
// in `originalMap`.
map.extends(originalMap);
}
}

return {
code: compilationResult.code,
map,
};
},
});

If your compiler supports the option to pass in an existing sourcemap, this may result in more accurate sourcemaps than using the method in the previous example.

An example of how this would work:

import {Transformer} from '@parcel/plugin';
import SourceMap from '@parcel/source-map';

export default new Transformer({
// ...

async generate({asset, ast, resolve, options}) {
// Get the original map from the asset.
let originalMap = await asset.getMap();
let compilationResult = someCompiler(await asset.getAST(), {
// Pass the VLQ encoded version of the originalMap to the compiler.
originalMap: originalMap.toVLQ(),
});

// In this case the compiler is responsible for mapping to the original
// positions provided in the originalMap, so we can just convert it to
// a Parcel SourceMap and return it.
let map = new SourceMap(options.projectRoot);
if (compilationResult.map) {
map.addVLQMap(compilationResult.map);
}

return {
code: compilationResult.code,
map,
};
},
});

Concatenating sourcemaps in Packagers

#

If you're writing a custom packager, it's your responsibility to concatenate the source maps of all the assets while packaging. This is done by creating a new SourceMap instance and adding new mappings to it using the addSourceMap(map, lineOffset) function. lineOffset should be equal to the line index at which the asset output starts.

Below is an example of how to do this:

import {Packager} from '@parcel/plugin';
import SourceMap from '@parcel/source-map';

export default new Packager({
async package({bundle, options}) {
// Read content and source maps for each asset in the bundle.
let promises = [];
bundle.traverseAssets(asset => {
promises.push(Promise.all([
asset.getCode(),
asset.getMap()
]);
});

let results = await Promise.all(promises);

// Instantiate a string to hold the bundle contents, and
// a SourceMap to hold the combined bundle source map.
let contents = '';
let map = new SourceMap(options.projectRoot);
let lineOffset = 0;

// Add the contents of each asset.
for (let [code, map] of assets) {
contents += code + '\n';

// Add the source map if the asset has one, and offset
// it by the number of lines in the bundle so far.
if (map) {
map.addSourceMap(map, lineOffset);
}

// Add the number of lines in this asset.
lineOffset += countLines(code) + 1;
}

// Return the contents and map.
return {contents, map};
},
});

Concatenating ASTs

#

If you're concatenating ASTs instead of source contents you already have the source mappings embedded into the AST, which you can use to generate the final source map. However, you must ensure that those mappings stay intact while editing the AST nodes. Sometimes this can be quite challenging if you're doing a lot of modifications.

An example of how this works:

import {Packager} from '@parcel/plugin';
import SourceMap from '@parcel/source-map';

export default new Packager({
async package({bundle, options}) {
// Do the AST concatenation and return the compiled result
let compilationResult = concatAndCompile(bundle);

// Create the final packaged sourcemap
let map = new SourceMap(options.projectRoot);
if (compilationResult.map) {
map.addVLQMap(compilationResult.map);
}

// Return the compiled code and map
return {
code: compilationResult.code,
map,
};
},
});

Postprocessing source maps in optimizers

#

Using source maps in optimizers is identical to how you use it in transformers. You get one file as input and are expected to return that same file as output, but optimized.

The only difference with optimizers is that the map is not provided as part of an asset but rather as a separate parameter/option, as you can see in the code snippet below. As always, the map is an instance of the SourceMap class.

import {Optimizer} from '@parcel/plugin';

export default new Optimizer({
// The contents and map are passed separately
async optimize({bundle, contents, map}) {
return {contents, map};
}
});

Diagnosing issues

#

If you encounter incorrect mappings and want to debug these, we have built tools that can help you diagnose these issues. By running the @parcel/reporter-sourcemap-visualiser reporter, Parcel creates a sourcemap-info.json file with all the necessary information to visualize all the mappings and source content.

To enable it, use the --reporter option, or add it to your .parcelrc.

parcel build src/index.js --reporter @parcel/reporter-sourcemap-visualizer

After the reporter has created the sourcemap-info.json file, you can upload it to the sourcemap visualizer.

API

#

SourceMap source-map/src/SourceMap.js:8

interface SourceMap {
  constructor(projectRoot: string, buffer?: Buffer): void,

Construct a SourceMap instance

Params:
  • projectRoot: root directory of the project, this is to ensure all source paths are relative to this path
  libraryVersion(): string,
  static generateEmptyMap(v: GenerateEmptyMapOptions): SourceMap,

Generates an empty map from the provided fileName and sourceContent

Params:
  • sourceName: path of the source file
  • sourceContent: content of the source file
  • lineOffset: an offset that gets added to the sourceLine index of each mapping
  addEmptyMap(sourceName: string, sourceContent: string, lineOffset: number): SourceMap,

Generates an empty map from the provided fileName and sourceContent

Params:
  • sourceName: path of the source file
  • sourceContent: content of the source file
  • lineOffset: an offset that gets added to the sourceLine index of each mapping
  addVLQMap(map: VLQMap, lineOffset: number, columnOffset: number): SourceMap,

Appends raw VLQ mappings to the sourcemaps

  addSourceMap(sourcemap: SourceMap, lineOffset: number): SourceMap,

Appends another sourcemap instance to this sourcemap

Params:
  • buffer: the sourcemap buffer that should get appended to this sourcemap
  • lineOffset: an offset that gets added to the sourceLine index of each mapping
  addBuffer(buffer: Buffer, lineOffset: number): SourceMap,

Appends a buffer to this sourcemap Note: The buffer should be generated by this library

Params:
  • buffer: the sourcemap buffer that should get appended to this sourcemap
  • lineOffset: an offset that gets added to the sourceLine index of each mapping
  addIndexedMapping(mapping: IndexedMapping<string>, lineOffset?: number, columnOffset?: number): void,

Appends a Mapping object to this sourcemap Note: line numbers start at 1 due to mozilla's source-map library

Params:
  • mapping: the mapping that should be appended to this sourcemap
  • lineOffset: an offset that gets added to the sourceLine index of each mapping
  • columnOffset: an offset that gets added to the sourceColumn index of each mapping
  _indexedMappingsToInt32Array(mappings: Array<IndexedMapping<string>>, lineOffset?: number, columnOffset?: number): Int32Array,
  addIndexedMappings(mappings: Array<IndexedMapping<string>>, lineOffset?: number, columnOffset?: number): SourceMap,

Appends an array of Mapping objects to this sourcemap This is useful when improving performance if a library provides the non-serialised mappings
Note: This is only faster if they generate the serialised map lazily Note: line numbers start at 1 due to mozilla's source-map library

Params:
  • mappings: an array of mapping objects
  • lineOffset: an offset that gets added to the sourceLine index of each mapping
  • columnOffset: an offset that gets added to the sourceColumn index of each mapping
  addName(name: string): number,

Appends a name to the sourcemap

Params:
  • name: the name that should be appended to the names array
  addNames(names: Array<string>): Array<number>,

Appends an array of names to the sourcemap's names array

Params:
  • names: an array of names to add to the sourcemap
  addSource(source: string): number,

Appends a source to the sourcemap's sources array

Params:
  • source: a filepath that should be appended to the sources array
  addSources(sources: Array<string>): Array<number>,

Appends an array of sources to the sourcemap's sources array

Params:
  • sources: an array of filepaths which should sbe appended to the sources array
  getSourceIndex(source: string): number,

Get the index in the sources array for a certain source file filepath

Params:
  • source: the filepath of the source file
  getSource(index: number): string,

Get the source file filepath for a certain index of the sources array

Params:
  • index: the index of the source in the sources array
  getSources(): Array<string>,

Get a list of all sources

  setSourceContent(sourceName: string, sourceContent: string): void,

Set the sourceContent for a certain file this is optional and is only recommended for files that we cannot read in at the end when we serialise the sourcemap

Params:
  • sourceName: the path of the sourceFile
  • sourceContent: the content of the sourceFile
  getSourceContent(sourceName: string): string | null,

Get the content of a source file if it is inlined as part of the source-map

Params:
  • sourceName: filename
  getSourcesContent(): Array<string | null>,

Get a list of all sources

  getSourcesContentMap(): {
    [key: string]: string | null
  },

Get a map of the source and it's corresponding source content

  getNameIndex(name: string): number,

Get the index in the names array for a certain name

Params:
  • name: the name you want to find the index of
  getName(index: number): string,

Get the name for a certain index of the names array

Params:
  • index: the index of the name in the names array
  getNames(): Array<string>,

Get a list of all names

  getMappings(): Array<IndexedMapping<number>>,

Get a list of all mappings

  indexedMappingToStringMapping(mapping: ?IndexedMapping<number>): ?IndexedMapping<string>,

Convert a Mapping object that uses indexes for name and source to the actual value of name and source
Note: This is only used internally, should not be used externally and will probably eventually get handled directly in C++ for improved performance

Params:
  • index: the Mapping that should get converted to a string-based Mapping
  extends(buffer: Buffer | SourceMap): SourceMap,

Remaps original positions from this map to the ones in the provided map
This works by finding the closest generated mapping in the provided map to original mappings of this map and remapping those to be the original mapping of the provided map.

Params:
  getMap(): ParsedMap,

Returns an object with mappings, sources and names This should only be used for tests, debugging and visualising sourcemaps
Note: This is a fairly slow operation

  findClosestMapping(line: number, column: number): ?IndexedMapping<string>,

Searches through the sourcemap and returns a mapping that is close to the provided generated line and column

Params:
  • line: the line in the generated code (starts at 1)
  • column: the column in the generated code (starts at 0)
  offsetLines(line: number, lineOffset: number): ?IndexedMapping<string>,

Offset mapping lines from a certain position

Params:
  • line: the line in the generated code (starts at 1)
  • lineOffset: the amount of lines to offset mappings by
  offsetColumns(line: number, column: number, columnOffset: number): ?IndexedMapping<string>,

Offset mapping columns from a certain position

Params:
  • line: the line in the generated code (starts at 1)
  • column: the column in the generated code (starts at 0)
  • columnOffset: the amount of columns to offset mappings by
  toBuffer(): Buffer,

Returns a buffer that represents this sourcemap, used for caching

  toVLQ(): VLQMap,

Returns a serialised map using VLQ Mappings

  delete(): void,

A function that has to be called at the end of the SourceMap's lifecycle to ensure all memory and native bindings get de-allocated

  stringify(options: SourceMapStringifyOptions): Promise<string | VLQMap>,

Returns a serialised map

Params:
  • options: options used for formatting the serialised map
}
Referenced by:
BaseAsset, BundleResult, GenerateOutput, MutableAsset, Optimizer, Packager, TransformerResult

MappingPosition source-map/src/types.js:2

type MappingPosition = {|
  line: number,
  column: number,
|}
Referenced by:
IndexedMapping

IndexedMapping source-map/src/types.js:7

type IndexedMapping<T> = {
  generated: MappingPosition,
  original?: MappingPosition,
  source?: T,
  name?: T,
}
Referenced by:
ParsedMap, SourceMap

ParsedMap source-map/src/types.js:15

type ParsedMap = {|
  sources: Array<string>,
  names: Array<string>,
  mappings: Array<IndexedMapping<number>>,
  sourcesContent: Array<string | null>,
|}
Referenced by:
SourceMap

VLQMap source-map/src/types.js:22

type VLQMap = {
  +sources: $ReadOnlyArray<string>,
  +sourcesContent?: $ReadOnlyArray<string | null>,
  +names: $ReadOnlyArray<string>,
  +mappings: string,
  +version?: number,
  +file?: string,
  +sourceRoot?: string,
}
Referenced by:
SourceMap

SourceMapStringifyOptions source-map/src/types.js:33

type SourceMapStringifyOptions = {
  file?: string,
  sourceRoot?: string,
  inlineSources?: boolean,
  fs?: {
    readFile(path: string, encoding: string): Promise<string>,
    ...
  },
  format?: 'inline' | 'string' | 'object',
}
Referenced by:
SourceMap

GenerateEmptyMapOptions source-map/src/types.js:46

type GenerateEmptyMapOptions = {
  projectRoot: string,
  sourceName: string,
  sourceContent: string,
  lineOffset?: number,
}
Referenced by:
SourceMap