Built-in search plugin¶

2019-12-182026-08-01

MaterialX 10.2.0 fully refactors the search module with a brand-new architecture, greatly improving search quality and indexing efficiency.

It supports multi-provider mode, chunked indexing, on-demand loading, index compression, multilingual search and cross-domain search. It is suitable for various complex scenarios and large-scale sites, and can handle sites with more than 100,000 pages.

Pagefind is the default provider. You may switch back to the original Lunr when using it in an offline environment (opened via the file:// protocol).

Objective¶

How it works¶

The plugin builds the selected provider's index after MkDocs has rendered the site. The frontend loads that provider through a common integration and renders the results with MaterialX's built-in search interface.

Pagefind scans the generated HTML and writes a chunked index beside the site. At search time, the browser loads only the chunks and result data needed for the current query. Lunr writes search_index.json, then constructs and queries its in-memory index in a Web Worker.

Configuration¶

The search plugin is built into MaterialX and doesn't need to be installed. Add search to the plugins list to enable it (pagefind is the default provider):

plugins:
  - search

Configuration structure¶

The example below demonstrates the provider-based structure and available options, all of which are optional.

plugins:
  - search:
      provider: pagefind # pagefind (default) or lunr

      pagefind:
        # Index configuration
        include_characters: ._
        keep_index_url: true
        exclude_selectors:
          - .md-banner
        # output_subdir: search
        # logfile: pagefind.log

        # Browser Search API configuration
        options:
          excerptLength: 30
          exactDiacritics: false
          ranking:
            termFrequency: 1.0
            termSimilarity: 1.0
            pageLength: 0.75
            termSaturation: 1.4
            diacriticSimilarity: 0.8
            metaWeights:
              title: 5.0

      lunr:
        lang:
          - en
        separator: '[\s\-]+'
        pipeline:
          - stemmer
          - stopWordFilter
          - trimmer
        # jieba_dict: dict.txt
        # jieba_dict_user: user_dict.txt

Provider¶

pagefind

Use this setting to select the search provider:

PagefindLunr

plugins:
  - search:
      provider: pagefind

plugins:
  - search:
      provider: lunr

Pagefind is the default provider. You may switch back to the original Lunr when using it in an offline environment (opened via the file:// protocol).

Pagefind¶

10.2.0

Pagefind is a static search library designed around a small initial payload and on-demand index loading. MaterialX integrates pagefind[extended] with specific support for Chinese and Japanese.

Features¶

High-quality search results: It matches keywords comprehensively by combining multiple metrics including term similarity, term saturation, term frequency, page length, diacritics, as well as weights for content and metadata, delivering more accurate and comprehensive search results
High performance for large sites: It implements chunked indexing and on-demand loading mechanisms. Regardless of the total number of pages on the site, only index chunks relevant to search keywords are loaded into memory instead of the entire index, which greatly boosts search performance
Multilingual search: It automatically detects page languages and generates corresponding chunked indexes for all supported languages
Cross-domain multi-site search: It can be configured to search across multiple sites and merge results and filters into a single response

Index configuration¶

Configure indexing options directly under the pagefind level:

Setting	Default	Description
`exclude_selectors`	`nav`, `footer`	CSS selectors and their descendants to omit from indexing
`include_characters`	`._`	Punctuation preserved as searchable characters
`keep_index_url`	`true`	Keep `index.html` at the end of result URLs
`logfile`	none	Also write logs to a file; relative paths are resolved inside `output_subdir`
`options`	`{}`	Browser Search API configuration, described in the next section

MaterialX marks the main content with data-pagefind-body and manages the index input, output, and result URL format. Other Pagefind index options remain available for advanced use, see Pagefind's index configuration.

Search API configuration¶

Configure the browser Search API options directly under the pagefind.options level:

Option	Default	Description
`excerptLength`	`30`	Maximum target length for generated result excerpts
`exactDiacritics`	`false`	Treat accented and unaccented characters as distinct
`ranking`	Pagefind defaults	Tune result ranking with the parameters below

MaterialX manages bundle routing, result URLs, and highlighting. The complete upstream option set remains available for special cases in Pagefind's Search API configuration.

The options for ranking are as follows:

Ranking option	Default	Purpose
`termFrequency`	`1.0`	Balance term frequency against weighted term count
`termSimilarity`	`1.0`	Prefer indexed terms whose length is closer to the query
`pageLength`	`0.75`	Control how strongly shorter-than-average pages are favored
`termSaturation`	`1.4`	Control how quickly repeated terms stop increasing relevance
`diacriticSimilarity`	`0.8`	Boost exact diacritic matches when normalization is enabled
`metaWeights`	`title: 5.0`	Weight matches in title or custom metadata fields

For value ranges and the remaining controls, see Pagefind's ranking documentation.

Excluding content¶

Excluding an entire page¶

Use search.exclude in the Markdown front matter to remove a complete page and all of its sections from the Pagefind index:

---
search:
  exclude: true
---

# Page title
...

The meta plugin can apply the same property to every page in a folder:

.meta.yml

search:
  exclude: true

Excluding certain types of elements¶

Use exclude_selectors for elements that should be ignored throughout the site, such as a custom banner or generated utility block:

plugins:
  - search:
      pagefind:
        exclude_selectors:
          - .md-banner
          - .generated-example

The matched element and all of its children are excluded.

Excluding part of a page¶

Use Pagefind's data-pagefind-ignore attribute to exclude part of a document. For a complete section, wrap the heading and its content in an HTML container:

<div data-pagefind-ignore markdown>

## Internal notes

This complete section is excluded from Pagefind.

</div>

Other Pagefind features¶

Pagefind also supports metadata, filters, sorting, content weighting, custom records, and search across multiple sites. These are upstream Pagefind capabilities; some require custom templates, indexing code, or frontend code beyond MaterialX's built-in integration. See the Pagefind documentation and multisite search guide for the complete workflows.

Lunr¶

Lunr builds one JSON search index and loads it into a Web Worker in the browser. It is less efficient for very large sites than Pagefind, but remains the right provider when a generated site must work without an HTTP server.

Language¶

`lang`¶

9.0.0

Use this setting to specify the language of the search index, enabling stemming support for languages other than English. The default is computed from the site language, but can be set to one or multiple languages:

Set languageAdd further languages

plugins:
  - search:
      provider: lunr
      lunr:
        lang: en

plugins:
  - search:
      provider: lunr
      lunr:
        lang: # (1)!
          - en
          - de

Including more languages increases the base JavaScript payload by around 20kb and by another 15-30kb per language, all before gzip.

Language support is provided by lunr languages, a collection of language-specific stemmers and stop words for Lunr maintained by the Open Source community.

The following languages are currently supported by lunr languages:

ar – Arabic
da – Danish
de – German
du – Dutch
en – English
es – Spanish
fi – Finnish
fr – French
hi – Hindi
hu – Hungarian
hy – Armenian
it – Italian
ja – Japanese
kn – Kannada
ko – Korean
no – Norwegian
pt – Portuguese
ro – Romanian
ru – Russian
sa – Sanskrit
sv – Swedish
ta – Tamil
te – Telugu
th – Thai
tr – Turkish
vi – Vietnamese
zh – Chinese

If lunr languages doesn't support the selected site language, the plugin falls back to the language that is expected to yield the best stemming results.

Tokenization¶

`separator`¶

9.0.0

Use this setting to specify the separator used to split words when building the search index. The default is computed from the site language, but can be set explicitly:

plugins:
  - search:
      provider: lunr
      lunr:
        separator: '[\s\-,:!=\[\]()"/]+|(?!\b)(?=[A-Z][a-z])|\.(?!\d)|&[lg]t;'

Separators support positive and negative lookahead assertions, allowing precise control over how words are split.

Broken into its parts, this separator induces the following behavior:

Special charactersCase changesVersion stringsHTML/XML tags

[\s\-,:!=\[\]()"/]+

This inserts token boundaries before and after whitespace, hyphens, commas, brackets, and other special characters. Adjacent separators are treated as one.

(?!\b)(?=[A-Z][a-z])

This splits programming identifiers at case changes, tokenizing PascalCase into Pascal and Case.

\.(?!\d)

The negative lookahead prevents version strings like 1.2.3 from being split into separate numbers and keeps them discoverable.

&[lg]t;

This accounts for < and > being encoded as < and > in code blocks, allowing users to search for tag names.

Pipeline¶

`pipeline`¶

9.0.0

Use this setting to specify the pipeline functions that filter and expand tokens after tokenization and before adding them to the index. The default is computed from the site language:

plugins:
  - search:
      provider: lunr
      lunr:
        pipeline:
          - stemmer
          - stopWordFilter
          - trimmer

The following pipeline functions can be used:

stemmer – Stem tokens to their root form, e.g. running to run
stopWordFilter – Filter common words such as a and the
trimmer – Trim whitespace from tokens

Segmentation¶

The plugin supports Chinese text segmentation with jieba. Japanese and Korean are segmented on the client side when Lunr is selected.

`jieba_dict`¶

9.2.0

Use this setting to specify a custom dictionary for jieba, replacing its default dictionary:

plugins:
  - search:
      provider: lunr
      lunr:
        jieba_dict: dict.txt

The following dictionaries are provided by jieba:

dict.txt.small – 占用内存较小的词典文件
dict.txt.big – 支持繁体分词更好的词典文件

The path is resolved from the project root directory.

`jieba_dict_user`¶

9.2.0

Use this setting to add a user dictionary to jieba without replacing the default dictionary. User dictionaries are useful for product names, technical terms, and other site-specific vocabulary:

plugins:
  - search:
      provider: lunr
      lunr:
        jieba_dict_user: user_dict.txt

The path is resolved from the project root directory.

Excluding sections and blocks¶

When Attribute Lists is enabled, a section can be excluded from the Lunr index by adding data-search-exclude to its heading. The heading and all content up to the next heading of the same or higher level are omitted:

# Page title

## Public section

This content is included.

## Internal section { data-search-exclude }

This complete section is excluded from Lunr.

Add the same attribute after an inline or block-level element to exclude only that element:

This paragraph is included.

This paragraph is excluded.
{ data-search-exclude }

Metadata¶

`boost`¶

8.3.0

Use this property to increase or decrease the relevance of a page in Lunr results. Values above 1 rank a page up and values below 1 rank it down:

Rank up Rank down

---
search:
  boost: 2 # (1)!
---

# Page title
...

When boosting pages, always start with low values.

---
search:
  boost: 0.5
---

# Page title
...

`exclude`¶

9.0.0

Use this property to exclude a page and all of its subsections from the Lunr index. The Pagefind provider recognizes the same property:

---
search:
  exclude: true
---

# Page title
...

Built-in search plugin¶

Objective¶

How it works¶

Configuration¶

Configuration structure¶

Provider¶

Pagefind¶

Features¶

Index configuration¶

Search API configuration¶

Excluding content¶

Excluding an entire page¶

Excluding certain types of elements¶

Excluding part of a page¶

Other Pagefind features¶

Lunr¶

Language¶

lang¶

Tokenization¶

separator¶

Pipeline¶

pipeline¶

Segmentation¶

jieba_dict¶

jieba_dict_user¶

Excluding sections and blocks¶

Metadata¶

boost¶

exclude¶

`lang`¶

`separator`¶

`pipeline`¶

`jieba_dict`¶

`jieba_dict_user`¶

`boost`¶

`exclude`¶