Building and Using a Custom Iconify Image Set For Mermaid

python
devops
front-end
Author

Adrian Cederberg

Published

February 18, 2025

Modified

February 18, 2025

Keywords

mermaid, kubernetes, architecture diagram, svg, iconify, icons, icon set, iconify icon set, github, api, python, typer

Recently I wanted to use mermaid.js to make an architecture diagram to describe my infrastructure I am currently using to deploy my blog.

In particular, I wanted to use the SVG icons defined in the kubernetes community github repo inside of my architecture diagram. However, I could not find a suitable icon set. To this end I made my own and would like to showcase the methods that I used.

After figuring out how to wrestle the SVG icons into a properly formatted JSON I was able to generate awesome diagrams using kubernetes icons such as the one below in Figure 1 and icons like that seen in the header of this section (the mermaid.js symbol).

architecture-beta
  group kubernetes(logos:kubernetes)
  group ns(k8s:ns)[Blog Namespace] in kubernetes

  group blog_service(k8s:svc)[Blog Service] in ns
  service blog_deployment(k8s:deployment)[Blog Deployment] in blog_service
  service blog_ingressroute(k8s:crd)[Traefik Production IngressRoute] in ns
  blog_ingressroute:L --> R:blog_deployment{group}

  group blog_service_preview(k8s:svc)[Blog Service Preview] in ns
  service blog_deployment_preview(k8s:deployment)[Blog Deployment Preview] in blog_service_preview
  service blog_ingressroute_preview(k8s:crd)[Traefik Production IngressRoute] in ns
  blog_ingressroute_preview:L --> R:blog_deployment_preview{group}

  service traefik(devicon:traefikproxy)[Traefik] in kubernetes
  service internet(internet)[Internet]

  junction tt in ns
  tt:T --> B:blog_ingressroute_preview
  tt:B --> T:blog_ingressroute
  tt:R <-- L:traefik
  internet:L --> R:traefik

  group blog_gha(k8s:crb)[Blog Github Actions ClusterRole] in ns
  service blog_gha_sa(k8s:sa)[Blog Github Actions Service Account] in blog_gha

  junction ss in ns
  ss:B -- T:blog_deployment{group}
  ss:T -- B:blog_deployment_preview{group}
  ss:L -- R:blog_gha_sa{group}

  service github_actions(logos:github-actions)[Github Actions]
  github_actions:R -- L:blog_gha_sa{group}
Figure 1: Diagram made with mermaid using the custom icon set.

Objective

{
  "prefix": "myprefix",
  "icons": {
    "myicon": {
      "body": "<g></g>",
      "height": 17,
      "width": 17
    }
  }
}
Figure 2: General shape of the iconify icon set JSON.

The goal is to take the SVG content and turn it into a string inside of some JSON like that in Figure 2. Every value inside of $.icons should at least have a body field containing SVG data. The exact JSON schema may be found on iconify.design.

Note

As you will see bellow, the only thing this has to with iconify is the JSON schema used by mermaid. It would appear that mermaid does not use iconify under the hood and therefore icon sets that work in mermaid may not work with iconify .

Gotchas

There were a number of things that I found unintuitive while writing the script and getting the icon set to work.

body Should Not be Wrapped in SVG

It turns out that having the SVG tag inside of the body results in the icons not loading. My solution was to use pythons built in xml library to parse out the content that was only the same. In the case of the kubernetes icons I had to find an element layer1 to extract from the xml.

In python, this is done like

from xml.dom import minidom

loaded = minidom.parse(str(path))
elements = loaded.getElementsByTagName("g")
element = next((ee for ee in elements if ee.getAttribute("id") == "layer1"), None)

A big part of debugging was parsing the xml files to query the right piece of code.

Certain SVG Icons will not Be Accepted by Iconify

Some icons contain SVG incompatible with iconify. However, such icons will often work with mermaid Since I only really want these icons for mermaid I do not care, however, I found that certain icons (in my case k8s:limits make iconify) unable to use the whole image set.

All that is Necessary is a Properly Formatted JSON File.

You do not need to build your own CDN or use the iconify API unless you would like to distribute the icon set to others.

The Script

After all of these considerations, I cooked up a python script to process all of the SVG icons and turn them into a nice JSON file.

First, install the dependencies like:

pip install typer requests

and then choose from one of the two setups described below.

Build using API Key

My best solution was to copy all of the files using the pull command instead of cloning the entire repo as in the alternative setup.

  1. Copy the script in Figure 3 and paste it into iconify.py.

  2. Read the instructions in the section on the code snippet for removing the small internal dependencies and replace them as specified.

  3. Get an appropriate github token. This token should be very minimal as it is included only to bypass API rate limiting by github. Export the token like export GH_TOKEN="<my-gh-token-goes-here>".

  4. Pull the raw SVGs from github using the pull command and then use the 4 make command to build the icon set:

    python ./iconify.py pull
python ./iconify.py make

Building from Cloned Repo

Warning

No longer recommended. See instead (the standalone setup)[#setup-standalone}

  1. Clone the kubernetes community repo, For instance

    git clone --depth 1 https://github.com/kubernetes/community

  2. Copy the build script in Figure 3 into the repo under icons/tools/iconify.py,

  3. Read the instructions in the section on the code snippet for removing the small internal dependencies and replace them as specified.

  4. Run the build command:

    python ./icons/tools/iconify.py make

The Build Script

"""This script should generate ``icons.json`` (e.g. ``iconify``) from existing svgs 
so that icons are available for use with tools [mermaid](https://mermaid.js.org) 
that load the svg from json.
"""

import base64
import json
import os
import pathlib
from typing import Any, Iterable, Optional
from xml.dom import minidom

import requests
import rich.console
import typer

from acederbergio import env, util

PATH_HERE = pathlib.Path(__file__).resolve().parent
PATH_ICONS_JSON = PATH_HERE / "icons.json"
PATH_SVG = PATH_HERE / "svg"
DELIM = "_"

abbr = {
    "pvc": "persistent_volume-claim",
    "svc": "service",
    "vol": "volume",
    "rb": "role-binding",
    "rs": "replica-set",
    "ing": "ingress",
    "secret": "secret",
    "pv": "persistent-volume",
    "cronjob": "cron-job",
    "sts": "stateful-set",
    "pod": "pod",
    "cm": "config-map",
    "deploy": "deployment",
    "sc": "storage-class",
    "hpa": "horizontal-pod-autoscaler",
    "crd": "custom-resource-definition",
    "quota": "resource-quota",
    "psp": "pod-security-policy",
    "sa": "service-account",
    "role": "role",
    "c-role": "cluster-role",
    "ns": "namespace",
    "node": "node",
    "job": "job",
    "ds": "daemon-set",
    "ep": "endpoint",
    "crb": "cluster-role-binding",
    "limits": "limit-range",
    "control-plane": "control-plane",
    "k-proxy": "kube-proxy",
    "sched": "scheduler",
    "api": "api-server",
    "c-m": "controller-manager",
    "c-c-m": "cloud-controller-manager",
    "kubelet": "kubelet",
    "group": "group",
    "user": "user",
    "netpol": "network-policy",
}


def walk(directory: pathlib.Path):
    """Iterate through ``directory`` content."""
    contents = os.listdir(directory)

    for path in map(lambda path: directory / path, contents):

        if os.path.isdir(path):
            yield from walk(directory / path)
            continue

        yield directory / path


def load(path: pathlib.Path) -> str:
    """Load and process the ``svg`` file at ``path``."""

    loaded = minidom.parse(str(path))
    elements = loaded.getElementsByTagName("g")
    element = next((ee for ee in elements if ee.getAttribute("id") == "layer1"), None)

    if element is None:
        rich.print(f"[red]Could not find ``layer1`` of ``{path}``.")
        raise typer.Exit(1)

    return element.toxml("utf-8").decode()


def create_name(path: pathlib.Path):
    """Create name from ``path``."""
    head, _ = os.path.splitext(os.path.relpath(path, PATH_SVG))
    pieces = head.split("/")

    # NOTE: Labeled icons are the default. When an icon is unlabeled, just
    #       attach ``unlabeled`` to the end.
    if "labeled" in pieces:
        pieces.remove("labeled")
    elif "unlabeled" in pieces:
        loc = pieces.index("unlabeled")
        pieces[loc], pieces[-1] = pieces[-1], "u"

    # NOTE: These prefixes are not necessary in icon names.
    if "infrastructure_components" in pieces:
        pieces.remove("infrastructure_components")
    elif "control_plane_components" in pieces:
        pieces.remove("control_plane_components")
    elif "resources" in pieces:
        pieces.remove("resources")

    return DELIM.join(pieces).replace("-", DELIM)


def create_alias(name: str) -> str | None:
    """For a short name, create its long alias."""

    split = name.split("-")
    for pos, item in enumerate(split):
        if item in abbr:
            split[pos] = abbr[item]
        elif item == "u":
            split[pos] = "unlabeled"

    alias = DELIM.join(split).replace("-", DELIM)
    if alias == name:
        return None

    return alias


def create_aliases(names: Iterable[str]):
    """Create aliases ``$.aliases``."""

    aliases = {
        alias: {"parent": name}
        for name in names
        if (alias := create_alias(name)) is not None
    }
    return aliases


def create_iconify_icon(path: pathlib.Path) -> dict[str, Any]:
    """Create ``$.icons`` values."""

    # NOTE: Height and width must be 17 to prevent cropping.
    return {"height": 17, "width": 17, "body": load(path)}


def create_icons():
    """Create ``$.icons``."""
    return {create_name(item): create_iconify_icon(item) for item in walk(PATH_SVG)}


def create_iconify_json(include: set[str] = set()):
    """Create ``kubernetes.json``, the iconify icon set."""

    icons = create_icons()
    if include:
        icons = {name: icon for name, icon in create_icons().items() if name in include}

    aliases = create_aliases(icons)
    return {"icons": icons, "prefix": "k8s", "aliases": aliases}


cli = typer.Typer(help="Tool for generating the kubernetes iconify icon set.")


@cli.command("pull")
def pull(gh_token: Optional[str] = None):
    """Download the svgs from github using the API."""

    url_icons = "https://api.github.com/repos/kubernetes/community/contents/icons"
    gh_token = env.require("gh_token", gh_token)
    headers = {"Authorization": f"Bearer {gh_token}"}

    def get(url: str):
        response = requests.get(url, headers=headers)
        if response.status_code != 200:
            print(
                f"Bad status code `{response.status_code}` from `{response.request.url}`."
            )
            if response.content:
                print(response.json())
            raise typer.Exit(5)

        return response.json()

    def walk_clone(directory_relpath: str):

        directory_path = PATH_HERE / directory_relpath
        directory_url = url_icons + "/" + directory_relpath

        if not os.path.exists(directory_path):
            rich.print(f"[green]Making directory `{directory_path}`.")
            os.mkdir(directory_path)

        rich.print(f"[green]Checking contents of `{directory_path}`.")
        data = get(directory_url)

        for item in data:

            # NOTE: If it is a directory, recurse and inspect content.
            item_relpath = directory_relpath + "/" + item["name"]
            item_url = directory_url + "/" + item["name"]
            if item["type"] == "dir":
                walk_clone(item_relpath)
                continue

            # NOTE: If it is not a directory, then just put it into the file
            #       with its respective name.
            # NOTE: Content is in base64 form. There is the option to use the
            #       download_url field, however it is probably faster to do
            #       this.
            item_path = directory_path / item["name"]
            rich.print(f"[green]Inspecting `{item_relpath}`.")
            if not os.path.exists(item_path) and item_path.suffix == ".svg":
                rich.print(f"[green]Dumping content to `{item_path}`.")

                item_data = get(item_url)
                with open(item_path, "w") as file:
                    file.write(base64.b64decode(item_data["content"]).decode())

    walk_clone("svg")


@cli.command("make")
def main(include: list[str] = list(), out: Optional[pathlib.Path] = None):
    """Create kubernetes iconify json."""

    iconify = create_iconify_json(set(include))

    if out is None:
        util.print_yaml(iconify, as_json=True)
        return

    with open(out, "w") as file:
        json.dump(iconify, file, indent=2)


@cli.command("aliases")
def aliases(include: list[str] = list()):
    """Generate aliases and print to console."""

    names: Any = map(create_name, walk(PATH_SVG))
    if include:
        _include = set(include)
        names = filter(lambda item: item in _include, names)

    aliases = create_aliases(names)
    util.print_yaml(aliases, as_json=True)


@cli.command("names")
def names():
    """Generate names and print to console."""

    util.print_yaml(list(map(create_name, walk(PATH_SVG))), as_json=True)


if __name__ == "__main__":
    cli()
Figure 3: Script for building the iconify icon set.

To learn about the dependencies in the acederbergio package, see the github repository for this website. To replace the two internal dependencies, just replace env.require with os.env["GH_TOKEN"] and util.print_yaml with print and remove the acederbergio import statements.

Source code is available here.

typer

typer makes it easy to run the script with flags and subcommands.

  1. The make subcommand will create the JSON output. It can limit the number of icons process using --include and set the output destination using --output.

    For instance, in the cloned setup the icon set with only the svc, etcd, and limits icon can be built like

    #| fig-cap: This assumes that the shell is currently in the root of the cloned community repo.
python ./icons/tools/iconify.py \
  make --include svc \
    --include etcd \
    --include limits \
    --output iconify.json

    and will be output to ./iconify.json.

  2. The names will print all of the icon names.

  3. The aliases command will print $.aliases and also has --include.

It might seem overkill, but this was extremely convenient to pick out some bugs while building the iconset.

Note

The JSON output is available on this website.

Using the Icon Set in Mermaid

Note

It is possible to use the icon set with iconify in pure HTML. However, for the reasons already state above, I will not be going into the details of this here. This is described within the iconify documentation. For those interested using iconify in quarto, see the iconify extension for quarto.

With Mermaid Inside of HTML

Note

The code displayed below is available as webpage here.

To get the icons working with mermaid, include mermaid and use mermaid.registerIconPacks. Then write out the declarative code for your diagram inside of some pre tags with class=mermaid:

<html>
<head>
  <script src="https://code.iconify.design/3/3.0.0/iconify.min.js"></script>
  <!-- start snippet script -->
  <script type="module">
    import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';

    async function setupMermaid()
    {
      await mermaid.initialize({startOnLoad: true});
      mermaid.registerIconPacks([{
        name: 'k',
        loader: () => fetch('/icons/kubernetes.json').then(response => response.json())
      }])
    }
    setupMermaid()
  </script>
  <!-- end snippet script -->
</head>
<body>
  <section>
    <h1>Using Mermaid</h1>
    <pre class="mermaid">
        architecture-beta
          group namespace(k:ns)[Kubernetes]
          service blog_deployment(k:svc)[Service] in namespace
    </pre>
  </section>
</body>
</html>

With Mermaid Inside Of Quarto

It is as simple as adding something like the script tags from above to quarto HTML output, e.g. make a file ./includes/mermaid.html that calls mermaid.registerIconPacks:

<script type="module">
  async function setupMermaid() {
    mermaid.registerIconPacks([
      {
        name: 'misc',
        loader: () => fetch('/icons/misc.json').then(response => response.json())
      },
      {
        name: 'k8s',
        loader: () => fetch('/icons/sets/kubernetes.json').then(response => response.json())
      },
      // NOTE: The CDN is not great. Build should expect these to be added into image, hopefully when cdn does not suck.
      // https://unpkg.com/@iconify-json/logos/icons.json
      // wget -O ./blog/icons/json/devicon-plain.json https://unpkg.com/@iconify-json/devicon-plain/icons.json
      {
        name: 'logos',
        loader: () => fetch("/icons/sets/logos.json").then(response => response.json())
      },
      {
        name: 'plain',
        loader: () => fetch("/icons/sets/devicon-plain.json").then(response => response.json())
      },
      {
        name: 'devicon',
        loader: () => fetch("/icons/sets/devicon.json").then(response => response.json())
      },
      {
        name: 'hugeicons',
        loader: () => fetch("/icons/sets/hugeicons.json").then(response => response.json())
      }
    ])
  }
  setupMermaid()
</script>

and include it in the html output by using format.html.include-in-header:

---
format:
  html:
    include-in-header:
      - file: ./includes/mermaid.html
---

This is an example, and the following diagram should be rendered by mermaid
in the browser:

```{mermaid}
architecture-beta
  group linode(logos:linode)[Linode]
  service blog_deployment(k8s:svc)[Blog Service and Deployment] in linode
```

The diagram rendered should look like

architecture-beta
  group linode(logos:linode)[Linode]
  service blog_deployment(k8s:svc)[Blog Service and Deployment] in linode
Figure 4: The final product.