Extracting Rich Output for fun and profit

Okay maybe not so much on the profit but definitely fun!

post python rich text ~1200 words lastmod

cover-extracting-rich-output
a screenshot of the HTML that I created to show that I don’t need screenshots

Somewhere in the middle of Tooting with Python, I mentioned I how I get Rich output into a post. That approach was a little clumsy though. I want to run my code and paste its output into whatever draft I’m editing.

So I’ll figure that one out now.

ℹ️ 2021-08-25

I initially posted a version of this post using BeautifulSoup for HTML extraction. Then Rich creator Will McGugan pointed out from Twitter that I could get what I need from Rich itself!

Great write up!

You may be able to skip the Beautiful Soup step with the following:

    console.export_html(code_format="<pre style="font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">{code}</pre>")

Yeah let’s do that instead. Much less to remember.

What are we printing?

How about a Table of the most popular pages on my site? I use Plausible for stats, and I’ve been meaning to play with their API. But I’m here to talk about Rich, not Plausible. Let’s use a static copy of API results so everyone’s using the same data.

    import rich
from rich.table import Table

STATS = {
    "results": [
        {
            "page": "/post/2017/11/drawing-grids-with-python-and-pillow/",
            "visitors": 1114,
        },
        {"page": "/post/2017/01/cinnamon-screenshot-shortcuts/", "visitors": 580},
        {"page": "/", "visitors": 458},
        {
            "page": "/post/2014/06/what-is-build-essentials-for-opensuse/",
            "visitors": 340,
        },
        {"page": "/config/emacs/doom/", "visitors": 303},
        {"page": "/post/2020/06/csv-and-data-tables-in-hugo/", "visitors": 293},
        {"page": "/post/2019/05/kitty-terminal/", "visitors": 265},
        {
            "page": "/post/2018/02/setting-task-dependencies-in-taskwarrior/",
            "visitors": 263,
        },
        {"page": "/post/2019/02/taskwarrior-projects/", "visitors": 260},
        {
            "page": "/post/2019/01/circular-grids-with-python-and-pillow/",
            "visitors": 242,
        },
    ]
}


def build_stats_table(stats):
    """Construct a Rich Table from site traffic breakdown."""

    table = Table(title="Plausible.io Traffic Breakdown")
    table.add_column("Page")
    table.add_column("Visitors", justify="right", style="green")

    for entry in stats["results"]:
        table.add_row(entry["page"], "{:,}".format(entry["visitors"]))

    return table


def show_stats():
    """Display Plausible's breakdown of site traffic."""

    table = build_stats_table(STATS)
    rich.print(table)


if __name__ == "__main__":
    show_stats()

Here’s a screenshot, so you know what this produces in my own terminal.

table output
table output

Okay. Now let’s start talking about exporting output.

xclip is usually good enough

This post focuses on the “blog writing and pretty reports” situations. For everyday sharing, all I need is a legibly formatted data dump. xclip works for those situations.

    python showstats.py | xclip

I don’t see anything on my screen, of course, because I piped everything to xclip. But when I paste from the clipboard:

                        Plausible.io Traffic Breakdown
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Page                                                    ┃ Visitors ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ /post/2017/11/drawing-grids-with-python-and-pillow/     │    1,114 │
│ /post/2017/01/cinnamon-screenshot-shortcuts/            │      580 │
│ /                                                       │      458 │
│ /post/2014/06/what-is-build-essentials-for-opensuse/    │      340 │
│ /config/emacs/doom/                                     │      303 │
│ /post/2020/06/csv-and-data-tables-in-hugo/              │      293 │
│ /post/2019/05/kitty-terminal/                           │      265 │
│ /post/2018/02/setting-task-dependencies-in-taskwarrior/ │      263 │
│ /post/2019/02/taskwarrior-projects/                     │      260 │
│ /post/2019/01/circular-grids-with-python-and-pillow/    │      242 │
└─────────────────────────────────────────────────────────┴──────────┘

xclip preserves the basic shape of my output. I see a table. The Visitors column is right-aligned. The title is centered. But it loses some of the finer formatting bits: bold, italicization, color.

ℹ️ Note

Also? This renders great on Chrome-based browsers and weird on Firefox. There are definite limitations to just copying and pasting from the terminal.

Let’s pull that clipboard management into the script with Al Sweigart’s Pyperclip library.

Let Rich and Pyperclip handle the clipboard

Pyperclip gives our code access to the system clipboard, letting us copy and paste from Python. The Rich Console can capture the characters it would have printed, and hand them to us when needed. Sounds like a great team.

    import pyperclip
from rich.console import Console

I set up Pyperclip and create a local Console to handle capturing.

    def show_stats(stats):
    """Display Plausible's breakdown of site traffic."""

    table = build_stats_table(stats)
    pyperclip.set_clipboard("xclip")
    console = Console()

    with console.capture() as capture:
        console.print(table)

    text_output = capture.get()
    pyperclip.copy(text_output)
    print(text_output)

I need to tell Pyperclip about xclip or it gets a bit confused on WSL. Also, since I captured the output, I need to print it myself. Why print instead of rich.print or console.print?

Let me answer that question by pasting the contents of my clipboard:

    [3m                    Plausible.io Traffic Breakdown                    [0m
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃[1m [0m[1mPage                                                   [0m[1m [0m┃[1m [0m[1mVisitors[0m[1m [0m┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ /post/2017/11/drawing-grids-with-python-and-pillow/     │[32m [0m[32m    1114[0m[32m [0m│
│ /post/2017/01/cinnamon-screenshot-shortcuts/            │[32m [0m[32m     580[0m[32m [0m│
│ /                                                       │[32m [0m[32m     458[0m[32m [0m│
│ /post/2014/06/what-is-build-essentials-for-opensuse/    │[32m [0m[32m     340[0m[32m [0m│
│ /config/emacs/doom/                                     │[32m [0m[32m     303[0m[32m [0m│
│ /post/2020/06/csv-and-data-tables-in-hugo/              │[32m [0m[32m     293[0m[32m [0m│
│ /post/2019/05/kitty-terminal/                           │[32m [0m[32m     265[0m[32m [0m│
│ /post/2018/02/setting-task-dependencies-in-taskwarrior/ │[32m [0m[32m     263[0m[32m [0m│
│ /post/2019/02/taskwarrior-projects/                     │[32m [0m[32m     260[0m[32m [0m│
│ /post/2019/01/circular-grids-with-python-and-pillow/    │[32m [0m[32m     242[0m[32m [0m│
└─────────────────────────────────────────────────────────┴──────────┘

Uh. Oops? console captured exactly what it would have printed, including terminal escape codes.

Rich supports exporting output beyond a raw dump, though.

Let Rich get you some HTML

ℹ️ Note

For safety reasons, most Markdown converters must be explicitly configured to allow raw HTML through. Check the documentation of your converter or blogging tools to see if and how you need to do that.

A Console created with the record option enabled remembers everything it prints. You can get export your copy at any point. The export_text method provides a copy with minimal formatting, while export_html produces HTML pages. That’s for sure something I can paste into my post source. Nice!

One slight wrinkle. Unless you tell it otherwise, export_html produces a complete HTML file — with <head>, <body>, and even a <style> section. All I want is the <pre>...</pre> describing my output.

Fortunately, export_html also lets us tell it exactly what we want:

Let’s make some HTML for Pyperclip to copy.

    def show_stats():
    """Display Plausible's breakdown of site traffic."""

    # print the stats
    table = build_stats_table(STATS)
    console = Console(record=True)
    console.print(table)

    # copy the stats
    pyperclip.set_clipboard("xclip")
    exported_html = console.export_html(
        inline_styles=True, code_format="<pre>{code}</pre>"
    )
    pyperclip.copy(exported_html)

What do the contents of my clipboard look like now?

                    Plausible.io Traffic Breakdown                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Page                                                    ┃ Visitors ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ /post/2017/11/drawing-grids-with-python-and-pillow/     │    1,114 │
│ /post/2017/01/cinnamon-screenshot-shortcuts/            │      580 │
│ /                                                       │      458 │
│ /post/2014/06/what-is-build-essentials-for-opensuse/    │      340 │
│ /config/emacs/doom/                                     │      303 │
│ /post/2020/06/csv-and-data-tables-in-hugo/              │      293 │
│ /post/2019/05/kitty-terminal/                           │      265 │
│ /post/2018/02/setting-task-dependencies-in-taskwarrior/ │      263 │
│ /post/2019/02/taskwarrior-projects/                     │      260 │
│ /post/2019/01/circular-grids-with-python-and-pillow/    │      242 │
└─────────────────────────────────────────────────────────┴──────────┘

That works well enough for a blog post!

If you’re curious about the exported HTML, here’s a chunk of it:

        <pre><span style="font-style: italic">                    Plausible.io Traffic Breakdown                    </span>
    ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
    ┃<span style="font-weight: bold"> Page                                                    </span>┃<span style="font-weight: bold"> Visitors </span>┃
    ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
    │ /post/2017/11/drawing-grids-with-python-and-pillow/     │<span style="color: #008000; text-decoration-color: #008000">    1,114 </span>│
    ...
    │ /post/2019/01/circular-grids-with-python-and-pillow/    │<span style="color: #008000; text-decoration-color: #008000">      242 </span>│
    └─────────────────────────────────────────────────────────┴──────────┘
    </pre>

Anyways, this was just another thing I wanted to get down before I forgot again.

What else?

There are a few more pieces that tie it into my particular workflow, but this covers what you’d need to export output from your own Rich programs for easy blogging or information sharing.